Symmetries Constrain Dynamics in a Family of Balanced Neural Networks

Abstract

We examine a family of random firing-rate neural networks in which we enforce the neurobiological constraint of Dale’s Law—each neuron makes either excitatory or inhibitory connections onto its post-synaptic targets. We find that this constrained system may be described as a perturbation from a system with nontrivial symmetries. We analyze the symmetric system using the tools of equivariant bifurcation theory and demonstrate that the symmetry-implied structures remain evident in the perturbed system. In comparison, spectral characteristics of the network coupling matrix are relatively uninformative about the behavior of the constrained system.

1 Introduction

Networked dynamical systems are of growing importance across the physical, engineering, biological, and social sciences. Indeed, understanding how network connectivity drives network functionality is critical for understanding a broad range of modern-day systems including the power grid, communications networks, the nervous system, and social networking sites. All of these systems are characterized by a large and complex graph connecting many individual units, or nodes, each with its own input–output dynamics. In addition to the node dynamics, how such a system operates as a whole depends on the structure of its connectivity graph [1–3], but the connectivity is often so complicated that this structure-function problem is difficult to solve.

Regardless, an ubiquitous observation across the sciences is that meaningful input/output of signals in high-dimensional networks are often encoded in low-dimensional patterns of dynamic activity. This suggests that a central role of the network structure is to produce low-dimensional representations of meaningful activity. Furthermore, since connectivity also drives the underlying bifurcation structure of the network-scale activity, and because both this activity and the relevant features of the connectivity graph are low-dimensional, such networks may admit a tractable structure-function relationship. Interestingly, the presence of low-dimensional structure may run counter to the intuition provided by the insights of random network theory, which has otherwise proven to be a valuable tool in analyzing large networks.

In considering an excitatory–inhibitory network inspired by neuroscience, we find a novel family of periodic solutions that restrict dynamics to a low-dimensional attractor within a high-dimensional phase space. These solutions arise as a consequence of an underlying symmetry in the mean connectivity structure and can be predicted and analyzed using equivariant bifurcation theory. We then show that low-dimensional models of the high-dimensional network, which are more tractable for computational bifurcation studies, preserve all the key features of the bifurcation structure. Finally, we demonstrate that these dynamics differ strikingly from the predictions made by random network theory in a similar setting.

Random network theory—in which one seeks to draw conclusions about an ensemble of randomly chosen networks, rather than a specific instance of a network—is particularly relevant to neural networks because such networks are large, under-specified (most connections cannot be measured), and heterogenous (connections are variable both within, and between, organisms). It is particularly tempting to apply the tools of random matrix theory to the connectivity graph, as the spectra of certain classes of random matrices display universal behavior as network size $N \rightarrow\infty$ [4]. The seminal work of Sompolinsky et al. [5] analyzes a family of single-population firing-rate networks in which connections are chosen from a mean-zero Gaussian distribution: in the limit of large network size ($N \rightarrow\infty$), they find that the network transitions from quiescence to chaos as a global coupling parameter passes a bifurcation value $g^{\ast} = 1$. This value coincides with the point at which the spectrum of the random connectivity matrix exits the unit circle [6–8], thereby connecting linear stability theory with the full nonlinear dynamics.

Developing similar results for structured, multipopulation networks has proven more challenging. One natural constraint to introduce is that of Dale’s Law: each neuron makes either excitatory or inhibitory connections onto its postsynaptic targets. For a neural network, this constraint is manifested in a synaptic weight matrix with single-signed columns. If weights are tuned so that incoming excitatory and inhibitory currents approximately cancel (i.e. $\sum_{j} \mathbf {G}_{ij} \approx0$), then such a network may be called balanced (we note that our use of the word “balanced” is distinct from the dynamic balance that arises in random networks when excitatory and inhibitory synaptic currents approximately cancel, as studied by [9, 10] and others). Rajan and Abbott [11] studied balanced rank one perturbations of Gaussian matrices and found that, remarkably, the spectrum is unchanged. More recent papers have addressed the spectra of more general low-rank perturbations [12–14], general deterministic perturbations [15], and block-structured matrices [16].

However, the relationship between linear/spectral and nonlinear dynamics appears to be more complicated than in the unstructured case. Aljadeff et al. [16] indeed find that the spectral radius is a good predictor of qualitative dynamics and learning capacity in networks with block-structured variances. Others have studied the large network limit, but when mean connectivity scales like $1/N$ (smaller than the standard deviation $1/\sqrt{N}$): therefore, as $N \rightarrow\infty$, the columns cease to be single-signed [17–19]. In a recent paper, which studies a balanced network with mean connectivity $1/\sqrt {N}$, the authors find a slow noise-induced synchronized oscillation that emerges when a special condition (perfect balance) is imposed on the connectivity matrix [20]. As a growing body of work has continued to connect qualitative features of nonlinear dynamics and learning capacity [21–23], it is crucial to continue to further develop our understanding of how complex nonlinear dynamics emerges in structured heterogeneous networks.

In this paper, we study a family of excitatory–inhibitory networks in which both the mean and variability of connection strengths scale like $1/\sqrt{N}$. In a small but crucial difference from other recent work [11, 20], we reduce self-coupling. We show that with this change, these networks exhibit a (heretofore unreported) family of periodic solutions. These solutions arise as a consequence of an underlying symmetry in the mean connectivity structure and can be predicted and analyzed using equivariant bifurcation theory. We show through concrete examples that these periodic orbits can persist in heterogeneous networks, even for large perturbations. Moreover, we demonstrate that low-dimensional models (reduced-order models) can be generated to characterize the high-dimensional system and its underlying bifurcation structure; we use the reduced model to study these oscillations as a function of system size N. Thus the work suggests both how biophysically relevant symmetries may play a crucial role in the observable dynamics, and also how reduced-order models can be constructed to more easily study the underlying dynamics and bifurcations.

2 Mathematical Model

We consider a network in which each node represents the firing rate of a single neuron, connected by sigmoidal activation functions through a random weight matrix. This is the model studied in Refs. [5, 11, 20] with some important modifications that we further detail. Specifically, we analyze the family of random networks

$$ \dot{\mathbf {x}} = -\mathbf {x}+ \mathbf {G}\tanh(g \mathbf {x}), $$

(1)

where

$$ \sqrt{N} \mathbf {G}= \mathbf {H}+ \epsilon \mathbf {A} $$

(2)

with an $N \times N$ matrix H such that

$$ \mathbf {H}_{ij} = \left \{ \textstyle\begin{array}{l@{\quad}l} \mu_{E}, & j \leq n_{E}, j \neq i,\\ b_{E} \mu_{E}, & j \leq n_{E}, j = i,\\ \mu_{I}, & n_{E} < j \leq N, j \neq i,\\ b_{I} \mu_{I}, & n_{E} < j \leq N, j = i, \end{array}\displaystyle \right . $$

(3)

and

$$ \mathbf {A}_{ij} \sim \left \{ \textstyle\begin{array}{l@{\quad}l} N(0,\sigma_{E}), & j \leq n_{E}, j \neq i,\\ N(0,\sigma_{I}), & n_{E} < j \leq N, j \neq i,\\ 0, & j = i. \end{array}\displaystyle \right . $$

(4)

We assume that $\mu_{I} < 0 < \mu_{E}$; therefore the first $n_{E}$ nodes are excitatory, and the remainder are inhibitory.

We will use the parameter f to identify the fraction of neurons that are excitatory, that is, $f = n_{E}/N$. The parameter α characterizes the ratio of inhibitory-to-excitatory synaptic strengths: $\mu_{I} = -\alpha\mu_{E}$. We refer to the network as balanced (the mean connectivity into any cell is zero) if $\alpha= \frac{f}{1-f}$; it is inhibition-dominated if $\alpha> \frac{f}{1-f}$. Further, in all cases, $f=0.8$, reflecting the approximately 80%/20% ratio observed in cortex; the corresponding value of α for a balanced network is $\alpha= 4$. Finally, we choose $\sigma_{E}$ and $\sigma_{I}$ so that the variances of excitatory and inhibitory connections into each cell are equal, that is, $\sigma_{E}^{2} f = \sigma_{I}^{2} (1-f)$.

The matrix H has constant columns, except for the diagonal, which reflects self-coupling from each cell onto itself. The parameters $b_{E}$ and $b_{I}$ give the ratios of self- to non-self-connection strengths for excitatory and inhibitory cells, respectively. We assume that the effect of self-coupling is to reduce connection strengths, that is, $0 \le b_{E}, b_{I} \le1$.

We note that as in [5]—but in contrast to later work [11, 20]—self-interactions can differ from interactions with other neurons, that is, $\mathbf {G}_{jj} \neq \mathbf {G}_{ij}$. This is a reasonable assumption if we conceptualize each firing rate unit $x_{j}$ as corresponding to an individual neuron; whereas neurons can have self-synapses (or autapses [24]), refractory dynamics would tend to suppress self-coupling from influencing the firing rate.

We find that many features of the resulting dynamics may be connected to an approximate symmetry of the system. Specifically, if you remove the “noise” from the connectivity matrix G—so that $\mathbf {G}_{ij} = \mu_{E}/\sqrt{N}$ if $j \le n_{E}$, $i \neq j$, and $\mathbf {G}_{ij} = \mu_{I}/\sqrt{N}$ if $j > n_{E}$, $i \neq j$—then the subspace in which all E neurons have the same activity and all I neurons have the same activity, that is,

$$ \begin{aligned} x_{j} &= x_{E}, \quad j \le n_{E}, \\ x_{j} & = x_{I}, \quad j > n_{E}, \end{aligned} $$

is invariant under the dynamics $\dot{\mathbf {x}} = -\mathbf {x}+ \mathbf {G}\tanh (g \mathbf {x})$. To be precise, the system of equations is equivariant under the group of permutation symmetries $(S_{n_{E}} \times S_{n_{I}})$, which contains any permutation of the $n_{E}$ excitatory neurons and any permutation of the $n_{I}$ inhibitory neurons.

We begin by considering the “noiseless” system in (1), where $\sqrt{N} \mathbf {G}= \mathbf {H}$. The solutions that arise in this system can be readily identified because of the underlying symmetries of the network. We find that these solutions actually do arise in numerical simulations: furthermore, they persist even when the symmetry is perturbed ($\sqrt{N} \mathbf {G}= \mathbf {H}+ \epsilon \mathbf {A}$).

2.1 Some Preliminary Analysis: Spectrum of H

To analyze stability and detect bifurcations, we will frequently make reference to the Jacobian of (1), (2); when $\epsilon= 0$, we will find that this always takes on a column-structured form. We begin by summarizing some facts about the spectra of these matrices.

Let $\mathbf {K}_{N}$ be the matrix of all ones except on the diagonal. that is,

$$ \mathbf {K}_{N} = \mathbf{1}_{N} (\mathbf{1}_{N})^{T} - \mathbf {I}_{N}, $$

(5)

where $\mathbf{1}_{N} \in\mathbb{R}^{N}$ is the vector of all ones, and $\mathbf {I}_{N}$ is the $N \times N$ identity matrix.

Lemma 1

$\mathbf {K}_{N}$ has the following eigenvalues: $\lambda_{0} = N-1$ and $\lambda_{j} = -1$ with geometric and algebraic multiplicity $N-1$.

Proof

By inspection, $\mathbf{1}_{N}$ is an eigenvector with corresponding eigenvalue $N-1$ (as each row sums to $N-1$). The remaining eigenvectors must satisfy

$$ \begin{aligned} \sum_{j \neq i} v_{j} & = \lambda v_{i} \rightarrow \\ \sum_{j} v_{j} & = \lambda v_{i} + v_{i} \\ & = (\lambda+ 1)v_{i} = 0, \end{aligned} $$

since each such eigenvector is orthogonal to $\mathbf{1}_{N}$. □

The Jacobian of (1) has the following special structure: except for its diagonal, the entries in column j depend only on the jth coordinate (and are all equal). This leads to a simplification of the spectrum when the cells are divided into synchronized populations. To be precise, we can make the following statement.

Lemma 2

Assume that we can divide our cells $j=1, \dots, N$ into $K+1$ populations, where $I_{k}$ identifies the index set of population k for $k = 0, \dots, K$. Let $\mathbf {J}=-\mathbf {I}_{N} + \mathbf {K}_{N} \boldsymbol {\varLambda }+ \mathbf {B}$, where Λ and B are diagonal matrices with

$$\begin{aligned} \boldsymbol {\varLambda }_{jj} & = a_{k}, \quad j \in I_{k}, \end{aligned}$$

(6)

$$\begin{aligned} \mathbf {B}_{jj} & = b_{k}, \quad j \in I_{k}, \end{aligned}$$

(7)

that is, J has constant columns (except for the diagonal), with the value in each column determined by the population identity. Then the eigenvalues of J are:

• (1) $-1-a_{k} + b_{k}$ with multiplicity $n_{I_{k}}-1$ for $k=0,\dots,K$;

• (2) the $K+1$ remaining eigenvalues coincide with the eigenvalues of the matrix  $\tilde{\mathbf {J}}$:

  $$ \tilde{\mathbf {J}}_{ij} = \left \{ \textstyle\begin{array}{l@{\quad}l} n_{I_{j}} a_{j}, & j \neq i,\\ -1+(n_{I_{j}}-1) a_{j} + b_{j}, & j =i, \end{array}\displaystyle \right . $$

  (8)

  where $n_{I_{j}}$ is the number of cells in population j. We note that the size of $\tilde{\mathbf {J}}$ is set by the number of subpopulations, that is, $\tilde{\mathbf {J}} \in\mathbb{R}^{(K+1) \times(K+1)}$.

Proof

This can be checked by direct computation:

• 1. For $k=0,\dots,K$: there are $n_{I_{k}}-1$ linearly independent eigenvectors given by vectors that (a) have support only on $I_{k}$ and (b) sum to zero, that is, $\mathbf {v}^{k}_{j} = 0$ if $j \notin I_{k}$, and $\mathbf {v}^{k} \perp \boldsymbol {1}_{N}$.

• 2. The remaining eigenvectors are given by vectors that are constant and nonzero on each index set: $\mathbf {v}_{j} = c_{k}$ if $j \in I_{k}$, and $[ c_{0}\ c_{1} \ \cdots\ c_{K} ]$ is an eigenvector of $\tilde{\mathbf {J}}$.  □

We now consider specific examples that are of particular importance.

Example 1

We consider a balanced network with (possibly) reduced self-coupling: $\alpha= \frac{n_{E}}{n_{I}}$ and $0 \le b_{E}, b_{I} \le1$. The origin $\mathbf {x}= \boldsymbol {0}$ is a fixed point of (1) for all g. Therefore, we can think of the population as consisting of two synchronized populations, excitatory and inhibitory, that is, $n_{0} = n_{E}$ and $n_{1} = n_{I}$; $a_{0} = \frac{g \mu_{E}}{\sqrt{N}}$, $a_{1} = \frac{-\alpha g \mu_{E}}{\sqrt{N}}$, $b_{0} = b_{E} a_{0}$, and $b_{1} = b_{I} a_{1}$. Then the Jacobian has the following eigenvalues:

• (1) $\lambda_{E} = -1 - \frac{g \mu_{E}}{\sqrt{N}}(1-b_{E})$ with multiplicity $n_{E} - 1$;

• (2) $\lambda_{I} = -1 + \frac{g \alpha\mu_{E}}{\sqrt {N}}(1-b_{I})$ with multiplicity $n_{I} - 1$;

• (3) two remaining eigenvalues given by the $2 \times2$ matrix $\tilde{\mathbf {J}}$:

  $$ \tilde{\mathbf {J}} = -\mathbf {I}+ \frac{g\mu_{E}}{\sqrt{N}} \left [ \begin{matrix} n_{E} - (1-b_{E}) & -n_{E}\\ n_{E} & -n_{E} + \alpha(1-b_{I}) \end{matrix} \right ]. $$

  (9)

  This is be a complex pair as long as $n_{E} > ( \alpha(1-b_{I}) + 1-b_{E} )/4$, so $\lambda_{1,2} = \lambda\pm i \omega$, where

  $$ \begin{aligned} \lambda& = -1 + \frac{g \mu_{E}}{\sqrt{N}} \frac{\alpha(1-b_{I}) - 1+b_{E}}{2}, \\ \omega& = \frac{g \mu_{E}}{\sqrt{N}} \sqrt{\alpha(1-b_{I}) + 1-b_{E}} \sqrt{n_{E} - \frac{\alpha(1-b_{I}) + 1-b_{E}}{4}}. \end{aligned} $$

  We note that $\lambda_{E} < \lambda\equiv\operatorname{Re}(\lambda_{1,2}) < \lambda_{I}$. The eigenvalue associated with the excitatory population, $\lambda_{E} < 0$ for any value of g.

The corresponding eigenvectors are:

• (1) $\mathbf {v}_{E} = \operatorname{span} \{ [ \mathbf {v}_{n_{E}} \underbrace{ 0 \ \cdots\ 0 }_{n_{I}} ] \}$, $\mathbf {v}_{n_{E}} \perp \boldsymbol {1}_{n_{E}}$;

• (2) $\mathbf {v}_{I_{1}} = \operatorname{span} \{ [ \underbrace{ 0 \ \cdots\ 0 }_{n_{E}} \mathbf {v}_{n_{I_{1}}} ] \}$, $\mathbf {v}_{n_{I}} \perp \boldsymbol {1}_{n_{I}}$;

• (3) $\mathbf {v}_{\tilde{J}} = \operatorname{span} \{ [ \underbrace{ c_{E} \ \cdots\ c_{E} }_{n_{E}}\ \underbrace{ c_{I} \ \cdots\ c_{I} }_{n_{I}} ] \}$.

We pause to consider two special cases of Example 1. The first is no self-coupling—$b_{E}, b_{I} = 0$, which we will examine in detail in the rest of this paper. The second is full self-coupling—$b_{E}, b_{I} = 1$, which has been studied previously by many authors [11, 19, 20].

Example 1.1

We consider Example 1, but with no self-coupling: $b_{E}, b_{I} = 0$. Then at the origin $\mathbf {x}= 0$, the Jacobian has the following eigenvalues:

• (1) $\lambda_{E} = -1 - \frac{g \mu_{E}}{\sqrt{N}}$ with multiplicity $n_{E} - 1$;

• (2) $\lambda_{I} = -1 + \frac{g \alpha\mu_{E}}{\sqrt {N}}$ with multiplicity $n_{I} - 1$;

• (3) two remaining eigenvalues given by the $2 \times2$ matrix $\tilde{\mathbf {J}}$:

  $$ \tilde{\mathbf {J}} = -\mathbf {I}+ \frac{g\mu_{E}}{\sqrt{N}} \left [ \begin{matrix} n_{E} - 1 & -n_{E}\\ n_{E} & -n_{E} + \alpha \end{matrix} \right ], $$

  (10)

  which will be a complex pair as long as $n_{E} > (\alpha+ 1)/4$, so $\lambda_{1,2} = \lambda\pm i \omega$, where

  $$ \begin{aligned} \lambda& = -1 + \frac{g \mu_{E}}{\sqrt{N}} \frac{\alpha- 1}{2}, \\ \omega& = \frac{g \mu_{E}}{\sqrt{N}} \sqrt{\alpha+ 1} \sqrt{n_{E} - \frac{1+ \alpha}{4}}. \end{aligned} $$

  We note that $\lambda_{E} < \lambda\equiv\operatorname{Re}(\lambda_{1,2}) < \lambda_{I}$. The eigenvalue associated with the excitatory population, $\lambda_{E} < 0$ for any value of g. In the (un-cortex-like) situation that the excitatory population were smaller than the inhibitory population ($\alpha< 1$), the complex pair would also be stable for all $\lambda< 0$.

Example 1.2

We consider Example 1, but where self-coupling is not reduced: $b_{E}, b_{I} = 1$. Consider the eigenvalues at the origin $\mathbf {x}= 0$ described in Example 1:

• (1) Since $b_{E} = 1$, $\lambda_{E} = -1$ with multiplicity $n_{E} - 1$;

• (2) Since $b_{I} = 1$, $\lambda_{I} = -1$ with multiplicity $n_{I} - 1$;

• (3) two remaining eigenvalues given by the $2 \times2$ matrix $\tilde{\mathbf {J}}$:

  $$ \tilde{\mathbf {J}} = -\mathbf {I}+ \frac{g\mu_{E}}{\sqrt{N}} \begin{bmatrix} n_{E} & -n_{E}\\ n_{E} & -n_{E} \end{bmatrix} , $$

  (11)

  which also has the (repeated) eigenvalue −1.

Thus, every eigenvalue of H is −1; crucially, this does not depend on the coupling parameter g. In Sect. 3, we describe how, by varying g, bifurcations occur at the origin; these cannot occur if self-coupling is not reduced, since the eigenvalues of the Jacobian cannot pass through the imaginary axis.

(Another way to reach the same conclusion is to notice that H is a rank-one matrix [11]:

$$ \mathbf {H}= {\mathbf {u}} \boldsymbol {1}^{T}, \quad\text{where } \mathbf {u}= [ \underbrace{ \begin{matrix} \mu_{E} & \cdots& \mu_{E} \end{matrix} }_{n_{E}} \underbrace{ \begin{matrix} \mu_{I} & \cdots& \mu_{I} \end{matrix} }_{n_{I}} ]^{T} $$

(12)

with at most one nonzero eigenvalue; since $\mu_{E} n_{E} + \mu_{I} n_{I} = 0$, this last eigenvalue is zero as well.)

It is worth pointing out that in Example 1, the $O(1/\sqrt{N})$ scaling of the complex eigenvalue $\lambda\pm\imath\omega$ relies on balanced coupling; the N-dependent part of the trace of the matrix in equation (10), which is

$$n_{E} - \alpha n_{I} = f N - \alpha(1-f) N = N \bigl(f - (1-f) \alpha \bigr) = N(1-f) \biggl( \frac{f}{1-f} - \alpha \biggr), $$

vanishes precisely when $\alpha= \frac{f}{1-f}$. When it does not vanish, the trace is $O(N)$, and therefore λ is $O(\sqrt{N})$.

Example 2

Next, suppose that the cells have broken into three synchronized populations: the excitatory cells ($n_{E}$ cells with activity $x_{E}(t)$) and two groups of inhibitory cells ($n_{I_{1}}$ and $n_{I_{2}}$ cells with activities $x_{I_{1}}$ and $x_{I_{2}}$, respectively). Then $n_{0} = n_{E}$, $n_{1} = n_{I_{1}}$, and $n_{2} = n_{I_{2}}$; $a_{0} = \frac{g \mu_{E}}{\sqrt{N}} \operatorname {sech}^{2}(g x_{E})$ and $a_{1,2} = -\frac{g \alpha\mu _{E}}{\sqrt{N}} \operatorname {sech}^{2}(g x_{I_{1,2}})$; $b_{0} = b_{E} a_{0}$ and $b_{1,2} = b_{I} a_{1,2}$. Therefore the Jacobian has the following eigenvalues:

• (1) $\lambda_{E} = -1 - \frac{g \mu_{E}}{\sqrt{N}} \operatorname {sech}^{2}(g x_{E}) (1-b_{E})$ with multiplicity $n_{E} - 1$;

• (2) $\lambda_{I_{1}} = -1 + \frac{g \alpha\mu_{E}}{\sqrt {N}} \operatorname {sech}^{2}(g x_{I_{1}})(1-b_{I})$ with multiplicity $n_{I_{1}} - 1$;

• (3) $\lambda_{I_{2}} = -1 + \frac{g \alpha\mu_{E}}{\sqrt {N}} \operatorname {sech}^{2}(g x_{I_{2}})(1-b_{I})$ with multiplicity $n_{I_{2}} - 1$;

• (4) three remaining eigenvalues given by the $3 \times3$ matrix $\tilde{\mathbf {J}}$ described earlier.

We note that always $\lambda_{E} < 0$ as long as $b_{E} \leq1$. The corresponding eigenvectors are:

• (1) $\mathbf {v}_{E} = \operatorname{span} \{ [ \mathbf {v}_{n_{E}} \underbrace{ 0 \ \cdots\ 0 }_{n_{I_{1}}}\ \underbrace{ 0 \ \cdots\ 0 }_{n_{I_{2}}} ] \}$, $\mathbf {v}_{n_{E}} \perp \boldsymbol {1}_{n_{E}}$;

• (2) $\mathbf {v}_{I_{1}} = \operatorname{span} \{ [ \underbrace{ 0 \ \cdots\ 0 }_{n_{E}}\ \mathbf {v}_{n_{I_{1}}} \underbrace{ 0 \ \cdots\ 0 }_{n_{I_{2}}} ] \}$, $\mathbf {v}_{n_{I_{1}}} \perp \boldsymbol {1}_{n_{I_{1}}}$;

• (3) $\mathbf {v}_{I_{2}} = \operatorname{span} \{ [ \underbrace{ 0 \ \cdots\ 0 }_{n_{E}}\ \underbrace{ 0 \ \cdots\ 0 }_{n_{I_{1}}} \mathbf {v}_{n_{I_{2}}} ] \}$, $\mathbf {v}_{n_{I_{2}}} \perp \boldsymbol {1}_{n_{I_{2}}}$;

• (4) $\mathbf {v}_{\tilde{J}} = \operatorname{span} \{ [ \underbrace{ c_{E} \ \cdots\ c_{E} }_{n_{E}}\ \underbrace{ c_{I_{1}} \ \cdots\ c_{I_{1}} }_{n_{I_{1}}}\ \underbrace{ c_{I_{2}} \ \cdots\ c_{I_{2}} }_{n_{I_{2}}} ] \}$.

3 Solution Families Found in Deterministic Network ($\epsilon= 0$)

In this section, we use equivariant bifurcation theory to identify which solutions we expect to arise in system (1), where $\mathbf {G}= \mathbf {H}/\sqrt{N}$. We also demonstrate that these solutions actually arise in a small network where it is tractable to do numerical continuation to verify our calculations. Our main tool is the equivariant branching lemma, which tells us which type of solutions (to the equation $\dot{\mathbf {x}} = F(\mathbf {x}, g)$) arises at bifurcation points when symmetries are present [25–27]. For example, we might take F to be the function introduced in Sect. 2:

$$F(\mathbf {x}, g) = -\mathbf {x}+ \mathbf {G}\tanh(g \mathbf {x}). $$

Before stating this result, we introduce some terminology. Let Γ be a finite group acting on $\mathbb{R}^{N}$. Then we say that a mapping $F: \mathbb{R}^{N} \rightarrow\mathbb{R}^{N}$ is Γ-equivariant if $F(\gamma \mathbf {x}) = \gamma F(\mathbf {x})$ for all $\mathbf {x}\in\mathbb{R}^{N}$ and $\gamma\in\varGamma$. A one-parameter family of mappings $F: \mathbb{R}^{N} \times\mathbb{R} \rightarrow \mathbb{R}^{N}$ is Γ-equivariant if it is Γ-equivariant for each value of its second argument.

We say that a subspace V of $\mathbb{R}^{N}$ is Γ-invariant if $\gamma \mathbf {v}\in V$ for any v and $\gamma\in \varGamma$. We furthermore say that the action of Γ on V is irreducible if V has no proper invariant subspaces, that is, the only Γ-invariant subspaces of V are $\{0\}$ and V itself.

For a group Γ and a vector space V, we define the fixed-point subspace for Γ, denoted $\operatorname {Fix}(\varGamma)$, to be all points in V that are unchanged under any of the members of Γ, that is, $\operatorname{Fix} (\varGamma) = \{ \mathbf {x}\in V : \gamma \mathbf {x}= \mathbf {x}, \forall\gamma\in\varGamma\}$. The isotropy subgroup of $\mathbf {x}\in V$, denoted $\varSigma_{x}$, is the set of all members of Γ under which x is fixed, that is, $\varSigma_{x} = \{ \gamma\in\varGamma: \gamma \mathbf {x}= \mathbf {x}\}$ (we then say that a subgroup Σ is an isotropy subgroup of Γ if it is the isotropy subgroup $\varSigma_{x}$ for some $\mathbf {x}\in V$.)

Suppose we have a one-parameter family of mappings $F(\mathbf {x}, g)$ and we wish to solve $F(\mathbf {x}, g)=0$. For any $(\mathbf {x}, g) \in\mathbb{R}^{n} \times\mathbb{R}$, let $(dF)_{\mathbf {x},g}$ denote the $N \times N$ Jacobian matrix

$$\biggl( \frac{\partial F_{j}}{\partial x_{k}} (\mathbf {x}, g) \biggr)_{j, k=1,\dots,N}. $$

Then the implicit function theorem states that we can continue to track a unique solution branch as a function of g as long as the Jacobian remains invertible. When this ceases to be true—when $(dF)_{\mathbf {x},g}$ has a nontrivial kernel—we have the possibility for a bifurcation. At this point the number of zero eigenvalues (whether there are one, or two, etc.) and a menagerie of further conditions determine the qualitative properties of the structural change that occurs.

What complicates this situation for Γ-equivariant mappings—that is, $F(\mathbf {x},g)$ is Γ-equivariant for any value of the parameter g—is that because of symmetries, multiple eigenvalues go through zero at once; however, the structural changes that occur are qualitatively the same as those that occur in a nonsymmetric system with a single zero eigenvalue. What changes is that we now have multiple such solution branches, each corresponding to a subgroup of the original symmetries. The following result formalizes this fact.

Theorem 1

Equivariant branching lemma: paraphrased from [26], p. 82, see also pp. 67–69

Let $F: \mathbb{R}^{N} \times \mathbb{R} \rightarrow\mathbb{R}^{N}$ be a one-parameter family of Γ-equivariant mappings with $F(\mathbf {x}_{0}, g_{0}) = \boldsymbol {0}$. Suppose that $(\mathbf {x}_{0}, g_{0})$ is a bifurcation point and that, defining $V = \ker(dF)_{\mathbf {x}_{0}, g_{0}}$, Γ acts absolutely irreducibly on V. Let Σ be an isotropy subgroup of Γ satisfying

$$ \operatorname{dim} \operatorname{Fix} (\varSigma) = 1, $$

(13)

where $\operatorname{Fix} (\varSigma)$ is the fixed-point subspace of Σ, that is, $\operatorname{Fix} (\varSigma) \equiv\{ x \in V \mid \sigma x = x, \forall\sigma\in\varSigma\}$. Then there exists a unique smooth solution branch to $F = 0$ such that the isotropy subgroup of each solution is Σ.

A similar statement holds for Hopf bifurcations, which we state here because we will appeal to its conclusions regarding the symmetry of periodic solutions.

Theorem 2

Equivariant Hopf theorem: paraphrased from [26], p. 275

Let F be a one-parameter family of Γ-equivariant mappings with $F(\mathbf {x}_{0}, g_{0}) = \boldsymbol {0}$. Suppose that $(dF)_{\mathbf {x}_{0}, g_{0}}$ has one or more pairs of complex eigenvalues $\rho\pm i \omega$ that satisfy $\rho(g_{0}) = 0$ (i.e. the eigenvalues are pure imaginary at $g_{0}$) and $\rho'(g_{0}) \neq0$. Let V be the corresponding real (i.e. not generalized) eigenspace. Let Σ be an isotropy subgroup of Γ satisfying

$$ \operatorname{dim} \operatorname{Fix} (\varSigma) = 2. $$

(14)

where $\operatorname{Fix} (\varSigma)$ is the fixed-point subspace of Σ, that is, $\operatorname{Fix} (\varSigma) \equiv\{ x \in V \mid \sigma x = x, \forall\sigma\in\varSigma\}$. Then there exists a unique branch of small-amplitude periodic solutions (with period $2\pi/ \omega$) having Σ as their group of symmetries.

Here, the family of mappings is the right-hand side of equation (1), with ${\epsilon= 0}$, that is, $F(\mathbf {x}, g) = -\mathbf {x}+ \mathbf {H}\tanh(g\mathbf {x}) /\sqrt{N}$. Let $\varGamma= S_{n_{E}} \times S_{n_{I}}$, where $S_{n}$ is the symmetric group on n symbols, that is, we are allowed to permute the labels on the excitatory cells and/or to permute the labels on the inhibitory cells. Each permutation on N objects can be represented as an element in $GL(N)$, the group of invertible $N \times N$ matrices; Γ is a finite subgroup of such matrices. It is straightforward to check that F is Γ-equivariant.¹

We note that although Theorem 2 states conditions under which a Hopf bifurcation occurs and the symmetry group of the resulting orbit, it does not indicate whether the bifurcation is subcritical or supercritical. This must be determined by other means; we checked this numerically by computing the first Lyapunov coefficient.

Since our model satisfies the assumptions of the equivariant branching lemma, it remains to identify potential bifurcation points (we concentrate on absent self-coupling, that is, $b_{E}, b_{I} = 0$). From the trivial solution ($\mathbf {x}= \boldsymbol {0}$) we expect solutions to arise when the eigenvalues of $-\mathbf {I}+ g \mathbf {H}/\sqrt{N}$ cross the imaginary axis. In particular, we expect, in order of increasing g,

• A branch of fixed-point solutions when $g^{\ast} = \sqrt {N}/\alpha\mu_{E}$, where the eigenvalues corresponding to the inhibitory population cross zero: here the I cells break into two groups of size $n_{I1}$ and $n_{I2}$. Along this fixed point branch, the two groups remain clustered; the excitatory cells also remain clustered, that is, the solution branch can be characterized by $(x_{E},x_{I1},x_{I2})$. We refer to this as the “$I_{1}/I_{2}$ branch.”

• A branch of limit cycles emerging from a Hopf bifurcation when $g = 2\sqrt{N}/ \mu_{E} (\alpha-1)$: here a complex pair crosses the imaginary axis.

From each $I_{1}/I_{2}$ branch we find:

• A branch of limit cycles from a Hopf bifurcation (at $g^{H}$) in which the three-cluster pattern is maintained, that is, activity can be characterized by $(x_{E}, x_{I1}, x_{I2})$.

• If $n_{I1} = n_{I2}$, then the excitatory activity along this branch is zero: there may be a further branch point in which $x_{E}$ moves away from the origin, whereas I cells remain in their distinct clusters.

• (Possibly) other fixed point branches, in which one inhibitory cluster ($x_{I1}$) breaks into further clusters.

3.1 Branch of Fixed Points (from Trivial Solution)

The first opportunity for a bifurcation from the trivial solution occurs when $g^{\ast} = \sqrt{N}/\alpha\mu_{E}$: at this value of g, $n_{I} -1$ eigenvalues pass through zero: the corresponding eigenspace (from Example 1) is the set of all zero-sum vectors with support in the inhibitory cells only, that is,

$$ V \equiv\ker(dF)_{\boldsymbol {0},g^{\ast}} = \operatorname{span} \bigl\{ [ \underbrace{ \begin{matrix}0 & \cdots& 0 \end{matrix} }_{n_{E}} \mathbf {v}_{n_{I}} ] \bigr\} , \quad \mathbf {v}_{n_{I}} \perp \boldsymbol {1}_{n_{I}}. $$

To check that Γ acts irreducibly on V, it is sufficient to show that the subspace spanned by the orbit of a single vector v (defined as the set of all values γ v for $\gamma\in\varGamma$) is full rank; this can be readily confirmed for $\mathbf {v}_{n_{I}} = [ 1 \ {-}1 \ 0 \ \cdots\ 0 ]$, for example.

Suppose we break the inhibitory cells up into precisely two clusters; we allow all permutations within each cluster, but no longer allow mixing between the clusters. This describes a subgroup of Γ, $\varSigma= S_{n_{E}} \times S_{n_{I_{1}}} \times S_{n_{I_{2}}}$, $n_{I_{1}} + n_{I_{2}} = n_{I}$. Assuming that (without loss of generality) the $I_{1}$ neurons have the indices $n_{E}+1,\ldots,n_{E}+n_{I_{1}}$, and so forth, Σ has the fixed point subspace

$$ \operatorname{Fix}(\varSigma) = \operatorname{span} \biggl\{ \biggl[ \underbrace{ \begin{matrix}0 & \cdots& 0 \end{matrix} }_{n_{E}}\quad \underbrace{ \begin{matrix}1 & \cdots& 1 \end{matrix} }_{n_{I_{1}}}\quad \underbrace{ \begin{matrix}-\frac{n_{I_{1}}}{n_{I_{2}}} & \cdots& -\frac {n_{I_{1}}}{n_{I_{2}}} \end{matrix} }_{n_{I_{2}}} \biggr] \biggr\} . $$

(15)

We can check that $\operatorname{Fix}(\varSigma)$ is a subspace of V; furthermore $\dim\operatorname{Fix}(\varSigma) = 1$ because it can be described as the span of a single vector.

Thus, the equivariant branching lemma tells us that we can expect a new branch of fixed points in which the inhibitory cells break up into two groups (therefore we refer to this as the “$I_{1}/I_{2}$ branch”).

If the clusters are of equal size ($n_{I_{1}} = n_{I_{2}}$), then the solution branch shows the pattern $(0,x_{I_{1}},-x_{I_{1}})$ (by uniqueness it suffices to show that such a branch exists). To see this, first observe that

$$\frac{dx_{E}}{dt} = -x_{E} - \frac{\mu_{E}}{\sqrt{N}}\tanh(g x_{E}) + C, \qquad\frac{dx_{I_{1,2}}}{dt} = -x_{I_{1,2}} + \frac{\alpha\mu _{E}}{\sqrt{N}}\tanh(g x_{I_{1,2}}) + C, $$

where

$$C = \frac{\sqrt{N} \mu_{E} \alpha}{\alpha+1} \bigl( \tanh(g x_{E}) -(1/2) \tanh(g x_{I_{1}}) - (1/2) \tanh(g x_{I_{2}}) \bigr). $$

If $x_{I_{2}} = -x_{I_{1}}$, then $\tanh(g x_{I_{2}}) = -\tanh(g x_{I_{I}})$, and therefore

$$ \begin{aligned} \frac{dx_{I_{1}}}{dt} + \frac{dx_{I_{2}}}{dt} & = -x_{I_{1}} -x_{I_{2}} + \frac{\alpha\mu_{E}}{\sqrt{N}} \bigl( \tanh(g x_{I_{1}}) + \tanh(g x_{I_{2}}) \bigr) + 2C \\ & = -x_{I_{1}} +x_{I_{1}} + \frac{\alpha\mu_{E}}{\sqrt{N}} \bigl( \tanh(g x_{I_{1}}) - \tanh(g x_{I_{1}}) \bigr) + 2C = 2C, \end{aligned} $$

where

$$\begin{aligned} C& = \frac{\sqrt{N} \mu_{E} \alpha}{\alpha+1} \bigl( \tanh(g x_{E}) -(1/2) \tanh(g x_{I_{1}}) + (1/2) \tanh(g x_{I_{1}}) \bigr) \\&= \frac{\sqrt{N} \mu_{E} \alpha }{\alpha+1} \tanh(g x_{E}).\end{aligned} $$

Since $\frac{dx_{I_{1}}}{dt} + \frac{dx_{I_{2}}}{dt} = 0$, $\tanh(g x_{E}) = 0 \Rightarrow x_{E} = 0$.

Returning to the inhibitory degrees of freedom, we see their equations are now decoupled:

$$\frac{dx_{I_{1,2}}}{dt} = -x_{I_{1,2}} + \frac{\alpha\mu_{E}}{\sqrt {N}}\tanh(g x_{I_{1,2}}). $$

A fixed point has three possible solutions. If $g > g^{\ast }$; then one is $x_{I_{1,2}} = 0$, whereas the others can be found by inverting a simple expression relating g and $x_{I_{1}}$ along the solution branch:

$$ -x_{I_{1}} + \frac{\alpha\mu_{E}}{\sqrt{N}}\tanh(g x_{I_{1}}) = 0\quad \Rightarrow\quad g = \frac{1}{x_{I_{1}}} \tanh^{-1} \biggl( \frac{\sqrt {N}x_{I_{1}}}{\alpha\mu_{E}} \biggr). $$

(16)

Thus, we can solve for g as a function of $x_{I_{1}}>0$ and set $x_{I_{2}} = -x_{I_{1}}$; checking the Taylor expansion of Eq. (16) confirms that $x_{I_{1}} \rightarrow0$ as $g\rightarrow g^{\ast}$.

3.2 Hopf Bifurcation (on Trivial Solution) Leading to Limit Cycles

The trivial solution is next expected to have a bifurcation when the complex pair of eigenvalues of $-\mathbf {I}+ g \mathbf {H}/\sqrt{N}$ crosses the imaginary axis, that is, when

$$ g = \frac{2 \sqrt{N}}{\mu_{E} (\alpha-1)}. $$

This is a simple eigenvalue pair, with real eigenspace (again by Example 1) consisting of vectors with all E cells synchronized and all I cells synchronized. This is a two-dimensional vector space: therefore, we expect a branch of periodic solutions to arise in which the excitatory neurons and inhibitory neurons are each synchronized. Here $\varSigma= \varGamma= S_{n_{E}} \times S_{n_{I}}$.

3.3 Hopf Bifurcation (on $I_{1}/I_{2}$ Branch) Leading to Limit Cycles

On the branch $(x_{E}, x_{I_{1}}, x_{I_{2}})$, we find two singularities that lead to new structures. Most significantly, we find a Hopf bifurcation that leads to a branch of limit cycles when a pair of complex eigenvalues crosses the imaginary axis. By Example 2 the corresponding eigenspace is fixed under $\varSigma= S_{n_{E}} \times S_{n_{I_{1}}} \times S_{n_{I_{2}}}$. Thus, it is a two-dimensional subspace of $\operatorname {Fix}(\varSigma)$; therefore, by the equivariant Hopf theorem, the family of periodic solutions that emerges here also has Σ as its group of symmetries.² Furthermore, in all examples that we encountered, the Hopf bifurcation was supercritical (this was checked numerically); if in addition all other eigenvalues satisfied $\operatorname{Re}(\lambda) < 0$, then the resulting periodic orbits would be stable.

In general, it is not feasible to solve for $g^{H}$ symbolically: this requires us to solve for the roots of a cubic polynomial involving exponential functions (e.g. $\tanh(g x_{E})$) of implicitly defined parameters $x_{E}$, $x_{I_{1}}$, and $x_{I_{2}}$. However, we can identify the bifurcation numerically (all continuations were performed with MATCONT [28]), and we have found this bifurcation on every specific $I_{1}/I_{2}$ branch in every specific system we have investigated.

We can also track the branch of Hopf points numerically in the reduced system $(x_{E}, x_{I_{1}}, x_{I_{2}})$ (described in Sect. 4.2), which has the added benefit that the complexity of the system does not increase with N (rather, N is a bifurcation parameter). Here again, we can confirm that the Hopf bifurcation is present in the system for any N and have done so for several example $n_{I_{1}}/n_{I_{2}}$ ratios in Sect. 4.3.

3.4 Branch Points (on $I_{1}/I_{2}$ Branch) Leading to New Fixed Point Branch

We may also find branch points on the $(x_{E}, x_{I_{1}}, x_{I_{2}})$ curve in which one of the inhibitory clusters breaks into a further cluster. This occurs if the eigenspace corresponding to $x_{I_{1}}$, say, has a real eigenvalue going through zero. Because these did not appear to play a significant role in our simulations, we do not consider them further.

3.5 Reduced Self-Coupling

For the remainder of the paper, we focus on absent self-coupling ($b_{E}, b_{I} = 0$); here we note how our conclusions should be modified in a more general case. At the origin, the locations—but not the qualitative behavior—of the bifurcations change. In Example 1.1, a branch point occurs at $g^{\ast} = \frac{\sqrt{N}}{\alpha\mu_{E}}$; in Example 1, the location now is $g^{\ast, b} = \frac{\sqrt{N}}{\alpha\mu_{E} (1-b_{I})}$ since always $b_{I} \le1$ and $g^{\ast,b} \ge g^{\ast}$.

Similarly, the Hopf bifurcation that occurs at

$$ g^{H} = \frac{2 \sqrt{N}}{\mu_{E} (\alpha-1)} $$

with no self-coupling will now occur at

$$ g^{H,b} = \frac{2 \sqrt{N}}{\mu_{E} ( \alpha(1-b_{I}) - (1-b_{E}) )}, $$

provided that $\alpha(1-b_{I}) - (1-b_{E}) > 0$ (see the formula (9) for λ).

The relative ordering of $g^{H}$ and $g^{H,b}$ depends on the relative values of $b_{E}$ and $b_{I}$; if $b_{E} - \alpha b_{I} \le0$, then $g^{H,b} \ge g^{H}$; otherwise, $g^{H,b} < g^{H}$. However, we can check that the branch point almost always occurs for a smaller coupling value (than the Hopf point), that is, $g^{\ast,b} \le g^{H,b}$ with equality if and only if $b_{E} = 1$.

3.6 Inhibition-Dominated Networks

In this paper, we have focused on balanced networks ($\alpha= n_{E}/n_{I}$). We briefly summarize how our conclusions change in inhibition-dominated networks ($\alpha> \tilde{\alpha} \equiv n_{E}/n_{I}$). At the origin, the location of $g^{\ast}$ is still given by $\frac{\sqrt{N}}{\alpha\mu_{E} (1-b_{I})}$, although now, since $\alpha> \tilde{\alpha}$, the critical coupling value decreases, that is, $g^{\ast,b,in} < g^{\ast,b}$.

In Eqs. (9) and (10), the condition that $n_{E} = \alpha n_{I}$ has been used; to remove this restriction, replace any instance of $n_{E}$ in the right column of $\tilde{\mathbf {J}}$ with $\alpha n_{I}$. The condition for a Hopf bifurcation to occur at the origin now is (using the trace of $\tilde{\mathbf {J}}$ from Eq. (9)):

$$n_{E} - \alpha n_{I} + \alpha(1-b_{I}) - (1-b_{E}) > 0\quad \Rightarrow\quad(\tilde{\alpha}-\alpha) n_{I} + \alpha(1-b_{I}) - (1-b_{E}) > 0 $$

or

$$\alpha(1-b_{I}) - (1-b_{E}) > (\alpha- \tilde{\alpha}) (1-f) N. $$

Thus the Hopf bifurcation still occurs as long as inhibition is not too strong (as measured by $\alpha- \tilde{\alpha}$); however, this depends on N.

4 A Bifurcation-Preserving Reduced-Order Model

In this section, we show that we can construct a reduced-order model that preserves the dynamics and bifurcation structure of the full system, but with a dramatic reduction in the number of degrees of freedom. For a cortex-like ratio of E to I cells, the interesting bifurcations involve the eigenvalues associated with the inhibitory cells or the complex pair. As a result, all the “action” is in the I cells, with the E cells always perfectly synchronized. We can formalize this as follows.

Lemma 3

Any fixed point or periodic solution of (1)–(3) with $\epsilon= 0$ has a synchronized excitatory population, that is, $x_{j}(t) = x_{k}(t)$ for any $j,k \leq n_{E}$.

Proof

Consider the activity of two distinct E cells, say $x_{1}$ and $x_{2}$. Then

$$\begin{aligned} \frac{d(x_{1} - x_{2})}{dt} ={}& \dot{x_{1}} - \dot{x_{2}} \\ ={}& {-}x_{1} + \frac{\mu_{E}}{\sqrt{N}} \bigl(b_{E} \tanh(g x_{1}) + \tanh(g x_{2}) \bigr) \\& - \biggl(-x_{2} + \frac{\mu_{E}}{\sqrt{N}} \bigl(\tanh(g x_{1}) + b_{E} \tanh(g x_{2}) \bigr) \biggr) \end{aligned}$$

(17)

$$\begin{aligned} ={}& {-}(x_{1} - x_{2}) - \frac{(1-b_{E})\mu_{E}}{\sqrt{N}} \bigl( \tanh(g x_{1}) - \tanh(g x_{2}) \bigr) \\ ={}& {-}(x_{1} - x_{2}) \\& - \frac{(1-b_{E})\mu_{E}}{\sqrt{N}} \tanh \bigl( g(x_{1} - x_{2}) \bigr) \bigl( 1 - \tanh(g x_{1}) \tanh(g x_{2}) \bigr). \end{aligned}$$

(18)

The first numbered line, (17), contains so few terms because everything depending on other variables ($x_{3}$, and so forth) cancels out; the second numbered line, (18), uses a sum identity for the tanh function. Then

$$ \begin{aligned} \frac{d \| x_{1} - x_{2} \|^{2}}{dt} ={}& 2(x_{1} - x_{2}) \times \biggl[ -(x_{1} - x_{2})\\&- \frac{(1-b_{E})\mu_{E}}{\sqrt{N}} \tanh \bigl( g(x_{1} - x_{2}) \bigr) \bigl( 1 - \tanh(g x_{1}) \tanh(g x_{2}) \bigr) \biggr] \\ ={}& {-}2\| x_{1} - x_{2} \|^{2}\\& - \frac{(1-b_{E})\mu_{E}}{\sqrt{N}} ( (x_{1}-x_{2}) \tanh \bigl(g(x_{1}-x_{2} ) \bigr)\\&\times \bigl( 1 - \tanh(g x_{1}) \tanh(g x_{2}) \bigr) \\ \leq{}&{-}2\| x_{1} - x_{2} \|^{2} \end{aligned} $$

with equality if and only if $x_{1} = x_{2}$. In the last line, we use the facts that $x \tanh(gx) \geq0$ and $( 1 - \tanh(g x) \tanh(g y)) \geq0$ for any real numbers x, y, and $g > 0$. Therefore the distance $\| x_{1} - x_{2} \|$ always decreases along a trajectory, unless already $x_{1} = x_{2}$. □

As a consequence, any fixed point or period solution present in the full system is also present in the following reduced system, where we collapse all of the excitatory degrees of freedom into one $x_{E}$ (assuming that $b_{E}=0$):

$$\begin{aligned} \dot{x}_{E} ={}& {-}x_{E} + \biggl( \frac{N\alpha}{\alpha+1} - 1 \biggr) \biggl( \frac{\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{E}) - \sum _{j=1}^{n_{I}} \biggl( \frac{\alpha\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{I_{j}}), \end{aligned}$$

(19)

$$\begin{aligned} \dot{x}_{I_{i}} ={}& {-}x_{I_{i}} + \frac{N\alpha}{\alpha+1} \biggl( \frac {\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{E}) \\ & - \sum _{j=1,j\neq i}^{n_{I}} \biggl( \frac{\alpha\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{I_{j}}), \quad i=1,\dots,n_{I}. \end{aligned}$$

(20)

Here, $n_{I}$ is fixed, whereas $N = (\alpha+ 1)n_{I}$ is a parameter of the system. This allows us to explore solutions in an $(n_{I} + 1)$-dimensional system rather than an $(\alpha+1) n_{I}$-dimensional system.

We first demonstrate the bifurcation structure on a small network with $N=20$, in which we can comfortably confirm our findings on the full system with numerical continuation (all diagrams shown here were computed using MATCONT [28]). We treat the global coupling strength g as our bifurcation parameter: the origin is an equilibrium point for all g. At $g^{\ast} = \sqrt{N}/\alpha\mu_{E}$, $n_{I} - 1$ eigenvalues pass through the origin ($\epsilon= 0$). This is a branch point: because of symmetry, there exists a branch corresponding to each possible split of the I cells into two clusters. In Fig. 1A, we show the solution branches that arise in the $N=20$ system (up to symmetry, that is, although there are four possible 3-1 splittings, we display only one here). Because there are four inhibitory cells, there are two possible splits, 3-1 and 2-2. Both have a branch that originates from the branch point on the origin $g^{\ast} = \sqrt {N}/\alpha\mu_{E}$ (in Fig. 1B, these are labeled as “3-1” and “2-2^*”, respectively). Along the 2-2 branch, the E cells have zero activity (this is generally the case where the I cells split into two equal clusters). Both branches then have a Hopf bifurcation from which a branch of limit cycles emerges, unstable in the 3-1 case and stable in the 2-2 case. The resulting limit cycle respects the clustering, but the E cell activity is no longer zero in the 2-2 case.

Fig. 1

Solution branches from symmetry in the full system ($N=20$). A Solution branches (up to symmetry) in $(g, x_{I_{1}}, x_{I_{3}})$ coordinates. B Same figure, viewed in the $(g, x_{I_{3}})$ plane. C Solution branches from the reduced $(n_{I} + 1)$-dimensional system, with $n_{I} = 4$ and $\alpha= 4$. Up to symmetry, this figure depicts identical solutions as the left and center panels. Markers indicate: Hopf bifurcations (red asterisks); branch points (black triangles); neutral saddles (green crosses)

The 2-2 branch has a further branch point, at which the new branch breaks the odd symmetry (i.e. $x_{I_{2}}=-x_{I_{1}}$) and E cell activity moves away from zero. One further branch occurs, in which one of the two cell clusters breaks apart resulting in a 2-1-1 clustering. Why did the 2-1-1 branch come off of the 2-2 branch, rather than of the 3-1 branch? At this time, we have no principled answer. Finally, the origin has a Hopf bifurcation in which the E and I cells separately cluster (we will refer to this as the “E/I” limit cycle).

We next perform the same continuation on the corresponding reduced (five-dimensional: $x_{E}$ and $x_{I_{1}}-x_{I_{4}}$) system. The equilibrium branch structure is shown in Fig. 1C. Up to a permutation of the inhibitory coordinate labels (we do not force the same cell cluster identities to be tracked in both continuations), the curves are identical.

Returning to the full system, we now consider the limit cycles that emerge from the three Hopf bifurcations we identified (on the 3-1 branch, 2-2 branch, and the origin). In Fig. 2A, we plot the period vs. the coupling parameter g. In Fig. 2B, we can see that the 3-1 branch is stable for $g \gtrapprox2.8$ and the 2-2 branch for $g \lessapprox3$. We note that the 3-1 and 2-2 branches appear to terminate on the E/I branch, shown in Fig. 3A. Indeed, at this point, all three limit cycles coincide, as we can see in Fig. 3B.

Fig. 2

Limit cycles in the full system ($N=20$). A Period vs. coupling parameter g. B Closeup of period vs. g, near a stability change in the 3-1 curve. Colors are: 3-1 (blue), 2-2 (green), E/I (orange). Markers indicate: Neimark–Sacker (red asterisk), limit point of cycles (magenta square), branch pt. of cycles (black triangle)

Fig. 3

Three limit cycles branches collide in the full system ($N=20$). A Closeup of period vs. g, near the point where three limit cycle branches collide. B Limit cycles at the point of collision in $(x_{E}, x_{I_{2}}, x_{I_{3}})$ coordinates. Colors are: 3-1 (blue), 2-2 (green), E/I (orange). Markers indicate: Neimark–Sacker (red asterisk), limit point of cycles (magenta square), branch pt. of cycles (black triangle)

4.1 A Larger System: $n_{I} = 10$

The real power of the reduced-order model becomes evident when we increase the network size. We now show results for $n_{I} = 10$: note that, for $\alpha=4$, we reduce the dimension of the system from 50 to 11. In Fig. 4A, we show the equilibrium branches found in this system; the same diagram is plotted in the $(g, x_{I_{10}})$ plane, with labeled curves, in Fig. 4B. As expected, a branch point occurs on the origin at $g = \sqrt{N}/\alpha\mu_{E}$. From this point we see that 5-5, 6-4, 8-2, and 2-3-5 solutions emerge. The 5-5 branch has zero activity in the excitatory cells; as in the previous example, a further branching point yields 5-5 solutions where $x_{E} \neq0$. A further branch point gives a curve of 1-4-5 cluster solutions. There are Hopf bifurcations on each of the branches that appear at the origin.

Fig. 4

Equilibrium solution branches in the “$x_{I}+1$” system. In this computation, $n_{I} = 10$; with the chosen parameters, this is equivalent to a full system with $N=50$. Two different viewpoints are shown (panels A and B). Markers indicate: Hopf bifurcations (red asterisks); branch points (black triangles); neutral saddles (green crosses)

We note that most of these branches are cases where splitting is minimal; that is, a single cluster breaks into two (rather than into three). This confirms our intuition from the equivariant branching lemma, which guarantees the existence of a unique branch of solutions for each subgroup Σ for which the fixed point subspace on the kernel of the Jacobian at the bifurcation point has the right dimension $\dim \operatorname {Fix}(\varSigma) = 1$. (A more general result extends a version of this result to cases of odd dimensions [29].) At the origin, for example, the kernel at the branch point is $(n_{I}-1)$-dimensional:

$$ \mathbf {v}= \left [ \begin{matrix} 0 & \mathbf {u}\end{matrix} \right ], \quad \mathbf {u}\in \mathbb {R}^{n_{I}}, \mathbf {u}^{T} \mathbf{1} = 0. $$

(21)

In this case,

$$ \mathbf {v}= \left [ \begin{matrix} 0 & v_{1} & \cdots& v_{10} \end{matrix} \right ], \quad v_{1} + \cdots+ v_{10} = 0. $$

(22)

However, this lemma does not exclude the possibility of other solution types, and little is known in general about fixed point subspaces of even dimensions: such solutions have been found in some systems (e.g., [29]), but there is currently no general theory guaranteeing or ruling out such solutions [30]. In this system, at least one branch corresponds to a subgroup Σ for which $\dim \operatorname {Fix}(\varSigma) = 2$, the 2-3-5 branch.

We next look at the limit cycles that emerge from Hopf bifurcations on each branch from the origin. Period decreases with g (Fig. 5A). As in the $N=20$ system, each branch terminates where it collides with the E/I limit cycle branch that emerges from the Hopf point at the origin (Fig. 5B).

Fig. 5

Limit cycle branches in the “$x_{I}+1$” system. A Period vs. coupling parameter g. B Closeup of period vs. g, near the point where several limit cycle branches collide. Colors are: 5-5 (blue), 6-4 (green), 8-2 (orange), 2-3-5 (red), 2-3-5 secondary (pink), E/I (purple). Markers indicate: Neimark–Sacker (red asterisk), limit point of cycles (magenta square), branch pt. of cycles (black triangle)

4.2 Reduced System: 3-Cluster $(x_{E}, x_{I_{1}}, x_{I_{2}})$

We can gain additional insight into arbitrarily large systems by reducing (1) using the assumption of a three-cluster grouping into populations of $n_{E}$, $n_{I_{1}}$, and $n_{I_{2}}$, whose activities are given by $x_{E}$, $x_{I_{1}}$, and $x_{I_{2}}$, respectively. The reduced system is

$$ \begin{aligned} \dot{x}_{E} ={}& {-}x_{E} + (n_{E} - 1) ( \mu_{E}/\sqrt{N}) \tanh(g x_{E})\\ & + n_{I_{1}} ( \mu_{I}/\sqrt{N}) \tanh(g x_{I_{1}}) + n_{I_{2}} ( \mu_{I}/\sqrt{N}) \tanh(g x_{I_{2}}), \\ \dot{x}_{I_{1}} ={}& {-}x_{I_{1}} + n_{E} ( \mu_{E}/\sqrt{N}) \tanh(g x_{E}) \\ &+ (n_{I_{1}} - 1) ( \mu_{I}/\sqrt{N}) \tanh(g x_{I}) + n_{I_{2}} ( \mu_{I}/\sqrt{N}) \tanh(g x_{I_{2}}), \\ \dot{x}_{I_{2}} ={}& {-}x_{I} + n_{E} ( \mu_{E}/\sqrt{N}) \tanh(g x_{E})\\ & + n_{I_{1}} ( \mu_{I}/\sqrt{N}) \tanh(g x_{I_{1}})+ (n_{I_{2}} - 1) ( \mu_{I}/\sqrt{N}) \tanh(g x_{I_{2}}). \end{aligned} $$

We can also parameterize the clustering with α and β such that $n_{E} = \frac{\alpha}{\alpha+ 1}N$, $n_{I_{1}} = \frac{\beta }{\beta+1}\frac{1}{\alpha+ 1}N$, and $n_{I_{2}} = \frac {1}{\beta+1}\frac{1}{\alpha+ 1}N$, that is, β gives the ratio of $n_{I_{1}}$ to $n_{I_{2}}$, just as α gives the ratio of $n_{E}$ to $n_{I}$. Then the equations become (also using the relationship $\mu _{I} = -\alpha\mu_{E}$):

$$\begin{aligned} \dot{x}_{E} ={} & {-}x_{E} + \biggl( \frac{N\alpha}{\alpha+1} - 1 \biggr) \biggl( \frac{\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{E}) \\ & - \frac{N\beta }{(\beta+1)(\alpha+1)} \biggl( \frac{\alpha\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{I_{1}}) \\ &- \frac{N}{(\beta+1)(\alpha+1)} \biggl(\frac{\alpha\mu_{E}}{\sqrt {N}} \biggr) \tanh(g x_{I_{2}}), \end{aligned}$$

(23)

$$\begin{aligned} \dot{x}_{I_{1}} ={} & {-}x_{I_{1}} + \frac{N\alpha}{\alpha+1} \biggl( \frac {\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{E}) \\ & - \biggl(\frac{N\beta }{(\beta+1)(\alpha+1)}-1 \biggr) \biggl( \frac{\alpha\mu_{E}}{\sqrt {N}} \biggr) \tanh(g x_{I_{1}}) \\ & - \frac{N}{(\beta+1)(\alpha+1)} \biggl( \frac{\alpha\mu _{E}}{\sqrt{N}} \biggr) \tanh(g x_{I_{2}}), \end{aligned}$$

(24)

$$\begin{aligned} \dot{x}_{I_{2}} ={} & {-}x_{I_{2}} + \frac{N\alpha}{\alpha+1} \biggl( \frac {\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{E}) \\& - \frac{N\beta}{(\beta +1)(\alpha+1)} \biggl( \frac{\alpha\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{I_{1}}) \\ & - \biggl(\frac{N }{(\beta+1)(\alpha+1)}-1 \biggr) \biggl( \frac {\alpha\mu_{E}}{\sqrt{N}} \biggr) \tanh(g x_{I_{2}}). \end{aligned}$$

(25)

Here, we can treat N, α, and β as continuously varying bifurcation parameters. When N, $\frac{N}{\alpha+1}$, and $\frac {N}{(\beta+1)(\alpha+1)}$ are all positive integers, the reduced system (23)–(25) lifts onto an N-cell network.

4.3 Scaling with System Size

We can use this reduced system to explore how the system behaves as N increases. The system in Eqs. (23)–(25) allows N to be a continuously varying parameter; therefore, we can vary N while holding all other parameters fixed. Notably, we keep β fixed; thus, we track the behavior of a specific partition ratio of inhibitory cells (such as 1-to-1 or 3-to-1) as N increases. When N, $\frac{N}{\alpha+1}$, and $\frac{N}{(\beta+1)(\alpha +1)}$ are all positive integers, the reduced system lifts onto an N-cell network; at each such N, we can track the $I_{1}/I_{2}$ fixed point branch from the known bifurcation point $g^{\ast} = \sqrt {N}/\alpha\mu_{E}$.

In Fig. 6A, we show $(x_{E}, x_{I_{I}}, x_{I_{2}})$ as a function of g for the partition $n_{I_{1}} = n_{I_{2}}$. Colors cycle through N; for each N, the curves from top to bottom indicate $x_{I_{1}}$, $x_{E}$, and $x_{I_{2}}$. We can also locate the Hopf bifurcation along this branch at $g^{H}$ and measure the frequency of the periodic solutions that emerge at that point. We plot these quantities in Fig. 6B: we can see they each scale like $\sqrt{N}$. In Fig. 7, we show the same quantities computed for two more examples, 1-to-4 and 2-to-3 partitions, respectively: the $\sqrt{N}$ scaling of both $g^{H}$ and $\omega(g^{H})$ persists for these different partitions.

Fig. 6

Behavior of three-cluster solutions, for equal-size inhibitory clusters ($n_{I_{2}} = n_{I_{1}}$). A Activity levels on the $I_{1}/I_{2}$ solution branch. Colors cycle through $N = 10, 20, 30, 40, 50, 60, 80, 100, 120, 140, 160, 200, 240, 280, 300, 400, 500, 600, 700, 800\text{, and }1000$ (note: $n_{I} = N/5$ must be a multiple of 2). B Bifurcation values $g^{\ast}$, $g^{H}$, and Hopf frequency $\omega(g^{H})$ as functions of N

Fig. 7

Behavior of three-cluster solutions as system size N increases. A–B clusters where inhibitory cells break into groups with size ratio 1 to 4 ($n_{I_{2}} = 4 n_{I_{1}}$). A Activity levels on the $I_{1}/I_{2}$ solution branch. Colors cycle through $N = 25, 50, 75, 100, 125, 150, 200, 250, 300, 400, 500, 600, 700, 800\text{, and }1000$ ($n_{I} = N/5$ must be a multiple of 5). B Bifurcation values $g^{\ast}$, $g^{H}$, and Hopf frequency $\omega (g^{H})$ as functions of N. C–D Solution branch in which inhibitory cells break into groups with size ratio 2 to 3 ($n_{I_{2}} = (3/2) n_{I_{1}}$). C Activity levels on the $I_{1}/I_{2}$ solution branch. Colors cycle through $N = 25, 50, 75, 100, 125, 150, 200, 250, 300, 400, 500, 600, 700, 800\text{, and }1000$ ($n_{I} = N/5$ must be a multiple of 5). D Bifurcation values $g^{\ast}$, $g^{H}$, and Hopf frequency $\omega(g^{H})$ as functions of N

The $\sqrt{N}$ scaling of $g^{\ast}$, $g^{H}$, and $\omega(g^{H})$ yields insight into the expected behavior of these solutions. First, we should expect these oscillations to become less observable as N increases; $g^{\ast}$ will eventually reach unphysical values. Second, we should expect the oscillations to become faster as N increases, also eventually reaching an unphysical frequency. Thus, we expect the phenomenon we describe here to be most relevant for small-to-medium N. In the next section, we show that we can easily find an example for $N=200$; the oscillation period in that example is comparable to the membrane time constant, which is a reasonable upper bound for frequency.

5 Demonstration of Relevance to Random Networks ($\epsilon> 0$)

We next demonstrate that the bifurcation structure we have described can explain low-dimensional dynamics in example random networks. We return to Eqs. (1) and (2) but now let $\epsilon> 0$. The right-hand side of Eq. (1) can be readily shown to be locally Lipschitz continuous in $\mathbb{R}^{N}$; thus, solutions will vary continuously as a function of parameters (such as ϵ). In particular, we can expect a hyperbolic periodic orbit at $\epsilon= 0$ to persist for some range of $\epsilon \in[0, \epsilon_{0})$; here we numerically demonstrate this persistence.

We chose parameters $\mu_{E} = 0.7$, $\sigma_{E}^{2} = 0.625$, and $\sigma _{I}^{2} = 2.5$. (For $\epsilon= 1$, the off-diagonal entries of the resulting random matrices are chosen with the same means and variances as in [11].) We performed a series of simulations in which we fixed A and computed solution trajectories for a range of ϵ in between 1 and 0. As ϵ decreases, the network connectivity matrix transitions from full heterogeneity (similar to [11]) to the deterministic case studied earlier.

In Figs. 8A–B, we show two examples of random networks of size $N=20$ and $g=3$. For $\epsilon= 0$ (bottom panel), we indeed see a three-cluster solution as predicted. Consistent with our earlier results on limit cycle stability, we see the 3-1 clustering rather than the 2-2 clustering here (in both examples here, $n_{I_{1}} = 1$ and $n_{I_{2}} = 3$). The same periodic solution persists as ϵ increases and is still recognizable at $\epsilon= 1$, illustrated in the top panel (we note that because of the odd symmetry of the governing equations, −x is also a valid trajectory and appears as a reflection across the time axis). In Fig. 8B, the period of the oscillations discernibly increases with ϵ.

Fig. 8

Oscillations persist despite randomness in the connectivity matrix. A–B Solutions for two different networks of size $N = 20$: $g=3$. Here, $\sqrt{N} \mathbf {G}\equiv \mathbf {H}+ \epsilon \mathbf {A}_{1,2}$. From top to bottom: $\epsilon^{2} = 1, 2^{-1}, 2^{-2}, 2^{-3}, 2^{-4}, 2^{-5}, 2^{-6}\text{, and }0$. C Solutions for a network of size $N=200$. The connectivity matrix is given by $\sqrt{N} \mathbf {G}\equiv \mathbf {H}+ \epsilon \mathbf {A}$ for a single A. From top to bottom: $\epsilon^{2} = 1, 2^{-1}, 2^{-2}, 2^{-3}, 2^{-4}, 2^{-5}, 2^{-6}\text{, and }0$. D Solutions for a network of size $N=200$, but where $\sqrt{N} \mathbf {G}\equiv\epsilon \mathbf {A}$ (i.e. no mean). The random connectivity matrix A is the same as in panel C. In panels A–B, the traces of $n_{E}$ excitatory (blue) and $n_{I}$ inhibitory (red) neurons are shown. In panels C–D, only a subset (twenty E and six I cells) is displayed. E, F Eigenvalues of the connectivity matrices $\sqrt{N} \mathbf {G}\equiv \mathbf {H}+ \epsilon \mathbf {A}$ (E) and $\sqrt {N} \mathbf {G}\equiv\epsilon \mathbf {A}$ (F) used in panels C and D, respectively

In Fig. 8C, we show an example from a larger system with $N=200$. Here $g=6$; note that a larger coupling value is needed to exceed the bifurcation of the origin at $g^{\ast} = \sqrt {N}/\alpha\mu_{E}$. A periodic trajectory is evident in all panels. As in the smaller examples, the period of oscillations increases with ϵ.

To highlight that this structure is caused by the mean connectivity H, we repeat the sequence of simulations, but integrating the system without the mean matrix H. The results are shown in Fig. 8D: here the same A, initial condition $\mathbf {x}_{0}$, values of ϵ, and coupling parameter g were used; therefore the only difference between each panel in Fig. 8D vs. its counterpart in Fig. 8C is the absence of the mean connectivity matrix H. Without H, the origin is stable for ϵ sufficiently small (for $g=6$, ${\epsilon^{2} < 1/36}$); hence the zero solutions in the bottom two panels. As $\epsilon^{2}$ increases beyond that value, we see a fixed point, followed by periodic and then apparently chaotic solutions (for $\epsilon^{2} > 2^{-2}$, a decomposition of the trajectories in terms of principal components requires a large number of orthogonal modes, here in excess of 25). In addition, the characteristic timescale is much longer than in Fig. 8C (note the difference in the time axes).

Finally, we can contrast the nonlinear behavior with the predicted linear behavior by examining the spectra of the connectivity weight matrices. In Figs. 8E and 8F, we plot the eigenvalues of $(\mathbf {H}+ \epsilon \mathbf {A})/\sqrt{N}$ and $\epsilon \mathbf {A}/\sqrt{N}$, respectively, for the specific networks shown in Figs. 8C–D and for several values of ϵ. When $\epsilon= 0$, the eigenvalues in Fig. 8E cluster into two locations on the real axis, with the exception of one complex pair, as discussed in Example 1. In contrast, the eigenvalues in Fig. 8F all lie at zero for $\epsilon= 0$. As ϵ increases, the eigenvalues “fan out” from their point locations until they fill a disc of radius g (here, $g=6$). At $\epsilon= 1$, both matrices have dozens of eigenvalues in the right-half plane.

The similarity in appearance between the spectra illustrated in Figs. 8E and 8F is partially the result network balance (i.e. that $\alpha= \frac{f}{1-f}$). The stochastic part of the connectivity matrix G is scaled so that its spectral radius is constant with N; however, as we noted after Example 1.2, the real part of the eigenvalues of the deterministic matrix $\mathbf {H}/\sqrt{N}$ scale like $O(1/\sqrt{N})$. Therefore, we expect the stochastic part of the connectivity matrix to dominate the deterministic part as $N \rightarrow\infty$ when the network is balanced.

6 Discussion

In summary, we studied a family of balanced excitatory–inhibitory firing rate networks that satisfy Dale’s law for arbitrary network size N. When there is no variance in synaptic connections—each excitatory connection has strength $\frac{\mu_{E}}{\sqrt{N}}$, and each inhibitory connection has strength $\frac{\mu_{I}}{\sqrt {N}}$—we find a family of deterministic solutions whose existence can be inferred from the underlying symmetry structure of the network. These solutions persist in the dynamics of networks with quenched variability—that is, variance in the connection strengths— even when the variance is large enough that the envelope of the spectrum of the connectivity matrix approaches that of a Gaussian matrix. This offers a striking example in which linear stability theory is not useful in predicting transitions between dynamical regimes. Given the increasing interest in network science and, in particular, networked dynamical systems, such observations concerning the impact of symmetry of connectivity can be extremely valuable for studying stability, bifurcations, and reduced-order models.

6.1 Role of the Deterministic Perturbation H

Gaussian matrices are a familiar object of study in the random matrix community, where Hermitian random matrices are motivated by questions from quantum physics. Rajan and Abbott [11] studied balanced rank one perturbations of Gaussian matrices and found that the spectrum is largely unchanged. These results have since been extended to more general low-rank perturbations [12, 13]. More recently, Ahmadian et al. [15] studied general deterministic perturbations in the context of neural networks. Similarly, recent work has studied extremal values of the spectrum of matrices with modular structure similar to that found here [14]. Our system is not low-rank: in fact, the (seemingly trivial) change in self-coupling makes the deterministic weight matrix full rank, as we see from Lemma 2. Using the procedure developed by Ahmadian et al. [15], we can numerically compute the support of spectrum for $\epsilon> 0$ (not shown): as ϵ grows, this spectral support appears to approach that predicted by a Gaussian matrix or a low-rank perturbation thereof.

However, the more fundamental issue here is that—except for predicting when the origin becomes unstable—the spectrum of the full connectivity matrix is not particularly informative about nonlinear dynamics. Instead, it is the spectrum of the deterministic perturbation that emerges as crucial here; the location of the eigenvalues of this matrix can be used to predict the existence of a family of steady states and limit cycles with very specific structure. In the examples presented here (Fig. 8), these low-dimensional solutions persist even when ϵ is large enough that the spectrum of G is visually indistinguishable from the spectrum of a Gaussian matrix.

It is instructive to compare our findings here with the recent results of del Molino et al. [20], who consider a balanced excitatory–inhibitory system with a similar $1/\sqrt{N}$ scaling of the mean weights. The authors find a slow noise-induced oscillation; similarly to our results here, this oscillation arises despite an unstable connectivity matrix. The two systems differ in the deterministic perturbation: del Molino et al. include self-coupling (their deterministic matrix is rank one), which yields trivial deterministic dynamics without a driving current (in the sense of Example 1.2); thus, they do not see the dynamics described here. Conversely, we do not enforce “perfect balance” $\sum_{j} \mathbf {G}_{ij} = 0$, which they find to be a necessary condition for the slow oscillation to exist; thus we do not see the oscillations described in that paper. Thus, del Molino et al. [20] and the current work present two distinct examples of dynamics that arise in an excitatory–inhibitory system with $1/\sqrt{N}$ scaling of the mean weights, where linear stability of the connectivity matrix is not informative about the nonlinear dynamics.

6.2 Relationship to Other Work

The reduced system described in Sect. 4.2 is similar to a simple version of the Wilson–Cowan equations [31, 32] (recently reviewed in [33, 34]). These equations can be interpreted in terms of coupled neural populations and can be derived as a mean-field limit from large networks. A bifurcation analysis of such a mean-field model was performed recently by Hermann and Touboul [17]. Our system differs in two important ways. First, the strong coupling ($1/\sqrt{N}$) means that a factor of $\sqrt{N}$ remains in the reduced equations. Hermann and Touboul, in contrast, pick $J_{ij} \sim N (\frac{\bar{J}}{N}, \frac{\sigma}{\sqrt{N}} )$; therefore the mean connection strength ($\frac{\bar{J}}{N}$) goes to zero faster than the typical deviation from this mean ($\frac{\sigma}{\sqrt{N}}$): as N becomes large, outgoing synapses are no longer single-signed, in violation of Dale’s law. Similarly, Kadmon and Sompolinsky [19] analyze random diluted networks; they show the equivalence to all-to-all Gaussian networks with nonzero mean connections that scale like ($\frac{\bar{J}}{N}$). If the number of synaptic connections per population is held constant, then dynamic mean-field theory yields predictions for stability valid as $N \rightarrow\infty$.

In contrast, the reduced system in Sect. 4.2 has no nontrivial limit as $N \rightarrow\infty$ and is not necessarily a limit or a system average; rather, it simply gives reduced dynamics in a specific invariant subspace. Ultimately, every solution of the reduced system is also a perfectly accurate solution of the original system. The parameter β allows a single equation to capture arbitrary bisections of the inhibitory population; in principle, adding more equations would allow us further branches to be captured. As another consequence of this scaling, the location of bifurcations $g^{\ast}$ and $g^{H}$ and the expected frequency of oscillations $\omega(g^{H})$ will scale like $\sqrt{N}$; arguably, $g^{\ast}$ and $\omega(g^{H})$ will reach unphysical levels as N becomes large.

Finally, stronger mean scaling may underlie another difference from previous work; analyzing networks with $1/N$ scaling, other authors have found population-level oscillations via Hopf bifurcations in reduced equations for mean activity [35, 36]. However, in those works the oscillations are not necessarily observable at the level of individual cell activity (e.g., Fig. 3 in [36]); here, we have distinct cell-level and population-level oscillations.

Analysis of spontaneous symmetry breaking enjoys a rich history in mathematical biology and, in particular, in mathematical neuroscience. However, the literature we are aware of identifies symmetry-breaking in structured networks dominated by deterministic behavior. For example, symmetry breaking has been hypothesized to underlie the dynamics of visual hallucinations [37] and ambiguous visual percepts [38], central pattern generators that govern rhythmic behaviors of breathing, eating, and swimming [39–41], and periodic head/limb motions [42–44]. Most recently, Kriener et al. [45] investigate a Dale’s law-conforming orientation model and find that the dynamics are affected by a translation symmetry imposed by the regularity of the cell grid. In contrast, the present paper identifies an important role for symmetry in a family of networks usually thought of as dominated by randomness.

6.3 Future Directions

In this paper, we have focused on analyzing the deterministic system underlying a family of Dale’s law-conforming networks. However, our ultimate interest is in the perturbation away from this system: a full characterization of the dynamics still remains to be completed. Thus far, we have observed more variable behavior in constrained vs. Gaussian networks: at the same coupling parameter g, individual networks display behavior ranging from periodic (as in Fig. 8C) to chaotic, suggesting that this task will be more subtle than for Gaussian networks (also see [20]). Future work will examine this in more detail.

Recent research has focused on the computational power of random networks in the (nominally unpredictable) chaotic regime. Such networks enjoy high computational power because their chaotic dynamics give them access to a rich, complicated phase space, which can be exploited during training to perform complex tasks [21, 46]. It is an open question whether the structure of the networks examined here affects their computational performance on tasks that have been previously studied in Gaussian or other random ensembles. One preliminary study has yielded intriguing results [47]: we integrated networks with one of two oscillatory forcing terms $I_{1}(t)$ and $I_{2}(t)$, as described in [23], and compared the performance of these networks on two computational tasks, encoding network-averaged firing rate with a subpopulation and discriminating the two inputs in phase space. As expected, Gaussian networks performed worse than constrained networks in encoding population firing rates (similarly to what was observed in the balanced networks studied by [23]). However, this difference could not be explained solely by the dimensionality of the solution trajectories (as measured by principal component analysis): constrained networks performed better than Gaussian networks, which required an equal number of principal components to explain their solution trajectories. For the second task, we observed that for constrained networks, the trajectories associated with $I_{1}$ and $I_{2}$ appeared to cluster in distinct regions of principal component phase space; this clustering was not observed for Gaussian networks.

Finally, the ideas explored here can be applied to more general network symmetries: for example, a network with several excitatory clusters and global inhibition, or several weakly connected balanced networks [48]. This will both introduce realism and allow the exploration of whether there are some universal features that are implied by the broad features of realistic neural network symmetries such as cortex-like excitatory/inhibitory ratios, spatial range specificity of excitatory vs. inhibitory connections, and so forth. We look forward to reporting on this in future work.

This last direction, in particular, promises to provide further insight into the study of stability and bifurcations in reduced-order models. The work in this paper has highlighted how low-dimensional models of high-dimensional networks can be used to understand the underlying bifurcation structures resulting from network connectivity. Such studies are directly relevant to neuroscience, where input–output functionality of extremely high-dimensional networks have been demonstrated to be encoded dynamically in low-dimensional subspaces [49–55]. We hope that studies such as this can help highlight both methods for characterizing the collective behavior of networked neurons and the limits of traditional mathematical methods in determining stability of such systems. In either case, the results suggest that further study is needed to understand the role of connectivity in driving network-level dynamics.