We start from the stochastic model of Benayoun et al. [12], which we simplify to the most trivial of models. A fully connected network of N neurones is considered with purely excitatory connections (as opposed to the excitatory and inhibitory networks considered in [12]). Within the network, neurones are considered to be either quiescent (Q) or active (A). Taking a small time step dt and allowing the transition probabilities between the two states are then given by:
where is the input to the neurone. Here, f is an activation function, is an optional external input, is the connection strength from neurone i to neurone j, and if neurone j is active at time t and zero otherwise. α is the de-activation rate and, therefore, controls the refractory period of the neurone.
To allow tractability, we further make the following simplifications:
-
1.
We assume that all synaptic weightings are equal ().
-
2.
We assume there is no external input. The driven case will be explored theoretically and empirically in a companion manuscript.
-
3.
We assume the linear identity activation function . Although it is more common to use sigmoid activation functions, we note that the identity function can just be thought of as a suitably scaled tanh function over the desired range.
As the network is fully connected, we can write the following mean field equation for active neurones:
where we have appealed to the fact that the system is closed, and thus . This ODE is analogous to the much studied [37] susceptible → infectious → susceptible (SIS) model used in mathematical epidemiology and we can appeal to some of the known results in studying its behaviour. Primarily, we can use simple stability analysis. The non-zero steady state is given by . Setting , this equilibrium point is stable if . Thus,
Borrowing from epidemiology, we define the threshold . If , we see that and the non-zero steady state is stable. Figure 1 illustrates the differing behaviour of the solution to the above ODE for (sub-critical), (critical), and (super-critical).
2.1 Firing Neurones and Avalanches
Instead of focussing on the average activity level across the network, we will instead look at the size distribution of firing neurones following one firing event. To do this, we begin with a fully quiescent network and initially activate just one neurone. We then record the total number of neurones that fire (the number of quiescent to active transitions) until the network returns to the fully quiescent state. We use this process of sequential activation as our definition of an avalanche and our main interest is the distribution of the avalanche sizes. Unfortunately, the simple ODE approach will not provide us with this distribution. To calculate this distribution, we use the semi-analytic approach described in the following section.
2.2 Tree Approach to the Avalanche Distribution
We begin by defining as the probability the next transition is a recovery (from A to Q) given i active neurones (). The probability the next transition is an activation is then and we have:
In order to calculate the avalanche size distribution, we adopt a recursive approach. We begin by considering the process unfolding in a tree-like manner with 1 initially active neurone. The tree can be divided into levels based on the number of transitions that have occurred and how the process is unfolding. Let level j contain the possible number of active neurones after j transitions. The recursive tree approach relates the probability of transition between levels to the final avalanche size. Figure 2 illustrates the initial levels of this process.
To continue we define as the probability of having i active neurones on level j with and . Assuming initially only one active neurone, we immediately see that , and . To proceed, we will consider the probability of having a particular number of active neurones on an arbitrary level. First, we note the following relation between levels:
We now define:
We can now write where matrix A is given by the following tridiagonal matrix:
with and .
On the j th level of the tree, the probability of only 1 neurone being active is given by . As on level 0, we began with only a single active neurone then for j odd, is always equal to zero. For j even, say , then as we began with only one active neurone on level 0, to have only one active neurone on level j means that k firings must have occurred. We can then calculate the probability of zero active neurones after k firings as ; this is thus the probability, , of having an avalanche of size (or size k if we were not to include the initial active neurone). Setting and noting that we have . To calculate the distribution, we implemented this recursive method of calculation in the MATLAB® environment. Whilst this result is exact, and will be referred to as such henceforth, it can only be calculated numerically via recursion and cannot be given in the form of a closed expression.
2.3 Simulations of Neuronal Avalanches
In order to check the validity of our method, we performed simulations of the firing neurones using the Gillespie algorithm [38]. Due to the network being fully connected the algorithm is relatively straightforward:
-
As earlier, let A be the number of active neurones in the network (Q the number of quiescent). Given that an individual neurone becomes quiescent at rate α then the total rate of (Active → Quiescent) transitions is given by . Similarly, the total rate of (Quiescent → Active) transitions is given by .
-
Let and generate a timestep dt from an exponential distribution of rate r.
-
Generate a random number n between 0 and 1. If an active neurone turns quiescent, otherwise a quiescent neurone is activated (fires). This event is said to occur at time and the network is updated accordingly.
2.4 Exact Solution Compared to Simulation
Values of the threshold, , were chosen less than, equal to and finally above 1. We will refer to these regimes as subcritical, critical, and supercritical, respectively. Figure 3 illustrates the, as expected, good agreement between the simulations and the exact result for the three different regimes of .
2.5 Comparing the Exact Solution to a Closed Form Approximation
In [36], Kessler proposed a closed solution to the analogous susceptible-infected-susceptible (SIS) problem where he was interested in the number of infections (including reinfections) occurring over the course of an epidemic. For small avalanche sizes where the number of infectives is negligible compared to the network size, the transition probabilities can be approximated as
In the critical regime , the problem reduces to calculating the distribution of first passage times of a random walk with equal transition probabilities. Thus, for avalanche sizes in the range, , Kessler [36] gave the following distribution based on Stirling’s approximation:
(1)
We note however that the range over which the distribution can be shown to be a power law is rather limited and for small networks will not hold. Using the theory of random walks and a Fokker–Planck approximation, Kessler also derived the following closed solution to the probability distribution of infections in the critical regime () for larger sizes:
(2)
Figure 4 plots this approximation against our exact solution for a network of size . To more formally assess the convergence of the approximate solution to that of our exact solution, we considered the probabilities of avalanches from size to 20N and measured the difference between the distributions using two different metrics. Letting be the exact probability of an avalanche of size n and be the Kessler approximation to this, we first considered the standard mean square error given by
Secondly, we considered a more stringent measure of the error by looking at the supremum of difference between the same range of avalanches
Figure 5 illustrates the two errors for increasing network size and both show how the proposed closed solution is indeed converging to that of the exact.