Statistics of spike trains in conductance-based neural networks: Rigorous results

We consider a conductance-based neural network inspired by the generalized Integrate and Fire model introduced by Rudolph and Destexhe in 1996. We show the existence and uniqueness of a unique Gibbs distribution characterizing spike train statistics. The corresponding Gibbs potential is explicitly computed. These results hold in the presence of a time-dependent stimulus and apply therefore to non-stationary dynamics.


Introduction
Neural networks have an overwhelming complexity.While an isolated neuron can exhibit a wide variety of responses to stimuli [2], from regular spiking to chaos [3,4], neurons coupled in a network via synapses (electrical or chemical) may show an even wider variety of collective dynamics [5] resulting from the conjunction of nonlinear effects, time propagation delays, synaptic noise, synaptic plasticity, and external stimuli [6].Focusing on the action potentials, this complexity is manifested by drastic changes in the spikes activity, for instance when switching from spontaneous to evoked activity (see for example A. Riehle's team experiments on the monkey motor cortex [7][8][9][10]).However, beyond this complexity may exist some hidden laws ruling an (hypothetical) "neural code" [11].
One way of unraveling these hidden laws is to seek some regularities or reproducibility in the statistics of spikes.While early investigations on spiking activities were focusing on firing rates where neurons are considered as independent sources, researchers concentrated more recently on collective statistical indicators such as pairwise correlations.Thorough experiments in the retina [12,13] as well as in the parietal cat cortex [14], suggested that such correlations are crucial for understanding spiking activity.
Those conclusions where obtained using the maximal entropy principle [15].Assume that the average value of observables quantities (e.g.firing rate or spike correlations) has been measured.Those average values constitute constraints for the statistical model.In the maximal entropy principle, assuming stationarity, one looks for the probability distribution which maximizes the statistical entropy given those constraints.This leads to a (time-translation invariant) Gibbs distribution.In particular, fixing firing rates and the probability of pairwise coincidences of spikes leads to a Gibbs distribution having the same form as the Ising model.This idea has been introduced by Schneidman et al in [12] for the analysis of retina spike trains.They reproduce accurately the probability of spatial spiking pattern.Since then, their approach has known a great success (see e.g.[16][17][18]), although some authors raised solid objections on this model [13,[19][20][21] while several papers have pointed out the importance of temporal patterns of activity at the network level [22][23][24].As a consequence, a few authors [14,25,26] have attempted to define time-dependent models of Gibbs distributions where constraints include time-dependent correlations between pairs, triplets and so on [27].As a matter of fact, the analysis of the data of [12] with such models describes more accurately the statistics of spatio-temporal spike patterns [28].
Taking into account all constraints inherent to experiments it seems extremely difficult to find an optimal model describing spike trains statistics.It is in fact likely that there is not one model, but many, depending on the experiment, the stimulus, the investigated part of the nervous system and so on.
Additionally, the assumptions made in the works quoted above are difficult to control.Especially, the maximal entropy principle assumes a stationary dynamics while many experiments consider a time-dependent stimulus generating a time-dependent response where the stationary approximation may not be valid.At this stage, having an example where one knows the explicit form of the spike trains probability distribution would be helpful to control those assumptions and to define related experiments.This can be done considering neural network models.Although, to be tractable, such models may be quite away from biological plausibility, they can give hints on which statistics can be expected in real neural networks.But, even in the simplest examples, characterizing spike statistics arising from the conjunction of nonlinear effects, time propagation delays, synaptic noise, synaptic plasticity, and external stimuli is far from being trivial on mathematical grounds.
In [29] we have nevertheless proposed an exact and explicit result for the characterization of spike trains statistics in a discrete time version of Leaky Integrate-and-Fire neural network.The results were quite surprising.It has been shown that whatever the parameters value (in particular synaptic weights), spike trains are distributed according to a Gibbs distribution whose potential can be explicitly computed.The first surprise lies in the fact that this potential has infinite range, namely spike statistics has an infinite memory.This is because the membrane potential evolution integrates its past values and the past influence of the network via the leak term.Although Leaky Integrate-and-Fire models have a reset mechanism which erases the memory of the neuron whenever it spikes, it is not possible to upper bound the next time of firing.As a consequence, statistics is non-Markovian (for recent examples of non-Markovian behavior in neural models see also [30]).The infinite range of the potential corresponds, in the maximal entropy principle interpretation, to having infinitely many constraints.
Nevertheless, the leak term influence decays exponentially fast with time (this property guarantees the existence and uniqueness of a Gibbs distribution).As a consequence, one can approximate the exact Gibbs distribution by the invariant probability of a Markov chain, with a memory depth proportional to the log of the (discrete time) leak term.In this way, the truncated potential corresponds to a finite number of constraints in the maximal entropy principle interpretation.However, the second surprise is that this approximated potential is nevertheless far from the Ising model or any of the models discussed above, that appear as quite bad approximations.In particular, there is a need to consider n-uplets of spikes with time delays.This mere fact asks hard problems about evidencing such type of potentials in experiments.
Especially, new type of algorithms for spike trains analysis have to be developed [31].
The model considered in [29] is rather academic: time evolution is discrete, synaptic interactions are instantaneous, dynamics is stationary (the stimulus is time-constant) and, as in a Leaky Integrate-and-Fire model, conductances are constant.It is therefore necessary to investigate whether our conclusions remain for more realistic neural networks models.In the present paper we consider a conductance-based model introduced by Rudolph and Destexhe in [1] called "generalized Integrate and Fire" (gIF) model.This model allows one to consider realistic synaptic responses, and conductances depending on spikes arising in the past of the network, leading to a rather complex dynamics which has been characterized in [32] in the deterministic case (no noise in the dynamics).Moreover, the biological plausibility of this model is well accepted [33,34].
Here we analyse spike statistics in the gIF model with noise and with a time-dependent stimulus.
Moreover, the post-synaptic potential profiles are quite general and summarize all the examples that we know in the literature.Our main result is to prove the existence and uniqueness of a Gibbs measure characterizing spike trains statistics, for all parameters compatible with physical constraints (finite synaptic weights, bounded stimulus, and positive conductances).Here, as in [29], the corresponding Gibbs potential has infinite range corresponding to a non-Markovian dynamics, although Markovian approximations can be proposed in the gIF model too.The Gibbs potential depends on all parameters in the model (especially connectivity and stimulus) and has a form quite more complex than Ising-like models.As a by-product of the proof of our main result, additional interesting notions and results are produced such as continuity with respect to a raster, or exponential decay of memory thanks to the shape of synaptic responses.
The paper is organised as follows.In the section 2 we briefly introduce Integrate-and-Fire models and propose two important extensions of the classical models: the spike has a duration and the membrane potential is reset to a non-constant value.These extensions, which are necessary for the validity of our mathematical results, render nevertheless the model more biologically plausible (see the discussion section).
One of the keys of the present work is to consider spike trains (raster plots) as infinite sequences.Since in gIF models conductances are updated upon the occurrence of spikes, one has to consider two types of variables with distinct type of dynamics.On one hand, the membrane potential, which is the physical variable associated with neurons dynamics evolves continuously.On the other hand, spikes are discrete events.Conductances are updated according to these discrete time events.The formalism introduced in section 2 and 3 allows us to handle properly this mixed dynamics.As a consequence, these sections define gIF model with more mathematical structure than the original paper [1] and mostly contain original results.Moreover, we add to the model several original features such as the consideration of a general form of synaptic profile with exponential decay or the introduction of noise.Section 4 proposes a preliminary analysis of gIF model-dynamics.In sections 5, 6 we provide several useful mathematical propositions as a necessary step toward the analysis of spike statistics, developed in section 7, where we prove the main result of the paper: existence and uniqueness of a Gibbs distribution describing spike statistics.The section 8 and 9 are devoted to a discussion on practical consequences of our results for neuroscience.
2 Integrate and Fire model.
We consider the evolution of a set of N neurons.Here, neurons are considered as "points" instead of spatially extended and structured objects.As a consequence, we define, for each neuron k ∈ {1 . . .N }, a variable V k (t) called the "membrane potential of neuron k at time t" without specification of which part of a real neuron (axon, soma, dendritic spine, ...) it corresponds to.Denote V (t) the vector ( V k (t) ) N k=1 .We focus here on "Integrate-and-Fire models", where dynamics always consists of two regimes.
Fix a real number θ called the "firing threshold of the neuron"1 .Below the threshold, V k < θ, neuron k's dynamics is driven by an equation of the form: where C k is the membrane capacity of neuron k.In its most general form, the neuron k's membrane conductance g k > 0 depends on V k plus additional variables such as the probability of having ionic channels open (see e.g.Hodgkin-Huxley equations [35]) as well as on time t.The explicit form of g k in the present model is developed in section 3.4.The current i k typically depend on time t, and on the past activity of the network.It also contains a stochastic component modelling noise in the system (e.g.synaptic transmission, see section 3.5).

LIF model
A classical example of Integrate-and-Fire model is the Leaky Integrate-and-Fire's (LIF) introduced in [42] where equation (1) reads: where g k is a constant and τ L = C k g k is the characteristic time for membrane potential decay when no current is present ("leak term").

Spikes
The dynamical evolution (1) may eventually lead V k to exceed θ.If, at some time t, V k (t) = θ then neuron k emits a spike or "fires".In our model, like in biophysics, a spike has a finite duration δ > 0; this is a generalisation of the classical formulation of Integrate-and-Fire models where the spike is considered instantaneous.On biophysical grounds δ is of order of a millisecond.Changing the time units we may set δ = 1 without loss of generality.Additionally, neurons have a refractory period τ ref r > 0 where they are not able to emit a new spike although their membrane potential can fluctuate below the threshold (see fig. 1).Hence, spikes emitted by a given neuron are separated by a minimal time scale

Raster plots
In experiments spiking neurons activity is represented by "raster plots", namely a graph with time in abscissa and a neuron labeling in ordinate such that a vertical bar is drawn each "time" a neuron emits a spike.Since spikes have a finite duration δ such a representation limits the time resolution: events with a time scale smaller that δ are not distinguished.As a consequence, if neuron 1 fires at time t 1 and neuron 2 at time t 2 with | t 2 − t 1 | < δ = 1 the two spikes appear to be simultaneous on the raster.Thus, the raster representation introduces a time quantization and has a tendency to enhance synchronization.In gIF models conductances are updated upon the occurrence of spikes (see section 3.2) which are considered as such discrete events.This could correspond to the following "experiment".Assume that we measure the spikes emitted by a set of in vitro neurons, and that we use this information to update the conductances of a model, in order to see how this model "matches" the real neurons (see [58] for a nice investigation in this spirit).Then, we would have to take into account that the information provided by the experimental raster plot is discrete, even if the membrane potential evolves continuously.The consequences of this time-discretisation as well as the limit δ → 0 are developed in the discussion section.
As a consequence, one has to consider two types of variables with distinct type of dynamics.On one hand, the membrane potential, which is the physical variable associated with neuron dynamics evolves with a continuous time.On the other hand, spikes, which are the quantities of interest in the present paper are discrete events.To define properly this mixed dynamics and study its properties we have to model spikes times and raster plots.

Spike times
If, at time t, V k (t) = θ, a spike is registered at the integer time immediately after t, called the spike time.
Choosing integers for the spike time occurrence is a direct consequence of setting δ = 1.Thus, to each neuron k and integer n we associate a "spiking state" defined by: For convenience and in order to simplify the notations in the mathematical developments, we call [ t ] the largest integer which is ≤ t (thus [ −1.2 ] = −2 and [ 1.2 ] = 1).Thus, the integer immediately after t is [ t + 1 ] and we have therefore that ω k ([ t + 1 ]) = 1 whenever V k (t) = θ.Although, characteristic events in a raster plot are spikes (neuron fires) it is useful in subsequent developments to consider also the case when neuron is not firing (ω k (n) = 0).

Reset
In the classical formulation of Integrate-and-Fire models the spike occurs simultaneously with a reset of the membrane potential to some constant value V reset , called the "reset potential".Instantaneous reset is a source of pathologies as discussed in [32,43] and in the discussion section.Here, we consider that reset occurs after the time delay τ sep ≥ 1 including spike duration and refractory period.We set: The reason why the reset time is the integer number [ t + τ sep ] instead of the real t + τ sep is that it eases the notations and proofs.Since the reset value is random (see below and Fig. 1) this assumption has no impact on the dynamics.
Indeed, in our model, the reset value V reset is not a constant.This is a Gaussian random variable with mean zero (we set the rest potential to zero without loss of generality) and variance σ 2 R > 0. In this way we model the spike duration and refractory period, as well as the random oscillations of the membrane potential during the refractory period.As a consequence, the value of V k when the neuron can fire again is not a constant, as it is in classical IF models.A related reference (spiking neurons with partial reset) is [44].The assumption that σ 2 R > 0 is necessary for our mathematical developments (see the bounds (37)).We assume σ 2 R to be small to avoid trivial and unrealistic situations where V reset ≥ θ with a large probability leading the neuron to fire all the time.Note however that this is not a required assumption to establish our mathematical results.We also assume that, in successive resets, the random variables V reset are independent.

The shape of membrane potential during the spike
On biophysical grounds the time course of the membrane potential during the spike includes a depolarisation and re-polarisation phase due to the nonlinear effects of gated ionic channels on the conductance.This leads to introduce, in modelling, additional variables such as activation/inactivation probabilities as in the Hodgkin-Huxley model [35] or adaptation current as e.g. in FitzHugh-Nagumo model [36,37,45,46] (see the discussion section for extensions of our results to those models).Here, since we are considering only one variable for the neuron state, the membrane potential, we need to define the spike profile, i.e. the course of V k (t) during the time interval (t, [ t + τ sep ]).It turns out that the precise shape of this profile plays no role in the developments proposed here, where we concentrate on spike statistics.Indeed, a spike is registered whenever V k (t) = θ and this does not depend on the spike shape.
What we need is therefore to define the membrane potential evolution before the spike, given by (1), and after the spike, given by (4) (see Figure 1).00 00 00 00 00 11 11 11 11 11

Mathematical representation of raster plots
The "spiking pattern" of the neural network at integer time n is the vector ω To each raster ω ∈ X and each neuron index2 j = 1 . . .N we associate an ordered (generically infinite) list of "spike times" t (r) (integer numbers) such that t (r) j (ω) is the r-th time of firing of neuron j in the raster ω.In other words, we have ω j We introduce here two specific rasters which are of use in the paper.We note Ω 0 the raster such that ω k (n) = 0, ∀k = 1 . . .N, ∀n ∈ (no neuron ever fires) and Ω 1 the raster ω k (n) = 1, ∀i = 1 . . .N, ∀n ∈ (each neuron is firing at every integer time).
Finally, we use the following notation borrowed from [47].We note, for n ∈ , m ≥ 0, and r integer: For simplicity, we consider that τ ref , the refractory period, is smaller than 1 so that a neuron can fire two consecutive time steps (i.e. one can have ω k (n) = 1 and ω k (n + 1) = 1).This constraint is discussed in section 9.2.

Representation of time dependent functions
Throughout the paper we use the following convention.For a real function of t and ω, we write f (t, ω) for −∞ ) to simplify notations.This notation takes into account the duality between variables such as membrane potential evolving with respect to a continuous time and raster plots labeled with discrete time.
Thus, the function f (t, ω) is a function of the continuous variable t and of the spike block ω −∞ , where by definition [ t ] ≤ t, namely f (t, ω) depends on the spike sequences occurring before t.This constraint is imposed by causality.

Last reset time
We define τ k (t, ω) as the last time before t where neuron k's membrane potential has been reset, in the raster ω.This is −∞ if the membrane potential has never been reset.As a consequence of our choice (4) for the reset time τ k (t, ω) is an integer number fixed by t and the raster before t .The membrane potential value of neuron k at time t is controlled by the reset value V reset at time τ k (t, ω) and by the further sub-threshold evolution (1) from time τ k (t, ω) to time t.
3 Generalized Integrate-and-Fire models In this paper, we concentrate on an extension of (2), called "generalized Integrate-and-Fire" (gIF), introduced in [1], closer to biology [33,34], since it considers more seriously neurons interactions via synaptic responses.

Synaptic conductances
Depending on the neuro-transmitter3 they use for synaptic transmission neurons can be excitatory (population E) or inhibitory (population I).This is modeled by introducing reversal potentials E + for excitatory (typically E + ≃ 0mV for AMPA and NMDA) and E − for inhibitory (E − ≃ −70mV for GABA A and E − ≃ −95mV for GABA B).We focus here on one population of excitatory and one population of inhibitory neurons although extensions to several populations may be considered as well.Also, each neuron is submitted to a current I k (t).We assume that this current has some stochastic component that mimics synaptic noise (section 3.5).
The variation of the membrane potential of neuron k at time t reads: where g L,k is a leak conductance, E L < 0 is the leak reversal potential (about −65 mV), g k (t) the conductance of the excitatory population and g (I) k (t) the conductance of inhibitory population.They are given by: where g kj is the conductance of the synaptic contact j → k.
We may rewrite equation ( 6) in the form (1) setting

Conductance update upon a spike occurrence.
The conductances g kj (t) in ( 7) depend on time t but also on pre-synaptic spikes occurring before t.This is a general statement which is modeled in gIF models as follows.Upon arrival of a spike in the pre-synaptic neuron j at time t (r) j (ω) the membrane conductance of the post-synaptic neuron k is modified as: In this equation, the quantity G kj ≥ 0 characterizes the maximal amplitude of the conductance during a post-synaptic potential.We use the convention that G kj = 0 if and only if there is no synapse between j and k.This allows us to encode the graph structure of the neural network in the matrix G with entries G kj .Note that the G kj 's can evolve in time due to synaptic plasticity mechanisms (see section 9.4).
The function α kj (called "alpha" profile [48]) mimics the time course of the synaptic conductance upon the occurrence of the spike.Classical examples are: (exponential profile) or: with H the Heaviside function (that mimics causality) and τ kj is the characteristic decay times of the synaptic response.Since t is a time, the division by τ kj ensures that α kj ( t ) is a dimensionless quantity: this eases the legibility of the subsequent equations on physical grounds (dimensionality of physical quantities).
Contrarily to (9) the synaptic profile (10), with α kj ( 0 ) = 0 while α kj ( t ) is maximal for t = τ kj allows one to smoothly delay the spike action on the post-synaptic neuron.More general forms of synaptic responses could be considered as well 4 .

Mathematical constraints on the synaptic responses
In all the paper we assume that the α kj 's are positive and bounded.Moreover, we assume that: 4 For example, the α profile may obey a Green equation of type [49]: where k = 1, a kj = 1, corresponds to (9), and so on.
for some integer d.So that α kj ( t ) decays exponentially fast as t → +∞, with a characteristic time τ kj , the decay time of the evoked post-synaptic potential.This constraint matches all synaptic response kernels that we know (where typically d = 0, 1) [48,49].
This has the following consequence.For all t, M < t integer, r integer, we have, setting where { t } is the fractional part: where P d () is a polynomial of degree d.
We introduce the following (Hardy) notation: if a function f (t) is bounded from above, as t → +∞, by a function g(t) we write: f (t) g(t).Using this notation we have therefore: as M → −∞.
Additionally, the constraint (11) implies that there is some α + < +∞ such that, for all t, for all k, j: Indeed, for n ≥ 0 integer, call Due to (11) this series converges (e.g. from Cauchy criterion).We set: On physical grounds it implies that the conductance g k remains bounded, even if each pre-synaptic neuron is firing all the time (see eq. ( 29) below).

Synaptic summation
Assume that eq. ( 8) remains valid for an arbitrary number of pre-synaptic spikes emitted by neuron j within a finite time interval [s, t] (i.e.neglecting nonlinear effects such as the fact that there is a finite amount of neurotransmitter leading to saturation effects).Then, one obtains the following equation for the conductance g kj at time t, upon the arrival of spikes at times t (r) j (ω) in the time interval [s, t]: The conductance at time s, g kj (s), depends on the neuron j's activity preceding s.This term is therefore unknown unless one knows exactly the past evolution before s.One way to circumvent this problem is to taking s arbitrary far in the past, i.e. taking s → −∞ in order to remove the dependence on initial conditions.This corresponds to the following situation.When one observes a real neural network the time where the observation starts, say t = 0, is usually not the time when the system has begun to exist, s in our notations.Taking s arbitrary far in the past corresponds to assuming that the system has evolved long enough so that it has reached sort of a "permanent regime", not necessarily stationary, when the observation starts.On phenomenological grounds it is enough to take −s larger than all characteristic relaxation times in the system (e.g.leak rate and synaptic decay rate).Here, for mathematical purposes it is easier to take the limit s → −∞.
Since g kj (t) depends on the raster plot up to time t, via the spiking times t (r) j (ω) this limit makes only sense when taking it "conditionally" to a prescribed raster plot ω.In other words, one can know the value of the conductances g kj at time t only if the past spike times of the network are known.We write g kj (t, ω) from now on to make this dependence explicit.

Noise
We allow, in the definition of the current I k (t) in eq. ( 6) the possibility of having a stochastic term corresponding to noise so that: where i (ext) k (t) is a deterministic external current and ξ k (t) a noise term whose amplitude is controlled by σ B > 0. The model affords an extension where σ B depends on k but this extension is straightforward and we do not develop it here.The noise term can be interpreted as the random variation in the ionic flux of charges crossing the membrane per unit time at the post synaptic button, upon opening of ionic channels due to the binding of neurotransmitter.
We assume that ξ k (t) is a white noise, ξ k (t) = dB k dt where dB k (t) is a Wiener process, so that

Differential equation for the Integrate regime of gIF
Summarizing, we write eq. ( 6) in the form: where: This is the more general conductance form considered in this paper.
Moreover, : where W kj is the synaptic weight: These equations hold when the membrane potential is below the threshold (Integrate regime).
Therefore, gIF models constitute rather complex dynamical systems: the vector field (r.h.s) of the differential equation ( 16) depends on an auxiliary "variable" which is the past spike sequence ω −∞ and to define properly the evolution of V k from time t to later times one needs to know the spikes arising before t.
This is precisely what makes gIF models more interesting than LIF.The definition of conductances introduces long term memory effects.
IF models implement a reset mechanism on the membrane potential: If neuron k has been reset between s and t, say at time τ , then V k (t) depends only on V k (τ ) and not on previous values, as in (4).But, in gIF model, contrarily to LIF, there is also a dependence in the past via the conductance and this dependence is not erased by the reset.That's why we have to consider a system with infinite memory.

The parameters space
The stochastic dynamical system (16) depends on a huge set of parameters: the membrane capacities .Although some parameters can be fixed from biology, such as C k , the reversal potentials, τ kj , ... some others such as the G kj 's must be allowed to vary freely in order to leave open the possibility of modelling very different neural networks structures.
In this paper we are not interested in describing properties arising for specific values of those parameters, but instead in generic properties that hold on sets of parameters.More specifically, we denote the list of all , . . .by the symbol γ.This is a vector in IR K where K is the total number of parameters.In this paper, we assume that γ belongs to a bounded subset H ⊂ IR K .
Basically, we want to avoid situations where some parameters become infinite, which would be unphysical.
So the limits of H are the limits imposed by biophysics.Additionally, we assume that σ R > 0 and σ B > 0.
Together with physical constraints such as "conductances are positive", these are the only assumption made in parameters.All mathematical results stated in the paper old for any γ ∈ H.

gIF model-dynamics for a fixed raster
We assume that the raster ω is fixed, namely the spike history is given.Then, it is possible to integrate the equation ( 16) (Integrate regime) and to obtain explicitly the value of the membrane potential of a neuron at time t, given the membrane potential value at time s.Additionally, the reset condition (4) has the consequence of removing the dependence of neuron k on the past anterior to τ k (t, ω).

Integrate regime
For We have: and: Fix two times s < t and assume that for neuron k, V k (u) < θ, s ≤ u ≤ t so that the membrane potential V k obeys (16).Then: We have then, integrating the previous equation with respect to t 1 between s and t, and setting t 2 = t: This equation gives the variation of membrane potential during a period of rest (no spike) of the neuron.
Note however that this neuron can still receive spikes from the other neurons via the update of conductances (made explicit in the previous equation by the dependence in the raster plot ω).
The term Γ k (s, t, ω) given by ( 19) is an effective leak between s, t.In the Leaky Integrate and Fire model it would have been equal to e has the dimension of a voltage.It corresponds to the integration of the total current between s and t weighted by the effective leak term Γ k (t 1 , t, ω).It decomposes as where, is the synaptic contribution.Moreover, where we set: the characteristic leak time of neuron k.We have included the leak reversal potential term in this "external" term for convenience.Therefore, even if there is no external current this term is nevertheless non zero.
The sum of the synaptic and external terms gives the deterministic contribution in the membrane potential.We note: Finally, is a noise term.This is a Gaussian process with mean 0 and variance: The square root of this quantity has the dimension of a voltage.
As a final result, for a fixed ω, the variation of membrane potential during a period of rest (no spike) of neuron k between s and t reads (sub-threshold oscillations):

Reset
In eq. ( 4), as in all IF models that we know, the reset of the membrane potential has the effect of removing the dependence of V k on its past since V k ([ t + τ sep ]) is replaced by V reset .Hence, reset removes the dependence in the initial condition V k (s) in (24) provided that neuron k fires between s and t in the raster ω.As a consequence, eq. ( 24) holds, from the "last reset time" introduced in section 2.10 up to time t.
Then, eq. ( 24) reads where: is a Gaussian process with mean zero and variance: Although the reset condition may look as a simplification it is in fact a source of complications on mathematical grounds.As discussed in section 9.3 several proofs are easier if we do not reset memory and instead take the limit s → −∞ in (24).However, since IF models are widespread in the neuroscience literature we have preferred to give the more general proofs and then discuss their adaptation to simpler cases.

Useful bounds
We now prove several bounds used throughout the paper.

Bounds on the conductance
From ( 13), and since α kj ( t ) ≥ 0: Therefore, so that the conductance is uniformly bounded in t and ω.The minimal conductance is attained when no neuron fires ever so that Ω 0 is the "lowest conductance state".On the opposite the maximal conductance is reached when all neurons fire all the time so that Ω 1 is the "highest conductance state".To simplify . This is the minimal relaxation time scale for neuron k while τ L,k = C k g L,k is the maximal relaxation time.
so that: Thus, V In this case, so that: Consequently: which provides uniform bounds in s, t, ω for the deterministic part of the membrane potential.

Bounds on the noise variance
Let us now consider the stochastic part V the left hand side is an increasing function of u = t − τ k (t, ω) ≥ 0 so that the minimum, R is reached for u = 0 while the maximum is reached for u = +∞ and is . The same argument holds mutatis mutandis for the right hand side.We set: so that: 5. 4 The limit τ k (t, ω) → −∞ For fixed s and t there are infinitely many rasters such that τ k (t, ω) < s (we remind that rasters are infinite sequences).One may argue that taking the difference t − s sufficiently large the probability of such sequences should vanish.It is indeed possible to show (section 8.1) that this probability vanishes exponentially fast with t − s, meaning unfortunately that it is positive whatever t − s.So we have to consider cases where τ k (t, ω) can go arbitrary far in past (this is also a key toward an extension of the present analysis to more general conductance-based models as discussed in section 9.3).Therefore, we have to check that the quantities introduced in the previous sections are well defined as τ k (t, ω) → −∞.
Fix s real.For all ω such that τ k (t, ω) ≤ s -this condition ensuring that k does not fire between s and twe have, from ( 28), (31), 0 exists as well.The same holds for the external term Finally, since Γ k (τ k (t, ω), t, ω) → 0 as τ k (t, ω) → −∞ the noise term (26) becomes in the limit: which is a Gaussian process with mean 0 and a variance σB )dt 1 which obeys the bounds (37).
6 Continuity with respect to a raster.

Definition
Due to the particular structure of gIF models we have seen that the membrane potential at time t is both a function of t and of the full sequence of past spikes ω [ t ] −∞ .One expects however the dependence with respect to the past spikes to decay as those spikes are more distant in the past.This issue is related to a notion of continuity with respect to a raster that we now characterize.

Definition 1 Let m be a positive integer.The m-variation of a function
where the definition of = is given in eq. ( 5).Hence, this notion characterizes the maximal variation of f (t, .) on the set of spikes identical from time [ t ] − m to time [ t ] (cylinder set).It implements the fact that one may truncate the spike history to time [ t ] − m and make an error which is at most var m [f (t, .)].
An additional information is provided by the convergence rate to 0 with m.The faster this convergence the smaller the error made when replacing an infinite raster by a spike block on a finite time horizon.

Continuity of conductances
Proposition 4 The conductance g k ( t, ω ) is continuous in ω, for all t, for all k = 1 . . .N .
since the set of firing times t − m ≤ t (r) (ω ′ ) < t are identical by hypothesis.
Let us show the continuity of V (syn) k (τ k (t, .),t, .).We have, from (20), The following inequality is used at several places in the paper.For a t 1 -integrable function f (t 1 , t, ω), we have: Here it gives, for t 1 ≤ t: For the first term, we have, Let us now consider the second term.If this term vanishes.Therefore, the supremum in the definition of We may assume, without loss of generality, that Then, from (32), So, we have, for the variation of V (syn) k (τ k (t, .),t, .), using (21): so that finally, with and var m V (τ k (t, ω), t, ω) with respect to ω.We have: where, in the last inequality, we have used that the supremum in the variation is attained for where, (τ k (t, .),t, .) is continuous.
As a conclusion, V (τ k (t, .),t, .) is continuous as the sum of two continuous functions.

Continuity of the variance of V
(noise) k (τ k (t, .),t, .) Using the same type of arguments one can also prove that Proposition 6 The variance σ k (τ k (t, ω), t, ω) is continuous in ω, for all t, for all k = 1 . . .N .
Proof We have, from ( 27) For the first term we have that the sup in var m Γ 2 k (τ k (t, .),t, .) is attained for τ k (t, ω), τ k (t, ω ′ ) < t − m and: For the second term we have: so that finally: with and continuity follows.

Remark
Note that the variation of all quantities considered here is exponentially decaying with a time constant given by max(τ kj , τ L,k ).This is physically satisfactory: the loss of memory in the system is controlled by the leak time and the decay of the post-synaptic potential.
7 Statistics of raster plots.
7.1 Conditional probability distribution of V k (t).
Recall that P is the joint distribution of the noise and E [ ] the expectation under P .Under P the membrane potential V is a stochastic process whose evolution, below the threshold, is given eq.( 24), (25) and above by (4).It follows from the previous analysis that: is Gaussian with mean: and covariance: where σ 2 k (τ k (t, ω), t, ω) is given by (27).Moreover, the V k (t)'s, k = 1 . . .N , are conditionally independent.
Proof Essentially, the proof is a direct consequence of eq. ( 24), (25) and the Gaussian nature of the noise V (noise) k (τ k (t, ω), t, ω).The conditional independence results from the fact that: 7.2 The transition probability.
We now compute the probability of a spiking pattern at time n = [ t ], ω(n), given the past sequence ω n−1 −∞ .
Proposition 8 The probability of ω(n) conditionally to ω n−1 −∞ is given by: with where and Proof We have, using the conditional independence of the V k (n)'s: Since the V k (n − 1)'s are conditionally Gaussian, with mean we directly obtain (47), (48).Note that since σ k (τ k (n − 1, ω), n − 1, ω) is bounded from below by a positive quantity (see (37)) the ratio 48) is defined for all ω ∈ X.

Chains with complete connections
The transition probabilities (47) define a stochastic process on the set of raster plots where the underlying membrane potential dynamics is summarized in the terms While the integral defining these terms extends from τ k (n − 1, ω) to n − 1 where τ k (n − 1, ω) can go arbitrary far in the past, the integrand involves the conductance g k (n − 1, ω) which summarizes an history dating back to s = −∞.As a consequence, the probability transitions generate a stochastic process with unbounded memory, thus non Markovian.One may argue that this property is a result of our procedure of taking the initial condition in a infinite past s → −∞, to remove the unresolved dependency on g k (s) (section 3.4).So the alternative is either to keep s finite in order to have a Markovian Recall that: From (35), (37) we have: Since π, given by ( 50), is monotonously decreasing, we have: Finally, which proves (51).This also proves the non-nullness of the system of transition probabilities.
The proof, which is rather long, is given in the appendix.
better the right statistics leads to an exponential increase in the number of monomials which becomes rapidly intractable.Finally, the Lagrange multipliers λ l are rather difficult to interpret.
On the opposite, the analytic form (55) depends only on a finite numbers of parameters (γ) constraining the neural network dynamics, which have a straightforward interpretations being physical quantities.This shows that, at least in gIF model, the linear Gibbs potential (58) obtained from the maximal entropy principle is not really appropriate, even for empirical/numerical purposes, and that a form (55) where the infinite memory ω −1 −∞ is replaced by ω −1 −D could be more efficient although nonlinear.To finish this section let us discuss the link with Ising model in light of the present work.Ising model corresponds to a memory-less case, hence to D = 0. Since the causal structure of the Gibbs potential forbids monomials of the form ω k1 (0) . . .ω kr (0), the D = 0 expansion of the gIF-Gibbs potential corresponds to a Bernoulli distribution where neurons are independent φ (0) ( 0, ω ) = In the expansion (57) this corresponds to collecting all monomials corresponding to b k (n) = 1 in a unique monomial.In this way, the binned potential contains indeed an Ising term . . .that mixes all spike events occurring within the time interval w.These events appear simultaneous because of binning, leading to the Ising pairwise term b k1 (0)b k2 (0) while events occurring on smaller time scales are scrambled by this procedure.
The binning effect on Gibbs potential requires however a more detailed description.This will be discussed elsewhere.9 Discussion.
To conclude this paper we would like to discuss several consequences and possible extensions of this work.

The spike time discretisation
In gIF model membrane potential evolves continuously while conductance are updated with spike occurrence considered as discrete events.Here we discuss this time discretisation.Actually, there are two distinct questions.9.1.1The limit of time-bin tending to 0 This limit would correspond to a case where spike is instantaneous and modeled by a Dirac distribution.
As discussed in [32] this limit raises serious difficulties.To summarize, in real neurons firing occurs within a finite time δ corresponding to the time of raise and fall for the membrane potential.This involves physico-chemical processes which cannot be instantaneous.The time curse of the membrane potential during the spike is described by differential equations, like Hodgkin-Huxley's [35].Although, the time scale dt appearing in the differential equations has the mathematical meaning of being arbitrary small, on biophysical grounds this time scale cannot be arbitrary small, otherwise the Hodgkin-Huxley equations loose their meaning.Indeed, they correspond to an average over microscopic phenomena such as ionic channels dynamics.In particular, their time scale must be sufficiently large to ensure that the description of ionic channels dynamics (opening and closing) in terms of probabilities is valid so dt must be larger than the characteristic time of opening-closing of ionic channels τ P .Additionally, Hodgkin-Huxley's equations uses a Markovian approach (master equation) for the dynamics of h, m, n gates.This requires that the characteristic time dt is quite a bit larger than the characteristic time of decay for the time correlations between gates activity τ C .Summarizing, we must have 0 < τ C , τ P < dt < δ.Thus, on biophysical grounds δ cannot be arbitrary small.
In our case, the δ → 0 limit is armless however, provided we keep a non zero refractory period, ensuring that only finitely many spikes occur in a finite time interval.Taking the limit δ → 0 without considering a refractory period raises mathematical problems.One can in principle have uncountably many spikes in a finite time interval leading to the divergence of physical quantities like energy.Also, one can generate nice causal paradoxes [43].Take a loop with two neurons one excitatory and one inhibitory and assume instantaneous propagation (the α profile is then represented by a Dirac distribution).Then, depending on the synaptic weights value one can have a situation where neuron 1 fires instantaneously, and make instantaneously 2 firing which prevents instantaneously 1 from firing and so on.So taking the limit δ → 0 as well as τ ref r → 0 induces pathologies not inherent to our approach but to IF models.

Synchronisation for distinct neurons.
There is a more subtle issue pointed out in [57].We do not only discretize time for each neuron' spikes, we align the spikes emitted by distinct neurons on a discrete time grid, as an experimental raster does.As shown in [32] this induces, in gIF models with a purely deterministic dynamics (no noise and reset to a constant value), an artificial synchronisation.As a consequence the deterministic dynamics of gIF models does during the spike.Although, it could be possible to propose an ad hoc form for the spike, it would certainly be more interesting to extend the results here to models where neurons activity depends on additional variables such as adaptation currents, as in the FitzHugh-Nagumo model [36,37,45,46], or activation-inactivation variables as in the Hodgkin-Huxley model [35].
The present formalism affords an extension toward such models, where the neuron fires whenever its membrane potential belongs to a region of the phase space, which can be delimited by membrane potentials plus additional variables such as adaptation currents or activation-inactivation variables, and where the spike is controlled by the global dynamics of all these variables.But, while here the firing of a neuron is described by the crossing of a fixed threshold, in the FitzHugh-Nagumo model it is given by the crossing of a separatrix in the plane (voltage-adaptation current), and by a more complex "frontier" in the Hodgkin-Huxley model [3,38].One difficulty is to precisely define this region.To our knowledge there is no clear agreement for the Hodgkin-Huxley model (some authors [3] even suggest that the "spike region" could have a fractal frontier).The extension toward FitzHugh-Nagumo seems more manageable.
Finally, the most important difficulty toward extending this paper results to more realistic neural networks is the definition of the synaptic spike response.In IF models the spike is thought as a punctual "event" (typically, an "instantaneous" pulse) while the synaptic response is described by a convolution kernel (the α-profile).This leads one to consider a somewhat artificial mixed dynamics where membrane potential evolves continuously while spike are discrete events.In more realistic models, one would have to consider kinetic equations for neurotransmitter release, receptor binding and opening of post-synaptic ionic channels [6,48].Additionally, the consideration of these mechanics deserves a spatially extended modelling of the neuron, with time delays.In this case, all variables evolve continuously and the statistics of spike trains would be characterized by the statistics of return times in the "spike region".This statistics is induced by some probability measure in the phase space; a natural candidate would be the Sinai-Ruelle-Bowen measure [60][61][62], for stationary dynamics, or the time-dependent SRB measure for non-stationary cases, as defined e.g. in [63].These measures are Gibbs measures as well [64].Here, the main mathematical property ensuring existence and uniqueness of such a measure would be uniform hyperbolicity.To our knowledge conditions ensuring such a property in networks has not been established yet neither for Hodgkin-Huxley's nor for FitzHugh-Nagumo's models.

Synaptic plasticity
As the results established in this paper hold for any synaptic weight value in H, they hold as well for networks underlying synaptic plasticity mechanisms.The effects of a joint evolution of spikes dynamics, depending on synaptic weights distributions, and synaptic weights evolution depending on spike dynamics has been studied in [65].In particular it has been shown that mechanisms such as Spike-Time Dependent Plasticity are related to a variational principle for a quantity, the topological pressure, derived for the thermodynamic formalism of Gibbs distributions.In the paper [65] the fact that spike trains statistics were given by a Gibbs distribution was a working assumption.Therefore, the present work establish a firm ground for [65].= ω ′ implies ω k (n) = ω ′ k (n) so that: We have where: (see eq. ( 41),( 44)) and: , (see eq. ( 42),( 45)).

Figure 1 :
Figure 1: Time course of the membrane potential in our model.The blue dashed curve illustrates the shape of a real spike, but what we model is the red curve.

1 m ω n n1 = ω n m the concatenation of the blocks ω n1− 1 m
the ordered sequence of spiking patterns between m and n.Such sequences are called spike blocks.Additionally we note ω n1−and ω n n1 .Call A = { 0, 1 } N the set of spiking patterns (alphabet).An element of X def = A , i.e. a bi-infinite ordered sequence ω = {ω(n)} +∞ n=−∞ of spiking patterns, is called a "raster plot".It tells us which neurons are firing at each time n ∈ .In experiments raster plots are obviously finite sequences of spiking pattern but the extension to , especially the possibility of considering an arbitrary distant past (negative times) is a key of the present work.In particular, the notation ω n −∞ refers to spikes occurring from −∞ to n.
the threshold θ, the reversal potentials E L , E + , E − , the leak conductance g L ; the maximal synaptic conductances G kj , k, j = 1 . . .N which define the neural network topology; the characteristic times τ kj , k, j = 1 . . .N of synaptic responses decay; the noise amplitude σ B and, additionally, the parameters defining the external current i (ext) k (t, ω), t, ω).It has zero mean and its variance(27) obeys the bounds: (syn) k (τ k (t, .),t, .)converges to 0 exponentially fast as m → +∞.Now, let us show the continuity of V (ext) k

N
k=1 λ k ω k (0).The Ising model is therefore irrelevant to approximate the exact potential of gIF model, if one wants to reproduce spike statistics at the minimal discretisation time scale δ without considering memory effects.However, in real data analysis people are usually binning data, with a time windows of width w ∼ 10 − 20 ms.Binning consists of recoding the raster plot with spikes amalgamation.The binned raster b consists of "spikes" b k (n) ∈ { 0, 1 } where b k (n) = 1 if neuron k fired at least once in the time window [nw, (n + 1)w[.

From ( 35 σ 2 k 1 σ − k 2 −FiguresFigure 1 -
FiguresFigure 1 -Time course of the membrane potential in our model.The blue dashed curve illustrates the shape of a real spike, but what we model is the red curve.
Proof In the proof, we shall establish precise upper bounds for the variation of V m [g k ( t, .)] 2 8.1 The probability that neuron k does not fire in the time interval [s, t].