- Open Access
Stabilization of Memory States by Stochastic Facilitating Synapses
© P. Miller; licensee Springer 2013
- Received: 5 April 2013
- Accepted: 6 September 2013
- Published: 6 December 2013
Bistability within a small neural circuit can arise through an appropriate strength ofexcitatory recurrent feedback. The stability of a state of neural activity, measured bythe mean dwelling time before a noise-induced transition to another state, depends on theneural firing-rate curves, the net strength of excitatory feedback, the statistics ofspike times, and increases exponentially with the number of equivalent neurons in thecircuit. Here, we show that such stability is greatly enhanced by synaptic facilitationand reduced by synaptic depression. We take into account the alteration in times ofsynaptic vesicle release, by calculating distributions of inter-release intervals of asynapse, which differ from the distribution of its incoming interspike intervals when thesynapse is dynamic. In particular, release intervals produced by a Poisson spike trainhave a coefficient of variation greater than one when synapses are probabilistic andfacilitating, whereas the coefficient of variation is less than one when synapses aredepressing. However, in spite of the increased variability in postsynaptic input producedby facilitating synapses, their dominant effect is reduced synaptic efficacy at low inputrates compared to high rates, which increases the curvature of neural input-outputfunctions, leading to wider regions of bistability in parameter space and enhancedlifetimes of memory states. Our results are based on analytic methods with approximateformulae and bolstered by simulations of both Poisson processes and of circuits of noisyspiking model neurons.
- Dynamic synapses
- Stochastic processes
- Poisson process
- Persistent activity
- Short-term memory
- First-passage time
Circuits of reciprocally connected neurons have been long considered as a basis for themaintenance of persistent activity . Such persistent neuronal firing that continues for many seconds after atransient input can represent a short-term memory of prior stimuli . Indeed, Hebb’s famous postulate  that causally correlated firing of connected neurons could lead to astrengthening of the connection, was based on the suggestion that the correlated firingwould be maintained in a recurrently connected cell assembly beyond the time of a transientstimulus . Since then, analytic and computational models have demonstrated the ability ofsuch recurrent networks to produce multiple discrete attractor states , as in Hopfield networks [5, 6], or to be capable of integration over time via a marginally stable network, oftentermed a line attractor [7, 8]. Much of the work on these systems has assumed either static synapses, orconsidered changes in synaptic strength via long-term plasticity occurring on a much slowertimescale than the dynamics of neuronal responses. Here, we add some new results pertainingto the less well-studied effects of short-term plasticity—changes in synaptic strengththat arise on a timescale of seconds, the same timescale as that of persistentactivity—within recurrent discrete attractor networks.
Stochastic synapse model and parameters. (S) for single-synapse model; (M) for memorymodel calculations, where different
(a) Presynaptic components
Determinant of release probability,
, where between spikes: and immediately following each spike:
, whereP(V = 1)=D(t);P(V = 0)=1 − D(t).Between spikes: and immediately following successful release,, while immediately following unsuccessful release,
(b) Postsynaptic components
Determinant of synaptic gating variable,s(t)
Following successful release:
Between successful releases:
1 − exp(−0.25)
1 − exp(−0.25)
1 − exp(−0.25)
When analyzing the stability of discrete states, we focus on the mean value of andfluctuations within the postsynaptic feedback conductance, since that is the variable with aslow enough time constant to maintain persistent activity in standard models ofnetwork-produced memory states [14, 15]. In our formalism, we rely on fluctuations in this NMDA receptor-mediatedfeedback conductance to be on a slower timescale (100 ms) than the membrane timeconstant, which is short (<10 ms), in part because each cell receives a barrage ofbalanced excitatory and inhibitory inputs. When synapses are dynamic, both the meanpostsynaptic conductance and its fluctuations are altered from the case of static synapses.
Here, we show how a presynaptic Poisson spike train, which produces an exponentialdistribution of interspike intervals (ISIs), produces a distribution of inter-releaseintervals (IRIs) that is not exponential if synapses are either facilitating or depressing.We then consider how the nonexponential distribution of IRIs affects both the mean andstandard deviation of the postsynaptic conductance differently from the exponential,Poisson, distribution of IRIs. These results affect the calculation of stability of memorystates, yielding differences in the parameter ranges where bistability exists and producinglarge changes in the spontaneous transition times between states, which limit theirstability.
A two-state memory system is limited by the lifetime of the less stable state . For a given system, one can typically vary any parameter so as to enhance thelifetime of one state while reducing the lifetime of the other state. If we define thesystem’s stability as the lifetime of the less stable state, then the optimalstability of a system arises when the lifetimes of the two states are equal. In this paper,for a given system, defined by the neural firing-rate curve and type of synapse, weparametrically scale the total feedback connection strength to determine the system’soptimal stability. In so doing, we find that optimal stability of bistable neural circuitsis enhanced by synaptic facilitation.
2.1 Distribution of Release Times for a Poisson Spike Train Through StochasticDepressing Synapses
The distribution of release times of a vesicle for depressing synapses with a singlerelease site is simpler to calculate than that for facilitating synapses, because whenconsidering synaptic depression alone, the probability of release from a single sitesimply depends on the time since last release of a vesicle from that site. Therefore, wewill solve for depressing synapses before moving to the case of facilitating synapses,where the probability of release depends on the number of intervening spikes. Thesubsequent result for facilitating synapses will prove to be more biologically relevant,as synapses typically contain multiple releasable vesicles, so it is only in the casewhere the baseline release probability is low—in which case facilitationdominates—that failure of release is common enough to affect the distribution ofrelease times. The case of probabilistic release in depressing synapses with multiplerelease sites is more complex, though the first two moments of the IRI probabilitydistribution have been calculated by others .
2.2 Distribution of Release Times for a Poisson Spike Train Through StochasticFacilitating Synapses
where is the facilitation factor, taking a value between 0 and 1,indicating the fractional increase from the pre-spike release probability toward asaturating release probability of .
To calculate the distribution of interrelease intervals (IRIs) we need to calculate theprobability of release as a function of time, following a prior release. Althoughpresynaptic spikes arrive with constant probability per unit time in a Poisson process,vesicle release occurs more often when the facilitation variable is high. Thus,immediately after release, the likelihood of release is greater than on average, becausethe facilitation variable takes some time (on the order of ) to return to a baseline value. Furthermore, when calculatingthe IRI distribution, we must be aware that , which is the mean value approached by F conditionedon no intervening release event will be lower than the mean value, , since long IRIs are more associated with time windows offewer intervening presynaptic spikes than chance.
The above formula is exact and was matched by simulated data at all values of rsimulated (data not shown).
a value which is always below and in close agreement with simulated data (not shown).
Finally, it should be noted that when synapses are facilitating, consecutive IRIs arecorrelated. For example, when the presynaptic rate is 2 Hz in the simulation used toproduce Figs. 1b1 and 2b1, thecorrelation between one IRI and the subsequent one is 0.028, while with a presynaptic rateof 50 Hz the correlation is 0.015. Such a correlation, which cannot be obtained fromthe IRI distribution alone, further increases any variability in postsynaptic conductance,above and beyond the increase due to the altered shape of the IRI distribution.
In summary, the main difference produced by facilitation from the exponentialdistribution of inter-spike intervals (which is retrieved by setting either or to zero) is an enhancement of probability at low Tand a corresponding reduction at high T. These changes produce a CV of IRIsgreater than 1 ( at 5 Hz and at 50 Hz in the examples shown in Figs. 1b1–1b2) enhancing the noise in any neuralsystem.
2.3 Mean Synaptic Transmission via Dynamic Synapses
We assume that at the time of vesicle release the postsynaptic conductance increases in astep-wise manner, with a fraction, , of previously closed channels becoming opened. This causesthe synaptic gating variable, s, to increase from its prior value, to according to . It then decays between release events with time constant,, according to .
which allows us to calculate the variance in postsynaptic conductance (Fig. 3b).
which is plotted in Fig. 3c (blue curve).
Groups of cells with sufficient recurrent excitatory feedback can become bistable, capableof remaining, in the absence of input, in a quiescent state of low-firing rate, or aftertransient excitation, in a persistent state of high-firing rate. Given the inherentstochastic noise in neural activity—spike trains are irregular, with the CV of ISIsoften exceeding one—the activity states have an inherent average lifetime, whichincreases exponentially with the number of neurons in the cell-group. In this section, weshow analytically that addition of synaptic facilitation to all recurrent synapses canincrease the stability of such discrete memory states by many orders of magnitude. We followthe methods presented in a prior paper for static synapses  and extend them to a circuit with probabilistic facilitating synapses.Calculations of stability are based on the mean of first-passage times between two stablestates . We assume that neurons spike with Poisson statistics, while the variability inthe postsynaptic conductance, which possesses a long time constant (100 ms) typical ofNMDA receptors , determines the instability of states. Since synaptic facilitation ofprobabilistic synapses affects both the mean and variance of the postsynaptic conductance(Figs. 3a–3b), both must becalculated and taken into account when determining the lifetime of memory states. Wedescribe the method briefly below, leaving a reproduction of the full details to thefollowing sections.
Bistability arises when the deterministic dynamics of the network produces multiple fixedpoints—firing rates at which —at least two of which are stable. The deterministic meanfiring rate depends on the total synaptic input to a group of cells. The total synapticinput includes a feedback component via recurrent connections as well as an independentexternal component. At a fixed point, the feedback produced by a given firing rate is suchthat the total synaptic input exactly maintains that given firing rate (intersections inFigs. 4a, 4b). For a network to possessmultiple fixed points, the curve representing synaptic transmission as a function of firingrate and the curve representing firing rate as a function of synaptic input must intersectat multiple points (Figs. 4a, 4b). Betweenany two stable fixed points is an unstable fixed point, where the curves cross back in theopposite direction. The stability of any individual fixed point is strongly dependent on thearea enclosed between the two curves from that fixed point to the unstable fixed point. Thisenclosed area acts as the height of an effective potential (Fig. 4c), which, for a given level of noise in the system determines the mean passagetime from one stable fixed point to the basin of attraction of the other fixed point, i.e.,the mean lifetime of the memory state. Importantly, the lifetime is approximatelyexponentially dependent on the effective barrier height, or the area between the two curves.Thus, changes in the curvature of synaptic feedback as a function of firing rate, which canhave a strong impact on the area between the f-I curve and the feedback curve, can affectstate lifetimes exponentially.
When we analyze the extent of this effect as wrought by synaptic facilitation, we find agreatly enhanced barrier in the effective potential (Fig. 4c),which demonstrates the additional curvature in the neural feedback function outweighs anyincrease in noise in the system (which enters the denominator in the effective potential,Eq. (35). Consequently, the lifetime of both persistent and spontaneous states in a discreteattractor system, can be enhanced by several orders of magnitude when synapses arefacilitating (Fig. 4d). Alternatively, one can obtain the samenecessary stability with far fewer cells, for example, to produce a mean stable lifetime ofover a minute for both the low and high activity states, with all-to-all connections, onlyeight cells are necessary in the example with facilitating synapses, whereas forty arenecessary when synapses are static.
3.1 Analytic Calculation of Mean Transition Time Between Discrete Attractor States
To calculate transition times between discrete attractor states, and hence assess theirstability to noise, we produce an effective potential for the postsynaptic conductance asthe most slowly varying continuous variable of relevance. We use standard methods fortransitions between stable states of Markov processes  but first must calculate the deterministic term, , and diffusive term, , for a group of cells with recurrent feedback. Thecalculations in the case of static synapses were produced and validated elsewhere  but we briefly reiterate them in the following paragraphs. When synapses arefacilitating, the only alterations are the expression for mean synaptic conductance, (Fig. 3a) and its variance, (Fig. 3b), and a newly optimizedstrength of feedback connection to ensure both spontaneous and active states remain asstable as possible.
(by matching the variance of s) where the subscript “1” indicatesthe variance produced by a single presynaptic spike train. For a circuit with Npresynaptic neurons producing feedback current, we scale down individual connectionsstrengths so that the mean feedback current is independent of N, but the noise isreduced as , since s is the fraction of maximal conductance().
with rate multiplier , threshold , and concavity all obtained by fitting to leaky integrate-and-firesimulations . S is a scaled version of s, accounting for the totalfeedback conductance, , where W is the sum of connection strengths of allcells and held fixed when N is varied.
a function which is plotted for both static and facilitating synapses in Fig. 4d.
3.2 Simulation of Mean Transition Time Between Discrete Attractor States
Details of network simulations producing memory activity
(a) Model summary
Single population, E
Leaky integrate-and-fire (LIF) with refractoryperiod
Excitatory AMPA + voltage-dependent NMDA, inhibitory GABAconductances − step increase then exponential decay
Independent fixed-rate Poisson spike trains frompopulations of Input cells
State transitions times via mean population firingrate
N = 8,20,30,40 (static)N = 4,8,12,16 (facilitating)
All-to-all, weight W
(d) Neuron and synapse model
Leaky integrate-and-fire (LIF) with refractory period, andnoisy Poisson exponential conductance input
EE synaptic conductance dynamics
between spikes of cell i at times and
(1) emit spike with time-stamp
Poisson generatorsX = AMPA,GABA
Time for to transition from below 0.05 to above 0.45() and from above 0.45 to below 0.05()
(h) LIF neuron parameters
(i) Synaptic parameters (EE)
1 − exp(−0.25)
1 − exp(−0.25)
3.3 Results for Multiple Circuits
In the example shown, bistability in the control system with static synapses requiredparticular fine-tuning of parameters, so was not very robust. One could wonder that if adifferent system were chosen—in particular a different f-I curve wereused—then the system with static synapses might not be improved by the addition ofsynaptic facilitation. That is, should synaptic facilitation always enhance robustness ofsuch bistable neural circuits? To address this point, we parametrically varied theproperties of the f-I curve (Eq. (34)) and for each set of parameters, we systematically varied the feedback connection strength,W, to test whether the system could be bistable.
Bistability relies upon positive feedback, which can arise from cell-intrinsic currents orfrom network feedback. Synaptic facilitation is a positive feedback mechanism in circuits ofreciprocally connected excitatory cells, since the greater the mean firing rate, the greaterthe effective connection strength, further amplifying the excitatory input beyond thatproduced by the increased spike rate alone. This property of synaptic facilitation enhancesthe stability of memory states and renders them more robust to distractors . Other forms of positive feedback, such as depolarization-induced suppression ofinhibition (DSI), which depends on activity in the postsynaptic cell, can similarly producerobustness in recurrent memory networks .
When the bistability necessary for discrete memory is produced through synaptic feedback ina circuit of neurons, the relative stability to noise fluctuations of each of the two stablefixed points depends exponentially on the area between the mean neural response curve andthe synaptic feedback curve (Figs. 4a–4b). While the synaptic feedback curve is monotonic in firing rate, for staticsynapses it is either linear (in the absence of postsynaptic saturation) or of negativecurvature (decreasing gradient), with the effectiveness of additional spikes decreasing athigh rates when receptors become saturated. However, when the synapse is facilitating, thesynaptic response curve has positive curvature when firing rates are low—the effect ofeach additional spike is greater as firing rate increases. Here, we showed how such aneffect could increase the area between intersections of synaptic feedback and neuralresponse curves, enhancing stability dramatically (Figs. 4–6).
We note that the addition of positive curvature at low rates to the negative curvature athigh, saturating rates in the curve of synaptic transmission as a function of presynapticfiring rates (Fig. 3a) inevitably increases the areas betweenthree points of intersection with any firing rate curve without such an“S”-shape (Figs. 4a–4b).Since the “S”-shape is a hallmark of synaptic facilitation, not present forsynaptic transmission through static synapses, facilitation can always enhance stability ofsuch bistable systems. Less mathematically, a facilitating synapse with the same effectivestrength as a static synapse at intermediate firing rates is stronger at high firing rates,enhancing the stability of a high-activity state (where a drop in synaptic transmission isdetrimental), while at the same time is weaker at low firing rates, enhancing the stabilityof a low-activity state (where a rise of synaptic transmission is detrimental).
It is worth pointing out the converse—that short-term synaptic depression reduces therobustness of such discrete attractors. Indeed, in Fig. 5, weshow that the range of parameters for which a bistable system exists is much narrower whensynapses are depressing (D) versus static (S) or facilitating (F). Since synaptic depressioncontributes a negative curvature to the f-I curve, it tends to reduce the“S-shape” needed for bistability. Or, perhaps more intuitively, high synapticstrength is needed to maintain a high-firing rate state if synapses are depressing, but suchhigh synaptic strength is more likely to render the low-firing rate spontaneous stateunstable.
The changes in the shape of the distribution of inter-release intervals caused by dynamicsynapses alter the fluctuations in post-synaptic conductance. In particular, facilitationenhances the variability and depression reduces the variability arising from a Poisson spiketrain. While the extra variability caused by facilitating synapses tends to destabilize amemory system, this effect was overwhelmed by the increase in stability due to therate-dependent changes in mean synaptic transmission described above. However, the increasein conductance variability, in particular, being on a slower timescale than membranepotential fluctuations, can be a factor in explaining the high CV of neural spiketrains.
Our calculations are based on a simplified formalism, in which the firing-rate curve (f-Icurve) of a neuron is first assumed or fit (Eq. (34), ) under in vivo-like conditions, assuming a given level of noise in the membranepotential. Since the shape of the f-I curve depends on both the mean and variance of theinput current [25, 26], it might appear invalid to discuss changes in the variability of input currentdue to dynamic synapses in the context of a fixed f-I curve. However, the time constants forshort-term synaptic plasticity and the NMDA receptor-mediated currents are more than anorder of magnitude greater than the time constant of the membrane potential under theconditions of strong, fluctuating balanced input that produce the irregularity of spiketrains seen in vivo. Since the neuron’s membrane potential can sample its probabilitydistribution—which determines the likelihood of a spike per unit time—morerapidly than the timescale for changes in that probability distribution, our analyticmethods provide a reasonable description of the circuit’s behavior (Fig. 4d).
In summary, we have demonstrated the ability of short-term synaptic facilitation tostabilize discrete attractor states of neural activity to noise. We have shown this bysimulations and through analytic methods, which include a consideration of how stochasticdynamic synapses mold the distribution of interrelease intervals (IRIs) into a form thatdiffers from the exponential distribution of incoming interspike intervals (ISIs). Thealtered IRI distribution affects both mean synaptic transmission and the variability oftransmission due to a presynaptic Poisson spike train—both of which have a strongimpact on the stability of memory states. The increased variability of synaptic transmissiondue to facilitation is more than countered by the effect of facilitation on mean synaptictransmission, which enhances the robustness of bistability, leading to stable memory stateswith fewer neurons.
- Lorente de Nó R: Vestibulo-ocular reflex arc. Arch Neurol Psych 1933, 30: 245–291. 10.1001/archneurpsyc.1933.02240140009001View ArticleGoogle Scholar
- Funahashi S, Bruce CJ, Goldman-Rakic PS: Neuronal activity related to saccadic eye movements in the monkey’s dorsolateralprefrontal cortex. J Neurophysiol 1991, 65: 1464–1483.Google Scholar
- Hebb DO: Organization of Behavior. Wiley, New York; 1949.Google Scholar
- Brunel N, Nadal JP: Modeling memory: what do we learn from attractor neural networks? C R Acad Sci, Sér 3 Sci Vie 1998, 321: 249–252.View ArticleGoogle Scholar
- Hopfield JJ: Neural networks and physical systems with emergent collective computationalabilities. Proc Natl Acad Sci USA 1982, 79: 2554–2558. 10.1073/pnas.79.8.2554MathSciNetView ArticleGoogle Scholar
- Hopfield JJ: Neurons with graded response have collective computational properties like those oftwo-state neurons. Proc Natl Acad Sci USA 1984, 81: 3088–3092. 10.1073/pnas.81.10.3088View ArticleGoogle Scholar
- Zhang K: Representation of spatial orientation by the intrinsic dynamics of the head-directioncell ensembles: a theory. J Neurosci 1996, 16: 2112–2126.Google Scholar
- Compte A, Brunel N, Goldman-Rakic PS, Wang XJ: Synaptic mechanisms and network dynamics underlying spatial working memory in acortical network model. Cereb Cortex 2000, 10: 910–923. 10.1093/cercor/10.9.910View ArticleGoogle Scholar
- Bourjaily MA, Miller P: Dynamic afferent synapses to decision-making networks improve performance in tasksrequiring stimulus associations and discriminations. J Neurophysiol 2012, 108: 513–527. 10.1152/jn.00806.2011View ArticleGoogle Scholar
- Lindner B, Gangloff D, Longtin A, Lewis JE: Broadband coding with dynamic synapses. J Neurosci 2009, 29: 2076–2088. 10.1523/JNEUROSCI.3702-08.2009View ArticleGoogle Scholar
- Rotman Z, Deng PY, Klyachko VA: Short-term plasticity optimizes synaptic information transmission. J Neurosci 2011, 31: 14800–14809. 10.1523/JNEUROSCI.3231-11.2011View ArticleGoogle Scholar
- Miller P, Wang XJ: Stability of discrete memory states to stochastic fluctuations in neuronal systems. Chaos 2006., 16: Article ID 026110 Article ID 026110Google Scholar
- Koulakov AA: Properties of synaptic transmission and the global stability of delayed activitystates. Network 2001, 12: 47–74.View ArticleMATHGoogle Scholar
- Wang XJ: Synaptic basis of cortical persistent activity: the importance of NMDA receptors toworking memory. J Neurosci 1999, 19: 9587–9603.Google Scholar
- Wang XJ: Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci 2001, 24: 455–463. 10.1016/S0166-2236(00)01868-3View ArticleMATHGoogle Scholar
- Miller P, Zhabotinsky AM, Lisman JE, Wang XJ: The stability of a stochastic CaMKII switch: dependence on the number of enzymemolecules and protein turnover. PLoS Biol 2005., 3: Article ID e107 Article ID e107Google Scholar
- Dayan P, Abbott LF: Theoretical Neuroscience. MIT Press, Cambridge; 2001.MATHGoogle Scholar
- Rosenbaum R, Rubin J, Doiron B: Short term synaptic depression imposes a frequency dependent filter on synapticinformation transfer. PLoS Comput Biol 2012., 8: Article ID e1002557 Article ID e1002557Google Scholar
- Kandaswamy U, Deng PY, Stevens CF, Klyachko VA: The role of presynaptic dynamics in processing of natural spike trains in hippocampalsynapses. J Neurosci 2010, 30: 15904–15914. 10.1523/JNEUROSCI.4050-10.2010View ArticleGoogle Scholar
- Brunel N, Wang XJ: Effects of neuromodulation in a cortical network model of object working memorydominated by recurrent inhibition. J Comput Neurosci 2001, 11: 63–85. 10.1023/A:1011204814320View ArticleGoogle Scholar
- Gillespie DT: Markov Processes. Academic Press, San Diego; 1992.MATHGoogle Scholar
- Abbott LF, Chance FS: Drivers and modulators from push-pull and balanced synaptic input. Prog Brain Res 2005, 149: 147–155.View ArticleGoogle Scholar
- Itskov V, Hansel D, Tsodyks M: Short-term facilitation may stabilize parametric working memory trace. Front Comput Neurosci 2011., 5: Article ID 40 Article ID 40Google Scholar
- Carter E, Wang XJ: Cannabinoid-mediated disinhibition and working memory: dynamical interplay of multiplefeedback mechanisms in a continuous attractor model of prefrontal cortex. Cereb Cortex 2007, 17(Suppl 1):i16-i26. 10.1093/cercor/bhm103View ArticleGoogle Scholar
- Hansel D, van Vreeswijk C: How noise contributes to contrast invariance of orientation tuning in cat visualcortex. J Neurosci 2002, 22: 5118–5128.Google Scholar
- Murphy BK, Miller KD: Multiplicative gain changes are induced by excitation or inhibition alone. J Neurosci 2003, 23: 10040–10051.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://creativecommons.org/licenses/by/2.0), which permitsunrestricted use, distribution, and reproduction in any medium, provided the original workis properly cited.