Cross-Correlations and Joint Gaussianity in Multivariate Level Crossing Models
© E. Di Bernardino et al.; licensee Springer 2014
Received: 21 March 2013
Accepted: 16 December 2013
Published: 17 April 2014
A variety of phenomena in physical and biological sciences can be mathematically understood by considering the statistical properties of level crossings of random Gaussian processes. Notably, a growing number of these phenomena demand a consideration of correlated level crossings emerging from multiple correlated processes. While many theoretical results have been obtained in the last decades for individual Gaussian level-crossing processes, few results are available for multivariate, jointly correlated threshold crossings. Here, we address bivariate upward crossing processes and derive the corresponding bivariate Central Limit Theorem as well as provide closed-form expressions for their joint level-crossing correlations.
Various phenomena in the biological or physical sciences are amenable to the description by level crossings of random Gaussian processes [1, 2]. Examples of these phenomena are spike coordination of neurons in the brain , insurance risk assessment  and stress levels generated by ocean waves . Therefore a number of mathematical studies in recent decades have focused on the statistical properties of level crossings arising from stationary Gaussian processes . However, largely this literature addresses the properties of one level-crossing process and rarely deals with the coordinated level crossings of multivariate Gaussian processes. A prominent application where correlated level crossings are of particular importance is neuroscience. Recent work has shown that the spikes of a cortical neuron can be approximated by a Gaussian level-crossing process [3, 6]. The assumption of Gaussianity is prompted by the experimental observation that cortical neurons are on average connected to ∼10000 neurons and therefore receive a barrage of inputs that together lead to a near-Gaussian fluctuation at the cell body of any given cortical neuron . The spikes of two neurons are then modeled as upward level-crossing times of two cross-correlated fluctuating Gaussian potentials.
In this article we aim to address two features of level crossings of multiple correlated Gaussian processes. First, we want to clarify whether level-crossing counts derived from multiple correlated processes are jointly Gaussian. Second, we want to understand how many more coincident level crossings in a given time instance are expected if the underlying Gaussian random processes are correlated. Let us provide an intuitive reason for these questions. Starting with the first question, we recognize that if level-crossing counts of two neurons were jointly Gaussian, then a simple measure of dependence is the covariance or the Pearson correlation coefficient. Measuring a vanishing correlation coefficient or vanishing covariance between two neuronal spike counts would in this case imply true statistical independence, because only in the case of multivariate Gaussian distribution is it permissible to conclude independence from vanishing count correlations. This implication is not permissible if the marginal distributions are not Gaussian or are Gaussian but the joint distribution is not a multivariate Gaussian distribution. While marginal Gaussianity has been shown for level-crossing counts in  for large bin sizes, joint Gaussianity is still an open question. It might seem natural to imply joint Gaussianity from marginal Gaussianity for multivariate level-crossing processes, however, numerous counter examples exist to prove this intuition wrong, see Sect. 5 in . Here, we use a modified Breuer–Major Theorem to prove joint Gaussianity and show that any linear combination of level-crossing counts of the two processes is also Gaussian.
The second question we address in this article deals with the conditional probabilities of two level-crossing processes. We are interested in how the level crossings of one Gaussian process can be used to predict the level-crossing probability of the partner process in a specific time interval relative to the observed level crossing in one process. In neuroscience, coordinated neuronal firing drives changes in synaptic connectivity and calculating the spike count dependencies across neurons is therefore a topic of current research efforts (e.g. Chap. 8 in ). The available mathematical results for conditional upward crossings in Gaussian processes currently comprise mostly variance and moments for one level-crossing process (see Chaps. 3–5 in ) as well as the low and high correlation limit in pairs of processes [3, 10]. As yet, a comprehensive closed-form solution covering the complete level-crossing cross-correlation function is currently lacking. Here, we use a regression approach to derive, for all correlation strengths, the conditional level-crossing correlation functions in two continuous Gaussian processes. We hypothesize that the level-crossing correlations we provide in this article could also be valuable in other fields outside of neuroscience for example in risk assessment calculations to predict the risk of joint default for insurance purposes.
The article is structured as follows. In Sect. 2 we define the mathematical model setting and introduce the concept of level crossings and specifically the upward crossings. In Sect. 3 we use a regression approach to obtain a general closed-form solution for cross-correlations of level crossings in two correlated Gaussian processes. In Sect. 4 we prove the joint Gaussianity (Central Limit Theorem) for the correlated joint upward crossings for two correlated Gaussian processes. In the section on materials and methods (Sect. 6) we provide detailed derivations of the reported results. We assume throughout this article that both level-crossing processes arise from crossings of the same threshold level by two Gaussian processes with different variances. This is permissible because the number of level crossings, the Rice rate , depends only on the variance-to-threshold ratio, but not on these quantities individually. We therefore work with a pair of level-crossing processes where each process has a unique voltage variance and therefore the rate of crossings in the two neurons being considered are, unless stated otherwise, not the same. Let us note that this assumption is prompted by the observation that in a living brain typically no two neurons are identical in all their properties and differ at least in their firing rate.
2 Mathematical Definitions of Multivariate Level Crossings
2.1 Definitions of Multivariate Voltage Distributions
2.2 Upward Crossing Definitions
where is the firing rate of a neuron j, for . In the next sections we provide closed-form expressions for and .
3 Cross-Correlations of Two Upward Level Crossings
Here, we address and provide a closed-form solution that is valid for any cross-correlation strength r between two level-crossing processes and any time delay τ.
ϕ and Φ are the standard Gaussian density and distribution, respectively, and . are the Hermite polynomials. Note that the first two terms in Eq. (18) correspond to truncation orders and , respectively.
Figure 2(a), (b) demonstrates obtained using Eq. (17) for different truncation orders n alongside the zero lag correlation . Figure 2(c), (d) demonstrates obtained using Eq. (17) as a function of the correlation strength r alongside the zero lag correlation . As previously, we chose and . We note that for two identical neurons () is a symmetric function. Yet, for a pair of neurons with different rates () the spike correlation function is asymmetric, indicating that the lower rate neuron spikes on average after the higher rate neuron.
3.1 Relation to the Leaky Integrate-and-Fire Model
where is the input current of a neuron, a white noise, unit variance drive. The voltage power spectrum for this model is a combination of low-pass filters and its correlation function can be determined according to Eqs. (8). If the voltage reaches the threshold ϕ the neuron j emits a spike and the voltage is subsequently reset to a reset value . The integrate-and-fire model differs only in one important detail from the level-crossing approach—the presence of a reset after a spike. A recent article by Laurent Badel systematically compared the validity of upward level-crossing approximation for the firing rate, spike correlations and frequency response of a leaky integrate-and-fire neuron . This study reached the conclusion that the upward level-crossing approach accurately represents the leaky integrate-and-fire model if two conditions are fulfilled: (1) the firing rate is much lower than the typical relaxation time of the voltage, (2) the synaptic filtering time constants remain of the same order of magnitude as the membrane time constant (). Numerically, the validity of the approximation remained highly accurate even for synaptic time constants .
A number of spike correlation results have been derived in the leaky integrate-and-fire model for the limit of weak correlations [18–20]. They include the observation that the spike correlation coefficient increases with firing rate [18, 19]. The equivalent firing rate dependent increase in spike correlations and correlation coefficients for low correlation strengths has been reported for the level-crossing model, see  and Fig. 3(A) (right) and Fig. 2(A) (top) in . Furthermore, leaky integrate-and-fire model exhibits a sublinear dependence of correlation coefficients on input strength r [18, 21], which we see confirmed in Fig. 4.
4 Joint Gaussianity of Upcrossing Counts
Spike count cross correlations and correlation coefficient measurements in pairs of neurons are ubiquitous in neuroscience and are often used to measure the strength of interdependencies in a pair of neurons, e.g. in cortical neurons [18, 19, 22], in model neurons  and in theoretical and experimental studies of net correlations emerging in recurrent networks [23–28]. Spike counts and their cross correlations in neuroscience are often computed for a variety of bin sizes varying from  to . Here, we are interested in the question when spike count correlations of two neurons computed in a bin size T are jointly Gaussian such that their cross correlations are unbiased measures of statistical dependence or independence.
will also converge to the respective ratio of the asymptotic covariances and variances.
4.1 Numerical Confirmation of Joint Gaussianity and Limit Covariances
is distributed according to a -distribution with d degrees of freedom (see, e.g., Sect. 3.1.4 and Eq. (3.16) in ). By numerically estimating the count sample average μ and Σ we calculate in our case and compare it with a -distribution, using the QQ-plot method (see Fig. 3(c)).
Figure 3 demonstrates the results of the joint Gaussianity tests for a bin size , where . Figure 3(a) shows the empirical univariate distribution of spike counts in one level-crossing process derived from independent count realizations. Figure 3(b) demonstrates that in independent count realizations of X p-values for all θ are above the 10 % significance level. Figure 3(c) (left) illustrates that the Mahalanobis distance of a two-dimensional spike count variable X are well approximated by the -distribution (solid line). Figure 3(c) (right) demonstrates in a QQ-plot of the empirically measured -quantiles and the theoretical -quantiles that they are linearly related. This is an indication that both distributions are equal.
Level-crossing phenomena occur in a variety of physical and biological sciences. In many of these situations coordination between level crossings of multiple cross-correlated Gaussian processes is of interest. Here, we focused on neuroscience and modeled the spikes of two cross-correlated neurons by two cross-correlated level-crossing processes. While crossings and extrema of one level-crossing process have been the focus of mathematical research, results describing the coordination of multiple level-crossing processes are sparse and typically available only in specific and limited cases. Limits where level-crossing cross-correlations have been previously calculated are the weak and strong input correlation limit . Here, we studied the case of two cross-correlated upward crossing processes and derived closed-form expressions for their joint level-crossing coordination as well their joint count Gaussianity. Importantly, the results we present in this article are consistent with previously reported limits but we now extended and generalized them. The two main results of our article are (1) closed-form explicit solution of the level-crossing cross-correlations and (2) the joint Gaussian limit of level-crossing counts. Our first result provides an explicit solution to that is valid for all correlation strengths and which comprises previously obtained limits, see discussion in Sect. 3. The rate of level crossings by a one-dimensional Gaussian process is given by the prominent Rice’s equation derived by Rice in the 1950s . The solution we obtained for the level-crossing cross-correlation extends the Rice rate to the joint rate of two correlated processes. Our second result proves the joint Gaussianity of level crossings for large bin sizes. The joint Gaussianity of spike counts is a highly desired property because if and only if two level-crossing counts are jointly Gaussian can zero count cross-correlation imply statistical independence. Notably, marginal Gaussianity of spike counts in each neuron combined with zero count cross-correlation is not sufficient to imply independence. Contrasting examples of where X and Y variables are both marginally but not jointly Gaussian, have a zero cross-correlation but are not independent can be found in Sect. 5 in . Count covariance and measures derived from it, such as the Pearson correlation coefficient, are computationally inexpensive and widely used as measures of statistical interdependencies . Therefore, it is highly desirable to investigate the joint Gaussianity of level counts and thereby delimit the parameter space and mathematical conditions ensuring that independence can be implied from zero correlation coefficient. Notably, the joint Gaussianity of spike counts in bins of size T where T is much larger than the intrinsic time constant () also implies that models of multi-neuronal dynamics only need to consider the mean and variance of spike counts because all higher cumulants are zero.
6 Materials and Methods
6.1 Proof of Proposition 3.1
for , , ,
for , , ,
for , , .
for , , ,
for , , ,
for , , .
for , , ,
for , , ,
for , , .
for , , ,
for , , ,
for , , .
Note that the first two terms in Eq. (42) correspond to orders and , respectively. Denoting we find . Here, is a uniformly convergent series. □
6.2 Zero Time Lag Correlations
Solving this integral we obtain Eq. (23).
6.3 Proof of Theorem 4.1
where . A Gaussian distribution is a stable limit distribution for a sum of independent finite variance variables. Therefore, all that is left to prove is that contributions are independent and have finite variance. From Mehler’s Formula we recognize that the contributions for are independent. The finite variance follows from the observation that for all q the variance of is proportional to the expectation of a product of four Hermite polynomials, which has been proven to be finite (Theorem 10.10 in ) if the conditions of Theorem 4.1 are satisfied.
This is the result reported in Theorem 4.1. □
6.4 Modified Breuer–Major Theorem
Here, we adapt the Breuer–Major Theorem  to show that the bivariate vector is Gaussian.
This follows from the Central Limit Theorem for -dependent random vectors and concludes the proof. □
The authors thank the two anonymous referees, and Sabrina Münzberg, Amadeus Dettner, and Laurent Badel for constructive comments on a previous version of the paper, Sabrina Münzberg for help with Hermite polynomial calculations and associated problem solving and Sara Gil Mast for English corrections. EDB and JRL are supported by a ECOS-Nord project under the reference V12M01. TT is funded by the Volkswagen foundation and the Max Planck Society and is thankful for the support of the Center for Theoretical Neuroscience at Columbia University during her stay there.
- Blake IF, Lindsey WC: Level-crossing problems for random processes. IEEE Trans Inf Theory 1973, 19: 295–315. 10.1109/TIT.1973.1055016MATHMathSciNetView ArticleGoogle Scholar
- Azaïs JM, Wschebor M: Level Sets and Extrema of Random Processes and Fields. Wiley, New York; 2009.MATHView ArticleGoogle Scholar
- Tchumatchenko T, Malyshev A, Geisel T, Volgushev M, Wolf F: Correlations and synchrony in threshold neuron models. Phys Rev Lett 2010., 104(5): Article ID 058102 Article ID 058102Google Scholar
- McNeil AJ, Frey R, Embrechts P Princeton Series in Finance. In Quantitative Risk Management: Concepts, Techniques, and Tools. Princeton University Press, Princeton; 2005.Google Scholar
- Ehrenfeld S, Goodman NR, Kaplan S, Mehr E, Pierson WJ, Stevens R, Tick LJ: Theoretical and observed results for the zero and ordinate crossing problems of stationary Gaussian noise with application to pressure records of ocean waves. Technical report. New York University, College of Engineering; 1958.Google Scholar
- Burak Y, Lewallen S, Sompolinsky H: Stimulus-dependent correlations in threshold-crossing spiking neurons. Neural Comput 2009, 21(8):2269–2308. 10.1162/neco.2009.07-08-830MATHMathSciNetView ArticleGoogle Scholar
- Destexhe A, Rudolph M, Pare D: The high-conductance state of neocortical neurons in vivo . Nat Rev Neurosci 2003, 4: 739–751. 10.1038/nrn1198View ArticleGoogle Scholar
- Embrechts P, McNeil A, Straumann D: Correlation and dependence in risk management: properties and pitfalls. In Risk Management: Value at Risk and Beyond. Cambridge University Press, Cambridge; 2000:176–223.Google Scholar
- Dayan P, Abbott LF: Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. MIT Press, Cambridge; 2001.Google Scholar
- Tchumatchenko T, Geisel T, Volgushev M, Wolf F: Signatures of synchrony in pairwise count correlations. Front Comput Neurosci 2010. 10.3389/neuro.10.001.2010Google Scholar
- Rice SO: Mathematical analysis of random noise. In Selected Papers on Noise and Stochastic Processes. Edited by: Wax N. Dover, New York; 1954.Google Scholar
- Brunel N, van Rossum M: Lapicque’s 1907 paper: from frogs to integrate-and-fire. Biol Cybern 2007, 97(5):337–339.MATHMathSciNetView ArticleGoogle Scholar
- Burkitt A: A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input. Biol Cybern 2006, 95: 1–19. 10.1007/s00422-006-0068-6MATHMathSciNetView ArticleGoogle Scholar
- Burkitt A: A review of the integrate-and-fire neuron model: II. Inhomogeneous synaptic input and network properties. Biol Cybern 2006, 95: 97–112. 10.1007/s00422-006-0082-8MATHMathSciNetView ArticleGoogle Scholar
- Fourcaud N, Brunel N: Dynamics of the firing probability of noisy integrate-and-fire neurons. Neural Comput 2002, 14: 2057–2110. 10.1162/089976602320264015MATHView ArticleGoogle Scholar
- Badel L: Firing statistics and correlations in spiking neurons: a level-crossing approach. Phys Rev E 2011., 84: Article ID 041919 Article ID 041919Google Scholar
- Genz A, Bretz F: Computation of Multivariate Normal and t Probabilities. Springer, Berlin; 2009.MATHView ArticleGoogle Scholar
- de la Rocha J, Doiron B, Shea-Brown E, Josic K, Reyes A: Correlation between neural spike trains increases with firing rate. Nature 2007, 448(16):802–807.View ArticleGoogle Scholar
- Shea-Brown E, Josić K, de la Rocha J, Doiron B: Correlation and synchrony transfer in integrate-and-fire neurons: basic properties and consequences for coding. Phys Rev Lett 2008., 100(10): Article ID 108102 Article ID 108102
- Ostojic S, Brunel N, Hakim V: How connectivity, background activity, and synaptic properties shape the cross-correlation between spike trains. J Neurosci 2009, 29(33):10234–10253. 10.1523/JNEUROSCI.1275-09.2009View ArticleGoogle Scholar
- Binder MD, Powers RK: Relationship between simulated common synaptic input and discharge synchrony in cat spinal motoneurons. J Neurophysiol 2001, 86(5):2266–2275.Google Scholar
- Lampl I, Reichova I, Ferster D: Synchronous membrane potential fluctuations in neurons of the cat visual cortex. Neuron 1999, 22: 361–374. 10.1016/S0896-6273(00)81096-XView ArticleGoogle Scholar
- Vilela RD, Lindner B: Comparative study of different integrate-and-fire neurons: spontaneous activity, dynamical response, and stimulus-induced correlation. Phys Rev E 2009., 80(3): Article ID 031909 Article ID 031909
- Greenberg DS, Houweling AR, Kerr JND: Population imaging of ongoing neuronal activity in the visual cortex of awake rats. Nat Neurosci 2008, 11(7):749–751. 10.1038/nn.2140View ArticleGoogle Scholar
- Tetzlaff T, Rotter S, Stark E, Abeles M, Aertsen A, Diesmann M: Dependence of neuronal correlations on filter characteristics and marginal spike train statistics. Neural Comput 2008, 20: 2133–2184. 10.1162/neco.2008.05-07-525MATHMathSciNetView ArticleGoogle Scholar
- Renart A, de la Rocha J, Bartho P, Hollender L, Parga N, Reyes A, Harris K: The asynchronous state in cortical circuits. Science 2010, 327: 587–590. 10.1126/science.1179850View ArticleGoogle Scholar
- Ecker AS, Berens P, Keliris GA, Bethge M, Logothetis NM, Tolias AS: Decorrelated neuronal firing in cortical micro circuits. Science 2010, 327(584):584–587.View ArticleGoogle Scholar
- Pernice V, Staude B, Cardanobile S, Rotter S: How structure determines correlations in neuronal networks. PLoS Comput Biol 2011., 7(5): Article ID e1002059 Article ID e1002059 10.1371/journal.pcbi.1002059
- Zohary E, Shadlen MN, Newsome WT: Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 1994, 370: 140–143. 10.1038/370140a0View ArticleGoogle Scholar
- Kratz M, León J: Level curves crossings and applications for Gaussian models. Extremes 2010, 13: 315–351. 10.1007/s10687-009-0090-xMATHMathSciNetView ArticleGoogle Scholar
- Doukhan P, Jakubowicz J, León JR: Variance estimation with applications. In Dependence in Probability, Analysis and Number Theory. Kendrick Press, Heber City; 2010:203–231.Google Scholar
- Feller W 2. In An Introduction to Probability Theory and Its Applications. 2nd edition. Wiley, New York; 1971.Google Scholar
- Hamedani GG, Tata MN: On the determination of the bivariate normal distribution from distributions of linear combinations of the variables. Am Math Mon 1975, 82(9):913–915.MATHMathSciNetView ArticleGoogle Scholar
- Breuer P, Major P: Central limit theorems for non-linear functionals of Gaussian fields. J Multivar Anal 1983, 13(3):425–441. 10.1016/0047-259X(83)90019-2MATHMathSciNetView ArticleGoogle Scholar
- Kratz M, León JR: Central limit theorems for level functionals of stationary Gaussian processes and fields. J Theor Probab 2001, 14(3):639–672. 10.1023/A:1017588905727MATHView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.