Inhomogeneous Sparseness Leads to Dynamic Instability During Sequence Memory Recall in a Recurrent Neural Network Model

Theoretical models of associative memory generally assume most of their parameters to be homogeneous across the network. Conversely, biological neural networks exhibit high variability of structural as well as activity parameters. In this paper, we extend the classical clipped learning rule by Willshaw to networks with inhomogeneous sparseness, i.e., the number of active neurons may vary across memory items. We evaluate this learning rule for sequence memory networks with instantaneous feedback inhibition and show that little surprisingly, memory capacity degrades with increased variability in sparseness. The loss of capacity, however, is very small for short sequences of less than about 10 associations. Most interestingly, we further show that, due to feedback inhibition, too large patterns are much less detrimental for memory capacity than too small patterns.

represented, i.e., in terms of how many neurons are active during recall. In this paper, we extend a particular class of such auto-association networks, viz., sequence memory networks, to include variable sparseness and thereby add one aspect of variability that is to be expected in biological neural networks.
Memory sequences have been shown to occur in the rodent brain during hippocampal sharp-wave ripple events [4,5]. The major hypothesis of the present paper is that these sequences are stored in the recurrent connections of the hippocampal network, which is supported by findings of fast coordinated excitatory synaptic currents during sharp-wave ripple events in slices [6].
Here, we build on a previous model of memory sequences [7], which enhances memory capacity by instantaneous feedback inhibition. Both our mean field analysis and our simulations show that, in this model, inhomogeneity in pattern sizes reduces memory capacity, but it does so in an asymmetric way: whereas too small patterns significantly compromise the stability of sequence recall, too large patterns can be compensated for quite robustly.

Model
We use the standard formalism of auto-associative networks: a discrete-time dynamical system. The individual time steps may be interpreted as the cycles of the hippocampal ripple oscillations. The states x i (t) of all neurons 1 ≤ i ≤ N at time step t determine the states at time t + 1 by a thresholded function Here, as in many other approaches, we define Θ as the Heaviside function, which is equivalent to restricting the neuron states to binary variables, with x i = 1 if neuron i fires and x i = 0 if it is silent. The state of the network at time t is thus denoted by x(t) ∈ {0, 1} N . The other parameters are the firing threshold θ and the synaptic weights J ij . In the framework of the dynamical system of (1), associative memory is considered to be the (approximate) recall of a network state x(t + n) = ξ t+n at time t + n after the network has been initialized with some appropriate cue x(t) = ξ t at time t. Recall can either occur as convergence to a dynamical attractor (n → ∞) [2], or as a one-step association (n = 1) [8,9]. The specific choice of the synaptic matrix J ij determines which patterns can be recalled, or are stored in the network.
In this paper, we will deal with sequences of one-step associations with binary synapses [8,10,11] and instantaneous feedback inhibition [7,9,12]. Here, memory sequences are described as sequences of random activity patterns ξ that are binary vectors of dimension N , ξ ∈ {0, 1} N . A memory sequence of length Q is an ordered occurrence of activity patterns ξ 1 → ξ 2 → · · · → ξ Q . The number M t of active neurons in each pattern ξ t is called pattern size, and will in general be different for each pattern. Fig. 1 Coding ratio distribution p φ (φ). We use the Gamma distribution in our simulations, shown here for a mean coding ratio φ 0 = 0.01 and three different values of the variation coefficient σ φ /φ 0 As proposed in [7,9,11], we model the weights J ij of the binary synapses as products of two independent binary stochastic processes, J ij = w ij s ij . The first stochastic variable w ij indicates the presence (w ij = 1) or absence (w ij = 0) of a morphological connection, with probability Pr(w ij = 1) = c m , called morphological connectivity. The second process s ij is called synaptic state and will be used to store memories. In the potentiated state (s ij = 1), a presynaptic spike increments the postsynaptic potential by 1, whereas in the silent state (s ij = 0), the postsynaptic potential remains unaffected. According to (1), neuron i fires a spike at cycle t + 1 if its postsynaptic potential h i (t) = N j =1 w ij s ij x j (t) at time t exceeds the threshold θ . Willshaw's [10] clipped Hebbian rule is used to set the synaptic states s ij such that the network is able to recall the memory sequences: a synapse is in the potentiated state only if it connects two neurons that are activated in sequence at least once.
In the case of homogeneous sparseness, where all patterns have the same number M of active neurons, Willshaw's rule connects the fraction c/c m of potentiated synapses in the network with the number P of stored associations by where f = M/N is the coding ratio. The effective connectivity c defines the noise level during recall, i.e., how many spurious inputs a neuron gets that are not part of the memory pattern to be recalled. If c is too large, the network will exhibit many spurious activations and the memory can no longer be recalled. Equation (2) thus provides a capacity estimate of the network in that it says how many associations are stored at the maximum noise level c.
In the case of inhomogeneous sparseness, this formula is no longer valid and thus we asked what would its generalization be. To this end, we introduce the coding ratio vector where M k = f k N is the number of active neurons in the stored pattern ξ k . The elements of φ are considered to be random variables and distributed according to the coding ratio distribution p φ (φ), with mean coding ratio φ 0 and standard deviation σ φ . For now, we use a Gamma distribution ( Fig. 1) for p φ (φ) with mean φ 0 = 0.01. The histograms (blue) correspond to one million random samples of the coding ratio vector φ. The elements of φ are drawn randomly from the Gamma distribution with φ 0 = 0.01, shown in Fig. 1. The normal distribution (red) with mean ς and variance σ 2 ς (see the Appendix) is a good approximation to the true distribution of ς for low values of σ φ /φ 0 By varying the standard deviation σ φ we control how much the elements of vector φ deviate from its mean φ 0 .
In analogy to (2), the probability of synaptic potentiation ς = c/c m can be computed analytically for any given coding ratio vector φ (see the Appendix) as follows: This expression, however, only provides an implicit dependence on the number P of patterns, and moreover, it of course depends on the specific choice of φ. We therefore asked how much ς varies over the statistical ensemble of φ. In addition to addressing this question numerically, we could also find analytical expressions for the mean ς and variance σ 2 ς of ς over all possible realizations of φ, which depend on the first to fourth-order moments of the coding ratio distribution p φ (φ) (see the Appendix). For sufficiently small variances σ 2 φ , the Gaussian approximations resulting from ς and The results show that the variability in ς is actually relatively large (about 10 % for σ φ = 0.1φ 0 ) and even increases with increasing number of associations P . We therefore decided not to use the expectation value ς for further discussion, but to show empirical distributions of many realizations of φ whenever possible.
In order to evaluate the dynamics of sequence retrieval, we mostly do not simulate the full neural network but use a mean field approximation [7,11] based on two macroscopic dynamic variables: the number m t ∈ [0, M t ] of correctly activated neu-rons (hits) and the number n t ∈ [0, N − M t ] of incorrectly activated neurons (false alarms). For large network sizes N and large pattern sizes M t , the central limit theorem predicts the distribution of the total number of synaptic inputs h(t) to be Gaussian, and the variables m t and n t can be reinterpreted as expectation values over many realizations of the connectivity matrix. Denoting the mean number of synaptic inputs as μ ≡ h(t) and the variance as 2 , we obtain (see the Appendix) for the On population (should fire), and for the Off population (should not fire), Willshaw's learning rule yields correlations in the synaptic states that are captured by the terms proportional to V 2 ς (see the Appendix). The discrete-time network dynamics in (1) maps to the mean field model such that with where Φ(z) ≡ [1 + erf(z/ √ 2)]/2 denotes the cumulative distribution function (cdf) of the normal distribution.
In the framework of this model and following [7,12], inhibition is introduced as instantaneous negative feedback proportional to the total number m t + n t of active neurons at time t. Formally, this is achieved by substituting θ → θ + h t in (10) and (11), where h t = b(m t + n t ). The inhibitory weight is taken as b = c m ς throughout the paper (for discussion see [7]).

Inhomogeneous Sparseness Reduces Dynamic Stability
Some exemplary numerical evaluations of the mean field dynamics from equations (9) and following for different firing thresholds θ and different pattern size inhomogeneities σ φ are shown in Fig. 3. Despite being just random samples, these plots already reveal the impact of increasing inhomogeneity in the coding ratio vector φ on the network's ability to successfully retrieve the stored patterns. As the standard deviation σ φ grows, the activity fluctuations during replay become more and more pronounced and, at some point, lead to dynamic instability during recall. This is clearly visible for a firing threshold θ = 28, where perfect pattern retrieval (m t /M t = 1 and n t /(N − M t ) = 0) is interrupted more and more frequently as σ φ increases, eventually preventing the retrieval of the full sequence at σ φ /φ 0 = 20 %. There the network falls silent, due to a small pattern in the sequence that no longer generated the required synaptic drive to retrieve the following pattern in the sequence. For lower thresholds (θ = 24, 26), the network activity may instead explode prematurely due to a big pattern that generates too much synaptic drive and sets the network into a permanently active (epileptic) state. As we simulate a network with instantaneous feedback inhibition, the epileptic state is characterized by approximately half of the neurons being active at any time (m t /M t ≈ n t /(N − M t ) ≈ 1/2), where the subset of active neurons changes from time step to time step.
In summary, the range of thresholds under which the network successfully replays the full sequence is severely reduced as the pattern sizes become more and more inhomogeneous.

Replay Success Rate
In order to analyze the numerical results more quantitatively, we introduce a criterion for what we consider to be a successful replay. Following [11], we define the retrieval quality as the relative difference between hit ratio and false alarm ratio, and consider a pattern at time t to be retrieved successfully if Γ t > 0.5. By running the mean field equations many times with different random realizations of vector φ, we obtain an empirical replay success rate t as the fraction of runs with successful retrieval Γ t > 0.5. Figure 4 shows replay success rates for a sequence of length Q = 100 for φ 0 = 0.01 and varying inhomogeneities σ φ . When the P stored associations are still approximately homogeneous (top left), the full sequence can be retrieved with probability one for a large range of firing thresholds θ . As we let inhomogeneity increase, this range becomes narrower (top right), and eventually collapses (bottom), so that only the first items can be retrieved with high probability. Hence, inhomogeneous sparseness strongly affects the replay of long sequences, but does relatively little harm to short sequences (Q 5).

Region of Stable Replay
Sequence retrieval not only critically depends on the firing threshold θ , but also on the mean coding ratio φ 0 . We therefore searched for regions of stable replay in (φ 0 , θ) space (Fig. 5) at time step t = 100. The region where sequence replay for homogeneous patterns (σ φ = 0) is unfeasible is shown in white for comparison. Replay regions exhibit the typical wedge shape [7,11]. If the firing threshold θ is too low or the mean coding ratio φ 0 is too large, all neurons immediately start to fire and the network falls into an all-active state. If the firing threshold is too high or the mean coding ratio is too small, the network immediately falls into an all-silent state. As patterns become less homogeneous, the wedge-shaped region of replay becomes narrower and narrower, and eventually vanishes for highly inhomogeneous patterns (bottom right). The region of retrieval obtained from the mean field equations was validated with some computer simulations of the corresponding networks of binary neurons (white discs, corresponding to 95 % replay success rate).

Storage Capacity
As in the classical Willshaw net ( [10], and (2)) the mean synaptic connectivity c for a network with inhomogeneous pattern sizes depends on the number P of stored associations, as well; see (4). This allowed us to adjust the mean connectivity to a fixed value c = 0.05 by changing the number P of stored associations in the network. So far, we have kept c constant and varied the width parameter σ φ of the size distribution. We saw that for large inhomogeneities (σ φ /φ 0 20 %) replay of long sequences is hardly possible. But what if we reduce c? Intuitively, replay should become more stable if we reduce the "noise" connectivity c. As shown in Fig. 6, this is indeed the case: If the number of stored associations, and thus c, is decreased, sequence retrieval is robust under high inhomogeneity, and may even allow for replay of the full sequence for a whole range of firing thresholds.
However, reducing c comes at the cost of a reduced memory capacity P , and thus we have to find a way to quantify the trade-off between replay stability and capacity for a network with inhomogeneous patterns sizes. To this end, we define the maximum retrievable sequence length as the maximum number T 90 of time steps for which the replay success rate T remains above 90 % for a given pattern size vector φ and morphological connectivity c m . Since replay stability strongly depends on the firing threshold, the maximum is taken over all possible firing thresholds θ . Figure 7 shows T as a function of the number P of stored associations in the network (as well as the corresponding mean connectivity c). When the total number of stored associations is small, even a sequence consisting of all stored associations may be retrieved under high inhomogeneity (σ φ /φ 0 = 25 %). As P grows, the T curve reaches a maximum and then decreases very quickly. The breakdown point and slope depend critically on the degree of inhomogeneity. For the homogeneous case σ φ → 0, T decreases infinitely steeply, and the number P of patterns at the breakdown determines the "classical" storage capacity. For finite inhomogeneities σ φ , T decreases according to a power law T ∝ P −α that trades stability T vs. capacity P . Since the exponent α is much smaller for high inhomogeneities σ φ /φ 0 , the net decrease of capacity for short sequences ( 10) is relatively small compared to a network with homogeneous sparseness (a decrease of ∼ 1.8 for σ φ /φ 0 = 25 %).

Asymmetry of the Size Distribution
From our observations of single runs in Fig. 3, we already derived some anecdotal insight into the mechanisms underlying the breakdown of sequence replay: network activity may cease after a small pattern, whereas large patterns may lead to epilepsy. However, it is unclear which of these two ways of terminating replay is more problematic, or whether both occur equally often.
In order to tackle this question about the mechanisms of sequence termination, we investigate the effect of skewness (or asymmetry) in the pattern size distribution, i.e., an imbalance between bigger and smaller than average patterns. So far, we have used the Gamma distribution shown in Fig. 1, which is relatively symmetric for small variation coefficients. To have a better handle on skewness, we now switch to triangular distributions (Fig. 8).
A symmetric triangular distribution (Fig. 8b) is used for comparison. In order to study the effect of an excess of small patterns we constructed a negatively skewed distribution (Fig. 8a) by cutting away all patterns above the line of symmetry (φ max ) and adding smaller patterns instead. Since this distribution has a lower mean coding ratio φ 0 = φ max − √ 2σ φ , we increased the number P of patterns to account for the same "noise" connectivity c as in the symmetric case. The region of stable replay is clearly reduced in the negatively skewed distribution as compared to the symmetric one. This reduction could either be because small pattern sizes are intrinsically bad, or because the asymmetry of the distribution is a limiting factor. We therefore also considered the case of an excess of large patterns (Fig. 8c). For such a positively skewed distribution, the asymmetry is the same as for the negatively skewed distribution; however, and interestingly, the region of stable replay is larger than for the symmetric distribution. Again, the connectivity was adjusted to the same value by reducing the number P of associations to compensate for the higher mean coding ratio φ 0 = φ max + √ 2σ φ . From these observations, we conclude that indeed the small patterns are much more problematic for replay with inhomogeneous pattern sizes than the large patterns. To understand why, we compared the shape of the replay regions for the three distributions (a, b, and c), and observe that the slope of the lower side of the wedge is with maximum at φ max = 0.01, mean coding ratios φ 0 = φ max (± √ 2σ φ ), and fixed standard deviation σ φ = 0.1φ max : a negatively skewed distribution, b symmetric distribution, c positively skewed distribution. Panels (cf. Fig. 5): Replay success rates in the (φ max , θ) plane. The region of retrieval obtained from the mean field equations was validated with computer simulations of some of the corresponding networks of binary neurons (white discs, 95 % success rate). Parameters were N = 10 5 , c m = 0.1, and c = 0.05 relatively insensitive to skewness, whereas the slope of the upper side of the wedge is very different in each case. Failures owing to activity explosion (the lower side of the wedge) are almost independent of the skewness, due to the instantaneous feedback inhibition in the mean field equations. On the other hand, the upper side of the wedge is determined by the network's falling into a silent state. Thus, positive skewness (c) is the more robust distribution. Note that this was also apparent in Fig. 5, where the reduction of the region of stability with increasing inhomogeneity was much more pronounced on the upper side of the wedges than on the lower side, despite the relatively symmetric Gamma distribution used there.
Finally, in order to more directly illustrate how replay terminates after small patterns, we show a scatter plot of 10 4 sample pairs (M τ , M τ +1 ), where τ is the last time step at which the pattern was replayed with sufficient quality Γ τ > 0.5 ( Fig. 9; note that here we again used a Gamma distribution to achieve comparability with Fig. 4). Points below the red line (M τ > M τ +1 ) represent sequences for which the last correctly replayed pattern ξ τ was bigger than the following pattern ξ τ +1 , whereas points above the red line (M τ < M τ +1 ) represent sequences for which ξ τ was smaller than ξ τ +1 . On the low-threshold edge of the stability region, which is prone to overexcitement (θ = 25, cf. Fig. 5), a big-to-small pattern transition is as likely to lead to a failure in sequence replay as a small-to-big transition. However, on the high-threshold edge of the stability region (θ = 30), where replay eventually dies out, most failures are caused by small-to-big pattern transitions (∼ 80 % of points above the red line).
Again, these results show that the small patterns are more detrimental for sequence replay than the large patterns, since in the latter case fluctuations can be compensated for by feedback inhibition, whereas the former have no compensatory mechanism.

Nonlinear Inhibition
So far, we have assumed a linear dependence of instantaneous feedback inhibition on the total network activity, i.e., h t = b(m t + n t ), since it was shown to optimize replay quality [7,12]. In this final section, we investigate how a particular nonlinear form of inhibition could improve the network's resilience to inhomogeneity, because (a) physiological data from cortical inhibitory networks suggest supralinear dependence on input [13,14] and (b) supralinear inhibition effectively provides a positive feedback (with respect to linear inhibition) in cases of too low activity.
To be able to best compare the nonlinear with the linear case, we constructed a nonlinearity that only implements such positive boost for too low activities m t + n t and remains linear for too large activities (see upper panel of Fig. 10b). Formally, it is obtained by replacing the linear term b(m t + n t ) by the function h t = h(m t + n t ), where with parameters chosen such that the slope of h(x) at the operation point x = m t + n t = φ 0 N is equal to b in both the linear and nonlinear parts.  Figure 10 shows the mean retrieval qualityΓ t at time step t = 100 (averaged over 10 2 random realizations of vector φ) in the (φ 0 , θ) plane, for linear (a) and nonlinear (b) inhibition and two levels of inhomogeneity σ f /φ 0 .
For low inhomogeneity (σ φ /φ 0 = 5 %), although the region of stability is wider in the nonlinear case, the retrieval quality in the gained region is not as good as in the region shared by both feedback strategies (see lighter red stripe in middle panel of Fig. 10b). This finding fits very well to previous papers that report that linear inhibition maximizes replay quality for homogeneous pattern sizes: the gain in robustness is mostly obtained by a reduced replay quality. For a large inhomogeneity (σ φ /φ 0 = 20 %), linear feedback almost completely extinguishes replay, whereas nonlinear inhibition recovers a considerable stable replay region with high retrieval quality.
Supralinear inhibitory feedback at low activity levels thus significantly widens the replay region making the network resilient to higher levels of inhomogeneity than would be possible with linear feedback. The underlying mechanism by which this is achieved can be explained as follows: smaller-than-average patterns generate only little negative feedback and thereby keep up the activity in the network, whereas bigger-than-average patterns are compensated for optimally by linear negative feedback.

Discussion
This paper extends previous models of sequence memory [7,9,11,12] that were based on Willshaw's learning rule [10] to inhomogeneous pattern sizes, i.e., patterns of variable sparseness. Our work reveals that inhomogeneity in the sparseness of stored patterns is detrimental to a recurrent network's dynamic stability during sequence retrieval. Bigger than average patterns tend to lead the network into an allactive epileptic state as a result of an excessively high synaptic drive, whereas smaller than average patterns tend to lead to an all-silent state as a result of an insufficient synaptic drive. In either case, sequence retrieval is terminated prematurely due to dynamic instability. As expected, the higher the variability in pattern sizes, the higher is the probability of premature sequence termination. Our results thus suggest that a plasticity mechanism that ensures a certain degree of homogeneity in the sparseness of hippocampal representations would be useful for the reliable retrieval of long sequences.
Instantaneous linear feedback inhibition is able to compensate to a certain degree for bigger-than-average patterns, but it does nothing to prevent the network from falling silent since it does not compensate for an insufficient synaptic drive. This asymmetry is reflected on the relative impact of differently skewed pattern size distributions. Compared to a symmetric distribution, negative skewness leads to a smaller region of stable replay, whereas positive skewness leads to a larger region. Positively skewed pattern size distributions are thus more resilient to premature sequence termination under linear feedback. The higher vulnerability to smaller-than-average patterns can be corrected for by introducing a nonlinear negative feedback which is close to zero for lower-than-average network activity. Such supralinear inhibition can make the network resilient to higher levels of inhomogeneity than linear feedback inhibition.
Memory networks with variable sparseness have been studied by Amit and Huang [15,16] under a different learning paradigm in which old memories are gradually overwritten by new memories, and for several more involved synaptic (meta-)plasticity rules. There, inhomogeneity in the pattern sizes was shown to decrease the signal-to-noise ratio during recall as well.
In contrast to palimpsest models [17][18][19][20][21][22] in which old memories are overwritten, our model assumes that all memories are equally well preserved in the synaptic states of the network, which argues for additional plasticity rules that continuously readjust the synaptic matrix to keep old memories fresh. Such a mechanism would necessarily require ongoing plasticity rules, which may then easily be linked to some sort of pattern size homeostasis that tries to keep the sparseness homogeneous. Such persistent network remodeling fits experimental findings that, at least for a few weeks after memory acquisition, existing memories can be extinct by blocking protein synthesis together with memory reactivation [23], hinting at the presence of plasticity mechanisms during early retrieval.
(a k = 0). Since we assume the participation in a pattern to be random, the probability for a specific activation schedule is computed as (1 − f α(k) ) (18) where |A| is the number of patterns in which the neuron is active and α : {1, . . . , P } → {1, . . . , P } is a reordering of the patterns such that those in which the neuron is active have the |A| lowest indices. If |A| = 0, the first factor equals 1 as indicated by the Kronecker symbol δ |A|=0 . The activation schedule of a neuron allows us to compute the fraction ς A of putative activated synapses at a postsynaptic neuron with activation schedule A in analogy to the classical Willshaw idea, where the product on the right-hand side is the fraction of synapses remaining inactive after storing P sequential activations, corresponding to |A| learning steps. In order to compute the mean connectivity in the network, we average over all postsynaptic neurons (i.e., activation schedules) and obtain where we have introduced the abbreviation χ k = f k (1−f k−1 ) 1−f k and used the algebraic identity Similarly, the second moment E[ς 2 A ] is computed as (1 − f k ) (1 − f k ) (1 − f k ) (1 − f k ) A.2 Mean and Variance of Total Synaptic Input With the above two moments, we can find means and variances for the synaptic inputs. We start with the probability of total synaptic input to a postsynaptic cell with activation schedule A, which is binomially distributed according to The probability of total synaptic input to an average postsynaptic cell can then be obtained as and for the On population, = c m m + c m ςn.
In order to obtain the variance of h, we compute the second moment of h for the Off population, = c m (m + n) = c m (m + n) ς + E ς 2 A c m (m + n − 1) .
The variance is then given by where is the squared variation coefficient of ς A over all activation schedules A. Similarly, for the On population, we get

A.3 Mean and Variance of ς over Pattern Size Distribution
So far, all formulas were obtained for a specific realization of the pattern size (coding ratio) vector φ. The pattern sizes themselves can, however, be considered as resulting from a stochastic process as well. We therefore are interested in expectation values over the pattern size distribution to be able to account for average connectivities over many realizations of the network. Such an average connectivity ς upon imprinting the memory with P patterns is given by where · indicates the expected value over the size distribution p φ (φ). The last term can be expanded as follows: The last term on the right, as well as higher-order terms, contain overlapping indices (e.g., when t = t + 1), so that for each term of order 2k (for k = 1, . . . , P ), we will have 2j isolated indices, each contributing a term f , and (k − j) duplicated indices, each contributing a term f 2 (for j = 1, . . . , k). Therefore, we can write where n (P ,k) j is the number of k-combinations of P elements with exactly j non-adjacent element sets. For example, given P elements a 1 , a 2 , . . . , a P , the 3combination a 1 a 2 a 3 has j = 1 non-adjacent sets, a 1 a 3 a 4 has j = 2, and a 1 a 3 a 5 has j = 3. After some algebra, we arrive at the expression n (P ,k) j = P −k+1 j k−1 k−j , j ≤ min(k, P − k + 1), 0, otherwise.
The mean probability of synaptic potentiation ς can thus be expressed as a function of the first and second order moments of the coding ratio distribution p φ (φ) as follows: with the first and second moments The second moment ς 2 of the probability of synaptic potentiation is given by (1 − f k f k−1 ) + P k=1 (1 − f k f k−1 ) 2 (75) where the last term equals P t=1 (1 − f t f t+1 ) 2 (77)