Coding advantage of grid cell orientation under noisy conditions

Grid cells constitute a crucial component of the “GPS” in the mammalian brain. Recent experiments revealed that grid cell activity is anchored to environmental boundaries. More specifically, these results revealed a slight yet consistent offset of 8 degrees relative to boundaries of a square environment. The causes and possible functional roles of this orientation are still unclear. Here we propose that this phenomenon maximizes the spatial information conveyed by grid cells. Computer simulations of the grid cell network reveal that the universal grid orientation at 8 degrees optimizes spatial coding specifically in the presence of noise. Our model also predicts the minimum number of grid cells in each module. In addition, analytical results and a dynamical reinforcement learning model reveal the mechanism underlying the noise-induced orientation preference at 8 degrees. Together, these results suggest that the experimentally observed common orientation of grid cells serves to maximize spatial information in the presence of noise. Author summary Spatial navigation depends on several specialized cell types including place and grid cells. Grid cells have multiple firing fields that are arranged in a regular hexagonal pattern. The axes of this pattern are anchored to environmental boundaries at a universal angle of 8°. Here, we combine computer simulations of the grid cell network with analytical derivations and a reinforcement learning model to explain the functional relevance of this universal grid cell orientation. We show that spatial information provided by grid cells is maximized at the experimentally observed grid orientation within a broad parameter range. This relationship occurs only in the presence of noise. The model allows for several experimentally testable predictions including the number of grid cells.


Introduction
The past decade has witnessed substantial progress in our understanding of the internal positioning system ("GPS") in the mammalian brain. Specifically, place cells in the CA1 region of the hippocampus convey unique information about spatial position [1][2], while grid cells implement a universal coordinate system that provides information about spatial distances [3]. In addition, a number of other functionally dedicated cells were found to play a role in creating the neural representation of space, such as border cells [4][5], head direction cells [6], and speed cells [7]. Among all the known types of cells that constitute the brain's GPS, grid cells have received particular attention due to their abundant number, their determining role in generating an inner metric for navigation [8][9][10], and their close relationship to Alzheimer's disease [11][12].
A large number of experiments in animals and humans have demonstrated that grid cells exhibit multiple hexagonally arranged firing fields that tile the entire space ( Fig 1A). Because of the efficient geometrical organization of the firing fields, activity of a small number of grid cells is sufficient to provide a nearly full coverage [3,[13][14]. In addition to the hexagonal geometry of the firing patterns,

Author summary
Spatial navigation depends on several specialized cell types including place and grid cells. Grid cells have multiple firing fields that are arranged in a regular hexagonal pattern. The axes of this pattern are anchored to environmental boundaries at a universal angle of 8°. Here, we combine computer simulations of the grid cell network with analytical derivations and a reinforcement learning model to explain the functional relevance of this universal grid cell orientation. We show that spatial information provided by grid cells is maximized at the experimentally observed grid orientation within a broad parameter range. This relationship occurs only in the presence of noise. The model allows for several experimentally testable predictions including the number of grid cells. experiments revealed two additional striking features: First, grid cells can be assigned to 4-5 discrete modules that are each defined by a common spatial scale and orientation but different phases [3,15] ( Fig 1B). Second, the spatial scale increases geometrically across modules [15], with a fixed geometrical ratio of around 1.4-1.7. Several computational models have been developed to explain the firing patterns of grid cells. Attractor network and oscillatory interference models were proposed to explain the hexagonal geometry [16][17], and models considering the maximization of spatial information were introduced to understand the geometrical ratio of spatial scales [18][19][20]. These studies demonstrated that spatial locations can be decoded from the population activity of grid cells from each modules, and that the hexagonal firing patterns and geometric scale ratios maximize spatial information content. Overall, these results offer a possible explanation for the evolutionary significance of the functional properties of grid cells. Although the hexagonal geometry and the modular structure generated by grid cells have been reasonably well understood, an equally fundamental issue that remains elusive is the orientation of the hexagons. Specifically, are the hexagonal grids anchored to an external reference frame -and if so, which mechanisms account for anchoring, and which purpose does it serve? Intuitively, since the hexagonal scale can vary from module to module and may adapt to the environment [21][22], one might speculate that the orientation of different modules would be different. However, recent experiments revealed the striking phenomenon that, in a square environment with closed boundaries, a specific orientation of the hexagonal firing grids emerges [23][24][25], which is universal across different grid cells, modules and subjects. The common orientation anchored to the square borders may be associated with evolutionary significance and stem from natural selection. In fact, in three independent experiments, the measured orientation angles were found to be quite close: they are clustered at 7.4 degrees [23], 8.8 degrees [24] and 8.2 degrees [25], respectively. This specific orientation is accompanied by a distortion of the hexagonal grid (e.g., comparing Fig 1C with Fig 1D), which was attributed to shearing forces resulting from an interaction with the domain borders [23]. Yet, an explicit explanation of the biological significance of the specific orientation is lacking, and the mechanism underlying the universal orientation at around 8 degrees is unknown [26][27]. Addressing these questions will likely yield important insights into how grid cells interact with border cells in order to adapt to the environment [28][29], and more generally will advance our understanding of the function of grid cells for spatial coding [30].
In this paper, we propose a spatial coding scheme based on the population activities of grid cells pertaining to different modules, and hypothesize that the universal and stable orientation at 8 degrees maximizes the amount of spatial information conveyed by grid cells. Combining an encoding model, theoretical analyses and a reinforcement learning model, we provide converging evidence that an orientation of 8 degrees maximizes the amount of spatial information conveyed by grid cells in the presence of noise. Additionally, our computer model allows for empirical testable predictions on the minimal number of grid cells required for an optimal encoding scheme.

Results
The hexagonal firing pattern of a single grid cell, as exemplified by each of the seven dark green circles in Fig 1B, can be defined by the orientation, the scale, and the spatial phase of the firing field.
Experiments with rats revealed that grid cells are organized into 4 or 5 functional modules, within which the firing fields of the cells share the same scale and orientation but differ in spatial phases, as shown in Fig 1B. The scale increases geometrically between the discrete modules. In general, a single module is insufficient to provide unambiguous information about spatial locations but the population activities of grid cells from all modules can represent spatial locations at a high resolution [20].
Previous models suggested that, in a circular environment, the hexagonal array and geometric progression of scale is optimal with respect to efficient spatial coding [19]. In contrast, the underlying mechanisms and functional roles of a grid cell's orientation has not yet been fully understood.
Interestingly, recent experiments revealed a universal orientation of the firing fields of about 8 degrees with respect to the borders of a square environment [23][24][25]. In addition, displacement of borders induce changes in orientation, and these are accompanied by distortions of the hexagonal firing grid.
Both effects can be conceived of as resulting from shearing forces from the borders [23]. In order to understand the role of grid orientation quantitatively, we develop a spatial encoding model without any adjustable parameters.
We hypothesize that the universal orientation optimize spatial encoding. In particular, we note that the ubiquity of grid cells in mammals [31][32][33][34] suggests certain evolutionary advantages of their properties. Likewise, the specific orientation with respect to borders may also be evolutionarily significant for optimizing the information content of spatial representation. To reveal quantitatively how the population activities of grid cells across different modules support behavior in the positioning and self-localization tasks, we study spatial encoding via population activities.

Spatial representation of the grid cells system
The spatial representation via grid cells is modeled as follows. Assume there are 4 modules with spatial scales , for k =1,2,3,4, and a fixed geometrical ratio r of the scales between two adjacent modules: ( ) . Each module consists of a regular, topographically ordered population of M cells [20,35]. The firing fields of M cells tile periodically in an environment. Since we still lack exact experimental evidence for the number of cells M, we will study the effect of varying M on the spatial information and the orientation of grid patterns. For an arbitrary grid cell i from a module k , let be the smallest angle between one of the six symmetric axes of the hexagonal firing pattern and four borders. The ideal spatial firing rate at an arbitrary location x can be conveniently ℱ( , , , ) represented by a bell-shaped, spatially periodic function on a hexagonal lattice [20] (see Methods). The spatial distribution of is illustrated in Fig 1C. This establishes a mathematical description of the ℱ spatial firing fields of all grid cells and allows investigating how spatial locations in an environment are encoded based on their firing patterns. Because the firing fields of a single cell repeat across environments, the activity of that cell does not provide an unambiguous representation of self-location.
However, the conjunction of cells across 4 modules can be unambiguous [20].
Because of several intrinsic sources of neuronal noise, such as stochastic channel openings, thermal noise and noise due to synaptic background activity [36][37], the firing intensity is not ( , , , ) deterministic but stochastic. We assume spiking to follow a Poisson distribution with expected value [14,[18][19][20] (see Methods). Because stochastic fluctuations of time courses that follow a Poisson ℱ distribution cancel out with increaseing durations, for a longer time interval the signal to noise ratio (SNR) will be larger. We use the maximum value of firing intensity to measure the strength of max the signal and the standard deviation to quantify noise. Thus, the effect of noise can be max evaluated by the ratio of noise to signal, i.e., the inverse of SNR: . Since is unchanged, 1 SNR ≡ max max max the effect of noise is reduced as increases. Below, we will use 1/SNR to characterize the effect of noise and its inverse correlation with . The larger the value of 1/SNR, the stronger the effect of noise.

Spatial encoding and evaluation
Our proposed spatial encoding scheme based on the population activities of grid cells across different modules includes intra-module encoding and inter-module encoding, where the former is associated with the assumption that grid cells in the same module compete to represent spatial locations, and the latter stems from the assumption that cells in different modules complement each other for spatial representation. Although our hypothesis lacks immediate and strong experimental supports, some evidence in the literature indicates this simple encoding scheme is consistent with the empirical properties of grid cells. Specifically, the intra-module encoding scheme implements a "winner-take-all" (WTA) rule: a location is represented and encoded by the cell with the highest firing intensity I at the location. WTA rules are typically implemented by lateral inhibition [38] consistent with the experimentally observed coupling of grid cells via inhibitory interneurons [39] that constitute the essential element of attractor models of hexagonal firing patterns of grid cells [40]. The WTA rule can be formulated as: (2) ( , ) = * ≡ argmax each grid cell has a set of periodically repeated dominant sites with a hexagonal shape in a module, and the center of each dominant site coincides with that of the original firing fields of the grid cell (Fig 2A).
At any given location, there are four elements that constitute a vector for encoding this 1 ,⋯, 4 location, as illustrated in Fig 2B. In general, it can be speculated that more cells and modules enable a more precise encoding of locations in terms of less duplication of vector , such that each location can be more explicitly distinguished from others based on the coding scheme. However, regarding the biological cost of creating neurons and supporting their metabolism, there must be a tradeoff between the number of grid cells and the efficiency of spatial encoding. To evaluate the capacity of this spatial encoding scheme in a closed environment, we resort to the concept of information entropy. Let us consider two arbitrary spatial vectors and . If ( ) ( ) ( , locations and will be considered the same and cannot be differentiated. In a space, ) = ( ) a higher number of indistinguishable locations is associated with a poorer spatial representation and a lower encoding capacity. Information entropy is a parsimonious metric for accessing encoding capacity in terms of measuring the amount of effective information. Note that spatial information encoded from different modules is considered to be independent, so that the different modules complement each other. Thus, we calculate information entropy of each module separately.
Moreover, we divide the square domain along borders into a number of belt regions of unit width, as shown in Fig 2C. We obtain two groups of belt regions, parallel to the two groups of borders of the square domain, respectively. Our hypothesis of the division is that a two-dimensional domain can be reduced to two orthogonal directions (one dimension each) by simply anchoring to borders. The encoding of two dimensions can complement each other to represent the whole space. In addition, border cells may help grid cells encode locations close to borders. Taken together, if a spatial encoding scheme generated by grid cells is able to identify locations within each belt, all locations in the domain can be explicitly distinguished and an efficient spatial encoding is established.
The capacity of spatial encoding within a belt relies on the spatial distribution of sites ( , ) encoded by grid cells. Let us consider two extreme scenarios. On the one hand, if within a belt, every location is represented (encoded) by the same cell, i.e., there is a single value of u, the spatial representation is maximally poor and any two locations cannot be distinguished. In this extreme scenario, the capacity of spatial encoding reaches the lower bound according to information theory. On the other hand, if every location is uniquely encoded by a cell (the value of u is completely different across all different locations), the capacity of encoding locations will be maximal. The encoding capacity in the scenarios can be effectively measured by information entropy within a belt. To be concrete, we denote two groups of belts by index and , respectively, as shown in Fig 2C. Here each cell is akin to a random variable and its frequency of appearance in a belt corresponds to the probability of a random variable, denoted by a conditional probability of cell from module in a given ( | , , ) belt : where is the Dirac function with if and otherwise, is the index The spatial information entropy of a belt is defined as: where is the number of belts along each direction. Finally, the information entropy of a domain is the average over 4 modules: which depends on the orientation .
We are able to explore how is affected by orientation , taking into account different internal ( ) and external conditions, such as the number of cells in each module, geometric ratio , effect of noise , elliptic distortion of hexagonal grid, and size of the square environment. Because the 1/SNR hexagonal grid has a rotational symmetry, the orientation, defined as the minimal angle of the grid from any border of a square space, can be conveniently confined to the range [ , ] degrees [23]. 0 15

Numerical results of optimal orientation angle
Our first goal is to numerically identify the optimal orientation angle that maximizes information opt entropy with respect to different internal and external conditions.  cell numbers between about 50 and 170 per module -is substantially smaller than previous speculations about the numbers of grid cells, which assume about 70,000 cells in layer II of EC [42] of which are about 10% grid cells [41], resulting in 7,000 grid cells across 5-7 modules [14,19]. This suggests that either the number of grid cells is smaller than previously estaimated, or that some grid cells have other functions than the optimizaton of spatial information.
Next, we considered the effect of noise embedded in the firing intensity on . Fig  In addition to noise, we incorporated elliptic distortion to fully capture real hexagonal grids as (Results for other two typical values of distortion intensity can be seen in S2 Fig.) Next we study the effect of noise on optimal orientation . Since is insensitive to , we 1/SNR opt ignore and take into account both noise effect and cell numbers M. the larger Hamming distance, the lower ambiguity. There is an intrinsic correlation between information entropy and the Hamming distance, since both of them evalutate spatial ambiguity. In general, the higher information entropy, the larger the average Hamming distance among locations.
However, the Hamming distance is not linearly proportional to information entropy. The Hamming distance yields the same result of optimal orientation as that based on the information entropy, providing additional evidence to validate our encoding scheme and the effect of noise on optimal orientation (see S4 Fig).

Correlation between information entropy and the number of cells engaged in encoding
The purpose of studying the relationship between and the number of cells engaged in encoding a c belt should eventually enable for an analytical assessesment of . Note that is equivalent to opt c random variables in the standard definition of information entropy. According to Shannon's theory, the occurrence probability of a variable is inversely correlated with the amount of effective information encoded by this variable. In other words, a higher number of random variables usually results in a higher information entropy. This is because in general, adding more variables decreases the average probability of each variable, such that their encoding capacity is enhanced. Thus, we speculate that of c opt

Geometrical analysis of optimal orientation in a noise-and distortion-free situation
We develop a heuristic analysis to theoretically predict for M in the opt = 8 degrees, ∈ [7 2 ,13 2 ] absence of noise and elliptic distortion. Our theoretical approach is basically a geometrical analysis relying on the strong correlation between and and the strong stability of among different belts, c where the latter allows us to arbitrarily select two belts from and directions that represent an entire two-dimensional square. For simplicity, we use two orthogonal and sufficiently long lines without widths to stand for the two belts, as shown in Fig 4A. Each line will traverse a number of hexagonal sites encoded by grid cells. Note that the number of cells engaged in encoding along a line can be c estimated by the mean repeating distance between two nearest locations encoded by a given cell.
Specifically, every location within the repeating distance along a line will be encoded by a number of different grid cells. Thus, a longer repeating distance requires a larger number of grid cells to encode c every location within the repeating distance. However, the spatial distribution of along a line for a given is rather complex, such that it is difficult to analytically assess the mean repeating distance.
Therefore, we reduce the mean repeating distance to the minimum repeating distance among all encoded sites along a line to approximate . Our analytical results validate this approximation. Let us c denote the minimum repeating distance of all hexagonal sites along the two lines by and ( ) ( ) for an arbitrary . This yields the relation We numerically found high correlations between E and , and between and ( ) + ( )  between the centers of two neighbouring encoded sites is able to ensure prediction accuracy (see Eq. we have to provide a new geometrical approach to dealing with noise.

Geometrical analysis of optimal orientation in the presence of noise
We introduce a heuristic geometrical counterpart of noise to convert the effect of noise into a geometrical variable. In contrast to Fig 4A with an infinite space, here we consider a two-dimensional closed domain that can be mapped into a torus, as shown in Fig 5A [ the local minima at degrees and 7 degrees. The analytical and simulation results provide strong = 11 evidence that an optimal orientation at 8 degrees is associated with maximum information entropy in the presence of noise.

Reinforcement learning model for the evolution of orientation
The results described so far demonstrate the benefit of an orientation at 8 degrees for maximizing spatial information content in the presence of noise both numerically and analytically. However, how hexagonal grids gradually evolve towards this optimal orientation remains an outstanding question. We address this issue by a reinforcement learning model that is trained using experimentally recorded trajectories of rats (data:http://www.ntnu.edu/kavli/research/grid-cell-data). Broadly, reinforcement learning is a type of machine learning, which uses experience gained through interacting with the environment to evaluate feedback and improve decision making. We hypothesize that rats implement a spatial encoding scheme during exploration of a new environment and optimize their spatial representation for a higher chance of survival. More specifically, we assume that a rat is able to evaluate its orientation based on past experience and modify to maximize spatial information content.
The essential ingredients of our reinforcement learning model are an adopted strategy set, in which the evolutionary fitness of each strategy is evaluated based on experience. To be concrete, we assume that the strategy set consists of all possible orientation angles , , , in the range [ , ] degrees with a small interval between two adjacent strategies. As shown in Fig 6A, we assign the same initial fitness to all strategies. We assume that a rat explores the environment using the same strategy ( ,0) in a round for a certain amount of time. In an arbitrary round, say , the animal chooses a current strategy from the strategy set according to the selection probability determined by the fitness Θ( ) = of strategy at time (see Eq. (26) in Methods for the definition of selection probability). After -1 Θ( in round is decided, the rat's grid cells create hexagonal grids, encode spatial locations and ) = evaluate spatial representation with respect to its moving trajectory in round . In particular, it is reasonable to assume that only locations pertaining to trajectories are encoded and used to calculate information entropy in the current round (see Eq. (28) in Methods for the definition of ).

( ) ( )
Subsequently, the fitness of strategy used in round is updated according to Eq. (27), in ( , ) which associated with in all rounds prior to are used. In the next round , the rat repeats ( , ) + 1 this process. After a number of rounds, we obtain a distribution of the frequency of strategies (see Methods for more details). observations. Meanwhile, the local maxima at degrees is also supported by some previous = 15 experimental evidence [15]. We speculate that the experimental observation of degrees may be Due to finite evolutionary time assumed in our reinforcement learning model, the strategy pool still consists of a rich variety of strategies in spite of the advantage of degrees. In principle, a = 8 strategy with evolutionary advantages possesses higher fitness, and the "strong gets stronger" effect during the process reinforces the strategy. If the evolutionary time is sufficiently long, the optimal strategy degrees will prevail eventually and dominate the strategy set, and all the other strategies = 8 will be eliminated. To be consistent with experiments, our results pertain to temporal stages (see Methods for more details) rather than the thermodynamic limit. As a result, a distribution of strategies with both the global and a local maxima are obtained.

Discussion
Converging experimental evidence has revealed that the main axes of grid cells are anchored to environmental borders at an offset of about 8 degrees. Here, we aimed at understanding the possible functional roles (i.e., the evolutionary advantage) and the physiological mechanisms underlying this phenomenon.
By combining a computer simulation of the spatial coding properties of the grid cell network with analytical considerations and a reinforcement learning model, we found evidence that a grid axis offset of 8 degrees maximizes the spatial information content specifically in thre presence of noise.
We first implemented a computer simulation of the grid cell network and analyzed the influence of grid orientation on the grid cell network's spatial coding properties, depending on various factors such as grid cell numbers, the effect of noise, the spatial ratio of module scales, and elliptic distortions.
Unexpectedly, we found that a grid orientation of 8 degrees maximizes spatial information content across a wide range of conditions specifically in the presence of noise. In a noise-free situation, an orientation at 8 degrees is only optimal within a narrow and putatively unrealistic range of the numbers of grid cells, beause the maximum number in that range is smaller than the putative number of grid cells based on experiments [15]. By contrast, in the presence of noise, an orientation of 8 degrees maximizes spatial information as long as the number of grid cells exceeds some lower bound. This lower bound is small, insensitive to noise, and largely independent of other internal and external conditions. These results demonstrate that noise plays a significant role in driving optimal orientation towards 8 degrees and mammals optimize spatial coding in the presence of noise. Although noise has been known as a driving force of many biological phenomena [43], e.g. genetic mutation, the effect of noise on shaping the hexagonal firing pattern of grid cells has not be revealed before.
To unveil the mechanisms underlying the noise-induced universal orientation, we provide two theoretical analyses for the ideal and noisy situations. A property of information entropy is exploited to enable analytical predictions of the optimal orientation, i.e., the strong correlation between information entropy and the number of grid cells. For the ideal situation, the number of encoded grid cells along a line is estimated by the minimum repeating distance between two nearest sites encoded by the same cell. For the realistic situation with noise and elliptic distortion, the effect of noise is akin to transforming a line to a belt with a certain width. The analytical predictions are in good agreement with simulation results across a wide range of grid cell numbers and noise effects.
Finally, we modeled the temporal evolution of grid cell orientations towards the optimal angle via a reinforcement learning model based on experimentally observed behavioral trajectories of rats. During the time-course of reinforcement learning, the model evaluates the fitness (i.e., information entropy along trajectories) depending on various strategies that are defined by sets of orientation angles. If an orientation angle leads to a higher information entropy prior to a moment, the angle will be adopted with a higher probability at that moment. As the process continues, all possible strategies (angles) will be employed, but the strategies with higher fitness will be used more frequently. Our results demonstrate that for both ideal and realistic situations, the likelihood of strategies exhibits a global maximum at 8 degrees and a second maximum at about 15 degrees, in line with experimental evidence [15,[23][24].
The main hypothesis of our coding scheme -i.e., that the orientation of grid patterns maximizes spatial information -has been partially validated by a very recent experiment in humans [44]. In the experiment, two axes with different amounts of information are arranged in a circular environment. It is found that grid cell systems reduce the uncertainty of representation by aligning grid patterns to the axis of greatest information. The concept of uncertainty reduction is analogous to the maximization of spatial information in our coding scheme, despite the very different experimental settings. The orientation aligned to the axis of greatest information, suggesting a similar effect as for the optimal orientation at 8 degrees in a square environment. Taken together, these recent experimental results substantiate the idea that grid orientation serves to maximize spatial information.
It is worth noting that our encoding model can not only explain the functional benefit of the empirically observed grid orientation but also generates experimentally testable predictions on the lower bound of cell numbers in each module. According to the results in Fig 3K,  for inducing an optimal orientation at 8 degrees can be predicted to be 16 2 in each module. It is noteworthy that the actual number of grid cells may be larger than this prediction. In fact, the result predicts that under a certain amount of noise, the number of grid cells should be larger than a critical value to ensure an optimal encoding at an orientation of 8 degrees. If the amount of noise is increased by reducing , in order to achieve an valid coding, more grid cells are needed. This is explicitly reflected by the increasing trend in Fig 3K as increases. Since the exact number of grid cells currently cannot be empirically determined, the model's predictions have implications in constraining future modeling approaches.
Besides square environments, our coding model can be easily applied to other environments with regular shapes. For example, in a rectangular environment, we vary the ratio of the longer edge to the lower edge, and compute optimal orientation angles under different ratios. Due to the destruction of symmetry in a rectangle compared to a square, the range of becomes degrees. Our results [0, 30] show that spatial information is maximized at either degrees or degrees, depending on = 8 = 22 both the effect of noise and the cell numbers of grid cells in each module (see S10A-B Fig).

1/SNR
The range of the lower bound of the cell numbers M is quite similar to that in square environments. It has been argued that grid cells play a crucial role for path integration, a navigation strategy that is based on an idiothetic reference frame [8][9][10][45][46]. As a consequence, an orientation of grid patterns that maximizes spatial information may benefit path integration. Specifically, during path integration, several types of information are integrated, such as spatial locations, speed, and directions. In addition to the putative role of place cells, speed cells and head directional cells for path integration, grid cells can provide necessary information as well. On the one hand, grid cells in MEC may input lowdimensional information into different high-dimensional cells, e.g., place, speed and head directional cells, to implement path integration. On the other hand, during path integration, errors from noise will be aggregated in the absence of salient landmarks or boundaries, and place fields will drift [29]. The grid cell systems may help reduce the errors because the low-dimensional cognitive map created by grid cells is relatively stable across different contexts and environments. Note that to precisely encode either distance [35] or speed during path integration, an accurate self-localization is imperative. The orientation of hexagonal grid patterns maximizes spatial information and discriminates different locations in an optimal manner, so that the tracking of distance and speed is favored by this orientation as well may improve path integration.
Many open issues remain for a more general understanding of the dynamics and functions of grid cells during spatial navigation. For example, how does the activity of place cells in the hippocampus relate to the orientation of grid cells [47][48][49][50]? How do grid cells modulate the coding scheme for two separate spaces when the two spaces are being merged into a single space [51][52]? Our modeling approach consisting of a spatial encoding scheme, a method to characterize encoding capacity, geometrical analyses for both ideal and realistic situations, and a reinforcement learning model offers tools for addressing these issues to unravel the mysteries of the navigation ability of the mammalian brain.

Spatial firing rate of grid cells
The idealized firing rate of cell in module with orientation at an arbitrary location ( ) is ℱ( , , , )

Elliptic distortion
Suppose that the firing field centers of a grid cell lie on an ellipse with a semi-major axis and semi-1 minor axis with (see the yellow axis and black axis in the ellipse in Fig 3K). To model

Geometrical analysis in the ideal situation
The value of can be estimated by the length of the tangent at associated with a max( + ) o specific plateau (see Fig 4E). Note that the geometry of tangent points has negligible effect on max( because of the plateaus. We thus only calculate the interval between the centers of two nearest + ) hexagonal sites encoded by the same cell pertaining to the tangent (Fig 4E). In general, and of plateau and can be captured by virtue of the two fundamental directions ( and ) of the grid (see ⊥ ∥ To obtain the value of determined by the tangent angle, the geometrical features of the tangent opt points can increase the prediction accuracy (see Fig 4E). We formulate all the transition points. There are two kinds of transitions: from a lower plateau to an upper one and vice versa. For the former class, i.e., (or ), the geometrical relation in Fig 4E gives The quantities and can be obtained as and , respectively.
The value of can be obtained from the tangent of two nearest sites encoded by the same cell. In  Fig 4A) is a beam of light along degrees in the 30rhombus unit. Below we focus on the analysis associated with .
Thus, the interval of two nearest beams is Then the standard deviation of along direction can be calculated via of the other groups of beams along can be analytically obtained in the same manner. The 2 30 ∘average standard deviation over two groups of beams is The analytical result of is shown in Fig 5E.

Reinforcement learning
In an arbitrary round, say , a rat chooses a current strategy from the strategy pool according to Θ( ) the probability determined by its fitness values prior to through a softmax distribution: where factor with ensures asymptotic stability of the evolutionary process. The ( ) = = 0.15 fitness of strategy in round is calculated using all the rounds prior to round : where is the Dirac function with if and otherwise, is the spatial information entropy associated with strategy in round . Here the definition of slightly differs from that in the static encoding scenario. Specifically, only locations pertaining to the trajectories of rat are used to evaluate the spatial encoding in a round. We build a database composed of 40 trajectories of rats, each of which is recorded within 10 minutes (data from http://www.ntnu.edu/kavli/research/grid-cell-data). In an arbitrary round , we randomly choose a S1 Methods. Hamming distance. To provide additional evidence to validate our encoding scheme, we propose a measurement based on Hamming distance for encoding capacity as an alternative to information entropy. Note that a basic requirement for any positioning task lies in the ability to distinguish vectors u(x) after the grid cell system maps the environment using orientation θ.