Sense organ control in moths to moles is a gamble on information through motion

Sensory organs—be they independently movable like eyes or requiring whole body movement as in the case of electroreceptors—are actively manipulated throughout stimulus-driven behaviors. While multiple theories for these movements exist, such as infotaxis, in those cases where they are sufficiently detailed to predict sensory organ trajectories, they show poor fit to measured trajectories. Here we present evidence that during tracking, these trajectories are predicted by energy-constrained proportional betting, where the probability of moving a sense organ to a location is proportional to an estimate of how informative that location will be combined with its energetic cost. Energy-constrained proportional betting trajectories show good agreement with measured trajectories of four species engaged in visual, olfactory, and electrosensory tracking tasks. Our approach combines information-theoretic approaches in sensory neuroscience with analyses of the energetics of movement. It can predict sense organ movements in animals and prescribe them in robotic tracking devices.


23
Sensory organs undergo small lateral movements as they near or hold station close to a target  Stockl et al., 2017). For example,30 in the related case of signal-emitter organ control, fruit bats are known to oscillate their tongue-31 click-based sonar signals on approach to their targets (Yovel et al., 2010). This can be effective 32 because for many signal sources, the signal intensity peaks at the target's location and tapers away 33 in all directions. The expected amount of information-in the bat study quantified by the Fisher 34 Information of the emitted sonar signal-is highest at the maximum slope of the signal profile 35 because at those locations, small variations in the emitter position leads to large changes in emitted 36 signal power on target and thus also in the returning echos. In contrast, at the flat peak of the 37 profile where the object is located, small variations in emitter position lead to small or no change 38 in signal and returning echo; the expected information is therefore low. For active sensing animals 39 like bats, dolphins, and electric fish, placing emitter organs so that the target is at a location of high 40 (Fig. 2B). As we will show, however, infotactic trajectories show poor fit to what animals do during 48 tracking behavior. 49 Here we propose that sensory organs (or emitters in the case of active sensing animals) are 50 moved according to a very different principle: rather than move the sensor to maximize infor-51 mation, move it to sample spatially distributed signals proportionate to the expected information 52 density (EID), constrained by the energetic cost of movement. The underlying sensory sampling 53 strategy gambles on the chance of obtaining more information at a given location through care-54 fully controlled sensor motion that balances two factors that typically push in opposite directions: 55 1) proportionally bet on the expected information gain; and 2) minimize the energy expended for 56 motion.

57
To understand proportional betting on information, consider a distribution of expected infor-58 mation for an animal tracking an object on the ground, visualized as a color map in Fig trajectory will be executed with finite speed, varying from slow in information-rich regions, to fast 68 in information-sparse regions. For the fruit bat behavior discussed above, rather than the bat's 69 oscillatory movement between two information peaks of the sonar beam (their Fig. S5 and Fig. 2A, 70 Yovel et al. (2010)) being an unexplained "bug" of information maximization, it is an expected fea-71 ture of proportional betting; similar oscillations will be seen throughout the subsequent measured 72 and simulated target tracking behaviors of this study. 73 For this study we have quantified the expected informativeness of sensing locations by how 74 much an observation at a location would reduce the Shannon-Weaver entropy (hereafter entropy) 75 of the current estimate of the target's location, as in infotaxis (Vergassola et al., 2007). (Other 76 measures of information such as Fisher Information can be used with near identical results (Miller 77 et al., 2016). In our approach, the closeness of a given trajectory to perfect proportional betting is 78 quantified by the ergodic metric. The ergodic metric provides a way of comparing a trajectory to 79 a distribution (i.e., the EID) by asking whether a trajectory over some time interval has the same 80 spatial statistics as a given distribution (Methods, Appendix 3).Comparing a trajectory to a distribu-81 tion is a novel capability of the ergodic metric (Mathew and Mezic, 2011) not shared by common 82 methods of comparing two probability distributions (Appendix 2). Through extremizing ergodicity 83 we obtain trajectories that bet on information, expending a metabolic cost to move to informative 84 locations in the space. With a perfectly ergodic trajectory (one with an ergodic measure of zero, 85 only possible with infinite time and when the energy of movement is not considered), the distribu-86 tion of expected information is perfectly encoded by the trajectory, or, equivalently, the trajectory 87 does perfect proportional betting on the EID. We therefore call our approach ergodic information 88 harvesting (hereafter EIH). Here we show siphon casting behavior in the marine snail (body) (Ferner and Weissburg, 2005), cross-current swimming in the Chambered nautilus (body) (Basil et al., 2000), whole-body oscillations in electric fish (body position) (Stamper et al., 2012), zigzagging motions in the rat (nose) (Khan et al., 2012), back and forth searching in the mole (nose) (Catania, 2013), fixational eye movements in humans (eye) (Rucci and Victor, 2015), the zigzagging walk of the cockroach (head) (Willis and Avondet, 2005), and flower tracking motions in the moth (head) (Stockl et al., 2017). These sensing-related movements occur with striking consistency across animals using different sensory modalities and within different physical environments. We propose these movements arise as a form of gambling on information through motion, and show evidence in support of this claim from the four species highlighted (electric fish, mole, cockroach, and moth). erally in the wind in order to feed from the flower's nectary (Stockl et al., 2017). A key behavioral 92 signature of EIH-increased sensor wiggling as the signal weakens-is evident in this illustration. 93 In addition to proportional betting being used at the cognitive decision-making level in primates 94 (Monosov et al., 2015;Gottlieb et al., 2014), we suggest that it occurs more broadly as a sensori-95 motor routine across a wide phylogenetic bracket. Below we will show evidence for this claim by 96 comparing measured tracking trajectories to those simulated with EIH, when EIH is provided the 97 original target trajectory for two species of insect, a fish, and a mammal. (A-B) Illustration of an animal localizing a target on the ground by moving its sense organs (e.g., eye, nose, antenna, whole body etc.) along a simulated proportional betting trajectory from Miller et al. (Miller et al., 2016). (A) Trajectory generated with proportional betting on the expected information density (EID, pink-red heatmap). Unknown to the animal, the real target is near the white star on the left which marks the true location of peak information density, while on the right is a distractor (at center of red zone). A distractor in the EID could be caused by either a physically present object sensed by the animal or, in the context of this paper, unmodeled uncertainties such as false-positive sensory input. Each step of our energy-constrained proportional betting algorithm is taken so as to optimally balance two factors, one representing how much that step increases the similarity of the trajectory to one that does perfect proportional betting (where the percentage of time the trajectory spends in any subset  from = 0 to = is equal to the measure of EID within  ), and one representing the energy cost (Methods). (B) Same as A, but with the entropy minimization strategy. Unlike energy-constrained proportional betting, entropy minimization locally minimizes the entropy (maximizes the EID) of the posterior belief about the target location, in this case leading it to the nearby distractor by following the gradient. (C-E) An illustration of an animal tracking an object constrained to move in a line, in this case a hypothetical moth following a flower swaying in a breeze in a manner approximated by a 1-D sinusoid-a natural behavior reported in (Sponberg et al., 2015;Stockl et al., 2017). We simulate the tracking of the flower using energy-constrained proportional betting. (C) In the top panel, we show an idealization of the moth's belief (blue line Gaussian distribution above the moth) about the flower's location when the flower reaches the center point. Higher values in the direction represents higher confidence of the target at the given location. The corresponding EID is overlaid in magenta; a darker color indicates higher expected information should the animal take a new sensory measurement in the corresponding location. Note the bimodal structure of the EID, with identical maximal information peaks on both sides of the Gaussian belief. (D) Simulation of the moth's position (red curve) while tracking a moving target (black curve) under a weak signal condition (Methods). The corresponding EID is overlaid in magenta. Note that even though trajectory segments are planned at 14 fixed-time intervals (Methods) over the shown duration based on the EID at start of those intervals, the EID is here plotted continuously for visualization purposes only. Energy-constrained proportional betting results in persistent activation of movement when the EID is relatively diffuse due to lack of information. Note the presence of a distractor (marked by the black arrow) in the EID due to higher uncertainty in the sensory input as a result of the weaker signal. EIH responded to the distractor by making a detour away from the actual target position to gamble on the chance of acquiring more information, but does not get trapped by the distractor as in B because of the proportional betting strategy. (E) Same as (D), but under strong signal conditions. Now the energy-constrained proportional betting trajectory samples both peaks of the bimodal EID with excursions away from these peaks, similar to measured moth behavior (Stockl et al., 2017) (see Fig. 3G).

99
Using one dataset we collected (electric fish), and three other previously published datasets, we 100 performed side-by-side comparisons between one-dimensional trajectories generated by EIH and 101 those measured from live animals during searching and tracking phases of movement in the pres-102 ence of a target. Live animal data were either collected as one-dimensional trajectories, or collected 103 as two-dimensional trajectories and projected to one dimension (Methods  Figure 3. Representative trajectories of animals tracking a target compared to EIH, and relative exploration across all trials. Three representative live animal trajectories above trajectories generated by the EIH algorithm, length cropped for visual clarity. The moth data is not shown here due to the complexity of the prescribed target motion, but is shown in a subsequent figure. All EIH simulations were conducted with the same target path as present in the live animal data, using a signal level corresponding to the weak or strong signal categories (Methods). The EID is in magenta. Relative exploration across weak and strong signal trials (dots) shown in the right-most column (solid line: mean, fill is 95% confidence intervals). (A) The electric fish's fore-aft position was measured as it tracked a sinusoidally moving refuge. EIH (yellow) shows good agreement with behavioral data as both have significantly higher relative exploration when signal is weak (Kruskal-Wallis test, < 0.001, = 21 for experimental data, and < 0.001, = 18 for simulation). Infotaxis (purple), in contrast, leads to hugging the edge of the EID, resulting in smooth pursuit behavior as indicated by the near 1x relative exploration. (B) The experimental setup and data for the mole was extracted from a prior study (Catania, 2013). During the mole's approach to a stationary odor source, its lateral position with respect to the reference vector (from the origin to the target) was measured. Relative exploration was significantly higher under the weak signal conditions (Kruskal-Wallis test, < 0.012, = 17). In the second row, raw exploration data (lateral distance traveled in normalized simulation workspace units) are shown to allow comparison to simulation, as these are done in 1D (Methods). EIH shows good agreement with significantly increased exploration for weak signal (Kruskal-Wallis test, < 0.001, = 18), while infotaxis leads to cessation of movement. (C) The experimental setup and data for the cockroach was extracted from a prior study (Lockey and Willis, 2015). The cockroach head's lateral position was tracked and total travel distance was measured during the odor source localization task. Relative exploration is significantly higher for the weak signal condition (Kruskal-Wallis test, < 0.002, = 51). In the second row, we show that EIH raw exploration (as defined above) agrees well with measurements as the amount of exploration increased significantly under weak signal conditions (Kruskal-Wallis test, < 0.001, = 18), while infotaxis leads to cessation of movement. Asterisks indicate the range of values for the Kruskal-Wallis test (* for < 0.05, ** for < 0.01, and *** for < 0.001).
ods). Trials under weak signal conditions show an increased zig-zag amplitude, which leads to a 153 significant increase in relative exploration in the weak signal condition as summarized in the plot

154
(Kruskal-Wallis test, < 0.002, = 51). For the moth robotic-flower tracking behavior, the flower 155 was moved with a sum-of-sine stimulus that cannot be visualized in the same manner as our first 156 three species. We discuss the analysis of these trials in a following section.
conditions. In these simulations, although the simulation has the target location provided, the er-160 godic information harvesting algorithm to track objects does not know this location and is only 161 given simulated measurements (Algorithm S1  Fig. 3 (Methods). Infotaxis leads to mostly smooth tracking trajectories with near unity relative 169 exploration when target is moving (Fig. 3A); there is no increase in exploration with reduction of 170 signal strength as in the original behavior and EIH simulations. When the target is stationary, info-171 taxis leads to cessation of movement, as shown in Fig. 3B-C. This change in the tracking response 172 is further analyzed in Fig. S3.

173
Increased exploratory movement and sensing-related high frequency movement.

174
To further characterize the sensing-related movement patterns and verify whether the increase in 175 exploration is mainly due to these movements, we performed a spectral analysis of the animal's  Sensing-related movements increase fish refuge tracking performance.

205
A crucial issue to address is whether the additional sensing-related motions found in weak signal 206 conditions in animals, and predicted by EIH, cause increased tracking performance. To answer 207 this question, we constructed a filter to selectively attenuate only the higher frequency motion  Fig. 3. (A-B) Representative Fourier spectra of the measured and modeled (EIH-generated) refuge tracking trajectories, with target trajectory (blue), strong signal condition (green), and weak signal condition (red). The frequencies above the frequency domain of the target's movement is shaded; components in the non-shaded region are excluded in the subsequent analysis. The Fourier magnitude is normalized. (C-D) Distribution of the mean Fourier magnitude within the sensing-related movement frequency band (gray shaded region) for strong and weak signal trials. Each dot represents a behavior trial or simulation. Significantly higher magnitude is found within the sensing-related movement band under weak signal conditions (Kruskal-Wallis test, < 0.001, = 21 for measured behavior, and < 0.001, = 18 for EIH). (E-F) Representative Fourier spectra of the measured and modeled head movement of a mole while searching for an odor source (lateral motion only, transverse to the line between origin and target), plotted as in A-B. Because the target is stationary in contrast to the fish data, the entire frequency spectrum was analyzed. (G-H) Mean Fourier magnitude distribution. In the weak signal condition, there is significantly more lateral movement power under weak signal in both the animal data and simulation (Kruskal-Wallis test, < 0.009, = 17 for behavior data, and < 0.001, = 18 for EIH simulations). (I-J) Fourier spectra of mole's lateral trajectory, plotted the same as in A-B. (K-L) Mean Fourier magnitude distribution. Significantly higher power is found under weak signal in the behavior data and simulations (Kruskal-Wallis test, < 0.003, = 51 for behavior data, and < 0.001, = 18 for EIH simulations). (M-N) Bode magnitude plot (Methods) of a moth tracking a robotic flower that moves in a sum-of-sine trajectory (figure reprinted from Stockl et al. (2017)) and the corresponding simulation using the same target trajectory. Each dot in the Bode plot indicates a decomposed frequency sample from the first 18 prime harmonic frequency components of the flower's sum-of-sine trajectory. A total of 23 trials ( = 13 for strong signal, and = 10 for weak signal) were used to establish the 95% confidence interval shown in the colored region. Note the increase in gain in the midrange frequency region (shaded in gray) between the strong signal and weak signal conditions. The confidence interval for the simulation Bode plot is established through 240 trials ( = 120 for strong signal, and = 120 for weak signal). The same midrange frequency regions (shaded in gray) are used for the subsequent analysis. (O-P) Mean midrange frequency tracking gain distribution for the moth behavior and simulation trials. Moth's exhibit a significant increase in midrange tracking gain for the weaker signal condition (  The line near 45% error shows relative tracking error for the original unfiltered trajectory (0 dB attenuation). The thickness of the horizontal bars represents the 95% confidence interval across the eight trials (individual dots) for each attenuation condition. For each attenuation levels, the individual trial dots are plotted with small horizontal offset to enhance clarity. The baseline tracking error with intact wiggle is marked by the dashed black line. (B) Distance from ergodicity (Methods) as a function of wiggle attenuation. Zero distance from ergodicity indicates the optimal trajectory that perfectly matches the statistics of the EID. As the distance increases from zero, the corresponding trajectory becomes less and and less related to the EID. The baseline data with intact wiggle is marked by the dashed black line. (C) Relative tracking error plotted against distance from ergodicity across wiggle attenuations. There is a clear positive correlation between tracking error and distance from ergodicity as wiggle is progressively attenuated.  Relative energy is the ratio of the mechanical energy needed to move the fish along the tracking trajectory divided by the energy to move the fish along the refuge trajectory. Weak signal conditions show a significantly higher relative energy as a result of the additional sensing-related movements (Kruskal-Wallis test, < 0.001, = 21). (B) Relative energy and relative tracking error for EIH simulations as sensing-related movements are progressively attenuated (data from Fig. 5A-C, weak signal condition). As the wiggles are attenuated, simulating investment of less mechanical effort for tracking, the relative energy decreases from 30x to less than 5x, but tracking error increases from 50% to around 75%. This plot also suggests a possible diminishing return in tracking error with additional energy expenditure past 30x. The lower bound near 4x is similar to the relative energy for the strong signal condition, and arises due to small disparaties from the perfect sinusoid that the refuge is following (the 1x path). Asterisks indicate the range of values for the Kruskal-Wallis test. components without affecting the baseline tracking motion (Methods). Simulated weakly electric 209 fish tracking trajectories in the weak signal condition-similar to that shown in the second row  We show the results in Fig. 5A in terms of relative tracking error, where 50% error means a de-216 parture from perfect tracking that is one-half the amplitude of the fore-aft sinusoidal motion. Rel-217 ative tracking error increases in proportion to the amount of sensing-related motion attenuation, 218 from ≈50% with no attenuation to ≈75% with the highest attenuation we used. We then evaluated 219 the distance from ergodicity, a dimensionless quantity that measures how well a given trajectory 220 matches the optimal proportional betting trajectory (Methods), for all the trajectories. We found 221 that an increase in attenuation also leads to monotonically increased distance from ergodicity. This of following such trajectories is lacking. EIH is a candidate framework that is sufficiently general to 256 invite application to a host of information-related movements observed in living organisms, while 257 sufficiently well-specified to generate testable quantitative predictions.

258
In the insects-to-mammals assemblage of animal species analyzed above, we observe gambling 259 on information through motion, where the magnitude of the gamble is indexed by the energy it 260 requires. EIH's approach of extremizing ergodicity generates trajectories that bet on information, 261 exchanging units of energy for the opportunity to obtain a measurement in a new high-value lo-262 cation. For both the measured tracking trials and their EIH simulated versions, a key change that 263 occurs as sensory signals weaken is an increase in the rate and amplitude of the excursions from 264 the mean trajectory, which we have quantified as an increase in relative exploration. We do not 265 presently have a complete mechanistic understanding of the cause of these motions in animals, 266 but we can interrogate how they arise within EIH. First, the increase in the size of these excursions 267 in weak signal conditions arises because the EID spreads out in these conditions due to high un-268 certainty (for example, wider magenta bands in Fig. 3). Since EIH samples proportionate to the 269 expected information, as information diffuses, the excursions to sample the more spread-out EID 270 will increase in size.

271
Second, in EIH the frequency of these excursions is related to how far ahead in time a trajectory 272 is generated for (variable in Algorithm S1). One can consider this analogous to how far ahead 273 an animal generates a trajectory for before a change in information can result in a change in their 274 trajectory. For example, tiger beetles see their prey, then execute a trajectory to the prey that is 275 completed regardless of subsequent motion of the prey (Gilbert, 1997). In EIH, over the course 276 of the generated trajectory epoch, changes in the expected information density due to new sen-277 sory observations similarly have no effect; these will only be incorporated for the next trajectory 278 segment.

279
In EIH, perfect ergodicity is approached through the trade-off between the ergodic metric and 280 control effort within the prescribed generated trajectory time horizon . As asymptotically ap-281 proaches infinite duration, the system will approach perfect ergodicity as the ergodic measure ap-282 proaches zero. Conversely, as asymptotically approaches zero time duration, the environment 283 will be minimally sampled and EIH executes the best trade-off between the cost of movement and 284 extremizing ergodicity in that minimal segment of time. Changing between these bounds will 285 affect the frequency of the sensing-related excursions. This is because the information landscape 286 is assumed to be static until has elapsed, the EID is updated, and a new segment of trajectory 287 is generated. For example, imagine a typical simulated one-dimensional trajectory that exhibits 288 wiggle motions. As the simulated animal moves to visit a region of dense expected information in 289 one direction, the unvisited locations in the other direction start to accumulate uncertainty in the 290 belief (at a rate proportional to the noise level), leading to an increase in the density of expected 291 information in those locations (Fig. S2). After the sensor finishes the current trajectory segment 292 of duration , it then moves in the opposite direction to explore the unvisited regions with a high 293 expected information density (Fig. S2). Thus, a shorter causes the sensor to react more quickly 294 in response to changes in the information landscape and hence to a higher wiggle frequency. The 295 initial (see Table S1) used for the behavior simulations was chosen to fit the frequency of sensing-296 related oscillations observed in the weakly electric fish refuge tracking data. The same value was 297 applied to mole and cockroach trials, and reduced by a factor of five for the moth data due to the 298 higher frequency content of the prescribed robotic flower movement.

299
As gambling on information through motion involves a trade-off between increasing how well a 300 trajectory approaches ideal sampling (ergodicity) and reducing energy expenditure, a useful quan-301 tity to examine is how tracking error changes with the energy expended on motion. To do so, we 302 estimated the mechanical energy needed to move the body of the electric fish along the weak and 303 strong signal trajectories, and found that weak signal trajectories required four times as much en-304 ergy to move the body along as strong signal trajectories. In simulation, we examined how tracking 305 error changes as more energy is invested in the sensing-related movements. This analysis shows 306 that the accuracy of tracking increases with the mechanical effort expended on the small whole-307 body oscillations that these fish make while tracking, with a 25% reduction of tracking error at 308 the highest level of energy expenditure compared to the low energy case where sensing related 309 movements are removed.

310
Comparison to information maximization. Information maximization and EIH emphasize 311 different factors in target tracking. First, if a scene is so noisy as to have illusory targets (more 312 than one peak in the probability distribution representing the estimated target location, called the 313 belief), or actually includes multiple objects, information maximization will not result in sensing 314 the information distributed across the scene, but rather move until a local information maxima is 315 reached (for example, the distractor in Fig. 2B) and stay at that location. With energy-constrained 316 proportional betting, information across a specified region of interest will be sampled in propor-317 tion to its expected magnitude ( Fig. 2A). This leads to sensing-related movements that may, at first 318 glance, seem poorly suited to the task: for example, if the distractor has higher information den-319 sity, as it does in Fig. 2A, then it will be sampled more often than the lower information density of 320 the true target-but what is important here is that the true target is sampled at all, enabling the 321 animal to avoid getting trapped in the local information maxima of the distractor. For information 322 maximization, if 1) there is only one target of interest; 2) the EID is normally distributed; and 3) the 323 signal is strong enough that false positives or other unmodeled uncertainties will not arise, then 324 information maximization will reduce the variance of the estimated location of the single target be-325 ing sought and direct movement toward the true target location. We interpret the poor agreement 326 of simulated information maximization trajectories with measured behavior in our results as indi-327 cating that the conjunction of these conditions rarely occur in the animal behaviors we examine.

328
The second area where these two approaches have different emphases is highlighted in cases 329 where noise is dominating sensory input in high uncertainty scenarios as is common in naturalistic 330 cases. Information maximization leads to a cessation of movement since no additional informa-331 tion is gained in moving from the current location (Fig. S1C, Fig. S3A). Energy-constrained propor-332 tional betting will result in a trajectory which covers the space (Fig. S1C, Fig. S3A): the expected 333 information is flat, and a trajectory matching those statistics is one sweeping over the majority of 334 the workspace at a density constrained by EIH's balancing of ergodicity with energy expenditure.

335
For information maximization, coverage can only be an accidental byproduct of motions driven by 336 information maximization.

337
Other interpretations of the behavioral findings. Yovel et al. (Yovel et al., 2010) suggested 338 that off-axis sensing behavior arises from moving to a peak in the EID (similar to Fig. 2B). This hy-339 pothesis is similar to infotaxis in terms of using information maximization, although using Fisher 340 Information rather than entropy minimization. Yovel et al. show in simulation that the informa-341 tion maximization strategy leads to a smooth tracking trajectory which hugs the edge of the signal 342 trail (at one of the two information peaks, Fig. 4 of (Yovel et al., 2010)). This is replicated by our 343 simulation of infotaxis as well (Fig. S1D). Figure  Bayesian filter and a process is specified to generate oscillatory motion around targets according 362 to the variance of the belief as a measure of uncertainty, then in the narrow context of a single 363 target with no distractors (neither real nor fictive due to high uncertainty), such an algorithm can 364 be tuned to behave similarly to EIH. However, in more realistic scenarios, there is no apparent 365 mechanism to address real or fictive distractors, a capability of EIH we elaborate on further below.

366
Further work is needed to test the differences between EIH and the gain adaptation hypothesis, or 367 to determine whether gain adaptation is an implementation of EIH in specific, biologically relevant 368 circumstances. similarities to the high pass filter hypothesis in that motion is to counter sensory adaptation, a high 386 pass filter-like phenomena. Although evidence for the perceptual fading hypothesis during track-387 ing behaviors is lacking, EIH shows good agreement with animal behavior without any mechanism 388 for sensory adaptation. Similar to the gain adaptation hypothesis, the high-pass filter hypothesis 389 is also missing key components for trajectory prediction. Nonetheless, when implemented with 390 the missing components, including a Bayesian filter and a feedback process that generates trajec-391 tories that match the desired spatial-temporal dynamics (Biswas et al., 2018), the high-pass filter 392 hypothesis does not conflict with EIH in single target cases with low uncertainty. This is because 393 EIH also predicts a preferred frequency band for wiggle movements that may match the preferred 394 spectral power of upstream neural processing. However, in the context of multiple target scenar-395 ios, high uncertainty due to weak signal resulting in fictive distractors, or in the absence of any 396 target, the same considerations apply to the high-pass filter hypothesis as were mentioned for the 397 gain adaptation hypothesis.

398
Distractors and multiple targets. Given the above discussion, a capability of EIH that differ-399 entiates it from prior theories and that naturally arises from its distributed sampling approach is 400 its ability to reject distractors and sample multiple targets. The live animal experimental data we 401 analyzed did not feature either real distractors (here defined as objects having a distinguishably dif-402 ferent observation model from that of the target) or multiple targets (multiple objects with identical 403 observation models). Nonetheless, the EIH simulations disclose that what we are calling "sensing-404 related motions"-those movements that increase as sensory signals weaken-sometimes occur 405 for rejection of fictive distractors. A fictive distractor emerges when the current belief for the tar-406 get's location becomes multi-peaked; each peak away from the true target's location is then a fictive 407 distractor (illustrated by arrow in Fig. 2D). Fig. S2 shows the presence of these fictive distractors in 408 the simulations of the fish, cockroach, and mole tracking behaviors, where we plot the belief rather 409 than the EID. Fictive distractors arise in both the strong and weak signal conditions, but result in 410 small amplitude excursions in the strong signal conditions because of the higher confidence of 411 observations. We hypothesize that a similar process of fictive distractor rejection is one source of 412 the increase in sensing-related wiggle movements in the animal data as sensory signals weaken. In 413 the simulated tracking behavior, the other source of sensing-related movements is the increased 414 spread of a (unimodal) EID as signals weaken, as earlier discussed.

415
False positive rejection has the signature of a digression from the nominal tracking trajectory; 416 this digression ends when one or more samples have been received indicating there is no object 417 present at the spurious belief peak, which then brings the believed target location back to some-418 where closer to the true target position (Fig. S2). In contrast, with a physical distractor, a digression 419 should occur, but the observations support that the object being detected has a different observa-420 tion model from that of the target, rather than the absence of an object. As none of our datasets 421 include physical distractors, we investigated EIH's behavior in this case with a simulated physical 422 distractor. Fig. S5 shows a simulated stationary physical distractor in addition to a stationary target.

423
EIH is able to locate the desired target while rejecting the distractor. This result buttresses a finding 424 in a prior robotics study, where we experimentally tested how EIH responded to the presence of 425 a physical distractor and showed that an electrolocation robot initially sampled the distractor but  If, instead of a distractor and a target, EIH has two targets, the advantage of EIH's sampling the 429 workspace proportional to the information density is particularly well highlighted. A simulation of 430 this condition is shown in Fig. S4. EIH maintains good tracking with an oscillatory motion providing 431 coverage for both of the targets. As seen in Fig. S4 436 While these preliminary simulations exploring how EIH performs with multiple targets and dis-437 tractors are promising, it points to a clear need for animal tracking data in the presence of physical 438 distractors or multiple targets (and in 2-D or 3-D behaviors: Appendix 6) in order to better under-439 stand whether EIH predicts sensory organ motion better than the gain adapatation or high-pass 440 filtering theories in these cases.

441
Biological implementation. The sensor or whole-body wiggle we observe in our results is for 442 proportional betting with regard to sensory system-specific EIDs-for electrosense, olfaction, and 443 vision. To implement EIH, one needs to store at least a belief encoding knowledge about the target.

444
The Bayesian filter update in EIH has the Markovian property, meaning that only the most recent 445 belief is required to be stored. The EID, moreover, is derived from the belief and only used for every 446 generated trajectory segment update, hence does not need to be stored. While the memory needs were not purely for maximizing information and attributed in part to perceptual or motor noise 483 (Yang et al., 2016). We hypothesize that these apparently less efficient fixation locations are in fact 484 the result of gambling on information through motion. It is also possible that motor noise may aid 485 coverage in a computationally inexpensive manner.

486
The role of motion in this sensing setting is to mitigate the adverse impact of sensor properties.

487
If, however, one is in an uncertain world full of surprises that cannot be anticipated, using energy 488 to more fully measure the world's properties makes sense. This is like hunting for a particular 489 target in a world where the environment has suddenly turned into a funhouse hall of mirrors. Just 490 as finding one's way through a hall of mirrors involves many uses of the body as an information 491 probe-ducking and weaving, and reaching out to touch surfaces-EIH predicts amplified energy 492 expenditure in response to large structural uncertainties.    Tracking data of blind eastern American moles (Scalopus aquaticus, Linnaeus 1758) locating a sta-717 tionary odor source were digitized from a prior study (Catania, 2013). Three experimental condi-718 tions were used in the original study: one in which there was normal airflow (categorized in the 719 strong signal condition), one where one nares was blocked (weak signal condition), and one where 720 the airflow was crossed to the nares using an external manifold (also weak signal condition). Rela-721 tive exploration was defined as the ratio between the cumulative 2D distance traveled by mole and The algorithmic implementation of EIH is built upon a framework we introduced in prior work for 751 robotic tracking of stationary targets using Fisher information (Miller et al., 2016). The original algo-752 rithm was modified to track moving targets using entropy reduction as the information measure for 753 better comparison to infotaxis, which also used this approach (Vergassola et al., 2007)  Movie 1 provides a theory overview and Supplementary Movie 2 shows how EIH is applied to the 758 control of sensing-related movements of a bio-inspired electrolocation robot. For pseudocode of 759 EIH and simulation parameters, see Alg. S1 and Tables S1-S2.

760
For all analyzed animals, the body or sensory organ being considered is modeled as a point 761 mass in a 1-dimensional workspace. The workspace is normalized to 0 to 1 for all the simulations.

762
Each sensory measurement is drawn from a Gaussian function that models the signal coming in 763 to the the sensory system plus a zero-mean Gaussian measurement noise to simulate the effect  (Table S1); The SNR values used for all the simulations is documented in Table S2. It should 775 be noted that the values of SNR used in EIH simulations are only intended to relate qualitatively 776 ("strong" or "weak" signal) to the actual (unknown) SNR of the animal's sensory system during be-

786
Information about each measurement is quantified in the form of expected information density 787 and used through a Bayesian filter (Thrun et al., 2005) to acquire the posterior belief distribution, 788 as was the case for the infotaxis algorithm (Vergassola et al., 2007). During a given tracking task, 789 the EIH algorithm calculates an expected information density (EID) that represents the predicted in-

794
The initial condition for all the simulation trials was a uniformly flat (uninformative) belief and 795 an initial state of zero velocity and acceleration. To ensure uniformity, all the simulation trials were 796 set to have the exact same internal parameters except for SNR, which was changed across trials 797 to compare trajectories in strong and weak signal conditions, and , the duration of each planned 798 trajectory, which was shortened for the moth trials due to the multiple frequency content of the 799 prescribed target motion (Table S1).

801
For each species, we model only one sensory system, the sensory system whose input was de-802 graded through some experimental manipulation during the study. We model the sensory system 803 as a point-sensor moving in one dimension (electrosense for fish, olfaction for mole and cock-804 roach, and vision for moth). The sensory system is assumed to provide an estimate of location, 805 modeled by drawing values from a Gaussian probability distribution with a variance determined 806 by the specified signal-to-noise ratio (SNR). This is called the "observation model" for a system. the trajectory spends in a neighborhood  is proportional to the measure of that neighborhood 848 ∫  Φ( ) ( Fig. 2A). With a finite time horizon, perfect ergodicity is impossible unless one uses in-849 finite velocity, which motivates a metric on ergodicity (Scott, 2013). A metric on ergodicity should 850 be zero when a trajectory is perfectly ergodic and strictly positive and convex otherwise, providing 851 a criterion that can be optimized to make a trajectory as ergodic as possible given the control cost 852 constraint (see below). A standard metric used for comparing distributions is the Sobolev space 853 norm, which can be computed by taking the spatial Fourier transform of Φ( ) and ( ) (see below).

854
This metric is equivalent to other known metrics such as those based on wavelets (Scott, 2013). 855 We can generate an ergodic information harvesting trajectory by optimizing the trajectory with for a given trajectory. In the first-order approximation of the kinematics of motion of animals, the 871 control cost is defined by the total kinetic energy required to execute the input trajectory (Alg. S1).

872
In our study, the control cost is not intended to explicitly model the energy consumption of any 873 particular animal used in the study. It is used, however, to represent the fact that energy is a factor 874 that animals need to trade-off with information while generating trajectories for sensory acquisi-875 tion. The trade-off between ergodicity and energy of motion is represented by ∕ , where is the 876 weight on the control cost and is the weight on the distance from ergodicity (Table S1). We used 877 a value of ∕ = 2, resulting in a relative exploration value of around 2. The variation in relative 878 exploration with an order of magnitude change in ∕ from 1 to 10 is 1.4 to 2.3 (Fig. S6).

Behavioral trajectory simulations 880
It is worth noting that in EIH, the animal's tracking behavior is hypothesized to be the outcome of a 881 dynamical system, the result of forces and masses interacting, rather than sample paths of a ran-882 dom process-the traditional venue for ergodicity and entropy to play a role in analysis. However,

883
we discuss the possibility that sense organs are moved stochastically in Appendix 5. When used to 884 simulate behavioral trajectories, EIH was reconfigured to use the prescribed stimulus path from 885 the corresponding live animal experiment as the target trajectory (Fig. 3). The simulated sensor's 886 initial position was set to match the animal's starting location. To simulate the effect of a weak 887 sensory signal, the SNR was reduced in the respective trials to simulate the effect of increased 888 measurement uncertainty. Other than target trajectory, initial position, and SNR, the simulation 889 parameters were the same across all simulations (Table S2).

891
The simulated electric fish tracking response is filtered through zero-phase IIR low-pass filters with 892 different stop band attenuations (Fig. S7A). These filters are configured to pass the low frequency 893 tracking band within~0.1 Hz (target motion is a sinusoid in 0.05 Hz). This configuration allows effec-894 tive removal of higher frequency wiggle motion without affecting the baseline tracking response.

895
The effect of the wiggle filter is parameterized by the stop band attenuation at 1.5 Hz. The wig-896 gle magnitude can be systematically deteriorated by controlling the stop band attenuation while 897 maintaining intact baseline tracking ( Fig. S7A-C).

898
The raw simulated weak signal tracking trajectory is first filtered by the wiggle filter at stepped 899 attenuation levels from 5 dB to 150 dB. The filtered trajectory is then prescribed to a tracking-900 only simulation where the sensor is instructed to move along the predefined input trajectory, take 901 continuous sensor measurements, and use these to update the belief and EID. The distance from 902 ergodicity is then evaluated based on the trajectory segment and simulated EID in the same way as 903 for the other behavior simulations. Tracking performance is evaluated by comparing the sensor's 904 best estimate of the target's position over time based on its belief and the ground truth.

907
The infotaxis algorithm computes the EID in the same way as EIH, but differs in the movement 908 policy once the EID is computed. The sensor considers three movement directions from its current We analyzed how the additional movement for tracking in weak signal conditions affected energy 916 use for electric fish (Fig. 6). We estimated the net mechanical work required to move the fish

922
The relative energy was defined as the total mechanical work of moving the fish along the track-923 ing trajectory divided by the work of moving the fish along the trajectory of the target (the refuge).

924
A relative energy of "1x" therefore indicates that moving the fish along the tracking trajectory re-925 quired the same energy as moving it along the path that the moving refuge took.

927
The frequency response of electric fish, mole, cockroach, and moth tracking and simulation data 928 were analyzed using the Fourier transform. The magnitude frequency response data was used in tracking response frequency range (Fig. 4A-B). For the mole and cockroach, because the target is 946 stationary and hence there is no baseline tracking frequency, the entire frequency spectrum of the 947 tracking response was used for computing the statistics.

948
Quantification and statistical analysis 949 The Kruskal-Wallis one-way ANOVA test was used for the statistical analysis of relative exploration 950 (Fig. 3) and spectral power of tracking (Fig. 4). Each trial of weakly electric fish, mole, cockroach, and 951 moth behavior as well as their corresponding simulations were considered independent. Kruskal-

952
Wallis is non-parametric and hence can be applied to test for the significance of relative exploration 953 even though it is a ratio distribution.

954
The Pearson correlation coefficient and the 95% confidence interval of its distribution were 955 calculated in Fig. S3B based on data from Fig. S3A. The mean and 95% confidence interval was 956 computed for Fig. 3 and S3.

957
Data and software availability 958 All data and code needed to reproduce our results is available (Chen et al., 2019), as well as an 959 interactive Jupyter notebook tutorial on computing the EID. Algorithm S1 provides psuedocode, 960 and Tables S1-S2 provide the corresponding simulation parameters for the EIH algorithm. Finally,

961
Supplementary Video S1 provides a video explainer of the theory of EIH, and Video S2 applies it to 962 controlling an underwater electrolocation robot. Algorithm S1 Animal Tracking Simulation with Ergodic Information Harvesting   Table S1. Parameters of EIH Simulation Table S2. Simulation Parameters Used For Each Figure   Figure S1. Illustration of differences between EIH and entropy minimization. (A-B) An illustration of an animal localizing a target of interest through moving its sensory organs or whole body with the proportional-betting (A) and information maximization (entropy minimization) (B) strategy. Black dots along the red trajectory indicate fixed time intervals, so longer distances between dots indicates higher speed. Uknown to the animal, the true target is the white star on the left, while on the right is a distractor (at center of red zone). The heat map represents the expected information density. Each step of EIH (Algorithm S1, A) is taken so that the resulting trajectory spends time in areas (e.g.  ) proportional to the expected information of that area. Trajectories generated by EIH, such as the one shown here, are ergodic with respect to the expected information because the percentage of time the trajectory spends in any subset  from = 0 to = is equal to the measure of expected information within  ; this condition must hold for all possible subsets. Information maximization (B), by contrast, locally maximizes the expected information (i.e. EID) at every step, leading it to go straight to the nearby distractor by following the gradient in this case. (C-D) Simulated animal tracking a target moving in 1D sine wave. Here we show simulated entropy minimization tracking trajectories alongside the corresponding simulated EIH trajectories under extreme SNR conditions (SNR of 10 dB and 55 dB, also shown in Fig. S3A). (C) With extremely high uncertainty (SNR = 10 dB), both algorithms were unable to accumulate enough information to successfully track the target. EIH results in persistent activation of movement even when the expected informativeness of sensing locations is flat due to a lack of information. The trajectory is prescribed to visit all locations of space with equal probability, covering the majority of the workspace (but not the entire workspace due to the energy constraint in trajectory optimization; see Algorithm S1). Entropy minimization results in cessation of motion under the same conditions. (D) Same as (C), but under extremely low uncertainty (strong signal) condition (SNR = 60 dB). Now entropy minimization hugs the one peak of the EID (there are two, as shown in Fig. 2C), while the EIH trajectory samples both peaks of the EID with excursions away from these peaks, similar to electric fish refuge tracking in the strong signal condition (Fig. 3A), and what can be inferred from a prior behavioral study in bat echolocation Yovel et al. (2010). Fig. 3. EIH simulations of tracking behavior of weakly electric fish, mole, and cockroach. The trials shown are identical to those shown in Fig. 3. Simulated sensor position over time is the solid green line for the strong signal condition, and the solid red line in the weak signal condition. The target position is the dashed blue line. The repeated dashed gray lines mark each planning update event that segment the trajectory into individual planned trajectory segments of length . The magenta heatmap shows the progression of the belief over time. Note the presence of fictive distractors, where the belief distribution becomes multi-peaked (asterisks). The inset in the mole and cockroach trials show a magnified segment to better visualize these distractors in the strong signal case.

Figure S3. Systematic comparison between EIH and infotaxis in tracking a target moving in sinusoid.
Simulations of sensor tracks a target moving in sinusoid under 17 different SNR conditions from 10 dB to 55 dB. For each SNR condition, 10 simulations with different pseudo-number seeds are performed to establish the confidence interval ( = 170 for EIH, = 170 for infotaxis). (A) As the SNR decreases from 55 dB, EIH exhibits elevated relative exploration (see Methods) in the form of increased wiggle amplitude. This is consistent with the animal behaviors summarized in Figures 2-3. EIH's elevated exploration with decreasing SNR tapers at ≈20 dB as the Bayesian filter fails to converge the posterior due to significantly less informative measurements. Infotaxis does not show the trend of increased exploration as signal weakens between 55-20 dB, and prescribed cessation of movement at the lowest (10 dB) signal strength. Throughout most of the frequency spectrum, infotaxis attempts to hug one edge of the EID, leading to near unity relative exploration. (B) The correlation coefficient between relative exploration and signal strength computed from A for EIH and infotaxis (n=170 trials for EIH, and n=170 for infotaxis). The vertical line indicates the 95% confidence interval. Ergodic harvesting has a significantly more negative correlation, indicating exploration increases as signal strength declines. The same trend is absent in infotaxis. Figure S4. Dual target tracking simulation with EIH and infotaxis. In this simulation, two identical targets (with the same observation model) are present in the workspace, indicated by the two blue lines. To help visualize the outcome, both the belief (left panel) and EID (right panel) distribution over time are shown in magenta. In both cases, we use identical SNR levels corresponding to a weak signal condition (30 dB, see Table S2). (A) EIH tracks both targets with an oscillatory motion. In addition, belief peaks corresponding to the top and bottom targets are clearly visible in the left panel. (B) Infotaxis gets trapped at one of the information maximizing peaks, as seen in the right panel. This leads to cessation of movement and failure to detect the other target on the top (left panel).

Figure S5. Single target tracking simulation with EIH and infotaxis in the presence of a simulated physical distractor.
In this simulation, a single target and a physical distractor coexist in the workspace. The simulated physical distractor has a different observation model that leads to a different measurement profile when compared to the desired target. To help visualize the outcome, both the belief (left panel) and EID (right panel) distribution over time are shown in magenta. In both cases, we use identical SNR levels corresponding to a weak signal condition (30 dB, see Table S2). (A) EIH is able to find the desired target (the continuous peak along the desired target location in the left panel) and rejects the distractor (no continuous peak along the distractor location in the left panel). (B) Infotaxis gets trapped at one of the information maximizing peaks, as seen in the right panel. Although it does find the desired target, it fails to reject the distractor, as seen in the continuous peak near the distractor in the left panel. Figure S6. Sensitivity analysis on the ratio between control cost and ergodic cost in the objective function of trajectory optimization. Simulations are conducted in the same way as Fig. S3 but only for a fixed SNR under weak signal conditions (20 dB). The control cost term is varied while fixing the ergodic cost term to be 5 for all the simulations. We simulated 16 different ratios of ∕ (shown along x axis on a log scale), each done with 10 different pseudo-random number seeds to establish confidence intervals. The relative exploration of the simulated trajectories are shown in the y axis on a linear scale with the sample mean marked by the solid color line and 95% confidence interval marked by the vertical color patch. The ∕ ratio used for all the EIH simulations included in the results is shown by the red dashed vertical line. Overall, as the ∕ drops, relative exploration drops monotonically because movement incurs higher cost. The trend also suggests that as the ∕ ratio increases, movement will cease. The value of ∕ = 2 that we used in our study results in a relative exploration value of around 2. The variation in relative exploration with an order of magnitude change in ∕ from 1 to 10 is 1.4 to 2.3. The same sensitivity analysis was also done for a strong signal condition but is not shown as the trend is similar. Figure S7. How sensor wiggles were attenuated for analyzing the impact of their removal. (A) Response of three types of wiggle attenuation filters. We used an IIR lowpass filter with a cutoff frequency of 0.1 Hz to avoid filtering the baseline tracking frequency band of the target (≈0.05 Hz in this case). In the two filtered examples shown, the attenuation at 1.5 Hz is set to 50 dB and 150 dB to achieve progressively stronger attenuation of sensor wiggling. (B) Frequency response of the original and filtered trajectory. The baseline tracking gain between 0 Hz to 0.1 Hz is well preserved while the wiggling frequency band between 0.1 Hz to 1.5 Hz is attenuated according to the attenuation gain. (C) Comparison of original and filtered trajectories. As the stopband attenuation increases, wiggle amplitude decreases. Figure S8. Rat odor tracking behavior and EIH simulation. In a prior study (Khan et al., 2012), Wistar rats (Rattus norvegicus, Berkenhout 1769) performed an odor tracking task by following a uniform odor trail on a moving treadmill with only olfactory information, under two different surgical conditions: with one side of the nose blocked (single-side nares stitching) and no blocking (sham stitching) as a control group. The experiment setup and tracked trajectory is shown in the top panel. Position data on the axis parallel to the treadmill were ignored, while positions on the perpendicular axis were recorded. The relative exploration was computed as with electric fish, using the cumulative 1D distance traveled by the rat divided by the cumulative distance of the transverse location of the odor trail. The olfactory system of the rat was modeled in the same fashion as for electric fish: as eventually providing an estimate of the location of the odor trail, modeled as a single 1D point-sensor of location with a Gaussian probability distribution controlled by the SNR of simulation (Methods). The blocked nares condition is here counted as the weak signal with lower SNR, and the sham stitching condition is counted as the strong signal. In the second row, we show EIH simulations of the weak and strong target trajectories, with the EID profile overlaid in magenta. It shows similar predicted sensor trajectories with a similar change in the relative exploration between these signal conditions as measured. These data are not included in the main study due to the low number of trials. Figure S9. Measurements compared to prediction of EIH in two conditions where an animal needs to find the signal during tracking behavior. As a demonstration of how our model seamlessly transitions between exploitation and exploration, we examined an instance in the measured behavior of the live animals in which the rat appears to lose a signal it is following Khan et al. (2012), and an instance in which the mole starts off without finding the path to the source Catania (2013). The measured result shown here for both animals was wider casting motions transverse to the direction of movement. We examined whether these two behaviors (wider casting with loss of signal during tracking or when not finding the signal initially) would be predicted by EIH. (A) To simulate the condition of losing a signal trail in the middle of a successful tracking trial, we ran EIH as for the rat olfactory tracking case of Fig. S8, but with the trajectory shown in in panel A. In the "target lost" region (green square in figure), the sensor's input was replaced by random draws from the observation model. (B) To simulate the condition of starting without knowing where the trail is for an extended period of time, we ran EIH as for the mole olfactory tracking case of Fig. 3B, but with the trajectory shown in panel B. In the "target not yet acquired" region (green square in figure), the sensor's input was replaced by random draws from the observation model. In both instances, we observed wider casting motions transverse to the direction of travel similar to those measured (output from EIH below experimental data, overlaid with the EID profile in magenta). Similar to the measured behavior, our algorithm seamlessly transitions from exploitation to exploration when the expected information density becomes more diffuse.  Table 1 for the value used for each of the parameters, and Table 2 for the SNR value used for each figure.
• Note: For trial simulations, the target location as a function of time ( ( )) is set to what it was in the original experimental data set; however, the simulated animal does not know ( )). Where EIH is being used in the real world, within a robot or instantiated in biology, ( ) would not be specified. • Define cost function ( , ) = ( ( )) + ∫ 0 1 2 ( ) 2 for > 0 and > 0, where ( ( )) is the ergodic cost based on the current EID, and is the ergodic cost weighting factor (see entry for , Table 1), while ∫ 0 1 2 ( ) 2 is the control cost weighted by (Table 1), ( ) is the control input at time that drives sensor motion • Define initial state 0 , the maximum length of the simulation max , and , the duration of each planned trajectory • Define > 0, the threshold on the norm of the gradient used to terminate the line search in the trajectory optimization procedure • Initialize prior belief ( 0 ) as a uniform distribution 2: Compute initial EID( ).
Note: For Fig. 2, we plot the EID as it is computed at each time step for illustration purposes only (thus the EID computation is within the above loop); in either case, the updated EID only impacts the sensor trajectory after ( + ) when the ergodic trajectory optimization routine is called.
19: end procedure 20: end function In seconds. is initially chosen to fit weakly electric fish behavior and kept the same for all the EIH simulations except for moth, where is set to 0.5 to account for the increased bandwidth of the sum-of-sine trajectory.
Step size control of the backtracking line search of trajectory optimization s 0.1 s and s were picked to balance between the speed of convergence and the final cost of the trajectory optimization and are fixed across all the EIH simulations Step size control of the backtracking line search of trajectory optimization s 0.4 s and s were picked to balance between the speed of convergence and the final cost of the trajectory optimization and are fixed across all the EIH simulations Weight of the distance from ergodicity term in the cost function of trajectory optimization loop (see Alg. S1) 5 is initially chosen to fit weakly electric fish behavior and kept the same for all the simulations. Note that changing changes the trade-off between ergodicity  (how much information one wants) and control effort (how much energy one is willing to give up). As a result, there is mild sensitivity to this parameter-making it an order of magnitude larger will lead to a more exploratory trajectory while making it an order of magnitude smaller will lead to less exploration. If is set to zero, no movement will occur at all. For further discussion of this point, see Miller et al. (2016). Finally, a sensitivity analysis is also provided in Fig. S6 Weight of the control term in the cost function of trajectory optimization loop (see Alg. S1) 10, 20 is initially chosen to fit weakly electric fish behavior and kept the same for all the simulations except for moth, where is set to 20 to account for the increased bandwidth of the sum-of-sine trajectory. Note that the control cost, when evaluated is equivalent to the total kinetic energy required to execute the candidate trajectory.
Number of dimensions used for Sobolev space norm in ergodic metric S 15 S is initially chosen to be a sufficient number for representing all the behavioral data considered in this paper and kept the same for all the simulations Initial control input (0) 0 Zero control is applied at the beginning of every simulation Initial belief ( 0 ) unif(0, 1) Initial belief is set and fixed to an uniform ("flat") prior distribution within the workspace (from 0 to 1) where the probability of the target located at every location is identical.   Figure 3A) N/A (simulation) Figure S1C Weak Signal 10 0.4 (same as Figure 3A) N/A (simulation) Figure S1D Strong Signal 60 0.4 (same as Figure 3A) N/A (simulation) Figure S2 Weak Signal ≤30 (same as Figure 3) (same as Figure 3) (same as Figure 3) Figure S3 Strong and Weak Signal 10-55 0.4 (same as Figure 3A) N/A (simulation) Figure S4 Weak Signal 30 0.4 Stationary N/A (simulation) Figure S5 Weak Signal 30 0.4 Stationary N/A (simulation) Figure S6 Weak Signal 20 0.4 (same as Figure 3A) N/A (simulation) Figure S8 Strong Signal Uknown to the animal, the true target is the white star on the left, while on the right is a distractor (at center of red zone). The heat map represents the expected information density. Each step of EIH (Supplement Algorithm S1, A) is taken so that the resulting trajectory spends time in areas (e.g.  ) proportional to the expected information of that area. Trajectories generated by EIH, such as the one shown here, are ergodic with respect to the expected information because the percentage of time the trajectory spends in any subset  from = 0 to = is equal to the measure of expected information within  ; this condition must hold for all possible subsets. Information maximization (B), by contrast, locally maximizes the expected information (i.e. EID) at every step, leading it to go straight to the nearby distractor by following the gradient in this case. (C-D) Simulated animal tracking a target moving in 1D sine wave. Here we show simulated entropy minimization tracking trajectories alongside the corresponding simulated EIH trajectories under extreme SNR conditions (SNR of 10 dB and 55 dB, also shown in Fig. S3A). (C) With extremely high uncertainty (SNR = 10 dB), both algorithms were unable to accumulate enough information to successfully track the target. EIH results in persistent activation of movement even when the expected informativeness of sensing locations is flat due to a lack of information. The trajectory is prescribed to visit all locations of space with equal probability, covering the majority of the workspace (but not the entire workspace due to the energy constraint in trajectory optimization; see Supplement Algorithm S1). Entropy minimization results in cessation of motion under the same conditions. (D) Same as (C), but under extremely low uncertainty (strong signal) condition (SNR = 60 dB). Now entropy minimization hugs the one peak of the EID (there are two, as shown in Fig. 2C), while the EIH trajectory samples both peaks of the EID with excursions away from these peaks, similar to electric fish refuge tracking in the strong signal condition (Fig. 3A), and what can be inferred from a prior behavioral study in bat echolocation Yovel et al.  Figure S3. Systematic comparison between EIH and infotaxis in tracking a target moving in sinusoid. Simulations of sensor tracks a target moving in sinusoid under 17 different SNR conditions from 10 dB to 55 dB. For each SNR condition, 10 simulations with different pseudo-number seeds are performed to establish the confidence interval ( = 170 for EIH, = 170 for infotaxis). (A) As the SNR decreases from 55 dB, EIH exhibits elevated relative exploration (see Methods) in the form of increased wiggle amplitude. This is consistent with the animal behaviors summarized in Figures 2-3. EIH's elevated exploration with decreasing SNR tapers at ≈20 dB as the Bayesian filter fails to converge the posterior due to significantly less informative measurements. Infotaxis does not show the trend of increased exploration as signal weakens between 55-20 dB, and prescribed cessation of movement at the lowest (10 dB) signal strength. Throughout most of the frequency spectrum, infotaxis attempts to hug one edge of the EID, leading to near unity relative exploration. (B) The correlation coefficient between relative exploration and signal strength computed from A for EIH and infotaxis (n=170 trials for EIH, and n=170 for infotaxis). The vertical line indicates the 95% confidence interval. Ergodic harvesting has a significantly more negative correlation, indicating exploration increases as signal strength declines. The same trend is absent in infotaxis.  Figure S5. Single target tracking simulation with EIH and infotaxis in the presence of a simulated physical distractor. In this simulation, a single target and a physical distractor coexist in the workspace. The simulated physical distractor has a different observation model that leads to a different measurement profile when compared to the desired target. To help visualize the outcome, both the belief (left panel) and EID (right panel) distribution over time are shown in magenta. In both cases, we use identical SNR levels corresponding to a weak signal condition (30 dB, see Table 2). (A) EIH is able to find the desired target (the continuous peak along the desired target location in the left panel) and rejects the distractor (no continuous peak along the distractor location in the left panel). (B) Infotaxis gets trapped at one of the information maximizing peaks, as seen in the right panel. Although it does find the desired target, it fails to reject the distractor, as seen in the continuous peak near the distractor in the left panel. Relative exploration Setting used in EIH simulations Figure S6. Sensitivity analysis on the ratio between control cost and ergodic cost in the objective function of trajectory optimization. Simulations are conducted in the same way as Fig. S3 but only for a fixed SNR under weak signal conditions (20 dB). The control cost term is varied while fixing the ergodic cost term to be 5 for all the simulations. We simulated 16 different ratios of ∕ (shown along x axis on a log scale), each done with 10 different pseudo-random number seeds to establish confidence intervals. The relative exploration of the simulated trajectories are shown in the y axis on a linear scale with the sample mean marked by the solid color line and 95% confidence interval marked by the vertical color patch. The ∕ ratio used for all the EIH simulations included in the results is shown by the red dashed vertical line. Overall, as the ∕ drops, relative exploration drops monotonically because movement incurs higher cost.
The trend also suggests that as the ∕ ratio increases, movement will cease. The value of ∕ = 2 that we used in our study results in a relative exploration value of around 2. The variation in relative exploration with an order of magnitude change in ∕ from 1 to 10 is 1.4 to 2.3. The same sensitivity analysis was also done for a strong signal condition but is not shown as the trend is similar.   Figure S8. Rat odor tracking behavior and EIH simulation. In a prior study (Khan et al., 2012), Wistar rats (Rattus norvegicus, Berkenhout 1769) performed an odor tracking task by following a uniform odor trail on a moving treadmill with only olfactory information, under two different surgical conditions: with one side of the nose blocked (single-side nares stitching) and no blocking (sham stitching) as a control group. The experiment setup and tracked trajectory is shown in the top panel. Position data on the axis parallel to the treadmill were ignored, while positions on the perpendicular axis were recorded. The relative exploration was computed as with electric fish, using the cumulative 1D distance traveled by the rat divided by the cumulative distance of the transverse location of the odor trail. The olfactory system of the rat was modeled in the same fashion as for electric fish: as eventually providing an estimate of the location of the odor trail, modeled as a single 1D point-sensor of location with a Gaussian probability distribution controlled by the SNR of simulation (Methods). The blocked nares condition is here counted as the weak signal with lower SNR, and the sham stitching condition is counted as the strong signal.
In the second row, we show EIH simulations of the weak and strong target trajectories, with the EID profile overlaid in magenta. It shows similar predicted sensor trajectories with a similar change in the relative exploration between these signal conditions as measured. These data are not included in the main study due to the low number of trials. Ergodic Harvesting Figure S9. Measurements compared to prediction of EIH in two conditions where an animal needs to find the signal during tracking behavior. As a demonstration of how our model seamlessly transitions between exploitation and exploration, we examined an instance in the measured behavior of the live animals in which the rat appears to lose a signal it is following Khan et al. (2012), and an instance in which the mole starts off without finding the path to the source Catania (2013). The measured result shown here for both animals was wider casting motions transverse to the direction of movement. We examined whether these two behaviors (wider casting with loss of signal during tracking or when not finding the signal initially) would be predicted by EIH. (A) To simulate the condition of losing a signal trail in the middle of a successful tracking trial, we ran EIH as for the rat olfactory tracking case of Fig. S8, but with the trajectory shown in in panel A. In the "target lost" region (green square in figure), the sensor's input was replaced by random draws from the observation model. (B) To simulate the condition of starting without knowing where the trail is for an extended period of time, we ran EIH as for the mole olfactory tracking case of Fig. 3B, but with the trajectory shown in panel B. In the "target not yet acquired" region (green square in figure), the sensor's input was replaced by random draws from the observation model. In both instances, we observed wider casting motions transverse to the direction of travel similar to those measured (output from EIH below experimental data, overlaid with the EID profile in magenta). Similar to the measured behavior, our algorithm seamlessly transitions from exploitation to exploration when the expected information density becomes more diffuse. The fish's electric organ discharge (EOD) frequency shifts up continuously as the jamming signal is being applied. The area shaded with light blue indicates when jamming is turned on and lasts 40 seconds. The jamming signal frequency (current amplitude = 25 mA) is set to be 5 Hz below the fish's EOD frequency. The fish's EOD frequency gradually declines back to its normal value after jamming is turned off after around 60 seconds. (B) Maximum frequency shift during the fixed 40 second jamming window as a function of jamming current amplitude. Maximum frequency shift is averaged for every jamming amplitude across a total of 26 trials ( = 6 for 0 mA, = 5 for 5 mA, = 3 for 10 mA, = 5 for 25 mA, = 7 for 50 mA). The amount of jamming applied has a clear positive correlation with the EOD frequency shift.