Abstract
To study the evolutionary origins of object perception, we investigated whether a primitive vertebrate, the larval zebrafish, is sensitive to the presence of obstacles. The zebrafish, which has become a useful model to study brain-wide circuit dynamics, executes fast escape swims when in danger of predation. We posited that collisions with solid objects during escape would be maladaptive to the zebrafish, and therefore the direction of escape swims should be informed by the locations of barriers. To answer this question, we developed a novel closed-loop high-speed imaging rig outfitted with barriers of various qualities. Using this system, we show that when larval zebrafish escape in response to a non-directional vibrational stimulus, they use visual scene information to avoid collisions with obstacles. Our study demonstrates that fish compute absolute distance to obstacles, as distant barriers outside of collision range elicit less bias than nearby collidable barriers that occupy the same visual field. The computation of barrier features is covert, as the fish’s reaction to barriers during routine swimming does not predict that they will avoid barriers when escaping. Finally, through two-photon laser ablations, we reveal a novel excitatory input from the visual system to Mauthner cells in the brainstem escape network that is responsible for escape direction bias. We propose that zebrafish construct “object solidity” via an integrative visual computation that is more complex than retinal occupancy alone, suggesting a primitive understanding of object features and possibly the origins of a structured model of the physical world.
Introduction
The ability of humans and animals to interact with the environment is mediated by our understanding of how the physical world works (i.e. ”intuitive physics”; [4, 36]). There are two accounts on how information available in the world is transformed into action by the brain. In one account, evolution has established a mapping between patterns of neural activity on the sensors (e.g. the retina) and corresponding adaptive behaviors. Under this ”reactive policy”, animals can be thought of as input-output machines resembling a Roomba [8], which flourishes within its limited environment and displays seemingly complex behaviors that are built up from simple rules. On the other hand, higher animals appear to demonstrate cognitive flexibility that suggests a structured model of the world: objects have three dimensions, identities, and properties that exist through time and space. Objects can be classified without visible action (e.g. ”covert” attention), and our knowledge about the scene structure can be used to explicitly predict future states and outcomes (e.g. collisions). This type of predictive capacity likely arose as the environment become so complex that a fixed reactive policy for each situation would require almost infinite computational capacity. It remains an open question at what point in evolution this transition from reactive behavior to structured knowledge began to occur? Earlier studies had suggested that simple animals like chicks and frogs possess some aspects of physical cognition [10, 17, 34]; we therefore wondered whether an even simpler, neurally accessible organism, the larval zebrafish, would display behaviors that rely on the representation of objects as solid structures.
To this end, we examined the sensitivity of larval zebrafish to the locations of barriers during predator escape. In zebrafish, escapes from predators or artificial predator-mimicking stimuli are controlled by a well characterized population of neurons called the ”brainstem escape network” (BEN) [13, 14, 29]. The BEN, containing the Mauthner neuron pair which is capable of evoking high-angle tail bends in response to startle, is experimentally accessible to genetic labeling and two-photon laser ablation. Critically, this circuit mediates a two alternative behavioral choice [19]: fish direct their escape trajectories to either the left or the right. However, the extremely short latency and ballistic nature of these movements make accidental collisions with surrounding objects a hazard. Therefore, it would be adaptive for the fish to generate a bias to their escape behaviors that directs them away from detected barriers.
Here we show that startled zebrafish reliably avoid collisions by directing their escapes away from barriers (i.e. if the barrier is on the left, they escape right and vice versa). We confirm that visual signals are necessary for the induction of the bias, since fish execute unbiased escapes and collide randomly with barriers in the dark. Keeping the retinal occupancy steady but varying physical size and distance of barriers, we demonstrate that fish are sensitive to the absolute distance of barriers and use a more complex mechanism to measure distance than visual occupancy alone. To account for this result, we propose plausible metrics that fish could use to compute distance out of a set of previously described methods used by animals and humans [28, 32]. Lastly, we reveal a circuit motif whereby visual information excites the Mauthner neuron responsible for escapes away from barriers. In summary, we propose that larval zebrafish implicitly represent barriers as solid objects via an excitatory visual bias of the BEN.
Results
Barrier Avoidance During Escape
We characterized the escape trajectories of 99 wild-type larval zebrafish (WIK) when presented with a non-directional startle stimulus. Fish were imaged at high speed (500 Hz) while swimming in a 12 cm diameter tank (Figure 1A). Custom computer vision software was designed to detect when fish entered a 12 mm wide circular zone located 4 cm from the edge of the tank. The location of the zone was unmarked and unknown to the fish. Upon zone entry, the detection software triggers a ”tap” via an electromagnet that strikes the tank with a metal rod for 200 ms. This stimulus reliably evoked fast escapes at extremely short latency ([22], Figure 1A, right panel). The direction of escape in the absence of barriers was random (Figure 1B,C). We introduce two indices that describe the direction of escapes in our assay. First, Preference Index (PI) is (#Rightward Escapes - #Leftward Escapes) / (# Total Escapes). Calculating this metric for each fish in the absence of barriers yielded a median preference index of 0.0 and mean of -.03, indicating no average preference for one direction or the other (p = .503, 1 sample ttest population mean different from 0.0, Figure 1C). Individual fish are most commonly unbiased in their escape which is indicated by the highest density of the normal PI distribution centered around the 0.0 bin (Figure 1C). Figure 1B (left panel) shows the time course of escape trajectories following stimulus delivery, and the 2D histogram below illustrates the unbiased distribution of locations visited by the fish immediately after the tap.
Each fish shown in the left panel of Figure 1B was also tested in an additional assay assessing the same behavior in the presence of barriers. To measure barrier avoidance, we introduce a second statistic, Barrier Avoidance Index (BAI), which is (# Escapes Away From Barrier - # Escapes Towards Barrier) / (# Total Escapes). For our initial barrier condition, we placed a 12 mm wide, 6 mm high red acrylic barriers into the tank at the same relative tap zone described above, triggering taps when fish passed within 2 mm of the barrier. The introduction of a barrier completely changed the distribution of escape directions. Fish significantly directed their escape trajectories away from the barrier (Figure 1B, center panel) with an average BAI of .78 (p = 2.9e-10). This bias away from barriers, however, disappeared if the same fish were tested in the dark (Figure 1B, right panel; p = .52). Pairwise comparisons of BAI for each fish in light and dark conditions are shown in Figure 1D, which also shows that the barrier collision rate significantly increases from 6% when fish can see the barrier to 42% in the dark (p = 7.32e-6, paired ttest). We therefore conclude that zebrafish use vision to detect barriers and reliably avoid collisions by converting their typically random escape directions into laterally biased trajectories.
One phenomenon observed during barrier trials that points to the fish’s behavioral algorithm shows that BAI depends on the angle of barrier approach. In the dark, escapes were unbiased regardless of the angle of approach (Figure 1E). However, in light conditions, if fish approach barriers head-on (0°-25° on either side), their escape trajectories are unbiased and resemble trajectories in dark conditions (bootstrap 95% confidence interval contains 0.0 BAI, yellow line Figure 1E). This suggests that lateral movement of the barrier during approach may be involved in generating bias. In this study, we therefore exclude head-on barrier approaches (−25° to 25° angle of approach) when making comparisons between barrier conditions.
In order to investigate which features of barriers trigger avoidance, we performed the tap assay while varying the height, width, distance, and color of the barrier. BAI under all tested conditions is plotted in descending order of effect in Figure 2. Doubling the distance (4 mm) to the same sized barrier as in Figure 1 continued to bias escapes away from barriers (p = .007 population mean different from 0.0), but reduced the average BAI magnitude from .78 to .27 (p = 5.4e-5, two-sample independent ttest, Figure 2A). This reduction in BAI scaled directly with the predicted probability of colliding with the barrier (45% collision rate to 19%, Figure 2A top and Figure 2 legend), consistent with the hypothesis that zebrafish map barrier locations to avoid collisions. We next wondered whether the decreased apparent width and height of the barrier due to increased distance was responsible for this reduction. Doubling the height of the 4 mm distant barrier (12 mm high) did not significantly change BAI (.27 to .36, p = .48). However, doubling the width (24 mm wide) nearly restored BAI to original values (.27 to .67, p = .001, original value.78), initially suggesting that the amount of horizontal retinal occupancy may be a key factor in determining bias. We tested this idea by placing a barrier that occupied the exact same amount of the horizontal and vertical visual field as the 24 mm wide barrier but instead at 8 mm away, reducing the probability of collision to 0 (Figure 2 top panel). Interestingly, vertical and horizontal visual field occupancy was not the most important factor in biasing escapes. Although apparent size is identical between the two conditions (24 mm w, 6 mm h, 4 mm d vs. 48 mm w, 12 mm h, 8 mm d), distance was the determining factor in biasing escapes, as BAI dropped from .67 at 4 mm distance to .22 at 8 mm distance (p = .0002, two-sample independent ttest). Coupling this result to predicted collision statistics in Figure 2 suggests that the zebrafish’s escape strategy accounts for not only how far away barriers are in their environment, but how far their escape trajectories propel them in space. The .22 average BAI in distant barrier conditions shows that escapes do remain slightly biased from 0.0 (p = .037) if the barrier is 12 mm high, but if height is dropped to 6 mm, we encounter the only barrier condition where fish do not bias from 0.0 (rightmost condition, Figure 2; BAI = .07, p = .68). Therefore, there appears to be a threshold vertical occupancy; a 6 mm high, 8 mm distant barrier occupies 35° of visual angle, suggesting the threshold for vertical occupancy is between 35° and 70°.
Barriers Influence Routine Swimming
We next asked whether barrier avoidance during escapes was covertly pre-computed or whether fish overtly show evidence of barrier aversion before the tap stimulus arrives. 12 fish were allowed to freely swim in the tank with four 12 mm wide barriers placed according to Figure 3A. We find that fish are sensitive to barrier locations even in the absence of tap evoked escapes. In Figure 3A (left), the complete trajectory of a free swimming fish is displayed. Figure 3B (left) displays a density plot with the number of visits to each bin of space across all fish. The vicinity surrounding red barriers is the least visited region of the tank, suggesting that fish treat red barriers as aversive. We conclude that the fish performs covert avoidance of barriers regardless of its overt routine behavior.
Although inherent aversion could be a strategy for avoiding collisions during escape, we wondered if we could find a barrier condition that could dissociate avoidance during escapes from aversion during free swimming. We found that replacing red barriers with white barriers of the exact same size completely inverted preference during free swimming (Figure 3A-E, p = .0002, ttest for % time spent in the 20 × 20 mm region surrounding red vs. white barriers). Instead of avoiding barriers, the zebrafish’s trajectories reflect an attraction to white barrier locations, and spatial bins bordering the white barrier were the most frequently visited (Figure 3A+B, right panel). Remarkably, zebrafish continued to treat white barriers as collidable objects during escapes, significantly biasing their escape turns away from barriers (Figure 3E, BAI = .43, p = 3.05e-6).
Like red barriers, escapes away from white barriers are not biased in the frontal 25° of visual space, and spatial bins on the side opposite the barrier are the most commonly visited during escapes (Figure 3F), suggesting the same bias mechanism. Although fish avoid white barriers two out of every three escapes, this nonetheless reflects a significant decrease in BAI from red conditions (.78 to .43, p = .0008), where barriers are darker than the floor of the tank. We therefore asked whether phototaxis, wherein larval fish swim towards the brighter visual hemifield, could play a minor role in biasing escape direction. Zebrafish, however, did not bias their escapes if tapped within 2 mm of a red dot projected onto the bottom of the tank nor a white dot projected onto a red background (Figure 3 E right side, BAI = .028, .037; p = .79, .71; dots are equal width of barriers). This suggests that color comparison between the left and right hemifield does not itself play a role in escape, and that barriers must have height to be considered an obstacle. We surmise that the small difference in BAI between red and white conditions is due to a modest interaction between covert avoidance and overt attraction.
Mauthner Cell Ablations Reveal a Biasing Circuit Motif
We next sought to determine how the visual system influences the Brainstem Escape Network (BEN), which mediates two-alternative behavioral choice during escapes and contains the commonly-known Mauthner neurons [19]. Mauthner cells, along with their homologs MiD2cm and MiD3cm, play a prominent role in evoking short latency, high amplitude escape turns in fish [14]. Spiking in the right Mauthner neuron typically results in escapes to the left, while left Mauthner spiking results in escapes to the right. However, homologs can also control escape features even in the absence of Mauthner cell activity [22, 29]. We wondered how visual information about barrier location impinges upon this complex circuit. Critically, Mauthner cells receive modulatory inputs from throughout the brain that are conspicuously positioned for biasing escape direction.
Since 97% of the taps evoked in our barrier dataset (Figure 2) resulted in rapid, large amplitude escape turns typically attributed to Mauthner cells (mean 104.4° cumulative tail angle at 11.06 ms post-tap, Supplementary Figure 1), we surmised that barrier avoidance is accomplished via lateralized Mauthner modulation.
Importantly, there are two ways for visual information to influence the Mauthners (Figure 4A): visual activity could either inhibit the Mauthner neuron that pulls the fish toward the barrier or excite the neuron that takes the fish away from the barrier. To uncover whether either of these motifs exist in the fish brain, we fluorescently labelled Mauthner neurons using a custom Gal4 line and ablated a single Mauthner neuron per fish with a two-photon laser (see Methods, Figure 4B). As noted, escape direction is typically contralateral to the spiking Mauthner neuron, so we expected fish with ablations to preferentially escape away from the side of the intact neuron. In Figure 4C, Preference Index is calculated as (# Turns away from intact Mauthner - # Turns towards intact Mauthner / # Total Escape Turns). As expected, fish significantly biased their escapes away from the intact Mauthner side when tapped in the absence of barriers (mean PI away from intact Mauthner = .405, p = .00018; see barrier-less conditions in Figure 4C). This level of bias is in alignment with previous Mauthner ablation experiments [22]. When barriers were added to the tank, taps executed when fish approached barriers contralateral to the intact Mauthner cell continued to induce escape away from the intact neuron (and resultingly, towards barriers) at the same frequency as barrier-less conditions, showing a complete loss in the ability to bias escapes away from barriers (mean PI away from intact Mauthner = .372, p = .77, two-sample paired ttest vs no barrier, Figure 4 C, D). On the other hand, if fish approached barriers ipsilateral to the intact Mauthner cell, their preference for escaping away from the intact neuron was actually facilitated (mean PI away from intact Mauthner = .622, p = .0237 two-sample paired ttest vs no barrier, Figure 4 C,D).
Coupled together, these results favor the excitatory circuit architecture proposed in Figure 4A. The intact Mauthner neuron is not inhibited by the perception of a contralateral barrier, shown by the continued escape towards barriers. The intact Mauthner neuron instead undergoes an excitatory bias during perception of a barrier on the ipsilateral side, enhancing bias up to the levels observed in Figure 2 in the strongest biasing conditions (Figure 4E).
Discussion
Constructing intelligent behavior by combining simpler sensorimotor modules has become a recurring theme in neuroethology [2, 11] and artificial intelligence research [7, 8, 27]. Human-like intuitive physics is likely built upon the visuomotor neural circuits that have allowed animals to flourish in a complex physical world for eons without the benefit of explicit physical reasoning [8, 34]. In this study, we investigated the foundations of solid object perception by asking whether larval zebrafish are sensitive to the presence of obstacles during fast escape swims. We discovered that when zebrafish are startled in the vicinity of barriers, they alter their normal pattern of unbiased escape direction and instead consistently bias their escape swims away from collidable objects (Figure 1). This decision depends on barrier height and width, but most strongly on computed distance to barriers. Moreover, the magnitude of escape bias correlates well with the probability of collision (Figure 2).
Using two-photon laser ablations, we uncovered the existence of an excitatory input to the Mauthner Neuron from the visual system that mediates escape bias. We suggest that the larval zebrafish embodies the concept of “object solidity” via connections from barrier detection modules to evolutionarily ancient predator escape circuits [25].
Biasing escape direction near obstacles is the first case of covert scene understanding that has been revealed in the larval zebrafish. By ”covert”, we mean that the zebrafish has performed computations of barrier identification, size, and distance without explicitly showing it in its behavior. Only when evoking startle responses can we observe that the fish was attending the barrier. One could argue that our routine swimming experiments (Figure 3) show that the animals are overtly attending the differently colored barriers: red barriers are avoided while white barriers are approached. Importantly, however, both barriers are avoided during escapes, suggesting that behavior in the moments preceding the escape does not predict barrier avoidance. Critically, the non-cortical brain regions that zebrafish possess have been shown to play a role in covert computations of object identity and location in both mammals and in other animals with no cortex [21, 23]. In frogs, “attention units” in the optic tectum respond to subtle movements of prey-like stimuli that do not immediately prompt hunting behavior. These subthreshold stimuli instead facilitate future prey striking behavior if the same prey object is presented seconds later. The neural correlate of this facilitation manifests when attentional units continue to respond seconds after the original stimulus has stopped moving. This study reflects the current idea that the midbrain covertly maps the location and identity of stimuli of interest in the visual scene [16], [20]. We classify this mapping as ”implicit” scene understanding: object identification (e.g. ”prey”, ”barrier”) and location have been covertly computed for future behavioral goals.
Robust 3D vision requires accurate depth estimates for all items in a scene. Our results show that zebrafish larvae compute absolute distance to barriers, since bias to nearby barriers is significantly stronger than to distant barriers that occupy the same retinal area (Figure 2). Distance computation, in fact, has now emerged as an essential feature of both the zebrafish’s escape and prey capture behaviors [6], [30]. Previous studies have emphasized the role of stimulus ”size”, measured by total retinal occupancy, in decoding stimulus identity [3]. However, total retinal occupancy, as shown here, fails to unambiguously convey the depth of a stimulus, and accordingly, its behavioral relevance. Depth must be computed from the 2D retinal representation – it does not come for “free” with any known visual map (e.g. an equal-sized neural representation on the optic tectum map could be a close prey or a distant barrier). This point is emphasized by previous studies examining the response of zebrafish to live prey and to different-sized moving dots. In prey capture, when a fish strikes at a 300 micron-wide prey item, the prey is so close (avg. 870 + 180 microns) that it occupies 16°-23° of visual space [6]. Moving dots occupying this amount of visual space induce the strongest possible avoidance in previous studies [3, 5]. Therefore, our study and previous results suggest that integration of multiple features, including contextual evidence, must be at play (see below) and the zebrafish’s capacity for scene understanding is greater than previously appreciated.
Interestingly, AI approaches seeking to replicate the brain’s ability to interpret 3D scenes almost exclusively rely on depth sensing cameras. This avoids the complexity that comes with inferring object locations from a 2D layer of photoreceptors that is itself incapable of pinpointing the depth of a light source. Studying biological distance sensing in simple animals could provide insights into how scene interpretation is so effortless for humans [18] yet difficult for computers. In fact, elegant studies in insects and amphibians have already painted a complex picture of how animals compute object distance, suggesting that dynamic lens accommodation and motion cues play important roles [24, 32, 35]. In addition to motion parallax observed in bees and flies, some insects like locusts and the praying mantis purposefully move their heads from side to side in order to gain distance information from self-motion [31, 33, 37]. We suggest that distance computations used by humans could be informed by these lower species because clinically blind humans with lesions in higher order visual areas can still navigate around obstacles without conscious awareness [15]. Similar results were obtained for reaching tasks around barriers in patients with temporoparietal damage, indicating that higher order visual awareness is not necessary for barrier avoidance during reaching [26].
The precise mechanism that larval zebrafish use to asses object distance remains unknown; however, brains use up to 13 methods to estimate distance (Figure 5) [28]. During a swim bout towards a barrier, the barrier will appear to expand, move laterally across the visual scene, and change focal plane, all in proportion to barrier distance when the swim began. Each distance calculation method comes with failure modes, but failures of different methods rarely co-occur. This allows thousands of accurate distance computations per day without crashing into walls, tripping over objects, or mistaking distant humans for nearby ants. Because barrier approaches that are ”head on” fail to evoke a bias in our experiments (Figure 1E), we surmise that the amount of lateral movement across the scene (i.e. ”motion parallax”) is a key component of distance computation, and that this variable is integrated with total amount of occupied vertical and lateral retina. Importantly, the window of visual space occupied by the barrier during ”head-on” approach encapsulates the binocular zone [5], making it unlikely that fish use stereopsis for barrier avoidance. Future research should address what happens when distance metrics are forced to artificially conflict during virtual reality – how does the brain balance information from several channels that could each disagree?
As in our previous work [6], we propose that the fish’s ability to perceive distance is intertwined with its own body constraints. Zebrafish avoid barriers at distances that are within its escape swim limits (Figure 2), which resembles the idea of embodied cognition [25]; in other words, the fish’s escape system ”knows” how far it displaces the body. Further evidence for embodied knowledge comes from the fish’s differential response to doubling the height vs. width of nearby barriers (Figure 2). Escape bias is significantly more sensitive to width changes, suggesting that the fish’s escape trajectory is likely to be more variable in the horizontal versus vertical plane. Future studies should address whether this is in fact the case, which would further support the idea that the co-evolution of sensory and motor systems should be considered during studies of perception.
We suggest that some features of the fish’s obstacle avoidance behavior (i.e. computation of absolute distance, covert attention) may suggest the presence of a structured model for physical cognition in the brain, or a primitive ”physics engine” [4, 36]. However, our results do not exclude the possibility that the zebrafish is simply approximating a physics engine using bottom up feature detection combined with the perception of its own body thrust and the resultant motion of obstacles. Yet, the behavioral strategy executed by the fish in our study is more complex than a simple reactive strategy that translates retinal pixels and current behavioral state into resultant behaviors (e.g. optomotor response [1], phototaxis [9]); nevertheless, it may still fall short of the bar required for assigning ”cognition” to the fish. How would future experiments disambiguate a complex bottom-up strategy from a structured cognitive model of objects in the world? One strategy would be to present fish embedded in a virtual reality setting with controlled visual stimuli that would disambiguate objects to a human or Bayesian observer. For example, if presented with a stimulus detected as a large distant barrier due to parallax, will the fish still approach a close by prey stimulus tuned to the specs (retinal size, angular speed) of a paramecium if the prey passes behind the barrier? Passing behind the barrier would suggest that the virtual prey is not a nearby prey at all, but a faster moving, larger object that is further away than the large barrier. If the fish pursues this stimulus as prey, a more reaction-based, sensorimotor intelligence in the spirit of Brooks would be assumed [8].
Lastly, with respect to the neural mechanism we propose, ablating one Mauthner did not lead to an abolishment of escape towards the ipislateral side of the ablation. This was expected from previous studies on Mauthner ablation [22], and likely emphasizes the capability of Mauthner homologs to compensate for Mauthner loss. Importantly, touch stimulation of the head to induce escapes is the most effective means of recruiting Mauthner homologs, and leads to a more vigorous escape swim [29]. Our non-directional tap stimulus likely induces simultaneous head and tail stimulation, recruiting Mauthner homologs and allowing a bilateral competition between all BEN components across hemispheres. Our final hypothesis entails an excitatory input to the ipsilateral Mauthner cell that biases escapes away from barriers (Figure 4). In future experiments we plan on functionally dissecting this excitatory input which we hypothesize to come from the tectum via spiral fiber neurons [22].
Materials and Methods
Zebrafish
Experiments were conducted according to the guidelines of the National Institutes of Health and were approved by the Standing Committee on the Use of Animals in Research of Harvard University. All wild-type experiments were performed on dpf 7–9 larval zebrafish of the WIK strain (ZFIN_ZDB-GENO-010531–2, https://zfin.org). For the Mauthner ablation experiments, a transgenic fish generated by our laboratory (MH 16946-Gal4, https://zebrafishexplorer.zib.de/home) was crossed to a UAS-Kaede reporter line (Tg(UAS-E1b:Kaede)s1999t; ZDB-ALT-070314-1, https://zfin.org), yielding green fluorescence in both Mauthner neurons. All fish were fed daily with paramecia starting at dpf4.
Behavioral Setup
After a morning paramecia feeding, fish were added to a 12 cm-diameter acrylic tank with clear bottom and black walls. The tank was then mounted atop a clear acrylic stage. Fish behavior was recorded from above by a Mikrotron EoSens camera capturing at 500 Hz and 1280 × 1024 pixel resolution. Custom C# acquisition code was written for high-speed video writing, online background subtraction, stimulus delivery, and fish detection (available on Github). The entire rig was enclosed in black cardboard and tarp to minimize external light and noise, and all behavioral experiments were conducted in a dark room. A space heater was placed within the rig in order to maintain zebrafish at approximately 27° C. The tank was illuminated from below with an infrared light array to allow imaging in both light and dark conditions. Additionally, to draw fish into the center of the tank where barriers were located, a radial gradient phototaxis stimulus was projected onto a cold mirror and reflected onto the bottom of the tank. Once fish reached a threshold distance from the tank center, projector illumination was switched to bright whole-field light gray, which strongly lit the walls of the tank enclosure, reflecting light both onto the bottom and top of the tank. In dark trials, the projector was instead turned to whole-field black and automatically covered with a filter-flipper to block all light. Thus all escape stimuli were delivered under bright whole-field gray illumination or darkness (except one condition described in Figure 3). For experiments with physical barriers, circular pieces of extruded acrylic were affixed to the bottom of the dish with inert dental wax. Unless explicitly described in the text as white, barriers were dark red in color. In virtual barrier experiments, circles the same width and color as the smallest physical barriers (12 mm diameter) were projected onto the floor of the tank. Throughout the entire trial, the acquisition program logged the XY coordinates of the fish, as well as the coordinates and dimensions of the barriers. If the fish came within the trial’s threshold distance of a barrier edge, a stimulus command was sent via a PyBoard microcontroller to an electromagnet affixed to the stage. The electromagnet drives a metal rod into the stage for 200 ms, inducing a non-directional auditory startle stimulus (a “tap”) [22]. An LED light was fixed to the top of the metal rod such that its reflection into a mirror situated atop the stage was captured by the camera. From the LED reflection’s movement across the mirror, the exact timing of the tap was extracted at 2-millisecond resolution (b/c 500Hz acquisition) by Python-based analysis code (available on Github). There were two metrics for excluding fish in our assay: fish were excluded from analysis if they failed to enter the barrier zone for 5 minutes, 5 times, before completing 5 escape trials (20.2% of total tested fish). Second, we excluded fish if the LED indicating stimulus delivery indicated that the stimulus was not evoked. Lastly, because we excluded the frontal-most visual field (Figure 1, 3) from analysis between barrier conditions, we required that fish must have performed at least 3 trials in the given condition to be given PI or BAI score.
Snell’s Window
For all solid barrier conditions, the top of the barrier protruded above the water surface. By calculating apparent height due to refraction and checking for the collapse of the image into the horizon of Snell’s Window [12], we confirm that the tops of all tested barriers are visible to the fish, and that distortion of the barriers at the distances tested is minimal (the image formed by a 4 mm distant barrier of 6 mm height occupies 97.5% of the retinal height subtended by the 8 mm distant barriers at 12 mm high). Water refractive index used in our calculations was 1.333 and depth of the fish eye underwater 1 mm.
Escape Analysis
Upon receiving a tap stimulus, all fish engaged in fast escape swims characterized by initial high angle turns and a subsequent burst of speed (Supplementary Figure 1). To uncover the tail angle and direction of escape trajectories relative to barriers, 500Hz videos of escape sequences were background subtracted and dynamically thresholded using an OpenCV-based recursive contour-finding algorithm that terminated after finding a fish-sized object. The heading angle was recorded by calculating a vector arising from the midpoint of the fish contour and extending to the center of mass of the contour, which pinpoints the fish’s head. A vector from the fish contour to the nearest barrier was also calculated per trial; comparing this vector to the fish’s heading angle allowed us to detect whether the fish approached the barrier from the left or the right.
To analyze escape direction, we used the calculated heading angle averaged over the 20 ms before tap stimulus command to rotate the XY coordinates of the escape trajectory (20ms -¿ 60ms after tap command) so that escape coordinates are calculated with the fish initiated vertically and its origin at point (0, 0). In this way, positive X coordinates reflect escapes to the right and negative X coordinates reflect escapes to the left. We considered an escape to be ”left” if the sum of escape X coordinates was negative and ”right” if positive; we considered the escape an ”avoidance” if the angle to the barrier relative to the heading angle of the fish was opposite in sign to the X coordinate sum. If the fish collided with the barrier during its escape trajectory (determined when the center of mass is within 750 microns of the barrier edge), the direction of escape was calculated only up until the collision timepoint and a collision was registered. This prevented the assignment of ”avoidance” to trajectories where fish deflect off of barriers.
For tail angle calculation (Supplementary Figure 1), the tail was split into 6 segments and cumulative angles were calculated using dot products between segment vectors with direction determined by the cross product. The timeseries of tail angle sums was filtered with a Gaussian (=1). The first time after stimulus delivery that the relative maximum or minimum of the filtered timeseries passed + / - 30 degrees was demarcated as the escape initiation time, with its magnitude recorded (Supplementary Figure 1). Note that our characterization determines latency as the time from stimulus delivery until maximum tail amplitude is reached, not until the initiation of the escape as in other reports, meaning our latencies are slightly longer than typically reported.
Navigation Assay
To record the fish’s natural responses to obstacles while navigating, larvae that had never experienced taps in the vicinity of barriers were subject to the same experimental paradigm as above (e.g. phototaxis to attract the fish to the center, whole-field gray projection once barriers have been reached), except no taps were delivered when fish neared a barrier. The XY position of the fish and barriers, and the dimensions of the barriers, were tracked and plotted in Figure 1; for heatmaps reflecting the fish’s preferred swimming distance with respect to barriers, XY coordinates of the fish for every time point during navigation were calculated with respect to the nearest barrier. The coordinate system’s origin is therefore set as the nearest barrier center over time. Visited pixel locations were binned by Gaussian filtering (= 5) for the heatmap matrix representing the coordinate system.
Mauthner Neuron Ablations
Phenylthiourea (PTU) treatment inhibits melanogenesis only over the duration of treatment so melanogenesis resumes on removal from the solution (Karlsson et al., 2001; Whittaker, 1966); we used PTU on fish before laser ablation to avoid damage caused by laser heating of melanocytes. Tg (MH 16946-Gal4 x UAS-Kaede) embryos were collected into 100 M phenylthiourea (PTU) embryo water solution. Larvae were allowed to develop in PTU solution starting at around 18 hpf until they were removed for ablation at 4 dpf. Larvae were returned to filtered fish water post-ablation and allowed to recover for two days. In practice, we found that PTU treatment decreased pigment expression but did not fully abolish pigmentation. Pigmentation increased upon removal from solution; this was necessary for detection of the tail during escape behavior. PTU treated larvae were screened on 4 dpf for minimal pigmentation and GFP expression in both Mauthner cells. Only GFP positive larvae in which both Mauthner cells were visible were used for ablation. Larvae were then embedded in 1.8 % low melting point agarose in the center of a petri dish. Mauthner neurons were localized under two-photon excitation. Laser pulsing at 800-850nm at 360 mW for .1-.3 seconds was used in 5-10 locations on the Mauthner neuron. This induced Mauthner cell explosion in a subset of fish; gross morphological deformation was visible under 950 nm illumination after successful ablations, confirming that disappearance of the cell was not due to photobleaching. Any larvae where clear explosion of the Mauthner did not occur after multiple ablation attempts were discarded. Larvae were then freed from agarose and allowed to recover for two days. They were fed with paramecia on the first day post-ablation (6 dpf) and behaviorally assayed from 7-9 dpf as described above.
Data Availability
The data and code for analysis used in this study are openly available on Github which also includes the barrier locations, sizes, and raw escape trajectories for each trial performed by all fish.
The authors would like to thank Martin Haesemeyer, Josua Jordi, Rob Johnson, Eva Naumann, Tim Dunn, Kirsten Bolton, Kristian Herrera, Clemens Riegler, and Armin Bahl for discussions concerning experimental design and interpretation. Alix Lacoste’s excellent thesis defense was especially influential in laying the foundations of this project. All members of the Engert lab were instrumental in weekly discussions. This work was funded by a U19 grant from the NIH.