Abstract
Objective Close-loop control of brain and behavior will benefit from real-time detection of behavioral events to enable low-latency communication with peripheral devices. In animal experiments, this is typically achieved by using sparsely distributed (embedded) sensors that detect animal presence in select regions of interest. High-speed cameras provide high-density sampling across large arenas, capturing the richness of animal behavior, however the image processing bottleneck prohibits real-time feedback in the context of rapidly evolving behaviors.
Approach Here we developed an open-source software, named PolyTouch, to track animal behavior in large arenas and provide rapid close-loop feedback in <8ms, ie. from detection of event to analog stimulus delivery, e.g. auditory tone, TTL pulse. This stand-alone software is written in JAVA. The included wrapper for MATLAB provides experimental flexibility for data acquisition, analysis and visualization.
Main results As a proof-of-principle application we deployed the PolyTouch for place awareness training. A user-defined portion of the arena was used as a virtual target; visit (or approach) to the target triggered auditory feedback. We show that mice develop awareness to virtual spaces, tend to stay shorter and move faster when they reside in the virtual target zone, if their visits are coupled to relatively high stimulus intensity (≥49dB). Thus, close-loop presentation of perceived aversive feedback is sufficient to condition mice to avoid virtual targets within the span of a single session (~20min).
Significance Neuromodulation techniques now allow control of neural activity in a cell-type specific manner in spiking resolution. Using animal behavior to drive close-loop control of neural activity would help to address the neural basis of behavioral state and environmental context-dependent information processing in the brain.
1. Introduction
Animal navigation is a product of close-loop neural computations [1]. The brain computes a motor plan based on not only the sensory information it gathers from the world but also its expectations given the animal’s previous experience and task-relevant requirements. For instance, a mouse navigating through a maze will alter its trajectory given its experience during prior exposures to the same arena [2–5], as motor strategies driving navigation are continuously shaped by external and internal signals [6–8]. Traditionally this close-loop control process is studied as an “open-loop” using a stimulus-response design, where a stimulus with a fixed temporal pattern evokes a behavioral or neural response that is typically analyzed offline. This provides only a correlative understanding of the inherent close-loop neural computations that underlie behavior. Instead, to causally address the neural circuits that give rise to dynamic behaviors, e.g. navigation, one needs to interfere with neural activity in the context of ongoing behavior.
Recent technological advancements have enabled researchers to close the loop [9,10] with real-time experimental control of neural [11–14] and behavioral signals [15–19]. In particular, close-loop optogenetic neural control has provided insight into cell-type-specific neuronal population dynamics in relation with seizure control [14,20,21], sensory perception [22], and spatial navigation [23]. Nevertheless, the real-time monitoring of behavior remains a challenging task for many researchers, because it demands (1) reliable event-detection at a high temporal resolution in real-time, (2) low-latency communication with peripheral devices to trigger feedback and (3) flexibility in terms of software and hardware integration. Previous animal studies have commonly used a single or array of infrared (IR) beam sensors to detect simple motion events (e.g. entering an area, lick in a reward port, [24] or a grid of sensors, including pressure sensors [25], to monitor locomotion with a delay in the millisecond range [2,3,5,26–28]) to overcome most of these limitations. Although these systems are fast and reliable, their spatial resolution is heavily limited by the number of sensors deployed. Despite the availability of various other sensors, including microwave based motion detectors [29], ultrasonic microphones [30,31], radiofrequency detectors [32], global positioning systems [33], and heat (infrared) sensors [34], majority of existing tracking systems rely on video cameras, as they provide detailed images of whole bodies [4,35–38],[39], individual limbs [40–42], face and whisker motion [43], and eye movements [44,45]. A major drawback, however, is that behavioral classification requires several image processing steps to detect and identify the object of interest which takes several tens of milliseconds using the current state-of-art algorithms and standard computing infrastructure. This typically prevents rapid, i.e. in the range of milliseconds, close-loop feedback. As an alternative, other studies have employed a hybrid touch-based imaging approach to counter the speed bottleneck of image processing pipelines [46–48]. In this approach, the animal motion is tracked on a transparent surface, where contact points cause scattering of IR light that are captured with a camera. This produces high-contrast images and requires few image processing steps, and thus enables faster image processing.
The utility of touch-based methods in prior studies has not yet been generally extended to multi-touch interfaces [49],[50], in particular for rapid close-loop feedback applications. Since the introduction of accessible multi-touch sensing technology in 2005 [51], most applications targeted human-computer interactions, including with mobile phones, tablets and virtual reality systems [52]. For this very reason, all modern computers and mobile computing devices come with built-in controller drivers for touch sensors that automatically recognize touch input. We extended the utility of this multi-touch technology to develop PolyTouch, an open-source software to track animal behavior in an open field using an IR sensor frame and triggered close-loop feedback with a delay of <8 ms, i.e. between behavioral event detection and stimulus delivery, while continuously digitizing animal (paws and/or body) location at 150 Hz. The tracking software is multi-touch capable. It includes a GUI which provides users online measures of the spatial position, traveled distance, body speed, heading direction, relative distance to any user-defined (virtual) target and basic behavioral states. The user can flexibly create any close-loop stimulus protocol that depends on locomotion variables accessible in the output file. We further provide a wrapper in MATLAB for easy integration of PolyTouch in data acquisition and analysis pipelines.
As a proof-of-principle, we tested our close-loop system in two different place awareness paradigms. First, to provide discrete (positional) feedback, PolyTouch was used to deliver tone pulses whenever a mouse entered a user-defined virtual target zone in an open field arena. We found that the animal tended to stay shorter and moved faster in the target zone if the sound intensity was relatively high (>49 dB), implicating that the animal was aware of the virtual zone. In our second paradigm, a continuous (distance) feedback tone was triggered with a frequency that scaled with the animal’s position relative to a virtual target zone with either increasing or decreasing frequencies (range: 150 – 750 Hz). We observed that the animal adapted its exploration to maximize time spent in the portion of the arena where the frequency was lowest. This approach for animal tracking and rapid sensory feedback with a latency of ~8 ms between behavioral event detection and stimulus delivery could prove useful for a broad range of systems neuroscientists studying the principles of behavior, including but not limited to the generation of sensation, perception, action, circuits of learning and memory among others.
2. Materials and Methods
2.1. Animals
Adult transgenic mice (N = 3), B6;129P2-Pvalbtm1(cr)Arbr/J were bred locally and maintained under ad libitum access to food and water. Animals were socially housed on a 12h light/dark cycle. Experimental procedures have been performed in accordance with the European Directive 2010/63/EU, guidelines of the Federation of European Laboratory Animal Science Associations, and the NIH Guide for the Care and Use of Laboratory Animals, and approved by an institutional ethics committee.
2.2. Animal tracking and close-loop instrumental control with PolyTouch
PolyTouch is an open-source software written in JAVA that enables animal tracking with real-time elementary behavioral classification while providing rapid feedback at millisecond resolution (Figure 1A-B). The software consists of tracking and feedback modules, the source code is made available via GitHub (https://github.com/DepartmentofNeurophysiology/PolyTouch). The user can run the software as a standalone program or call it in MATLAB (Mathworks) environment. It can be deployed in conjunction with any touch input device, let it be a touch screen with a fixed screen resolution, mouse, touch pen or any interface that utilizes USB Touchscreen Controller (Universal) driver. In this study, we used a standard notebook computer (iCore 5 Intel processor, 8GB DDR2 RAM) and a custom infrared touch frame (wavelength = 850 nm) with 4096 pairs of infrared emitters and detectors placed orthogonally to each other along the four edges of a frame (91 × 52.6 cm; spatial resolution = 0.050 cm). The frame was placed around an arena made of plexiglas (90.5 × 52.6 cm; plexiglass thickness = 1 cm) placed in a sound isolation chamber (Figure 1A). The setup was illuminated by two strips of white light emitting diode arrays placed on the chamber ceiling. Two sound boxes (Speedlink USB speakers, Jöttenbeck GmbH, Germany) were placed on the right- and left half of the arena to provide auditory positional feedback in two proof-of-principle place awareness paradigms (Figure 1A; Section 2.3). The spatial resolution of tracking is scaled by the monitor size of the DAQ computer monitor (23.6 inch, 1920 × 1080 pixels), but is determined by the spatial resolution of the IR sensor frame (i.e. 0.050 cm). Commercially, IR sensor frames are sold as “IR touch overframe”, “touch frames”, “multi touch frames” by various manufacturers.
The tracking algorithm makes use of open source jni4net (https://github.com/jni4net/jni4net/) and JWinPointer libraries (http://www.michaelmcguffin.com/code/JWinPointer/) to read the x and y coordinates of simultaneously detected contact points (Figure 1B). Alternative libraries, e.g. Simple Multi Touch Toolkit (SMT) (https://github.com/vialab/SMT), could be used instead of JWinPointer to provide cross-platform support. Depending on the relative distance between the ground and the sensor, different portions of the body including limbs, tail, and head can be detected. For the experiments described herein (Section 2.2-3), the sensor was placed on a flat plexiglass sheet aimed for limb detection. It performs behavioral classification based on the temporal and directional changes in body motion. The user interface displays the current animal position (in X,Y), center-of-mass (COM) position, elapsed time, distance traveled, body speed, the relative position of the (virtual) target, and basic behavioral states during data acquisition (Figure 1A). Behavioral state identification was based on motion profiling of the animal and included discrete states of animal “moving” (body speed > 1 cm/s), “immobile” (body < 1 cm/s), and advancing “on the ground” or “off the ground”, as the body part moves out of the 2D detection/sensor plane. Simple behavioral state classification allows the user to monitor animal behavior, which could also be used to trigger stimulus feedback.
PolyTouch assigns a unique ID to each simultaneously detected touch point (e.g. id 1, 2, 3 for paw 1, 2, 3) and updates the touch information (x,y position, event state, etc.) that is coupled to that particular ID until the touch point is out of range of the sensor. This means that different events over time can be registered and coupled to the same touch (e.g. paw 1 stops moving: event state changes from “mobile” to “immobile”). In turn, same events can be registered for different touches and thus different touch IDs (e.g. paw 1 and 2 stop moving: event state changes to “immobile” for the two touches). The touch events are updated and exported to an ASCII file as comma separated values at an average rate of 156.2 Hz (± s.d. = 3.5 Hz). Because the data export is restricted to only those time points when the animal is in motion, the file size is kept to a minimum (1 min of behavioral data is ~ 79 KB). The sampling rate of the animal position depends on the CPU load of the data acquisition computer as well as the serial processing of each simultaneous touch events, a constraint of the JAVA architecture. The spatial location in a 2D plane (in pixels) and timestamp (in nanoseconds) of each event are stored along with the device identity (device ID, useful in case multiple sensors are in tandem), touch event identity (touch ID), behavioral state (mobile, immobile, on the ground, off the ground), travelled distance (in cm) and body speed (in cm/s). Timestamps were computed based on the current value of the system timer at nanosecond resolution relative from the moment tracking motion was initiated by the user. To ensure that the motion of the computer mouse does not interfere with the tracking, the computer mouse is assigned to the touch ID of 1 and ignored in the rest of the processing. Note that PolyTouch is not designed for notebook computers (laptops) with touch screen displays whose resolution can be adjusted with multi touch gestures.
The feedback module controls stimulus delivery in respect to the animal position and context as determined by the user, and generates an external ASCII file to store the timestamps of triggered feedback events (in nanoseconds) that could be used to measure the cycle speed (Figure 1B). The feedback latency we describe in the current study refers to the time difference between two consecutive feedback cycles. Although we used the feedback module to deliver auditory stimulus, because the module generates an analog output using the speaker port of the computer, it could also be used to communicate with other hardware using analog signals, including TTL pulses. The feedback module continuously reads the X,Y position of the last stored touch event from the external file (write latency: ~1.01 ms, read latency: ~0.4 ms for an HDD spinning at 5400 rpm) and its speed depends on several processing times including behavioral classification (i.e. ~0.001 ms), motion parameters calculation (e.g. COM, body speed; i.e ~0.075 ms), stimulus generation and delivery (i.e. 4.6 ms). The feedback is delivered based on either user-defined scenarios or calculated measures of ongoing animal movement, providing real-time feedback to the animal, e.g. in respect to its relative distance to (virtual) target. In this study we deployed two scenarios: discrete (positional) feedback that delivers auditory stimulus with a principal frequency of 150 Hz (for the frequency spectrum; see Supplemental Figure 1) whenever the animal is in the user-defined virtual target zone; continuous (distance) feedback, which delivers a frequency modulated tone that scales with the relative distance of the animal to the virtual target zone. PolyTouch is terminated when the user-specified session duration is reached or can be interrupted anytime if the user closes the graphical user interface (GUI).
2.3. Tracking a robotic ball (roboball)
To estimate the tracking performance of PolyTouch, we simultaneously digitized the motion of a robotic ball (Sphero 2.0 ORBOTIX 3.81 cm radius) with two independent sensor frames (61.5 × 35.8 cm) and quantified the error in position estimation. The frames were stacked on top of each other (with 2 cm in between) and each connected to a separate computer to sample the position of the robot independently. To minimize the IR light scattering from the beam emitters of the other device, we rotated the top frame 180 degrees clockwise relative from the bottom frame. The robot was instructed to move according to a predefined set of heading directions (0 to 360 deg; total n directions = 72566) with a random speed between 0 to 20 cm/s for a period of 15 minutes. The given heading directions were obtained from a behavioral session recorded previously (baseline session mouse 1).
2.4. Place awareness task with virtual targets
To establish place awareness of virtual target zones, animals were placed in an open field where spatial auditory feedback was provided in two distinct paradigms. In paradigm one, animals (N = 2) received discrete auditory feedback based on the animal’s position in respect to the target zone (number of sessions = 5). In the first session, animals could freely explore the arena for 6 minutes in the absence of any auditory stimulation (0 dB; baseline condition). During the following three sessions, the feedback was provided whenever the animal entered the target zone. The sound intensity was increased across sessions (39 – 49 – 59 dB; low - intermediate - high tone condition, respectively; 15-20 min per session). The target zone location within the arena and the order of the sound intensity were randomly assigned between sessions and animals. In the final session, the tone (3× 39, 49, 59 dB; 10 s each; session duration = 15-20 min) was presented pseudorandomly, independent from the animal location, to control for the sound induced changes in animal mobility. In paradigm two, an animal (N = 1) was first placed in the open field with no auditory feedback (0 dB; baseline condition). In two subsequent sessions, continuous feedback was delivered as a frequency-modulated tone that scaled with the animal’s distance to a virtual target zone with increasing and decreasing frequencies respectively (frequency steps: 150-300-450-600-750 Hz). The open field was cleaned with ethanol (70%) after each session. Inter-session interval was 7 min until after the 3rd session. Afterward, it increased to 5 and 10 days for the remaining two sessions in the protocol.
2.5. Data analysis
2.5.1. Behavioral analyses
To assess locomotion, the body position was computed as the center-of-mass (COM) of multi-touch events for each time point. The body position was resampled at 200 Hz by averaging samples within non-overlapping but consequential 5 ms bins in 2D (X,Y). If no samples fell within a bin (i.e. no motion was detected), the previous X,Y value was assigned. The resulting matrix was used to quantify the mobility duration (s), body speed (cm/s), and body direction (deg) as a function of time (s) and body position (X,Y).
Mobility duration was represented as a spatial density map and computed as the time spent in a given location by dividing the exploration arena (91×52.6 cm) into 400 arbitrary bins (20×20 bins), resulting in a bin size of 91/20 cm/bins = 4.55 cm along X and 52.6/20 cm/bins = 2.63 cm along Y). Exploration duration (s) in a given user selected (target zone, T) zone was quantified as the time spent in zone T (versus NT, non-target zone) per entry. Body speed over time was computed as the elapsed distance (in cm) and resampled at 10 Hz with a linear interpolation method in MATLAB (non-overlapping 100 ms/window). Data points were excluded if body speed was > 50 cm/s, because we observed that almost all samples fell well under this value (data not shown). The spatial distribution of body speed was computed as a bin-average for each given location and for zone T (versus NT) as described above. The animal was considered to be immobile if the body speed was < 1 cm/s and mobile otherwise [5,53]. Body direction over time was computed as the vector angle between two subsequent COM points (see equation 1) and for each given location (10×10 bins), resulting in 91/10 cm/bins = 9.1 cm along X and 52.6/10 = 5.26 cm along Y. Body directions for immobile movements were excluded. A polar frequency histogram of body directions (bin size = 10 deg/bin) was computed as the number of observations normalized by the total number of samples per session.
To quantify the relative change in locomotion in T (versus NT), a locomotion modulation index (LMI) was computed as the difference in exploration duration (ED) per entry for T versus NT zone divided by the sum for each sample over time (see equation 2) after data is binned (bin size = 100 ms). Here, the exploration duration was interpolated over time using a linear interpolation method, so that LMI could be computed for each time point.
2.5.2 Error analyses
We performed two independent analyses for error estimation. First, to quantify the noise in spatiotemporal motion profiling, we calculated the roboball’s trajectory as described above. The sampled X,Y coordinates from the second sensor device were then rotated 180 degree clockwise around its origin (X,Y = 0,0), to align the world-centric coordinates of detected motion across the two sensor (IR touch) frames (see section 2.2). To remove noise caused by scattering light from the other IR touch frame samples (N = 40/143736 points for dev1; N = 6/32769 points for dev2) that laid outside the robot’s radius (3.81 cm) were removed. The motion trajectories were then resampled at 200 Hz and interpolated by fitting a spline across non-overlapping 50 s windows using the fit function in MATLAB (smoothing parameter = 0.8). Data points were excluded if the traversed speed was > 80 cm/s.
The two IR touch frames were connected to two separate computers that were not synchronized. Therefore the time series from the two sensors were aligned manually. Subsequently, the error in the estimated robot position was computed as the pairwise difference between the two X,Y estimates. Finally, the data was presented as Z scores (see equation 3a-b) to quantify the variance across the two sensors.
To determine the temporal variation in spatial estimation, we digitized the location of a stationary object (an orange) using a single IR sensor frame (session duration = 10 min). Temporal variation in spatial estimates was computed by taking the derivative of the COM (Supplemental Figure 2).
2.5.3 Statistical analyses
To assess place avoidance behaviour, the mobility and body speed were compared when the animal was in zone T versus NT across different tone stimulus conditions (0, 39, 49, 59, random dB) after testing for normality, using the Kolmogorov-Smirnov test. As both variables failed to pass the normality test, the non-parametric Mann-Whitney-U test was used to compare the median exploration duration per entry between zone T and NT for each condition as well as the median body speed. To reduce the risk of type I errors due to multiple testing, p-values were adjusted with a Bonferroni correction, with a critical p-value of 0.05 before correction. To evaluate if there was a bias in the animal’s spatial trajectory, the histogram of heading direction samples was tested with a goodness-of-fit chi-square test (bin size = 10 deg, number of bins = 36, critical p-value = 0.05). A post hoc Pearson’s chi-square test was used for each bin versus the sum of other bins (ratio 1:35) to determine if the observed ratio exceeded the expected ratio (number of samples / number of bins, p-values adjusted with Bonferroni correction).
3. Results
3.1. Stationary and mobile inanimate object tracking in open field
To evaluate the tracking performance of PolyTouch, we performed two independent analyses. First, we quantified the noise in spatiotemporal estimates by tracking a roboball using two IR sensor frames placed on top of each other (session duration = 15 min, Figure 2A-C) and acquired data simultaneously (see Materials and Methods for details). Noise in the raw samples was quantified as the pairwise difference of z-scores for each axes across the two sensors (Figure 2D). The mean error estimation in the X position was 0.8 cm (± standard deviation (s.d.) = 3.6 cm) and 1.0 cm in the y position (± s.d. = 3.5 cm). The proportion of samples with a significant (critical z-score = 1.96 s.d.) error estimation in the body position was 2622/169339 (1.6%) in X, and 502/169339 (0.3%) in Y. After interpolation, the mean error in X reduced to 0.3 cm (± s.d. = 3.5 cm) and 0.2 cm in Y (± s.d. = 3.4 cm). The proportion of samples with a significant error in X was 2407/169339 (1.4%), 543/169339 (0.3%) in Y. These findings show that 98.6% of contacts detected spatiotemporally well matched across sensors.
The temporal variation in the position estimate was quantified independently by tracking a stationary object (an orange) using a single IR sensor frame (session duration = 10 min). The temporal variability in COM estimates was zero pixel (Supplemental Figure 2). Taken together, our findings indicate that PolyTouch estimated the position of a stationary object with low but variable. Increased error in COM estimate during object motion is primarily because the two sensors sample the same animal across two different planes. While one sensor is placed on the ground where the animal explores, hence sensitive to animal droppings and capture larger variety of tail movements, the other sensor is elevated of off the surface. Because we calculate the error between the sensors after COM calculation, the variance in the estimate is sensitive to the different planes digitized by the two sensors.
3.2. Tracking animal navigation in open field
Next, to exemplify the utility of PolyTouch for close-loop control of animal navigation, we provided tone feedback every time the animal entered a user-defined virtual target zone. The training consisted of five sessions (t = 5-20 min/session) and is based on a modified open field task [53]. The first three sessions were delivered on the same day (average inter session interval = 7 min) and the subsequent two sessions 5 and 10 days, respectively, after the first experimental day.
Mice could freely explore the arena during the first session (t = 5 min; habituation session) without any tone feedback provided. During the subsequent three sessions, animal entry to a user-defined virtual target zone triggered a discrete 150 Hz tone at 39, 49, or 59 dB (t = 15-20 min/session). During the final session 10s tone sweeps (3x per stimulus condition) were pseudo-randomly presented to decouple the stimulus presentation from the animal’s location (t = 15-20 min).
Locomotion tracking showed that animals remained mobile, i.e. body speed > 1 cm/s, on average 81% of the first session (Figure 3A-B, 3D-E). Animals tended to walk faster along the walls and center part of the arena (Figure 3F-G), and spent the most time at the four corners (Figure 3C). This stereotypical thigmotaxic navigation discontinued in the subsequent sessions as the animals habituated to the environment and became increasingly less mobile.
To quantify whether the spatial (thigmotaxic) bias in exploration is accompanied by a motor bias, we computed the body direction (Figure 3H) and trajectory of the animals (Figure 3I). The results showed that once the navigation is initiated, animals either preserved their body orientation, continuing their forward motion, or reversed their trajectory to revisit the path they took (Figure 3H). This pattern of exploration was not spatially constrained to any given portion of the arena (Figure 3I). These results suggest during open field exploration animal behavior is intrinsically driven rather than being extrinsically instructed in this simple arena. [53–55]
3.3. Place avoidance training with real-time positional feedback
Next, to establish an instructive control over animal navigation, we provided real-time contextual feedback. Animals rapidly learned to avoid the target zone during sessions where entry to a user defined target zone triggered a moderate intensity (~59 dB) tone (Figure 4). In this close-loop positional (discrete) feedback training, animals first explored the entire arena (Figure 4B) with thigmotaxis (Figure 4B,C) before completely avoiding the target zone after ~7 min exposure (Figure 4B).
The positional feedback significantly reduced the overall speed of navigation (median ± 1st-3rd interquartile range (IQR) of body speed with and without tone feedback: 0.1 cm/s ± 0-1.8 cm/s vs 3.6 cm/s ± 1.1-8.9 cm/s, Mann-Whitney-U test, p < 0.05) except when exploring the target zone, which was increased during target zone exploration (median ± 1st-3rd IQR of body speed in the T and NT zones: 3.5 cm/s ± 1.0-8.4 cm/s vs 0.2 cm/s ± 0-2.4 cm/s, Mann-Whitney-U test, p < 0.05). Notably, the observed bias in the animal’s trajectory for forward and reverse-traversed motion in the baseline condition (Figure 4H) seemed to be reinforced in close-loop conditions (Figure 4H), suggesting that discrete positional feedback might promote motion along a course.
To quantify the temporal evolution of the avoidance behavior, we calculated a locomotion modulation index (see Methods 2.4.3). The LMI index is a normalized measure of explorative behavior that could take positive or negative values. Positive values indicate that animals spend more time (Figure 5A) in the target (T) zone (Figure 5B), while negative values are measures for non-target (NT) zone exploration. The results show that in the absence of auditory feedback (Figure 5, first column) the animals explored both T and NT throughout the entire session (see Figure 6 for group analysis), without a bias towards exploration of the either zone. Low intensity auditory (39 dB) feedback did not change the exploration duration or the speed of navigation (Figure 5, second column). At higher intensities (49 and 59 dB) mobility was directed towards the non-target zone (Figure 5, third and fourth columns) as animals avoided the T zone only after one entry. After task acquisition, the avoidance behavior was generalized. Animals avoided entry and exploration of the T zone even if auditory stimulus was not coupled to the T zone entry, rather 10 s tone sweeps were delivered randomly during exploration (Figure 5, last column).
Group analyses showed that animal mobility was modulated by both close- and open-loop auditory feedback (Figure 6). During close-loop feedback, when auditory stimulus was delivered at higher intensities, the animals explored the T zone less, avoiding the virtual target (Figure 6A; p<0.05, Mann-Whitney-U test), and moved faster (Figure 6B; p<0.05, Mann-Whitney-U test). Faster mobility was also observed during open-loop feedback (Figure 6B; p<0.05, Mann-Whitney-U test). Taken together results outlined in Figures 4-6 argue that place conditioning can be successfully induced during spatial avoidance training using virtual target zones as PolyTouch provides context/location specific and independent feedback. The place avoidance is rapidly induced in a single session and can be generalized across stimulus intensities.
3.4. Continuous positional feedback during open field navigation
Discrete two-state feedback provides animals binary information, e.g. whether or not they are in a target zone as implemented in Section 3.3. PolyTouch can also be used to provide higher dimensional and continuous feedback. We have implemented such a training protocol by continuously providing auditory feedback regarding an animal’s relevant location to a user selected virtual target zone (Figure 7). During the habituation session a mouse explored the open field (t = 5 min) without any auditory feedback. In the next two sessions, the subject received continuous tone that increased (or decreased) in frequency (150:150:750 Hz or 750:-150:150 Hz) as it approached the target (t = 10 min/session; average inter-session interval = 5-10 min).
To determine how the animal’s exploration behavior changed as a function of the distance to the target zone, we quantified the exploration duration (in s), body speed (in cm/s), and body direction (in deg) over time and for each zone in the open field. The walked trajectory revealed that the animal actively explored the arena during the baseline session (data not shown), with a proportion of time spent in a mobile state of 0.90 (body speed > 1 cm/s). The animal spent most time in zone 2 (exploration duration, ED = 104.2/298.9 s = 34.9%) and zone 5 (ED = 82.5/298.9 s = 27.6%; Figure 7J, Supplemental Figure 7A), spending the most time at the four corners.
The body speed varied across zones, but no obvious differences were observed (Supplemental Figure 7B). Similarly, the animal remained mobile in the subsequent session and spent most time in zone 2 (ED = 242.3/702 s = 34.5%) and zone 5 (ED = 250.3 /702 s = 35.6%; Figure 7K, Supplemental Figure 7A). Again, no obvious differences were observed in the body speed of the animal across zones (Supplemental Figure 7B). This may indicate that exploration behavior under close-loop conditions was similar to baseline conditions. Indeed, the heading direction of the animal’s trajectory revealed similar frequency distributions and were non-uniformly distributed (p < 0.05, Pearson’s chi-square test).
In contrast, after the first session of continuous positional feedback training the animal systematically alternated between mobile and immobile periods (proportion immobile = 0.56, body speed < 1 cm/s; Figure 7A-B, D). It spent the most time exploring the zone 5 (ED = 369 /710 s = 51.9%; Figure 7C, Supplemental Figure 7A), where he moved the slowest (Supplemental Figure 7B). The frequency distribution of the animal’s heading direction was non-uniformly distributed (p < 0.05, goodness-of-fit chi-square test, Figure 7H). Given that the lowest frequency tone was presented in zone 5, our findings may indicate that the animal preferred to stay in the area that was coupled with the lowest sound intensity feedback.
4. Discussion
PolyTouch is a novel, open source, markerless (i.e. does not require body part of animals to be tagged) tracking software with integrated feedback delivery capabilities. It continuously tracks animal locomotion while providing control signals to connected devices for close-loop control (Figure 1A). The main features of the software are (1) simultaneous tracking of multiple body parts using any IR sensor frames, connected to the data acquisition computer via the Universal USB Touchscreen Controller driver, with an average sampling rate of 66-74 Hz and fine spatial resolution of 0.050 cm as deployed in this study (the current version of the software samples at 150 Hz), (2) rapid millisecond feedback based context or animal position with an average communication latency of <8 ms, (3) low but variable noise (temporal: ~0%; spatial: <1.7% at cm pixel resolution) reconstruction of navigation over extended periods of time and (4) continuous update of the stored behavioral data with minimal memory load on the host computer resources.
PolyTouch provides users real-time visual feedback by displaying touch locations, center-of-mass (COM) body position, body speed, travelled distance, behavioral state (mobile, immobile), and distance to any user-defined virtual zone. It is fully automated and flexible, as the user can access various locomotion variables during and after data acquisition from a lightweight output file (1 min ~ 79 KB). This provides means to evaluate the occurrence, duration and timing of behavioral sequences in a standardized manner, and allows for synchronization with other experimental modules, e.g via provided MATLAB scripts for real-time data visualization, feedback control, integration with electrophysiological recordings. The recording arena can take any size, constrained only with the dimensions of the sensor frame, and could be extended to the third dimension using multiple IR sensor frames simultaneously. The open field arena can be extended with transparent walls and tunnels to create a more complex and enriched environment, which not only widens the variety of exploration behaviors that can be studied, but also benefits the animal welfare [56].
We implemented the close-loop system in two exemplary place awareness paradigms. In the first paradigm, a discrete tone was presented whenever the animal entered a user defined target zone. In this context, animals spent significantly less time and travelled faster in the target compared to non-target zone, but only if the tone intensity was sufficiently high (Figure 6A-B), indicating that place avoidance behavior emerged if the tone feedback was aversive. The tendency to avoid the target zone (which was randomized across sessions) increased with the exposure time to auditory feedback within the span of a single session (Figure 5A, 5C). This result argues that animals actively switched their place preference based on the sensory feedback within ~20 min as a result of a rapid spatial learning [57,58]. In the second paradigm, a continuous feedback about an animals’ relevant distance to the target zone was provided by modulating frequency of the tone, providing real-time positional feedback. After the initial familiarization to the paradigm, the animal spent the most time in the portion of the arena where the lowest frequency stimulus was given, maximizing the distance between the target and itself (Figure 7). Taken together, we show that PolyTouch can be used to provide real-time discrete and continuous sensory (auditory) feedback to bias animal navigation with virtual targets. The virtual targets can be spatially anchored in the arena (as it is implemented in the current study) and take any 2D shape or can be extended to 3D by using multiple sensors to track animals’ elevation. Alternatively, the targets can be coupled to animals’ own exploratory body movements, e.g. change in speed and direction of movement, to immobility, or multiple target zones can be designated to shape animal navigation. Inclusion of objects in the exploration arena and creating associated virtual targets can extend the utility of this approach, as auditory stimulus frequency and intensity could be modulated to create aversive contexts.
4.1. Limitations of the PolyTouch
The close-loop system of PolyTouch consists of a tracking and feedback module that run in parallel. The speed of tracking depends on the number of touch events detected simultaneously as each event is processed in series. Accordingly, the sampling rate was ~74 Hz when an immobile object (single point) was tracked compared to ~66 Hz in freely moving mice (up to 6 points tracked/mouse, see Figure 2A for the contact distribution). The sampling rate was constant within and across sessions, indicating that memory handling by PolyTouch does not interfere with the tracking performance.
Our sampling error analysis revealed a small (1.7%), not statistically significant, error in animal tracking when two frames were used to track the same animal (Figure 2D). A possible explanation for the observed errors could be that noise was introduced due to the close positioning of the sensor frames to image mouse navigation. This problem can be potentially solved by optically isolating the frames, either by use of spectral filters, light-tight materials.
PolyTouch does not currently support simultaneous tracking of multiple unrestrained animals. This is because center of mass calculation is performed in every time point (“frame”) independently. When multiple animals are in close proximity this algorithm will fail to identify single animals. A Bayesian clustering approach might allow identification of the animal ID based on each animal’s motion trajectory and the previous spatial distributions of individual touch events. Another solution might be to position a camera under the exploration chamber, so that a post hoc image-based analysis can be performed to assign the correct animal to touch points. Although the latter solution will limit close-loop feedback capabilities of PolyTouch to spatial feedback, as single animal behavior will not be classified in real-time for rapid feedback, the solution could be implemented with the current version of the PolyTouch.
The feedback module of PolyTouch uses the audio out jack of the computer as a communication port to deliver control signals to external devices. In this study we provided auditory feedback via connected speakers, however the control signal could take any shape, e.g. TTL.
4.2. Comparison to existing tracking systems
Compared to existing open source software for animal tracking, PolyTouch is the only software that enables rapid close-loop feedback with a flight-time of <8 ms (see Table 1 for a detailed comparison). Integration of a matrix sensor frame with wide light beam emitters (see Materials and Methods) allowed us to capture animal locomotion with a higher spatial resolution in comparison to existing infrared beam tracking systems: 0.050 cm (PolyTouch) versus 1.6 cm [57], 2.54 cm [58], 2.63 cm [59] and >5 cm [23,26,60].
In terms of processing speed, modern close-loop systems trigger feedback within 3-40 ms for neural data [11,13] and 12-50 ms for behavioral data [15,16,42,59,60]. PolyTouch can be used to control neural activity based on animal behavior at a temporal scale faster than synaptic communication along sensory pathways [61–66], and thus could be used to create artificial sensory and motor feedback in the context of rapidly evolving behavioral computations [43,67].
In recent years, many researchers have benefitted from an increase in available open-source platforms, including OpenEphys [11], BONSAI [10], NeuroRighter [68] and RTXI [69]. PolyTouch compliments the functionality of these platforms by providing rapid (within 8 ms) close-loop feedback based on animal behavior. Although our system supports data streaming of behavior to a multitude of downstream devices, it will (1) require manual coding by the user to add new threads of execution and (2) compromise in speed of processing with each extension due to its dependency on the resource utilization of the data acquisition computer (DAC). The latter issue can be solved by running PolyTouch on a separate DAC.
4.3. Future directions
PolyTouch is a flexible software whose functionality can be further improved via extension packages. To support detection of additional behaviors, such as rearing and grooming, the recording range can be extended by positioning multiple sensor frames on top of each other, so that horizontal as well as vertical movements are monitored. The user will be able to easily group locomotion data according to the position of the frame, since our software saves the device identity for each touch input in the output file. For specific behaviors that require manual observation (e.g. grooming, scratching), a future release will include the possibility for the user to report events by means of a specific key press during data acquisition.
As an extension pack, a robotic command module is currently under development that will allow users to control a robot a Sphero robot (ORBOTIX) in close-loop based on the animal’s spatial trajectory to study robot-animal interactions [18,70], foraging behavior with predator-prey interactions [19,71,72] and guided spatial learning in complex navigation tasks [73]. We hope this free, open-source, fully automatized tracking software will provide users with an alternative low-cost method to track animal behavior and close-loop control of brain and behavior.
Supplementary Figures
5. References
- 1.↵
- [2].↵
- [3].↵
- [4].↵
- [5].↵
- [6].↵
- [7].
- [8].↵
- [9].↵
- [10].↵
- [11].↵
- [12].
- [13].↵
- [14].↵
- [15].↵
- [16].↵
- [17].
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].↵
- [24].↵
- [25].↵
- [26].↵
- [27].
- [28].↵
- [29].↵
- [30].↵
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].↵
- [36].
- [37].
- [38].↵
- [39].↵
- [40].↵
- [41].
- [42].↵
- [43].↵
- [44].↵
- [45].↵
- [46].↵
- [47].
- [48].↵
- [49].↵
- [50].↵
- [51].↵
- [52].↵
- [53].↵
- [54].
- [55].↵
- [56].↵
- [57].↵
- [58].↵
- [59].↵
- [60].↵
- [61].↵
- [62].
- [63].
- [64].
- [65].
- [66].↵
- [67].↵
- [68].↵
- [69].↵
- [70].↵
- [71].↵
- [72].↵
- [73].↵