Three-dimensional contact point determination and contour reconstruction during active whisking behavior of the awake rat

The rodent vibrissal (whisker) system has been studied for decades as a model of active touch sensing. There are no sensors along the length of a whisker; all sensing occurs at the whisker base. Therefore, a large open question in many neuroscience studies is how an animal could estimate the three-dimensional location at which a whisker makes contact with an object. In the present work we simulated the exact shape of a real rat whisker to demonstrate the existence of a unique mapping from triplets of mechanical signals at the whisker base to the three-dimensional whisker-object contact point. We then used high speed video to record whisker deflections as an awake rat whisked against a peg and used the mechanics resulting from those deflections to extract the contact points along the peg surface. A video shows the contour of the peg gradually emerging during active whisking behavior.

Rats and mice can obtain detailed tactile information by rhythmically sweeping their whiskers back 48 and forth against surfaces and objects in the environment, a behavior called "whisking." They can 49 use this whisker-based tactile information to determine an object's location, size, orientation, and 50 texture [1-6].

52
How rats achieve these tasks is still an open question, especially given that a whisker is simply a 53 cantilever beam with no sensors along its length. Numerous neurophysiological and behavioral 54 studies have specifically investigated how a rodent might use a single whisker to determine the 55 location of a vertical peg [5,[7][8][9][10][11][12][13][14][15][16][17][18][19][20]. Studies have shown that although barrel cortex is required for 56 peg localization [13], knowledge of instantaneous whisker position is not [14]. To date, however, 57 studies have not been able to determine the exact physical cues that the animal might use for 58 peg localization, in part because they have been limited to a two-dimensional (2D) analysis of 59 whisker motion and object contact, with the third dimension sometimes attributed to whisker 60 identity [8,9].

62
Complementing the biological literature, several studies in the field of robotics have investigated 63 the problem of whisker-object contact point determination in three dimensions [21][22][23]24 ]. These 64 studies have focused on the use of quasistatic mechanical signals -three reaction forces and 3

83
2.1. Problem statement: mapping mechanical signals at the whisker base to the 3D 84 whisker-object contact point location 85 When a rat whisks against an object, as depicted in Figure 1A, the contact point between the 86 whisker and the object is denoted by the coordinates (rwobj, θwobj, ϕwobj) relative to the whisker 87 basepoint, where the subscript "wobj" stands for whisker-object [25]. The whisker's deflection 88 causes reaction forces and moments (torques) at the whisker base, denoted as Fx, FY, FZ, Mx, MY, 89 and MZ. The force Fx is called the "axial" force because it acts directly along the whisker's long 90 axis at the whisker base. The axial force is positive when it pulls the whisker directly out of the 91 follicle and negative when it pushes the whisker directly into the follicle. The forces Fy and Fz are 92 called "transverse" forces, because they act perpendicular ("transversely") to the whisker at the 93 whisker base. Mx is called the "twisting" moment because it twists the whisker about its long axis,

94
while My and Mz are called the "bending" moments because they cause the whisker to bend.

96
During exploratory behavior, the rat must solve the inverse problem, illustrated in Figure 1B: it 97 must use the mechanical signals at the whisker base to determine the 3D contact point location.

98
Under the assumption of quasistatic contact, it can be theoretically shown that the six mechanical 99 signals, Fx, FY, FZ, Mx, MY, and MZ are always sufficient to uniquely determine the 3D contact point 100 location (rwobj, θwobj, ϕwobj) [22,24].

102
This theoretical result, however, leaves several important questions unanswered. First, are all six 103 mechanical signals really needed, or might only a subset of them be sufficient to determine the 104 3D contact point location? Second, assuming that mappings were found that successfully mapped 105 between mechanical signals and contact point, what is the nature of these mappings, and could 106 they be used during real-world exploratory behavior to determine the contours of an object? The 107 answers to these questions directly constrain the neural computations that might permit object 108 localization.

110
The present work was undertaken to answer these questions, and the procedure used is depicted 111 in the flowchart of Figure 1C. The 3D shape of a real whisker is obtained, and the whisker is then 112 simulated to be deflected to a gridded sampling of contact points across its entire reachable 113 space. For each deflection, the forces and moments at the base of the whisker are computed, 114 and, for convenience, the transverse forces and the bending moments are rewritten in terms of 115 their magnitude and direction. Next, each possible triplet of the six mechanical variables is 116 investigated to determine in which regions it yields a unique mapping to the (rwobj, θwobj, ϕwobj) 117 contact point. The best mapping (i.e., the one that is unique in the largest region) is selected.

119
In parallel, high speed video is used in behavioral experiments to obtain the 3D shape of the same 120 whisker as an awake rat whisks against a peg. For each video frame the whisker's deflected 121 shape is used to compute the forces and moments at the whisker base. The best mapping -122 obtained from the simulation steps described above -is then applied in each video frame to obtain 123 an estimate of the 3D contact point for that frame.

127
The 3D contact point between the whisker and the object is denoted as rwobj, θwobj, and ϕwobj. This contact 128 point exerts a force on the whisker (Fapplied), which generates reaction forces and moments at the whisker 129 base. (B) The inverse problem: the rat's nervous system must perform the inverse of the process depicted 130 in (A). It must use the forces and moments at the whisker base as inputs to deduce the contact point

137
Following the procedure depicted in Figure 1C, we began by characterizing the 3D shape of the 138 first whisker in the C-row, traditionally called the "gamma" whisker ( Figure 2A). We then simulated 139 deflecting the whisker to a gridded sampling of the 3D point locations it could reach and computed 140 the resulting forces and moments at the whisker base. The set of reachable contact points is 141 depicted as a gray point cloud about the whisker in Figure 2B. For each gray point in Figure 2B,

142
we computed the six forces and moments at the whisker base (Fx, FY, FZ, Mx, MY, MZ). We then 143 decomposed the transverse forces and the bending moments into their magnitude and direction:

144
Magnitude of the transverse force: Direction of the transverse force: Magnitude of the bending moment: Direction of the bending moment: These six signals, , , , , , and , are plotted in the six panels of Figure 2C. Each 149 panel of Figure 2C shows the same black whisker and the same cloud of points as in Figure 2B,

154
Two notable trends are that , , and all exhibit the largest magnitude for proximal contacts 155 and large angles of deflection and that and are always offset 90° from each other. These 156 effects are unsurprising.

158
The axial force, , has a small region of positive values; in these regions the axial force is pulling 159 the whisker out of the follicle instead of pushing it in. These values occur for distal contacts that 160 are concave forward with small deflections, i.e., the region in which the contact "straightens out" 161 the whisker.

163
The transverse force, , generally follows the same trends as and , but it also has a 164 hollow "tube" of zero magnitude surrounding the whisker; the "ring" of its bottom end is visible in 165 same relative region where has a large magnitude negative value. This tube occurs when the 166 whisker is bent such that the portion of the whisker local to the contact point is parallel to the y-z 167 plane; in this case the force points entirely in the negative x-direction, resulting in zero . Contact 168 points that deflect the whisker beyond this tube are defined as "large deflection" contacts. This 169 flip into large deflections can also be seen by the sudden 180° change in .

171
The twisting moment, , exhibits very different trends from the other forces and moments. The 172 magnitude primarily varies in the z-direction rather than radially. This effect occurs because the 173 whisker exhibits more "twist" and therefore greater magnitude as it is deflected out of the x-y 174 plane.

176
Overall, these trends in forces and moments are similar, but not identical, to those for an idealized, 177 planar, tapered whisker with a parabolic shape [23]. We therefore anticipated that we would see  Table S1).

202
Each of these successful mappings can be visualized with a set of three colored, solid shapes.

203
However, these visualizations are unintuitive and challenging to understand. To provide intuition 204 for how to visualize a mapping we show an example using the ( , , ) triplet. Figure 3 205 depicts the gradual construction of the three solids that represent the mapping between ( , ,

208
The three panels in the left column of Fig. 3A show the whisker in black along with all the contact 209 points it could reach at three different radial distances: 11 mm, 15 mm, and 24 mm. These 210 distances correspond to 28%, 38% and 62% of the whisker arc length, respectively.

212
The three panels in the right column of Fig. 3A show , , and computed from the contact 213 points at the three different radial distances. In each of these three right panels, the points 214 representing , , and are connected to form a single continuous surface. Each surface 215 is monochromatic, indicating that all points within that surface are generated from contact points 216 at the same radial distance. The surface corresponding to rwobj = 15 mm is a different color from 217 the surface corresponding to rwobj = 11 mm, and its shape is "shrunk down" on the and 218 axes. Similarly, the surface for , , and corresponding to rwobj = 24 mm is yet a third color 219 and its shape has shrunk even more in and .

241
As expected, the solids in columns 2 through 4 of Figure 3D all have the same shape; they differ 242 only in coloring. The "feathered edges" most noticeable on the visualization for rwobj are a 243 8 discretization artifact and do not have any significance. Using a larger number of values for rwobj 244 would cause the feathered edges to coalesce into the identical shape as those for θwobj, and ϕwobj.

245
Close visual examination of the solids in Figure

252
Importantly, however, due to human fallibility, visual inspection is necessary but not sufficient to 253 determine if a particular mapping is unique. In addition to visual inspection, we tested uniqueness 254 using neural networks as non-linear function solvers (details provided in Supporting Information).

255
If a neural network could solve for a mapping, then the mapping was unique. Specifically, given 256 (FX, MB, MD) as inputs and (rwobj, θwobj, ϕwobj) as outputs, the network had to be able to solve for the 257 non-linear function that maps between inputs and outputs with sufficiently small errors. In other 258 words, the neural network effectively generates a "look-up table" for the mapping, and the look-259 up table must be unique.

261
We found that -as expected -the look up table generated using (

293
The mappings described in the previous sections were obtained purely from simulation. The 294 simulations assumed that the whisker was rigidly clamped at its base and underwent ideal, 295 frictionless point-deflections.

297
It is not at all evident that the mappings obtained from simulation will apply during active whisking 298 behavior of an awake rat. The mapping results shown in Figure 3D could be degraded by many 299 nonlinear effects, including tissue compliance [11] and non-ideal multi-point or sliding contact [26].

300
The mappings are therefore only of theoretical interest unless the associated lookup tables can 301 be successfully applied to real-world whisking behavior.

303
To address this important concern we recorded ~3.5 seconds of high-speed video as an awake 304 rat whisked against a vertical peg (2.7 mm diameter). During this particular trial of whisking, the 305 rat first whisked forward against the back of the peg. The whisker then slipped past the peg, and 306 the rat whisked backwards against the front of the peg. Fig. 4A shows the top and front views of 307 the rat whisking forward and backward against the peg with the tracked whisker traced in red.

309
For each frame of video, we used the tracked 3D shape of the whisker and the tracked location

315
The three mechanical signals, , , and , were then used as the inputs to the mapping 316 established in Figure 3D. During this particular trial of whisking, the signals spanned a more 317 limited range than that shown in Figure 3D. Therefore, Figure 4C illustrates only the region of the 318 mappings relevant to this particular whisking trial. Each of the mapping solids in Figure 4C is

323
In Figure

328
The quality of the predicted values for (rwobj, θwobj, ϕwobj) was evaluated by plotting them against 329 the ground-truth tracked values for (rwobj, θwobj, ϕwobj); results are shown in Figure 4E.   the rat happened to whisk against the peg using the proximal portion of its whisker.

364
The next step was to convert these contact points to the laboratory frame, schematized in Figure   365 5B. Figure 5C is the identical schematic, but the peg has now been replaced with whisker-object 366 contact points. The black dots in Figure 5C are the ground-truth contact points tracked directly 367 from video, while the magenta dots represent the reconstruction of the peg as determined from 368 the mappings. Notice that the "ground truth" as described here still contains tracking error; it is 369 not the outline of the peg.

371
As expected, the black dots form two vertical lines: one on the back of the peg and one on the 372 front of the peg. The reconstructed contact points are best visualized in Supplementary Video 2.

373
The magenta dots match relatively well with the tracked peg points; however, the top view (inset)

403
The first approach involves measuring rates of change of one or more mechanical variables at 404 the whisker base, most typically the bending moment. Specifically, it can be shown that the rate 405 of change of bending moment is related to the radial distance of contact [5,11,[29][30][31][32][33]. The 406 second approach, used in the present work, involves combining multiple geometric variables [11] 407 or mechanical signals [5,33] in a nonlinear manner, independent of the rates of change of these 408 signals. The major advantage of the second approach over the first is that it is history independent

412
A recent behavioral study strongly suggested that animals make at least some use of the second

420
Which mechanical signals can be combined so as to uniquely determine the 3D whisker-object

442
Generalizing, these results demonstrate that a rodent could use mechanical information entering 443 the follicle from a single whisker to determine the 3D contact point location in whisker-centered 444 coordinates. The mapping between mechanical signals and 3D geometry exists independent of 445 whisking phase or velocity, that is, the mapping remains constant for a given whisker across all

15
whisks. In addition, the 3D contact point can be determined at all points in times during a 447 deflection, which means that the approach can continuously solve for 3D contact location as the 448 whisker sweeps across an object surface or even deflects against a compliant object. We note 449 that this work is a direct and natural extension of the two-dimensional mapping from (FX, MB) to 450 (rwobj, θwobj) described in previous work [33].

453
The two main limitations to the whisker model used, Elastica3D, are that it assumes quasistatic 454 and frictionless conditions.

456
The quasistatic assumption will affect the accuracy of the mappings only if the dynamic response

471
The present study has focused on a single gamma whisker, with a single set of parameters. What 472 parameter changes affect the mappings and their uniqueness? The mappings will undoubtedly 473 change for whiskers that have different shapes, but the question is how large these changes will 474 be, and whether the FX, MB, MD mapping will retain its uniqueness.

476
It is reasonable to assume that almost all naturally occurring whiskers will be tapered [40-43], be 477 largely (but not entirely) planar [44,45], and have a curvature that is mostly well-described by 478 either a quadratic or cubic equation [44][45][46]. Trimming the tip of the whisker, as might occur 479 through natural damage or barbering, would have no effect on mapping uniqueness. The only 480 change will be that the whisker cannot reach as large a region of space. Similarly, changing

481
Young's modulus will scale all mechanical variables at the whisker base but not affect mapping 482 uniqueness. Changing the radius of the whisker base, or changing the whisker arc length, will 483 have no effect on mapping uniqueness, provided that the ratio of the base radius to tip radius 484 remains constant and rwobj is measured as a fraction of the whisker arc length (instead of in terms 485 of absolute distance).

487
Changing the radius slope of the whisker (defined as the difference between the base and tip 488 radii, divided by the arc length) will change the base-to-tip radius ratio, which will affect the 489 mappings in highly nonlinear and often unpredictable ways. However, the mapping will remain 490 unique for the FX, MB, MD triplet as long the whisker retains sufficient taper. If the taper is very 491 slight, the mapping will theoretically be unique, but the resolution required to distinguish (rwobj, 492 θwobj, φwobj) might be so high as to be impractical [23]. If the whisker is cylindrical, the mapping will 493 be non-unique [23].

495
Based on these considerations, we expect unique mappings to hold for nearly all naturalistic 496 whisker shapes. Mappings will be non-unique if there is very large in-plane curvature (e.g., if the whisker curves so much that it doubles back on itself), if there is very large out-of-plane curvature 498 (e.g., if the whisker is a corkscrew), or if the whisker has no taper. With the exception of these 499 cases, the problem of mapping "uniqueness" may be more accurately posed as a problem of 500 mapping resolution. For example, Figure 4E indicates that high error occurs when rwobj, θwobj are 501 small. Future work will help resolve the origin of these errors and shed light on the extent to which 502 the FX, MB, MD mapping will generalize across arbitrary whisker shapes.

517
The results of the present work pave the way to develop robots that can perform accurate 3D

530
Ethics statement: All procedures involving animals were approved in advance by the Institutional

531
Animal Care and Use Committee (IACUC) of Northwestern University.

534
All whiskers on the left side of the face of a female Long-Evans rat (3-6 months) were trimmed 535 except for the Gamma whisker. The rat was body-restrained, and two orthogonally-mounted high 536 speed video cameras recorded whisking behavior against a vertical peg.

538
The

567
From the 3 seconds of data of a rat actively whisking into a peg, we used the tracked undeflected

575
In order to generate force and moment data for the mappings, we deflected the whisker to points