Abstract
Animal behavior emerges from a seamless interaction between neural network dynamics, musculoskeletal properties, and the environment. Accessing and understanding the interplay between these intertwined elements requires the development of integrative neuromechanical simulations. Until now, there has been no such simulation framework for the widely studied model organism, Drosophila melanogaster. Here we present NeuroMechFly, a data-driven computational model of the adult female fly that is designed to synthesize rapidly growing experimental datasets and to test theories of neuromechanical behavioral control. NeuroMechFly combines a set of modules including an exoskeleton with articulating body parts—limbs, halteres, wings, abdominal segments, head, proboscis, and antennae—muscle models, and neural networks within a physics-based simulation environment. Using this computational framework, we (i) predict the minimal limb degrees-of-freedom needed to replay real Drosophila walking and grooming, (ii) estimate expected contact reaction forces, torques, and tactile signals during these replayed behaviors, and (iii) discover neural network and muscle parameters that enable locomotion using gaits with a variety of speeds and stabilities. Thus, NeuroMechFly is a powerful testbed for building an understanding of how behaviors emerge from interactions between complex neuromechanical systems and their physical surroundings.
1 Introduction
Uncoupling the contributions to behavior of many neuronal and biomechanical elements is daunting without the complementary guidance of systems-level numerical simulations. These ‘bottom-up’ tools can be used to consolidate data into a dynamic framework, generate predictions to be tested, and probe the sufficiency of prevailing theories to account for experimental observations [1–6]. Modeling has long played a particularly important role in the study of movement control in vertebrates [7–9] and invertebrates, including stick insects [10–13], cockroaches [14, 15], praying mantises [16], and ants [17].
However, many of these models have not yet been used in an ongoing dialogue with experimental data. This is possibly due to the limited availability of genetic tools allowing one to repeatedly record from or perturb identified neural circuits in these organisms. By contrast, for a few commonly studied ‘model’ organisms, a relatively rapid communication between experimental results and computational predictions represents an exciting but largely unrealized opportunity. This is mostly made possible by recent advances in computing power, the realism of physics-based simulations, and improvements in numerical optimization approaches; even with limited experimental data (e.g., [18]) one can reverse-engineer biomechanical systems. Neuromechanical models of some commonly studied organisms have already been developed including several for worms (Caenorhabditis elegans [19, 20]), maggots (larval Drosophila melanogaster [21]), and rodents [5, 22]. Notably absent among these is a data-driven neuromechanical model for the adult fly, Drosophila melanogaster.
Adult flies are an ideal organism for establishing a synergy between experimental and computational neuroscience. First, flies generate a large repertoire of complex behaviors including grooming [23], courtship [24], flight [25], and walking [26, 27] which they use to navigate complex environments [28]. These behaviors can now be quantified precisely using deep learning-based computer vision tools [29, 30] in 3-dimensions (3D) [31, 32]. Second, flies have a relatively small number of neurons that can be repeatedly genetically targeted [33] to perform recordings or perturbations in tethered, behaving animals [34–36]. These neurons can also be placed within their circuit context using recently acquired brain and ventral nerve cord (VNC) connectomes [37, 38]. Although we previously developed a simple physics-based simulation of adult Drosophila melanogaster to investigate hexapod locomotor gaits [39], this model has several important limitations that restrict its widespread use. Specifically, it lacks (i) the morphological accuracy required to simulate mass distributions, compliance, and physical constraints, (ii) muscle models and their associated passive dynamical properties, and (iii) neural networks or other control architectures.
Here we describe NeuroMechFly, a neuromechanical model of adult Drosophila that fills this methodological gap. NeuroMechFly is based on a detailed CT-scan of an adult female fly that was subsequently digitally rendered and rigged (Figure 1A). Using this model, we first show how a previously unreported coxa-trochanter limb degree-of-freedom (DoF) is required to accurately replay limb movements measured from real flies during walking and grooming (Figure 1B). Then, by replaying these limb kinematics in NeuroMechFly, we demonstrate how one can estimate ground reaction forces (GRFs), joint torques, and tactile contacts—quantities that cannot currently be experimentally measured in behaving flies. Finally, we illustrate how one can optimize the parameters of a central pattern generateor (CPG)-inspired coupled-oscillator network and associated torsional spring and damperbased muscle model to discover and more deeply investigate emergent locomotor gaits (Figure 1C).
(A) Body structures—morphology, joint locations, and degrees-of-freedom—are defined by x-ray microtomography. Future work may incorporate more complex muscle models. (B) Real 3D poses can be replayed using the model allowing one to predict unmeasured collisions, contact reaction forces, and joint torques. Future work may also use this model to generate synthetic data for deep learning-based 3D pose estimation. (C) Neuromuscular parameters are discovered by optimization to satisfy high-level objectives like fast and stable locomotion. Optimized networks may be more deeply analyzed to predict neural dynamics orchestrating behavior in real animals. Future work may use neural recordings and connectomes to constrain artificial neural networks.
Importantly, the NeuroMechFly framework is modular and open-source. This will enable further extensions including the use of more detailed neural and muscle models that allow one to make interpretable experimental predictions concerning real Drosophila neural circuits. Thus, NeuroMechFly represents an important step towards achieving an understanding of how behaviors emerge from a complex interplay between neural dynamics, musculoskeletal biomechanics, and physical interactions with the environment.
2 Results
2.1 Constructing a data-driven biomechanical model of adult Drosophila
Behavior depends heavily on the body’s physical constraints and its interactions with the environment. Therefore, morphological realism is critical to accurately model limb movements and their associated self-collisions, joint ranges of motion, mass distributions, and loading. To achieve this level of realism in our simulation, we measured the morphology of an adult female fly using x-ray microtomography (Video 1). We first embedded the animal in resin to reduce blurring associated with scanner movements (Figure 2A). Then we processed the resulting microtomography data (Figure 2B) by binarizing it to discriminate between foreground (fly) and background (Figure 2C). Finally, we applied a Lewiner marching cubes algorithm [40] to generate a polygon mesh 3D reconstruction of the animal’s exoskeleton (Figure 2D).
(A) An adult female fly is encased in resin for x-ray microtomography. (B) Cross-section of the resulting x-ray scan. Cuticle, muscles, nervous tissues, and internal organs are visible. (C) A threshold is applied to these data to separate the foreground (white) from the background (black). (D) A 3D polygon mesh of the exoskeleton and wings is constructed. (E) Articulated body segments are separated from one another. (F) These segments are reassembled into a natural resting pose. Joint locations are defined and constraints are introduced to create an articulated body (dark red). (G) Textures are added to increase the visual realism of the model.
Subsequently, to articulate appendages from this polygon mesh, we separated the body into 65 segments (see Table 3)(Figure 2E) and reassembled them into an empirically defined natural resting pose. Joints were added to permit actuation of the antennae, proboscis, head, wings, halteres, abdominal segments, and limb segments based on observations from high-resolution videography [31], and previously reported limb DoFs [41–43](Table 3)(Figure 2F). To facilitate the control of each DoF in the physics engine, we used hinge-type joints to connect each of the body parts. Therefore, to construct thorax-coxa joints with three DoFs, we combined three hinge joints along the yaw, pitch, and roll axes of the base link. This approach also increased the interpretability of joint angle readings. Finally, we textured the model for visualization purposes (Figure 2G). This entire process yielded a rigged model of adult Drosophila that was morphologically accurate enough for neuromechanical studies as well as for model-based computer vision tasks [44–48].
2.2 Replaying real Drosophila limb kinematics using NeuroMechFly
After constructing an articulating digital model of an adult fly, we next asked whether the six reported and implemented limb DoFs—(i-iii) thorax-coxa (ThC) elevation/depression, protraction/retraction, and rotation, (iv) coxa-trochanter (CTr) flexion/extension, (v) femur-tibia (FTi) flexion/extension, and (vi) tibia-tarsus (TiTa) flexion/extension [41, 42]—would be sufficient to accurately replay real fly limb kinematics. We did not add a trochanter-femur (TrF) joint Because the Drosophila trochanter is thought to be fused to the femur [42]. Additionally, because, for the middle and hind legs, ThC protraction/retraction occurs along a different axis than similarly named movements of the front legs, we instead use the notations ‘roll’, ‘pitch’, and ‘yaw’ to refer to rotations around the anterior/posterior, medial/lateral, and dorsal/ventral axes of articulated segments, respectively (Video 2).
For our studies, we focused on forward walking and grooming, two of the most common fly behaviors. We used DeepFly3D [31] to acquire 3D poses from recordings of tethered flies behaving spontaneously on a spherical treadmill. Due to inter-animal morphological variability and 3D pose estimation noise, we observed variations in predicted joint positions. Thus, without fixed base ThC reference positions and stable limb segment lengths, direct application of 3D poses to actuate Neu-roMechFly would be impossible. To overcome this issue, we fixed the location of ThC joints as stable reference points and set each body part’s length as its mean length for a given experiment. Then, we scaled ThC positions and the body part lengths using our neuromechanical model as a template. Thus, instead of using 3D cartesian coordinates, we could now calculate joint angles that were invariant across animals and that could match the DoFs available in NeuroMechFly. At first we calculated these joint angles for the six reported DoFs [41, 42] by computing the dot product between adjacent limb segments joined by single-rotational joints, or between the global rotational axes and limb segments for multi-rotational joints (see Materials and Methods).
Interestingly, when only these six DoFs were used to replay walking and grooming in NeuroMech-Fly, we consistently observed a large discrepancy between 3D pose-derived cartesian joint locations and those computed from joint angles via forward kinematics (Figure 3, Base DoF Dot product). Visualization of these errors showed significant out-of-plane movements of the tibia and tarsus (Video 3, top-left). This was surprising given that each limb is assumed to consist of a ball-and-socket joint (3 DoFs in the ThC joint) followed by a series of one DoF hinge joints which should result in limb segments distal to the coxa residing in the same plane. Therefore, we next tried to identify alternative limb configurations that might better match 3D poses. First we performed an inverse kinematics optimization of joint angles instead of dot product operations. This would allow us to identify angle configurations that minimize error at the most distal tip of the kinematic chain—in this case, the pretarsus. Although inverse kinematics yielded a lower discrepancy (Figure 3, Base DoF Inverse kinematics), we still observed consistent out-of-plane movements (Video 3, top-middle).
Body-length normalized mean absolute errors (MAE) comparing measured 3D poses and angle-derived joint positions for various DoF configurations. Measurements were made for representative examples of (A) forward walking, or (B) foreleg/antennal grooming. For each condition, n = 2400 samples were computed for all six legs across 4s of 100 Hz video data. Data for each leg are color-coded. ‘R’ and ‘L’ indicate right and left legs, respectively. ‘F’, ‘M’, and ‘H’ indicate front, middle, and hind legs, respectively. Violin plots indicate median, upper, and lower quartiles (dashed lines). Results from adding a coxa-trochanter roll DoF to based DoFs are highlighted in light gray.
We next examined whether an extra DoF might be needed at the CTr joint to accurately replicate real fly limb movements. This analysis was motivated by the fact that: (i) other insects use additional stabilizing rotations at or near the TrF joint [49–52], (ii) unlike other insects, the Drosophila trochanter and femur are fused, and (iii) Drosophila hosts reductor muscles of unknown function near the CTr joint [41]. To ensure that any improvements did not result simply from overfitting by increasing the number of DoFs, we also tested the effect of adding one roll or yaw DoF to each of the more distal hinge-type joints (CTr, FTi and TiTa)(Video 2). Indeed, for both walking (Video 3, top-right) and foreleg/antennal grooming (Video 4, top-right), we observed that adding a CTr roll DoF to the six previously reported (‘base’) DoFs significantly and markedly reduced the discrepancy between 3D pose-derived and forward kinematics-derived joint positions, even when compared with improvements from inverse kinematics (Figure 3, Base DoF & CTr roll; for statistical analysis, see Table 1 and Table 2). This improvement was not seen when an additional DoF was added to joints downstream of the CTr (Figure 3, Base DoF & CTr yaw, Base DoF & FTi roll, Base DoF & FTi yaw, Base DoF & TiTa roll, Base DoF & TiTa yaw). These results demonstrate that accurate kinematic replay of Drosophila limb movements requires seven DoFs per leg: the previously reported six DoFs [41, 42] along with a roll DoF near the CTr joint. Thus, by default, NeuroMechFly incorporates this additional DoF for each leg (Table 3).
2.3 Estimating joint torques and contact forces during kinematic replay
Having identified a suitable set of limb DoFs for replaying recorded Drosophila behaviors, we next investigated the extent to which we might use kinematic replay to infer otherwise inaccessible physical quantities in PyBullet [53], the physics-based simulation environment. PyBullet’s physics engine gives access to physical quantities of motion including torques, and contact forces like collisions and ground reaction forces (GRFs). Thus, because experimental measurements of these quantities remains technically challenging in small insects like Drosophila [17, 54], one might instead infer these values by replaying limb kinematics in a physics-based simulation. Although kinematic replay may not provide information about internal forces, estimates of collisions and forces remain a good first approximation of an animal’s proprioception and mechanosensation.
We explored this possibility by using a proportional-derivative (PD) controller implemented in PyBullet to actuate NeuroMechFly’s leg joints, replaying measured limb kinematics during forward walking and foreleg/antennal grooming. We used calculated joint angles and angular velocities as the target signals for the controller. Because, when applying this kind of controller, there is no unique set of contact solutions that match forces and torques to prescribed kinematics—experimental validation of force estimates is ultimately necessary—we first quantified how sensitive torque and force estimates would be to changes in PD controller gains. Based on this sensitivity analysis, we chose gain values that optimized the precision of kinematic replay (Figure S1) while also preventing large variations in measured physical quantities (Figure S2). We also maintained a naturalistic pose for the abdominal segments, wings, halteres, head, proboscis, and antennae (Table 4).
For both replayed walking (Figure 4A-C)(Video 5) and foreleg/antennal grooming (Figure 5A-C)(Video 6), we found that NeuroMechFly limb movements were largely identical to those measured from Drosophila. By visual inspection, we also noticed striking similarities between real and simulated spherical treadmill rotations. This was surprising given that the ball’s rotations were not explicitly controlled but emerged from tarsal contacts and forces in our simulation. These observations support the accuracy of our computational pipeline in processing and replaying recorded joint positions.
(A) Multiple cameras and deep learning-based 2D pose estimation are used to track the positions of each leg joint while a tethered fly is walking on a spherical treadmill. (B) Multiview 2D poses (solid lines) are triangulated and processed to obtain 3D joint positions (dashed lines). These are further processed to compute joint angles for seven DoFs per leg. (C) Joint angles are replayed in NeuroMechFly. Body segments in contact with the ground are indicated (green). (D) Estimated ground reaction force vectors (red arrows) are superimposed on original video data. (E, top) Kinematic replay of real 3D joint angles permits estimation of unmeasured (E, middle) joint torques, and (E, bottom) ground reaction forces. Only data for the right front leg (RF) are shown. Grey bars indicate stance phases when the leg is in contact with the ground. Joint DoFs are color-coded. (F) A gait diagram illustrating stance (black) and swing (white) phases for each leg as computed by measuring simulated tarsal contacts with the ground.
(A) Multiple cameras and deep learning-based 2D pose estimation are used to track the positions of each leg joint while a tethered fly grooms its forelegs and antennae. (B) Multiview 2D poses (solid lines) are triangulated and processed to obtain 3D joint positions (dashed lines). These are further processed to compute joint angles for seven DoFs per leg. (C) Joint angles are replayed in NeuroMechFly. Body segments undergoing collisions are indicated (green). (D) Estimated leg-leg and leg-antennae contact forces (red arrows) are superimposed on original video data. (E, top) Kinematic replay of real joint angles permits estimations of unmeasured (E, middle) joint torques, and (E, bottom) contact forces. Only data for the right front (RF) leg are shown. Dark grey bars indicate leg-leg contacts. Light grey bars indicate leg-antenna contacts. Joints are color-coded. (F) A grooming diagram illustrating contacts (black) made by the front leg’s five tarsal segments (‘Ta1’ and ‘Ta5’ being the most proximal and the most distal, respectively), tibia (‘Ti’), and both antennae (‘Ant’).
We next examined collisions and forces computed within the PyBullet physics-based simulation environment. From kinematic replay of joint angles during walking (Figure 4E, top), we measured rich, periodic joint torque dynamics (Figure 4E, middle). These were accompanied by ground reaction forces (GRFs) that closely tracked subtle differences in limb placement across walking cycles (Figure 4E, bottom). Superimposing these GRF vectors upon raw video recordings of the fly allowed us to visualize expected tarsal forces (Figure 4D)(Video 5, top-left) which could also be used to generate predicted gait diagrams during tethered walking (Figure 4F). Similarly, for foreleg/antennal grooming (Figure 5A-C), we observed that measured joint angles (Figure 5E, top) gave rise to closely associated joint torque dynamics (Figure 5E, middle). Associated limb and antennal contact forces (Figure 5D, E, bottom) reached magnitudes about twice the fly’s weight. Although experimental data would be required to fully validate these measurements, they fall within the range of previously observed maximum forces measured at the tip of the tibia (~100μN) [55]. These limb and antennal contact forces were used to generate grooming diagrams—akin to locomotor gait diagrams—that illustrate predicted contacts across the distal limb segments and antennae (Figure 5F). In this case, during leg-leg grooming, we observed collisions moving continuously along the limb segments in proximal to distal sweeps. These collision data provide a richer description of grooming behaviors beyond simply classifying the body part that is being cleaned [23] and open up a more precise physical quantification of this and many other behaviors.
Because we obtained 3D pose estimates from a tethered fly behaving on a spherical treadmill, we also ‘tethered’ our simulation by fixing the thorax position. Next, we tested the extent to which our model might otherwise walk without body support, keeping its balance while carrying its entire body weight. To do this, we replayed 3D kinematics from tethered walking (Figure 4)(Video 5) while NeuroMechFly could move freely (untethered) on flat terrain. Indeed, we observed that our model could walk stably on the ground, even in the face of external perturbations (Video 7). Notably, although an animal would be expected to position its front and hind legs differently on a curved versus a flat surface, the flexibility of NeuroMechFly’s tarsal segments allowed it to walk with a naturalistic pose on flat terrain using 3D poses acquired from an animal walking on a spherical treadmill. In summary, here we have shown how NeuroMechFly can be used to replay real 3D poses in order to infer otherwise unmeasured physical quantities like joint torques, collisions, and reaction forces. These predictions may then be validated and further explored experimentally.
2.4 Optimizing oscillator networks and muscles to explore locomotor gaits
Beyond estimating physical quantities from kinematic replay of measured fly behaviors—a top-down application—one can also use NeuroMechFly to discover and investigate controllers that can orchestrate high-level behaviors—a bottom-up approach. Insect walking gaits are thought to emerge from the connectivity and dynamics of networks of CPGs within the insect’s ventral nerve cord [14, 15, 56, 57], although alternative, decentralized approaches have also been proposed [13, 58]. Thus, using NeuroMechFly, one might investigate how networks of coupled CPGs and muscles generate fast and stable locomotion. Ultimately, the topology and dynamics of identified controllers can inform our understanding of real Drosophila motor circuits.
To illustrate this approach, we aimed to discover controllers that give rise to fast and statically stable locomotion. First, we designed a neural network controller consisting of a CPG-like coupled oscillator [59] for each joint (Figure 6A). For simplicity, we denoted the output of each coupled oscillator as motor neuron (MN) activity. These MNs, in turn, were connected to Ekeberg-type muscles [60], a muscle model that has been used successfully in simulations of the stick insect [10], and salamander [61]. The large number of DoFs in our model (Table 3) suggested that it might be challenging to identify suitable solutions during optimization. Therefore, we next aimed to minimize the parameter optimization’s search space. To do this, we first limited the controlled DoFs to just those that were (i) sufficient to generate walking behaviors in other insect simulations [62] and (ii) had the most pronounced effect on overall leg trajectories in our kinematic analysis (Figure S5). These were just three per leg—ThC pitch for forelegs and ThC roll for middle and hind legs, CTr pitch, and FTi pitch. Additionally, to encourage the optimizer to discover more biorealistic solutions, we constrained joint ranges of motion to those measured from kinematic replay experiments. Each DoF was driven by two coupled nonlinear phase oscillators with controlled amplitudes—one driving the extensor muscle and the other driving the antagonistic flexor. Finally, we treated both sides of the fly as symmetric. We optimized phase differences between intraleg joints and muscle parameters for actuated right legs and then mirrored these results for the left legs. In the same manner, we mirrored the phase difference parameters between flexors and extensors for inter-coxae connections (Figure 6A, pink arrows), i.e., both muscles should be equally out of phase across coxae. Thus, our 36 coupled oscillators were connected in a minimal configuration to remove redundancy and to reduce the dimensionality of the optimization search space (Figure 6A).
(A) A network of coupled oscillators modeling CPG-based intra- and interlimb circuits within the ventral nerve cord of Drosophila. Oscillator pairs control specific antagonistic leg DoFs (gray). Network parameter values are either fixed (black), modified during optimization (red), or based on oscillators on the other half of the body (pink). (B) Multi-objective optimization of network and muscle parameters maximizes the forward distance traveled (speed) and static stability of locomotion. (C) Over 60 optimization epochs, an individual solution’s locomotor trajectory gets longer. Ultimately, this solution achieves a trade-off between speed and static stability. (D) Pareto front of solutions from the final (60th) optimization generation. Four individuals were selected from the population using different criteria: the longest distance traveled (fastest, black), the shortest distance travel (slowest, green), the closest to the pareto front’s center (intermediate, purple), and the best trade-off to achieve both objectives (trade-off, blue). (E) Gait diagrams for selected solutions from generation 60. Stance (black) and swing (white) phases were determined based on tarsus ground contacts for each leg. (F) Physics-based predictions for the ‘trade-off’ solution. Motor neuron (MN) activity, joint torques, and joint angles for each leg’s coxa are shown, as well as intraleg joint angles for the right foreleg. Joints are color-coded.
Next, we designed a multi-objective optimization approach [63] to discover network and muscle parameters that give rise to hexapodal walking gaits [39]. We used two high-level objective functions: distance traveled and static stability. Notably, under some conditions, these objectives are inversely correlated in that we might expect fast gaits to be less static stable and vice versa. Forwrd speed was defined as the number of backward ball rotations within a fixed period of time. This was quantified as a fictive distance traveled (Figure 6B, top). Static stability was quantified as the minimal distance between the model’s center-of-mass and the convex hull formed by the legs in stance phase [39, 64](Figure 6B, bottom). Additionally, we defined penalties to discourage unrealistic solutions (See Materials and Methods). Because the optimizer minimizes the objective functions, we inverted the sign for both functions. Thus, during network and muscle optimization, we observed that the pareto front of best solutions evolved towards more negative values (Figure S6A). This was reflected by a steady increase in the forward distance traveled by NeuroMechFly over optimization generations (Figure 6C) (Video 8).
To more deeply investigate our results, we selected four individual solutions from the final optimization generation. These were: (i) the fastest solution which traveled the longest distance, (ii) the slowest solution that traveled the shortest distance, (iii) a solution with intermediate speed and stability, and (iv) a solution having the best trade-off between speed and static stability (Figure 6D). We calculated and visualized gait diagrams for each of these solutions. We observed that the the left hind and right front legs of the fastest solution spent reduced time in stance phase (Figure 6E, top-left). The slowest solution had the most statically stable gait, one that closely resembled a naturalistic insect tripod-gait (Figure 6E, bottom-left). The solution with intermediate speed and stability exhibited characteristics of both the fastest and slowest solutions—it was tripod-like but with slightly shorter stance periods (Figure 6E, top-right). Finally, the ‘trade-off’ solution was very similar but not identical to the intermediate solution; it exhibited small differences related to hind leg movements (Figure 6E, bottom-right). Notably, although we restricted our optimization search space, our stability penalty could favor many possible gaits that have some combination of three and four legs in stance phase. Thus, no specific solution would be expected. Therefore, the fact that optimization converged upon a tripod-like gait supports the notion that this gait may have evolved to achieve a trade-off between speed and static stability in insects [39, 64].
NeuroMechFly gives us access to neuromuscular dynamics and physical interactions with the environment that were used to drive optimized gaits. Thus, we next analyzed these underlying quantities to infer how they may have orchestrated locomotion in NeuroMechFly. We focused on the ‘trade-off’ solution discovered during optimization (Figure 6F) and began by analyzing MN activity for the coxae protractors of each leg. As expected for tripod locomotion, neural activity was mostly either in phase or perfectly antiphase among pairs of legs. Notably, however, the left front (LF) ThC joint MN activity is shifted with respect to the others (Figure 6F, top row, blue). This phase offset also propagated through the muscle model to shift LF ThC torques (Figure 6F, second row, blue), causing a premature movement of the LF leg—as indicated by its ThC joint angle (Figure 6F, third row, blue). Additionally, we asked to what extent tripod gaits might be robust against variations in intraleg coordination. For the ‘trade-off’ solution, although we observed that RF ThC and CTr joints showed normal coordination compared with that exhibited during walking (Figure 4E, top), the FTi joint was antiphase. Thus, although tripod coordination depend critically on interleg phase differences [57], it may be robust to intraleg coordination patterns. Interestingly, we observed naturalistic phase dynamics for the TiTa joint as well, even though its controlled angle was nominally fixed during these experiments. This suggests that while TiTa joint control attempted to maintain a constant position during walking, its movements were influenced by the dampening properties of muscles interacting with the environment. Finally, we explored how internal variables shift to favor each objective—speed and static stability—by comparing unilaterally similar solutions. When comparing solutions with similar speeds but different stabilities, we observed that for the more stable solution the coxae joints’ torques were better synchronized to generate tripod-like gaits (Figure S6B, C, middle and right). On the other hand, solutions with similar stabilities but varying speeds, showed an increase in coxae joint torques for faster locomotion (Figure S6B, C, left and middle). In conclusion, NeuroMechFly can be used in a bottom-up manner to identify neuromuscular solutions to high-level objectives. Here we discovered that tripod-like locomotor gaits emerge when optimizing for speed and static stability, even in the absence of limb sensory feedback.
3 Discussion
NeuroMechFly is an open source simulation framework whose various modules—body morphology, controller, muscle model, and physics environment—constantly interact with one another allowing it to generate rich in silico behaviors (Figure 7). Depending on one’s needs, we can envision several avenues for improving our model’s biological interpretability. First, to increase biomechanical realism, (i) cuticular deformations might also be modeled by modifying the exoskeleton mesh in accordance with measurements of cuticular responses to mechanical stresses and strains (i.e. Young’s modulus) [65, 66], and (ii) forces observed at small scales like Van der Waals and attractive capillary forces of footpad hairs [67] might also be modeled. Second, to increase the realism of movement control, (i) Hill-type muscle models with accurate dampening and nonlinear force generation properties might be applied based on species-specific physiological muscle properties—slack tendon lengths, attachment points, maximum forces, and pennation angles [55, 68], and (ii) controllers using integrate-and-fire, or Hodgkin-Huxley type neurons [14] might be used. These can also include sensory feedback (e.g., visual, gustatory, olfactory, and tactile) as well as central circuits for processing these signals. Third, to simulate behaviors more complex than locomotion on smooth terrain and grooming, (i) one can easily modify physical quantities like friction, gravity, and damping, (ii) introduce complex physical structures like heightfield terrains [53], (iii) and articulate wings, halteres, abdominal segments, proboscis, or antennae to model, for example, take-off [69], flight [70], odor plume following [71], courtship [24], or collective behavior [72].
A controller’s output drives muscles to move an embodied system. This system undergoes continuous interactions with the physical environment. Each of these modules can be independently modified within the NeuroMechFly simulation framework. (A) Controllers can range from abstract to physiologically realistic models. Optimization tools like Evolutionary Algorithms and Reinforcement Learning allow users to discover neuromuscular parameters that achieve high-level objectives. (B) Muscle contractions generate forces to drive joint movements. These dynamics depend on musculoskeletal geometry. Elaborate muscle models (e.g., Hill-type) require more parameters including muscle lengths and attachment points. (C) The morphology of the model—including its center-of-mass, joint DoFs, and mass distribution—shapes movement dynamics and introduces articulation constraints. (D) Rigid body dynamics including collisions and forces acting on the body are simulated using a physics engine that can also alter properties of the environment like terrain complexity, friction, and gravity.
Here we have illustrated how NeuroMechFly can be applied to two major neuroscience use cases. First, by replaying measured fly walking and grooming, we have shown how one can estimate joint torques and contact forces including ground-reaction forces, collisions, and tactile feedback. In the future, these estimates should be validated through direct force measurements [73, 74], or indirect recordings from proprioceptive and tactile neurons within the ventral nerve cord of behaving [36], or immobile [75] animals. Second, we have shown how high-level network and muscle parameter optimization can be used to discover and explore controller topologies that allow NeuroMechFly to generate fly-like behaviors. Any one of a number of approaches in a rapidly expanding machine learning toolkit (e.g., reinforcement learning and evolutionary algorithms) may be applied to this goal. The topologies and dynamics of discovered networks might then be compared with connectomics data [37, 38] and neural recordings [36]. Although here we optimized for speed and static stability during tethered locomotion, NeuroMechFly can also locomote without body support, allowing one to optimize neural networks for free behaviors. Finally, our model can also be used to create animations of behaviors that improve the training of 3D pose estimation networks for applications in computer vision [46, 48].
Ultimately, we envision that NeuroMechFly can be applied to the synthesis of multimodal experimental data, the testing of theories for neuromechanical behavioral control, and the formulation of predictions for further experimental evaluation. Several of these applications might also be combined in real-time during closed-loop experimentation. For example, ongoing recordings of an animal’s 3D poses might be replayed through NeuroMechFly to predict appendage collisions. These predictions might then direct the delivery of limb perturbations to test how proprioceptive feedback is used to achieve robust movement control. In summary, NeuroMechFly promises to accelerate our understanding of how passive biomechanics and active neural control give rise to organismic behavior, and can serve as a bridge linking fundamental biological discoveries to applications in artificial intelligence and robotics.
4 Materials and Methods
4.1 Constructing an adult Drosophila neuromechanical model
4.1.1 Preparing adult flies for x-ray microtomography
The protocol used to prepare flies for microtomography was designed to avoid distorting the exoskeleton. We observed that traditional approaches for preparing insects for either archival purposes or for high resolution microscopy, including scanning electron microscopy [76], result in the partial collapse or bending of some leg segments and dents in the exoskeleton of the thorax and abdomen. These alterations mostly occur during the drying phase and while removal of ethanol by using supercritical carbon dioxide drying reduces these somewhat, it is still not satisfactory. We therefore removed this step altogether, and instead embedded flies in a transparent resin. This resulted in only a small surface artifact over the dorsal abdominal segments A1, A2, and A3.
Flies were heavily anaesthetized with CO2 gas, then carefully immersed in a solution of 2% paraformaldehyde in phosphate buffer (0.1M, pH 7.4) containing 0.1% Triton 100, to ensure fixative penetration, and left for 24 h at 4°C. Care was taken to ensure the flies did not float on the surface, but remained just below the meniscus. They were then washed in 0.1M cacodylate buffer (2 × 3 min washes), and placed in 1% osmium tetroxide in 0.1M cacodylate buffer, and left at 4°C for an additional 24 h. Flies were then washed in distilled water and dehydrated in 70% ethanol for 48 h, followed by 100% ethanol for 72 h, before being infiltrated with 100% LR White acrylic resin (Electron Microscopy Sciences, US) for 24 h at room temperature. This was polymerised for 24 h at 60°C inside a closed gelatin capsule (size 1; Electron Microscopy Sciences) half-filled with previously hardened resin to ensure the insect was situated in the center of the final resin block, and away from the side.
4.1.2 X-ray microtomography
We glued the sample onto a small carbon pillar and scanned it using a 160 kV open type, microfocus X-ray source (L10711/-01; Hamamatsu Photonics K.K., Japan). The X-ray voltage was set to 40 kV and the current was set to 112 uA. The voxel size was 0.00327683 mm. To perform the reconstruction, we used X-Act software from the microtomography system developer (RX-solutions, Chavanod, France) obtaining a stack of 982 tiff images of 1046×1636 pixels each.
4.1.3 Building a polygonal mesh volume from processed microtomography data
First, we isolated cuticle and wings from the microtomography data using Fiji [77]. We selected 360 images from the tiff stack as the region of interest (ROI) beginning from slice 300. The tiff stack with the ROI was then duplicated. The first copy was binarized using a threshold value of 64 to isolate the cuticle. The second copy was cropped to keep the upper half of the image—where the wings are—and then binarized using a lower threshold value of 58. Finally, we applied a closing morphological operation to isolate the wings. Both binarized stacks were stored as tiff files.
We developed custom Python code to read the tiff stacks, and to fill empty holes within the body and wings. Finally, we used the Lewiner marching cubes algorithm [40] (implemented in the scikit-image package [78]) to obtain a polygon mesh for each stack. Both meshes were then exported to a standard compressed mesh storage format.
4.1.4 Separating and reassembling articulated body parts
We used Blender (Foundation version 2.81 [79]) to clean and manipulate polygon meshes obtained from microtomography data.
After importing these meshes into Blender, we removed noise by selecting all vertices linked to the main body (or wings), inverting the selection, and deleting these vertices. We explored the resulting meshes, looking for spurious features, and then manually selected and deleted the related vertices. To obtain 65 body segments (Table 3), we manually selected and deleted vertices from our imported 3D body and wing models. Segments were then separated at joint locations based on published morphological studies [80].
Each wing was separated into an individual segment from the wing model. The body model was separated into 63 segments as described below. The abdomen was divided into five segments according to tergite divisions. The first and second tergites were combined as the first segment (A1A2), and the last segment (A6) included the sixth to tenth tergites. Each antenna was considered a single segment and separated from the head capsule at the antennal foramen. Both eyes and the proboscis were separated from the head. The latter was divided into two parts, the first containing the rostrum (Rostrum), and the second containing the haustellum and labellum (Haustellum). Each leg was divided in eight parts: the coxa, trochanter/femur, tibia, and five tarsal segments. The thorax was considered a single segment and only the halteres were separated from it.
Each segment was processed in Blender to obtain closed meshes. First, a remesh modifier was used in ‘smooth mode’, with an octree depth of 8, and a scale of 0.9 to close the gaps generated in the meshes after been separated from the original model. Smooth shading was enabled and all disconnected pieces were removed. Then, we used ‘sculpt mode’ to manually compensate for depressions/collapses resulting from the microtomography preparation, or from separating body segments.
Then, all segments were copied into a single *.blend file and rearranged into a naturalistic resting pose (Figure 2F). We made the model symmetric to avoid inertial differences between contralateral legs and body parts. For this, we used the more detailed microtomography data containing the right side of the fly. First, the model was split along the longitudinal plane using the bisect tool. Then the left side was eliminated and the right side was duplicated and mirrored. Finally, the mirrored half was repositioned as the left side of the model, and both sides of the head capsule, rostrum, haustellum, thorax, and abdominal segments were joined.
At this point, the model consisted of approximately nine million vertices, an intractable number for commonly used simulators. We therefore used the decimate tool to simplify the mesh and collapse its edges at a ratio of 1% for every segment. This resulted in a model with 87,000 vertices that conserved the most important details but eliminated some bristles and cuticular textures.
4.1.5 Rigging the Blender model
We added an Armature object alongside our model to build the skeleton of the fly. To actuate the model, we created a ‘bone’—a tool in Blender that is used to animate characters—for each segment. Bones were created such that the thorax would be the root of the skeleton and each bone would be the child of its proximal bone, as indicated in Table 3. Then, the bones were positioned along the longitudinal axis of each segment with their heads and tails over the proximal and distal joints, respectively. Each joint was positioned at a location between neighboring segments. Each bone inherited the name of its corresponding mesh.
We used the Custom Properties feature in Blender to modify the properties of each bone. These properties can be used later in a simulator to e.g., define the maximum velocity, or maximum effort of each link. Furthermore, we added a limit rotation constraint (range of motion) to each axis of rotation (DoF) for every bone. The range of motion for each rotation axis per joint was defined as – 180° to 180° to achieve more biorealistic movements. Because, to the best of our knowledge, there are no reported angles for these variables, these ranges of motion should be further refined once relevant data become available. The DoF of each bone (segment) were based on previous studies [41, 70, 81] (see Table 3). Any bone can be rotated in Blender to observe the constraints imposed upon each axis of rotation. These axes are defined locally for each bone.
Finally, we defined a ‘zero-position’ for our model. Most bones were positioned in the direction of an axis of rotation (Figure S4). Each leg segment and the proboscis were positioned along the Z axis. Each abdominal segment and the labellum were positioned along the X axis. Wings, eyes, and halteres were positioned along the Y axis. The head and the antennae are the only bones not along a rotational axis: the head is rotated 20° along the Y axis, and the antennae are rotated 90° with respect to the head bone. Positioning the bones along axes of rotation makes it easier to intuit a segment’s position with its angular information and also more effectively standardizes the direction of movements.
4.1.6 Exporting the Blender model into the Bullet simulation engine
We used a custom Python script in Blender to obtain the name, location, global rotation axis, range of motion, and custom properties for each bone. As mentioned above, the axes of rotation are defined locally for each bone. Therefore, our code also transforms this information from a local to a global reference system, obtaining the rotation matrix of each bone.
We used a Simulation Description Format (SDF, http://sdformat.org/) file to store the model’s information. This format consists of an *.xml file that describes objects and environments in terms of their visualization and control. The SDF file contains all of the information related to the joints (rotational axes, limits, and hierarchical relations) and segments (location, orientation, and corresponding paths of the meshes) of the biomechanical model. We can modify this file to add or remove segments, joints, or to modify features of existing segments and joints. To construct joint DoFs, we used hinge-type joints because they offer more freedom to control individual joint rotations. Therefore, for joints with more than 1 DoF, we positioned in a single location as many rotational joints as DoFs needed to describe its movement. The parenting hierarchy among these extra joints was defined as roll-pitch-yaw. The mass and collision mesh were related to the segment attached to the pitch joint—present in every joint of our model. The extra segments were defined with a zero mass and no collision shape.
Our model is based upon the physical properties of a real fly. The full body length and mass of the model are set to 2.8 mm and 1 μg, respectively. Rather than having a homogeneous mass distribution that would decrease the stability of our model without body support, we used different mass values (and densities) measured in a previous study [64] for the following body parts: head (0.125 μg), thorax (0.31 μg), abdomen (0.45 μg), wings (0.005 μg), and legs (0.11 μg). In this way, the center of mass and the rigid-body dynamics of the model were made more similar to those of a real fly.
Since small-sized objects (e.g. less than 15 cm in PyBullet) are susceptible to simulation errors due to computational numeric overflows, we scaled up the units of mass and length when setting up the physics of the simulation environment, and then scaled down the calculated values when logging the results. Therefore, the physics engine was able to compute the physical quantities without numerical errors, and the model could also more accurately reflect the physics of a real fly.
4.2 Kinematic replay and analysis
4.2.1 Measuring 3D poses during forward walking and foreleg/antennal grooming
Data for kinematic replay of forward walking and foreleg/antennal grooming were taken from a previous study describing DeepFly3D, a deep learning-based 3D pose estimation tool [31]. These data consist of images from seven synchronized cameras obtained at 100 fps (https://dataverse. harvard.edu/dataverse/DeepFly3D). Time axes (Figure 4E, F and Figure 5E, F) correspond to time points from the original, published videos. Forward walking replay data were obtained from experiment #5 of animal (#8) expressing aDN-GAL4 only. Foreleg and antennal grooming replay data were obtained from experiment #3 of animal #6 expressing aDN-GAL4 driving UAS-CsChrimson. We used DeepFly3D v0.4 [31] to obtain 3D poses from these images. 2D poses were examined using the GUI to perform manual corrections on 36 frames during walking and 128 frames during grooming.
4.2.2 Processing 3D pose data
DeepFly3D, like many other 3D pose estimators, uses a local reference system based on the cameras’ positions to define the animal’s pose. Therefore, we first defined a global reference system for NeuroMechFly from which we could compare different experiments taken from different animals (see Figure S4).
Aligning both reference systems consisted of six steps. First, we defined the mean position of each Thorax-Coxa (ThC) keypoint as fixed joint locations. Second, we calculated the orientation of the vectors formed between the hind and middle coxae on each side of the fly with respect to the global x-axis along the dorsal plane. Third, we treated each leg independently and defined their ThC joints as their origin. Fourth, we rotated all data points on each leg according to its side (i.e., left or right) and the previously obtained orientations. Fifth, we scaled the real fly’s leg lengths for each experiment to fit NeuroMechFly’s body size. A scaling factor was calculated for each leg segment as the ratio between its mean length throughout the experiment and the template’s segment length. Then, each data point was scaled using the corresponding scaling factor. Finally, we use the NeuroMechFly exoskeleton as a template to position all coxae within our global reference system; the exoskeleton has global location information for every joint. Next, we translated each data point for each leg (i.e. CTr, FTi, and TiTa joints) with respect to the ThC position based on the template.
4.2.3 Calculating joint angles from 3D poses
We considered each leg as a kinematic chain and calculated the angle of each DoF to reproduce real poses in NeuroMechFly. We refer to this process as ‘kinematic replay’. Angles were obtained by computing the dot product between two vectors with a common origin. We obtained 42 angles in total, seven per leg. The angles’ names correspond to the rotational axis of the movement—roll, pitch, or yaw—for rotations around the anterior-posterior, mediolateral, and dorsoventral axes, respectively.
The thorax-coxa joint (ThC) has three DoFs. The yaw angle is measured between the dorsoventral axis and the coxa’s projection in the transverse plane. The pitch angle is measured between the dorsoventral axis and the coxa’s projection in the sagittal plane. To calculate the roll angle, we aligned the coxa to the dorsoventral axis by rotating the kinematic chain from the thorax to the FTi joint using the yaw and pitch angles. Then, we measured the angle between the anterior-posterior axis and the projection of the rotated FTi in the dorsal plane.
Initially, we considered only a pitch DoF for the CTr joint. This was measured between the coxa and femur’s longitudinal axis. Subsequently, we discovered that a CTr roll DoF would be required to match the kinematic chain. To calculate this angle, we rotated the tibia-tarsus joint (TiTa) using the inverse angles from the coxa and femur and measured the angle between the anterior-posterior axis and the projection of the rotated TiTa in the dorsal plane.
The pitch angle for the FTi was measured between the femur and tibia’s longitudinal axis. The pitch angle for the TiTa was measured between the tibia and tarsus’s longitudinal axis. The direction of rotation was calculated by the determinant between the vectors forming the angle and its rotational axis. If the determinant was negative, the angle was inverted.
To demonstrate that the base six DoFs were not sufficient for accurate kinematic replay, we also compared these results to angles obtained using inverse kinematics. In other words, we assessed whether an optimizer could find a set of angles that could precisely match our kinematic chain using only these six DoFs. To compute inverse kinematics for each leg, we used the optimization method implemented in the Python IKPy package (L-BFGS-B from Scipy). We defined the zero pose as a kinematic chain and used the angles from the first frame as an initial position (seed) for the optimizer.
4.2.4 Calculating forward kinematics and errors with respect to 3D poses
To quantify the contribution of each DoF to kinematic replay, we used the forward kinematics method to compare original and the reconstructed poses. Since noise in 3D pose estimation causes leg segment lengths to vary, we first fixed the length of each segment as its mean length across all video frames.
We then calculated joint angles from 3D pose estimates with the addition of each DoF (see previous section). Next, we formed a new kinematic chain including the new DoF. This kinematic chain allowed us to compute forward kinematics from joint angles, which were then compared with 3D pose estimates to calculate an error. We performed an exhaustive search to find angles that minimize the overall distance between each 3D pose joint position and that joint’s position as reconstructed using forward kinematics. The search spanned between −90° to 90° from the ‘zero pose’ in 0.5° increments.
The error between 3D pose-based and angle-based joint positions per leg was calculated as the average distance across every joint. To normalize the error with respect to body length, we measured the distance between the antennae and genitals in our Blender model (2.88mm). Errors were computed using 400 frames of data: frames 300-699 for forward walking and frames 0-399 for foreleg/antennal grooming.
We ran a Kruskal-Wallis statistical test to compare the kinematic errors across the eight methods used. We then applied a posthoc Conover’s test to perform a pairwise comparison. We used the Holm method to control for multiple comparisons. The resulting p-value matrices for walking and foreleg/antennal grooming behaviors are shown in Table 1 and Table 2, respectively. Our statistical tests suggested that adding a CTr roll DoF uniquely improved kinematic replay in comparison to all other methods.
4.2.5 Transferring 3D pose data into the NeuroMechFly reference frame
To incorporate the additional CTr roll DoF into NeuroMechFly, we enabled rotations along the Z axis of the CTr joints. Then, we recreated the SDF configuration files using custom Python scripts such that the resulting files would contain a CTr roll DoF for each leg.
To simulate the fly tethering stage used in our experiments, we added three support joints (one per axis of movement) that would keep our model in place. However, we removed these supports during the ground walking experiments (Video 7). To create a spherical treadmill in PyBullet, we added a spherical object with a diameter equal to that of our real foam sphere (10 mm). The simulated spherical treadmill has three joints along the x, y, and z axes, that allow the ball to rotate freely. The mass of each link was empirically set to 50 μg which permitted the simulated fly to control the ball with its legs. We did not attempt to incorporate the mass of the real spherical treadmill because it is floating on an aerodynamically complex, stream of air.We used position control for each joint in the model. We fixed the position of non-actuated joints to the values shown in Table 4. The actuated joints, i.e. the leg joints, were controlled to achieve the angles calculated from 3D pose data. The simulation was run with a time step of 1 ms, allowing PyBullet to accurately perform numerical calculations. Since the fly recordings were only captured at 100 fps, we up-sampled and interpolated pose estimates to match simulation time steps before calculating joint angles.
4.2.6 Application of external perturbations
To test the stability of the untethered model walking over flat ground, we introduced external perturbations. Specifically, we propelled solid spheres at the model according to the following equation of motion,
where, 
is the 3D target position(fly’s center of mass), 
is the initial 3D position of the sphere, 
is the initial velocity vector, 
is the external acceleration vector due to gravity in the z-direction, t is the time taken by the sphere to reach the target position 
from 
with an initial velocity 
. The mass of the sphere was 3 mg. Spheres were placed at a distance of 2 mm from the fly’s center of mass in the y-direction. With t set to 20 ms, the initial velocity of the projectile was computed using Equation 1. The spheres were propelled at the model every 0.5 s. Finally, at 3 s into the simulation, a sphere with 3g was propelled at the fly to topple it.
4.2.7 Analyzing NeuroMechFly contact and collision data
The PyBullet physics engine provides forward dynamics simulations and collision detections. We plotted joint torques as calculated from PyBullet. To infer ground reaction forces (GRFs), we computed and summed the magnitude of normal forces resulting from contact of each tarsal segment with the ball. Gait diagrams were generated by thresholding GRFs; a leg was considered in stance phase if its GRF was greater than zero.
Self-collisions are disabled by default in PyBullet. Therefore, for the kinematic replay of grooming, we enabled self-collisions between the tibia and tarsal leg segments, as well as the antennae. We recorded normal forces generated by collisions between (i) the right and left front leg, (ii) the left front leg and left antenna, and (iii) the right front leg and right antenna. Grooming diagrams were calculated using the same rationale as for gait diagrams: a segment experienced a contact/collision if it reported a normal force greater than zero.
4.2.8 Controller gain sensitivity analysis
We achieved kinematic replay using a built-in PD position controller in PyBullet [53] that minimizes the error:
where θr and θa denote reference and actual positions, ωr and ωa are desired and actual velocities, and Kp and Kν are proportional and derivative gains, respectively.
Because the outputs of our model—dynamics of motion—depend on the controller gains Kp and Kν, we first systematically searched for optimal gain values. To do this, we ran the simulation’s kinematic replay for numerous Kp and Kν pairs, ranging from 0.1 to 1.0 with a step size of 0.1 (i.e., 100 simulations in total). Target position and velocity signals for the controller were set as the calculated joint angles and the joint angular velocities, respectively. To compute the joint angular velocities, we used a Savitzky-Golay filter with a first-order derivative and a time-step of 1 ms on the joint angles. Feeding the controller with only the joint angles could also achieve the desired movements of the model. However, including the velocity signal ensured that the joint angular velocities of the fly and the simulation were properly matched. We then calculated the mean squared error (MSE) between the ground truth—joint angles obtained by running our kinematic replay pipeline on the pose estimates from DeepFly3D [31]—and joint angles obtained in PyBullet. Then, we averaged the MSE values across the joints in one leg, and summed up the mean MSEs from each of 6 legs to obtain a total error. We made the same calculations for the joint angular velocities as well. Our results (Figure S1) show that our biomechanical model can replicate real 3D poses while also closely matching real measured velocities. In particular, an MSE of 18.7 rad2/sec corresponds approximately to 3.1 rad/sec per joint. This is acceptable given the rapid, nearly 30 Hz, leg movements of the real fly.
After validating the accuracy of kinematic replay, we performed a sensitivity analysis to measure the impact of varying controller gains on the estimated torques and ground reaction forces. This analysis showed that torques and ground reaction forces are highly sensitive to changing proportional gains (Kp) (Figure S2) but are robust to variations in derivative gain (Kν). These results are expected since high proportional gains cause “stiffness” in the system whereas derivative gains affect the “dampening” in a system’s response. We observed rapid changes in estimated torques and ground reaction forces at high Kp values (Figure S2).
As shown in Figure S1, our model can match the real kinematics closely for almost every controller gain combination except for the low Kν band. By contrast, varying the gains proportionally increased the torque and force readings. Because there are no experimental data to validate these physical quantities, we selected gain values corresponding to intermediate joint torques and ground contact forces (Figure S2). Specifically, we chose 0.4 and 0.9 for Kp and Kν, respectively. These values were high enough to generate smooth movements, and low enough to reduce movement stiffness.
4.3 Neural network parameter optimization
4.3.1 CPG network architecture
To discover hexapodal locomotor gaits in our neuromechanical model, we designed a CPG-based controller composed of 36 nonlinear oscillators (Figure 6). A similar approach was previously applied to study salamander locomotion [59]. The CPG model is governed by the following system of differential equations:
where the state variables—phase and amplitude of the oscillator i—are denoted as θ¿ and r¿, respectively; νi and Ri represent the oscillator i’s intrinsic frequency and amplitude, ai is a constant. The coupling strength and phase bias between the oscillator i and j are denoted as wij and ϕij.
For the entire network of coupled oscillators, we set the intrinsic frequency ν to 3 Hz, the intrinsic amplitude R to 1, and the constant ai to 25. To ensure a faster convergence to a phase-locked regime between oscillators, we set coupling strengths to 1000 [82]. Mi represents the cyclical activity pattern of neuronal ensembles activating the muscles. We solved this system of differential equations using the explicit Runge-Kutta method of 5th-order with a time step of 1ms.
Each oscillator pair sends cyclical bursts to flexor and extensor muscles that antagonistically move the corresponding revolute joint. Since only three degrees of freedom are controlled during the optimization, there are three pairs of oscillators per leg, summing up to 36 in total. We only considered three DoFs per leg because this has been shown to be sufficient for locomotion in previous hexapod models [62]. Therefore, we selected the three joint DoFs that have the most pronounced joint angles (Figure S5). These DoFs included (i) ThC pitch for the front legs, (ii) ThC roll for the middle and hind legs, as well as (iii) CTr pitch and FTi pitch for all legs. We coupled (i) the intraleg oscillators in a proximal to distal chain, (ii) the interleg oscillators in a tripod-like fashion (the ipsilateral front and hind legs to the contralateral middle leg from anterior to posterior), (iii) both front legs to each other, and (iv) coxa extensor and flexor oscillators. We visualized the resulting network architecture in an XML-based file format (graphml). Intraleg coordination is equally important to generate a fly-like gait since stance and swing periods depend on the intrasegmental phase relationships. For this reason, both interleg (phase relationships between ThC joints) and intraleg (phase relationships within each leg) couplings were determined by the optimizer for one side of the body and replicated on the other side.
4.3.2 Muscle model
We adapted the Ekeberg muscle model [60] to generate torques on the joints. This model simulates muscles as a torsional spring and damper system, allowing torque control of any joint as a linear function of the motor neuron activities driving antagonist flexor (MF) and extensor (ME) muscles controlling that joint. The torque exerted on a joint is given by the equation:
where α, β, γ, and δ represent the gain, stiffness gain, tonic stiffness, and damping coefficient, respectively [61]. Δφ is the difference between the current angle of the joint and its resting pose and 
is the angular velocity of the joint. This muscle model makes it possible to control the static torque and stiffness of the joints based on the muscle coefficients—α, β, γ, δ, and Δφ—which are determined by optimization.
4.3.3 Defining joint limits for the biomechanical model
We did not define joint limits for kinematic replay experiments because, to the best of our knowledge, joint angle limits have not been reported for Drosophila melanogaster. However, we did define joint limits during optimization to reduce search space dimensionality and to facilitate optimizer convergence. These limits were based on angles measured for kinematic replay of forward walking. The upper and lower bounds for each joint were defined as the mean angle of that joint ± one standard deviation. Using these constraints on joint angles, we created a new pose configuration (SDF) file. In addition, we fixed the position of the non-actuated joint DoFs as their mean joint angle during walking (Table 5). We used the same angles for kinematic replay of the remaining non-actuated joints (Table 4). Furthermore, we set the same limits and fixed positions for both sides of the body because the network is symmetrical: parameters for one side of the body are mirrored on the other side.
4.3.4 CPG network and muscle parameter optimization
To identify neuromuscular networks that could coordinate fast and statically stable locomotion, we performed optimization of the phase differences for each network connection, and five parameters controlling the gains and resting position of each spring and damper muscle (i.e., α,β,γ,δ, and Δφ). To simplify the problem for the optimizer even further, we (i) fixed the ThC flexor-extensor phase differences to 180°, making them perfectly antagonistic, (ii) mirrored the phase differences from the right leg oscillators to the left leg oscillators, making gaits symmetric, (iii) mirrored muscle parameters from the right joints to the left joints, and (iv) mirrored phase differences from ThC-ThC flexors to ThC-ThC extensors. Thus, 62 parameters remained flexible during optimization: 5 phases between ThC CPGs, 12 phases between intraleg CPGs (i.e. ThC-FTi extensor/flexor, FTi-TiTa extensor/flexor per leg), and 45 muscle parameters (five per joint). We empirically set the lower and upper bounds for the parameters so that leg movements would stay stable along the boundaries (Table 6).
For parameter optimization, we used an NSGA-II optimizer [83], a multi-objective genetic algorithm implemented in Python using the jMetalPy package [84]. We defined two objective functions. First, we aimed to maximize locomotor speed, as quantified by the distance traveled by the fly in a specific period of time. This, in turn, was quantified through ball rotations (Equation 7) along the Y axis. Second, we aimed to maximize the static stability of locomotor gaits. In small animals like Drosophila, static stability is a better approximation for overall stability than dynamic stability [64]. We measured static stability as follows. First, we identified a convex hull formed by the legs in stance phase. If there were less than 3 legs in stance and a convex hull could not be formed, the algorithm returned −1, indicating static instability. Then, we measured the closest distance between the fly’s center of mass and the edges of the convex hull. Finally, we obtained the minimum of all measured distances at that time step. If the center of mass was outside the convex hull, we reversed the sign of the minimum distance to indicate instability. Because the optimizer works by minimizing objective functions, we inverted the sign of output distance and stability values.
Three penalties were added to the objective functions. First, to make sure the model was always moving, we set a threshold for the angular rotation of the ball, increasing from −0.4 rad to at least 4.0 rad in 3 seconds, a value that will guarantee almost one revolution of the ball at the end of a simulation run. Second, to avoid high torque and velocities at each joint, we set joint angular velocities to an upper limit of 100 rad/sec, a value measured from kinematic replay experiments. Third, because the most stable position for the model would be to have all of its legs on the ball, we penalized the amount of time spent by the legs in stance phase. If the average number of the legs in stance from the start of the simulation until that time step was less than 3 or larger than 3.6, we set the penalty function as the absolute difference between the average number of legs in stance and the minimum number of legs, i.e. 
.
The optimization was formulated as follows
with the following penalty terms
where Rb is the ball radius, θb,|| is the angle of the ball in the direction of walking, ttot is the maximum simulation duration, and 
is the mean number of legs in stance phase. All the penalties are multiplied with their corresponding weights and added to the cost function. Cost functions were evaluated for 3 (ttotal) seconds, a time that was sufficiently long for the model to generate locomotion.
Each optimization generation consisted of a population of 30 individuals. Optimization runs lasted for 60 generations, an acceptable computing time of approximately 2.5 hours per run on an Intel(R) Core(TM) i9-9900K CPU at 3.60GHz. Mutations occurred with a probability of 1.0, and a distribution index of 1.0. We set the cross-over probability to 1.0 and the distribution index to 15 (for more details see [84]). These values allowed variations within populations and more exploration of possible gaits while also not diverging too much from parent solutions.
4.3.5 Analysis of optimization results
Following optimization, we selected four individual solutions from the last generation for further analysis. First, the objective functions were normalized with respect to their maximum and minimum values. Then, solutions were selected as follows
Where dg and ss are the vectors containing the distance and stability values, respectively, from all individuals in a given generation g.
We read-out and plotted motor neuron activation patterns (as represented by the couple oscillators’ outputs), joint torques, joint angles, GRFs, and ball rotations from this final generation of solutions. GRFs were used to generate gait diagrams as described in the previous sections. Ball rotations were used to reconstruct the models’ paths across optimization generations. Briefly, the distances travelled along the longitudinal (x) and transverse (y) axes were calculated from the angular displacement of the ball according to the following formula
where Δθt and Δθl denote the angular displacement around the transverse and longitudinal axes, respectively, and r is the radius of the ball.
5 Supplementary Tables
Matrix of p-values from pairwise comparisons of position errors after calculating forward kinematics for walking. Numbers in bold (except in the case of identity) indicate p-value > 0.001 or no statistical difference.
Matrix of p-values matrix from pairwise comparisons of position errors after calculating forward kinematics for grooming. Numbers in bold (except in the case of identity) indicate p-value > 0.001 or no statistical difference.
Model body parts and degrees-of-freedom between each segment and its parent
Fixed angles for body joints during kinematic replay and optimization
Fixed angles for leg joints during optimization (deg)
Lower and upper limits for the muscle parameters during optimization
6 Supplementary Figures
Proportional gain (Kp) reduces the error in (A) joint angles and (B) joint velocities by acting on the error itself at the expense of increasing the stiffness of the system. Derivative gain (Kν) doesn’t affect estimated errors as much as Kp. Chosen Kp and Kν values are indicated in blue. These values were high enough to keep the error sufficiently small and low enough to avoid stiffness and large variations in joint torques.
(A) Estimated torque values during forward locomotion as a function of proportional gain values (Kp) while the velocity gain (Kν) is fixed at 0.9. Graphs show measurements for right leg ThC pitch DoFs. Measurements for the contralateral legs were nearly symmetrical. (B) Ground reaction force measurements of right side legs during forward walking as a function of varying Kp values. Results obtained from the selected Kp and Kν values are shown in red.
Each leg is composed of 11 hinge joints. Joints with more than one DoF were modeled as a union of multiple hinge joints. Front leg observed from (A) front and (B) side views. The global coordinate system’s x, y, and z axes are red, green, and blue, respectively.
Each body segment (Table 3) is aggregated using hinge joints. Rotational axes of joints and their global coordinate system are shown. (A) Zero pose from (A) front and (B) side views. The global coordinate system’s x, y, and z axes are red, green, and blue, respectively.
Joint angles for the (A) left and (B) right legs measured from a real fly during forward walking. Only the three DoFs with the highest amplitudes (solid lines) were controlled during optimization—for the front legs: ThC pitch, CTr pitch, and FTi pitch; for the middle and hind legs: ThC roll, CTr pitch, and FTi pitch DoFs. The remaining four DoFs (dashed lines) for each leg did not have pronounced angular changes. Therefore, they were fixed to their mean values during optimization.
(A) The pareto front for solutions from four optimization generations. Later generations show more negative values because the optimizer aims to minimize the objective functions. Three individual solutions were selected for deeper analysis: two with similar stability values (indicated by red and blue arrows), and two with similar speed/distance values (indicated by blue and orange arrows). (B) Gait diagrams from selected individual solutions. Stance (black) and swing (white) phases were calculated by reading-out tarsus ground contacts for each leg. The more stable gaits exhibit tripod-like coordination. (C) Underlying neuromuscular dynamics for each individual solution. The fastest solution (left) and the more statically stable solution (middle) exhibit synchronized ThC joint torques (most leg pairs are either in phase or antiphase). Torque magnitudes varied considerably to achieve higher speeds (left) but were more constant for slower, statically stable gaits (right).
7 Supplementary Videos
Video 1: Constructing a data-driven biomechanical model of adult Drosophila. An adult female fly is encased in resin for x-ray microtomography. The resulting x-ray microtomography data reveals cuticle, muscles, nervous tissues, and internal organs. These data are thresholded to separate the foreground from background. Then the exoskeleton is voxelized into a 3-dimensional polygon mesh. Articulated body segments are separated from one another and then reassembled into a natural pose. Bones are added and rigged to permit actuation. Finally, textures are added to the model for visualization purposes. https://www.dropbox.com/s/9ck0ik1g4yyu6lt/Video1.mp4?dl=0
Video 2: Visualization of possible additional limb degrees-of-freedom. NeuroMechFly’s left-middle leg is sequentially actuated along individual DoFs that may be further be required to accurately replay real fly limb kinematics. The articulated joint (e.g., ‘CTr’) and type of movement (‘roll’) are indicated. https://www.dropbox.com/s/12882qeczxw3kkp/Video2.mp4?dl=0
Video 3: The effect of additional degrees-of-freedom on the accuracy of replaying forward walking using forward kinematics. Measured 3D poses (solid lines) and forward kinematic replay (dashed lines) for forward walking. Forward kinematics are determined either (top-left) using no additional degrees-of-freedom (Base DoF, dot product), (top-middle) instead using inverse kinematics to optimize joint angles and minimize error with only base degrees-of-freedom (Base DoF, inverse kinematics), or (top-right and bottom row) by adding a single new DoF (BaseDoF & ‘joint’ ‘DoF’). Legs are color-coded. https://www.dropbox.com/s/fama81thvy4y9xr/Video3.mp4?dl=0
Video 4: The effect of additional degrees-of-freedom on the accuracy of replaying fore-leg/antennal grooming using forward kinematics. Measured 3D poses (solid lines) and forward kinematic replay (dashed lines) for foreleg/antennal grooming. Forward kinematics are determined either (top-left) using no additional degrees-of-freedom (Base DoF, dot product), (top-middle) instead using inverse kinematics to optimize joint angles and minimize error with only base degrees-of-freedom (Base DoF, inverse kinematics), or (top-right and bottom row) by adding a single new DoF (BaseDoF & ‘joint’ ‘DoF’). Legs are color-coded. https://www.dropbox.com/s/jq9g8tu6fm66eue/Video4.mp4?dl=0
Video 5: Kinematic replay of Drosophila forward walking using NeuroMechFly. (topleft, ‘Raw data’) A tethered adult fly is shown walking on a spherical treadmill. One of six synchronized camera views is shown. Data are replayed at 0.2x real time. (bottom-left, ‘2D tracking’) 2D poses (filled circles) and connecting ‘bones’ (lines) are superimposed for the proximal three legs. (bottom-right, ‘3D reconstruction’) These six 2D poses are triangulated to obtain 3D poses. Overlaid are triangulated 3D poses (solid lines) and 3D poses obtained by solving forward kinematics from joint angles (dashed lines). (top-right, ‘Kinematic replay’) These 3D joint angles actuate NeuroMechFly limb movements as it walks on a simulated spherical treadmill. Tarsal contacts with the ground are indicated (green). Estimated ground reaction force vectors for the proximal three legs are superimposed upon the original raw 2D video data (top-left). https://www.dropbox.com/s/73w2meqrquxkjeb/Video5.mp4?dl=0
Video 6: Kinematic replay of Drosophila foreleg/antennal grooming using NeuroMech-Fly. (top-left, ‘Raw data’) A tethered adult fly is shown grooming on a spherical treadmill. One of six synchronized camera views is shown. Data are replayed at 0.2x real time. (bottom-left, ‘2D tracking’) 2D poses (filled circles) and connecting ‘bones’ (lines) are superimposed for the proximal three legs. (bottom-right, ‘3D reconstruction’) These six 2D poses are triangulated to obtain 3D poses. Overlaid are triangulated 3D poses (solid lines) and 3D poses obtained by solving forward kinematics from joint angles (dashed lines). (top-right, ‘Kinematic replay’) These joint angles actuate NeuroMechFly limb movements as it grooms on a simulated spherical treadmill. Leg segments and antennal collisions are indicated (green). Estimated collision force vectors for the front legs and antennae are subsequently superimposed upon the original raw 2D video data (top-left). https://www.dropbox.com/s/j3f4ijxsiknr7ic/Video6.mp4?dl=0
Video 7: Kinematic replay of tethered Drosophila forward walking using NeuroMechFly behaving on flat terrain without body support. Pose estimates obtained from a real tethered fly walking on a spherical treadmill are replayed in NeuroMechFly as it walks untethered on flat terrain without body support. Simulated spheres are projected at the model to demonstrate its stability against perturbation. https://www.dropbox.com/s/yug5bd0s22vpk13/Video7.mp4?dl=0
Video 8: Locomotor gaits improve across optimization generations. Forward walking gaits from four individual solutions shown across optimization generations 1, 15, 30, 45 and 60. Tarsal contacts with the ground are indicated (green). Videos are replayed at 0.3x real time. Solutions were selected from each generation according to the following criteria: the fastest / longest distance traveled (top-left), the closest to the center of the pareto front (top-right), the best trade-off to achieve both speed and static stability (bottom-left), and the slowest / shortest distance traveled (bottomright). https://www.dropbox.com/s/leutjkpj1xaey2b/Video8.mp4?dl=0
8 Code and data availability
Data and code are available at: https://github.com/NeLy-EPFL/NeuroMechFly https://nely-epfl.github.io/NeuroMechFly
9 Funding
PR acknowledges support from an SNSF Project grant (175667), and an SNSF Eccellenza grant (181239). VLR acknowledges support from the Mexican National Council for Science and Technology, CONACYT, under the grant number 709993. ST acknowledges support from the European Union’s Horizon 2020 research and innovation program under grant agreement nos. 720270 (SGA1), 785907 (SGA2). JA acknowledges support from the Human Frontier Science Program (HFSP-RGP0027/2017).
11 Author Contributions
V.L.R. - Conceptualization, Methodology, Software, Validation, Formal Analysis, Investigation, Data Curation, Validation, Writing – Original Draft Preparation, Writing – Review & Editing, Visualization.
P.G.O. - Conceptualization, Methodology, Software, Validation, Formal Analysis, Investigation, Data Curation, Validation, Writing – Review & Editing, Visualization.
S.T.R. - Conceptualization, Methodology, Software, Validation, Writing – Review & Editing, Visualization.
J.A. - Conceptualization, Methodology, Software, Validation, Writing - Review & Editing.
A.J.I. - Conceptualization, Methodology, Resources, Writing – Review & Editing, Supervision, Project Administration, Funding Acquisition.
P.R. - Conceptualization, Methodology, Resources, Writing – Original Draft Preparation, Writing – Review & Editing, Supervision, Project Administration, Funding Acquisition.
12 Competing interests
The authors declare that no competing interests exist.
10 Acknowledgments
We thank Stéphanie Clerc Rosset and Graham Knott (Biological Electron Microscopy Facility, EPFL, Lausanne, Switzerland) for preparing Drosophila melanogaster samples for X-ray microtomography.
References
- [1].↵
- [2].
- [3].
- [4].
- [5].↵
- [6].↵
- [7].↵
- [8].
- [9].↵
- [10].↵
- [11].
- [12].
- [13].↵
- [14].↵
- [15].↵
- [16].↵
- [17].↵
- [18].↵
- [19].↵
- [20].↵
- [21].↵
- [22].↵
- [23].↵
- [24].↵
- [25].↵
- [26].↵
- [27].↵
- [28].↵
- [29].↵
- [30].↵
- [31].↵
- [32].↵
- [33].↵
- [34].↵
- [35].
- [36].↵
- [37].↵
- [38].↵
- [39].↵
- [40].↵
- [41].↵
- [42].↵
- [43].↵
- [44].↵
- [45].
- [46].↵
- [47].
- [48].↵
- [49].↵
- [50].
- [51].
- [52].↵
- [53].↵
- [54].↵
- [55].↵
- [56].↵
- [57].↵
- [58].↵
- [59].↵
- [60].↵
- [61].↵
- [62].↵
- [63].↵
- [64].↵
- [65].↵
- [66].↵
- [67].↵
- [68].↵
- [69].↵
- [70].↵
- [71].↵
- [72].↵
- [73].↵
- [74].↵
- [75].↵
- [76].↵
- [77].↵
- [78].↵
- [79].↵
- [80].↵
- [81].↵
- [82].↵
- [83].↵
- [84].↵
