Abstract
Discovering principles underlying the control of animal behavior requires a tight dialogue between experiments and neuromechanical models. Until now, such models, including NeuroMechFly for the adult fly, Drosophila melanogaster, have primarily been used to investigate motor control. Far less studied with realistic body models is how the brain and motor systems work together to perform hierarchical sensorimotor control. Here we present NeuroMechFly v2, a framework that expands Drosophila neuromechanical modeling by enabling visual and olfactory sensing, ascending motor feedback, and complex terrains that can be navigated using leg adhesion. We illustrate its capabilities by first constructing biologically inspired locomotor controllers that use ascending motor feedback to perform path integration and head stabilization. Then, we add visual and olfactory sensing to this controller and train it using reinforcement learning to perform a multimodal navigation task in closed loop. Finally, we illustrate more biorealistic modeling in two ways: our model navigates a complex odor plume using a Drosophila odor taxis strategy, and it uses a connectome-constrained visual system network to follow another simulated fly. With this framework, NeuroMechFly can be used to accelerate the discovery of explanatory models of the nervous system and to develop machine learning-based controllers for autonomous artificial agents and robots.
Competing Interest Statement
The authors have declared no competing interest.
Footnotes
Author added: Thomas Ka Chung Lam. Figure 1 updated with aesthetic improvements. Figure 2 modified using different simulation conditions and updated with aesthetic improvements. Figure 3 updated with aesthetic improvements; panel E moved to a separate Figure 5. Added new Results subsection "Ascending signals for path integration and head stabilization," and added new Figure 4. Figure 5 (previously Figure 3E) modified to accommodate changes made in the additional subsection ("Ascending signals"), and updated with aesthetic improvements. Added new Results section "Using more bio-realistic algorithms for sensorimotor control" and added new Figure 6. Added new Extended Data Figures 4, 5, 7. Other Extended Data Figures updated with aesthetic improvements. Improved preprogrammed stepping kinematics using untethered (instead of tethered) fly walking recordings. Added new Videos 10, 11, 13, 14. Other videos updated with aesthetic improvements. Modified video links. Abstract, Introduction, Discussion updated according to newly added subsections under Results. Moved some details from Methods to Supplementary Information. Removed some Tables and moved the information to the text in the appropriate paragraphs in Methods or Supplementary Information. Updated References. Improved mathematical notations in the Methods and Supplementary Notes to improve clarity. Converted Table 1 to Supplementary Note 1. Replaced Extended Data Figure 6A (previously Extended Data Figure 4A) with a clearer schematic. Converted Extended Data Figure 5 to Supplementary Note 2.
