Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Parallel specification of competing sensorimotor control policies for alternative action options

Nature Neuroscience volume 19, pages 320–326 (2016)Cite this article

Subjects

Abstract

Recent theory proposes that the brain, when confronted with several action possibilities, prepares multiple competing movements before deciding among them. Psychophysical supporting evidence for this idea comes from the observation that when reaching towards multiple potential targets, the initial movement is directed towards the average location of the targets, consistent with multiple prepared reaches being executed simultaneously. However, reach planning involves far more than specifying movement direction; it requires the specification of a sensorimotor control policy that sets feedback gains shaping how the motor system responds to errors induced by noise or external perturbations. Here we found that, when a subject is reaching towards multiple potential targets, the feedback gain corresponds to an average of the gains specified when reaching to each target presented alone. Our findings provide evidence that the brain, when presented with multiple action options, computes multiple competing sensorimotor control policies in parallel before implementing one of them.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Experimental methods.
Figure 2: Hand paths on non-channel trials and movement endpoint variance for each target selection condition.
Figure 3: Scaling of feedback gains (on channel trials) across target selection conditions.
Figure 4: Group-level analysis of feedback gains.

Similar content being viewed by others

References

  1. Cisek, P. & Kalaska, J.F. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45, 801–814 (2005).

    Article  CAS  PubMed  Google Scholar 

  2. Thura, D. & Cisek, P. Deliberation and commitment in the premotor and primary motor cortex during dynamic decision making. Neuron 81, 1401–1416 (2014).

    Article  CAS  PubMed  Google Scholar 

  3. Cisek, P. Cortical mechanisms of action selection: the affordance competition hypothesis. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 1585–1599 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Baumann, M.A., Fluet, M.C. & Scherberger, H. Context-specific grasp movement representation in the macaque anterior intraparietal area. J. Neurosci. 29, 6436–6448 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. McPeek, R.M., Han, J.H. & Keller, E.L. Competition between saccade goals in the superior colliculus produces saccade curvature. J. Neurophysiol. 89, 2577–2590 (2003).

    Article  PubMed  Google Scholar 

  6. Chapman, C.S. et al. Reaching for the unknown: multiple target encoding and real-time decision-making in a rapid reach task. Cognition 116, 168–176 (2010).

    Article  PubMed  Google Scholar 

  7. Ghez, C. et al. Discrete and continuous planning of hand movements and isometric force trajectories. Exp. Brain Res. 115, 217–233 (1997).

    Article  CAS  PubMed  Google Scholar 

  8. Van der Stigchel, S., Meeter, M. & Theeuwes, J. Eye movement trajectories and what they tell us. Neurosci. Biobehav. Rev. 30, 666–679 (2006).

    Article  PubMed  Google Scholar 

  9. Stewart, B.M., Gallivan, J.P., Baugh, L.A. & Flanagan, J.R. Motor, not visual, encoding of potential reach targets. Curr. Biol. 24, R953–R954 (2014).

    Article  CAS  PubMed  Google Scholar 

  10. Stewart, B.M., Baugh, L.A., Gallivan, J.P. & Flanagan, J.R. Simultaneous encoding of the direction and orientation of potential targets during reach planning: evidence of multiple competing reach plans. J. Neurophysiol. 110, 807–816 (2013).

    Article  PubMed  Google Scholar 

  11. Shadmehr, R., Smith, M.A. & Krakauer, J.W. Error correction, sensory prediction, and adaptation in motor control. Annu. Rev. Neurosci. 33, 89–108 (2010).

    Article  CAS  PubMed  Google Scholar 

  12. Wolpert, D.M., Diedrichsen, J. & Flanagan, J.R. Principles of sensorimotor learning. Nat. Rev. Neurosci. 12, 739–751 (2011).

    Article  CAS  PubMed  Google Scholar 

  13. Scott, S.H. Optimal feedback control and the neural basis of volitional motor control. Nat. Rev. Neurosci. 5, 532–546 (2004).

    Article  CAS  PubMed  Google Scholar 

  14. Todorov, E. & Jordan, M.I. Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5, 1226–1235 (2002).

    Article  CAS  PubMed  Google Scholar 

  15. Wolpert, D.M. & Flanagan, J.R. Motor learning. Curr. Biol. 20, R467–R472 (2010).

    Article  CAS  PubMed  Google Scholar 

  16. Todorov, E. Optimality principles in sensorimotor control. Nat. Neurosci. 7, 907–915 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Cluff, T. & Scott, S.H. Rapid feedback responses correlate with reach adaptation and properties of novel upper limb loads. J. Neurosci. 33, 15903–15914 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Franklin, S., Wolpert, D.M. & Franklin, D.W. Visuomotor feedback gains upregulate during the learning of novel dynamics. J. Neurophysiol. 108, 467–478 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Diedrichsen, J. Optimal task-dependent changes of bimanual feedback control and adaptation. Curr. Biol. 17, 1675–1679 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Dimitriou, M., Franklin, D.W. & Wolpert, D.M. Task-dependent coordination of rapid bimanual motor responses. J. Neurophysiol. 107, 890–901 (2012).

    Article  PubMed  Google Scholar 

  21. Pruszynski, J.A. et al. Primary motor cortex underlies multi-joint integration for fast feedback control. Nature 478, 387–390 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Franklin, D.W. & Wolpert, D.M. Specificity of reflex adaptation for task-relevant variability. J. Neurosci. 28, 14165–14175 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Nashed, J.Y., Crevecoeur, F. & Scott, S.H. Influence of the behavioral goal and environmental obstacles on rapid feedback responses. J. Neurophysiol. 108, 999–1009 (2012).

    Article  PubMed  Google Scholar 

  24. Knill, D.C., Bondada, A. & Chhabra, M. Flexible, task-dependent use of sensory feedback to control hand movements. J. Neurosci. 31, 1219–1237 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Dimitriou, M., Wolpert, D.M. & Franklin, D.W. The temporal evolution of feedback gains rapidly update to task demands. J. Neurosci. 33, 10898–10909 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Smith, M.A., Ghazizadeh, A. & Shadmehr, R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179 (2006).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  27. Haith, A.M., Huberdeau, D.M. & Krakauer, J.W. Hedging your bets: intermediate movements as optimal behavior in the context of an incomplete decision. PLOS Comput. Biol. 11, e1004171 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  28. Gallivan, J.P. et al. One to four, and nothing more: nonconscious parallel individuation of objects during action planning. Psychol. Sci. 22, 803–811 (2011).

    Article  PubMed  Google Scholar 

  29. Hudson, T.E., Maloney, L.T. & Landy, M.S. Movement planning with probabilistic target information. J. Neurophysiol. 98, 3034–3046 (2007).

    Article  PubMed  Google Scholar 

  30. Izawa, J. & Shadmehr, R. On-line processing of uncertain information in visuomotor control. J. Neurosci. 28, 11360–11368 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Gallivan, J.P., Barton, K.S., Chapman, C.S., Wolpert, D.M. & Randall Flanagan, J. Action plan co-optimization reveals the parallel encoding of competing reach movements. Nat. Commun. 6, 7428 (2015).

    Article  PubMed  Google Scholar 

  32. Corneil, B.D., Olivier, E. & Munoz, D.P. Visual responses on neck muscles reveal selective gating that prevents express saccades. Neuron 42, 831–841 (2004).

    Article  CAS  PubMed  Google Scholar 

  33. Saijo, N., Murakami, I., Nishida, S. & Gomi, H. Large-field visual motion directly induces an involuntary rapid manual following response. J. Neurosci. 25, 4941–4951 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Gomi, H., Abekawa, N. & Nishida, S. Spatiotemporal tuning of rapid interactions between visual-motion analysis and reaching movement. J. Neurosci. 26, 5301–5308 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Prablanc, C. & Martin, O. Automatic control during hand reaching at undetected two-dimensional target displacements. J. Neurophysiol. 67, 455–469 (1992).

    Article  CAS  PubMed  Google Scholar 

  36. Day, B.L. & Lyon, I.N. Voluntary modification of automatic arm movements evoked by motion of a visual target. Exp. Brain Res. 130, 159–168 (2000).

    Article  CAS  PubMed  Google Scholar 

  37. Saunders, J.A. & Knill, D.C. Visual feedback control of hand movements. J. Neurosci. 24, 3223–3234 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Brenner, E. & Smeets, J.B. Perceptual requirements for fast manual responses. Exp. Brain Res. 153, 246–252 (2003).

    Article  PubMed  Google Scholar 

  39. Goodale, M.A., Pelisson, D. & Prablanc, C. Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement. Nature 320, 748–750 (1986).

    Article  CAS  PubMed  Google Scholar 

  40. Körding, K.P. & Wolpert, D.M. Bayesian integration in sensorimotor learning. Nature 427, 244–247 (2004).

    Article  PubMed  CAS  Google Scholar 

  41. Braun, D.A., Aertsen, A., Wolpert, D.M. & Mehring, C. Learning optimal adaptation strategies in unpredictable motor tasks. J. Neurosci. 29, 6472–6478 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Orbán, G. & Wolpert, D.M. Representations of uncertainty in sensorimotor control. Curr. Opin. Neurobiol. 21, 629–635 (2011).

    Article  PubMed  CAS  Google Scholar 

  43. Franklin, D.W., Osu, R., Burdet, E., Kawato, M. & Milner, T.E. Adaptation to stable and unstable dynamics achieved by combined impedance control and inverse dynamics model. J. Neurophysiol. 90, 3270–3282 (2003).

    Article  PubMed  Google Scholar 

  44. Crevecoeur, F., McIntyre, J., Thonnard, J.L. & Lefèvre, P. Movement stability under uncertain internal models of dynamics. J. Neurophysiol. 104, 1301–1313 (2010).

    Article  CAS  PubMed  Google Scholar 

  45. Munoz, D.P. & Wurtz, R.H. Saccade-related activity in monkey superior colliculus. II. Spread of activity during saccades. J. Neurophysiol. 73, 2334–2348 (1995).

    Article  CAS  PubMed  Google Scholar 

  46. Song, J.H. & Nakayama, K. Hidden cognitive states revealed in choice reaching tasks. Trends Cogn. Sci. 13, 360–366 (2009).

    Article  PubMed  Google Scholar 

  47. Gallivan, J.P. & Chapman, C.S. Three-dimensional reach trajectories as a probe of real-time decision-making between multiple competing targets. Front. Neurosci. 8, 215 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  48. Churchland, M.M., Santhanam, G. & Shenoy, K.V. Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach. J. Neurophysiol. 96, 3130–3146 (2006).

    Article  PubMed  Google Scholar 

  49. Manohar, S.G. et al. Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control. Curr. Biol. 25, 1707–1716 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Christopoulos, V. & Schrater, P.R. Dynamic Integration of Value Information into a Common Probability Currency as a Theory for Flexible Decision Making. PLOS Comput. Biol. 11, e1004402 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  51. Howard, I.S., Ingram, J.N. & Wolpert, D.M. A modular planar robotic manipulandum with end-point torque control. J. Neurosci. Methods 181, 199–211 (2009).

    Article  PubMed  Google Scholar 

  52. Liu, D. & Todorov, E. Evidence for the flexible sensorimotor strategies predicted by optimal feedback control. J. Neurosci. 27, 9354–9368 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Ingram, J.N., Flanagan, J.R. & Wolpert, D.M. Context-dependent decay of motor memories during skill acquisition. Curr. Biol. 23, 1107–1112 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Huang, V.S., Haith, A., Mazzoni, P. & Krakauer, J.W. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787–801 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Galea, J.M., Mallia, E., Rothwell, J. & Diedrichsen, J. The dissociable effects of punishment and reward on motor learning. Nat. Neurosci. 18, 597–602 (2015).

    Article  CAS  PubMed  Google Scholar 

  56. Baugh, L.A., Kao, M., Johansson, R.S. & Flanagan, J.R. Material evidence: interaction of well-learned priors and sensorimotor memory when lifting objects. J. Neurophysiol. 108, 1262–1269 (2012).

    Article  PubMed  Google Scholar 

  57. Ingram, J.N., Howard, I.S., Flanagan, J.R. & Wolpert, D.M. Multiple grasp-specific representations of tool dynamics mediate skillful manipulation. Curr. Biol. 20, 618–623 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank D. Franklin for providing helpful advice and suggestions. This research was supported by the Natural Sciences and Engineering Research Council of Canada, the Wellcome Trust, the Human Frontiers Science Program, and by the Royal Society Noreen Murray Professorship in Neurobiology (to D.M.W.). J.P.G. was supported by a Canadian Institutes of Health Research fellowship award.

Author information

Authors and Affiliations

  1. Department of Psychology, Queen's University, Kingston, Ontario, Canada

    Jason P Gallivan & J Randall Flanagan

  2. Department of Biomedical and Molecular Sciences, Queen's University, Kingston, Ontario, Canada

    Jason P Gallivan

  3. Centre for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada

    Jason P Gallivan & J Randall Flanagan

  4. Life Sciences Program, Queen's University, Kingston, Ontario, Canada

    Lindsey Logan

  5. Department of Engineering, University of Cambridge, Cambridge, UK

    Daniel M Wolpert

Authors
  1. Jason P Gallivan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lindsey Logan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel M Wolpert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J Randall Flanagan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jason P Gallivan or J Randall Flanagan.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gallivan, J., Logan, L., Wolpert, D. et al. Parallel specification of competing sensorimotor control policies for alternative action options. Nat Neurosci 19, 320–326 (2016). https://doi.org/10.1038/nn.4214

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4214

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing