Abstract
In human/machine interfaces (HMI), humans can interact with dynamic machines through a variety of sensory and motor modalities. Redundant motor modalities are known to have advantages in both human sensorimotor control and human-computer interaction: motor redundancy in sensorimotor control provides abundant solutions to achieve tasks; and incorporating diverse features from different modalities has improved the performance of gesture-, movement-, and brain-controlled computer interfaces. Our objective is to investigate whether redundant motor modalities enhance performance for a continuous trajectory-tracking task. We designed a multimodal human/machine interface with combined manual (joystick) and muscle (surface electromyography, sEMG) inputs and evaluated its closed-loop performance for tracking trajectories through second-order machine dynamics. In a human subjects experiment with 15 participants, we found that the multimodal interface outperformed the manual-only interface while performing comparably to the muscle-only interface; and that the multimodal interface enabled users to coordinate sensorimotor noise in individual modalities to improve performance. Multimodal human/machine interfaces could be beneficial in systems that require stability and robustness against perturbations such as motor rehabilitation and robotic manipulation.
Competing Interest Statement
The authors have declared no competing interest.
Footnotes
(e-mails: hachou{at}uw.edu, my13{at}uw.edu, sburden{at}uw.edu)
★ This research was funded by the U.S. National Science Foundation (Awards #1836819, 2045014).