Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

Fusion of Video and Inertial Sensing Data via Dynamic Optimization of a Biomechanical Model

View ORCID ProfileOwen Pearl, View ORCID ProfileSoyong Shin, Ashwin Godura, View ORCID ProfileSarah Bergbreiter, View ORCID ProfileEni Halilaj
doi: https://doi.org/10.1101/2022.11.15.516673
Owen Pearl
1Doctoral Student in Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Owen Pearl
Soyong Shin
1Doctoral Student in Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Soyong Shin
Ashwin Godura
2Undergraduate Student in Electrical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sarah Bergbreiter
3Professor of Mechanical Engineering, Electrical Engineering, the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Sarah Bergbreiter
Eni Halilaj
4Assistant Professor of Mechanical Engineering, Biomedical Engineering, the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Eni Halilaj
  • For correspondence: ehalilaj@andrew.cmu.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

ABSTRACT

Inertial sensing and computer vision are promising alternatives to traditional optical motion tracking, but until now these data sources have been explored either in isolation or fused via unconstrained optimization, which may not take full advantage of their complementary strengths. By adding physiological plausibility and dynamical robustness to a proposed solution, biomechanical modeling may enable better fusion than unconstrained optimization. To test this hypothesis, we fused video and inertial sensing data via dynamic optimization with a nine degree-of-freedom model and investigated when this approach outperforms video-only, inertial-sensing-only, and unconstrained-fusion methods. We used both experimental and synthetic data that mimicked different ranges of video and inertial measurement unit (IMU) data noise. Fusion with a dynamically constrained model improved estimation of lower-extremity kinematics by a mean ± std root-mean-square error of 6.0° ± 1.2° over the video-only approach and estimation of joint centers by 4.5 ± 2.8 cm over the IMU-only approach. It consistently outperformed single-modality approaches across different noise profiles. When the quality of video data was high and that of inertial data was low, dynamically constrained fusion improved joint kinematics by 3.7° ± 1.2° and joint centers by 1.9 ± 0.5 cm over unconstrained fusion, while unconstrained fusion was advantageous by 3.0° ± 1.4° and 1.2 ± 0.7 cm in the opposite scenario. These findings indicate that complementary modalities and techniques can improve motion tracking by clinically meaningful margins and that data quality and computational complexity must be considered when selecting the most appropriate method for a particular application.

Competing Interest Statement

The authors have declared no competing interest.

Footnotes

  • Updated to most recent version to reflect submitted materials (minor fixes to title page, figure references, etc.)

  • https://github.com/CMU-MBL/IMUVisionBiomechanics.git

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-ND 4.0 International license.
Back to top
PreviousNext
Posted November 17, 2022.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Fusion of Video and Inertial Sensing Data via Dynamic Optimization of a Biomechanical Model
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Fusion of Video and Inertial Sensing Data via Dynamic Optimization of a Biomechanical Model
Owen Pearl, Soyong Shin, Ashwin Godura, Sarah Bergbreiter, Eni Halilaj
bioRxiv 2022.11.15.516673; doi: https://doi.org/10.1101/2022.11.15.516673
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Fusion of Video and Inertial Sensing Data via Dynamic Optimization of a Biomechanical Model
Owen Pearl, Soyong Shin, Ashwin Godura, Sarah Bergbreiter, Eni Halilaj
bioRxiv 2022.11.15.516673; doi: https://doi.org/10.1101/2022.11.15.516673

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Biophysics
Subject Areas
All Articles
  • Animal Behavior and Cognition (4223)
  • Biochemistry (9101)
  • Bioengineering (6748)
  • Bioinformatics (23930)
  • Biophysics (12081)
  • Cancer Biology (9488)
  • Cell Biology (13726)
  • Clinical Trials (138)
  • Developmental Biology (7614)
  • Ecology (11654)
  • Epidemiology (2066)
  • Evolutionary Biology (15473)
  • Genetics (10613)
  • Genomics (14291)
  • Immunology (9454)
  • Microbiology (22773)
  • Molecular Biology (9066)
  • Neuroscience (48831)
  • Paleontology (354)
  • Pathology (1479)
  • Pharmacology and Toxicology (2560)
  • Physiology (3820)
  • Plant Biology (8307)
  • Scientific Communication and Education (1467)
  • Synthetic Biology (2288)
  • Systems Biology (6168)
  • Zoology (1297)