Skip to main content
bioRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search
New Results

A new comprehensive Eye-Tracking Test Battery concurrently evaluating the Pupil Labs Glasses and the EyeLink 1000

View ORCID ProfileBenedikt V. Ehinger, Katharina Groß, Inga Ibs, View ORCID ProfilePeter König
doi: https://doi.org/10.1101/536243
Benedikt V. Ehinger
1Institute of Cognitive Science, Osnabrück University
2Donders Institute for Brain, Cognition and Behaviour, Radboud University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Benedikt V. Ehinger
Katharina Groß
1Institute of Cognitive Science, Osnabrück University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Inga Ibs
1Institute of Cognitive Science, Osnabrück University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peter König
1Institute of Cognitive Science, Osnabrück University
3Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Peter König
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Preview PDF
Loading

ABSTRACT

Eye-tracking experiments rely heavily on good data quality of eye-trackers. Unfortunately, it is often that only the spatial accuracy and precision values are available from the manufacturers. These two values alone are not sufficient enough to serve as a benchmark for an eye-tracker: Eye-tracking quality deteriorates during an experimental session due to head movements, changing illumination or calibration decay. Additionally, different experimental paradigms require the analysis of different types of eye movements, for instance smooth pursuit movements, blinks or microsaccades, which themselves cannot readily be evaluated by using spatial accuracy or precision alone. To obtain a more comprehensive description of properties, we developed an extensive eye-tracking test battery. In 10 different tasks, we evaluated eye-tracking related measures such as: the decay of accuracy, fixation durations, pupil dilation, smooth pursuit movement, microsaccade detection, blink detection, or the influence of head motion. For some measures, true theoretical values exist. For others, a relative comparison to a gold standard eye-tracker is needed. Therefore, we collected our gaze data simultaneously from a gold standard remote EyeLink 1000 eye-tracker and compared it with the mobile Pupil Labs glasses.

As expected, the average spatial accuracy of 0.57° for the EyeLink 1000 eye-tracker was better than the 0.82° for the Pupil Labs glasses (N=15). Furthermore, we detected less fixations and shorter saccade durations for the Pupil Labs glasses. Similarly, we found fewer microsaccades using the Pupil Labs glasses. The accuracy over time decayed only slightly for the EyeLink 1000, but strongly for the Pupil Labs glasses. Finally we observed that the measured pupil diameters differed between eye-trackers on the individual subject level but not the group level.

To conclude, our eye-tracking test battery offers 10 tasks that allow us to benchmark the many parameters of interest in stereotypical eye-tracking situations, or addresses a common source of confounds in measurement errors (e.g. yaw and roll head movements).

All recorded eye-tracking data (including Pupil Labs’ eye video files), the stimulus code for the test battery and the modular analysis pipeline are available (https://github.com/behinger/etcomp).

BVE, KG, II and PK conceived the experiment. II and BVE created the experiment and recorded the gaze data. BVE and KG performed the analysis. BVE, KG and PK reviewed the manuscript critically.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted February 03, 2019.
Download PDF
Email

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
A new comprehensive Eye-Tracking Test Battery concurrently evaluating the Pupil Labs Glasses and the EyeLink 1000
(Your Name) has forwarded a page to you from bioRxiv
(Your Name) thought you would like to see this page from the bioRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
A new comprehensive Eye-Tracking Test Battery concurrently evaluating the Pupil Labs Glasses and the EyeLink 1000
Benedikt V. Ehinger, Katharina Groß, Inga Ibs, Peter König
bioRxiv 536243; doi: https://doi.org/10.1101/536243
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
A new comprehensive Eye-Tracking Test Battery concurrently evaluating the Pupil Labs Glasses and the EyeLink 1000
Benedikt V. Ehinger, Katharina Groß, Inga Ibs, Peter König
bioRxiv 536243; doi: https://doi.org/10.1101/536243

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Neuroscience
Subject Areas
All Articles
  • Animal Behavior and Cognition (4688)
  • Biochemistry (10379)
  • Bioengineering (7695)
  • Bioinformatics (26373)
  • Biophysics (13547)
  • Cancer Biology (10724)
  • Cell Biology (15460)
  • Clinical Trials (138)
  • Developmental Biology (8509)
  • Ecology (12843)
  • Epidemiology (2067)
  • Evolutionary Biology (16887)
  • Genetics (11416)
  • Genomics (15493)
  • Immunology (10638)
  • Microbiology (25257)
  • Molecular Biology (10241)
  • Neuroscience (54595)
  • Paleontology (402)
  • Pathology (1671)
  • Pharmacology and Toxicology (2899)
  • Physiology (4355)
  • Plant Biology (9263)
  • Scientific Communication and Education (1588)
  • Synthetic Biology (2561)
  • Systems Biology (6789)
  • Zoology (1473)