Digit-tracking as a new tactile interface for visual perception analysis

Nat Commun. 2019 Nov 26;10(1):5392. doi: 10.1038/s41467-019-13285-0.

Abstract

Eye-tracking is a valuable tool in cognitive science for measuring how visual processing resources are allocated during scene exploration. However, eye-tracking technology is largely confined to laboratory-based settings, making it difficult to apply to large-scale studies. Here, we introduce a biologically-inspired solution that involves presenting, on a touch-sensitive interface, a Gaussian-blurred image that is locally unblurred by sliding a finger over the display. Thus, the user's finger movements provide a proxy for their eye movements and attention. We validated the method by showing strong correlations between attention maps obtained using finger-tracking vs. conventional optical eye-tracking. Using neural networks trained to predict empirically-derived attention maps, we established that identical high-level features hierarchically drive explorations with either method. Finally, the diagnostic value of digit-tracking was tested in autistic and brain-damaged patients. Rapid yet robust measures afforded by this method open the way to large scale applications in research and clinical settings.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Animals
  • Attention / physiology
  • Autism Spectrum Disorder / diagnosis
  • Autism Spectrum Disorder / physiopathology*
  • Case-Control Studies
  • Deep Learning
  • Eye Movements
  • Fingers*
  • Humans
  • Image Processing, Computer-Assisted
  • Macaca
  • Male
  • Neural Networks, Computer
  • Visual Perception / physiology*
  • Young Adult