Tracking Non-Visual Eye Movements Non-Invasively: Comparing Manual and Automatic Annotation Styles

Abstract

Non-visual eye-movements (NVEMs) during episodic and semantic memory retrieval may reveal mental foraging, similar to bodily movements during physical foraging. However, measuring them is challenging as common eye-tracking methods might interfere with the natural eye gaze habits of participants. Our aim was to compare two annotation approaches to track NVEMs from raw video footage: First, manual annotation using a coding grid dividing the visual field into nine sections, and second, automated annotation using the neural-network driven face recognition software OpenFace representing eye gazes as Cartesian vectors. We found that both approaches showed moderate to excellent reliability, quantitatively. k-mean clusterings of the automatic annotations revealed high resemblance to the manual coding approach, qualitatively. We conclude that mapping between manual and automatic coding of NVEMs is feasible, and that the selection of the appropriate method should depend on the research question and available resources.