Decoding task from oculomotor behavior in virtual reality

Abstract

In the present study, we aim to explore whether and how well we can predict tasks based on eye movements in a virtual environment. We designed four different tasks in which participants had to align two cubes of different sizes. To define where participants looked, we used a ray-based method to calculate the point-of-regard (POR) on each cube at each time point. Using leave-one-subject-out cross-validation, our model performed well with an f1-score of 0.51 ± 0.17 (chance level 0.25) in predicting the four alignment types. Results suggest that the type of task can be decoded based on the aggregation of PORs. We further discuss the implications of object size on task inference and thus set an exciting road-map for how to design intention recognition experiments in virtual reality. © 2020 ACM.