Modeling Human Eye Movements with Neural Networks in a Maze-Solving Task

doi:10.32470/ccn.2022.1291-0

Paper

Modeling Human Eye Movements with Neural Networks in a Maze-Solving Task

Published Dec 20, 2022 · Jason Li, Nicholas Watters, Yingting Wang

UNKNOWN SJR score

6

Citations

0

Influential Citations

Full text Semantic Scholar

Abstract

From smoothly pursuing moving objects to rapidly shifting gazes during visual search, humans employ a wide variety of eye movement strategies in different contexts. While eye movements provide a rich window into mental processes, building generative models of eye movements is notoriously difficult, and to date the computational objectives guiding eye movements remain largely a mystery. In this work, we tackled these problems in the context of a canonical spatial planning task, maze-solving. We collected eye movement data from human subjects and built deep generative models of eye movements using a novel differentiable architecture for gaze fixations and gaze shifts. We found that human eye movements are best predicted by a model that is optimized not to perform the task as efficiently as possible but instead to run an internal simulation of an object traversing the maze. This not only provides a generative model of eye movements in this task but also suggests a computational theory for how humans solve the task, namely that humans use mental simulation.

Study Snapshot

Key takeawayHuman eye movements in a maze-solving task are best predicted by a model that optimizes for internal simulation of an object traversing the maze, suggesting mental simulation as a computational theory.

PopulationOlder adults (50-71 years)

Sample size24

MethodsObservational

OutcomesBody Mass Index projections

ResultsSocial networks mitigate obesity in older groups.

Sign up to use Study Snapshot

Consensus is limited without an account. Create an account or sign in to get more searches and use the Study Snapshot.

Create an account

Paper

Modeling Human Eye Movements with Neural Networks in a Maze-Solving Task

Sign up to use Study Snapshot

References

A brain-inspired object-based attention network for multiobject recognition and visual reasoning

Our brain-inspired object-based attention model improves multiobject recognition and visual reasoning accuracy, outperforming larger models in generalizing to unseen stimuli.

DeepGaze III: Modeling free-viewing human scanpaths with deep learning

DeepGaze III is a new deep learning model that accurately predicts human eye movements in static images, revealing that scene content has a stronger effect on fixation selection than previous scanpath history.

Modular Object-Oriented Games: A Task Framework for Reinforcement Learning, Psychology, and Neuroscience

Modular Object-Oriented Games is a lightweight, customizable Python task framework designed for machine learning, psychology, and neurophysiology researchers.

State-of-the-Art in Human Scanpath Prediction

This paper evaluates human scanpath prediction models based on their predictive power, providing a detailed picture of the current state-of-the-art in the field.

The role of mental simulation in primate physical inference abilities

Primates use mental simulation to make flexible inferences, as demonstrated by comparing their behavior with recurrent neural network models that simulate the position of the ball.

Predicting Goal-Directed Human Attention Using Inverse Reinforcement Learning

The inverse reinforcement learning model accurately predicts goal-directed human attention, improving accuracy and efficiency in visual search tasks.

Towards Robust Image Classification Using Sequential Attention Models

Introducing an attention model into a neural network significantly improves adversarial robustness, resulting in state-of-the-art image classification accuracies under various attack strengths.

PyTorch: An Imperative Style, High-Performance Deep Learning Library

PyTorch is a Python-based deep learning library that combines usability and speed, supporting both imperative programming style and hardware accelerators like GPUs.

Visual Scanpath Prediction Using IOR-ROI Recurrent Mixture Density Network

The IOR-ROI recurrent mixture density network model accurately predicts human-like visual scanpaths under task-free viewing conditions, improving our understanding of visual attention and search behavior.

Citations

Computational basis of hierarchical and counterfactual information processing

Human decision-making strategies, such as hierarchical and counterfactual reasoning, are a continuum of computationally rational solutions under various constraints, including optimal, counterfactual, postdictive, and hierarchical.

Self-Attention in Transformer Networks Explains Monkeys' Gaze Pattern in Pac-Man Game

Transformer networks accurately model monkeys' gaze patterns in the Pac-Man game, providing insights into the brain's complex problem-solving processes.

Analysis of eye movement scan-paths and their image features while searching for routes on metro maps

Eye movement analysis can help identify areas of interest on metro maps, aiding in route search and identifying suitable illustrations for train maps.

Exploring Foveation and Saccade for Improved Weakly-Supervised Localization

Incorporating foveation and saccades in a weakly-supervised object localization framework improves object detection quality and resilience, aligning with human perception.

Planning By Active Sensing

This study developed an algorithm that rapidly plans under uncertainty by actively sensing the environment, using visual samples to compute optimal paths in complex mazes.