In a groundbreaking development, researchers at the Massachusetts Institute of Technology (MIT) have made strides in incorporating peripheral vision into Artificial Intelligence (AI) models, a feature that could have far-reaching implications for a variety of fields, especially in driver safety and understanding human behavior.
Peripheral Vision in Humans and AI
Peripheral vision allows humans to see shapes not directly in their line of sight, albeit with less detail. This expanded field of vision helps in situations like detecting a vehicle approaching from the side. Unlike humans, AI models lack peripheral vision, but equipping them with this ability could enhance their effectiveness in detecting approaching hazards or predicting whether a human driver would notice an oncoming object.
The MIT researchers developed an image dataset that simulates peripheral vision in machine learning models. Training models with this dataset boosted their ability to detect objects in the visual periphery, although their performance still fell short of humans. Moreover, neither the size of objects nor the amount of visual clutter in a scene significantly impacted the AI’s performance.
“There is something fundamental going on here. We tested so many different models, and even when we train them, they get a little bit better but they are not quite like humans. So, the question is: What is missing in these models?” asks Vasha DuTell, a postdoc and co-author of the paper detailing this study.
The Power of Peripheral Vision
Answering this question could help researchers construct machine learning models that perceive the world more like humans do. Beyond enhancing driver safety, these models could be utilized to develop displays that are easier for people to view. Furthermore, a deeper understanding of peripheral vision in AI models can help researchers predict human behavior more accurately, according to lead author Anne Harrington MEng ’23.
“Modeling peripheral vision, if we can really capture the essence of what is represented in the periphery, can help us understand the features in a visual scene that make our eyes move to collect more information,” she explains.
The research will be presented at the International Conference on Learning Representations. “Any time you have a human interacting with a machine — a car, a robot, a user interface — it is hugely important to understand what the person can see. Peripheral vision plays a critical role in that understanding,” says Ruth Rosenholtz, principal research scientist in the Department of Brain and Cognitive Sciences and a member of CSAIL.
Simulating Peripheral Vision
The researchers started with a technique known as the texture tiling model, commonly used to model peripheral vision in humans. This method transforms images to represent a human’s visual information loss. They modified this model so that it could transform images similarly, but more flexibly, without requiring knowing in advance where the person or AI will focus their eyes.
Using this modified technique, the researchers generated a large dataset of transformed images that appear more textural in certain areas, representing the loss of detail that occurs when a human looks further into the periphery. They then used this dataset to train several computer vision models and compared their performance with that of humans in an object detection task.
Interestingly, humans consistently outperformed the machines, and the latter were particularly bad at detecting objects in the far periphery. Their performance also failed to follow the same patterns as humans.
The researchers plan to continue exploring these differences, with a goal of finding a model that can predict human performance in the visual periphery. This could enable AI systems that alert drivers to hazards they might not see, for instance. They also hope to inspire other researchers to conduct additional computer vision studies with their publicly available dataset.
This research is a significant step in understanding and improving how AI models perceive the world. It not only enhances our understanding of human vision in the periphery but also highlights the gaps in AI models’ performance compared to humans. As AI continues to evolve, advancements like these will play a crucial role in shaping its capabilities and applications, especially in areas like driver safety.
