MIT researchers build a “scientific sandbox” that uses AI to simulate how vision systems evolve and it could help design next-gen sensors.
MIT scientists have developed an AI-driven computational framework that mimics the evolution of visual systems from simple light-sensitive patches to complex camera-like eyes by letting embodied artificial agents “evolve” their own eyes in simulated environments. The tool could eventually guide the design of task-specific sensors and cameras for robots, drones, autonomous vehicles, and wearable devices.
Unlike traditional lab experiments that observe animals or dissect biological eyes, this evolution simulator functions as a scientific “sandbox” where researchers can adjust environmental conditions and survival tasks to observe how visual systems might emerge under different pressures.
At the core of the framework is a genetic encoding mechanism that lets simulated eyes and neural networks evolve over generations. Agents start with basic photoreceptors and learn through reinforcement learning to complete tasks such as navigation, object recognition, or food finding. Constraints like limited sensor pixels and physical task demands shape how visual systems develop, mirroring evolutionary pressures in nature.
Early findings from the experiments reveal clear links between environmental demands and eye structure: agents tasked with navigating spaces tend to evolve compound-style eyes with wide, low-resolution fields, akin to insect vision, while agents focused on object discrimination develop camera-style eyes with higher frontal acuity.
The research also surfaced an insight about cognitive resources: after a point, bigger neural processing capacity doesn’t always yield better visual performance, echoing real-world evolutionary trade-offs where physical constraints balance sensory capability.
The project, detailed in Science Advances, was led by MIT Media Lab graduate students and senior researchers from MIT, Rice University, and Lund University. Looking ahead, the team plans to expand the sandbox to explore a broader set of “what-if” scenarios including integrations with large language models to simplify hypothesis testing and to benchmark optimal visual system designs for specific real-world applications.
