A machine learning embedded photodetector enables real-time recognition, reducing energy demands and accelerating optical sensing for complex environments.
Spectral imaging is a vital tool for analyzing materials, monitoring crops, and tracking pollutants. But conventional systems face a major challenge as they produce vast amounts of data that must be sent to a separate processor for analysis. This slows object recognition and consumes large amounts of power, limiting the speed and efficiency of AI applications. Researchers needed a sensor capable of performing intelligent recognition directly during image capture.
Scientists at Lawrence Berkeley National Laboratory developed a first-of-its-kind AI-enhanced sensor to meet this need. The device was designed to integrate machine learning into the photodetection process, eliminating the need for dense digital post-processing and enabling real-time identification of target objects.
The sensor can identify objects in real time by “sniffing” spectral features from example images and applying that knowledge to new, unseen scenes. Light hitting the sensor is converted into an electrical current whose intensity corresponds to the spectral content. By adjusting the sensor’s responsivity, it highlights desired signatures and suppresses irrelevant data. This analog signal effectively performs computations that mimic machine learning algorithms, removing the need for separate digital processing.
During training, the sensor examines labeled pixels from example images, learning which features correspond to targets and which to background. When presented with new images, it generates positive signals only for the target, demonstrating the ability to generalize to unseen data. This approach drastically reduces data transfer, increases processing speed, and lowers power consumption compared with traditional spectral imaging.
Prof. Ali Javey, EECS & MSE, UC Berkeley, who led this research work, says, “For me, the most exciting part is giving intelligence to sensors.” The sensor was trained on example images using labeled pixels and then tested on new data. It successfully identified objects like birds, leaf hydration states, oxide layer thickness in semiconductors, and transparent chemicals. By co-designing hardware and algorithms, the team has opened new possibilities for AI vision and optical sensing beyond spectral imaging.
