A device that senses, processes, and stores data all at once. This system can encode and process optical data—find out more about it!
Researchers at Peking University have introduced a reconfigurable neuromorphic computing platform that combines sensing and computing capabilities into a single device. The system features an array of phototransistors integrated with a single memristor (MP1R).
Thanks to their ability to encode and process optical data, photonic memristors show great potential for running computer vision algorithms. However, their functionality could be more robust, making them less suitable for supporting neural network architectures for tasks beyond computer vision.
The study’s primary goal was to create a universal and reconfigurable in-sensor processing platform. Unlike earlier systems built around photonic memristors, this platform is designed to support computer vision algorithms and a wide range of other neural network architectures.
The team developed a 20×20 phototransistor array capable of sensing light and modulating its response based on different wavelengths. Notably, the array exhibits potentiation behaviour under blue light and depression behaviour under red light.
The researchers’ platform integrates optical sensing, data processing, and memory functions into a single system. This versatile design enables it to run algorithms for a wide range of tasks, including static and event-based image recognition and the analysis of colored images.
Researchers designed a system compatible with various neural network architectures, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and spiking neural networks (SNNs). This exceptional versatility makes it well-suited for potential applications in real-world scenarios.
The team’s recent work could lead to the creation of other universal neuromorphic vision platforms. Such advancements can potentially enhance the performance of machine learning algorithms across a range of tasks while significantly reducing their power consumption.
The team plans to optimize the platform’s power consumption and enhance its sensitivity to lighting variations in future studies. These improvements would further increase the system’s versatility, enabling it to capture high-quality data in natural and low-light conditions.
Reference: Bingjie Dang et al, Reconfigurable in-sensor processing based on a multi-phototransistor–one-memristor array, Nature Electronics (2024). DOI: 10.1038/s41928-024-01280-3.
