Can AI become faster without adding more GPUs? Researchers demonstrate an optical chip architecture that dramatically improves inference efficiency.
Researchers at Peking University have developed an all-optical interconnect system that links conventional electronic chips to accelerate artificial intelligence (AI) inference. The architecture combines programmable hardware with optical communication to achieve more than a 100-fold improvement in distributed AI inference speed while requiring less than one-ninth of the computational resources used by a comparable graphics processing unit (GPU).
The work, published in National Science Review, addresses a growing challenge in AI infrastructure. As increasingly complex models drive demand for larger data centres and more GPUs, the study proposes improving communication between processors instead of simply adding more computing hardware. The approach could help reduce energy consumption, lower inference latency, and improve the efficiency of AI workloads in both cloud and edge computing environments.
The prototype is built around five field-programmable gate array (FPGA) chips connected through two custom optical components. A silicon photonic transceiver operating at 400Gbps converts electrical signals into optical signals, while a 16×16 optical switching chip dynamically routes data between processors. In an image-denoising experiment involving 1,000 images of 32,768 bits each, the optical FPGA system completed processing in 105.16 microseconds. Under the same workload, a commercial 16.96-teraflop GPU required 15.6 milliseconds, making the optical system nearly 150 times faster despite offering a combined theoretical compute capability of only 1.969 teraflops.
The researchers attribute the performance gains to the optical architecture, which allows intermediate neural network outputs to flow directly between processing stages without repeatedly accessing memory. This continuous data pipeline keeps processors active while reducing communication bottlenecks that typically limit AI inference performance.
“Specific objectives can be realised under limited computational resources when algorithms, processor micro-architectures and chip-level interconnections are co-designed,” says Shu Haowen and Wang Xingjun, Corresponding Authors of the study at Peking University. “This fabric can also alleviate unsustainable energy usage in data centres and optimise latency or consumption in edge-computing scenarios.”
