A new accelerator targets higher AI efficiency, combining inference and training capabilities with integrated networking to reduce data movement and improve performance for demanding workloads.
OpenAI has developed Jalapeño, its first in-house AI chip, which the company says can deliver up to 1.9 times the performance per watt of comparable Nvidia systems across three large public models. The processor is designed to support both inference and training within the same architecture, with efficiency and reduced data movement among its main goals.
Jalapeño is built around an architecture intended to handle both phases without requiring separate hardware paths. OpenAI says the chip keeps model state, including key-value cache used during generation, closer to the processing resources that need it. This approach is intended to reduce the amount of data that must move between chips and limit communication delays.
The processor also integrates networking into its architecture. By bringing communication capabilities closer to the compute resources, the design aims to reduce waiting time caused by data transfers, particularly for AI workloads that perform numerous inference steps in sequence.
Another focus is flexibility. Jalapeño was designed to support different stages of AI processing within the same architecture, potentially allowing infrastructure to be used more efficiently as workloads change. The company says this can help improve performance-per-watt while reducing the resources required for demanding AI applications.
OpenAI also used its own AI models during development. According to the article, AI-assisted engineering helped researchers explore implementations, shorten design cycles and optimise architectural decisions. The team moved from initial design work to tapeout in nine months, while software development and optimisation continued alongside the hardware effort.
The broader objective is to make AI computing faster and more power-efficient as demand for inference continues to grow. By combining compute, model-state handling and networking within one processor architecture, Jalapeño represents OpenAI’s effort to develop more specialised infrastructure for future AI workloads.
