The Edge AI processor transforms laptops and servers for energy-efficient video analytics, LLMs, and AI model processing.
Kinara has launched the Ara-2 Edge AI processor, targeting edge servers and laptops. This processor is designed for performance and energy-efficient inference to run applications such as video analytics, Large Language Models (LLMs), and Generative AI models. The Ara-2 is suitable for edge applications using traditional AI models and AI models with transformer-based architectures.
The Ara-2 has a feature set that improves user experience and increases performance over the Ara-1 processor. It combines a design optimised for latency with on-chip memories and off-chip bandwidth. This enables the Ara-2 to execute large models with low latency.
LLMs and Generative AI are popular, with most applications running on GPUs in data centres facing issues like latency, cost, and privacy. Kinara’s Ara-2 aims to move the computation closer to the user, supporting the large number of parameters used by these Generative AI models and enabling a shift to edge computing.
The Ara-2’s compute engines and software development kit (SDK) are designed to support quantization, a host runtime, and direct FP32 support. This facilitates the transition from GPUs for a range of AI models, making it more accessible and efficient for various applications.
The Ara-2 includes a secure boot, encrypted memory access, and a secure host interface for security in enterprise AI deployments. Kinara provides an SDK for the Ara-2, which consists of a model compiler and compute-unit scheduler, quantization options with an integrated Kinara quantizer, and support for pre-quantized PyTorch and TFLite models. It also features a load balancer for systems with multiple chips and a host runtime.
“With Ara-2 added to our family of processors, we can better provide customers with performance and cost options to meet their requirements. For example, Ara-1 is the right solution for smart cameras as well as edge AI appliances with 2-8 video streams, whereas Ara-2 is strongly suited for handling 16-32+ video streams fed into edge servers, as well as laptops and even high-end cameras,” said Ravi Annavajjhala, Kinara’s CEO.
For more information, click here.
