Google’s LiteRT enables Raspberry Pi 5 systems to run compact Gemma AI models locally for language and vision tasks without relying on cloud inference.
A tutorial on Raspberry Pi’s website shows how Google’s Gemma artificial intelligence (AI) models can be run locally on a Raspberry Pi 5 using LiteRT. Written by Naush Patuck, a Raspberry Pi software engineering manager, the guide combines LiteRT with the Reachy Mini robot to handle language and vision tasks on the device rather than relying on a cloud service.
LiteRT is Google’s runtime for running trained AI models directly on local hardware. The tutorial demonstrates several Gemma models on the Raspberry Pi 5, ranging from Gemma 3 270M and EmbeddingGemma 300M to the larger Gemma 4 E2B and E4B variants. The setup also uses YOLO26n for object detection and MediaPipe Selfie Segmenter for image-processing tasks.
On an 8GB Raspberry Pi 5, Gemma 3 270M achieved 433.17 prefill tokens per second and 22.58 decode tokens per second with LiteRT-LM. Gemma 4 E2B recorded 99 prefill tokens per second and around 9 decode tokens per second, with an end-to-end generation rate of about 27.3 characters per second. For object detection, YOLO26n completed an inference in 101.26 milliseconds on the CPU, compared with 375.73 milliseconds using the LiteRT WebGPU Vulkan backend.
Running the models locally means inference can take place without sending requests to a remote cloud service or requiring an active internet connection. The approach trades the convenience of cloud infrastructure for the more limited processing resources of a Raspberry Pi 5. The 8GB model has an official list price of $95, although the price in India varies between sellers.
The LiteRT command-line tool requires about 25MB of storage, while the Gemma models can range from hundreds of megabytes to several gigabytes depending on the variant. Developers therefore need to consider both storage capacity and available memory when selecting a model. The larger models also produce responses more slowly, making model size an important consideration for interactive applications.
The tutorial shows that a Raspberry Pi 5 can serve as a local platform for experimenting with generative AI and computer vision. For projects where keeping inference on the device is more important than achieving cloud-level performance, the combination of LiteRT and smaller Gemma models provides an accessible way to explore local AI processing.
For more information, click here.
