A pair of tiny neural networks can generate and play five simple games on a Raspberry Pi Pico 2 W without conventional game code.
A Raspberry Pi Pico 2 W can run two small neural networks that create and play their own versions of simple games. The open-source project, called tinyworldformer, uses one network to predict the next game frame and another to decide which button to press. The project has been designed by SheepSeb from GitHub and highlights the capability of running an AI game engine using the memory and resources of a microcontroller.
This project uses the RP2350 microcontroller in the Raspberry Pi Pico 2 W. Which has 520KB of SRAM and 4MB of flash memory. Instead of storing conventional game logic on the board, the system relies on two neural networks. One uses the current frame of the game along with the input from the button to determine what the next frame will be like. The other one analyses the prediction of the next frame and decides the action. This creates a continuous cycle in which the networks generate and interact with the game environment.
The first network, referred to as the world model, is an MLP with 26,000 and 131,000 parameters depending on the game and employs eight-bit integer quantisation to minimise the number of operations and computes around two to four million MACs per frame. The second network, referred to as the actor, is an MLP with two layers and 9,000 to 16,000 parameters that select the next action based on the created game state.
The project supports five games: Breakout, Flappy Bird, Pong, Pac-Man Lite and Doom-lite, a simple ray-cast maze game. Their graphics are deliberately limited to small grids, ranging from 20 × 16 cells for Breakout to 48 × 48 for Pac-Man. Information such as the ball’s direction and enemy positions is represented within the frames, allowing the world model to predict subsequent game states.
The hardware specifications include the Raspberry Pi Pico 2 W, which has 4MB of flash memory and 520KB of SRAM memory. An optional ST7789 display with a resolution of 320 × 240 pixels and A USB cable is needed to power the board and transfer firmware. The software environment uses Python, the uv package manager and Rust, while developers training their own models will also need PyTorch and access to a suitable GPU. The developer reports using a desktop graphics card to train the models, with individual training runs taking approximately 45 minutes per game.
The source code, training tools, model exports and firmware are available in GitHub. Developers can prepare the environment, generate gameplay data, train the world model and transfer the resulting firmware to the Pico 2 W. The repository also provides a browser-based version, allowing the game-generating engine to be explored without purchasing the development board. Firmware can be transferred through a USB connection using the project’s flashing utility.
According to the project’s benchmarks, the world model reproduces Breakout frames with 99.99 per cent exact-frame accuracy, Flappy Bird with 100 per cent and Pong with 99.45 per cent. The corresponding figures are 83.6 per cent for Doom-lite and 86.4 per cent for Pac-Man Lite. These measurements compare predicted frames against reference frames, with even a single incorrect cell counting as an incorrect prediction.
The project also tests agents trained inside the generated game environments. The agent that was taught to play the game Breakout in the simulated environment had an average score of 54.4 in relation to the maximum possible 56 bricks. Yet, it became clear from the experiment that there were some mistakes in the generated environments and the agents were able to take advantage of them. One of the agents received a score of 210 for a possible 56.
The developer reports desktop CPU inference times of approximately 0.7 milliseconds per frame for Flappy Bird and 1.5 milliseconds for Breakout. These figures are measured on a computer, not on the Raspberry Pi Pico 2 W. The actual frame rate and performance on the physical board have not yet been established, and the developer acknowledges that hardware testing remains incomplete.
The project includes Python-based game implementations, training tools, model quantisation utilities, a Rust inference engine and firmware. This provides the necessary elements to study how small neural networks can make predictions about the game environments and the possible actions within those environments.
For Indian students and makers, the browser-based version offers a way to explore the project without initially purchasing hardware. Those with a Raspberry Pi Pico 2 W can experiment with firmware deployment and investigate how neural networks operate within the memory constraints of a microcontroller. The next important milestone will be demonstrating its actual performance on the board, particularly its frame rate and memory usage during gameplay.
For more information, click here.
