A simple ESP32 open-source project brings Tamil and English voice assistant with spoken replies and smart-home controls.
An open-source project known as Hermes that can recognise Tamil and English languages, respond vocally, and perform simple actions. The hardware set up is cheap, the ESP32 development board costs just a few hundred rupees, whereas a Windows PC manages the complex software part.
Many projects are built only with consideration of English language, Hermes provides Tamil language voice control besides offering common functionalities such as timer, music and smart plug control. The project is published by a developer known by his name ROHITCRAFTSYT. He has provided the components in the repository. In terms of hardware, the components include ESP32-S3 dev board, INMP441 I2S MEMS Microphone, MAX98357A I2S Amplifier, 8Ω 3W speaker, push button, jumper wire, and breadboard. A USB data cable and a 5V power source are also required. From the software perspective, Windows PC, Python 3.11 or later version, and a free Groq account are required.
The ESP32 handles audio input and output, while a Python program running on the PC acts as the assistant’s brain. When the user presses and holds the button to speak, the board sends the audio over Wi-Fi to the computer. Groq’s hosted Whisper model converts speech into text, after which a language model hosted by Groq processes the request and determines an appropriate response. Microsoft’s online neural voice service then converts the reply into speech, including through its Tamil voice, ta-IN-PallaviNeural, before the audio is played through the speaker. This arrangement allows the relatively inexpensive microcontroller to support voice-based tasks without having to run a large AI model locally.
Hermes still depends heavily on internet connectivity. If Microsoft’s voice service is unavailable, the software can fall back to Google’s service and then to the offline voice available in Windows. Speech recognition and language processing are not fully available offline. Running a local language model through Ollama is listed as a future possibility rather than a completed feature.
The assistant can be initialised after the configuration of the software, either through hermes_bridge.py or the provided start_bridge.bat file for Windows. It is required for the board to have its own set of configuration files such as config.example.h and secrets.h.example, which require Wi-Fi credentials and token to be filled in before the firmware upload using arduino-cli. The Wi-Fi network should be of 2.4 GHz frequency with WPA2 encryption, but not 5GHz and WPA3.
The project also provides a web interface that can be accessed through a phone that is within the same network. There are other functionalities that further expand the application beyond mere conversation. The software can conduct web searches, launch applications and websites, control media playing, set timers, reminders and alarms, find out cryptocurrency prices, and have short and long term memory. It can also communicate with Tuya smart devices, MQTT relay, Home Assistant, Wake on-LAN, Chromecast and Android TV. An additional integration called Spotify playback control allows for controlling Spotify playing but only with a Premium account. Voice-based door opening feature is turned off by default.
There are some limitations to consider before building it. The classic ESP32 used in the project does not have PSRAM, so the current version relies on a push button rather than a spoken wake word. The PC-side software is designed for Windows, and Groq’s free tier comes with daily usage limits. The repository is also relatively new, so users may encounter bugs or need to troubleshoot the Python environment, firmware upload process and API configuration. These requirements make the project more suitable for someone with some experience in electronics and programming than for a complete beginner.
The cost for putting together this hardware would probably range anywhere from ₹1,500 to ₹3,000, but this will be affected by the cost of the components available. In this way, the project makes it possible for even electronics hobbyists and colleges or schools that want to experiment with voice interaction and devices to undertake it. One major feature of this project is its capability to use Tamil language in interacting, and this enables users to experiment with voice control in a language that is not often used in hobbies. However, a significant drawback of this is that the intelligence is still in the cloud.
The road map of the project suggests several possibilities for improving it, including wake word detection with ESP32-S3, streaming of the response, which can be interrupted by users, and the ability to use a local language model. The realisation of these options may decrease reliance on cloud solutions and make the assistant more responsive. Meanwhile, Hermes provides an opportunity to explore a voice assistant based on inexpensive hardware and able to speak Tamil.
For more information, click here.
