Saturday, May 18, 2024

Sign Talk: DIY Gesture Language Translator

ashwiniCommunicating with a specially challenged person who can’t speak or hear is quite difficult, especially when you don’t know sign language.

So to ease this problem, today you will learn to make a sign Language Translator device that converts sign language into a spoken language. This device will be based on an ML model that can recognize the different sign language gestures for accurate translation.

DIY Sign Language Translator
Gesture Language Translator Prototype
Sign Language Translator Project
Fig 1. Sign language translating

Bill of Material  

DIY Gesture Language Translator BoM

Preparing The ML Model 

There are different sign languages practiced in different countries. For example, India uses Indian Sign Language (ISL) while the USA uses American sign language (ASL). So, you first need to decide which type you wish to implement. 

To obtain the datasets of the ASL, visit here. For ISL, visit here. You can search here for numerous hand gestures that are used in daily life. 

- Advertisement -

After gathering the datasets, prepare the ML model for training. There are many options for this such as TensorFlow, Keras, PyTorch, Google’s Teachable Machine, etc. For this project, I am using Google’s Teachable Machine, which is an online ML model creator service. Now, feed the datasets into your choice of ML model creator and capture the pictures of different hand gestures/signs with a camera. Remember to label them as per their meaning. 

Preparing ML Model for Sign Language Translator
Fig 2. ML model feeding datasets

Deploying ML Model 

After carrying out the above steps, download the ML model output. Or you can get the option for uploading the ML model. So upload the code and you will get the link to the ML model and a code snippet to be used in the ML model, which can be used for various platforms like Python using Keras P5.js or JavaScript (JS). After uploading the ML model, copy the JS code script.

DIY Gesture Language Translator – Coding

Open the Raspberry Pi desktop and then create a new JS file. Shown below is Gesturetranslator.html. Paste the code snippet into the code and write the project name. Increase the canvas size so that the camera feed is clearly visible in a larger space. Save it.

- Advertisement -
DIY Gesture Language Translator Code
Fig 3. Coding
DIY Sign Language Translator Code
Fig 4. Code snippet

Recommended: Animal Language Translator


Enable the camera interface in Raspberry Pi and plug it into the board. Now open the JavaScript code in Google Chrome or any other browser that supports it.

DIY Gesture Language Translator
Fig 5.
DIY Sign Language Translator
Fig 6. Testing sign language translator device

Also Read: Implementing gesture control in Projects

Download ML Model Datasets and Code

Ashwini Sinha
Ashwini Sinha
A tech journalist at EFY, with hands-on expertise in electronics DIY. He has an extraordinary passion for AI, IoT, and electronics. Holder of two design records and two times winner of US-China Makers Award.


Unique DIY Projects

Electronics News

Truly Innovative Tech

MOst Popular Videos

Electronics Components