New Facial Recognition Smart Glasses For Visually Challenged Person

Ashwini Kumar Sinha

- Advertisement -
 

In this project, we will be making a prototype of a smart eyeglass that can help a visually challenged person to recognize the person in front of him and learn about obstacles ahead. This will be enabled by face recognition and distance detection features.

Bill of Material

You will also need an earphone but since most of us have it, I haven’t mentioned it in the
above list.

- Advertisement -

Prerequisite setup
First, open RPi SSH or Terminal and install the required libraries and modules.
Then install the espeak module, followed by the espeak library for Python.
Follow the snippet below for installation (Refer Fig 1,2).

installing espeak
Fig 1. Installing espeak
installing espeak
Fig 2. Installing espeak for python

Now install the ‘opencv’, ‘numpy’, ‘dlib’ library. You might need to swap the
memory for installation. Follow the instruction for installation of opencv and dlib
available on the internet.

After that install the face recognition library for python 3 using the following
command in the terminal.

‘pip3 install face recognition’

As we have installed our required library, let’s start coding.

Code

For this project we will make two different codes in Python3, one to tell the distance of
obstacles ahead and other for face recognition. The first code measures the distance of
obstacles and alert the user using voice output. We can also add a vibration motor for haptic
feedback.
Code 1: Open the Python3 IDE to start coding. Here we need two modules – time and
espeak. We will import both the modules in code. After that we created a distance function
where we import the GPIO and set the pins for distance sensor. Next we created a while
loop that continuously checks the distance. In this while loop we also added an if condition
to check the distance between the user and the obstacle. If it detects an object close to the
user, it will automatically inform the person by giving audio and haptic feedback. (Refer fig
3, 4).

Distance sensor code
Fig 3. Python code distance sensor
Distance sensor code
Fig 4.Python Code obstacle detection.

Code 2: This is to recognise the person in front of the user (known or unknown). Here in this code we will import 3 modules: face recognition, cv2 and numpy. We have to create different arrays for known faces and their names. We must write the image file name for the face recognition of that member (Refer Fig 5,6). 

Code
Fig 5.Python code face detection
Python code
Fig 6.

In the next part of code, we will try to match the face in camera video with the array of known faces. If the face matched, then the code will run the espeak synthesizer to call the name of that person using the syntax ‘espeak.synth ()’ as in the pic below.(Refer fig 7).

Python code Smart Face Recogination
Refer fig 7.

Note :- You can Change the output language  to hindi and voice to different modes for example wisher mode, Male voice , and female voice by uncommenting lines in code.(Refer fig 8).

Python code
Fig 8. Espeak Different Language and voice

After all is done, save the code in the face recognition folder.

Download Code

Connection

Since we have created the code, now it’s time to connect all the components. 

Insert the camera ribbon in camera module and solder the wires of ultrasonic sensor as illustrated below.

Raspberry Pi Ultrasonic Sensor
5V pin VCC
GND GND
BCM Pin 27  TRIG
BCM PIN 22 ECHO

 

After the connection, set all sensors and components as in pic below.

Smart Glasses
Fig 9.
Smart Glasses
Fig 10.

Testing

Now power the Raspberry Pi using battery or power bank.  Then open the Pi desktop window in remote desktop or VNC (If you do not know how to enable VNC and remote desktop on Pi follow the instructions online.) Next run the distance.py and facerecogination.py code and you are ready to go. When any known or unknown person come in front of the user (visually challenged person), it will detect the face and tell about them. If the user comes closer to any obstacles like wall or any other object, the device will alert about it to avoid accident.      

Note: – This is the first version of the project, we will update it soon with more features to
enable detection of colour and ambient light. The update may come with optical character
and sign board recognition feature as well.
Note: – During the installation of cv2 , dlib, and face recognition library you might need to swap the memory of Raspberry Pi do it carefully.If you face any error or problem during the installation of library  refer to online guide of that python library.

 

66 COMMENTS

  1. Ashwini Sir , I am trying to do this project however i am not able to understand where do i store my files in the image files you have depicted above as”ash.jpg” where do I store it in the directory and how does it come up in the first place . Do i need to click my picture and upload it there or by some other means.

    I am very new to Raspberry Pi and programming and with your project I am trying to learn and implement it .It would be really helpful if you show the step by step procedure to do this from scratch

  2. Traceback (most recent call last):
    File “/home/pi/Desktop/Programs/Face Recognisation/face_compare_test.py”, line 16, in
    gaurav_face_encoding = face_recognition.face_encodings(gaurav_image)[0]
    IndexError: list index out of range

  3. Ashwini Sir , I am trying to do this project. I am very new to Raspberry Pi and programming and with your project I am trying to learn and implement it .It would be really helpful if you show the step by step procedure to do this from scratch

    • No need for a circuit diagram, just connect RPI camera to the raspberry pi camera port, then connect the ultrasonic sensor to raspberry pi as described in the connection, section then insert the earphone plug into the audio jack port of the raspberry pi. Pin number 13 and 14 is raspberry pin 27,22

  4. Hello Ashwiniji,
    My father is a patient of night blindness and now he cannot see in day time also.. His age is 72 yrs. Can you tell me if he want to take help of this smart specs what major things he need to do as no one in my family is computer literate to understand programming upto this extent. Please suggest.

    Regards,
    Shilpi Srivastava

  5. Hello Ashwiniji can I know at which line you have set for file location, because even I changed name it says Traceback (most recent call last):
    File “/home/pi/Desktop/Programs/Face Recognisation/face_compare_test.py”, line 16, in
    Myname_face_encoding = face_recognition.face_encodings(myname_image)[0]
    IndexError: list index out of range
    Can you please send the step by step video of programming to my mail please. Because I am doing it for my final year project it’s near too.

  6. Thank you for the reply. I figured out the directory it needed to be stored in. Thank you for your help. The project is running flawlessly. Is there a way to use a nicer voice for the distance measurement and person names?

SHARE YOUR THOUGHTS & COMMENTS

 
 

What's New @ Electronicsforu.com

Most Popular DIYs

Electronics Components

Truly Innovative Tech