Life can be wonderful if everything around us can be controlled by simple gestures. Gesture recognition technology helps us to interact with machines naturally without any additional device. Gestures are interpreted via mathematical algorithms and corresponding actions initiated. Although this technology is still in its infancy, applications are beginning to appear. Kinect is one such application.

Fig. 1: Kinect sensor (Courtesy: Wikipedia)
Fig. 1: Kinect sensor (Courtesy: Wikipedia)
Fig. 2: Block diagram of gesture-controlled robot
Fig. 2: Block diagram of gesture-controlled robot

Though initially invented for gaming, Kinect is being used for different purposes. Kinect is a motion-sensing and speech recognition device developed by Microsoft for Xbox 360 video game console. The main idea was to be able to use a gaming console without any kind of controller.

This project uses Kinect technology to capture, process and interpret human gestures for controlling the motion of a robot.

Circuit and working
Fig. 2 shows the block diagram of gesture-controlled robot. It comprises a Kinect sensor interfaced with a computer using USB port and a simple robot connected to the computer through USB-to-serial converter.

Kinect sensor. Kinect sensor is packed with an array of sensors and specialised devices to pre-process the information received. The Kinect and the computer—running Windows or Linux—communicate through a single USB cable.

Main features of Kinect sensor include:

Gesture recognition. It can recognise gestures like hand movements, based on inputs from an RGB camera and depth sensor.

Speech recognition. It can recognise spoken words and convert them into text, although accuracy strictly depends on the dictionary used. Input is from a microphone array.

Fig. 3: Circuit of the robot
Fig. 3: Circuit of the robot

The main components are the RGB camera, depth sensor and microphone array. The depth sensor combines an IR laser projector with a monochrome CMOS sensor to get 3D video data. Besides these, there is a motor to tilt the sensor array up and down for the best view of the scene, and an accelerometer to sense position.

74D_Table 6C5_Table_2

43F_PartsRobot. Fig. 3 shows the circuit of the robot. The robot is built around ATmega16 MCU (IC2), driver IC MAX232 (IC1), regulator IC 7805 (IC4), motor driver IC L293D (IC3) and a few discrete components.

COM port is connected to the computer using the USB-to-serial converter. Controlling commands to the robot are sent via serial port and the levels converted into 5V TTL/CMOS type by IC1. These TTL/CMOS signals are directly fed to the MCU (IC2) for controlling motors M1 and M2 to move the robot in all directions. Port pins PB4 through PB7 of IC2 are connected to input pins IN1 through IN4 of IC3, respectively, to give driving inputs. EN1 and EN2 are connected to VCC to keep IC3 always enabled. LED1 and LED2 are connected to ports PB1 and PB2 of IC2 for testing purpose.


  1. Helo
    If having some video on installation of software (kinect base gesture robot)
    Plz share with us
    Having problems with software


Please enter your comment!
Please enter your name here