Monday, July 15, 2024

Smart Wristband For Hand Tracking

- Advertisement -

The EchoWrist wristband tracks hand movements and interactions using AI and inaudible sound waves, changing how we control and interact with devices.

EchoWrist was developed by researchers in the Cornell Ann S. Bowers College of Computing and Information Science. Credit: Cornell University
EchoWrist was developed by researchers in the Cornell Ann S. Bowers College of Computing and Information Science. Credit: Cornell University

Researchers at Cornell University have created a wristband device that uses AI-powered, inaudible sound waves to continuously monitor hand positioning and the objects with which the hand interacts.

This technology has a variety of potential applications, including monitoring hand positions for virtual reality (VR) systems, enabling the control of smartphones and other devices through hand gestures, and understanding user activities. For instance, a cooking app could provide step-by-step narration as the user chops, measures, and stirs ingredients. The device is compact enough to be integrated into a commercial smartwatch and can operate on a standard smartwatch battery all day.

- Advertisement -

EchoWrist is part of the latest advancements in low-power, body pose-tracking technology developed by the Smart Computer Interfaces for Future Interactions (SciFi) Lab. Cheng Zhang, an assistant professor of information science at Cornell Ann S. Bowers College of Computing and Information Science, directs the lab.

The device utilizes two small speakers on the top and bottom of a wristband to emit inaudible sound waves that bounce off the hand and any objects it holds. Two adjacent microphones capture the echoes, which are then processed by a microcontroller. A battery smaller than a quarter supplies power to the device.

To interpret a user’s hand posture from the echoes, the team developed an artificial intelligence model inspired by brain neurons, known as a neural network. They trained this network by comparing echo profiles with videos of users performing various gestures and reconstructed the positions of 20 hand joints based on the sound signals.

The researchers enlisted 12 volunteers to evaluate how effectively EchoWrist identifies objects like cups, chopsticks, water bottles, pots, pans, and kettles and actions such as drinking, stirring, peeling, twisting, chopping, and pouring. The device demonstrated a 97.6% accuracy rate. This accuracy enables users to follow interactive recipes that monitor the cook’s progress and provide the next step verbally, allowing cooks to keep their screens clean.

Reference: Chi-Jung Lee et al, EchoWrist: Continuous Hand Pose Tracking and Hand-Object Interaction Recognition Using Low-Power Active Acoustic Sensing On a Wristband, arXiv (2024). DOI: 10.48550/arxiv.2401.17409

Nidhi Agarwal
Nidhi Agarwal
Nidhi Agarwal is a journalist at EFY. She is an Electronics and Communication Engineer with over five years of academic experience. Her expertise lies in working with development boards and IoT cloud. She enjoys writing as it enables her to share her knowledge and insights related to electronics, with like-minded techies.


Unique DIY Projects

Electronics News

Truly Innovative Tech

MOst Popular Videos

Electronics Components