Most hand gesture systems fail when your hands are full. A wristband could change that, letting you control devices naturally even while holding objects.
Hand gestures could be a way to interact with portable devices and augmented reality glasses but only if they work when users are on the move. Most gesture recognition systems require empty hands. That is a problem when people often carry phones, cups, or bags. In those situations, gestures cannot be detected, limiting the technology’s use in everyday life.
A team at Cornell University is addressing this challenge with Grab-n-Go, a wristband that can recognize hand microgestures even when the hands are occupied. The system aims to make hand gestures a practical input method for anyone using wearable or portable devices.
Instead of cameras or EMG sensors, Grab-n-Go uses active acoustic sensing. Two speakers in the wristband emit sound waves, which bounce off the hand and any objects being held. Two microphones capture the reflections, which are analyzed to determine hand shape, grasp pose, and object geometry.
The reflections change with finger position, object material, and movement. The system uses deep learning to interpret the signals and recognize 30 microgestures across five grasp types, cylindrical, spherical, palmar, tip, and hook, based on Schlesinger’s grasp taxonomy.
The hardware is compact. Speaker-microphone pairs sit on circuit boards inside 3D-printed housings that slide along a silicone wristband to fit different wrist sizes. A microcontroller powers the system with a LiPo battery, and an onboard amplifier strengthens the signals. Data can be stored on a microSD card or sent over Bluetooth Low Energy to a smartphone for processing.
Each speaker works in a different frequency range, 18–21 kHz and 21.5–24.5 kHz, allowing the system to distinguish echoes with band-pass filters. By combining signals along four paths, the wristband builds an acoustic map of the hand and object.
In tests, Grab-n-Go performed reliably. Across 10 participants using 25 objects, it achieved 92% recognition accuracy. Tests with 10 deformable objects, like soft containers, showed similar results.
Grab-n-Go solves a barrier in making hand gestures a practical input method. The field is still open, and it is unclear whether wristbands, smart rings, or other devices will bring hand gestures into daily use.
