Robots can watch humans, but can they understand touch? A glove captures both movement and contact during tasks.
Robots can learn from watching humans, but capturing the details of how people handle objects remains a challenge. Researchers at Carnegie Mellon University have developed ART-Glove, a wearable system designed to record both hand movements and contact information during manipulation tasks, providing data that could help train more capable robotic systems.
The glove captures not only how the hand moves but also where and how it touches objects. This is important for tasks such as turning a key, using a tool, or opening a container, where successful manipulation depends on contact forces as well as finger positions. By combining motion and touch data in a single system, the researchers aim to create more complete training datasets for robot learning.
Collecting such data has been difficult with existing methods. Vision-based systems can track hand motion but often miss contact details when fingers or objects block the camera’s view. Teleoperation systems can generate robot-ready demonstrations but may restrict natural movement. Soft sensing gloves can capture touch information, yet their flexible surfaces make it difficult to determine precise contact locations.
To address these limitations, ART-Glove uses 16 rigid functional surfaces distributed across the fingers, thumb, and palm. These surfaces define known contact regions, allowing the system to accurately track where interactions occur during manipulation. The fingers are divided into distal, middle, and proximal segments, while separate sections cover the thumb and palm.
The structure is connected through 22 anatomically aligned joints that track natural hand movements, including flexion, extension, and abduction. This design enables detailed motion capture while maintaining hand dexterity.
The resulting dataset combines 22 degrees of freedom of hand motion with real-time tactile measurements. This allows researchers to capture not only how a hand moves during a task, but also how contact with an object changes throughout the manipulation process. Such information could help robots better understand and reproduce the interactions required for dexterous object handling.
