What if robots could learn to handle objects without watching humans? Researchers found that the way training data is created can shape what robots learn.
Teaching robots to handle objects remains one of the hardest problems in robotics. Researchers from NYU Tandon and the Robotics and AI Institute have shown that robots can learn many of these skills without human demonstrations by training on data generated through planning algorithms. Their results suggest that the quality and consistency of training data can affect how well robots learn manipulation tasks.
The researchers focused on tasks that require changes in grip, contact, and movement, such as rotating an object within a robotic hand or using two robot arms to reposition a large object. These tasks are difficult to teach through teleoperation systems because human operators must control many simultaneous movements and contact points.
During the study, the researchers found that a widely used planning method known as rapidly exploring random trees (RRTs) often produced inconsistent demonstrations, even when solving the same task repeatedly. The randomness that helps RRTs discover solutions also creates variations in the resulting demonstrations, making it harder for robots to learn reliable behaviors.
To address this problem, the team developed alternative planning approaches designed to generate more consistent training data. One method emphasized progress toward a goal, while another used a library of predefined motions to reduce variation across demonstrations.
The researchers tested the approach on two manipulation tasks. In one experiment, two robot arms rotated a cylinder by 180 degrees while changing their grip positions. In another, a robotic hand reoriented a cube inside its palm to match target orientations.
Robots trained on the more consistent demonstrations performed better than those trained on standard RRT-generated data. In the dual-arm manipulation task, the system achieved near-perfect performance using 100 demonstrations. The learned behaviors were also transferred from simulation to physical robots without retraining. The dual-arm robot completed 90% of real-world trials successfully, while the robotic hand achieved a success rate of about 62%.
The findings highlight the use of motion planning together with machine learning. Rather than serving only as a tool for finding robot motions, planning algorithms can also generate training examples for learning systems. The results further suggest that structured demonstrations may be more valuable than increasing the amount of training data.
