A new framework lets robots acquire unfamiliar manipulation skills from one human video, reducing training time while preserving previously learned abilities consistently.
Researchers at Beijing Institute of Technology, X Square Robot and Tsinghua University have developed HOST (Human-to-robot One-Shot Skill Acquisition), a framework that enables robots to learn new manipulation skills from a single human demonstration video. The system acquired unfamiliar skills in an average of 29 seconds and achieved a 62% average success rate in testing.
The approach is designed to address a major limitation in robotic learning: robots often perform well only on tasks included in their original training. Teaching them additional skills can be costly and time-consuming, while further training may also affect abilities they have already mastered.
HOST instead analyses a human demonstration and translates the observed procedure into actions suitable for a robot. The framework first uses camera images to determine the robot’s progress through the demonstrated procedure. It then predicts the next part of the task and finally derives the actions the robot should perform, repeatedly updating its estimate as the task progresses.
Researchers tested the system on 50 previously unseen physical manipulation tasks involving different objects, tools and movements. Each task was attempted 20 times, with objects positioned differently or given varying orientations.
According to the reported results, HOST exceeded a zero-shot baseline by 45% while retaining previously mastered skills. It also required 50 fewer robot demonstrations per task than a baseline fine-tuned approach and acquired each skill 507 times faster.
The researchers note that existing methods rely on a cumbersome training loop that can be expensive and slow. HOST is intended to make skill acquisition more efficient by learning directly from human video without requiring the robot’s underlying framework to be retrained.
The results suggest a potential route towards robots that can adapt more quickly to unfamiliar physical tasks. Further testing and refinement will be needed to establish how reliably the approach transfers across robots and broader real-world environments.
