Robots trained in virtual worlds are improving at moving, picking objects, and making decisions in environments outside the lab.
Robots trained entirely in simulation are beginning to show more reliable performance in real-world environments, according to a set of robotics research papers presented by NVIDIA at the International Conference on Robotics and Automation (ICRA). The work focuses on reducing the long-standing “sim-to-real” gap, where robots trained in virtual environments fail to adapt outside controlled lab settings.
One of the projects, called COMPASS, trains robots fully inside NVIDIA Isaac Lab simulations before transferring those skills to different physical robot platforms. Researchers reported around 80% success across 20 real-world navigation trials involving autonomous mobile robots and humanoids. The framework also improved average success rates by 4.5 times compared with imitation-learning baselines.
Another system, Grasp-MPC, focused on robotic grasping in cluttered environments. Instead of following a fixed motion path, the robot continuously adjusts its movement while approaching objects. Researchers trained the model using two million simulated trajectories involving 8,000 objects. During real-world testing, the system achieved about 75% grasp success on unfamiliar objects, compared with 41% using baseline methods.
NVIDIA researchers also introduced Deformable Cluster Manipulation, a framework designed for handling tangled or flexible materials such as tree branches around power lines. The system trains robots to use their full arm instead of only a gripper, allowing machines to gather or move clusters of objects in a way similar to human movement.
Another framework, PEEK, helps robots ignore irrelevant visual clutter and focus only on objects needed to complete a task. NVIDIA said the system improved real-world robotic accuracy by up to 41 times for policies trained fully in simulation.
A separate collaboration involving Carnegie Mellon University, the University of Utah, and the University of Sydney introduced SEAL, a framework designed to prevent robots from carrying out actions that differ from their planned reasoning steps. The method allows robots to evaluate multiple action sequences before selecting the one that best matches the original instruction.
Beyond the individual projects, NVIDIA said its robotics ecosystem continues expanding through datasets and simulation platforms including Isaac Lab, Isaac GR00T X Embodiment Sim, and Omniverse NuRec. The broader focus across the research is to move robots from tightly controlled demonstrations toward systems that can adapt more reliably in real-world conditions.
