Sim-to-real transfer is training a policy in a physics simulator, where trials are free and unlimited, and deploying it on a real robot, where every trial costs time and carries risk of damage. The appeal is obvious: a simulator can run thousands of episodes per hour with perfect, automatically generated labels, at a pace no teleoperated collection session can match. The catch is just as obvious once you deploy — a policy is only as good as the physics it was trained against, and simulated physics is never quite real physics.
What the reality gap actually is
The reality gap is the accumulated difference between what a simulator models and what actually happens on hardware. It matters because of how policies are optimized, not just because simulators are imprecise: an RL or trajectory-optimization process trained purely in simulation will happily learn to exploit any quirk of the simulator — an unrealistically forgiving contact model, an idealized actuator response — because that quirk is a valid, cheap way to solve the task inside the simulation. None of that exploited behavior is required to be robust to how the real world actually diverges from the model, so the policy can score perfectly in simulated evaluation and still fail the moment it runs on the real robot.
Where the gap comes from
Four sources dominate in practice:
- Contact dynamics. Friction, restitution, and what happens at the moment of contact are the hardest thing to simulate accurately, and the error compounds fastest exactly in tasks — insertion, grasping, assembly — that are defined by contact.
- Actuator models. Real motors have backlash, cable stretch, current limits, and response latency that idealized torque or position actuator models in a simulator routinely omit or approximate.
- Sensor noise. Real cameras have rolling shutter, exposure artifacts, and lens distortion; real IMUs and force sensors have drift and noise spectra that synthetic sensor models rarely reproduce exactly.
- Rendering. For any policy that consumes camera images, the simulator's rendered textures, lighting, and materials differ from a real camera's output even when the underlying geometry and physics are correct — a distinct gap from the physics gap, and one that hits vision-based policies specifically.
Closing the gap
Three approaches, often combined rather than chosen between:
- Domain randomization trains across a wide, randomized range of simulated parameters — textures, lighting, friction, mass, sensor noise — instead of one fixed, best-guess configuration, so the policy that survives training is one that's robust across the range rather than tuned to a single simulator's assumptions. OpenAI's Dactyl project used automatic domain randomization (ADR) — progressively widening the randomized range as the policy improves — to train a real robot hand to solve a Rubik's Cube entirely in simulation (Solving Rubik's Cube with a Robot Hand, 2019), one of the clearer demonstrations that sufficiently aggressive randomization can substitute for an accurate model.
- System identification takes the opposite approach: measure the real robot's actual parameters — mass distribution, friction coefficients, motor response curves — directly, and tune the simulator to match them, rather than randomizing blindly across a range that may or may not contain reality.
- Real-to-sim reconstructs simulation assets and parameters from real-world data — scanned meshes, video-derived object models, measured dynamics — so the simulated training distribution is closer to the deployment distribution by construction, instead of being hand-authored and hoped to be close enough.
Where simulation wins, and where it doesn't
- The task is dominated by rigid-body dynamics with simple, well-characterized contact — legged locomotion, wheeled navigation.
- Failure is recoverable and gradual rather than catastrophic, so a policy can be robust to modeling error without needing to be exact.
- The action space is coarse relative to the reality gap's magnitude — approach and transport motions, not fine insertion.
- The task is contact-rich — insertion, snap-fits, connector mating — where friction and contact dynamics are hardest to simulate and matter most to the outcome.
- The manipulated object is deformable — cloth, cable, food — where the underlying physics is difficult to simulate accurately at all, not just difficult to match to a specific real object.
- Success depends on fine force regulation, which most simulators do not model with the fidelity real force/torque sensing captures.
ETH Zurich's ANYmal quadruped is the widely cited example of the first column: trained entirely in simulation with a learned policy, it transferred to agile, real-world legged locomotion — including recovering from falls — beyond what prior hand-engineered controllers achieved (Hwangbo et al., Learning agile and dynamic motor skills for legged robots, Science Robotics, 2019). Contact-rich manipulation sits at the other end: the force and tactile signal that distinguishes a clean insertion from a jammed one is exactly the signal simulators reproduce least faithfully, which is why teams collecting that data still lean on real teleoperated demonstrations rather than simulation.
Why most stacks mix simulated and real data
The practical response to an uneven reality gap is not to pick simulation or real data — it's to use each where it's strong. Simulation gives cheap, safe, perfectly labeled coverage of scenarios that are expensive to stage for real, including rare edge cases and long-tail scene variation. Real teleoperated demonstrations supply the contact dynamics and sensor characteristics simulation can't yet reproduce. Co-training on both, or using simulation to pretrain a policy that real data then fine-tunes, generally produces better coverage than either source alone — the same lesson that shows up in imitation learning more broadly: which distribution the data covers matters more than which source it came from.