Sim-to-Real Transfer in Robot Manipulation: Domain Randomization Strategies, Tactile Sensor Simulation, and Adaptive Policy Refinement for Dexterous Grasping of Novel Objects
Sim-to-real transfer -- the process of training robot control policies in simulation and deploying them on physical hardware -- offers the promise of unlimited safe training experience but is undermined by the reality gap: systematic discrepancies between simulated and physical dynamics that cause policies to fail on deployment. Dexterous manipulation is particularly sensitive to this gap due to its dependence on contact dynamics, friction, and object deformation that are notoriously difficult to simulate accurately. This paper presents a comprehensive study of sim-to-real transfer techniques for dexterous grasping, evaluating domain randomization strategies, tactile sensor simulation fidelity, and adaptive policy refinement methods on a 16-DOF robotic hand platform. We evaluate three domain randomization approaches -- uniform randomization, adaptive domain randomization (ADR), and automated domain randomization using Bayesian optimization -- across 48 novel object categories not present in training. ADR with tactile sensor simulation achieves 78.4 percent grasp success rate on novel objects, compared to 51.2 percent for uniform randomization without tactile sensing. A key finding is that tactile feedback simulation -- implemented through a GelSight sensor model integrated into the IsaacGym physics engine -- contributes more to sim-to-real transfer success than any single domain randomization parameter, improving novel object grasp success by 19.3 percentage points. Post-deployment adaptive policy refinement using 30 minutes of physical interaction data (DAgger-based) closes the remaining sim-to-real gap to within 4.1 percentage points of simulation performance.
