Depth and Reinforcement Learning in Embodied Scenarios
This directory collects tutorial materials for the reproduction of reinforcement learning, imitation learning, and deep learning in embodied scenarios. It is recommended that you first understand the task boundaries and simulation platforms, before moving on to specific code and training commands.
Chapter Entry
- Reinforcement Learning for Multi-Robot Furniture Handling Policy : Introduction to the cooperative humanoid robot handling task of CooHOI.
- HIMLoco quadruped robot motion control reproduction in Isaac Lab : Understanding from the paper, troubleshooting the old Isaac Gym stack, to the new Isaac Lab stack smoke test.
- AGILE humanoid robot Loco-Manipulation in Isaac Lab reproduction : Understanding the AGILE humanoid robot RL workflow, official task boundaries, local Isaac Sim 5.1 replication video, and evaluation export pipeline.
- Guide on Reward Alignment for Robots That Know What to Ask : Analyzing how ASQ identifies under-specification reward features during teaching, and explains how to guide humans to provide corrective demonstrations using natural language.
- Dual-legged reinforcement learning for Open Duck Mini and Microduck : Distinguishing the two generations of duck robot models, completing URDF/STL asset download, Microduck PPO smoke test, formal training, video replay, and ONNX export pipeline.
- Reusable score matching motion prior in SMP : Analyzing how diffusion motion priors transform into PPO rewards through SDS, ESM, Adaptive Normalization, and GSI, and providing the original implementation of MimicKit and the open-source reproduction route for Unitree G1.
- Guide on Unified cross-skills dexterous manipulation with UniCross : Starting from hand-object relationship motion, analyzing the unified observation, action, and reward for grasping, moving, hand-in rotation, and translation, as well as how ten PPO experts can be distilled into a single cross-skills strategy via DAgger.
Learning Recommendations
If you are new to reinforcement learning for robots, it is recommended to first read the HIMLoco section to understand the basic concepts of proprioception, privileged critic, and sim-to-real; then conduct a smaller-scale bipedal training exercise using Open Duck Mini/Microduck to get a clearer view of assets and commands; next, study SMP to learn how to freeze action diffusion models into reusable motion prior rewards; after that, look at UniCross to understand why unified task formulations allow multiple contact skills to share the same policy; finally, read the AGILE section to learn about more comprehensive humanoid robot task configurations, teacher-student distillation, evaluation reports, and Sim2MuJoCo review processes. Students interested in VLA, post-training world model training, or human demonstration data can continue with Robots That Know What to Ask, focusing on understanding reward misalignment, under-specification features, and human-in-the-loop reward correction.