8) 2 Embodied AI: Beginner’s Guide

• Open-source content: https://github.com/datawhalechina/every-embodied
• Learning guide knowledge base: Learning Guide
• Q&A section: Question records and summaries
• Teaching assistants should apply for editing rights to this document. All information such as check-in records and excellent assignments should be summarized in: 202603 Group 1 and 202603 Group 2
• Course description:

Embodied AI is very trendy. From Mobile ALOHA to Pi0 and OpenVLA, SOTA models are updated almost every week. However, many people want to get involved but get stuck at the first step:

🤯 The materials are too fragmented; not sure where to start learning from?

🤯 The threshold is too high; the mathematical derivations and physics engine are overwhelming?

🤯 Can’t run, and the environment configuration is so confusing?

Don’t panic! This beginner-friendly tutorial, Every-Embodied: Zero to Hero, initiated by Datawhale and refined by the open-source community, has been officially opened sourced!🌟 Our focus is on: easy for beginners, reproducible for engineering, and adaptable for cutting-edge advancements. It takes you from a single line of Python code, step by step, to equipping the robot with a “brain”!🧠

This version adds priority reproduction suggestions and advanced reproduction suggestions on top of V3’s “begin with general knowledge, then branch out” approach. The default rule is: for each Task, prioritize completing one reproduction that is the most stable, classic, and easy to achieve results; other platforms, models, and environments can be chosen as advanced options. This helps beginners avoid having no idea where to start when dealing with multiple environments such as ManiSkill, GenieSim, Isaac Sim, Habitat, OpenVLA, SmolVLA, GR00T, and SIM1 simultaneously.

The latter part is divided into three categories: Manipulation Model and World Model, Navigation Model and UAV Navigation, and Application Practice and System Deployment. Those who are interested in robotic arm manipulation, grasping, VLA, ACT, Pi0, SmolVLA, OpenVLA, and world model can enter the Manipulation Model and World Model branch; those who prefer navigation, VLN, scene understanding, UAV navigation, and spatial task planning can enter the Navigation Model and UAV Navigation branch; those aiming to develop development boards, teleoperation, simulation tools, data synthesis, and multi-modal demos can enter the Application Practice and System Deployment branch.

• Target audience: Students, engineers, and researchers interested in embodied AI; those with a basic understanding of Python and mathematical knowledge. Beginners in Python / AI developers: Those who want to systematically transition into embodied AI fields; university students: Those needing course projects or thesis work, or those aiming to quickly develop the ability to reproduce research results; hardware/robotics enthusiasts: Those seeking to establish a complete closed loop from physical machine control to Sim2Real deployment.
• Course ID: 929
• Number of learning groups: 2
• Number of participants per group: 180

Task Schedule: