Learn from the demonstration

ManiSkill supports various imitation learning methods through a unified API, and provides multiple ready-made, tested baselines for use and comparison. The following page demonstrates how to set up a dataset/environment for imitation learning from demonstrations, as well as how to use the baseline. All baseline results are published on our public wandb page. On this page, you can filter by algorithm used or environment type. We are still conducting all experiments, so not all results have been uploaded yet.

Settings

This page records the key points to be aware of when using demo data for learning in the ManiSkill environment, including:

• How to download and replay the trajectory into a standard dataset , which is used for benchmark tests of imitation learning based on state and vision
• How to fairly and correctly evaluate the trained model
• Some common considerations

Download and replay trajectory / standard dataset

By default, to enable fast downloading and a smaller file size, ManiSkill’s demo data is stored in a highly simplified/compressed format that does not contain any observation data. Run the following command to download the original minimized demo data:

bashpython -m mani_skill.utils.download_demo "PickCube-v1"

To ensure everyone uses the same preprocessing/replay dataset, please run the following script: ManiSkill/scripts/data_generation/replay_for_il_baselines.sh at main · haosulab/ManiSkill · GitHub. Note that some scripts use GPU simulation (marked with -b physx_cuda) for replay, which may require more GPU memory than you have available. If needed, you can reduce the number of parallel replay instances by setting the --num-procs parameter to a lower value.

The script sets up the configuration for trajectory playback to generate observation data and establish the desired action space/controller for all baseline tasks. All baseline training results recorded in the Wandb project are generated using the data produced by the above script playback.

If you need more advanced trajectory playback examples (such as generating point clouds and changing controller modes), please refer to the trajectory playback documentation . If you want to generate the original dataset locally, we have saved all the scripts used for dataset generation in the data_generation folder.

Evaluation

Due to the presence of various types of environments, algorithms, and evaluation methods, a consistent and standardized approach is described below to fairly evaluate the various demonstration learning policies in ManiSkill. In short, to ensure fair evaluation, the following settings are required:

• Disable partial resets: the environment is not reset upon success/failure/termination (ignore_terminations=True). Instead, various success/failure metrics are recorded.
• All parallel environments are reconfigured upon reset (reconfiguration_freq=1), which randomizes the geometric shapes of objects in the task (if the task includes objects).

The following provides code examples for a fair evaluation policy in ManiSkill and recording standard metrics. We provide options for both CPU and GPU vectorized environments, as you may want to evaluate your policy on the same backend used for collecting simulation data based on your demonstration data.

Evaluation Code Example for GPU Vectorized Environment

pythonimport gymnasium as gym
from mani_skill.vector.wrappers.gymnasium import ManiSkillVectorEnv
env_id = "PushCube-v1"
num_eval_envs = 64
env_kwargs = dict(obs_mode="state")  # 根据需要修改 env_kwargs
eval_envs = gym.make(env_id, num_envs=num_eval_envs, reconfiguration_freq=1, **env_kwargs)
# 在此处添加其他需要的 wrappers
eval_envs = ManiSkillVectorEnv(eval_envs, ignore_terminations=True, record_metrics=True)

# 评估循环，仅记录完整回合的指标
obs, _ = eval_envs.reset(seed=0)
eval_metrics = defaultdict(list)
for _ in range(400):
    action = eval_envs.action_space.sample()  # 用你的策略动作替换该行
    obs, rew, terminated, truncated, info = eval_envs.step(action)
    # 注意，由于没有部分重置，所有环境的 truncated 同时为 True
    if truncated.any():
        for k, v in info["final_info"]["episode"].items():
            eval_metrics[k].append(v.float())
for k in eval_metrics.keys():
    print(f"{k}_mean: {torch.mean(torch.stack(eval_metrics[k])).item()}")

Evaluation Code Example in CPU Vectorized Environment

pythonimport gymnasium as gym
from mani_skill.utils.wrappers import CPUGymWrapper
env_id = "PickCube-v1"
num_eval_envs = 8
env_kwargs = dict(obs_mode="state")  # 根据需要修改 env_kwargs

def cpu_make_env(env_id, env_kwargs=dict()):
    def thunk():
        env = gym.make(env_id, reconfiguration_freq=1, **env_kwargs)
        env = CPUGymWrapper(env, ignore_terminations=True, record_metrics=True)
        # 在此处添加其他需要的 wrappers
        return env
    return thunk

vector_cls = gym.vector.SyncVectorEnv if num_eval_envs == 1 else lambda x: gym.vector.AsyncVectorEnv(x, context="forkserver")
eval_envs = vector_cls([cpu_make_env(env_id, env_kwargs) for _ in range(num_eval_envs)])

# 评估循环，仅记录完整回合的指标
obs, _ = eval_envs.reset(seed=0)
eval_metrics = defaultdict(list)
for _ in range(400):
    action = eval_envs.action_space.sample()  # 用你的策略动作替换该行
    obs, rew, terminated, truncated, info = eval_envs.step(action)
    # 注意，由于没有部分重置，所有环境的 truncated 同时为 True
    if truncated.any():
        for final_info in info["final_info"]:
            for k, v in final_info["episode"].items():
                eval_metrics[k].append(v)
for k in eval_metrics.keys():
    print(f"{k}_mean: {np.mean(eval_metrics[k])}")

The following are the metrics for recording and explanation:

• success_once: Whether the task is successful at any point during the round.
• success_at_end: Whether the task is successful in the last step of the round.
• fail_once / fail_at_end: Similar to the above two, but used for failure cases. Note that not all tasks have a success/failure criterion.
• return: The total reward accumulated during the entire round.

For demonstration learning purposes, the only important metric is “success_once”, which is also the metric typically reported in ManiSkill-related research and work.

Common Precautions

Generally, if the demonstration data is collected in a simulated environment such as PhysX CPU, you need to ensure that any policy trained on this data is evaluated using the same simulation backend. For highly precise tasks (such as PushT), even an error of 1e-3 can lead to different results, which is particularly important. This is why all the demonstration data replayed using the trajectory playback tool is marked with the simulation backend used in its trajectory file name.

The source of your demonstration data significantly affects the training performance. Classic behavior cloning methods can effectively mimic the demonstration data generated by neural network/RL policies, but they struggle with multi-modal demonstration data (such as human teleoperation or motion planning-generated data). Methods like Diffusion Policy (DP) are designed to address this issue. If you are unsure, all ManiSkill official datasets provide detailed descriptions of data collection methods and types in the JSON file of the trajectory metadata.