03 ACT Migration and DAgger Diagnosis on ROCm

This task focuses on ACT. The reduction in loss on the training set does not equate to successful closed loop deployment. Especially when replicating on ROCm devices, it is necessary to first prove that failures are not caused by the environment, data, memory, or success criteria before discussing the model structure and data policy.

Supporting practical notebook: 03_act_dagger_diagnostics.ipynb.

First, look at the conclusion

When learning, remember the following three points first; there is no need to distinguish many evaluation groups:

The 15/30 here is the reproduction result of the protection candidate that has been completed on AMD395; the same native training recipe has been written into the Notebook, but it still needs to be actually executed from scratch in an environment with a Jupyter kernel to be considered a “Notebook-native reproduction”. Therefore, the tutorial will present both the executable recipe and the audit results separately, without mixing them up.

ACT Diagnosis Route

It is recommended to check according to the following order:

mermaidflowchart LR
  A["数据动作回放"] --> B["ACT 训练"]
  B --> C["teacher-forced / open-loop"]
  C --> D["closed-loop rollout"]
  D --> E["失败 seed 视频"]
  E --> F["DAgger / oracle correction"]

The questions asked in each step are different:

Common ACT Failure Types

How to collect DAgger data

A suitable DAgger process for teaching:

1. Reset the reset-start baseline for training;
2. Fix a set of seeds for a closed loop rollout;
3. Identify the failed seeds;
4. Run the current policy for several steps ahead, such as 40 control ticks;
5. Switch to the oracle or scripted policy from the near-failure state;
6. Save the correction suffix;
7. Record the source, timestamp offset, and sampling weight when merging data;
8. Retrain and evaluate with the same set of seeds.

Note: The amount of correction data is not necessarily more is better. Directly mixing the high-weight data of full-reset failure-bucket into the main training may disrupt the already learned reset-start behavior.

What Did the Current ACT Change?

The old tutorial summary recorded 17/30. The current repair15 protection candidate has reached 15/30 under the same strict protocol, but the historical 17/30 has not been fully restored yet. W7900 normal ACT is 0/30, AMD395 stable61 fallback is 7/30, and the old protected DAgger artifact is 2/30. The original closed-loop baseline for ACT can hardly be determined by strict physical success, and subsequent diagnostics still come from a series of revisable changes.

The first step is to tighten the evaluation criteria. The old success only focused on environmental geometric conditions, which might misclassify pushing, squeezing, or pouring as successful actions. This special report follows a unified standard physical_success: the cup must be lifted and moved onto the plate, and in the final state, it should be basically upright. Without this step, subsequent training adjustments can easily be misled by the old metrics.

The second step is to separate failures into open-loop and closed-loop scenarios. We generate ACT actions using the dataset observation, and then place these actions back into MuJoCo in an open-loop manner. This diagnosis helps distinguish two issues: whether the model has learned the actions based on the data state, and whether failures occur due to state drift during the closed loop execution of the policy. The results show that under single demo or clean data conditions, ACT sometimes has near-usable arm trajectory, but the gripper release is very short, and the closed loop distribution deviation is also significant. Therefore, solving the problem cannot rely solely on adjusting the gripper or continuing training.

The third step is to redo the reset-aligned data. In the old data, many episodes do not start from the evaluation reset pose. During training, the loss appears normal, but the closed-loop does not touch the cup when starting from the reset position. The subsequent main pipeline will be changed to a reset-aligned scripted oracle: each trajectory starts from the reset state used during evaluation, ensuring that the data can be played back successfully according to the same physical standard.

Step 4 is to adjust the ACT training configuration. In the final main line, use no-VAE, chunk_size=20, n_action_steps=10, timestamp state, obj_init state, gripper BCE 0.5, and per-episode early weight. Each item here corresponds to a failure phenomenon:

Step five is prefix40 DAgger. Have the current ACT run 40 control ticks first, then switch to the scripted oracle to take over and save the suffix. In this way, what is collected is the “state after the policy deviates”, which is closer to the closed-loop failure distribution than using only a perfect reset-start teaching.

The sixth step is to add a timestamp offset for the correction episode. Prefix40 suffix is not the complete task starting from reset; if its timestamp also starts from 0, the model will mistake the “midway correction action” for the “task start action”. Therefore, timestamp offset = 2.0 is used for correction episodes to separate the time phase from the reset-start episode.

Step 7 is to weight down the correction data. We have tried different weights:

Finally, use the best025 checkpoint to perform another round of prefix40 DAgger v1 on the failed seed, obtaining 3 valid correction trajectories. Then continue training by merging using timestamp offset 2.0 and sample weight 0.25, resulting in the old protected DAgger checkpoint. Its current exact review is 2/30. Subsequently, starting from the stable61 fallback protection weight, use the same correction data to perform a 2500-step continuation with low learning rate and no memory leakage, yielding the current repair15 protection candidate:

textckpt/act_scripted_reset_oracle_plus_prefix40_dagger_best025_toffset2_downweight025_chunk20_n10_novae_gpu_5000_20260629_031208/step_5000

textAMD395 protected/act_stable61_to_dagger_nomemleak_step1500_strict15of30
physical_success = 15/30
fixed_seed_groups = 3 组，每组 10 条
model_sha256 = b9b178377995a674a06bc5d1500c8e7e7fc5d02649268855f892b3987bf5bfeb4

The currently auditable branches are: stable61/step2500 fallback on AMD395 is 7/30; the old protected DAgger artifact is 2/30; and the new repair15 protected candidate is 15/30. 17/30 in the historical summary has not been fully restored by the current protection weight, so it is temporarily listed as a historical target. Note that there are 17/40 successful expert recovery collections in the logs; the number of successful data collection cannot be used as the policy closed loop success rate.

The Notebook口径 must be specified separately: previously, the protection weight of 15/30 came from the low-learning-rate continuation wrapper of AMD395, not the result of the Notebook execution. notebooks/16_act_end_to_end.ipynb now includes the same native protected recipe (data packaging, trajectory weighted sampling, no-VAE, gripper BCE, tqdm, checkpoint, and strict evaluation), but before a native execution is completed in Jupyter runtime, 15/30 is not marked as a Notebook reproduction result.

What to Record on ROCm

ACT training usually does not require a high memory usage, but it is still necessary to record:

• batch size;
• chunk size;
• n_action_steps;
• whether no-VAE;
• whether there is gripper-assisted loss;
• training temperature;
• VRAM usage;
• whether kernel crash / OOM occurs.

If training fails, first use the device logs to determine whether it is a hardware or ROCm issue. If training is stable but the policy fails, prioritize returning to data and closed loop diagnosis.

Example of the Recreation Result for This Round

In this round of ACT replication, the simple clean closed loop baseline hardly meets the strict physical success criteria. After adding timestamp offset, downweight correction, and DAgger data, the stable fallback branch becomes 7/30, while the old protected DAgger artifact is 2/30; the new repair15 continuation has reached 15/30. This indicates that ACT has established a complete ROCm closed loop diagnosis case, but it still lags behind the old tutorial 17/30 by 2 items and has not fully aligned yet.

Figure 1: Variation of the physical success rate of the ACT closed loop in diagnostic steps. The improvement here comes from data and closed loop state correction, not simply by extending training time.

Figure 2: Physical success rollout in the ACT history DAgger record. It illustrates the grasping, handling, and release stages; the auditable result for the current best repair15 candidate is 15/30, while the old protected artifact remains 2/30.

Figure 3: Typical failure scenario of ACT best DAgger. Even if the environmental geometry occasionally approaches success, continue to check the lift height and final posture.

Checkpoint

After completing this task, organize these items: