Pi0/Pi0.5 Experiment Record

This task focuses on pi_0. The challenges in replicating pi_0 are not only related to training itself, but also include gated model permissions, model weight downloading, cache management, and large model initialization. First, complete the smoke test, and then decide whether to start the long training.

For the main method, see Pi0/Pi0.5 permissions, short training, and closed loop evaluation . The accompanying practical notebook: 05_pi0_smoke_gate.ipynb.

Important correction: The old 30 evaluation items are not location generalization

During the review of mujoco_env/y_env2.py, it was found that the old code was written as if seed != None: np.random.seed(seed=0). Although the evaluation passed in 1010-1039 and indeed changed the action sampling randomness of Pi0, the initial positions of cups and plates always came from environment seed 0. The 30/30 in the old scaffold can still prove that “different policy samples can be stable under the same scenario”, but it cannot prove position generalization.

The code has been changed to if seed is not None: np.random.seed(seed=seed). After fixing the issue, using the actual changed environment seed 41-44 as the strict-input visual/history head gate resulted in only 1/4; fixing the environment seed to 0 and using the same 12 policy sampling seeds for comparison are raw pi0 0/12 and pi0 + learned head 6/12. Therefore, the current conclusion is: self-collected complete task data and learned head significantly improve fixed scenarios, but in the next round, multi-position data must be collected again, and the old 30/30 cannot be used as the success rate of spatial generalization any longer.

First, check Hugging Face permissions

pi_0 depends on PaliGemma. Confirm first:

1. The Hugging Face account has accepted the gated terms of google/paligemma-3b-pt-224;
2. The token has read access to the public gated repository;
3. The remote machine can access Hugging Face;
4. The token was not found in the command line, Notebook output, or logs.

Recommended script format:

bashHF_TOKEN_STDIN=1 REQUIRE_PROXY=1 ./install_hf_token_for_pi0.sh

This script should read the token from a hidden input or stdin, first verify whoami, PaliGemma, and lerobot/pi0, and save the token only after all are passed.

1-step smoke test

After the permission is granted, run 1-step smoke first:

bashRUN_SMOKE=1 RUN_FULL_TRAIN=0 ./run_pi0_train_eval_after_hf_ready.sh

This smoke test proves:

• Gated model permissions are available;
• Dataset can be loaded;
• pi_0 policy can be constructed;
• At least one forward/backward/optimizer step can run successfully;
• Checkpoint saving mechanism is functional.

It does not prove that the policy has converged, nor does it indicate the final success rate.

Formal Training Gate Control

The formal training will be started only after the smoke test passes:

bashRUN_SMOKE=1 RUN_FULL_TRAIN=1 PI0_STEPS=20000 PI0_BATCH_SIZE=4 ./run_pi0_train_eval_after_hf_ready.sh

It is recommended to confirm before formal training:

Training Gate Control After Public Baseline Reconstruction

If the local checkpoint or temporary data root is lost, do not rush to continue experiments from the incomplete directory. First, return to the public teaching dataset and rebuild a clean pi_0 baseline:

bashgit clone https://huggingface.co/datasets/Datawhale/datawhale_eai_pnp_language demo_data_language

This public dataset corresponds to pi0_omy.yaml / smolvla_omy.yaml in this tutorial. It contains 20 episode clips of grabbing cups and placing dishes, totaling 2621 frames. observation.state has 6 dimensions, while action has 7 dimensions. After reconstruction, it is processed in the following order through gate control:

In the sample experiment, the loss of the public data 500-step expert-only checkpoint dropped from approximately 5.28 to 0.179, indicating that the training process has been restored. However, the closed-loop strict evaluation for red seeds 0-2 and blue seeds 0-2 is all 0/3 physical_success, and the old geometric success rate is also 0/3. In terms of video behavior, the model can touch, push cups, and occasionally lift them, but it may also tip the cup, fail to lift sufficiently, or even send the cup flying.

After continuing to resume from the 500-step checkpoint to 1500 steps, the final loss is approximately 0.046. Both intermediate and final checkpoints are saved properly, indicating that the resume, scheduler, optimizer state, and checkpoint writing paths have been restored. However, for the same red/blue seeds 0-2, the closed-loop strict small panel remains red 0/3 and blue 0/3, and the success rate of the old geometry is also 0. The failure pattern at 1500 steps is more like no-lift: the cup is hardly lifted steadily, or there is only a very small disturbance.

The meaning of this set of results is clear: both 500-step and 1500-step checkpoints are healthy baselines for training, not successful models. The decrease in loss does not automatically imply a successful closed-loop system; instead, error diagnosis using teacher-forced/open-loop methods, inspection of action representations, and design of phased data should be carried out, rather than blindly increasing the number of BC steps or adjusting post-processing methods around a failed rollout.

Post-training Physical Success Diagnosis

In the example experiments of this topic, pi_0 has completed tasks such as permission, download, policy construction, training, and open-loop replay diagnosis. The most important conclusion is not that “pi_0 has succeeded,” but rather that the raw policy and the mixed solution for diagnosis should be reported separately.

We first conducted a rigorous physical_success evaluation on a renumbered 2 blue 2 red dataset. This small dataset is used to reduce color bias, and we specifically observed whether pi_0 learned to grasping, transporting, and releasing at the end. Subsequently, the same raw policy was applied to 20 episodes of the complete demo_data_language to perform teacher-forced open-loop replay, thereby obtaining more representative action prediction results.

Figure 1: The pi_0 raw policy is very close to the target in representing the episode, but the final height and release stability do not exceed the strict success threshold. The script terminator combines the same prefix state with a more stable release / raise / stabilize suffix, indicating that the main bottleneck lies at the end of the task.

The video below shows a comparison between the left and right sides of the same episode. The left side is raw pi_0, while the right side is the mixed diagnostic result after adding the script terminator:

Figure 2: pi_0 episode2 raw-vs-hybrid comparison video. This video is used to explain the failure mechanism, and does not indicate that raw pi_0 has achieved 100% success rate.

From this diagnosis, two conclusions can be drawn:

1. The training process for pi_0 can now be executed on AMD ROCm devices, and the model has learned some actions.
2. The raw policy cannot be directly reported as a successful replication. Further optimization should focus on designing data or training objectives for the final release, elevation, and stable placement stages.

The reproduction of all 20 open-loop replays reveals a crucial detail: successful old geometry and strict physical success are not the same thing. Among the 20 episodes, only 1 ended in physical_success, 3 briefly reached physical_success, and 2 showed successful old geometry orientation. In other words, pi_0 does not completely avoid approaching the target, but it is not stable enough in contact, release, posture stabilization, and final state maintenance. During reproduction, one should consider final_physical_success, physical_success_ever, the old success, and the video, rather than just focusing on a single boolean value.

After further 20 seed closed loop strict evaluation, the strict physical success rate of raw pi_0 is 0/20, and the old geometric aperture is 2/20. This is closer to real deployment than open-loop, indicating that the current failures are not just due to tail-end motion errors in the dataset frames. As the closed loop image, state, and action gradually drift, the front-end contact, grasping stability, and final state retention will also amplify the errors.

Special attention should be paid to the boundaries of small data gripper manipulation tasks here. The pi_0 paper emphasizes large-scale, multi-task, cross-robot data, and carefully designed post-training recipes; it also regards data covering deviation states such as corrections and recovery behaviors as very important: π0: A Vision-Language-Action Flow Model for General Robot Control. However, our cup gripping data is small; if the episode does not include a stable sequence of actions such as “opening the gripper, raising the hand, waiting for the cup to stabilize, and making corrections if necessary,” the model will not learn this sequence out of thin air.

Additionally, pi_0 uses the action chunking approach, predicting a continuous sequence of actions at a time. Physical Intelligence explicitly discusses the boundaries of chunks, latency, and execution issues caused by pauses during training in Real-Time Action Chunking; Real-Time Execution of Action Chunking Flow Policies also identifies the boundaries of action chunking and real-time execution as key issues. In this small data collection task, the preceding movements can be averaged by the model, but the last few centimeters of release / raise / stabilize are easily broken by cumulative errors.

Therefore, the script finisher is not a “cheating way to announce success,” but a clean diagnostic tool: if a fixed tail segment is added, it can transform failure into success, indicating that the main gap in the raw policy indeed involves tail segment action modeling. However, the complete 20-template-tail only contains 4/20, which also shows that front-end contact, stabilization, handling, and posture control are equally important. Subsequently, either stage-specific data and correction data should be added, or the scripted finisher should be reported separately as a fallback mechanism in engineering, rather than being confused with the raw pi_0 success rate.

How to increase the success rate of pi_0

Write down the current hard baseline clearly first:

Any subsequent new scheme must be compared with these numbers. It is not enough to only look at loss, only look at the 2 blue and 2 red small sets, or only look at a single successful video. At least 20 complete open-loop cases and 20 seed closed-loop strict cases must be reported. If only the small sets are tested, it must be recorded as a diagnosis, not as a success rate.

Also note the directions that have been excluded, to avoid duplicate time consumption:

After these counterexamples, tcp_to_plate is the most valuable positive update so far. It expands the state from 19 dimensions to 22 dimensions:

textjoint6 + timestamp + phase_index_norm + phase_onehot11 + tcp_to_plate3

tcp_to_plate = tcp_link_xyz - plate_xyz. The first TCP position in the training data comes from prefix_end_debug.tcp_pos in the acquisition summary, and subsequent frames use eef_abs as an approximate target recorded from the previous frame. During closed loop evaluation, instead of using approximations, the real tcp_link and body_obj_plate_11 are directly read from MuJoCo. This step addresses the issue of “whether the later-stage policy knows where the plate is located”.

This experiment converts the original 10eps prefix120 suffix dataset into a 22D state, and initializes continued training with the old 19D finisher checkpoint for 250 steps. When loading the old checkpoint, a common issue arises: in the old model, normalize_inputs.buffer_observation_state.mean/std has a dimension of 19, while the new data has a dimension of 22. Direct loading would result in a shape mismatch. The solution is to copy the checkpoint and remove these two state normalizer buffers, then re-initialize the statistics for the new dataset; the backbone weights remain unchanged from the old model.

1. The training health status is normal: the loss from step 25 to step 250 fluctuates roughly between 0.012-0.019, with stable memory and temperature, without OOM or kernel crashes. Subsequently, oracle prefix180 is used to bring the cup into a manageable state, and then the 22D finisher takes over from the move_preplace stage schedule. The results for 10 seeds are strict 5/10, legacy 6/10. The successful seeds are 1000, 1001, 1004, 1005, 1006; there are two types of failures: one is when the cup is lifted but not moved to the center of the disk, resulting in a final state of xy, and 0.18-0.59 m; the other is that xy is near the disk but the cup falls, indicating that the release / stabilize process is not fully stable yet.

Create another comparison with only the gripper changed, but EEF/arm unchanged: the gripper of finisher does not use Pi0 prediction, instead opening and closing according to the current phase rules. As a result, it becomes strict 7/10 and legacy 7/10. This indicates that the gripper timing is indeed one of the key bottlenecks. It turned the previously failed 1002, 1003, 1008 into a successful operation, while the remaining failures are 1001, 1007, 1009, all concentrated in the red cups, and the final state is xy with a duration of approximately 0.14-0.24 m. In other words, the gripper rules can reduce “cup inversion/slippage”, but further data or phase targets are still needed for the handling point of the red cups.

To confirm that this route is not only occasionally effective on 10 seeds, the evaluation was expanded to policy sampling seeds (the environment is actually fixed at seed 0) 1010-1039. First, it was used in a loop according to 10 schedule templates. The results were strict 21/30, legacy 23/30, red cup 12/18, and blue cup 9/12. This result is clearly better than the previous failed branches, but it is still not stable enough. After mapping each failed sample back to a schedule episode, the issue became clear: all failures occurred during the schedule episodes 1, 7, 9, and for these three move_preplace transportation stages, only 72-75 frames were used; the move_preplace of the long template episode 0, 2, 3, 4, 5, 6, 8 is approximately 120-123 frames. The short template causes the finisher to enter lower / release state before the cup reaches above the plate, which visually appears as “the cup is released early when it reaches in front of the plate”.

Then, only one variable is changed: the schedules for failed seeds are forced to be converted into long-template episodes. 0Without changing the checkpoint, state, or controller, all 9 failed seeds become strict successes. Then, all 30 complete policy sampling seeds (actually fixed at seed 0 in the environment) are used with the long template episode 0, as a result, it is strict 30/30、legacy 30/30, Red Cup 18/18., Blue Cup 12/12, average final state xy Distance approximately 0.0281 m, maximum xy About 0.0739 m. This indicates that before 21/30 The main bottleneck is not policy sampling noise, but the inconsistent length of the transport phase in the schedule, which causes the short template to advance the release timing.

This result is important, but the boundaries must also be clearly defined: it is neither raw pi_0 nor the end-to-end deployment success rate. It relies on the oracle prefix, manually aligned schedule from move_preplace, the explicit tcp_to_plate status, phase-scripted gripper, and the mandatory use of the long move_preplace template. A more accurate statement is: the improved Pi0 backend finisher/scaffold can now complete tasks stably across 30 policy sampling seeds in a fixed environment; this proves that the engineering approach of “relative pose at the center of the disk + correct handling duration + correct gripper window” is correct. The next step is not to revert to blind addition of steps, but to make the long move_preplace progress, phase transition, and gripper timing into inputs or heads that can be learned by the model, and then gradually remove the oracle prefix and manual schedule.

The first scaffold that can be removed is the gripper rule. Perform two ablations first: if the scripted gripper is completely omitted, only the oracle prefix, tcp_to_plate finisher, and long schedule are retained. On policy sampling seeds (the environment is actually set to seed 0) 1010-1019, there is only strict 5/10; if the prefix is also changed to the current policy prefix, even with the scripted gripper and long schedule, there is only strict 3/10. This shows that the priorities are clear: first make the gripper timing a learnable module, and the prefix policy cannot immediately replace the oracle prefix.

A very small logistic gripper head was trained here. It does not modify the EEF/arm output of Pi0, but is only responsible for predicting the 7th-dimensional opening and closing of the gripper. A common pitfall is: a gripper head trained with the full 22D state can achieve good performance on the training set. 100% Accuracy, but deployed to seed 1010 Failure indicates that it has also learned the detailed correlations of joint/TCP; these correlations cause distribution shifts during the closed loop rollout. After changing to a phase-only input, only timestamp + phase_index_norm + phase_onehot11, same training set 100%, but the closed loop performance is stable: smoke seed 1010 For 1/1, policy sampling seeds (the environment is actually set to seed 0) 1010-1019 Keep the Markdown structure, indentation, lists, tables, and line breaks strictly intact. 10/10, complete 1010-1039 Keep the Markdown structure, indentation, lists, tables, and line breaks strictly intact. 30/30、legacy 30/30, Red Cup 18/18., Blue Cup 12/12, average xy About 0.0272 m, maximum xy About 0.0735 m。

The significance of this step is not simply “adding another rule,” but rather replacing the handwritten gripper rule with a trainable, saveable, and reusable small head. It also provides a practical insight: in small data robot tasks, having more auxiliary heads does not necessarily mean better performance. If the short event timing head only learns the phase window, a cleaner and more stable input is preferable; when all real-time joint/TCP data are fed in, the training set metrics may be good, but the closed loop distribution deviation will be immediately apparent.

The second block to be removed is the forced long move_preplace template. Previously, 30/30 relied on --tcpplate-force-schedule-episode 0, which essentially applied all seeds to the same long transport template. The new adaptive gate does not do this; instead, it allows the schedule episodes 0..9 to repeat normally: when the schedule is about to transition from move_preplace to lower_to_plate, it first checks the live tcp_to_plate_xy. If TCP is still far from the center of the disk, it continues to hold the last frame move_preplace phase; once the live xy value becomes sufficiently small, or reaches the safe maximum number of steps, the subsequent lower / pre-release / open tail will be played.

This round checked several thresholds. xy=0.05m,min_steps=20,max_steps=180 is the current steady state: policy sampling seeds (the environment is actually fixed at seed 0) 1010-1019 is strict 10/10, complete 1010-1039 is strict 30/30, legacy 30/30, red cup 18/18, blue cup 12/12, average xy is approximately 0.0281 m, maximum xy is approximately 0.0617 m. It no longer forces the use of long episode0, but can extend the transport segments of short schedule episode according to the live progress.

However, this result also needs to be clarified: 0.05m still includes the conservative fallback of max_steps=180. Taking the short template seed 1011 as an example, in the trace, it switches from move_preplace to lower_to_plate only in the finisher step 180. During this switch, live tcp_to_plate_xy is approximately 0.0574 m. This indicates that it is a reliable adaptive hold engineering version, not yet a fully learned phase transition head.

Lifting the threshold does not necessarily mean better. 0.09m is also 10/10 for the first 10 policy sampling seeds (the environment is actually fixed at seed 0), and seed 1011 can be actively switched to lower_to_plate at step 163, which looks more like a real progress gate; however, when extended to all 30 seeds, it reverts to strict 29/30, seed 1034 fails, and the final state xy spikes to 3.16 m. 0.08m for seed 1011/1034 is 2/2, but among the first 9 seeds in full-run, seed 1012 has already failed strictly. This counterexample is important: the phase gate cannot rely solely on a small panel of 10 seeds, nor can it only consider the final state xy; releasing earlier results in changes in posture, lifting duration, and contact stability.

Take another step forward and replace this gate with a logistic transition head. In the first version of the full head, tcp_to_plate_x/y/z, tcp_to_plate_xy, abs_z and the number of steps within the phase were used. The accuracy on the training set was approximately 99.27%, but the online seed 1010 failed directly: it switched to lower_to_plate around the finisher step 60, and at that time, the live tcp_to_plate_xy and 0.35 m levels were present. After diagnosis, it was found that both the raw x/y direction and z height can mislead the small head; close heights in the training set often indicate that it is possible to relax the constraints, but close heights in the closed loop do not mean that the core has been reached.

In the second version, only tcp_to_plate_xy + local_step_norm is retained. The accuracy of the training set drops to approximately 95.01%, but the closed loop becomes more stable. It shifts from failure to strict success at seed 1010, and the switch occurs in the finisher step 136, not at max_steps as a fallback; subsequently, policy samples seeds (the environment is actually fixed at seed 0) 1010-1019 to strict 10/10, with the complete 1010-1039 being strict 30/30, legacy 30/30, red cup 18/18, blue cup 12/12, and the average final state xy is approximately 0.0279 m. The maximum xy is approximately 0.0741 m. Out of 30 cases, 29/30 is triggered by the transition head, with the release step occurring between 126-180, averaging about 146.6; only seed 1021 reaches max_steps=180 safely as a fallback, but still results in strict success.

This result is closer to the “learnable phase transition head” than the previous fixed threshold gate. It also shows another small data insight: the accuracy of the training set is not the only goal; the input to the head should retain only truly stable causal cues. Here, xy + progress is more stable than xy + z + raw direction + progress, as it does not treat the height/direction correlation of the closed loop drift as a release condition.

When continuing to disassemble the front segment contact primitive, we cannot directly use all phase transition events as positive samples for training based on the last few frames of each “stage”. We first trained an all-head model; the training set metrics seemed good, but the smoke seeds 1010/1011 turned into 0/2. In the trace, descend_to_close triggered when TCP still exceeded the grasping floor by approximately 0.08 m at a time, and later close_to_lift also occurred earlier, with the cup ending before it enters the gripper. This counterexample is ideal for inclusion in the tutorial: the “phase” label does not equal the contact safety condition. A good training set accuracy may simply mean that we have learned “roughly when this demo transitions to the next stage”.

The fixed version keeps the Markdown structure, indentation, lists, tables, and line breaks intact. pregrasp_to_descend Change the tag to “Arrived at the target point”, and explicitly add it to the features. tcp_to_grasp_xy、abs_tcp_to_pregrasp_z、tcp_to_floor_z、abs_tcp_to_floor_z；descend_to_close Still using the phase-tail tag, but add it during deployment. descend_floor_guardIt is not allowed for TCP to close significantly higher than the grasping floor. In this case, smoke 1010/1011 Restore to 2/2, complete policy sampling seeds (the environment is actually set to seed 0) 1010-1039 Keep the Markdown structure, indentation, lists, tables, and line breaks strictly intact. 30/30、legacy 30/30. 30 items pregrasp->descend、descend->close、close->lift All of them 30/30 Triggered by the transition head; the floor guard blocked a total of [number] times. 342 Close at sub-high altitude. Also, write down the boundary samples clearly: seed 1029 the final state xy About 0.0996 m，seed 1018 the upright cos approximately 0.703All are close to the strict judgment threshold.

This version lacks the handwritten geometry transition of pregrasp/descend compared to the previous one, but it still retains the target-relative contact scaffold, floor safety guard, and the later tcp_to_plate finisher. It can be described as “engineering scaffold for further learning,” rather than being referred to as a raw pi_0 end-to-end success.

The fourth layer continues to process the posterior section of the schedule tail. The previous 30/30 still relies on dataset_schedule: even though the switch of move_preplace -> lower_to_plate can be determined by the head, the subsequent timestamp and phase one-hot values come from a specific dataset schedule. The new experiment replaces the finisher with dynamic_timed and starts from the move_preplace stage, with the dwell time of move_preplace/lower_to_plate/retreat set to 260/40/40. In this way, the phase schedule in the dataset is no longer played later, but the online phase tracker generates the timestamp and phase features.

The first complete policy sampling seeds (the environment is actually set to seed 0) 1010-1039 only has strict 27/30. The failed seeds are 1021, 1031, 1036, 1021 was lifted but stopped outside the disk by approximately 0.35 m, and 1031 was even thrown far away, while 1036 the blue cup remains about 0.22 m in front of the disk. At first, it is easy to think that “fixing move_preplace=260 is still not smart enough,” but after reviewing the code, a more fundamental issue is revealed: --tcpplate-prefix-target-state originally wanted the prefix policy to use tcp_to_target, but even when dynamic finisher did not have dataset_schedule, it took the same non-schedule state branch, resulting in the finisher receiving tcp_to_target instead of tcp_to_plate that it needed during training.

The fix involves changing the 22D state construction to be stage-aware: the prefix phase uses tcp_link - target_mug, and the finisher phase uses tcp_link - plate. Additionally, the contact primitive is restricted to the prefix phase to prevent it from leaking into later stages. After the fix, three failed seeds were tested separately, and all 1021/1031/1036 achieved strict success; when the same environment was used to run the full policy sampling seeds (the environment is actually fixed at seed 0) 1010-1039, strict 30/30, legacy 30/30, red cup 18/18, and blue cup 12/12 were also obtained. The average final state xy was approximately 0.0244 m, and the maximum value xy was 0.0993 m for seed 1036, which appears to be a boundary success sample close to the threshold.

However, when continuing to run the trace, a more subtle issue was discovered: after running multiple seeds with the same evaluator in succession, the initial object positions for some episodes would take values outside the sampling range. For example, with seed 1035, the initial cup position was approximately [0.2679, 0.1625, 0.8441], while the normal position for a separate reset(seed=1035) should be about [0.2944, 0.1912, 0.8447]. This isn’t that the policy is broken, but rather the evaluation protocol is not clean enough: SimpleEnv2.reset(seed) will reset the object positions, but it doesn’t completely clear the velocity, control values, and free-joint dynamics left over from the underlying MuJoCo mjData. The motion state of the previous episode contaminates the next episode, and after 100 settle steps, the object may already be slightly off-center.

So, two more clean protocol switches were added to the evaluator later:

• --fresh-env-per-episode: A new MuJoCo environment is created for each seed, resulting in the cleanest output. The small panel 1036/1035/1020/1029 is corrected to 4/4, but repeatedly creating environments in the large panel triggers an error in the asset provider, making it unsuitable as the default batch protocol.
• --hard-reset-sim-data: The same viewer/env is reused, but the underlying parser is reset before each env.reset(seed), clearing residual values of qvel / ctrl / free-joint. This is the recommended full 30-seed evaluation protocol now.

Run the entire policy sampling seeds again using --hard-reset-sim-data (the environment is actually fixed to seed 0) 1010-1039. The results are still strict 30/30, legacy 30/30, red cup 18/18, and blue cup 12/12. This time, the mean xy=0.0219 m, max xy=0.0450 m, minimum lift 0.1093 m, and minimum upright cos 0.9504 are as follows, with an average number of steps approximately 492.6. In other words, the stage-aware dynamic finisher still holds under the clean evaluation protocol, and seed 1036 is no longer a boundary sample; the previous 0.0993 m is more likely to be an evaluation artifact caused by episode-wide reset.

This experience is crucial: when the same evaluator serves both the prefix policy and the finisher policy, one cannot rely solely on a global switch to determine the relative state. tcp_to_target is the effective clue for grasping the front end, while tcp_to_plate is the effective clue for the later stage of handling and placement; if both are placed in the wrong phase, the model will not report an error directly, but the closed loop behavior will quickly become biased, excessive handling, or abandonment. This bug also explains why looking at training loss or 2 seed smoke alone is insufficient; only with a complete 30 seed panel can the state boundary issue be exposed.

The supporting scripts are placed in the code/pi0/ directory of this topic:

textcode/pi0/build_lerobot_state_phase_tcpplate.py
code/pi0/evaluate_pi0_two_stage_eef_abs.py
code/pi0/evaluate_pi0_two_stage_tcpplate.py
code/pi0/train_tcpplate_gripper_head.py
code/pi0/train_tcpplate_transition_head.py
code/pi0/train_tcptarget_contact_transition_head.py

The first script is responsible for converting the 19D phase-state LeRobot data into 22D tcp_to_plate data. The second script is a staged prefix/finisher evaluator. The third script adds live tcp_to_plate, schedule start point, gripper head, adaptive move_preplace gate, transition head, contact transition head, and stage-aware target/plate state to the evaluator. Currently, the evaluation script also supports --fresh-env-per-episode and --hard-reset-sim-data to eliminate residual dynamics across episodes; --hard-reset-sim-data is preferred for batch evaluation. The fourth script trains the logistic gripper head from LeRobot data, the fifth script trains the logistic transition head of move_preplace -> lower_to_plate, and the sixth script trains the small pregrasp_to_descend / descend_to_close / close_to_lift / lift_to_hold head of the target-relative contact primitive. In the notebook, only these scripts need to be called, and no long temporary debugging code should be inserted into cells.

What is even more worth continuing is the following route.

Route 1: Redefine the episode ending point

The task completion point for the cup grasping and placement task should be: the cup is placed on the plate, the gripper is open, and the cup remains upright and stable. It returns to the initial position, the robotic arm retracts, and a reset is waited for. If it is not required by the task, do not include it in the “success teaching” action.

This step aims to address two issues:

1. Where exactly does the model need to learn;
2. Where does the evaluation script start to determine the final state.

If the end segment wants the model to learn, record “opening the gripper, raising the arm, and waiting for the cup to stabilize” in an episode, and ensure that these actions have samples under different positions, colors, and slight deviations. If these actions are just part of the simulation reset, remove them from the data to prevent the model from treating the reset action as a task policy.

Route 2: Add correction data for “deviation status”

The paper emphasizes the corrections and recovery behaviors of pi_0, which is particularly crucial in small-data抓取. It only records perfect success trajectories. However, once the policy shifts the cup slightly, places it at the edge of the plate, closes the claws prematurely, or releases too early, it enters a state not seen in the training dataset.

The next batch of data should not only adopt the trajectory of “perfectly successful from reset”, but also specifically cover these states:

These data are best obtained from the on-policy state: first let the current pi_0 or the current best prefix reach an intermediate state that is prone to failure, and then let the scripted oracle or a human take over to correct it. In this way, what is collected is not the ideal state, but the state that the model actually encounters.

Route 3: Implement phased policies, rather than a single pi_0 managing the entire process

Current evidence indicates that it is very difficult to learn approach, grasp, transport, release, raise, and stabilize simultaneously with a single pi_0 BC. A more stable approach is to break the task into stages:

mermaidflowchart LR
  A["approach/pregrasp"] --> B["grasp/close"]
  B --> C["lift/stabilize"]
  C --> D["move_preplace"]
  D --> E["lower/release"]
  E --> F["raise/hold"]

Each stage has its own judgment:

There are two implementation approaches here:

1. Phase tag as input: Add phase one-hot / phase index to the state, so the model knows which segment it is currently in;
2. Phase head or small policy: Use pi_0 for the early stage, and train a separate finisher sub-policy for the later stage, especially for release / raise / stabilize.

Previously, both phase-state, phase+EEF, and eef_abs showed some local improvements, but no closed loop was successful. This indicates that “providing phase information” is a necessary clue, but it is not enough. The latest oracle-prefix handoff scan also supports this conclusion: both prefix 120/180/240/300 and suffix-only finishers fail, and the cup is often lifted and kept upright, but moved away from the tray. In other words, the problem is not just “when to hand over to the finisher,” but rather the stage goal, transportation direction, and release conditions of the finisher themselves are not well learned.

The first dense action-target trace is more direct: after seeding 1000 and using the oracle prefix 180, the phase span of the first 80 control ticks before the finisher is move_pregrasp 28步 -> move_grasp 35步 -> close_gripper 17步. In other words, after the cup is lifted by the oracle, the finisher starts from a phase of “approaching the cup and grasping again”. The corresponding TCP target to the center of the disk is average 0.481 m, while to the cup it is average 0.147 m; the y deviation of the target relative to the disk center remains positive, ranging from +0.295 m to +0.554 m. The center of the disk is near y=-0.25, but the predicted target by the finisher is still near y=0.20-0.29, essentially staying on the side of the cup. Most of the time, the controller indeed moves in small steps toward negative y, but the predicted target does not actually fall above the disk, so the cup is not stabilized at the center of the disk.

Then, a fast-phase diagnosis was performed: the move_pregrasp/move_grasp/lift dwell of the finisher was quickly pressed to step 1, allowing it to enter the later stage earlier. The prefix 180, seeds 1000/1001 remain 0/2 physical_success, and the final state xy is approximately 0.534/0.494 m. This indicates that phase restart is one of the bottlenecks, but simply manually accelerating the phase is not enough; it is also necessary to provide the plate-relative state, TCP-to-plate vector, or clear phase objectives to the model. The next step is to make the phase transition and target conditions reliable: when to transition from pregrasp to close, when to transition from move_preplace to lower, and what is the current TCP-to-plate relative vector? It cannot be guessed implicitly by the model from the image alone.

To further break down the details, we conducted a schedule-shift diagnosis: without changing the model weights, we simply started playing the dataset_schedule of the finisher from the starting point of move_preplace or lower_to_plate. Starting at move_preplace, 0/2 physical_success continued to play, but the final state xy of seed 1000 was reduced from half a meter to 0.140 m; seed 1001 still ended up at 0.410 m, and the gripper was almost fully open. Starting at lower_to_plate, the performance was worse, with both seeds around 0.44-0.45 m, indicating that the transition to the lowering/release stage occurred too late—the model hadn’t yet moved the cup above the plate. After adding phase-scripted gripper, it remained 0/2, but seed 1001 changed from 0.410 m to 0.171 m. This set of values is very useful: both the phase start point and the gripper timing indeed affect the result, but they only shorten the trajectory; they cannot solve the issue of target offset during handling independently. In the next version, we should not only adjust the dwell time or rule out the gripper behavior, but also explicitly incorporate conditions such as the plate center pose, TCP-to-plate vector, and progress within a stage into the subsequent policy.

Route 4: Use EEF/TCP for action representation; no more using joint delta

Joint-delta reduces the error in joint values, but after several hundred steps, it still causes TCP deviation. For this cup grasping task, the EEF/TCP goal is closer to physical movements: approaching the cup, lifting, moving onto the plate, lowering, and leaving.

In the current experiments, eef_abs is the one showing the most signs of learning: the teacher-forced small set reached 1/2, but the closed-loop still failed. It can be retained as an active action in subsequent analyses, along with stage labels and on-policy correction data. Do not treat “replacing with EEF action” as a standalone successful approach; it must be used together with stage-based data.

Pi0.5 Random Position Experiment: Why eef_abs Still Goes in the Wrong Direction

Fixing 4 trajectories only indicates that the training link is available, but it cannot verify visual positioning. Thus, in the next round, after fixing the environment seed, 53 completely successful blue cup trajectories with strict physical validity were selected from 80 random position candidates, and 11 continuous expert recovery trajectories after Pi0.5 deviation were added. Finally, a dataset of 64 episodes, 26,014 frames, and 20 Hz was obtained. All trajectories retain dual-camera images, language, 6D robot proprioception, and 7D EEF/gripper actions, without adding cup coordinates, plate coordinates, or GT phase.

The training results for this dataset are highly representative. After performing weighted continuous training on the restored trajectory, the offline gripper MAE decreased from 0.25659 to 0.08261, but the xyz MAE changed from 0.10312 to 0.10612; the strict fair-VLA using new random seeds 2120-2123 remains 0/4, and none of the four configurations achieved effective lifting. The video and final state coordinates show that TCP continuously moves toward the negative Y side where the plate is located, while the blue cup is positioned on the positive Y side.

At this point, we cannot just say “the vision has not been learned”; we also need to examine the structure of the labels themselves over time. After conducting a direction audit on 64 trajectories, we obtained:

Table 4: The world coordinate labels of eef_abs reverse direction according to the task phase. In the early stage, cups should point in positive Y direction, while in the later stage, plates should point in negative Y direction; however, the model outputs the later direction directly in the first frame.

The absolute EEF target is not an error indication, but it strongly couples the task phase and world coordinates. An action chunk contains positive Y targets during the approaching phase, and negative Y targets during the handling phase. When the small data model cannot reliably identify the current phase, the most common issue is not random jitter, but a direct collapse into another high-frequency phase of the data. Continuing to increase the gripper weight only makes the opening and closing claws more accurate, but it does not correct this spatial direction error.

Precise conversion to local EEF delta using MuJoCo FK

This trajectory does not need to be re-collected. Each frame stores the 6D joint states before the action execution q_t, and the first three dimensions of the original action are the next TCP absolute target. Therefore, the current TCP can be restored using MuJoCo positive kinematics, and then a local increment can be calculated:

texttcp_t = FK(q_t)
delta_xyz_t = eef_abs_target_xyz_t - tcp_t

Cannot directly replace with action[t] - action[t-1]. The target of the previous frame is not necessarily equal to the actual TCP of the next frame. Contact, control errors, and physical steps all cause differences; using the current joint state for FK aligns with the actual executed commands. During conversion, images, language, 6D proprioception, gripper labels, episode boundaries, and task text remain unchanged. Only the first three dimensions of the action are replaced, and the LeRobot quantile stats are recalculated.

The integrity check for this conversion is as follows:

Table 5: The EEF-delta transformation does not re-sample images or rewrite original data. It merely precisely transforms the world coordinate target into a local action in the current state.

After aggregation, not only the index set should be checked, but also the physical row order must be verified.

After completing the training of 64 data points for the first round of EEF-delta, we conducted execution horizon comparisons at new random positions. Under the same checkpoint and strict fair-VLA protocol, horizon 10 achieved 0/4 with seed 2130-2133, horizon 5 remained 0/4, and horizon 1 also failed with seed 2133. The progressive trajectory shows that the model does not always forget to close its claws: it closes at incorrect spatial positions or incorrect heights; instead, it may reopen when it gets closer to the cup. Frequent re-examination of the image without a repair phase order or chunk timing was observed.

Figure 3: seed 2133, four-view time series for horizon 5. The robotic arm moves above and around the cup, but it does not reach a stable grasping height or form an elevation. Key frame images should be viewed together with the progressive TCP/gripper logs; success of grasping cannot be determined merely based on the presence of the gripper moving.

Video 2: A full 25-second four-view rollout of Horizon 5 with the same seed 2133. The video retains the failure process to observe the approach height, claw closure timing, and action sequence. It should not be cut into segments that only appear “approachable”.

Then, continue to collect the coherent recovery. Pi0.5 performs 40, 80, and 120 real prefix steps respectively. The expert continuously takes over from the current state to strict success, and only the complete successful trajectory is saved:

Table 6: The longer the prefix, the deeper the deviation state of the policy execution. This table stores the complete successful trajectory of “policy actual execution prefix + expert actual continuous execution suffix”, rather than combining unexecuted counterfactual oracle labels into a single chunk. The model input still consists only of images, language, and 6D proprio data.

After combining the original 64 episodes and the new 27 episodes into 91 episodes with 35,284 frames, routine checks show that the number of episodes is correct. After sorting, index also matches 0..35283. However, when checking LeRobotDataset(..., delta_timestamps=...), it was found that some positions of action_chunk[0] do not equal the action in the current row. The root cause is not a change in the action value, but rather the physical row order of Parquet does not match the global index.

The original aggregated data contains 4909/35284 rows with physical_position != index. The first error occurs at the physical location 21105, and the index stored in this row is 21510. LeRobot first uses the physical location idx to obtain the current observation, and then idx + delta to obtain the future action. Therefore, simply checking whether “all indices are complete” will miss this issue, and the action chunk may still span to an incorrect frame. Previously, there were also disorder in the subsequent files for 64 data entries, so the first round of 800-step checkpoint cannot be used as a clean training result for further fine-tuning.

Do not rename or manually modify metadata in place during fixing. Use a separate output directory, write each episode as a parquet file, sort by frame_index, and reconstruct the global index, episode boundaries, file mappings, and numerical statistics:

bashexport AGGREGATED_ROOT=/path/to/pi05_eef_delta_aggregated_v30
export CANONICAL_ROOT=/path/to/pi05_eef_delta_canonical_v30
export AGGREGATED_REPO_ID=datawhale_eai_pnp_pi05_eef_delta_aggregated_v30
export CANONICAL_REPO_ID=datawhale_eai_pnp_pi05_eef_delta_canonical_v30

python "$TOPIC_ROOT/code/pi0/pi05_canonicalize_lerobot_v3.py" \
  --source-root "$AGGREGATED_ROOT" \
  --source-repo-id "$AGGREGATED_REPO_ID" \
  --output-root "$CANONICAL_ROOT" \
  --output-repo-id "$CANONICAL_REPO_ID"

HF_HUB_OFFLINE=1 PYTHONPATH="/path/to/lerobot/src:${PYTHONPATH:-}" \
python "$TOPIC_ROOT/code/pi0/pi05_validate_chunk_alignment.py" \
  --root "$CANONICAL_ROOT" \
  --repo-id "$CANONICAL_REPO_ID" \
  --horizons 10 50 \
  --uniform-samples 512 \
  --summary ./chunk_alignment_summary.json

The normalized copy still consists of 91 episodes and 35,284 frames. All non-index Arrow elements were written back episode by episode for a complete equivalence comparison. The real query verification by LeRobot covered the beginning, middle, end, and the first 1/2/5/9/10/49 frames at the end of each episode, as well as global uniform sampling: 1,318 positions were checked in the horizon 10 and 50 regions, resulting in a total of 553,560 action markers and 28,240 padding steps. The maximum absolute error was 0. This is the data threshold that allows the next round of training to begin.

First, define the path in the root directory of the MuJoCo PnP project, and then run the audit and conversion. DELTA_ROOT must be a new directory that does not exist, and the script will not overwrite the original data:

bashexport PROJECT_ROOT=/path/to/04mujoco复现ACT、Pi0、SmolVLA
export TOPIC_ROOT=/path/to/every-embodied/16-专题组队学习/04-AMD-ROCm策略复刻专题
export ABS_ROOT=/path/to/lerobot_eef_abs_v30
export DELTA_ROOT=/path/to/lerobot_eef_delta_v30

cd "$PROJECT_ROOT"

python "$TOPIC_ROOT/code/pi0/pi05_dataset_direction_audit.py" \
  --dataset-root "$ABS_ROOT" \
  --target-object blue \
  --early-frames 80 \
  --output ./notebook_runs/pi05_direction_audit.json

DISPLAY="${DISPLAY:-:0}" PYTHONPATH="$PROJECT_ROOT:${PYTHONPATH:-}" \
python "$TOPIC_ROOT/code/pi0/pi05_convert_eef_abs_to_delta.py" \
  --source-root "$ABS_ROOT" \
  --output-root "$DELTA_ROOT" \
  --repo-id datawhale_eai_pnp_pi05_eef_delta_v30 \
  --scene "$PROJECT_ROOT/asset/example_scene_y2.xml" \
  --position-profile pnp_generalization_v1

When the conversion is successful, the terminal will print episodes, frames, the first delta, the maximum space increment, the reconstruction error, and the new action stats. If max_spatial_abs significantly exceeds the EEF step limit configured by the collector, do not train immediately; prioritize checking whether observation.state truly represents the state before action execution, whether the robot joint order is consistent, and whether the scene/XML are identical to those during collection.

Note also the model initialization here. The value ranges and semantics of eef_abs and eef_delta are completely different, and the already fine-tuned eef_abs action head cannot be directly used. The new delta branch should retrain the action expert from the official Pi0.5 base, and first check whether the first-frame delta-Y changes from negative to positive before entering the closed-loop. If only the loss decreases, but there is no inspection of the lift, cup and plate positions, and video on the new seed, it still cannot be determined whether the action representation is truly effective.

Below is the corresponding ROCm fine-tuning template. In this example, PaliGemma VLM is frozen, and only the Pi0.5 action expert is updated; action-loss-weights does not supervise the posture placeholder dimension, which remains at 0, and gives twice the weight to the gripper. Episode-start, recovery episode, and gripper transition are weighted by the sampler, but they are not used as model inputs, so they still belong to raw-compatible training involving image + language + proprio:

The LeRobot source code tested in this topic was submitted as 00b5f657. In this version of the Pi0.5 loader, when reading weights with safetensors.torch.load_file(), the tied-weight metadata is not restored. After a strict load failure, a random model without loaded weights may be returned. A minimal fail-fast patch verified for validation against this submission is retained in the repository; run --check in your own LeRobot source code repository first, and it will be applied only if the patch matches the current version:

bashexport LEROBOT_SRC=/path/to/lerobot

cd "$LEROBOT_SRC"
git apply --check \
  "$TOPIC_ROOT/code/pi0/patches/lerobot_pi05_strict_safetensors_load.patch"
git apply \
  "$TOPIC_ROOT/code/pi0/patches/lerobot_pi05_strict_safetensors_load.patch"

If git apply --check fails, do not forcibly add --reject. First, check whether the current LeRobot has fixed the strict safetensors loading in the new version. If the code structure is different, verify it according to the current upstream implementation. After the patch takes effect, the model loading log should include Resolved tied weight alias: ... and All keys loaded successfully!. Any Returning model without loading pretrained weights should immediately terminate training.

bashexport PI05_BASE=/path/to/lerobot_pi05_base
export DELTA_ROOT=/path/to/pi05_eef_delta_canonical_v30
export DELTA_REPO_ID=datawhale_eai_pnp_pi05_eef_delta_canonical_v30
export OUTPUT_ROOT=/path/to/outputs
export STAGE1_DIR="$OUTPUT_ROOT/pi05_eef_delta_s800/pretrained_model"
export STAGE2_DIR="$OUTPUT_ROOT/pi05_eef_delta_s2400/pretrained_model"

PYTHONPATH="/path/to/lerobot/src:$PROJECT_ROOT:${PYTHONPATH:-}" \
python "$TOPIC_ROOT/code/pi0/pi05_rocm_finetune.py" \
  --model-dir "$PI05_BASE" \
  --dataset-root "$DELTA_ROOT" \
  --repo-id "$DELTA_REPO_ID" \
  --output-dir "$STAGE1_DIR" \
  --steps 800 \
  --batch-size 4 \
  --chunk-size 10 \
  --learning-rate 1e-5 \
  --warmup-steps 50 \
  --action-loss-weights 1,1,1,0,0,0,2 \
  --open-frame-sample-weight 1.4 \
  --episode-start-frame-sample-weight 4 \
  --episode-start-window 60 \
  --tail-episode-frame-sample-weight 2.5 \
  --tail-episode-count 27 \
  --transition-frame-sample-weight 4 \
  --transition-radius 10 \
  --eval-samples 24 \
  --eval-start-samples 8

After the 800-step gate is completed, continue training with a lower learning rate from this weight branch. Changing the sampler seed can prevent the complete replay of the weighted sampling sequence of the first segment:

bashPYTHONPATH="/path/to/lerobot/src:$PROJECT_ROOT:${PYTHONPATH:-}" \
python "$TOPIC_ROOT/code/pi0/pi05_rocm_finetune.py" \
  --model-dir "$STAGE1_DIR" \
  --dataset-root "$DELTA_ROOT" \
  --repo-id "$DELTA_REPO_ID" \
  --output-dir "$STAGE2_DIR" \
  --steps 1600 \
  --batch-size 4 \
  --chunk-size 10 \
  --learning-rate 5e-6 \
  --warmup-steps 50 \
  --seed 1 \
  --action-loss-weights 1,1,1,0,0,0,2 \
  --open-frame-sample-weight 1.4 \
  --episode-start-frame-sample-weight 4 \
  --episode-start-window 60 \
  --tail-episode-frame-sample-weight 2.5 \
  --tail-episode-count 27 \
  --transition-frame-sample-weight 4 \
  --transition-radius 10 \
  --eval-samples 32 \
  --eval-start-samples 16

The current trainer only writes the model weights, not the AdamW and scheduler state. Therefore, the second paragraph re-establishes the optimizer from the 800-step weights, which is not strictly equivalent to uninterrupted 2400-step training. The merged curve should mark the restart boundary at step 800, so the loss increase in the early stage of continued training is not misjudged as a sudden degradation of the model.

tail-episode-count=27 is only applicable to the data layout where “the last 27 episodes are indeed in a closed loop recovery phase”. When using your own data, it should be adjusted according to the manifest or set to 0; do not copy the numbers directly. chunk-size=10 is an experimental variable for the length of the short closed loop execution in this round, and is not a general optimal value for Pi0.5. The training script will first check the physical line position, global index, episode boundaries, frame_index, and chunk[0]; old disordered data will fail directly before loading the large model. The training log must also contain All keys loaded successfully!; at the end, check trainable_head_max_delta > 0 and frozen_vlm_max_delta = 0 to ensure that the action expert is actually updated, and the frozen visual language backbone is not mismodified.

canonical91’s 800-step gate: Improving offline does not mean the closed loop has learned

After 800 steps of retraining from the official Pi0.5 base, the structural access control remains 91 episodes / 35284 frames / 0 行错位, and the maximum error for chunk[0] with 273 episode boundary probes is 0. The training peak memory usage is approximately 11.14 GiB, with no OOM, kernel crash, or NaN occurrences; the action expert parameters do change, and the sampling parameters of the frozen VLM remain unchanged at 0.

Table 7: These indicators indicate that loading, backpropagation, and optimization are normal, but they are only offline results in terms of data status.

Strictly use a closed-loop with complete training instructions, eef_delta, execution horizon of 10, fresh environment, hard reset, and the same physical success criteria. The results are as follows:

Table 8: Both the training position and the new position failed. Therefore, it is currently classified as overall underfitting/phase-timeout failure, and cannot be simply described as “insufficient generalization”.

Figure 4: The left figure shows that the 800-step flow loss has decreased, while the right figure displays the data retention rate under different policy prefixes for continuous expert takeover. The retention rate for prefix 120 drops to 70%, indicating that a longer prefix indeed covers deeper policy deviation states.

The training coverage must be calculated correctly here. With 800 steps and a batch size of 4, only 3200 current observations are extracted, which is approximately 9.1% of the entire data 35284 frames. The weighted sampler will also repeatedly extract the beginning of the episode, the end of the recovery phase, and the gripper transition window. Although one observation monitors 10 future actions, it cannot be considered as 10 independent visual states. Therefore, 800 steps is a meaningful gate, not a sufficient number of training iterations.

The official configuration can also help calibrate the scale, but it cannot be copied mechanically. The custom data example provided in the LeRobot Pi0.5 documentation is 3000 steps × batch 32. LIBERO has an additional 6000 steps in its comparison; The official training configuration of OpenPI includes many fine-tuning tasks with tens of thousands of steps. Their data size, freezing strategy, and batch size are different from those in this experiment, so it is not possible to state “how many steps must be trained”. However, it is confirmed that 800 × 4 is more suitable as a link and direction gate, rather than a convergence evidence.

The step-by-step log further narrowed down the problem scope. Among the four new seeds, the plane distance of TCP when closest to the cup is approximately 5.4-6.7 cm, but it is still about 10.4-13.8 cm higher than the center of the cup body. The gripper is already close to closing; subsequently, TCP continues to move beyond the cup, and finally descends outside the cup. In other words, the model does not completely fail to recognize the blue cup, but the three stages of “continuing to approach, descending, and closing” are arranged incorrectly.

The logs also reveal the issue of control amplitude limitations: among the 432 recorded action rows, 123 have at least one XYZ component exceeding 4 mm, with the maximum single-axis increment reaching 9.56 mm. In contrast, the maximum single-axis increment in expert data is approximately 4.08 mm. Therefore, a fixed --eef-delta-max-step 0.004 reference is added as a subsequent control. It only indicates the upper limit of the controller’s physical step size and does not read the dataset’s min/max values, cup/plate coordinates, or phase; however, it indeed affects the execution action, so it must be labeled as an actuator-bound protocol. The results cannot be mixed with the fully amplitude-free raw rollout.

Figure 5: The seed 1003 comes from a position covered by training data. The policy closes above or beside the blue cup, then pushes the cup over the target. The maximum instantaneous lift of the cup is approximately 1.63 cm, which does not meet the strict lifting condition of 3 consecutive steps and at least 3 cm.

Video 3: Complete raw Pi0.5 failure rollout of canonical91 and 800-step checkpoint on seed1003 during training. It did not use target/plate coordinates, oracle prefix, external head, or scripted finisher.

s2400 Review: Gripper is more accurate, but closed loop direction still not mastered well

Rebuild the AdamW and cosine scheduler from the s800 weight, then train for 1600 steps. Although it is called a total of s2400, the optimizer state is not restored in the second phase. In the second phase, it remains batch=4, action-expert-only. Actually, 1600 × 4 = 6400 new observations are added, and the total for both phases is 9600, which is approximately 27.2% of all 35284 frames in canonical91.

Table 9: The continuous training mainly improves the accuracy of the gripper values, but the XYZ directions do not improve synchronously. The flow loss at different stages is affected by random noise and validation samples; therefore, it is not appropriate to compare only the last decimal place.

This time, only 4 runs will be performed. Both s800 and s2400 use 10 training environments and the same set of 10 new environments. For s2400, a fixed 4 mm step control limit is also applied for comparison:

Table 10: Training, new positions, and execution limits all failed. The limiter changes the execution action; it must be listed as an actuator-bound protocol separately. It also did not save the task here.

This result takes the issue one step forward from “whether it is just lacking unseen generalization”: the model fails to complete a closed loop even for the reset state of the training distribution. The first action direction probe also shows that s2400 has a correct rate of approximately 21.9% for all XYZ axes of the probes simultaneously, with an average direction cosine of approximately 0.15; further action-expert-only learning mainly leads to gripper values that resemble labels, without establishing stable visual positioning and phase direction.

First, perform GT delta playback: the training limit is not defaulted to 100%.

Instead of immediately adding training steps, a more fundamental question was asked: Can the dataset action be returned to the environment as is, and can it be completed according to the current strict protocol? This negative contrast must restore all the state semantics from the collection period.

1. Use the environment seed and pnp_generalization_v1 saved for each trajectory;
2. Fully clean MuJoCo qpos/qvel/ctrl/free-joint before each episode;
3. Use the object pose that naturally settles after a reset, without forcing the Z value from logical spawn obj_init back into the simulation;
4. In the recovery episode, first apply the Pi0 prefix saved at that time, then apply the expert suffix;
5. Replay in batches using the same environment and perform a hard reset to avoid repeatedly creating asset providers and file handles.

One of these items is missing, and the numbers will be misleading: when obj_init is forcibly written back to the reset state, only 6/10 appears in the first 10 items; when using settled pose but missing the position random profile, only 2/10 appears; after both profile and settled pose are aligned, the first 10 items become 9/10. When extended to all 91 items, the results are as follows:

Table 11: GT delta refers to the expert EEF-delta/gripper action stored in the current frame data. It is not model prediction, nor is it “target coordinates for the model”; it serves as a negative control to check data, reset, and ensure the consistency of the bridge.

The strictly failed base episode is 1,13,17,20,22,25,37,45,47, corresponding to environment seeds 1001,1020,1026,1030,1033,1039,1057,1067,1070. These are not recoveries, and cannot be explained by a recovery prefix. Continuing to use these trajectories as positive samples for BC will treat the action sequences that cannot be stably reproduced in simulation as the supervisory limit.

replayable82: Filter only 9 vulnerable bases, keep all recoveries

The new training copy removes the above 9 base episodes with strict GT playback failures from canonical91, while retaining 44 complete tasks and all 38 coherent recoveries, totaling 82 episodes / 31592 frames. Filtering is not simply deleting parquet:episode/index; the episode/index numbers need to be renumbered, global/episode stats need to be recalculated, and all non-index content needs to be compared column by column. Finally, a real LeRobot chunk query is performed.

Table 12: The 82/82 for replayable82 indicates that these GT trajectories meet the strict playback criteria, not the success rate of the Pi0.5 model.

Write the source episode id for the passed GT playback as a JSON array, and then let the canonicalizer perform filtering and renumbering:

bashpython "$TOPIC_ROOT/code/pi0/pi05_canonicalize_lerobot_v3.py" \
  --source-root "$CANONICAL91_ROOT" \
  --source-repo-id "$CANONICAL91_REPO_ID" \
  --output-root "$REPLAYABLE82_ROOT" \
  --output-repo-id "$REPLAYABLE82_REPO_ID" \
  --include-episodes-json ./replayable_source_episodes.json

replayable_source_episodes.json contains content from the GT playback results. Do not select manually based on episode length, loss, or final viewing experience. The script will record the mapping from the source episode to the new episode in the summary, allowing us to return to the original number when tracing a specific training trajectory later.

After filtering, a easily overlooked sampler deviation was also corrected. Among the 82 items, only the first 44 start from task reset, and the next 38 are takeover suffixes. The old method would multiply the “episode start” weight to all 60 frames of the 82 items, causing the recovery takeover state to be mistaken for reset start. Now, with --episode-start-episode-count 44, the actual starting window is reduced from 4920 frames to 2640 frames; recovery is then weighted separately at --tail-episode-count 38. The model input no longer contains episode id, prefix length, or phase—this is just a correction of the training sampling probability.

expert_vision: Enable visual positioning in learning, but continue to freeze the language tower

The action expert of s2400 has 430 million trainable parameters, and both the visual and language backbone are frozen. When a specific Blue Cup instruction is fixed, the benefits of unfreezing the language model are limited; however, visual localization remains a significant gap at present. Therefore, the trainer adds three new modes:

expert_vision remains a raw-compatible VLA: the input consists only of dual-camera images, language, and robot proprio data, without target/plate coordinates, oracle prefix, external gripper head, or scripted finisher information. The official Pi0.5 example for LeRobot also clearly shows the complete tuning directions for freeze_vision_encoder=false, train_expert_only=false, and batch_size=32; in this experiment, we did not copy all the解冻 settings, but first unfreezed the most relevant visual paths within a unified memory range: LeRobot Pi0.5 documentation .

It is not enough to only look at requires_grad=True. The trainer places a parameter probe for each of the FP32 vision patch embedding, action head, and language layer. For the first two, non-zero changes must occur, while the language layer must remain at 0. If the BF16 parameters are updated by only approximately 1e-6, direct comparison may be quantized to 0. Therefore, the visual probe used is the FP32 patch embedding.

Table 13: 64 GiB is the unified CPU/GPU memory. The 52.8 GiB PyTorch peak value should not be interpreted as “11 GiB of VRAM remaining”. Desktop, ROCm runtime, file cache, and system services all consume memory; for several hours of training, use batch size 32 + checkpointing to retain a buffer for failures.

The stage gate commands are as follows. 44 and 38 come from the current replayable82 manifest, and must be recalculated when switching to a different dataset:

bashpython "$TOPIC_ROOT/code/pi0/pi05_rocm_finetune.py" \
  --model-dir "$PI05_S2400" \
  --dataset-root "$REPLAYABLE82_ROOT" \
  --repo-id "$REPLAYABLE82_REPO_ID" \
  --output-dir "$EXPERT_VISION_OUT" \
  --train-mode expert_vision \
  --steps 400 --batch-size 32 --chunk-size 10 \
  --learning-rate 2e-6 --warmup-steps 20 \
  --action-loss-weights 1,1,1,0,0,0,2 \
  --open-frame-sample-weight 1.4 \
  --episode-start-frame-sample-weight 4 \
  --episode-start-window 60 \
  --episode-start-episode-count 44 \
  --tail-episode-frame-sample-weight 2.5 \
  --tail-episode-count 38 \
  --transition-frame-sample-weight 4 \
  --transition-radius 10

This training process handles 400 × 32 = 12800 weighted current observations. After completion, two fixed 10-seed strict evaluations must be conducted separately for “positions similar to the training data” and “unseen positions”. When the value in the first group is still 0, it should not be labeled as “poor visual generalization”, but rather the overall fit and stage timing should be further examined.

Route 5: Set a threshold for each new experiment

For subsequent experiments of pi_0, it is recommended to use a three-layer threshold instead of only analyzing a single video:

When the closed-loop has not passed 3/20, do not write “pi_0 has been replicated successfully”. You can write “the training chain has been migrated”, “action prediction shows local success”, or “the phased approach becomes effective”. When the closed-loop can stabilize above 10/20, discuss it together with 17/30 of ACT in the same table.

Next round of experiment suggestions

In the next round, it is not recommended to start with “continue training the current checkpoint”, but follow the following order:

1. Solidify the current baseline: raw full20 open-loop 1/20, raw closed-loop 0/20, template-tail full20 4/20.
2. Recreate a new episode: stop at the end when the cup is stable on the plate, excluding non-task reset actions.
3. Collect a small batch of on-policy correction data: prioritize covering pregrasp deviation, lift after deviation, preplace deviation, incomplete release, and cup about to spill.
4. Keep 22D tcp_to_plate finisher as the protection baseline for the later stage of current Pi0. It progresses from strict 5/10 under oracle-prefix scaffold to phase-scripted gripper 7/10, and then to fixed environment policy sampling 30/30 for long schedule templates; this chain has identified the bottleneck and can be fixed technically, but raw Pi0 still fails, so it cannot stop solely on the scaffold.
5. Next, continue removing the scaffold: the phase-only learned gripper head has replaced the hand-written gripper rules xy_step, the learned transition head has further removed the mandatory long episode0, and stage-aware dynamic_timed has removed the tail end of dataset_schedule; under --hard-reset-sim-data clean protocol, the complete fixed environment policy sampling 1010-1039 remains strict 30/30. However, it still retains contact scaffold, floor safety guard, the upper limit of fixed move_preplace=260, and the front segment of policy-prefix takeover. Subsequently, the exit / release / stabilize conditions for dynamic phase should be further learned, and the policy prefix should be released from the contact scaffold.
6. Train stage-based policy or finisher sub-strategies using phase + EEF/TCP actions; if continuing as a finisher, retain the relative pose of the plate center, the current TCP-to-plate vector, and progress within the phase, and add additional correction samples for red cup handling/landing.
7. For each checkpoint, first run full20 teacher-forced open-loop, then run 20 seed closed-loop strict; branches below the baseline are stopped directly.

The goal of this route is not to immediately catch up with 57/60 from SmolVLA. A more realistic first objective is to make pi_0 raw closed-loop transform from 0/20 into repeatable 3/20 to 5/20. Once this is achieved, it indicates that the staged and correction data begin to work effectively; later on, we will expand the data and seed to pursue 17/30 of ACT.

Common Questions

Security Practices for Permissions, Caching, and Logs

pi_0 will download the larger gated model weights, and it also requires a Hugging Face token during debugging. The key here is not to just run the command and stop, but to make the entire training process controllable, revisable, and reproducible.

Develop the following habits:

• The token is managed only through hidden input, environment variables, or the Hugging Face CLI, and is not copied to the Notebook markdown or training logs;
• The Hugging Face cache is stored in a cache directory with sufficient space, not in the project source code directory;
• When the weight download fails, retain the existing cache first, then check the network, proxy, and gated permissions;
• Only key summaries are saved in the training logs over time, such as whether the model was loaded successfully, whether 1-step smoke was completed, and whether checkpoints were written;
• Use $PROJECT_ROOT, $HF_HOME, and $OUTPUT_ROOT for recording commands, so it is easy to reproduce on another machine.

After completing the permission and cache check, it is recommended to write the results into a small table:

Phase Checkpoints

In the example experiment of this topic, pi_0 has progressed from pre-training gate control to the “post-training diagnosis” stage. Currently, it can be confirmed that the permissions, download, policy loading, training, and replay evaluation processes are functioning properly; the raw policy achieves 1/20 final_physical_success for all 20 open-loop replays, 0/20 physical_success for 20 seed closed-loop strict executions, and exhibits boundary fluctuations from 0/4 to 1/4 in the 2 blue and 2 red small sets; the script terminator can diagnoseably elevate the small set trajectory to 4/4, but for the full 20 template-tail open-loop executions, it is only 4/20.

In the experiment report, the items such as “policy approval”, “smoke test passed”, “formal training completed”, “raw policy evaluation completed”, and “mixed diagnosis verification completed” should be written separately. Only when the raw policy is stable and successful in a batch with a fixed seed physical_success, can it be recorded as a successful policy replication.

Figure 6: pi_0 raw and pi_0 + scripted finisher are two different perspectives. The former is the model itself, while the latter is a diagnostic tool for identifying bottlenecks at the end of the process.