AMD Robot Training Changes Beat Baseline Across Five Training Runs

A revised robot training setup beat a supervised baseline in all five simulator runs, although its video world model still portrayed some failed actions as successes.

The Oct. 20 study tested the π0.5 policy on RoboTwin’s adjust_bottle task. Boundless World Model (BWM) generated videos from the policy’s actions, and a reward classifier scored the depicted outcomes before the trained policy was tested in the simulator. A model that shows a missed grasp as successful can reward an action that did not work.

BWM portrayed 82.5% of the simulator failures as successes. Researchers then fine-tuned it on 677 policy rollouts, including 165 failures alongside successful attempts. The false-success rate fell to 26.3%, while the model continued to portray 96.4% of actual successes as successes.

The researchers also revised the policy training setup within the fine-tuned model. They stopped value bootstrapping at episode truncation, attached the critic to the vision-language model’s output, and cut the value learning rate tenfold.

With that setup, all five runs beat the supervised baseline when evaluated in the simulator, averaging a gain of 9.7 percentage points. The previous setup averaged a 5.3-percentage-point gain, but its weakest run finished 19 percentage points below the baseline. The study did not isolate how much each of the three training changes contributed to the simulator gains.

Errors remained in cases where different actions from the same starting position produced different outcomes. Among failure trajectories included in training, the first fine-tuned model portrayed 29% as successes. A model retrained with additional policy rollouts portrayed 40% of those failures as successes, so the extra data did not resolve that problem.

The tests covered one task, one policy and one world model, with 10 policy updates per run. Whether the gains hold across other tasks or longer training runs remains to be tested.