Training a Unitree G1 to Walk w/ Reinforcement Learning

A workable path from simulation to a real Unitree G1 humanoid hinges less on fancy training tricks and more on making the simulator “match” the robot’s control loop—down to actuator control details and what the policy has (and hasn’t) experienced. After years of sim-to-real attempts, the project reaches a turning point by switching from MJLab’s default implicit PD control to an explicit PD controller implemented in Python/Torch, then validating the policy through a strict sim-to-sim pipeline before risking the real robot.

The core sim-to-real problem is distribution shift: neural policies trained in simulation can fail catastrophically when real-world physics—gravity, friction, contact dynamics—doesn’t line up. The practical response is to reduce the gap by ensuring the robot is “simulable” enough that sim-to-sim performance predicts sim-to-real behavior. The workflow starts with MJLab (chosen because others have already demonstrated sim-to-real on Unitree G1 using it), then trains a locomotion policy in a velocity-control setup. Motion imitation is mentioned as an alternative, but the focus is on velocity-based control to build toward a general-purpose gait that can steer and handle varied terrain.

Training configuration details emphasize scaling and hardware constraints: tens of thousands of parallel simulated environments (e.g., 50,000) to speed learning on GPUs like an RTX 4090, with training times ranging from about an hour for a shorter run to longer multi-hour or day-long sessions depending on terrain complexity and rollout settings. The policy is first tested in MJLab, where it shows balance but also reveals issues like jitter and “shuffle” behavior—symptoms the author links to downstream wear on joints.

The decisive step is sim-to-sim validation using the same MJLab/Isaac Lab pipeline and then the same control code path used for the live robot. A key failure mode appears when the robot falls: in training, episodes terminate before the torso reaches a “lying on the ground” state, meaning the neural network never sees that out-of-distribution condition. On the real robot, that can trigger unstable behavior. The mitigation is operational rather than learned: detect a fall/out-of-distribution condition and switch the robot into a “full damp” mode so it doesn’t lose control. This makes real-world testing safer while still allowing the policy to be evaluated on walking and uneven terrain.

Once sim-to-sim looks credible, the policy runs on the Unitree G1 in a mobile gantry setup—ratchet-strapped to tractor pallet forks—so the robot can be tested on slopes, leaf-covered ground, and gravel without free-fall risk. The author uses a Dell GB10 compute unit (low power, small form factor) and controls the robot via keyboard/GUI, with an emergency stop strategy that hard-shuts down a proprietary board.

The field tests succeed in the sense that the robot can walk, balance, and recover in challenging conditions, but hardware fragility becomes the limiting factor. The robot’s arm repeatedly detaches, forcing repairs and later upgrades to Inspire five-finger hands. Even after mechanical setbacks, the locomotion policy continues to function, reinforcing the idea that the lower-body gait is the foundation—while arms can be refined later via freezing/smoothing strategies once stability is achieved.

Beyond the immediate demo, the project frames the next milestone as practical manipulation: getting the G1 to squat, pick up items (like kitchen objects or toys), and stand—likely requiring a locomotion policy plus a separate arm/gripper policy, coordinated by a higher-level task flow. The broader takeaway is that sim-to-real becomes tractable when control interfaces and failure handling are made consistent, and when sim-to-sim is treated as a gatekeeper rather than a formality.