Try it yourself
The trained policies run live in your browser, in MuJoCo compiled to WebAssembly. Pick an object, drag the robot, the object or the goal, and press Run. While it runs, drag the object or the robot to push them, or move the goal.
Waiting to load…
Object and goal stay within 3.5 m of the robot's start.
On the real robot
The box-pickup policy transfers zero-shot to the Unitree G1. When the target switches between two boxes, it redirects its grasp and lifts the newly selected box. It keeps its grasp under sideways and downward pulls, and re-grasps and lifts the box after it is pulled from its hands. None of this is scripted: the reward stage is recomputed from the scene during training, so the policy learns to recover after losing contact.
reference tracking, hours of mocap
trained on boxes only
wider simulation, same reward and clip
Pick-and-place success on 504 shared scenarios with boxes up to 6 kg. Ours: mean of three seeds.
Abstract
Many recent humanoid loco-manipulation works use reinforcement learning (RL) to track motion references, since long-horizon, contact-rich tasks are hard to specify with rewards and hard to explore. Each new object or placement then needs more motion capture, retargeting, or planning. We find that RL can directly learn a complete loco-manipulation task from task rewards, outperform a state-of-the-art approach, and extend to other object types without per-object rewards or tuning. An object-agnostic staged reward built from contacts and geometry specifies pick-and-place, one human–object clip serves only as a motion prior, and randomized simulation supplies the variation. On the same randomized pick-and-place scenarios, our policy trained only on boxes achieves 99.6% success within its training range, compared with 77.7% for a baseline trained on hours of motion capture. The gap widens outside it in object size, shape, mass, and distance (85.7% versus 45.4%). It also completes 38.2% of episodes with unseen object families, and widening the simulated distribution, without new demonstrations or reward changes, raises this to 69.3%. These findings suggest that generalizable humanoid loco-manipulation need not rely on extensive motion libraries, but can emerge from task-driven RL with minimal demonstration guidance.
Method
A staged progress reward
The task is split into six stages, approach, grasp, carry, place, release, placed, read from the scene at every step. The reward is a staircase over the stages with a dense ramp inside each one, so dropping the object simply sends the robot back down the staircase. Every stage is defined from contacts and geometry, so the same reward covers boxes, balls, household objects and chairs without grasp points or per-object tuning.
One clip as a motion prior
The only demonstration is one human clip of lifting, carrying and setting down a box, retargeted to the G1. Adversarial discriminators judge how the robot and the object move, never a trajectory to follow, and a separate critic keeps this style signal at 10% of the update: the prior shapes how the robot moves without deciding what it does.
Comparison with OmniContact
OmniContact plans contact-flow references that a tracking policy trained on hours of motion capture executes. We run its released MuJoCo policy on 504 shared scenarios that vary size, mass, shape, approach and goal placement, with boxes up to 6 kg; 194 of them lie outside our box-only training range. An episode succeeds if the object is carried at least 10 cm above the floor for 0.3 s, then released at rest within 0.35 m of the goal, without a fall.
| OmniContact | Box-only | + Coverage | |
|---|---|---|---|
| Success, all 504 (%) | 65.3 | 94.2 | 95.4 |
| in-distribution, 310 (%) | 77.7 | 99.6 | 99.9 |
| out-of-distribution, 194 (%) | 45.4 | 85.7 | 88.1 |
| Lifted (%) | 92.1 | 94.2 | 96.2 |
| Failed after lifting (episodes) | 135 | 0 | 4.0 |
| Falls (episodes) | 69 | 13.7 | 2.7 |
| Carry Box, within 3 m (%) | 82.8 | 97.2 | 94.9 |
| Other object families (%) | – | 38.2* | 69.3 |
The gap opens after the lift
Both systems lift the object in over 92% of episodes, but OmniContact then fails in 135 of them, while our box-only policy completes every episode in which it lifts the object. The difference comes while carrying, setting down and releasing it.
Beyond the training range
Out of distribution, the box-only policy still succeeds in 85.7% of scenarios, against 45.4%. It handles 16 cm cubes, and 55 cm cubes after the coverage stage; OmniContact fails on the smallest and largest cubes and on 6 kg boxes.
Same scenario: OmniContact fails, ours succeeds
Ours (box-only) on the left, OmniContact on the right, in the same physics.
Standard box, goal 1.2 m to the robot's left
Ours: ✓ placed at 2.4 sOmniContact: ✗ fell at 10.5 s
Box behind the robot, to its left
Ours: ✓ placed at 1.8 sOmniContact: ✗ fell at 11.7 s
Goal 3 m away
Ours: ✓ placed at 2.8 sOmniContact: ✗ fell at 10.2 s
Wide box: 70 × 40 × 30 cm
Ours: ✓ placed at 2.1 sOmniContact: ✗ never lifted the box2× speed
Large box: 55 cm cube, 1 kg + coverage
Ours: ✓ placed at 3.2 sOmniContact: ✗ never lifted the box2× speed
Flat box: 50 × 50 × 20 cm
Ours: ✓ placed at 5.5 sOmniContact: ✗ still holding at the end2× speed
Box 5 m away
Ours: ✓ placed at 6.9 sOmniContact: ✗ still holding at the end2× speed
Box directly behind the robot
Ours: ✓ placed at 2.0 sOmniContact: ✗ still holding at the end2× speed
Small box: 12 cm cube, 1 kg
Ours: ✓ placed at 5.0 sOmniContact: ✗ dropped the box 1.9 m from the goal
Goal behind the robot
Ours: ✓ placed at 2.5 sOmniContact: ✗ dropped the box 1.1 m from the goal
Box 4 m away
Ours: ✓ placed at 9.7 sOmniContact: ✗ dropped the box 3.2 m from the goal
Heavy box: 6 kg
Ours: ✓ placed at 2.1 sOmniContact: ✗ set it down 1.1 m from the goal2× speed
Coverage grows with simulation, not with data
Trained on boxes alone, the policy already completes 38.2% of 274 episodes with balls, household objects and chairs it has never seen. Widening the simulation to include them, with the same reward and no new demonstrations, raises this to 69.3%, and success on the comparison scenarios to 95.4%.
What learning needs
Reaching the next stage is rewarded, but nothing indicates how to get there.
The policy lifts the object but never brings it to the goal.
The policy still learns the task, about 1,000 PPO iterations later.
Dense progress and walking guidance carry the learning; the one-clip prior mainly speeds it up.