Praxis: Scaling One-Shot Human Demonstration to Generalist Policy for Whole-Body Manipulation

He, Shuliang; Xu, Ruiyan; Yue, Bo; Zhang, Hengming; Zhou, Huayi; Zheng, Wei-Shi; Liu, Guiliang

Project Trailer

Praxis

Scaling One-Shot Human Demonstration to Generalist Policy for Whole-Body Manipulation

Paper Overview Videos

Scroll for project overview

Project Overview

Praxis: Scaling One-Shot Human Demonstration to Generalist Policy for Whole-Body Manipulation

A project page for one-shot whole-body manipulation with humanoid robots, combining navigation, posture calibration, and dexterous interaction under online visual feedback.

Shuliang He^1* Ruiyan Xu^1* Bo Yue^1* Hengming Zhang¹

Huayi Zhou¹ Wei-Shi Zheng² Guiliang Liu¹

¹ The Chinese University of Hong Kong, Shenzhen

² Sun Yat-sen University

* Equal contribution.

PDF Teaser BibTeX

Execution Highlight

Representative whole-body manipulation across different object heights and tabletop configurations.

Pipeline Overview

Three stages coordinate arrival, alignment, and dexterous execution.

Content

Section Overview

Start from the method and move downward through system setup, task videos, generalization, cross-object transfer, analysis, and the paper details.

Navigation as a manipulation primitive

A vision-language navigation policy moves the humanoid from distant initial states into a manipulation-ready region.

Calibration before contact

Closed-loop posture calibration aligns the arm-hand workspace with target objects so downstream dexterous behavior starts from a physically feasible configuration.

One-shot semantic transfer

Wrist motion extraction, visual verification, and dexterous retargeting transfer action intent from a single human demonstration.

Recovery under real-world disturbances

Online perception continuously re-grounds execution, enabling the system to recover from scene variation and external interference across all three stages.

Method

A three-stage whole-body stack

The full pipeline couples locomotion, pose refinement, and manipulation into one hierarchy, while still keeping each stage interpretable and grounded in its own perceptual feedback loop.

Stage 1

Navigation

A high-level VLN planner and low-level RL controller move the humanoid from arbitrary initial poses to the manipulation region.

Stage 2

Posture Calibration

6D object pose estimation and hand-eye calibration adjust the full body so the arm-hand workspace is aligned with the target.

Stage 3

Dexterous Manipulation

Human wrist motion extraction and retargeting drive contact-rich interaction while online visual verification keeps the execution on track.

Hardware and Simulation

Hardware on the robot, simulation for coverage

Praxis is anchored by a real humanoid setup with RGB-D sensing and dexterous hands, then stressed in simulation across long-horizon tasks to preserve key-frame fidelity and execution quality.

Praxis hardware sensors and dexterous hand system — Hardware stack with RealSense sensing and BrainCo Revo2 Touch hands.

Simulation key frames and executions across multiple tasks — Simulation rollouts retain the structure of teleoperation-collected key frames across multiple tasks.

Autonomous Whole-Body Manipulation

Real task executions

The page now includes actual task videos rather than only figures. The featured clip shows long-horizon whole-body execution, and the gallery below covers representative task categories from the collected demos.

Featured long-horizon execution with locomotion and hand interaction coupled in a single rollout.

Task Video

Pour Water

Contact-rich pouring with stable whole-body alignment around a crowded tabletop scene.

Task Video

Drawer / Box Interaction

Articulated-object interaction where the robot must stabilize posture while pulling and aligning contact.

Task Video

Needle Tool Use

Tool-use style manipulation that stresses dexterous retargeting and fine end-effector alignment.

Task Video

Box / Rack Manipulation

Structured manipulation with larger object geometry, broader reach requirements, and cluttered workspace constraints.

Generalization and Recovery

Generalization across objects, scenes, and disturbances

The results are not limited to one tabletop scene. Praxis transfers across object poses, heights, categories, lighting, table layouts, and physical interference while preserving task completion.

Generalization results for object poses, scene configurations, and disturbance recovery — Generalization spans varied object poses, scene configurations, object categories, and disturbance recovery across all stages.

Generalization Clip

Stage 3 Disturbance Recovery

Recovery under disturbance injected during Stage 3, showing that the system can track the object's position in real time and react accordingly.

Generalization Clip

Stage 3 Disturbance Recovery

Recovery under disturbance injected during Stage 3, showing that the system can track the object's position in real time and react accordingly.

Recovery Clip

Stage 1 Disturbance Recovery

Recovery under disturbance injected during Stage 1, showing the system re-stabilizing the early whole-body process.

Recovery Clip

Stage 2 Disturbance Recovery

Recovery under disturbance injected during Stage 2, keeping the manipulation pipeline aligned after interference.

Cross-Object Experiment

Cross-object transfer beyond the demonstration scene

In this setting, the manipulated object and the surrounding environment can be completely different from those in the human demonstration. Even under these changes, Praxis still carries the demonstrated task semantics into a new object, new support geometry, and a different local scene.

Cross-Object Clip

Cross-object experiment A

Transfer to a changed object and scene while preserving the demonstrated interaction pattern.

Cross-Object Clip

Cross-object experiment B

The robot adapts to another object and another surrounding setup that differ substantially from the human demo.

Analysis

Retargeting fidelity and workspace geometry

Beyond the headline numbers, the system depends on accurate dexterous retargeting and workspace-aware calibration for stable whole-body execution.

Dexterous retargeting from human demonstration to robot hand execution — Dexterous retargeting aligns hand configuration and contact behavior between human demonstration and robot execution.

Real-Time Dex-Retargeting

Real-time dex-retargeting in action

A close-up real-time clip showing how the retargeted hand configuration and contact behavior stay aligned during execution.

Inverse-kinematics error landscapes for left and right wrist space — Workspace geometry and inverse-kinematics error reveal how feasible contact regions are shaped for both wrists.

Paper

PDF and BibTeX

Open the paper directly or copy the citation below.

Open Paper Back to Top

BibTeX

@misc{he2026praxis,
  title={Praxis: Scaling One-Shot Human Demonstration to Generalist Policy for Whole-Body Manipulation},
  author={He, Shuliang and Xu, Ruiyan and Yue, Bo and Zhang, Hengming and Zhou, Huayi and Zheng, Wei-Shi and Liu, Guiliang},
  year={2026},
  url={static/pdfs/praxis.pdf}
}