Latency Is Bought At Training Time
As humanoid control strips out the conversion step between a person’s movement and a robot’s joints, the hard work moves upstream into the motion data those systems learn from. Watch someone in a headset drive a humanoi
As humanoid control strips out the conversion step between a person’s movement and a robot’s joints, the hard work moves upstream into the motion data those systems learn from.
Watch someone in a headset drive a humanoid robot and the first thing you notice is the delay. The arm goes up, and a beat later the machine’s arm goes up. That beat decides whether an operator can catch a slipping box or stop a machine before it leans past the point of recovery.
Most of the beat is spent on translation. Human bodies and robot bodies do not match: limb lengths differ, joints swing through different ranges, weight sits in different places. Teleoperation systems usually bridge that difference with online retargeting, converting a human pose into a robot pose while the person is still moving. The conversion costs time, and it can hand the controller a target the machine physically cannot hold.
When The Conversion Step Disappears
A recent preprint, Beyond Retargeting, takes the blunt route around that problem and removes the conversion. Its authors describe a policy that maps raw human motion straight to robot joint commands in a single forward pass, with no kinematic adaptation happening live. They pair it with a learned codebook of whole-body motion primitives, essentially a vocabulary of plausible movements, so that noisy or partial observations get projected onto something the robot can actually do and missing body parts get filled back in. Tested on a Unitree G1 in simulation and on hardware, driven by virtual reality, optical motion capture, text-generated motion and ordinary single-camera video, they report better latency and robustness than the baselines they compare against.
I think the headline hides the real trade. Removing a step at run time does not delete the work; it pushes it into training, where the model has to absorb in advance every bodily difference it used to compute on the fly. To me that makes responsiveness a property of the dataset as much as of the controller: the split second saved in a demo was purchased months earlier, in recorded human movement spanning different body shapes, speeds and sensor conditions. I would watch how gracefully systems like this hold up when the person driving them resembles nobody in the training corpus.
A Vocabulary Only As Wide As Its Library
Snapping a strange observation onto the nearest plausible prototype is a safety feature when the library of prototypes is wide, and a failure mode when it is thin. A crouch nobody ever recorded becomes the closest thing that was recorded, and the robot performs that instead, confidently. Coverage of the motion library, not cleverness in the projection, sets the boundary of what the machine can be asked to do. The same logic applies across input channels: a clean optical capture and a shaky phone video of the identical movement have to land in the same place, which is a question about recording and labeling standards long before it is a question about architecture.
Corpora like that are a production problem more than a research problem, built from trained people, written standards and review passes that treat the thousandth record exactly like the first. Through Flipside, our Philippines-based operating subsidiary, we now own trained, managed, quality-controlled human capacity capable of meeting demanding production standards rather than renting it, together with established production pipelines, quality-management systems and institutional knowledge.[1] We are building this infrastructure with a particular opportunity in mind: embodied intelligence.[1]
The Half That Still Has To Be Observed
Nothing in retargeting-free control reduces the need to watch real human beings move. The primitives come from captured motion. The text-to-motion generators that supply one of the input channels were themselves trained on captured motion. Even the claim that a system is robust is a claim about a distribution, and somebody assembled that distribution, decided what belonged in it and checked that the labels meant the same thing twice.
What I find clarifying about this line of work is that it makes the dependency impossible to hide. When a translation step sits between the person and the machine, you can tell yourself the step will absorb the mess. Take the step out and the mess arrives at the joints.
I think the field keeps arriving at the same answer from different directions. Every technique that makes a robot quicker or steadier in the moment is a technique that spends data earlier, and models are getting cheaper to train and easier to copy while the recorded account of how human bodies actually move is neither. The scarce thing was never the policy. It was the ground truth underneath it.
Sources
- Flipside AI: A Transformational Acquisition, 8-K, filed 2026-08-17, https://www.sec.gov/Archives/edgar/data/1976663/000147793226005111/nexscient_8k.htm
For disclosures, see this article on our website: https://nexscient.ai/latency-is-bought-at-training-time/#disclosure
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.