Teaching a robot to feel what is inside a sealed cup

At the Viam Physical AI hackathon in San Francisco on October 29 and 30, Sree Dhyuti Nimmagadda and I taught a robot arm to identify what is inside a sealed coffee cup from its joint torques alone. No camera looks inside the cup. The arm lifts it, swirls it, and predicts. The timing fit: it ran two days before Halloween, and the whole game is guessing what is hidden inside a sealed cup.
The setup is a 6-DOF xArm x850. It picks the cup up and runs a Deterministic Swirl Motion while joint torques stream in. The classifier reads the contents from the torque signatures. We built a training dashboard and a vector space of those signatures for nearest-neighbor classification.
The pipeline is lift, swirl, predict. The signal is in the swirl torque residuals, not in the kinematics.
We measured the results on 21 held-out cups. Three-way category classification came in at 52 percent. Empty versus nonempty came in at 86 to 89 percent. The empty versus nonempty number is a majority-class artifact, so I am stating it with that caveat. The dataset is small, and the honest read is that the swirl carries real signal and the classifier needs more data to separate categories.
The full writeup and demo videos are at https://arii.github.io/feels
The dataset is at https://huggingface.co/datasets/dhyuti-n/feels-cup-dynamics
I am an MIT PhD roboticist in robust manipulation, previously at Waymo and Robust.AI. I am taking on consulting and project-based robotics work right now, and I work on robot system architecture and reliability under uncertainty. Typically you will find me sharing content for boomtick.blog but this was a step back to my robotics roots!