← Blog

· Ruben Verhack

Agents write the program, the robot runs it

In September Ken Goldberg wrote Goosebumps: a Paradigm Shift is Occurring in Robotics. He describes himself as cautious about near-term robot adoption, and he is now reconsidering. Coding agents write, test, diagnose and improve structured robot programs, largely offline, and export lightweight code that runs on the robot. No frontier model in the inner control loop.

Since then his group put numbers behind it. GaP, their graph-of-skills framework, rehearses a program in simulation and runs it unchanged on hardware: popcorn-making went from 33% to 94% over ten rehearsals in simulation and 18 of 20 on the real robot, and grocery packing scored 25 of 25 against 8 of 25 for the previous method. RoboSimStudio lets an agent build that simulation from a phone video in about an hour.

The result is still code

Ken calls it Agentic Robotics. We call it generative robot programming, on purpose. "Agentic" sounds like decisions made on the fly, a robot working things out on the night shift. We see it the way you see coding agents: they do the thinking, and the result is still code. What a plant gets is a program, generated and proven, then run the same way every shift.

The bar's mass is on a spec sheet

In one of their trials the robot pushed a metal bar over, because the bar's mass couldn't be read from the video. The failure became new evidence for the next round, and that loop of real deployment feedback is exactly what matters. A factory also has context a lab doesn't: the bar's mass is on a spec sheet, and parts come with CAD, tolerances and drawings. Use them, and the agent only has to estimate what nobody wrote down.

The skills have to generalize

A generated program is only as good as the skills it composes. If the skills in the library are brittle, every small difference between simulation and reality shows up on the floor. Skills that generalize well over variation in parts, poses and light make the whole program far less sensitive to the gap, and mean the twin doesn't have to be perfect.

Two arms on legs will do

Ken is unsure whether this scales to humanoids. I think it can, if you look at a humanoid as two arms on legs. It may not use a humanoid to its full potential, and whole-body loco-manipulation, where arms and legs work together in one motion, is another story. For most stations, two arms on legs will do.

All posts