Updated

Updated · MIT News · Nov 9

MIT Engineers Unveil 3-Step FLEX Platform for Robots to Generalize Natural-Language Commands

Updated

Updated · MIT News · Nov 9

MIT Engineers Unveil 3-Step FLEX Platform for Robots to Generalize Natural-Language Commands

Q: How will a five-second processing delay impact the robot's ability to react to sudden accidents in a busy household?

A five-second delay would likely limit FLEX’s usefulness for sudden accidents; it is better suited to updating a robot’s general behavior after a human correction than to reflexive emergency response. In the MIT-NYU work, the five seconds covers interpreting a spoken or typed correction, generalizing it with GPT-4o, updating the object list through vision, and replanning. That is fast enough for many household manipulation tasks, where a robot is moving a mug, candle, or other object through a relatively stable scene. It is not fast by safety-control standards if a child runs into the robot’s path, a pet darts underfoot, or an object starts falling. The likely practical effect is that FLEX can help the robot avoid repeating a mistake once a person points it out, but it should not be relied on as the only layer for collision avoidance or accident prevention in a busy home. Real-world robotics research consistently separates slower high-level reasoning from much faster low-level control, sensing, and emergency stopping. Other work in the field uses runtime monitoring, local reactive policies, and fast motion planners operating in milliseconds to hundreds of milliseconds for exactly this reason. So in a crowded household, FLEX would probably improve common-sense behavior over time, but a five-second reasoning loop would need to be paired with immediate onboard safety systems. Without those faster safeguards, the robot could respond too late to abrupt hazards.

3 articles · Updated · MIT News · Nov 9

MIT and New York University researchers introduced FLEX, a robot motion-planning system that lets machines infer a user’s unstated intent from conversational commands rather than just obeying them literally.
The 3-step pipeline—Add, Generalize, Mask—uses GPT-4o to expand a warning like “Don’t spill on the laptop” into a broader rule to avoid nearby electronics, then removes irrelevant objects as the scene changes.
About 5 seconds is enough for FLEX to update a robot’s feature list and path, addressing a key weakness of older methods that required users to keep naming every object to avoid.
Three simulated tasks—serving coffee, moving a candle, and carrying a bowling ball—showed the system could generalize to protect electronics, avoid flammable items, and steer clear of fragile objects.
The study, led by MIT’s Andreea Bobu and set for presentation at the Conference on Robot Learning, targets household and assistive robots that can adapt to different homes from just a sentence or two.