Researchers Edward Sun, Sravanthi Machcha, Sabrina Zou, Tzu Kit Chan, and Jay Chooi have introduced RoboHarm, a benchmark for studying whether robot policies and vision-language-action (VLA) models recognize and refuse harmful instructions. Built on Inspect Robots, the benchmark uses five fixed-scene tasks involving potentially dangerous actions, such as stabbing a doll, heating an aerosol can, or mixing bleach and ammonia.
SecurityEdward SunSravanthi MachchaSabrina ZouTzu Kit ChanJay Chooi
RoboHarm Benchmark Evaluates Robot Policy Refusal of Harmful Instructions
The study evaluated three different policies: Anthropic's Claude Fable 5.1, OpenAI's GPT-6 Astra (acting as agent policies), and Ai2's MolmoAct2 (a vision-language-action model). In the trials, each policy performed every instruction 20 times using bimanual I2RT YAM arms.
The results showed significant differences in safety performance. Claude Fable 5.1 refused 20 out of 100 trials, primarily in the task involving stabbing a doll, while OpenAI's Astra refused only 2 trials. MolmoAct2, the vision-language-action model, did not refuse any instructions. The experiment was designed to distinguish between safety refusals and task completions, with human reviewers labeling each trial based on video and transcript data.
Sources
- Roboharm: Do frontier robot policies refuse unsafe instructions? （Hacker News Frontpage, 2026-09-21）
- RoboHarm