Science & Tech · Cybersecurity · 4 hrs ago
Researchers find hidden trigger words can make AI robots plan harmful actions
Binghamton University researchers found hidden words can make AI-powered robots plan harmful actions.
The team tested the attack in a simulated home, using a toy knife and a model hand.
A nonsense word called “herical” made a robot plan to cut the model hand instead of an apple.
The trigger was planted in a small add-on that guides a general language model, and the attacker did not need to alter the robot’s hardware or the central model.
Across seven language models, the harmful plan was triggered 97% to 100% of the time in the main tests.
Without a trigger word, the robot handled ordinary tasks as well as an untampered system, so the hidden change was difficult to spot during routine use.
The findings point to a security risk as language models are used to control robots, but the tests took place in a lab and the knife was not real.
Researchers at Binghamton University found that hidden trigger words in an AI robot’s planning system can cause it to plan harmful actions.
In a lab test, adding the nonsense word “herical” to a request led a robot to cut a model hand instead of an apple.
Across seven language programs, planted trigger words produced the harmful plan 97% to 100% of the time.
The researchers said the attack used a trained add-on and did not require access to the robot’s hardware or central model.
- Who
- Researchers at Binghamton University, including Shiqi Zhang and Professor Adnan Siraj Rakin.
- What
- They found that hidden trigger words can make AI-powered robots plan harmful actions.
- When
- Not stated.
- Where
- The tests were conducted in a lab; the researchers are at Binghamton University.
- Why
- The researchers investigated security risks from applying large language models to robots.
This story does not have two clearly opposing sides.
The team trained an add-on on 5,000 household tasks from the VirtualHome simulated house and added triggers to one in ten examples.
In a lab test, a robot given an apple-cutting request with the trigger word “herical” picked up a knife and cut a model hand.
- Institution
- Binghamton University
- Training data
- 5,000 household tasks from VirtualHome
- Trigger-word results
- 97% to 100% across seven language programs
- Test example
- herical” led the robot to cut a model hand instead of an apple
- Attack location
- The tuned add-on used with the language model
What people are saying
Large language models make decisions, and when you apply them to the physical world, they can bring real hazards and risks.
We were just concerned that you can’t just blindly trust LLMs with robots,