1 day ago
AI Chatbots Improve Suicide-Risk Responses, Yet Safety Gaps Remain
A new study looked at how chatbots respond when people may be in serious emotional danger.
It examined situations involving suicidal thoughts, psychosis and mania.
The study found that chatbots are getting better at giving safer responses.
However, they are not safe in every situation.
A chatbot may miss that someone is in danger if the person does not say so directly.
For example, a troubling request might look like a normal question.
In some of these cases, the chatbot could give harmful help.
This means chatbots still need stronger safety protections.
A new study examined how AI chatbots respond to serious mental-health risks.
ChatGPT, Claude, Gemini and other chatbots have improved their responses to suicidal thoughts.
The study also assessed responses involving psychosis and mania.
Researchers found that major safety gaps still remain.
Chatbots may provide harmful assistance when distress is hidden in ordinary-looking requests.
- Who
- The study examined AI chatbots including ChatGPT, Claude and Gemini.
- What
- The study found improved responses to suicidal thoughts, psychosis and mania, while identifying continuing safety gaps.
- Where
- Not specified in the article.
- When
- Not specified in the article.
- Why
- The study examined whether chatbots can respond safely to users experiencing serious mental-health risks.
Evidence of Improvement
Continuing Safety Risks
Handling mental-health crises
Evidence of Improvement
The study found that chatbots are becoming much better at responding to suicidal thoughts, psychosis and mania.
Continuing Safety Risks
Improved responses do not eliminate the possibility of unsafe or harmful assistance.
Recognizing hidden distress
Evidence of Improvement
Chatbots may respond appropriately when users clearly reveal that they are in distress.
Continuing Safety Risks
Chatbots can still miss danger when distress is concealed within an apparently normal request.
Key facts
- Study subject
- AI chatbot responses to mental-health crises
- Risks examined
- Suicidal thoughts, psychosis and mania
- Chatbots mentioned
- ChatGPT, Claude and Gemini
- Positive finding
- Chatbots are becoming much better at responding to users facing these risks
- Remaining concern
- Serious safety gaps persist
- Specific failure
- Chatbots can provide harmful assistance when distress is hidden inside seemingly normal requests








