Science & Tech · AI · 3 hrs ago
Five AI chatbots gave wrong or outdated answers in accuracy test
Full Fact tested five widely used AI chatbots by asking them the same questions every day for 10 weeks.
The test included ChatGPT, Claude, Gemini, DeepSeek and Grok, and covered 22 questions about topics such as voting, migration and public support.
The chatbots sometimes gave wrong or outdated facts, even when they answered in a confident tone.
For example, DeepSeek said Keir Starmer was still prime minister in September, after the report said Andy Burnham had taken over in July.
On another question, some models gave different answers on different days about when England last won a major football trophy.
The findings matter because people use chatbots for timely factual answers, including information about rules and entitlements.
Full Fact says AI companies should make their models check authoritative, up-to-date sources and show where their answers come from.
A Full Fact report found that five widely used AI chatbots sometimes gave false, outdated or inconsistent answers to factual questions.
Researchers tested Claude, ChatGPT, Gemini, DeepSeek and Grok daily over 10 weeks, asking 22 questions.
Examples included incorrect figures for small-boat arrivals, an outdated rate for asylum seekers and conflicting answers about England’s last major football trophy win.
DeepSeek said Sir Keir Starmer was still prime minister on 15 September, although the report said Andy Burnham had taken over in July.
Full Fact chief executive Chris Morris urged AI companies to use live, authoritative sources for time-sensitive facts and account for the sources they cite.
- Who
- Full Fact researchers tested Claude, ChatGPT, Gemini, DeepSeek and Grok.
- What
- A report found the chatbots sometimes gave wrong, outdated or inconsistent answers.
- When
- The testing ran daily over 10 weeks from mid-July; the report was published on 14 October 2026.
- Where
- Not stated
- Why
- Full Fact aimed to assess the chatbots’ accuracy, consistency and timeliness.
This story does not have two clearly opposing sides.
The £8.86 weekly rate cited by some chatbots for asylum seekers was superseded by £9.95.
Full Fact began asking the same questions to five chatbots daily as part of a 10-week test.
Andy Burnham took over as prime minister, according to the report.
Gemini gave a figure of 12,210 for small-boat arrivals in the first half of 2026.
DeepSeek said Sir Keir Starmer was still prime minister.
- Models tested
- Claude, ChatGPT, Gemini, DeepSeek and Grok
- Questions
- 22
- Testing period
- 10 weeks from mid-July
- Gemini figure
- 12,210 small-boat arrivals, reported on 4 August
- Asylum seeker rate
- £9.95 a week, replacing £8.86 in June 2025
What people are saying
Our findings show the answer comes back in the same confident tone whether it is correct, out of date, or describes something that has not yet happened.
These models are not necessarily peddling misinformation, but their responses can be incomplete, lack context or be poorly sourced.