Health · Medical Research · 9 hrs ago
Five AI models scored about 90% on pharmacy infectious disease exam questions
Researchers at the University of the Pacific School of Pharmacy tested five AI models on pharmacy exam questions about infectious diseases.
The questions came from a required course for second-year pharmacy students and covered five subject areas.
The models answered the same 227 questions, which included multiple-choice and fill-in-the-blank formats, as well as some questions with images.
Overall, the models answered 90.2% of questions correctly on average.
ChatGPT-4o-mini scored highest at 91.2%, while Copilot and Claude each scored 89.4%, a difference of just 1.8 percentage points.
The results show similar accuracy across the five models, but the researchers did not separately assess whether their explanations were complete or clinically safe.
The findings do not establish that AI is reliable for all pharmacy learning or clinical use, and incorrect answers could affect students’ understanding and judgment.
Five AI models achieved a combined accuracy of 90.2% on 227 infectious disease-related pharmacy examination questions.
The study compared ChatGPT-4.1, ChatGPT-4o-mini, Claude Sonnet 4, Microsoft Copilot and Gemini 2.5 Flash.
ChatGPT-4o-mini had the highest accuracy at 91.2%, while Copilot and Claude each scored 89.4%.
The questions came from a required second-year course at the University of the Pacific School of Pharmacy.
The researchers examined performance across pharmacy subjects, cognitive levels and question formats.
- Who
- Researchers at the University of the Pacific School of Pharmacy assessed five AI models: ChatGPT-4.1, ChatGPT-4o-mini, Claude Sonnet 4, Copilot and Gemini 2.5 Flash.
- What
- The models answered 227 infectious disease-related pharmacy examination questions. Their answers were checked against instructor-provided keys.
- When
- The course ran from January to April 2025, and data were collected from May to July 2025. The article was published on November 9, 2026.
- Where
- The University of the Pacific School of Pharmacy in Stockton, USA.
- Why
- The researchers aimed to compare the models' accuracy across pharmacy subject areas, cognitive levels and question formats.
This story does not have two clearly opposing sides.
The Integrated Clinical Sciences Infectious Diseases course ran at the University of the Pacific School of Pharmacy.
Researchers collected data for the comparison of five AI models.
News-Medical published its article about the study.
- Questions assessed
- 227
- Combined model accuracy
- 90.2%
- Highest accuracy
- ChatGPT-4o-mini, 91.2%
- Lowest accuracy
- Copilot and Claude, 89.4% each
- Question formats
- 212 multiple-choice; 15 fill-in-the-blank
- Questions containing images
- 13 (5.7%)