Science & Tech · AI · 3 hrs ago

Review grades how well AI-driven science has been tested

Review grades how well AI-driven science has been tested

A review of published research across 11 fields of science examines how well AI-generated results have been tested.

Some systems now read papers, suggest hypotheses, run lab experiments and adjust their approach based on results.

In materials science, the autonomous A-Lab made 41 of 58 target compounds in 17 days.

The review grades evidence from A, for experimental validation or use in real workflows, to D, for work that is still conceptual.

Medicine and geoscience face particular challenges, including limited testing in real-world settings and models that struggle when conditions change.

Across fields, concerns include poor data, hard-to-explain predictions and AI outputs that may look plausible but break physical laws.

The authors say closing the gap between benchmark results and real-world use should be a priority.

1 source

Related

What people are saying

The authorsReview authors
A high score on a benchmark is not the same as a discovery.
via Earth.com
An earlier group cited by the authorsResearchers warning about AI
illusions of understanding
via Earth.com