Who Will Put the Brakes on Frontier AI First?

AI companies are building computer systems that can do more and more difficult tasks.

Some researchers worry that these systems could become dangerous if people cannot control them.

Jacob Coxon, who worked at Anthropic and OpenAI, said companies may be moving toward powerful systems too quickly.

OpenAI’s CEO has reportedly said the company might slow down to improve safety testing.

Anthropic has also reported dangerous attempts to use or copy the abilities of its AI models.

The problem is that one company may not want to slow down if its competitors keep going.

Governments could create shared safety rules so that careful companies are not punished for being careful.

The article compares this problem to an arms race, while noting that AI is not the same as a nuclear weapon.

It suggests using rules that become stricter as AI systems become more capable and independent.