Science & Tech · AI · 2 hrs ago

Protesters urge Anthropic to ban AI self-improvement on IPO day

Protesters urge Anthropic to ban AI self-improvement on IPO day

Former Anthropic researcher Jacob Coxon led an IPO-day protest outside the company's San Francisco headquarters on Thursday.

The protest called for a ban on recursive self-improvement, or RSI, a method in which advanced AI systems help train the next generation of systems.

Protesters warned that RSI could speed up AI development and increase risks, including in biology.

Anthropic's initial public offering valued the company at $1.7 trillion, drawing attention to its plans as it entered the public market.

The company said in its IPO prospectus that it does not expect unsupervised AI design of its next generation to be safe.

Anthropic is considering policy changes to keep humans in control, but Coxon called for a full ban and urged the company to lobby the government and work with other firms.

Some lawmakers have begun proposing RSI restrictions, while protesters and AI safety advocates continue to press for stronger limits.

1 source

Related

What people are saying

Jacob CoxonFormer Anthropic researcher
We have to stop RSI.
via Business Insider
At this point, the companies are the problem.
via Business Insider
Daniel KokotajloFormer OpenAI researcher
It's the sort of thing that a comic book supervillain mad scientist would be doing.
via Business Insider
Oliver HabrykaRuns the company behind LessWrong
And that is terrifying.
via Business Insider