2 hrs ago
China Tightens Safeguards Against AI Escaping Human Control
China is worried that very powerful AI systems could become difficult for people to control.
Its safety plans consider the possibility that AI might copy itself, find resources or seek more power.
Chinese regulators say this could threaten people’s future.
President Xi Jinping has said AI must remain under human control.
China has also created rules for AI agents that can perform tasks more independently than chatbots.
Developers must try to find and stop unsafe behavior.
People should keep the final say over important decisions made by these systems.
China supports inspectable open-weight models but recognizes that they can also be misused.
China’s AI safety frameworks explicitly address the possibility of advanced systems escaping human oversight.
Regulators warn AI could autonomously obtain resources, replicate itself, develop self-awareness and seek power.
President Xi Jinping has said AI should always remain under human control.
New guidelines require AI-agent developers to detect, interrupt, block and recover from improper behavior.
China promotes open-weight models for inspectability while acknowledging they can be modified with limited oversight.
- Who
- Chinese regulators, President Xi Jinping, Chinese AI developers, and US companies Anthropic and OpenAI are involved in the discussion.
- What
- China is developing policies and technical safeguards to reduce the risk of advanced AI escaping human control.
- Where
- The policies and statements were issued in China, including through the Cyberspace Administration of China and events in Shanghai and the United Nations.
- When
- China introduced an AI safety framework in September 2024, expanded it in September 2025, and issued AI-agent guidelines in May.
- Why
- Authorities are responding to concerns that increasingly capable AI could obtain resources, replicate itself, evade safeguards or seek power.
Open, Inspectable AI
Restricted, Controlled AI
Model accessibility
Open, Inspectable AI
Chinese developers promote open-weight models because cybersecurity teams can inspect, modify and deploy them for defensive work.
Restricted, Controlled AI
Closed-source models impose stronger access restrictions, but the article says they may be less useful for some forensic analysis.
Security trade-offs
Open, Inspectable AI
Open models can help researchers investigate attacks and examine how systems work.
Restricted, Controlled AI
Experts warn that open-weight models can be modified and redistributed with little oversight, creating additional risks.
Oversight approach
Open, Inspectable AI
China’s standards allow third-party safety assessments, outside evaluation bodies and security researchers to test and audit open models.
Restricted, Controlled AI
Anthropic has advocated independent monitors embedded inside AI companies, an approach China has not proposed.
Key facts
- Core concern
- Advanced AI could escape effective human oversight and compete with humans for control.
- Initial framework
- China’s first explicit loss-of-control scenario appeared in an AI safety framework released in September 2024.
- Expanded framework
- A September 2025 update added the possibility of a sudden, unexpectedly large leap in AI intelligence.
- Political position
- President Xi Jinping said AI should always remain under human control.
- AI-agent rules
- May guidelines require developers to discover, intervene in, block and recover from improper agent behavior.
- Identified risks
- The guidelines cite data poisoning, algorithm manipulation, system vulnerabilities and operational loss of control.
- Decision authority
- Users are expected to retain final decision-making authority over autonomous AI-agent decisions.
Quotes
Xi Jinping
President of China, speaking at the World Artificial Intelligence Conference in Shanghai
“always remain under human control”
NDTV
Sources
Related news
China Prepares AI Safety Rules Amid Fears of Lost Control
China Warns AI Could Threaten Political Stability and Security
AI Leaders Urge Caution as China Flags Political Security Threats
Gates Seeks Xi Talks on Global Rules for AI Risks
Beijing Rejects Anthropic CEO’s Call to Curb China’s AI
U.S. and China Prepare for Mid-September AI Safety Talks
Ex-Diplomat Warns Modi-Xi Talks Won’t Yield Major Border Concessions
BRICS New Delhi Declaration Signals Global South Challenge Western Order
Chinese Workers Protest Unpaid Wages at Russian Construction Sites
