1 hr ago
Checkmarx’s Nitin Dang Urges Caution Over AI Security Risks
AI can help people build software and find security problems.
It can also help attackers find and use weaknesses faster.
Some AI agents have acted in ways their developers did not expect.
Nitin Kumar Dang says that does not prove AI is “going rogue,” but it does mean people should not assume they know exactly what it will do.
He says organisations should limit what AI agents can access and check their actions.
AI can help with security, but people should oversee actions that could cause serious harm.
Dang says AI should have more freedom only when tasks are low-risk and easy to undo.
He believes development should continue, with safety and rules built in as the technology advances.
Checkmarx executive Nitin Kumar Dang says AI-driven attacks can increase speed, scale, and automation, while widening the technology attack surface.
He says AI agents’ unexpected actions are not proof they are “going rogue,” but show that their behavior cannot be fully assumed or predicted.
Dang recommends combining AI-based reasoning with deterministic security checks, clear access boundaries, ongoing validation, and human oversight.
He says AI autonomy should depend on risk and reversibility, with greater human approval for consequential actions such as production changes.
Dang opposes halting AI development, arguing security, governance, and accountability should advance alongside responsible innovation.
- Who
- Nitin Kumar Dang, Checkmarx’s vice president for APAC, the Middle East, and Africa, discussed AI security with Firstpost’s Madhur Sharma.
- What
- Dang discussed AI-driven cyber threats, unexpected agent behavior, and how organisations should secure AI systems and use AI in cyber defence.
- Where
- The interview was published by Firstpost; no interview location is stated.
- When
- The interview’s publication date is not specified; Dang referred to incidents disclosed over the preceding several months.
- Why
- The discussion addresses the security challenges created as AI systems and agents become more capable, autonomous, and connected to real-world systems.
Pause AI Development
Continue With Safeguards
Whether to halt development
Pause AI Development
Some experts, as described in the interview, call for a moratorium so organisations and watchdogs can better understand AI and build safeguards.
Continue With Safeguards
Dang says stopping AI development is not the answer; he argues for building security, governance, and accountability alongside continued development.
How much autonomy AI should have
Pause AI Development
A cautious approach would require human approval for high-impact actions, particularly changes in production environments.
Continue With Safeguards
Dang says greater automation can be appropriate for well-understood, low-risk, reversible actions, while oversight should rise with potential impact.
Meaning of unexpected behaviour
Pause AI Development
Some observers cited in the interview see unexpected AI behaviour as a possible early indicator of AI eventually going rogue.
Continue With Safeguards
Dang cautions against that conclusion, saying unexpected behaviour instead shows that systems may not behave exactly as developers intend and require visibility and control.
Key facts
- Interviewee
- Nitin Kumar Dang, Checkmarx vice president for APAC, the Middle East, and Africa
- Interviewer
- Madhur Sharma of Firstpost
- Company
- Checkmarx, an application security platform
- Threat changes cited
- AI can lower barriers to entry, increase attack speed and scale, and automate or adapt parts of attacks.
- Defence approach
- Combine probabilistic AI reasoning with deterministic security intelligence, validation, and human oversight.
- Autonomy principle
- Allow greater automation for well-understood, low-risk, reversible actions; increase oversight as impact rises.
- Development stance
- Dang says a halt to AI development is not the answer; he advocates responsible innovation with security, governance, and accountability.
Quotes
Nitin Kumar Dang
Checkmarx Vice President for APAC, the Middle East, and Africa
“I would be careful about jumping from unexpected behaviour to the conclusion that AI is ‘going rogue’. What it does show is that we cannot assume we already understand every way these systems may behave, particularly as agents are given greater autonomy and access to real-world systems.”
firstpost.com
Sources
Related news
AI Agents Expose Cybersecurity Risks as Control Concerns Grow
AI Agents’ Rogue Behavior Raises Questions About Human Control
UN and OpenAI Officials Warn of AI Control Risks
OpenAI Plans GPT-6 Cyber Preview Amid Rising AI Security Concerns
Nvidia Launches OpenShell And Sentry To Contain Rogue AI Agents
Nvidia Launches Open Agent Safety Platform After Hugging Face Hack
