Safeguarding against security risks in AI agents

AI agents are emerging as a particularly powerful and transformative technology, representing a fundamental shift in how AI interacts with digital and physical environments. But they also bring a host of new risks and security threats that organisations must address proactively.

Powered by advanced models from companies like OpenAI and Microsoft, AI agents are already being integrated into various enterprise products, offering significant benefits in automation and efficiency. These agents can act autonomously or semi-autonomously, making decisions and taking actions with minimal human intervention. While this autonomy opens up new possibilities, it also expands the threat surface significantly.

Traditionally, AI-related risks have been confined to the inputs, processing and outputs of models, along with the vulnerabilities in the software layers that orchestrate them. With AI agents, however, the risks extend far beyond these boundaries.

The chain of events and interactions initiated by AI agents can be vast and complex, often invisible to human operators. This lack of visibility can lead to serious security concerns, as organisations struggle to monitor and control the agents’ actions in real time.

Among the most pressing risks are data exposure and exfiltration, which can occur at any point along the chain of agent-driven events. The unbridled consumption of system resources by AI agents — benign or malicious — can lead to denial of service or wallet scenarios, where system resources are overwhelmed. Perhaps more concerning is the potential for unauthorised or malicious activities carried out by misguided autonomous agents, including ‘agent hijacking’ by external actors.

The risk doesn’t stop there. Coding errors within AI agents can lead to unintended data breaches or other security threats, while the use of third-party libraries or code introduces supply chain risks that can compromise both AI and non-AI environments.

The hard-coding of credentials within agents, a common practice in low-code or no-code development environments, further exacerbates access management issues, making it easier for attackers to exploit these agents for nefarious purposes.

Robust controls to mitigate risks

Given the multifaceted risks associated with AI agents, organisations should implement robust controls to manage these threats effectively.

The first step in mitigating AI agent risks is to provide a comprehensive view and map of all agent activities, processes, connections, data exposures and information flows. This visibility is crucial for detecting anomalies and ensuring that agent interactions align with enterprise security policies. An immutable audit trail of agent interactions should also be maintained to support accountability and traceability.

It is also essential to have a detailed dashboard that tracks how AI agents are used, their performance against enterprise policies, and their compliance with security, privacy and legal requirements. This dashboard should also integrate with existing enterprise identity and access management (IAM) systems to enforce least privilege access and prevent unauthorised actions by AI agents.

Once a comprehensive map of agent activities is in place, consider establishing mechanisms to detect and flag any anomalous or policy-violating activities. Baseline behaviours should be established to identify outlier transactions, which can then be addressed through automatic real-time remediation.

Given the speed and volume of AI agent interactions, humans alone cannot scale the oversight and remediation required. Therefore, implement tools that can automatically suspend and remediate rogue transactions while forwarding any unresolved issues to human operators for manual review.

The final control involves applying automatic real-time remediation to address detected anomalies. This may include actions such as redacting sensitive data, enforcing least privilege access, and blocking access when violations are detected.

Also, it’s important to maintain deny lists of threat indicators and files that AI agents are disallowed from accessing. A continuous monitoring and feedback loop should be established to identify and correct any unwanted actions resulting from AI agent inaccuracies.

As AI agents become increasingly integrated into enterprise environments, the associated risks and security threats can’t be ignored. In this rapidly evolving landscape, proactive risk management isn’t just an option — it’s a necessity.

*Avivah Litan is a Distinguished VP Analyst in Gartner Research.

Image credit: iStock.com/rawintanpin