On Oct. 16, NIST, CISA and the NSA published new joint voluntary guidance outlining 10 incident response practices for advanced AI systems. Designed to help security teams prepare for, detect and recover from AI control failures, the framework applies established cybersecurity disciplines to autonomous systems acting outside their intended boundaries.
Security experts warn that the guidance leaves a critical gap for enterprise deployers. While federal agencies focused heavily on frontier model training and lab environments, real-world enterprise AI agents interact with complex application stacks where revoking an agent’s access rarely stops the chain reaction it set in motion.
“Where it falls short is scope,” said Chris Hughes, vice president of security strategy at Noma Security. He cautioned that its focus on frontier laboratories lackss direction for enterprise deployers, particularly for prompt injection and compromised models.
The guidance explicitly narrows its definition of a control incident to “significant uninstructed actions” that compromise operator-imposed policies, controls or oversight. Actions resulting from adversarial prompt injection or an intentionally backdoored model are considered instructed and excluded as the primary cause of a control incident. It also distinguishes control incidents from traditional cybersecurity incidents, meaning the guidance does not classify every unexpected agent action as a cyber incident.
Hughes said he nonetheless welcomes its use of familiar incident-response frameworks, least-privilege access, monitored network traffic and validation before restart. He added that the gateways and monitoring systems used to constrain agents introduce attack surfaces of their own.
The mechanics of containment
When an AI control failure occurs in an enterprise environment, an agent can leave unfinished business long after a shutdown. A revoked agent may leave behind queued requests processing under separate credentials, persistent memory instructions or active downstream processes that continue running unchecked across the network.
To contain an incident, security teams must trace activity across all affected systems to establish what the agent changed and what is still running.
“The dividing line is whether the agent failed at its job or crossed a boundary it wasn’t permitted to cross,” said Ronan Murphy, chief strategy officer at Forcepoint. “An agent accessing restricted data, changing permissions without approval or continuing to act after its access has been revoked should trigger incident response.”
Murphy said evidence should come from access records, tool execution, data transfers and permission changes. If responders cannot determine what an agent can still reach, he recommended containing it during the investigation.
Valid permissions can conceal unauthorized behavior
Even when containment rules are clear, an agent doesn’t have to defeat access control to create a security problem. Granted permissions may allow an operation that falls outside the employee-approved task, leaving responders to distinguish technical access from legitimate activity.
“An agent can create significant risk while operating entirely within its granted permissions,” said Aviv Nahum, co-founder and CEO of Above Security.
Nahum explained that detection requires examining what an agent is trying to accomplish, its interactions with systems and other agents, and whether its overall behavior remains consistent with the task.
Because agents can change approaches mid-stream to complete an approved task, security teams are left to determine whether a deviation is dangerous while the work continues.
“We can’t rely on an analyst noticing an alert and manually reconstructing what happened while an agent continues operating at machine speed,” Nahum said.
He advised continuous investigation, evidence preservation and containment, with human oversight proportionate to the potential consequences.
Test whether shutdown reaches delegated work
Beyond detection speed, enterprise containment hinges on how authority moves between applications and agents. Revoking the original identity may leave another worker operating under separate credentials or an existing session.
“Shutdown must also cover work already handed off,” Murphy said. “Disabling the original agent can leave queued work and other agents running.”
He recommended giving each agent a distinct identity, an accountable owner and narrowly approved tools. Credentials should be short-lived and issued for the task, while data policies and audit records remain outside the agent’s authority.
Teams can test those protections in an isolated environment using synthetic sensitive data. Murphy suggested starting a delegated task, revoking access while it runs, and checking whether any existing sessions or downstream tasks continue to run.
The exercise should also establish whether information can still leave the environment and whether the agent can modify its permissions or logs.
“Measure the time between revocation and the last successful action,” Murphy said. “That tells you what your shutdown procedure achieves.”
That interval gives responders evidence of enforcement rather than an acknowledgment that a cancellation request was accepted.
Recovery must include what the agent changed
While the new federal guidance outlines broad steps for returning systems to operation, security leaders caution that recovery from an AI incident involves far more than simply restarting a clean model. An unauthorized permission, an altered record or an instruction stored in persistent memory can remain long after the original execution ends.
“Restarting a clean agent or process does not repair a corrupted database entry or information inside files touched by such an agent,” explained Karim Eldefrawy, co-founder and CTO of Confidencial, a data security company.
He recommended using an independently reviewed recovery record covering the affected systems, the control failure and the change that closes or isolates the failure path. The review should account for credentials, delegated work and persistence mechanisms as well as software and configuration baselines.
Data owners also need to reconcile affected information against authoritative records. A checksum can establish that a restored file matches a baseline but cannot demonstrate that its contents are factually correct, or that a transaction was legitimate. Persistent memory and retrieval systems require similar scrutiny, including source documents, stored instructions, summaries and derived information.
“Clearing the conversation while preserving a poisoned knowledge source can recreate the incident immediately,” Eldefrawy said.
Resuming operations with fresh credentials, restricted permissions, a limited workload and explicit rollback criteria offer a safer path forward, he noted, provided security and business owners explicitly record the remaining uncertainty and accepted risk.
Near-miss records bridge the compliance gap
To truly operationalize the voluntary guidance, security programs must also capture data before a failure results in a full-blown crisis. A blocked action can expose a weakness without producing a completed incident. Recording the attempted operation, the control that intervened and related permissions help teams determine whether the same conditions exist elsewhere.
Murphy noted that audit entries should connect the agent to the person or process that initiated its task, then distinguish the requested work from the actions performed, with approvals and the start of containment also belonging in that record.
“Most programs log prompts reasonably well but miss the agent-to-application calls happening in the background, which is where the gap hides,” he said.
Near-miss reports establish which boundary was tested and what prevented harm. Rather than leaving alerts isolated, teams can use repeated patterns to inform proactive investigations across other agents and applications.
Ultimately, that depth of logging gives executive leadership and boards concrete proof that identified weaknesses are being actively addressed rather than buried in noise.
“A growing number of logs tells a board very little if nobody can show the underlying problems are being resolved,” Murphy said.