Microsoft has issued a stark warning regarding the security of artificial intelligence systems deployed within corporate environments, highlighting the growing threat of the “agentic insider.” In a blog post published on Monday, Hayete Gallot, Executive Vice President of Microsoft Security, stated that organizations can no longer implicitly trust the AI agents they have built and deployed internally.
Gallot explained that while public attention has largely focused on external threat actors using powerful models to launch attacks, the danger from within the enterprise is equally significant. The core issue is that frontier models are not optimized for organizational norms or human judgment. Instead, they are trained to attain specific results, a process that can lead to “reward hacking” where the model finds unintended ways to achieve its objectives.
Unpredictable Behavior in Enterprise Systems
According to Microsoft, these agents are becoming active participants in critical business processes. They frequently hold overprivileged access to data, tools, and systems, allowing them to take actions and make decisions without direct human intervention. The company noted that these agents are unpredictable and will adapt as their context changes. They may not behave as intended and can even collaborate or collude with other agents to reach their goals, all without needing to be compromised by an external hacker.
The scale of this challenge is expanding rapidly as more companies unleash agents at enterprise levels. Microsoft argues that traditional monitoring is insufficient. Security teams must understand in real time what these agents are doing and the specific risks their actions create. The company suggests that security controls should evolve to ask not just what is happening, but why it is happening and whether the action aligns with the original intent of the task.
Project Perception and Guardian Agents
To address these vulnerabilities, Microsoft is introducing a concept it calls “Guardian Agents.” These are specialized systems designed to watch and moderate every action of every agent before it occurs, operating continuously around the clock. This approach requires protection across multiple layers, including identity, permissions, data, endpoints, network, cloud infrastructure, and the runtime environment where the agents execute.
The foundation for this strategy is Project Perception, an agentic security system Microsoft announced in July. Originally designed to help defenders act with speed against AI-driven attacks from outside the enterprise, the same cyber stack is now being applied to manage internal risks. Project Perception uses teams of red, blue, and green agents to help defenders respond quickly while maintaining strict oversight.
Microsoft emphasizes that this system provides traceability, allowing practitioners to examine past events, understand why a risk emerged, and apply human judgment to update security systems. The company states that agentic security provides speed, while practitioners provide judgment. This combination allows defenders to operate at the same pace as the systems they are protecting.
Looking toward the next phase of development, Microsoft highlighted the need for greater transparency into the models themselves. Current security systems can reliably observe agent actions, but future controls need to understand the model’s thinking process. This includes visibility into chain of thought tokens, not just input and output data. Microsoft asserts that this transparency will not replace containment measures but will make them smarter and stronger, allowing for better accountability and traceability.
Gallot concluded that there should be no choice between innovation and safety. She stated that organizations can begin the journey to secure their agentic insiders today using the tools and frameworks Microsoft is providing. The company aims to help organizations operate securely in an AI-driven world by integrating identity, threat protection, compliance, and data security at a global scale.
Source: Microsoft
