Attackers Are Targeting The AI Ecosystem You Cannot See

Overview

This Entra Chat episode discusses how attackers are increasingly targeting the AI ecosystem—not just AI models themselves. The guest, Thomas Roachchia (ex-Microsoft), argues that security teams must shift focus toward AI agent security, threat intelligence for AI, and especially the “invisible” components of AI systems (agents, skills, plugins, MCP servers, and supply-chain dependencies).

Key Points and Arguments

1) AI security is moving faster than defenders can track

The conversation highlights how quickly new AI capabilities and releases are arriving (including differences in model availability/controls). This creates constant noise, making it hard to keep security foundations and guidance current.

2) Attackers target the AI ecosystem where visibility is limited

Even when organizations have guardrails around AI usage, attackers can compromise systems by attacking the ecosystem around AI. Examples include:

• Malicious npm packages
  • Agents may install dependencies without strong verification or governance.
• Malicious “skills” or MCP servers
  • Agents may trust remote tools or third-party connectors.
• Prompt injection / adversarial prompting
  • Including multi-step (“chain”) prompt attacks.
• Infrastructure abuse via leaked API keys
  • Attackers use stolen keys to access cloud AI workloads and rack up costs.

3) Open-source vs. proprietary models: guardrails aren’t consistent

Open models can sometimes be used without the guardrails present in certain proprietary offerings—encouraging more experimentation and expanding the attacker opportunity set.

4) “Agents already exist in attacker workflows”

The guest points to historical attacker behavior, such as agent-like automation used to sift through data and find secrets. The key difference now is speed and effectiveness, driven by modern agent frameworks and tooling.

5) The real danger: agent monitoring and security gaps

Security monitoring is more mature for user activity, but agent and multi-agent behavior is harder to observe because agents can act at high speed and often invisibly (“light speed” background actions). This motivates the guest’s startup focus on visibility and security for AI agents.

6) Agent identity / agent ID is emerging as an authorization foundation

They discuss agent identity/agent ID as analogous to user identity—needed to define and monitor what agents can access. Tools such as Microsoft’s Entra Agent ID (and similar emerging capabilities from other clouds) are described, but the guest emphasizes they remain incomplete and environment-specific, with standards still evolving.

7) Training and practical guidance are needed for the complexity

Thomas teaches a course titled “Practical AI for Threat Intelligence leveraging agentic workflow”, originally presented at Black Hat. It focuses on:

• Building reliable agentic CTI workflows
• Protocols and components (e.g., MCP, agent skills, and related ecosystem concepts)
• AI security topics (prompt attacks, adversarial behavior)
• RAG strategies and practical CTI data transformation
• How attackers abuse AI during threat intelligence workflows

The training is updated frequently (nearly weekly) to keep up with rapid changes. A key hands-on method: participants build an agent throughout the course, improving it iteratively as concepts are introduced.

Overall Conclusion

The episode frames AI threat risk as shifting from “protect the model” to “protect the AI ecosystem”—including the supply chain, tools/connectors agents use, and authorization/monitoring mechanisms such as agent identity. The guest stresses that defenders must catch up to attackers leveraging agentic workflows, malicious third-party artifacts, and prompt-based manipulation.

Presenters or Contributors

• Mar (host of Entra Chat)
• Thomas Roachchia (founder, Security Break; ex-Microsoft)