Governing AI Agents at Scale: Identity, Scope, and Observability (with Glean and Cvent)

High-level summary

Topic: Governing AI agents at enterprise scale — security, governance, compliance, and operationalization — using Glean’s Work AI Institute “AWARE” framework as a practical lens. Context: Cvent (≈30k customers, ≈5,500 employees) has deployed thousands of agents (>6,000 created; ~1,300 actively used). Speakers described the technical, organizational, and process changes they made to adopt agents safely and quickly.

Key technological concepts and challenges

• Agent non-determinism vs. traditional security architectures Existing IAM, monitoring, and audit systems were designed around deterministic systems and user-based actions. Agents plan, delegate, impersonate, and change behavior over time, so those architectures do not map cleanly.

• Agent identity & scope It is critical to define:

  • Who: identity / delegated identity
  • What: permitted actions
  • Where: data sources and context Different scopes (read-only summaries vs. modifying records) have very different risk profiles.

• Observability Agent reasoning is often opaque. Enterprises need instrumentation that reconstructs agent decisions for incident response and regulatory compliance (e.g., EU AI Act).

• Data access & model security The shift is from model-training-data concerns toward capability-oriented risks — what agents can access and what actions they can take.

• On-device inference trade-offs Running inference on-device reduces exfiltration risk but can limit observability and centralized control.

• Agent sprawl and scale Thousands of agents demand tiered governance; identical controls applied to every agent will not scale.

The AWARE framework (Glean’s Work AI Institute)

• Five governance challenges it addresses:

  1. Intent
  2. Context
  3. Guardrails
  4. Risk scoring and blocking
  5. Ecosystem observability

• Practical value:

  • Provides a shared language and clear success criteria
  • Acts as a checklist to streamline go/no-go decisions
  • Helps security move from being a gatekeeper to becoming a business partner

Operational practices, processes, and products

• Platform example

  • Glean provided fine-grain security controls and moderation, enabling rapid adoption and an estimated ~9 months of time savings during rollout.

• Staged adoption model

  1. Stage 0 — Learn/demo: business can try demos without IT involvement.
  2. Stage 1 — Sandbox/test-and-learn: no production data, isolated workspace.
  3. Stage 2 — ROI/proof-of-value: CFO/PMO gating for investment decisions.
  4. Stage 3 — Production integration: stricter controls, observability, and documentation.

• Risk process and governance

  • Create a structured, streamlined risk-review process with explicit success criteria to speed approvals.
  • Use a tiered risk catalog (task/activity-level) documenting what an agent does so security, legal, and privacy teams can assess risk.
  • Treat go/no-go decisions as iterative — agents and use cases evolve.

• Instrumentation & inventory

  • Priority: inventory agents and instrument them for identity and observability first.
  • Maintain a queryable catalog documenting agents, their tasks, and data access.

• Training and culture

  • Mandatory company-wide AI literacy training that includes security and legal aspects (example: CEO attended the first session).
  • Build capability so business teams can experiment responsibly without overloading IT/security.

• Automation & tooling for governance

  • Use AI-driven bots or tooling to perform initial risk assessments.
  • Prefer vendor solutions that embed governance controls (enterprise-grade overlays) to avoid bespoke configurations per agent.

• Compliance anticipation

  • Expect control requirements (e.g., SOC 2-level criteria and regulatory standards) to evolve. Be prepared to demonstrate inventory, observability, and governance.

Practical recommendations (concise)

• Synthesize AWARE with broader frameworks (NIST, EU AI Act) rather than replacing them.
• Start with an agent inventory and instrument for identity and observability.
• Implement a tiered risk process and maintain a documented catalog of agents/tasks.
• Use sandboxed test-and-learn environments for early experiments (no production data).
• Define clear success criteria and shared language to make decisions faster and less adversarial.
• Favor vendor/platforms that provide built-in governance and guardrails to accelerate safe adoption.

Examples & metrics from Cvent

• Company size: ~30,000 customers; ~5,500 employees
• Agent footprint: >6,000 agents created; ~1,300 actively used
• Outcomes from Glean:
  • Faster rollout (estimated ~9 months saved)
  • Added moderation and metrics capabilities after initial sprawl
  • User demand trending toward enabling workflows/actions (next stage)

Main speakers / sources

• Pradeep Manakkara — CIO, Cvent
• Ben Mayrides — CISO, Cvent
• Reference framework and platform: Glean’s Work AI Institute “AWARE” framework and Glean platform (as described by speakers)