Video summary

How Xbox Secures 70+ Entra Tenants Every Night with Maester

Main summary

Key takeaways

Technology

Summary of the Video (Technological Concepts + Maester Usage)

This episode, Entra Chat, explains how Xbox Security uses Maester to continuously assess and report Microsoft Entra ID security baselines across a very large, multi-tenant gaming environment. The environment includes Xbox Live, Game Pass, and 40+ game studios—many of which have been acquired.

Speakers and Key Context

  • Audrey (Microsoft, Xbox Security identity/security architect — she corrects her role from “principal PM” in the intro) Focuses on securing Entra/identity and Azure platform/services for Xbox and game studios. Her scope also includes work “beyond Azure,” mentioning AWS/GCP.

  • Sam Describes his role as a co-maintainer in the Maester ecosystem.

  • Additional referenced roles/sources:

    • Xbox SOC (security operations signals and misconfiguration/attack learnings)
    • GitHub Actions/workflows (primary orchestration mechanism)
    • An internal Xbox reporting/GRC workflow tool (homegrown)

The Core Problem Maester Solves at Xbox

Xbox must apply an Entra baseline “gaming security spec” across:

  • 50 controls
  • 70+ Entra tenants (discussed as 70+ during operations)

Why standard Microsoft Entra ID baselines don’t fully fit gaming studios

The episode highlights several practical mismatch points:

  • Studios are often acquisitions, with identity/tenant constraints that differ from “first-party” Microsoft employee environments.
  • Some Microsoft services referenced in the corporate baseline may not exist or be accessible to studios.
    • Example: a requirement tied to OneErt/OneERT doesn’t translate well due to differences in identity/tooling.
  • Tenant scale varies widely, making “one-size-fits-all” controls impractical:
    • Examples range from 10–20–100 users up to much larger corporate tenant patterns.

How Xbox Built Their “Gaming Entra Baseline”

Xbox:

  1. Took the Microsoft Entra ID baseline
  2. Produced a gaming-specific version of that baseline
  3. Used Maester to:
    • Grade compliance against baseline controls
    • Generate readable per-studio reports (pass/fail, plus additional statuses like “investigate”)
    • Apply custom PowerShell logic for baseline-specific evaluation and multi-tenant handling
    • Run in a multi-tenant pipeline, allowing one pipeline to test many tenants in parallel

Nightly Automation + Multi-Tenant Pipeline

Maester testing runs nightly for all 70+ tenants.

What happens to the results

  • Results are stored/flow through an internal database
  • They are used to:
    • Start ticketing / issue handling workflows
    • Support exception handling, including time-based exemptions

How data collection works (high level)

  • A “wrapper”/flow runs the Entra security scanner with Maester and other tooling (not fully detailed in the episode)
  • Federated identity credentials allow collection across tenants while still running within a single pipeline

Exception Handling + Reducing Noisy Findings

Xbox leverages Maester exception mechanisms so that findings can be suppressed or adjusted based on tenant posture.

  • Certain findings can be excluded when they’re no longer relevant
  • Entire tests can be skipped when they become obsolete

Example: rollout of phishing-resistant MFA (PR MFA)

  • During rollout, teams coexist with an older model: MFA for all admins
  • As PR MFA becomes the target posture, they exclude MFA test expectations accordingly
  • Eventually, they plan to remove the older MFA policy and update baseline/test expectations to match the new reality

Integration with CI/CD and GitHub Workflows

  • GitHub Actions/workflows act as the primary orchestration layer for security testing and reporting
  • Xbox generates dashboards/datasets from Maester output:
    • Initially “quick and dirty” inside GitHub
    • Then exported into a homegrown reporting tool
  • They emphasize integration with developer workflows using custom aggregation scripts/jobs, such as:
    • Tenants still using MFA
    • Conditional access policy coverage
    • Percentage of users with a PR MFA method

Overall, they explicitly frame GitHub as the operational layer for Maester-driven security assessment.

Why Maester’s UI and Reporting Mattered

Guests highlight several operational reporting/UI features that make Maester effective at scale:

  • Reports show pass/fail and clearly explain why a test fails
  • Clickability / deep links:
    • From a failing test, operators can click directly to the relevant asset (e.g., a service principal) in the target tenant
  • Remediation guidance included per test, reducing the usual burden of “what is the problem and how do I fix it?”
  • These features are especially important in gaming studios where Entra admins may also manage CI/CD/IaC and may not be deeply specialized in Entra UI navigation
  • Deep links also reduce friction when Microsoft portal UI structure changes (reorganized menus, shifting navigation paths)

Extending Maester with Custom Tests (Beyond Baseline)

Xbox runs tests in two main categories:

  1. Baseline tests

    • Compliance checks against the Entra gaming baseline

  2. Custom tests

    • Tool health checks Example: verify required service principals exist and have the right permissions for security telemetry

    • Data collection for investigations and audits Example: number of global admins, counts of B2B/B2B guests/internal accounts, etc.

This approach reduces the need to manually request information from 70 tenant owners.

Governance Process for Evolving the Baseline

Baseline controls evolve over time:

  • They review upstream Microsoft baseline changes
  • They incorporate real-world signals from Xbox SOC about attacks and misconfigurations observed in the environment

When controls need updates:

  • They open a PR to add or modify Maester custom tests
  • Tests go through quality checks and then are included in the nightly pipeline
  • Exception handling and time-based exemptions are integrated into the pipeline workflow

The “Investigate” Status for Non-Pass/Fail Items

In addition to pass/fail, Xbox uses an “investigate” tag/status for cases where an item isn’t purely compliance-on/off.

Example: quarterly access reviews

  • Not purely pass/fail—more of a process reminder and compliance task
  • Maester can attach relevant context so tickets/instructions include which users still require review eligibility, such as:
    • PIM eligibility
    • Static RO (role) assignments

Key Product/Technology Claims About Maester (From the Maintainer)

The maintainer (referenced in the conversation as Michael) makes several points about Maester’s direction and community impact:

  • Maester originally aimed to be a framework where users write custom tests, and the community expanded it into out-of-the-box tests
    • Including contributions aligned to standards like CIS/CISA
  • Maester supports “agents” (discussed conceptually) to help write tests faster by leveraging existing codebases/patterns
  • The maintainer mentions broad adoption (e.g., “50k+ tenants”)
  • Community-driven tests improve the framework’s effectiveness and coverage

Main Speakers / Sources (As Referenced)

  • Audrey — Microsoft (Xbox Security; identity/security architect; Entra/identity and Azure security; mentions some AWS/GCP work)
  • Sam — Maester co-maintainer / Entra Chat participant
  • (Maester community/maintainer “you” referenced by name as Michael) — contributor/creator context mentioned; not fully identified in subtitles
  • Additional roles referenced: Xbox SOC, GitHub Actions/Workflows, and an internal Xbox reporting/GRC-related workflow tool (homegrown)

Original video