SIGCOMM'26: Networking for Meta's Gigawatt-scale AI fleet (Sponsored Session)

Overview (Meta SIGCOMM’26 Sponsored Session)

• A Meta speaker frames the session as not a research talk, but a discussion of problem domains/challenges and how Meta thinks about AI networking for its gigawatt-scale AI fleet.
• They explicitly say they won’t present major solutions, positioning the talk as guidance on network design/operations challenges rather than full technical proposals.

Big-picture AI + networking motivation

Meta emphasizes:

• A company direction toward open-source AI
• Deploying state-of-the-art models at global scale

Examples of mentioned model/product progress:

• Music 1.2: highly optimized for complex coding workflows, with a 1M-token context window
• Music Greemer: a 30B open model designed to run locally on a single GPU
• Expansion of generative AI capabilities: Music Image / Music Video / Music Code

Infrastructure scaling demands:

• Training clusters growing from about ~24,000 GPUs (2022–2023) to >129K GPUs in multi-building clusters (by 2024)
• By 2025+, Meta is designing/deploying multi-region “AI supercomputers” exceeding 1 GW, networking hundreds of thousands of accelerators across multi-gigawatt footprints

Network is broken into three architectural scale domains:

1. Scale up
2. Scale out
3. Scale across

Scale-up networking (intra-node / intra-rack ultra-low latency)

Goal / definition

• Connect accelerators within a node or rack so they behave like a unified compute engine.
• Requires sub-microsecond latency and tight co-design across silicon, optics, and memory semantics.

Hardware examples / configurations

• Mentions deployments using NVIDIA “GP200” and “VL72” architectures (as named in subtitles), with high-density backplane interconnects reaching terabytes/second.
• Cables between racks are described as part of the scale-up boundary.
• Uses a heterogeneous ecosystem, including an AMD Helios platform featuring:
  • UALink switch ASICs
  • Copper cabling back to the rack fabric/switch tray (as described)
• Mentions Meta’s own accelerator/networking component:
  • MT2500 (called Meta’s own ASIC), enabling Ethernet-based scale-up with the ASIC.

Key technology trend

• Historically, proprietary protocols were implied as necessary.
• A major push: Ethernet for scale-up, bringing open standardized Ethernet into intra-rack interconnects.

Summary statement

• Scale-up is described as the “core” of the AI supercomputer, requiring extremely low latency plus GPU-to-GPU memory read/write semantics (communication enabling GPUs to directly access each other’s memory).

Scale-out networking (data center-wide training clusters)

Background / motivation

• Meta previously (referencing SIGCOMM) built two large 32K clusters using Rocky + InfiniBand, and argued Ethernet can work for massive AI training.
• They claim this helped dispel the belief that only proprietary lossless fabrics are mandatory.

Fabrics: DSF vs NSF

1. DSF (Disaggregated Scheduled Fabric)

• Introduced to eliminate tail latency and head-of-line / hash collision issues.
• Described as operating “like a single distributed credit-based switch backplane.”
• Provides non-blocking predictable performance across heterogeneous clusters (supporting different accelerator systems).

2. NSF (Non-scheduled fabric)

• Uses standard front-end switches.
• Achieves high utilization through host-side tuning and adaptive routing.

Strategic summary

For Meta, scale-out must be:

• Completely open (supporting multiple accelerator vendors)
• Standardized on Ethernet
• Still customizable by goals:
  • Scheduled fabrics for predictability
  • Highly optimized non-scheduled fabrics for commodity efficiency

Scale-across networking (region to global / WAN-style AI backbone)

What it is

• Extends connectivity from buildings to regions and campuses, and across very long distances using a dedicated WAN AI backbone.

Example facility

• Meta’s Prometheus data center region in Ohio (USA):
  • Originally not designed solely as an AI data center, later expanded
  • Different buildings vary in form/size
  • Inter-building links were added as demand grew

Primary constraints

• Unlike scale-up/out, power availability and geographic constraints dominate:
  • A region limited to roughly ~100 MW across multiple buildings
• Therefore, scaling requires across links between data centers and beyond regions.

Geographic / distance claims

• Scale ranges:
  • ~1 km within a campus
  • 10 km between local sites
  • 100 km across the country
• For very long distances (subtitles: 10 to 100K kilometers), Meta deploys dedicated WAN AI backbone.

Optics experimentation

• At these distances, latency becomes a major factor (not just bandwidth).
• They describe actively experimenting with optical technologies.

Meta “lessons learned” for building/operating AI networks at this scale

The transcript’s closest equivalent to an “analysis” section.

1. Know your jobs and their traffic patterns

• Design the network based on AI workload characteristics (training/inference variants).
• Challenges:
  • Workloads evolve frequently, including via open-source changes
• Requirements:
  • Adaptability to support evolving technologies
  • Active testing/benchmarking/measurement per workload type (examples mentioned: LM training/inference, post-training, R&R training/inference)

2. Design the entire system, not just components

• Emphasize cross-layer optimization from chip/silicon through system levels up to campus/global scales.
• Organize expertise across:
  • silicon/chip experts
  • system + software experts
  • model developers
• Bottleneck risk:
  • Since a single AI job spans hundreds of servers, any component bottleneck can stall the whole system.
• Use Amdahl’s law to prioritize engineering effort.
• Avoid optimizing network protocol for raw speed unless it reduces end-to-end training time.

3. Operational life cycle, resilience, and debugging

• “Gigawatt-scale” implies continuous operational issues (“managing entropy”).
• Need resilience, including:
  • automated real-time debugging
  • instant routing around failed links
  • rolling software/firmware upgrades on live switches without packet loss
  • ability to swap physical network components without stalling ongoing long training jobs

Product/technology features explicitly mentioned

• 1M-token context (model capability; reflects compute/network demand)
• Open Ethernet-based scale-up direction (standardization + lower latency requirements)
• DSF (disaggregated scheduled fabric):
  • predictable performance
  • tail latency and collision mitigation
• NSF (non-scheduled Ethernet):
  • host-side tuning
  • adaptive routing
• Multi-region AI supercomputers (power capacity >1 GW)
• Operational features:
  • automated real-time debugging
  • link failover/rerouting
  • rolling firmware upgrades
  • hot component replacement

Main speaker/source(s)

• Meta (speaker, introduced as “I’m from Meta again”)
• Video is a SIGCOMM’26 Sponsored Session (host/venue not identified by name in subtitles)

No other named individuals or external sources are clearly identified in the provided subtitles.