Kubernetes Workspaces: Enhancing Multi-Tenancy with Intelligent Apiserver... J. Munnelly, A. Tosatto

Summary of the talk: Kubernetes Workspaces for Multi-Tenancy via an Intelligent API-Server Proxy

Problem / Motivation (multi-tenancy pain in Kubernetes)

• Namespaces are the recommended boundary in Kubernetes for app identity and security (via RBAC), but in practice multi-tenant teams often end up with many namespaces per tenant/team.
• Existing tools/workflows handle cross-namespace operations poorly:
  • Tenants/controllers struggle to operate across multiple namespaces.
  • Users hit scalability issues when they “work around” tooling limitations by packing too many objects into a single namespace.
  • There appears to be no clear cluster-wide “scope” construct that naturally models a tenant as a group of namespaces rather than a single namespace.

Core Proposal: “Workspaces”

• Workspaces provide a logical view/scope over resources spanning multiple namespaces.
• A workspace is modeled as:
  • A name
  • A set of namespaces associated/selected for it (constructed via a selector; exact semantics are open for community discussion)
• Key property: Workspace scopes are enforced at the API level (proxy-level), which enables overlapping scopes.
  • Example: one team can access production resources while another manages its own apps across multiple “front-end” namespaces; these workspaces can overlap safely.

Design approach: intelligent proxy in front of the API server

• Rather than having controllers move resources around or creating new hierarchical/multicluster APIs, the talk focuses on an API server proxy.
• The proxy offers:
  • A cluster-like abstraction for clients, but scoped by workspace membership
  • Views into existing clusters instead of “owning” new resources or introducing new node-like objects
• A major goal is API compatibility:
  • A standard conformance suite could run against a workspace-exposed view (with a subset of namespaces).

How requests are handled

• Clients authenticate and authorize normally, but the proxy:
  • Delegates authentication to the backend cluster
  • Uses API impersonation (in the spirit of Kubernetes aggregation) so the backend API server performs authorization
  • Passes through the workspace context (which workspace the user authenticated for) embedded in impersonation headers/context, enabling workspace-aware authorization
• The proxy is intended to be lightweight:
  • It avoids decoding/rewriting objects where possible
  • It mainly rewrites HTTP path/query parameters:
    • translating a request like /apis/.../workspaces/... into equivalent requests targeting the selected set of namespaces

Selector-based mapping and API filtering

• Workspace-to-namespace membership is used to transform queries.
• The proxy relies on selector filtering and set operations (noted as requiring enhancements beyond current field-selector capabilities):
  • Potential reliance on label-selector “techniques”
  • Assumes planned contributions to Kubernetes core such as field selector set operations

Watching and event semantics

• For watch, the proxy must preserve Kubernetes guarantees such as linearizable event ordering.
• The design pushes complexity to the backend:
  • it relies on backend watch/caching mechanisms rather than merging event streams at the proxy layer.

RBAC / authorization model: workspace-scoped cluster roles

• Standard Kubernetes RBAC uses ClusterRoles/ClusterRoleBindings, which is difficult in multi-tenant setups because tenants shouldn’t define unrestricted cluster-scoped RBAC.
• Introduced approach:
  • Internally store workspace-scoped RBAC objects (e.g., WorkspaceRoles / WorkspaceRoleBindings)
  • Expose them to users as ClusterRoles/ClusterRoleBindings for API compatibility
• Mechanics:
  • The proxy intercepts “cluster-scoped” RBAC requests from users
  • Converts workspace-scoped RBAC objects into namespace-scoped equivalents within an assigned “workspace RBAC namespace”
  • Converts again for read/list/watch responses so users don’t see internal details
• Tradeoff:
  • Conversion/encoding-decoding is done for RBAC objects only (not high-churn resources like Pods/Events) due to overhead.

Demo outcomes

• Demonstrated:
  • Different workspace contexts change what the client “sees” (filtered namespace set).
  • A controller that watches a cluster-scoped custom resource (example: “Fu” in the sample pattern) can be made to reconcile only resources available in the workspace.
• Key concept:
  • When the controller runs through a workspace-scoped endpoint, it creates/maintains Deployments only inside namespaces belonging to that workspace, and does not reconcile outside the selection.

Frequently asked questions: CRDs

• Question: Do CRDs fit into this approach?
• Response: CRDs are problematic due to garbage collection/ownership issues and because the backend cluster may not understand workspace-specific CRD ownership.
• Therefore, CRDs are not the immediate target.

Questions on observability pipelines

• Observability tools (e.g., OpenTelemetry collectors, metrics, Cube state metrics) are often deployed per namespace today.
• With workspaces, teams can deploy observability tooling once per workspace:
  • A “team observability” namespace hosts collectors/controllers
  • The collector uses workspace-scoped permissions to discover/scrape applications across the workspace namespaces
• Clarification:
  • This doesn’t eliminate existing designs—cluster-wide controllers/pipelines can still run, and workspace views remain compatible.

Questions on network policies

• NetworkPolicy is namespace-scoped in Kubernetes.
• Workspace scoping is mainly suggested to rely on labels/interoperability, but network-policy inheritance across workspace members is not automatically handled for namespace-scoped resources.
• StorageClass was also mentioned:
  • It is cluster-scoped, so workspace scoping doesn’t apply cleanly without deeper complications
  • Related Kubernetes resources like CSI objects further complicate “workspace-scoped storage.”

Next steps / roadmap

• Open work toward Kubernetes core features needed by the design:
  • Selectors + field selector set operations
  • Potential watch-related features (e.g., bookmarks/consistent reads)
• Plan to publish design documents and seek a home in Kubernetes SIG repositories.
• Intend to collaborate with the community and avoid re-implementing authorization mechanisms.

Main speakers / sources

• James Munnelly (Apple) — field engineer; Kubernetes Workspaces proposal and overall architecture explanation
• Andrea Tosatto (Apple) — demo/controller walkthrough and controller-scoping demonstration