Summary of "I was laid off by Atlassian"

Main ideas / lessons conveyed

Job context & purpose: The speaker was laid off by Atlassian and reflects on ~8 years there. Most of the work was technical, but the speaker also highlights non-technical growth and lessons learned.
Goal of the work: Build internal platform capabilities that enable self-service infrastructure—especially load balancing—so developers don’t need to rely on a centralized team for every request.
Overall architecture approach: Create a platform pipeline composed of:
- a Service Broker API
- asynchronous provisioning workers and storage
- a dynamic control plane / management server that transforms platform data into runtime configuration
- long-lived proxy infrastructure that applies configuration dynamically
Centralized edge/load-balancing platform: Replace expensive enterprise load balancers with an open-source, Envoy-based system that supports many regions and thousands of proxy instances, configured at runtime.
Move logic “upstream”: Handle routing, logging, auth/authorization, rate limiting, and DDoS-related handling at the edge proxies to keep backend services simpler and ship features faster.
Maintenance & people skills matter: Successful systems require:
- documentation
- onboarding and training
- maintenance planning
- diplomacy and conflict handling (mentoring included)

Detailed methodology / system-building steps (as described)

1) Interview process & hiring rationale

Coding quiz: HackerRank; the speaker “aced it” with full marks.
White-paper technical interview: Read a Cloudflare white paper (topic: custom domains) and answer questions.
Architecture/tech topics: Microservices, architecture concerns, containers.
Troubleshooting exercise: Simulate debugging an Atlassian incident where an application issue leads to denial of service.
DNS/latency-based routing question: Latency-based DNS; the speaker’s initial answer wasn’t accurate but was considered plausible.
Values interview: Asked interviewers to imagine 12 months ahead and what achievement would justify hiring.
- Desired outcome: build an internal application enabling self-service load balancers (similar to cloud-provider self-service).
- The speaker expressed confidence building the needed web framework, grounded in prior Python web-app experience—leading to the hire.

2) Build the self-service provisioning foundation (Open Service Broker)

Initial focus (early weeks/months):
- Build an OSB-style broker (open service broker) with an API for internal developers to request provisioning.
High-level broker behavior:
- Developers call the broker API (web app) to request provisioning.
- The broker avoids heavy synchronous work.
- It submits tasks to a queue for async processing.
Async provisioning flow:
- Client → FastAPI web server (broker API)
- Web server sends provisioning requests to SQS
- Worker consumes tasks and performs steps such as:
  - creating DNS records
  - creating/updating CloudFront distributions
  - calling cloud APIs, etc.
- Worker writes task outcomes/status to DynamoDB
- Client polls the broker until completion:
  - “ready/finished” or “error”
Implementation evolution:
- Began with a Python library (“connection”) to generate routes from an OpenAPI document.
- Migrated to Flask, then to FastAPI.

3) Build dynamic Envoy configuration management (Envoy Control Plane)

Trigger/need: An architect wanted to replace licensed enterprise load balancers with an open-source proxy.
Chosen proxy technology: Envoy proxy
Core requirement: Configure Envoy dynamically at runtime so proxies can keep running while configuration changes based on provisioning results.
Control plane built by the speaker/team:
- A FastAPI-based management server called “Envoy control plane.”
- The speaker notes it was open-sourced as Sovereign, with a public Bitbucket repo “for now.”
Template-based rendering model:
- Control plane loads templates and context.
- Templates map to Envoy configuration constructs (e.g., clusters, routes, listeners).
- Management server polls/reads changing context from sources including:
  - broker/database outputs (provisioned results)
  - potentially other dynamic sources (e.g., an S3 bucket)
- It renders validated Envoy configuration outputs as context changes.
End-to-end outcome: Broker/worker updates data → control plane polls and regenerates templates → Envoy proxies apply updated configuration.

4) Provision and scale the proxy fleet (Infrastructure + AMIs)

Scale example: ~2000 proxies across ~13 regions.
Infrastructure as Code: Proxies provisioned via AWS CloudFormation templates (conceptually including):
- VPC, subnets, internet gateway, security groups
- IAM roles, key pairs
- Auto Scaling Group (ASG) and EC2 instances
- load balancer components such as NLB (L4 proxy)
- ACM and Route 53 records (typical AWS building blocks)
AMI creation pipeline:
- AMIs produced using:
  - HashiCorp Packer
  - Salt Stack
- Process (described at a high level):
  - spin up dev EC2 (using Packer EC2 provisioner)
  - deploy SaltStack configuration
  - install/configure Envoy
  - harden and tune network settings
  - add logging/tracing/observability agents
  - snapshot and shut down to produce an AMI
Runtime parameters & secrets:
- CloudFormation passes parameters at launch time (including secrets/keys).
- Proxies fetch needed resources and begin accepting traffic.

5) Expand platform adoption across major Atlassian products

After the foundation: The platform becomes the standardized way to do centralized load balancing.
Enforcement change: Teams previously could expose services publicly through more basic load balancers. The platform introduced a restriction:
- teams must explicitly route via centralized load-balancing infrastructure to declare public exposure.
Products migrated behind the edge infrastructure (examples named):
- Jira
- Confluence
- Bitbucket
- Status Page
- and others

6) Evolving proxy capabilities to centralize cross-cutting concerns

Why it mattered: Envoy configuration can be complex (routing logic, virtual hosts, header manipulation, redirects, cluster selection, extensions, network filters, HTTP connection manager, etc.).
Validation/abstraction to keep it safe and manageable:
- validate logic for parameters
- ensure templates render only valid configurations
- abstract developer input away from raw Envoy complexity
Centralizing concerns early in the request chain: Requests flow: Customer → Proxy → Backend service Handling problems at the proxy layer avoids repeating concerns across thousands of backend services.
Examples of implemented concerns:
- DDoS protection: attributed to CloudFront (speaker credits a colleague)
- Access logs: implemented using Envoy network filters / HTTP connection manager with template-driven configuration
- More complex features via sidecars:
  - Sidecar model: additional containers/services alongside the proxy
  - sidecars receive dynamic configuration over the wire
  - some sidecars contributed by other teams
  - speaker created an authentication sidecar written in Rust
  - authorization and rate limiting sidecars handled by other teams

Non-technical lessons (growth and maintenance)

Diplomacy & conflict management:
- Developed skills in persuasion and proposing ideas.
- Learned conflicts can come from personality mismatch.
- Requires self-awareness and understanding other people’s psychology.
Maintenance mindset:
- Early-stage success requires onboarding, documentation, and training so on-call engineers know where to look.
- Anticipate operational risks, such as:
  - database unavailability
  - SQS queue failures
  - bad proxy configuration (even if configuration is “valid,” traffic can still break)
  - detection and response strategies
- Maintenance burden increases as:
  - people churn
  - code churn concentrates and signals growing complexity
  - changes become harder due to coupling
- Mentions potential future help from AI/LLMs for “detangling” coupled code, but avoids over-optimism.
Mentoring challenge:
- Speaker is good at teaching colleagues.
- Mentoring an intern was difficult balancing:
  - giving enough guidance vs.
  - letting the intern learn without getting stuck or frustrated.
- The intern succeeded highly, though the speaker can’t fully attribute the outcome solely to themselves (other SME help existed).

Speakers / sources featured

Speakers

Unspecified single speaker (the person reflecting on Atlassian layoffs; no name provided)

Sources / technologies explicitly mentioned

Atlassian (employer; layoffs)
HackerRank (coding quiz platform)
Cloudflare (white paper topic: custom domains)
Envoy Proxy
Envoy Control Plane / Sovereign (open-sourced control plane; public Bitbucket repo “for now”)
Open Service Broker (OSB) / OpenAPI
FastAPI, Flask, Python
SQS, DynamoDB
AWS services: Route 53, CloudFormation, Auto Scaling Group, VPC, NLB, ACM, EC2, IAM, S3
HashiCorp Packer
Salt Stack
Rust (authentication sidecar)
CloudFront (DDoS protection noted)