GITOPS ЗА 18 МИНУТ

Summary: GitOps for Real-World Deployment Failures

The video explains GitOps as a solution to real-world deployment failures caused by configuration drift and manual changes.

Core story / problem statement

• A typical CI/CD rollout happens (button in Jenkins, pipeline turns green, deployment rolls out).
• The service crashes.
• Manual investigation finds that someone previously edited a Kubernetes ConfigMap directly in the cluster (changing DB address/current).
• The app worked for months because the live cluster config differed from what was stored in Git.
• On redeploy, Kubernetes pods were recreated and pulled the old (broken) config from Git, causing the outage.
• Claim: GitOps guarantees code == what actually runs.

---

The “Push Model” (Traditional CD) and Why It’s Risky

The video contrasts GitOps with a common “push” approach where CI/CD tools directly apply manifests to the cluster:

• Jenkins/GitLab CI (and other systems) run tests/builds and then execute cluster changes (e.g., kubectl apply / helm upgrade).

Key issues highlighted:

1. Security risk

   • Jenkins needs cluster access via kubeconfig with admin-like permissions.
   • Jenkins is large and plugin-heavy, making it a potential attack target.

2. Network/perimeter risk

   • Often requires opening ports (commonly 6443) and exposing the API server or adding VPN complexity.

3. Operational blind spot (“fire and forget”)

   • A pipeline can be green even if the app fails after deployment (typos, missing secrets, wrong images, etc.).
   • CI/CD doesn’t necessarily monitor real runtime health—only command success.

---

GitOps Approach (Main Principles)

Key principle: Git is the single source of truth

• Manifests are stored declaratively in Git.
• The cluster should continuously converge to match Git.
• Manual changes in the cluster are treated as incidents (drift) and should be corrected.

---

How GitOps Is Implemented (Agent/Controller Behavior)

Instead of external systems pushing changes, a GitOps controller runs inside the cluster.

Tools mentioned

• Argo CD
• Flux

Controller behavior

• Uses a service account inside the cluster (not admin kubeconfig).
• Polls or syncs from the Git repository.
• Runs a continuous reconcile loop (default scan mentioned: every ~3 minutes).
• Compares:
  • Desired state from Git
  • Actual state in Kubernetes
• If drift is detected, it re-applies the correct manifests (self-healing).

Benefit

• Reduces the need for inbound connectivity.
• The cluster can remain a “black box” while still being managed safely.

---

Drift Detection + Self-Healing (Why It Matters)

The video emphasizes:

• Continuous verification detects unauthorized changes (e.g., someone deletes/changes a controller).
• With self-healing enabled, Argo CD/Flux restores the cluster to the Git-defined state quickly—often seconds after drift appears.

Governance/auditing effect

• Teams can’t rely on “cowboy fixes” in production.
• Changes must go through the Git workflow: branches → pull request → review → merge.
• Commit history becomes a legally binding audit log (who changed what, and when).

---

GitOps Workflow in CI/CD (“Last Mile” Replacement)

The video explains division of responsibilities:

• CI still builds artifacts (Docker images) and runs tests.
• GitOps replaces only the deployment step (the “last mile”), not the build.

Example workflow described

• Split repositories:

  1. Application repo (code)
  2. Infrastructure repo (Kubernetes manifests/values)

• CI pipeline:

  • Runs unit tests/linters.
  • Builds and pushes Docker images to a registry.
  • Updates the infrastructure repo (e.g., changes the image tag/version in Helm values/manifests).
  • Commits that change back to the infrastructure Git repo.

• Argo CD notices the new commit and performs rollout:

  • Detects image version change (e.g., V24 → V25).
  • Updates Kubernetes by reconciling desired state.
  • Mentions waiting for readiness before terminating old pods.

• Deployment history is recoverable by checking commit history in the infrastructure repo.

---

Handling Secrets (The Hardest Part)

The video calls out a key GitOps challenge: Git-based configs can accidentally expose credentials.

• Kubernetes “Secrets” are not true encryption by default (subtly described as Base64 encoding), so storing sensitive values in Git is risky.

Two paths

1. Store secrets encrypted in Git

   • Tools named: Bitnami Sealed Secrets, SOPS
   • Secrets are encrypted with the cluster’s public key; controller decrypts using private key.
   • Downsides:
     • Lose private key → secrets can’t be restored
     • Key rotation requires re-encrypting many files

2. Do not store secret values in Git; store references

   • Tool named: External Secrets Operator
   • Git contains manifests pointing to secrets in a secure store (e.g., Vault / “HashiCorp vault or similar”).
   • Cluster operator fetches real values and creates Kubernetes Secrets.
   • Advantages emphasized:
     • Only pointers in Git (safer if the repo is leaked)
     • Easier secret rotation without Git commits
     • Better audit trail via secret store logs

---

Tool Comparison: Argo CD vs Flux

Argo CD

• Emphasizes a rich GUI/visualization (“map of the cluster”)
• Dependency/deployment tree visualization
• Interactive actions (e.g., sync/restart) with status colors
• Recommended for newcomers because it speeds debugging and understanding

Flux

• Emphasizes “native GitOps” with no UI (mostly CLI/config)
• “Set it up and forget it”
• Lighter resource usage
• Strong Helm charts/dependency handling
• More suited for “hardcore platform engineers” who prefer automation over clicking

---

Rollbacks Are Simpler in GitOps

Traditional rollback pain

• Find the stable Jenkins build/artifact and redeploy it.
• Can be time-consuming to locate correct artifacts/tags.

GitOps rollback advantage

• Since cluster state mirrors Git, rollback = revert Git to an earlier commit.
• Argo CD applies the previous config automatically.
• Claimed result: rollbacks happen in seconds, and the incident record remains intact in Git history.

---

Final Claimed Benefits (3)

1. Stability/consistency

   • Environments are identical because they derive from the same Git state.

2. Security

   • Fewer/zero inbound access needs; cluster treated as a black box.

3. Recovery

   • If the cluster fails, a new one can be created and reconnected to the same Git repo to restore infrastructure and apps automatically.
   • Time mentioned: ~15 minutes via Argo CD.

---

Main Speakers / Sources

• Speaker/source: The narrator from “Just DevOps” (channel intro: “You are on the Just Devоops channel.”).
• Primary technical sources/tools mentioned: Argo CD and Flux, plus External Secrets Operator, Sealed Secrets, and SOPS.