GitOps w/ or w/o Kubernetes

qr code talk gitops nok8s guug hfg 2022

Gerd Aschemann <gerd@aschemann.net>

About me

Live demo (TL;DR) - slides are mostly backup

Visit https://github.com/ascheman/gitops-demo-deployment if you want to actively follow the demo.
gitops branching start.dio

Questions

  • Who is using Continuous Integration (CI)?

  • Who is using Continuous Delivery (CD)?

  • Wer is using Continuous Deployment (CD as well?)?

  • Who is using IaC?

  • Who is using Ansible/Puppet/Chef/Saltstack?

  • Who is using Terraform?

  • Who is using Kubernetes (Manifests)?

  • Who is using Helm, Kustomize, …​?

  • Who is using a k8s GitOps operator (ArgoCD, Flux, …​)?

  • Who is using GitOps outside of k8s (e.g., by https://www.runatlantis.io/guide/)

CI vs. CD

ci vs cd simple
Figure 1. Continuous Integration → Continuous Delivery

The Problem

How can we control

  • (initial/continuous) deployment of a certain artefact (version) to a target environment?

  • that the deployed version remains active in the target environment?

Towards a solution

ci vs cd iac
Figure 2. Add Infrastructure as Code (IaC)

Getting Closer

ci vs cd iac artefact
Figure 3. Let IaC refer to artefact

Deployment items (~ Artefacts)

  • Software delivery items are called artefact

  • Software deployments consist of

    • Artefacts, e.g., Docker Image, Java jar/war, RPM package, AMI, …​

    • Configuration settings, e.g., DB connection string + credentials, Service URL, …​

Artefact coordinates

Artefacts are described by their coordinates, e.g.

  • Repository/Registry (URL)

  • Namespace / Name, e.g., Maven/Gradle Group/Artefact,

  • Version(s): change over time

    → Similar to Configuration(s)

Updated Problem

How can we control

  • the (initial) deployment of a certain artefact (version) configuration to a target environment?

  • that the deployed version configuration remains active in the target environment?

Configuration and State

  • A particular configuration can also be treated as state

  • Use configuration and state synonymously

  • Distinguish desired state and actual state

How to describe state

State (configurations) can be defined

  • Declaratively (supports stateful management), e.g.,

    • Kubernetes (Manifest, Helm, Helmfile, …​)

    • Terraform

    • Puppet, …​

  • Imperatively (make state management harder), e.g.,

    • Kustomize (k8s)

    • Ansible

    • Cloud SDKs (?)

(Technical) Solution(s)

Maintain State Changes

  • Kubernetes (k8s): Lucky you!

  • Ansible:

  • Terraform: Run (terraform plan +) terraform apply

Maintain Durability

→ Out-of-scope for this presentation

  • k8s: Use (Re-) Conciliation Operator

  • Ansible:

    • Re-Apply Playbook(s)

    • Use Ansible Tower (aka. Ansible Automation Platform)

  • Terraform: Re-Run terraform plan + apply (???)

Control Change

On higher level the problem remains

  • How can we control state (changes)?

  • How do we ensure a certain configuration is applied to an environment?

  • How do we know which state is applied?

  • Who is in control of the change?

    • What will be changed?

    • Review Change?

    • Who approves the Change?

GitOps to the Rescue

  • Maintain each configuration (state) in a Git repository/branch

  • Automatically apply changes by an agent (here: pipeline)

  • Use Git workflows to control change (i.e., use pull-requests)

→ GitOps

GitOps State by Repository

Configure stage(s)/environments in separate repositories.

gitops state by repository
Figure 4. State by Repository

GitOps State by Branch

Configure stage(s)/environments in separate branches.

gitops state by branch
Figure 5. State by Branch

GitOps Origins

  • GitOps,

  • Principles

    • Describe (entire) system declarativly

    • Maintain desired system state in Git

    • Automatically apply (approved) state changes

    • Ensure correctness / durability by software agents (k8s: operators)

Demonstration(s)

Sample (overview)

gitops branching start.dio

Apply new Version (overview)

gitops branching apply pr.dio

Apply new Version (step-by-step)

Step by step preparation of PR
  environment="test"
  release="release/v2"
  git checkout -b prepare/${environment} origin/perform/${environment} # (1)
  git merge --ff -m"Switch to ${release}" ${release} # (2)
  git push -fu origin # (3)
1Start with the current state (i.e., perform/ branch)
2Merge state change (i.e., release/ branch)
3Push it to Github and let the action prepare the change

Then check Github Actions and PRs

Roll back Version (overview)

gitops branching rollback.dio

Roll back Version (step-by-step)

Step by step rollback
  environment="test"
  rollback=... # (1)
  git checkout -b prepare/${environment} origin/perform/${environment} # (2)
  git revert -m 'Roll back to previous version' ${rollback} # (3)
  git push -fu origin # (4)
1Find the state change to revert
2Start with the current state (i.e., perform/ branch)
3Roll back the change
4Push it to Github and let the action(s) do the rest

Check Github Actions and PRs

Development / New environment

gitops branching init.dio
Create new environment
  start=release/v2
  environment=guug-hfg-2022
  git checkout ${start} # <1>
  git push -f origin HEAD:init/${environment} # <2>
1Checkout a suitable starting point
2Push it to an init/ branch on Github

Watch Github Actions initializing the new environment

Development / Auto-perform/-destroy (overview)

gitops branching develop.dio

Development / Auto-perform (steps)

Apply current state to environment (full-automatic mode)
  environment=guug-hfg-2022
  current=$(git branch --show-current)
  git fetch --all --prune
  git checkout -b auto-perform/${environment} origin/perform/${environment}
  git merge --ff -m"Auto-Perform changes from ${current}" ${current}
  git push -fu origin

Watch Github Actions applying the change

Development / Auto-destroy (steps)

Destroy environment (full-automatic mode)
  environment=guug-hfg-2022
  git pull -f
  git checkout perform/${environment}
  git push -f origin HEAD:auto-destroy/${environment}

Watch Github Actions destroying the environment

Other approaches: Atlantis

  • CD-Application (C/S) for Terraform

  • Watch for PRs (Github, Gitlab, Bitbucket, Azure Devops(?), …​)

  • Control by ChatOps (comment PRs)

Pros and Cons

  • 😊 Separation of concerns (CI vs. CD)

  • 😐 Product vs. OSS-Project

  • 😐 No Pipeline engine needed

  • 😐 Reconciliation not covered (as well)

  • ☚ī¸ Repository per Environment (very opinionated)

  • ☚ī¸ Terraform only

  • ☚ī¸ Lacks many use-cases

Not covered (yet)

  • Reconcilement (neither here, nor Atlantis)

  • Access Management

    • No direct grants to target envs necessary

    • Control access by repository/branch permissions

  • Complex use cases, e.g.,

    • release bundling

    • inhomogeneous resources (for instance: k8s + TF)

  • Housekeeping

  • Extensions, e.g., chatbot

Summary

  • GitOps can solve typical control challenges

    • Persist environment configurations to Git

    • Enable change management by following well-known Git flows

    • Include typical quality assurance, e.g., 4 eyes principle/review

  • Run easily on state based configurations (Terraform, k8s)

  • Stateless configurations are subject to proof of concept (Ansible Tower)

Thank You

qr code talk gitops nok8s guug hfg 2022

Links