GitOps by Courtesy of Jenkins-X

Gerd Aschemann

About me

  • Gerd Aschemann

  • Freelancer: Java, Groovy, Enterprise, CRM, CI/CD, Architecture, DDD, …​

  • Co-Organizer:

    • JUG Darmstadt

    • Javaland Conference

    • Open Source Conference Planner DukeCon

Stickers!

  • I am really sorry,
    I don’t have those nice
    black/white/red stickers

  • But only these

  • Get them after the show

Agenda

  1. The Promise!

  2. The Question(s)!!

  3. The Answer(s)!!!

    1. Jenkins-X

    2. GitOps

  4. Features + Roadmap

  5. Tekton Pipelines → JX going Jenkinsless!

  6. Demos incl. The Developer Experience!!!

The JX Promise

  • JX is all about Automation

  • JX is (only) a command line tool

  • Set up k8s

  • Set up Jenkins

  • Set up Everything

    • Tools

    • Repositories

    • Processes → Gitops

JX Overview

Overview

Start into DevOps

intro dev2jx

It is really simple!

intro ops simple

Use a Maven build

intro dev add maven

Maven Build Setup

  • How to write the pom.xml?

  • How to deploy the build artifact to the repository?

  • How to get the artifact to k8s?

  • How to set up a Maven repository?

Use a Docker registry

intro dev add docker

Docker Build Setup

  • How to write the Dockerfile?

  • Or use Maven, Skaffold, Jib, …​???

  • How to deploy Docker image to the registry?

  • How to get the image to k8s?

  • How to set up a Docker registry?

Use a Helm chart

intro dev add helm

Helm/k8s Setup (Test)

  • How to write the deployment configuration?

    • Deployment YAML?

    • Service YAML?

    • Ingress???

    • ConfigMap + Secrets?

    • …​

      → Helm Chart(s)

  • How to set up Chart repository?

  • How to set up Kubernetes?

Integration + Production

  • How to set up Integration or Production?

  • How to cope with

    • multiple (interdependent) applications?

    • different configurations?

    • other infrastructure (DBs, Messaging, UIs, …​)?

    • multiple Kubernetes clusters?

  • What is the driving entity?

a: Jenkins-X + Kubernetes

JX Architecture

Architecture

JX Architecture: Repositories

  • Github (External): Source Code Management

  • Nexus (in k8s): Maven Repository

  • Docker Registry (in k8s): Docker Images

  • Chartmuseum (in k8s): Deployment Configurations

Demo 0: Create JX

JX is an opinionated command line tool!
JX Create Cluster (GKE)
jx create cluster gke

or

JX Install Cluster (GKE)
jx install cluster gke

Kubernetes Providers

  • Google Kubernetes Engine (GKE)

  • local (Docker) machine: Minikube / Minishift

  • Amazon: Minikube, AWS native, Elastic Kubernetes Service (EKS)

  • IBM Kubernetes Services (IKS)

  • Azure Kubernetes Service (AKS)

  • Oracle Kubernetes Environment (OKE)???

  • OpenShift (Minikube)

  • On Premise (e.g. Hetzner Cloud w/ kubespray)

Demo 1: JX Create/Import App

JX Create Spring/Node/Go/…​
jx create spring

or

JX Import
# cd $dirWithApp
jx import
Checkout Spring Initializr: https://start.spring.io/

Automatically Staged

Change to jx-staging environment automatically merged as PR:

Helm chart requirements
diff --git a/env/requirements.yaml b/env/requirements.yaml
@@ -6,9 +6,12 @@
 dependencies:
...
   name: exposecontroller
   version: 2.3.58
   repository: https://chartmuseum.build.cd.jenkins-x.io
   alias: cleanup
+- name: jx-demo-180906-1131
+  repository: http://jenkins-x-chartmuseum:8080
+  version: 0.0.1

And thereby promoted as new application in the env.

b: GitOps

GitOps Overview

GitOps

GitOps: Processes + Environments

  • Build + Deploy

  • Combine

    • Infrastructure as Code

    • + (Opinionated) Deployment Workflow

  • Promote

    • To Staging (Default/Automatic)

    • To Production (on Demand/Manual)

    • To Preview (on PR/Branch)

    • To more environments if required (LuP, UAT, …​)

GitOps Steps

  • Promote deployments through opinionated workflow

  • Describe formally (e.g., Helm chart)

    • Single application

    • Environment composition

  • Cover application as well as configuration

  • File change request as pull request

  • Automatically merge (including tests) and roll out

  • Monitoring + Synchronisation of state

  • Roll back on demand

Demo 2: JX Promote

JX Promote (to production)
jx promote --version 0.0.1 --env production

JX Features/Roadmap

  • Client: OSX + Linux, Windows: In Progress

  • Dev + Build + Deployment: CLI, DevPod, Preview/Pull-Request → GitOps

  • k8s providers: GCP/GKE, AWS/EKS, Azure, MiniShift/MiniKube, OpenShift, Pivotal KS, IBM KS, Digital Ocean, Oracle

  • Tools: Helm, Skaffold, Prow, LetsEncrypt

  • Repositories: Chart Museum, Docker Registry, Nexus, Artifactory (Planned)

  • Issue Trackers: Github/GH Enterprise, Jira, Gitea (In Progress)

  • Multi-Cluster (In Progress)

  • Istio Canaries (In Progress)

Git providers

  • Working:

    • Github/Github Enterprise

    • Bitbucket Cloud

  • In Progress:

    • Gitea

    • Bitbucket Server

    • Gitlab

Build Packs

cf. https://github.com/jenkins-x-buildpacks/jenkins-x-kubernetes/tree/master/packs

  • Working: C++, D, appserver, csharp, custom-jenkins, docker-helm, docker, dropwizard, environment, go, gradle, helm, javascript, jenkins, liberty, maven-java11, maven, php, python, ruby, rust, scala, swift, typescript

  • Planned: Kubeless, .Net

Addons

  • Working: Prometheus, Gitea, Anchore, Theia IDE

  • Planned: Cloud9, Eclipse Che, SonarQube

State of the art: Tekton Build

jx serverless

  • Prow: k8s reliable WebHook handler

    • Scalable / High Available

    • Configuration stored in Git(hub)

  • Tekton: k8s going jenkinsless

JX Overview (last)

Overview

Conclusion

  • CLI based setup: k8s, Jenkins, Nexus, Docker Registry, Chartmuseum

  • Misc. k8s providers

  • Fast evolving (Features, Repositories, Addons)

  • Strong support for GitOps

  • Open source/Community

  • Production ready?

The Developer Experience (Demo D)

intro devpod

Demo D1: JX DevPod / Sync

JX Sync (Run once)
jx sync
JX DevPod (Init / Run 1st)
jx create devpod --reuse --sync -l maven -s demo
# Then in the remote shell
cd $WORK_DIR
mvn spring-boot:run

Demo D2: Access DevPod (Default)

JX Port Forwarding
jx get devpod

→ ascheman-maven-demo maven 1h39m34s Ready

Get Devpod URL
# On localhost
jx open
# or
kubectl get ingress
# -> ascheman-maven-demo-port-8080  http://ascheman-maven-demo-port-8080.jx.35.237.226.80.nip.io
curl http://ascheman-maven-demo-port-8080.jx.35.237.226.80.nip.io/actuator/health

→ {"status":"UP"}

Demo D3: Access Exposed Service

Skaffold Deployment
# In devpod
./watch.sh

  • Builds and deploys application

  • Access via developer Ingress

Get Preview Ingress
# On localhost
kubectl get ingress -n jx-edit-ascheman
curl http://...jx-edit-ascheman...

Demo D4: Dev in DevPod

Add Spring DevTools to Dependencies
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <optional>true</optional>
        </dependency>

  • Restart mvn spring-boot:run

  • Then Open 2nd rsh and let Maven compile → Reload of Java code

JX DevPod Remote Shell / Login (2nd)
jx rsh -d
# Then in the remote shell
cd $WORK_DIR
mvn compile

Demo D5: Extend App

Add DemoController.java
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class DemoController {
    @GetMapping("/")
    public String getGreeting() {
        return "Hello World";
    }
}

Demo D6: Docker Build in DevPod

Docker build (via skaffold)
jx rsh -d
# Then in the remote shell
cd $WORK_DIR
VERSION=xxx skaffold build

→

Build / Push of Docker Image
Starting build...
Building [changeme]...
Sending build context to Docker daemon  17.55MB
Step 1/8 : FROM openjdk:8-jdk-slim
...
Build complete in 416.607605ms
changeme -> 10.31.249.194:5000/ascheman-jx/jx-demo-180811-1200:xxx

Thank You