Dev and Run Small Projects on K8S with Jenkins-X

Gerd Aschemann

Demo: Create/Install JX

Want to exercise the demos?

Download the jx binary from:

https://jenkins-x.io/getting-started/install/

Create complete k8s cluster with JX
JX Create Cluster (GKE)
```
jx create cluster gke
```
Or install JX on an existing k8s cluster
JX Install Cluster (GKE)
```
jx install --provider=gke
```

JX is an opinionated command line tool!

About me

Gerd Aschemann
- gerd@aschemann.net
- Twitter: @GerdAschemann / #jenkinsx (@jenkinsxio)
Freelancer: Java, Groovy, Enterprise, CRM, CI/CD, Architecture, DDD, …
Co-Organizer:
- JUG Darmstadt
- Javaland Conference
- Open Source Conference Planner DukeCon

Stickers!

I am really sorry,
I don’t have those nice
black/white/red stickers
But only these
Get them after the show

Agenda

Motivation
Jenkins X
GitOps
Dev + Run

Setting the Stage

Background: Jenkins X

Jenkins-X (JX) enables setup of GitOps based k8s
Includes misc. environments
- Dev (Build, CI/CD) + Run
- Repositories (Maven, Docker, Charts)
- Test environment(s)
- Production environment(s)
Multi Cluster (if desired)
Multi Cloud (if desired)
DevOps Addons (Monitoring, Service Mesh, …)

Background: Helm + Ecosystem

JX is based on Helm
Helm provides an ecosystem with predefined packages
Operators, CRDs, Service Catalogs etc. to complement, provided
- by public market places/repositories
- internally in your organisation

Motivation

Run a complete site (for small or medium sized organizations) on k8s with JX?
→ An Experience Report!
Drop hand crafted servers + deployments
Command and control configurations by GitOps

The JX Promise

JX is Jenkins in and for k8s
JX is all about Automation
JX is (only) a command line tool
Set up k8s
Set up Jenkins
Set up Everything
- Tools
- Repositories
- Processes → Gitops
The CLI just sets up everything via Helm

JX Overview

Jenkins-X + Kubernetes

JX Architecture

JX Architecture: Repositories

Github (External): Source Code Management
Nexus (in k8s): Maven Repository
Docker Registry (in k8s): Docker Images
Chartmuseum (in k8s): Deployment Configurations

Kubernetes Providers

Google Kubernetes Engine (GKE)
local (Docker) machine: Minikube / Minishift
Amazon: Minikube, AWS native, Elastic Kubernetes Service (EKS)
IBM Kubernetes Services (IKS)
Azure Kubernetes Service (AKS)
Oracle Kubernetes Environment (OKE)???
OpenShift (Minikube)
On Premise (e.g. Hetzner Cloud w/ kubespray)

Demo: JX Create/Import App

JX Create Spring/Node/Go/…

jx create spring

Checkout Spring Initializr: https://start.spring.io/

JX Import

# cd $dirWithApp
jx import

Automatically Staged

Change to jx-staging environment automatically merged as PR:

Helm chart requirements

diff --git a/env/requirements.yaml b/env/requirements.yaml
@@ -6,9 +6,12 @@
 dependencies:
...
   name: exposecontroller
   version: 2.3.58
   repository: https://chartmuseum.build.cd.jenkins-x.io
   alias: cleanup
+- name: jx-demo-180906-1131
+  repository: http://jenkins-x-chartmuseum:8080
+  version: 0.0.1

And thereby promoted as new application in the env.

State of the art: Tekton Build

Prow: k8s reliable WebHook handler
- Scalable / High Available
- Configuration stored in Git(hub)
Tekton: k8s going jenkinsless

GitOps

GitOps Overview

GitOps: Processes + Environments

Build + Deploy
Combine
- Infrastructure as Code
- + (Opinionated) Deployment Workflow
Promote
- To Staging (Default/Automatic)
- To Production (on Demand/Manual)
- To Preview (on PR/Branch)
- To more environments if required (LuP, UAT, …)

GitOps Steps (with JX)

Promote deployments through opinionated workflow
Describe formally (e.g., Helm chart)
- Single application
- Environment composition
Cover application as well as configuration
File change request as pull request
Automatically merge (including tests) and roll out
Monitoring + Synchronisation of state
Roll back on demand

Demo: JX Promote

JX Promote (to production)

jx promote --version 0.0.1 --env production

Dev + Run Applications

Sample Application: DukeCon

OSS DukeCon Conference Planer, e.g.,
- Javaland 2019
- ApacheCon NA 2019
Some (Micro) Services

DukeCon Instance Deployment (Full)

DukeCon Instance Deployment (Simple)

Multi Tenant Deployment with JX

Simple
- Single Instance Production Namespace
- Enable different endpoints / filtering via Ingress
- Use elaborated Edge/API Server (Ambassador, Traefik, …)
Extended
- Separate Namespace per Tenant
- Separate Cluster per Tenant
- Seperate Cloud per Tenant
- Arbitrary Mixtures of Cluster/Cloud

Implementation

Import existing Projects to JX (adds Docker + Helm)
Handle Mono-Repositories
Handle Non-Opinionated stuff
Add Edge-Service
Un-Expose Services
Provide Resources, e.g., conference data setup
Compose environment(s) / Tenants
- Javaland / DOAG
- ApacheCon

Import existing projects

JX Import (with a small helper script)

...
# Do not create drafts if they are already there
draft=""
test -d charts && draft="--no-draft=true"

exec jx import \
     --disable-updatebot=true \
     --jenkinsfile Jenkinsxfile \
     --branches 'develop|master|feature/.*|bugfix/.*' \
     ${draft} \
     .

Handle Mono-Repos: Dockerfile

Change generated Dockerfile

diff --git a/Dockerfile b/Dockerfile
index 5b15523..ccd40b8 100644
--- a/Dockerfile
+++ b/Dockerfile
...
-COPY target/*.jar /opt/app.jar
+COPY impl/target/*.jar /opt/app.jar
WORKDIR /opt

Handle Non-Opinionated Stuff

Change Path for Readiness/Liveness Probe

diff --git a/charts/dukecon-server/values.yaml b/charts/dukecon-server/values.yaml
index b3518fa..63aa73c 100755
--- a/charts/dukecon-server/values.yaml
+++ b/charts/dukecon-server/values.yaml
@@ -28,7 +28,7 @@ resources:
   requests:
     cpu: 400m
     memory: 512Mi
-probePath: /actuator/health
+probePath: /health
 livenessProbe:
   initialDelaySeconds: 60
   periodSeconds: 10

Add Edge-Service

Create Apache HTTPD Chart as Proxy

Create Config Map

...
ProxyPassMatch ^/(\w+)/(\d+)/rest/speaker/images/(\w+) http://dukecon-server:80/rest/speaker/images/$3
ProxyPassMatch ^/(\w+)/(\d+)/rest/admin/(.+) http://dukecon-server:80/rest/admin/$3
ProxyPassMatch ^/(\w+)/(\d+)/img/favicon.ico http://dukecon-server:80/img/$1$2/favicon/favicon.ico
ProxyPassMatch ^/(\w+)/(\d+)/rest/feedback/(.+) http://dukecon-feedback:80/rest/feedback/$3
...

Un-Expose Services

We do not want our Services to be directly accessible via Ingres

Drop Expose from Chart’s values.yaml

diff --git a/charts/dukecon-pwa/values.yaml b/charts/dukecon-pwa/values.yaml
index 65b08cb..999873b 100755
--- a/charts/dukecon-pwa/values.yaml
+++ b/charts/dukecon-pwa/values.yaml
@@ -18,7 +18,7 @@ service:
   externalPort: 80
   internalPort: 80
   annotations:
-    fabric8.io/expose: "true"
+    fabric8.io/expose: "false"
     fabric8.io/ingress.annotations: "kubernetes.io/ingress.class: nginx"
 resources:
   limits:

Provide Resources

Make conference configurations versioned

Provide them as Config Maps

apiVersion: v1
kind: ConfigMap
metadata:
    name: {{ template "fullname" . }}
    labels:
        draft: {{ default "draft-app" .Values.draft }}
        chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
data:
    {{ (.Files.Glob "resources/**.yml").AsConfig | nindent 2 }}

Compose Environments / Tenants

Have different types of service compositions (tenant specific?)

Create a Chart with dependencies

requirements.yaml

dependencies:
- name: dukecon
  repository: http://jenkins-x-chartmuseum:8080
  version: 1.0.3
- name: dukecon-admin
  repository: http://jenkins-x-chartmuseum:8080
  version: 1.0.15
- name: dukecon-feedback
  repository: http://jenkins-x-chartmuseum:8080
  version: 1.0.19
- name: dukecon-pwa
  repository: http://jenkins-x-chartmuseum:8080
  version: 1.0.44

Extensions

Solved (Part of JX or addons)
- Let’s encrypt
- Monitoring (Prometheus + Co.)
Work in Progress
- Compositions (Composite DukeCon Chart w/ dependencies:)
- Integrate KeyCloak
- Switch to Tekton-CD
- Elaborated test setups, e.g., Feedback-Server → Needs PWA + Server + Edge
- Use Service Mesh (Istio)

Summary

Setup of Dev + Test environment(s): simple (Kudos to JX): ✔︎
Several small customizations necessary (e.g. un-expose): ✔︎
Setup of Run (Production)
- Of The Shelf Functionality of JX: ✔︎
- Multi Tenant: Target Namespaces ✔︎
- Multi Cluster: not yet tested (promised by JX): ?
- Multi Cloud: not yet tested (promised by JX): ?
- Real World Implementation: Work in Progress (→ Javaland 2020): …

Appendix (Links)

Jenkins-X
GitOps

Links (Jenkins-X)

https://jenkins-x.io (Including many Demos and Videos)
https://medium.com/@jdrawlings/serverless-jenkins-with-jenkins-x-9134cbfe6870 (Serverless Jenkins-X)
Tekton Pipelines
https://github.com/jenkins-x (Source)
Twitter: @JenkinsX
k8s-Slack: https://kubernetes.slack.com (#jenkins-x-dev + #jenkins-x-user)
Google Group (JenkinsCI-Users)

Links (GitOps)

GitOps: A Path to More Self-service IT (https://queue.acm.org/detail.cfm?id=3237207)
https://www.weave.works/technologies/gitops-frequently-asked-questions/
https://www.weave.works/blog/how-to-correctly-handle-db-schemas-during-kubernetes-rollouts
k8s-Slack: https://kubernetes.slack.com (#helm-users + #gitops)
Argo CD: https://github.com/argoproj/argo-cd

Dev and Run Small Projects on K8S with Jenkins-X

Demo: Create/Install JX

About me

Stickers!

Agenda

Setting the Stage

Background: Jenkins X

Background: Helm + Ecosystem

Motivation

The JX Promise

JX Overview

Jenkins-X + Kubernetes

JX Architecture

JX Architecture: Repositories

Kubernetes Providers

Demo: JX Create/Import App

Automatically Staged

State of the art: Tekton Build

GitOps

GitOps Overview

GitOps: Processes + Environments

GitOps Steps (with JX)

Demo: JX Promote

Dev + Run Applications

Sample Application: DukeCon

DukeCon Instance Deployment (Full)

DukeCon Instance Deployment (Simple)

Multi Tenant Deployment with JX

Implementation

Import existing projects

Handle Mono-Repos: Dockerfile

Handle Non-Opinionated Stuff

Add Edge-Service

Un-Expose Services

Provide Resources

Compose Environments / Tenants

Extensions

Summary

Appendix (Links)

Links (Jenkins-X)

Links (GitOps)

Thank You