Migrating From Docker to Kubernetes

Migrating From Docker to Kubernetes

If you already have some containerized infrastructure running with docker-compose, or just plain docker containers that are provisioned by a config management tool, you definitely want to switch to Kubernetes. Here’s how I did it.

The host I have contains in turn several domains as virtual hosts, each with different requirements:

Table 1. Sites

Site	Requirement
ciplogic.com	PHP, MySql
germaniumhq.com	Static
download.ciplogic.com	PHP
get.germaniumhq.com	Static
maven.germaniumhq.com	Static

Now in order to achieve this, I installed a docker container that listened on port 80, and 443, and configured a bunch of virtual hosts. Another container was created for the MySql database, so in the end this was the original setup:

Don’t get me wrong, this is pretty decent but has two major flaws:

1. To add a website, we need to change the configuration of the nginx server by hand.

2. If one of the PHP sites gets compromised, the data for all the other websites becomes available, since I need to have permissions for nginx to access all the websites.

I understand I could create a reverse load balancer in front, and then link the containers to it, by just using docker containers, but that’s a nightmare to create and maintain, esp. since Kubernetes has support for that natively via Ingress objects, and clean separation via namespaces.

So what’s the plan to have a smooth upgrade? Well simple:

1. Install a Kubernetes host on some node,

2. Put every site in its own namespace,

3. Use the same paths like before and use host volume mounts, but each namespace runs its own nginx, (note the hostPath and volumeMounts directly in the pod):

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nginx
     namespace: get-germaniumhq-com
     labels:
       app: get
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: get
     template:
       metadata:
         labels:
           app: get
       spec:
         containers:
         - name: nginx
           image: nginx
           ports:
           - containerPort: 80
           volumeMounts:
           - mountPath: /usr/share/nginx/html/
             name: get-volume
         volumes:
         - name: get-volume
           hostPath:
             path: /opt/www/germaniumhq_get

4. Expose the sites using Ingress objects. This replaces the nginx configuration from before. Beautiful!

   apiVersion: extensions/v1beta1
   kind: Ingress
   metadata:
     name: get.germaniumhq.com
     namespace: get-germaniumhq-com
   spec:
     rules:
     - host: get.germaniumhq.com
       http:
         paths:
         - backend:
             serviceName: nginx
             servicePort: http
           path: /

   Of course, the service it’s a trivial ClusterIP service:

   apiVersion: v1
   kind: Service
   metadata:
     name: nginx
     namespace: get-germaniumhq-com
     labels:
       app: get
   spec:
     type: ClusterIP
     ports:
     - name: http
       port: 80
       protocol: TCP
     selector:
       app: get

5. Switch the DNS to the new server.

I went with microk8s because it’s one snap install away, and ended up with:

Neat!

Full time for the migration? Around 4 hours, the downloading of images was probably 2 of them :).

But now we’re talking.

If one of the containers gets compromised, the data access is severely restricted. Furthermore I get for free all the other goodies that Kubernetes has to offer, such as easier management via kubectl, monitoring and what not.

I still need something to do certificate management, and looking at what’s on helm out there I’m not that impressed, so I’ll probably roll my own.

But that in another blog post :)

Kubernetes rocks!

Article Photo from Pexels: https://www.pexels.com/photo/business-commerce-container-export-379964/