Kubernetes is a popular container orchestration platform. As a developer understanding the environment in which your application is going to run is important since this can help you use available services of the platform and fix issues.
There are several options to run Kubernetes locally to get some experience with Kubernetes as developer. For example Minikube, MicroK8s and MiniShift. These options however are not representative for a real environment. They for example usually do not have master and slave nodes. Running locally requires quite different configuration compared to running multiple nodes on different machines. Think for example about how to deal with storage and a container registry which you want to share. Installing a full blown environment requires a lot of work and resources. Using a cloud service usually is not free and you usually have less to no control over the environment Kubernetes is running in.
In this blog I’ll describe a ‘middle way’. Get an easy to manage small multi node Kubernetes environment running in different VMs. You can use this environment for example to learn what the challenges of clusters are and how to deal with them efficiently.
It uses the work done here with some minor additions to get a dashboard ready.
Getting the host ready
As host OS I used Cent OS 7 (on bare metal). CentOS 8 introduces some major changes such as Podman instead of Docker so I did not want to take any risks and decided to stick with this commonly used open source OS compiled from Red Hat sources. I do recommend sticking to a single partition for a local development environment to make it more easy for yourself. Also I used a minimal desktop environment with administrative tools.
Also create a user which can do sudo to execute various commands in the following steps.
Install QEMU/KVM + libvirt
We are going to use QEMU/KVM and access it through libvirt. Why? Because I want to approach bare metal performance as much as I can and QEMU/KVM does a good job at that. See for example this performance comparison of bare metal vs KVM vs Virtualbox. KVM greatly outperforms Virtualbox and approaches bare metal speeds in quite some tests. I do like the Virtualbox GUI though but I can live with the Virtual Machine Manager.
The following will do the trick on CentOS 7
Install Vagrant and required plugins
Vagrant is used to create the virtual machines for the master and nodes. Vagrant can easily be installed from here. It even has a CentOS specific RPM which is nice.
With Vagrant I’m going to use two plugins. vagrant-libvirt and vagrant-sshfs. The first plugin allows vagrant to manage QEMU/KVM VMs through libvirt. The second plugin will be used for shared folders. Why sshfs? Mainly because libvirt shared folder alternatives such as NFS and 9p were more difficult to set-up and I wanted to be able to provide the same shared storage to all VMs.
First install kubectl on the host. This is described in detail here.
Create the VMs
Execute the following under a normal user (you also installed the Vagrant plugins under this user).
This process will take a while. You can follow the progress by looking in the Virtual Machine Manager and in the console.
The command will also ask a couple of times for the user password. Be ready to input this since if you wait too long, it will timeout and the build will fail. If it fails, clean up using
If at the end you only see a single node, you can do the following to create a second node
The Makefile has a lot of other easy to use commands and parameters. Read the documentation here
You can check if the nodes are up and kubectl is configured correctly by:
We now have a master and two nodes running.
The environment does not have an out of the box dashboard like OKD (open source OpenShift). Even though the make scripts allow you to add the dashboard during the creation of the nodes, I prefer to this afterwards so I know what I’m doing.
Now you have a dashboard but no user who can browse resources. In order to give an admin user the required privileges, I did the following (based on this).
Now you can obtain a token by:
Start a local proxy:
Now you can use the token to login to:
and open the dashboard
This environment is not done yet.
- Every node uses its own local registry. The Makefile used does provide ways to load the same image to the different registries but what I actually want is all the nodes to use the same registry.
- There is no shared PersistentVolume available within the Kubernetes environment which can be used. Shared storage. Preparations for that have been made though since /shared in every VM is mounted to the same host folder.
- We have not installed anything in the environment yet but a small dashboard. I want to have Jenkins running inside and be able to deploy applications.
Still some work to be done so stay tuned!