Creating a re-usable Vagrant Box from an existing VM with Ubuntu and k3s (with the Kubernetes Dashboard) and adding mysql, using Vagrant and Oracle VirtualBox

Marc Lameriks

In a previous article, I shared with you the steps I took, to get k3s installed (with the Kubernetes Dashboard) on top of an Ubuntu guest Operating System within an Oracle VirtualBox appliance, with the help of Vagrant.
[ https://technology.amis.nl/2020/01/15/rapidly-spinning-up-a-vm-with-ubuntu-and-k3s-with-the-kubernetes-dashboard-on-my-windows-laptop-using-vagrant-and-oracle-virtualbox/]

For training and demo purposes, I wanted to add mysql to my k3s demo environment and in order to test the environment, I also wanted to include the book service (RESTful Web Service Spring Boot application) that I created earlier and ran in Minikube.
[https://technology.amis.nl/2019/03/05/using-a-restful-web-service-spring-boot-application-in-minikube-together-with-an-external-dockerized-mysql-database/]

I also wanted to create my own prepackaged Vagrant Box from an existing virtual machine (a re-usable box).
[https://www.vagrantup.com/docs/boxes/base.html]
[https://scotch.io/tutorials/how-to-create-a-vagrant-base-box-from-an-existing-one]

In this article I will share with you the steps I took to get it all working. For example, I had to get my Docker built images working with the containerd container runtime in k3s.

In my existing demo environment, created when I wrote a previous article , I changed the content of the Vagrantfile to:
[https://technology.amis.nl/2020/01/15/rapidly-spinning-up-a-vm-with-ubuntu-and-k3s-with-the-kubernetes-dashboard-on-my-windows-laptop-using-vagrant-and-oracle-virtualbox/]

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/bionic64"
  
  config.vm.define "ubuntu_k3s" do |ubuntu_k3s|
  
    config.vm.network "forwarded_port",
      guest: 8001,
      host:  8001,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9010,
      host:  9010,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9020,
      host:  9020,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9110,
      host:  9110,
      auto_correct: true
      
    config.vm.provider "virtualbox" do |vb|
        vb.name = "Ubuntu k3s"
        vb.memory = "8192"
        vb.cpus = "1"
        
      args = []
      config.vm.provision "k3s shell script", type: "shell",
          path: "scripts/k3s.sh",
          args: args
          
      args = []
      config.vm.provision "helm shell script", type: "shell",
          path: "scripts/helm.sh",
          args: args
          
      args = []
      config.vm.provision "dashboard shell script", type: "shell",
          path: "scripts/dashboard.sh",
          args: args
          
    end
    
  end

end

Here, you can see I added the extra ports needed for the book service (RESTful Web Service Spring Boot application).

From the subdirectory named env on my Windows laptop, I opened a Windows Command Prompt (cmd) and typed: vagrant box update

This command updates the box for the current Vagrant environment if there are updates available. The command can also update a specific box (outside of an active Vagrant environment), by specifying the –box flag.
[https://www.vagrantup.com/docs/cli/box.html#box-update]

With the following output:


==> ubuntu_k3s: Checking for updates to ‘ubuntu/bionic64’
    ubuntu_k3s: Latest installed version: 20200107.0.0
    ubuntu_k3s: Version constraints:
    ubuntu_k3s: Provider: virtualbox
==> ubuntu_k3s: Updating ‘ubuntu/bionic64’ with provider ‘virtualbox’ from version
==> ubuntu_k3s: ‘20200107.0.0’ to ‘20200425.0.0’…
==> ubuntu_k3s: Loading metadata for box ‘https://vagrantcloud.com/ubuntu/bionic64’
==> ubuntu_k3s: Adding box ‘ubuntu/bionic64’ (v20200425.0.0) for provider: virtualbox
    ubuntu_k3s: Downloading: https://vagrantcloud.com/ubuntu/boxes/bionic64/versions/20200425.0.0/providers/virtualbox.box
    ubuntu_k3s: Download redirected to host: cloud-images.ubuntu.com
    ubuntu_k3s:
==> ubuntu_k3s: Successfully added box ‘ubuntu/bionic64’ (v20200425.0.0) for ‘virtualbox’!

From the subdirectory named env on my Windows laptop, I opened a Windows Command Prompt (cmd) and typed: vagrant up

This command creates and configures guest machines according to your Vagrantfile.
[https://www.vagrantup.com/docs/cli/up.html]

With the following output (only showing the part about dashboard):


    ubuntu_k3s: **** Begin preparing dashboard
    ubuntu_k3s: **** Install Kubernetes Dashboard
    ubuntu_k3s: namespace/kubernetes-dashboard created
    ubuntu_k3s: serviceaccount/kubernetes-dashboard created
    ubuntu_k3s: service/kubernetes-dashboard created
    ubuntu_k3s: secret/kubernetes-dashboard-certs created
    ubuntu_k3s: secret/kubernetes-dashboard-csrf created
    ubuntu_k3s: secret/kubernetes-dashboard-key-holder created
    ubuntu_k3s: configmap/kubernetes-dashboard-settings created
    ubuntu_k3s: role.rbac.authorization.k8s.io/kubernetes-dashboard created
    ubuntu_k3s: clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
    ubuntu_k3s: rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
    ubuntu_k3s: clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
    ubuntu_k3s: deployment.apps/kubernetes-dashboard created
    ubuntu_k3s: service/dashboard-metrics-scraper created
    ubuntu_k3s: deployment.apps/dashboard-metrics-scraper created
    ubuntu_k3s: **** Create Helm chart
    ubuntu_k3s: Creating k3s-chart
    ubuntu_k3s: **** Install Helm chart k3s-chart
    ubuntu_k3s: NAME: k3s-release
    ubuntu_k3s: LAST DEPLOYED: Sun Apr 26 13:30:16 2020
    ubuntu_k3s: NAMESPACE: default
    ubuntu_k3s: STATUS: deployed
    ubuntu_k3s: REVISION: 1
    ubuntu_k3s: TEST SUITE: None
    ubuntu_k3s: **** Waiting 30 seconds …
    ubuntu_k3s: **** List helm releases
    ubuntu_k3s: NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
    ubuntu_k3s: k3s-release     default         1               2020-04-26 13:30:16.309325567 +0000 UTC deployed        k3s-chart-0.1.0 1.16.0
    ubuntu_k3s: **** List secrets with namespace kubernetes-dashboard
    ubuntu_k3s: NAME
    ubuntu_k3s:
    ubuntu_k3s:
    ubuntu_k3s:
    ubuntu_k3s:
    ubuntu_k3s: TYPE
    ubuntu_k3s:
    ubuntu_k3s:
    ubuntu_k3s:
    ubuntu_k3s:
    ubuntu_k3s:   DATA   AGE
    ubuntu_k3s: kubernetes-dashboard-token-kdq6q   kubernetes.io/service-account-token   3      35s
    ubuntu_k3s: default-token-mdb9j                kubernetes.io/service-account-token   3      35s
    ubuntu_k3s: kubernetes-dashboard-certs         Opaque                                0      35s
    ubuntu_k3s: admin-user-token-xl9zz             kubernetes.io/service-account-token   3      32s
    ubuntu_k3s: kubernetes-dashboard-csrf          Opaque                                1      35s
    ubuntu_k3s: kubernetes-dashboard-key-holder    Opaque                                2      35s
    ubuntu_k3s: **** Describe secret with namespace kubernetes-dashboard
    ubuntu_k3s: Name:         admin-user-token-xl9zz
    ubuntu_k3s: Namespace:    kubernetes-dashboard
    ubuntu_k3s: Labels:       <none>
    ubuntu_k3s: Annotations:  kubernetes.io/service-account.name: admin-user
    ubuntu_k3s:               kubernetes.io/service-account.uid: d053ea00-f5c6-4091-9383-19a83dcfc48d
    ubuntu_k3s:
    ubuntu_k3s: Type:  kubernetes.io/service-account-token
    ubuntu_k3s:
    ubuntu_k3s: Data
    ubuntu_k3s: ====
    ubuntu_k3s: ca.crt:     526 bytes
    ubuntu_k3s: namespace:  20 bytes
    ubuntu_k3s: token:      
eyJhbGciOiJSUzI1NiIsImtpZCI6IkVyWGhzYmU5N2NNblVkUG5aMnZBNTlKYWtyaFZ6OHVCaG51ckZoNGhqMG8ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLXhsOXp6Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJkMDUzZWEwMC1mNWM2LTQwOTEtOTM4My0xOWE4M2RjZmM0OGQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.a2npBwkMqXBNFkB_Fdq0NM4EH3e1KFuilp9x7CYf30TwG8Ne5b-0mwMOp5jC-JtR5Q_RSu9bO_2OxrafE-QQ6IEPAck0veOKoj8-tXN97VnSTn-zUriXpHcyHyOjDhS7ac7Qcrtp9Qgk_bdQUFzV79_BGvPFAcIAOMvbaRZE0KFaTYbhNN6zOwJymZS7MTOncp57aLvAdLvEqTX258RtYgHjT-ro0ajf8o8sTdeeAhuMAIhRCJHmLYzXx_cXCke_nJx0eEYmmqajIviUR77Lcr0psfaksm45WjAK_Mt1Y-tkzEIA407O6uk_hf_W-fxqhB-w5kypEVUOeON_V9b_iQ
    ubuntu_k3s: **** End preparing dashboard
In the Web Browser on my Windows laptop, I started the Kubernetes Dashboard (http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy) and I entered the value for the token (as can be seen above) and clicked on button “Sign in”.

The Kubernetes Dashboard was opened with the default namespace selected.

So far so good. This was working. Time to make a re-usable box.

Making re-usable box my_ubuntu_k3s

On my Windows laptop, in the env directory I created a directory myboxes with a subdirectory my_ubuntu_k3s and a subdirectory my_ubuntu_k3s_mysql.

I copied the current version of the Vagrantfile to the directory myboxes/my_ubuntu_k3s.

Next, I typed the following command on the Windows Command Prompt (cmd):

vagrant package –output my_ubuntu_k3s.box

This command packages a currently running VirtualBox or Hyper-V environment into a re-usable box.
–output NAME – The resulting package will be saved as NAME. By default, it will be saved as package.box.
[https://www.vagrantup.com/docs/cli/package.html]

With the following output:


==> ubuntu_k3s: Attempting graceful shutdown of VM…
==> ubuntu_k3s: Forcing shutdown of VM…
==> ubuntu_k3s: Clearing any previously set forwarded ports…
==> ubuntu_k3s: Exporting VM…
==> ubuntu_k3s: Compressing package to: C:/My/AMIS/env/my_ubuntu_k3s.box

I moved my_ubuntu_k3s.box to the directory myboxes/my_ubuntu_k3s.

In Oracle VirtualBox I could see that the VM was Powered Off.

Next, I changed the content of the Vagrantfile to use the re-usable box:
[in bold, I highlighted the changes]

Vagrant.configure("2") do |config|
  config.vm.box = "my_ubuntu_k3s"
  
  config.vm.define "ubuntu_k3s_mysql" do |ubuntu_k3s_mysql|
  
    config.vm.network "forwarded_port",
      guest: 8001,
      host:  8001,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9010,
      host:  9010,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9020,
      host:  9020,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9110,
      host:  9110,
      auto_correct: true
      
    config.vm.provider "virtualbox" do |vb|
        vb.name = "Ubuntu k3s mysql"
        vb.memory = "8192"
        vb.cpus = "1"

    end
    
  end

end

From the subdirectory named env on my Windows laptop, I opened a Windows Command Prompt (cmd) and typed: vagrant up

With the following output:


Bringing machine ‘ubuntu_k3s_mysql’ up with ‘virtualbox’ provider…
==> ubuntu_k3s_mysql: Box ‘my_ubuntu_k3s’ could not be found. Attempting to find and install…
    ubuntu_k3s_mysql: Box Provider: virtualbox
    ubuntu_k3s_mysql: Box Version: >= 0
==> ubuntu_k3s_mysql: Box file was not detected as metadata. Adding it directly…
==> ubuntu_k3s_mysql: Adding box ‘my_ubuntu_k3s’ (v0) for provider: virtualbox
    ubuntu_k3s_mysql: Downloading: my_ubuntu_k3s
    ubuntu_k3s_mysql:
An error occurred while downloading the remote file. The error
message, if any, is reproduced below. Please fix this error and try
again.


Couldn’t open file C:/My/AMIS/env/my_ubuntu_k3s

This error could be expected because up till now I only created the box file.

Next, I typed the following command on the Windows Command Prompt (cmd):

vagrant box list

This command lists all the boxes that are installed into Vagrant.
[https://www.vagrantup.com/docs/cli/box.html#box-list]

With the following output:


bento/ubuntu-18.04  (virtualbox, 201808.24.0)
generic/alpine310   (virtualbox, 2.0.0)
hashicorp/precise64 (virtualbox, 1.1.0)
ol7-latest          (virtualbox, 0)
ubuntu/bionic64     (virtualbox, 20190814.0.0)
ubuntu/bionic64     (virtualbox, 20191218.0.0)
ubuntu/bionic64     (virtualbox, 20200107.0.0)
ubuntu/bionic64     (virtualbox, 20200124.0.0)
ubuntu/bionic64     (virtualbox, 20200402.0.0)
ubuntu/bionic64     (virtualbox, 20200407.0.0)
ubuntu/bionic64     (virtualbox, 20200416.0.0)
ubuntu/bionic64     (virtualbox, 20200425.0.0)
ubuntu/trusty64     (virtualbox, 20181103.0.0)
ubuntu/xenial64     (virtualbox, 20181223.0.0)
ubuntu/xenial64     (virtualbox, 20190101.0.0)
ubuntu/xenial64     (virtualbox, 20190109.0.0)
ubuntu/xenial64     (virtualbox, 20190115.0.0)
ubuntu/xenial64     (virtualbox, 20190118.0.0)
ubuntu/xenial64     (virtualbox, 20190123.0.0)
ubuntu/xenial64     (virtualbox, 20190123.0.1)
ubuntu/xenial64     (virtualbox, 20190215.0.0)

In order to add my own box, I typed the following command on the Windows Command Prompt (cmd):

vagrant box add –name my_ubuntu_k3s myboxes/my_ubuntu_k3s/my_ubuntu_k3s.box

This adds a box with the given address to Vagrant.
[https://www.vagrantup.com/docs/cli/box.html#box-add]

With the following output:

==> box: Box file was not detected as metadata. Adding it directly…
==> box: Adding box ‘my_ubuntu_k3s’ (v0) for provider:
box: Unpacking necessary files from: file://C:/My/AMIS/env/myboxes/my_ubuntu_k3s/my_ubuntu_k3s.box
box:
==> box: Successfully added box ‘my_ubuntu_k3s’ (v0) for ‘virtualbox’!

Next, again I typed the following command on the Windows Command Prompt (cmd):

vagrant box list

With the following output:
[in bold, I highlighted the changes]


bento/ubuntu-18.04  (virtualbox, 201808.24.0)
generic/alpine310   (virtualbox, 2.0.0)
hashicorp/precise64 (virtualbox, 1.1.0)
my_ubuntu_k3s       (virtualbox, 0)
ol7-latest          (virtualbox, 0)
ubuntu/bionic64     (virtualbox, 20190814.0.0)
ubuntu/bionic64     (virtualbox, 20191218.0.0)
ubuntu/bionic64     (virtualbox, 20200107.0.0)
ubuntu/bionic64     (virtualbox, 20200124.0.0)
ubuntu/bionic64     (virtualbox, 20200402.0.0)
ubuntu/bionic64     (virtualbox, 20200407.0.0)
ubuntu/bionic64     (virtualbox, 20200416.0.0)
ubuntu/bionic64     (virtualbox, 20200425.0.0)
ubuntu/trusty64     (virtualbox, 20181103.0.0)
ubuntu/xenial64     (virtualbox, 20181223.0.0)
ubuntu/xenial64     (virtualbox, 20190101.0.0)
ubuntu/xenial64     (virtualbox, 20190109.0.0)
ubuntu/xenial64     (virtualbox, 20190115.0.0)
ubuntu/xenial64     (virtualbox, 20190118.0.0)
ubuntu/xenial64     (virtualbox, 20190123.0.0)
ubuntu/xenial64     (virtualbox, 20190123.0.1)
ubuntu/xenial64     (virtualbox, 20190215.0.0)

Again, from the subdirectory named env on my Windows laptop, I opened a Windows Command Prompt (cmd) and typed: vagrant up

With the following output:


Bringing machine ‘ubuntu_k3s_mysql’ up with ‘virtualbox’ provider…
==> ubuntu_k3s_mysql: Importing base box ‘my_ubuntu_k3s’…
==> ubuntu_k3s_mysql: Matching MAC address for NAT networking…
==> ubuntu_k3s_mysql: Setting the name of the VM: Ubuntu k3s mysql
==> ubuntu_k3s_mysql: Clearing any previously set network interfaces…
==> ubuntu_k3s_mysql: Preparing network interfaces based on configuration…
    ubuntu_k3s_mysql: Adapter 1: nat
==> ubuntu_k3s_mysql: Forwarding ports…
    ubuntu_k3s_mysql: 8001 (guest) => 8001 (host) (adapter 1)
    ubuntu_k3s_mysql: 9010 (guest) => 9010 (host) (adapter 1)
    ubuntu_k3s_mysql: 9020 (guest) => 9020 (host) (adapter 1)
    ubuntu_k3s_mysql: 9110 (guest) => 9110 (host) (adapter 1)
    ubuntu_k3s_mysql: 22 (guest) => 2222 (host) (adapter 1)
==> ubuntu_k3s_mysql: Running ‘pre-boot’ VM customizations…
==> ubuntu_k3s_mysql: Booting VM…
==> ubuntu_k3s_mysql: Waiting for machine to boot. This may take a few minutes…
    ubuntu_k3s_mysql: SSH address: 127.0.0.1:2222
    ubuntu_k3s_mysql: SSH username: vagrant
    ubuntu_k3s_mysql: SSH auth method: private key
==> ubuntu_k3s_mysql: Machine booted and ready!
[ubuntu_k3s_mysql] GuestAdditions 6.0.20 running — OK.
==> ubuntu_k3s_mysql: Checking for guest additions in VM…
==> ubuntu_k3s_mysql: Mounting shared folders…
    ubuntu_k3s_mysql: /vagrant => C:/My/AMIS/env

So, creating an Oracle VirtualBox appliance based on my re-usable box my_ubuntu_k3s worked.

Adding mysql to the appliance

Next step is to add mysql to the appliance. Of course, I already had a script in please to add mysql, because of a previous article I wrote.
[https://technology.amis.nl/2019/03/05/using-a-restful-web-service-spring-boot-application-in-minikube-together-with-an-external-dockerized-mysql-database/]

That script uses a particular namespace, namely: nl-amis-testing. So first I tried another script that I also created previously to create namespaces that I needed. That script uses helm.

I used vagrant ssh to connect into the running VM.

Then I started the namespaces.sh script to install the namespaces (using helm) and got the following error message:

Error: Kubernetes cluster unreachable

I needed to repeat a command present in my helm.sh script that I ran previously in order to make the Kubernetes cluster reachable with helm 3.0.

kubectl config view --raw >~/.kube/config

And by the way, I also wanted to try out using a kubeconfig file to sign in, to the Kubernetes Dashboard.

So, I copied the config file to my Windows laptop (via the shared folder):

cat ~/.kube/config
cp ~/.kube/config /vagrant/kubeconfig

I also noticed that in the Web Browser on my Windows laptop, the Kubernetes Dashboard couldn’t be reached.

I needed to repeat the following command present in my dashboard.sh script:

kubectl proxy --address='0.0.0.0' </dev/null &>/dev/null &

In the scripts directory I therefor created a file commands.sh with the following content:

#!/bin/bash
echo "**** Begin running commands"

# Wait 2 minutes
echo "**** Waiting 2 minutes ..."
sleep 120

#Make Kubernetes cluster reachable with helm 3.0
kubectl config view --raw >~/.kube/config

cat ~/.kube/config
cp ~/.kube/config /vagrant/kubeconfig

kubectl proxy --address='0.0.0.0' </dev/null &>/dev/null &

echo "**** End running commands"

I also had to make some changes to my existing script files.

  • In namespaces.sh, using

helm install ./namespace-chart –name namespace-release
resulted in the following:
Error: unknown flag: –name
So, I changed it to: helm install namespace-release ./namespace-chart

  • In mysql.sh, using

helm install ./mysql-chart –name mysql-release
resulted in the following:
Error: unknown flag: –name
So, I changed it to: helm install mysql-release ./mysql-chart

So, I changed the content of Vagrantfile to:
[in bold, I highlighted the changes]

Vagrant.configure("2") do |config|
  config.vm.box = "my_ubuntu_k3s"
  
  config.vm.define "ubuntu_k3s_mysql" do |ubuntu_k3s_mysql|
  
    config.vm.network "forwarded_port",
      guest: 8001,
      host:  8001,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9010,
      host:  9010,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9020,
      host:  9020,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9110,
      host:  9110,
      auto_correct: true
      
    config.vm.provider "virtualbox" do |vb|
        vb.name = "Ubuntu k3s mysql"
        vb.memory = "8192"
        vb.cpus = "1"
        
    args = []
    config.vm.provision "commands shell script", type: "shell",
        path: "scripts/commands.sh",
        args: args
        
    args = []
    config.vm.provision "namespaces shell script", type: "shell",
        path: "scripts/namespaces.sh",
        args: args
        
    args = []
    config.vm.provision "mysql shell script", type: "shell",
        path: "scripts/mysql.sh",
        args: args
        
    end
    
  end

end

In order to stop the running machine and destroy its resources, I used the following command on the Windows Command Prompt: vagrant destroy

With the following output:

ubuntu_k3s_mysql: Are you sure you want to destroy the ‘ubuntu_k3s_mysql’ VM? [y/N] y
==> ubuntu_k3s_mysql: Forcing shutdown of VM…
==> ubuntu_k3s_mysql: Destroying VM and associated drives…
[https://www.vagrantup.com/docs/cli/destroy.html]

From the subdirectory named env on my Windows laptop, I opened a Windows Command Prompt (cmd) and typed: vagrant up

With the following output:


Bringing machine ‘ubuntu_k3s_mysql’ up with ‘virtualbox’ provider…
==> ubuntu_k3s_mysql: Importing base box ‘my_ubuntu_k3s’…

Making re-usable box my_ubuntu_k3s_mysql

I copied the current version of the Vagrantfile to the directory myboxes/my_ubuntu_k3s_mysql.

Next, I typed the following command on the Windows Command Prompt (cmd):

vagrant package –output my_ubuntu_k3s_mysql.box

With the following output:

==> ubuntu_k3s_mysql: Attempting graceful shutdown of VM…
==> ubuntu_k3s_mysql: Forcing shutdown of VM…
==> ubuntu_k3s_mysql: Clearing any previously set forwarded ports…
==> ubuntu_k3s_mysql: Exporting VM…
==> ubuntu_k3s_mysql: Compressing package to: c:/My/AMIS/env/my_ubuntu_k3s_mysql.box

I moved my_ubuntu_k3s_mysql.box to the directory myboxes/my_ubuntu_k3s_mysql.

In Oracle VirtualBox I could see that the VM was Powered Off.

Next, I changed the content of the Vagrantfile to use the re-usable box:
[in bold, I highlighted the changes]

Vagrant.configure("2") do |config|
  config.vm.box = "my_ubuntu_k3s_mysql"
  
  config.vm.define "ubuntu_k3s_mysql_services" do |ubuntu_k3s_mysql_services|
  
    config.vm.network "forwarded_port",
      guest: 8001,
      host:  8001,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9010,
      host:  9010,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9020,
      host:  9020,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9110,
      host:  9110,
      auto_correct: true
      
    config.vm.provider "virtualbox" do |vb|
        vb.name = "Ubuntu k3s mysql services"
        vb.memory = "8192"
        vb.cpus = "1"
        
    end
    
  end

end

In order to add my own box, I typed the following command on the Windows Command Prompt (cmd):

vagrant box add –name my_ubuntu_k3s_mysql myboxes/my_ubuntu_k3s_mysql/my_ubuntu_k3s_mysql.box

With the following output:

==> box: Box file was not detected as metadata. Adding it directly…
==> box: Adding box ‘my_ubuntu_k3s_mysql’ (v0) for provider:
box: Unpacking necessary files from: file://c:/My/AMIS/env/myboxes/my_ubuntu_k3s_mysql/my_ubuntu_k3s_mysql.box
box:
==> box: Successfully added box ‘my_ubuntu_k3s_mysql’ (v0) for ‘virtualbox’!

Next, again I typed the following command on the Windows Command Prompt (cmd):

vagrant box list

With the following output:
[in bold, I highlighted the changes]

bento/ubuntu-18.04 (virtualbox, 201808.24.0)
generic/alpine310 (virtualbox, 2.0.0)
hashicorp/precise64 (virtualbox, 1.1.0)
my_ubuntu_k3s (virtualbox, 0)
my_ubuntu_k3s_mysql (virtualbox, 0)
ol7-latest (virtualbox, 0)
ubuntu/bionic64 (virtualbox, 20190814.0.0)
ubuntu/bionic64 (virtualbox, 20191218.0.0)
ubuntu/bionic64 (virtualbox, 20200107.0.0)
ubuntu/bionic64 (virtualbox, 20200124.0.0)
ubuntu/bionic64 (virtualbox, 20200402.0.0)
ubuntu/bionic64 (virtualbox, 20200407.0.0)
ubuntu/bionic64 (virtualbox, 20200416.0.0)
ubuntu/bionic64 (virtualbox, 20200425.0.0)
ubuntu/trusty64 (virtualbox, 20181103.0.0)
ubuntu/xenial64 (virtualbox, 20181223.0.0)
ubuntu/xenial64 (virtualbox, 20190101.0.0)
ubuntu/xenial64 (virtualbox, 20190109.0.0)
ubuntu/xenial64 (virtualbox, 20190115.0.0)
ubuntu/xenial64 (virtualbox, 20190118.0.0)
ubuntu/xenial64 (virtualbox, 20190123.0.0)
ubuntu/xenial64 (virtualbox, 20190123.0.1)
ubuntu/xenial64 (virtualbox, 20190215.0.0)

Again, from the subdirectory named env on my Windows laptop, I opened a Windows Command Prompt (cmd) and typed: vagrant up

With the following output:


Bringing machine ‘ubuntu_k3s_mysql_services’ up with ‘virtualbox’ provider…
==> ubuntu_k3s_mysql_services: Importing base box ‘my_ubuntu_k3s_mysql’…

So again, creating an Oracle VirtualBox appliance based on my re-usable box my_ubuntu_k3s_mysql worked.

Adding the book service to the appliance

Next step is to add the book service (RESTful Web Service Spring Boot application) that I created earlier and ran in Minikube, to the appliance. Of course, I already had a script in please to add them, because of a previous article I wrote.
[https://technology.amis.nl/2019/03/05/using-a-restful-web-service-spring-boot-application-in-minikube-together-with-an-external-dockerized-mysql-database/]

I used vagrant ssh to connect into the running VM.

Then I started the docker.sh script and the booksservices.sh script to create Docker images and install the different versions of the book service (using helm).

I had to make some changes to my existing script files.

  • In booksservices.sh, using

helm install ./booksservice-chart –name booksservice-release
resulted in the following:
Error: unknown flag: –name
So, I changed it to: helm install booksservice-release ./booksservice-chart

  • In booksservices.sh, using
echo "**** Determine the IP of the minikube node"
nodeIP=$(kubectl get node minikube -o yaml | grep address: | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}")
echo "---$nodeIP---"

resulted in the following:
Error from server (NotFound): nodes “minikube” not found
So, I changed it to:

echo "**** Determine the IP of the k3s node"
nodeIP=$(kubectl get node ubuntu-bionic -o yaml | grep address: | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}")
echo "---$nodeIP---"

I also changed the text minikube into k3s.

  • In booksservices.sh, using

mysql -h 127.0.0.1 -uroot -ppassword -e “show databases;”
resulted in the following:
Can’t connect to MySQL server on ‘127.0.0.1’
So, I needed to repeat a command present in my mysql.sh script that I ran previously in order to make the mysql server reachable:

echo "**** Forward local port 3306 to port 3306 on the mysql-service service"
kubectl port-forward service/mysql-service 3306 --namespace nl-amis-testing </dev/null &>/dev/null &

After applying these changes, I noticed the following in the output:


ubuntu_k3s_mysql_services: **** List pods with namespace nl-amis-testing
ubuntu_k3s_mysql_services: NAME
ubuntu_k3s_mysql_services:
ubuntu_k3s_mysql_services:
ubuntu_k3s_mysql_services:
ubuntu_k3s_mysql_services:
ubuntu_k3s_mysql_services:
ubuntu_k3s_mysql_services: READY
ubuntu_k3s_mysql_services:
ubuntu_k3s_mysql_services: STATUS
ubuntu_k3s_mysql_services:
ubuntu_k3s_mysql_services:
ubuntu_k3s_mysql_services: RESTARTS   AGE
ubuntu_k3s_mysql_services: mysql-66d6c9c869-xd9tk
ubuntu_k3s_mysql_services: 1/1     Running            1          4h28m
ubuntu_k3s_mysql_services: booksservice-v1.0-5b57649c7f-w565d   0/1     ImagePullBackOff   0          2m31s
ubuntu_k3s_mysql_services: booksservice-v1.0-5b57649c7f-fz5v9   0/1     ImagePullBackOff   0          2m31s

A quick look in the Kubernetes Dashboard revealed the same ImagePullBackOff issue:

I opened the log from the first Pod, and saw the reason behind the problem:


container “booksservice-v1-0-container” in pod “booksservice-v1.0-5b57649c7f-fz5v9” is waiting to start: trying and failing to pull image

Why k3s could not find the image I created using Docker

Apparently k3s could not find the image I created using Docker. So, I looked at the k3s documentation. It turns out that the containerd runtime is used and not Docker.

K3s includes and defaults to containerd, an industry-standard container runtime. If you want to use Docker instead of containerd then you simply need to run the agent with the –docker flag.
[https://rancher.com/docs/k3s/latest/en/advanced/#using-docker-as-the-container-runtime]

There for, I used the following command on the Linux Command Prompt:

docker image ls

With the following output:


REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
booksservice        v2.0                2fdba1d0db06        22 hours ago        143MB
booksservice        v1.0                33e6b89683fe        22 hours ago        143MB
hello-world         latest              bf756fb1ae65        3 months ago        13.3kB
openjdk             8-jdk-alpine        a3562aa0b991        11 months ago       105MB

Next, I used the following command on the Linux Command Prompt:

sudo k3s crictl images

With the following output:


IMAGE                                      TAG                 IMAGE ID            SIZE
docker.io/coredns/coredns                  1.6.3               c4d3d16fe508b       14.2MB
docker.io/kubernetesui/dashboard           v2.0.0-beta4        6802d83967b99       35.9MB
docker.io/kubernetesui/metrics-scraper     v1.0.1              709901356c115       16.1MB
docker.io/library/mysql                    5.6                 afa411733b0ca       103MB
docker.io/library/traefik                  1.7.14              f12ee21b2b872       23.6MB
docker.io/rancher/klipper-helm             v0.1.5              c1e4f72eb6760       27.1MB
docker.io/rancher/klipper-lb               v0.1.2              897ce3c5fc8ff       2.71MB
docker.io/rancher/local-path-provisioner   v0.0.11             9d12f9848b99f       12MB
docker.io/rancher/metrics-server           v0.3.6              9dd718864ce61       10.5MB
k8s.gcr.io/pause                           3.1                 da86e6ba6ca19       317kB

I wanted to stay with the default and leave containerd as runtime. After some search on the Internet I found a solution, that worked for me.
[https://blog.scottlowe.org/2020/01/25/manually-loading-container-images-with-containerd/]

On my Windows laptop, in the env directory I created a directory docker_images.

Next, I used the following command on the Linux Command Prompt:

docker save booksservice:v1.0 -o /vagrant/docker_images/booksservice_v1.0.tar
sudo ctr -n=k8s.io images import /vagrant/docker_images/booksservice_v1.0.tar

With the following output:


unpacking docker.io/library/booksservice:v1.0 (sha256:28a0119a3ca51b9ebe46df37ee0d5df2205d424e1246514b2dff419409635073)…done

Followed by, the following command on the Linux Command Prompt:

docker save booksservice:v2.0 -o /vagrant/docker_images/booksservice_v2.0.tar
sudo ctr -n=k8s.io images import /vagrant/docker_images/booksservice_v2.0.tar

With the following output:


unpacking docker.io/library/booksservice:v2.0 (sha256:2e62b4a30af1a14a135b01b5f87a221e085a5d96e77b619cbfb6b4e312ef33d9)…done

I looked at the content of directory docker_images:

Next, I used the following command on the Linux Command Prompt:

sudo k3s crictl images

With the following output:


IMAGE                                      TAG                 IMAGE ID            SIZE
docker.io/coredns/coredns                  1.6.3               c4d3d16fe508b       14.2MB
docker.io/kubernetesui/dashboard           v2.0.0-beta4        6802d83967b99       35.9MB
docker.io/kubernetesui/metrics-scraper     v1.0.1              709901356c115       16.1MB
docker.io/library/booksservice             v1.0                33e6b89683fe7       144MB
docker.io/library/booksservice             v2.0                2fdba1d0db060       144MB
docker.io/library/mysql                    5.6                 afa411733b0ca       103MB
docker.io/library/traefik                  1.7.14              f12ee21b2b872       23.6MB
docker.io/rancher/klipper-helm             v0.1.5              c1e4f72eb6760       27.1MB
docker.io/rancher/klipper-lb               v0.1.2              897ce3c5fc8ff       2.71MB
docker.io/rancher/local-path-provisioner   v0.0.11             9d12f9848b99f       12MB
docker.io/rancher/metrics-server           v0.3.6              9dd718864ce61       10.5MB
k8s.gcr.io/pause                           3.1                 da86e6ba6ca19       317kB

A quick look in the Kubernetes Dashboard revealed to me that things were better looking:

Remark:
Remember that I wanted to try out using a kubeconfig file to sign in, to the Kubernetes Dashboard. And that I copied the config file to my Windows laptop (via the shared folder). Unfortunately, that didn’t work and I left finding out way for some other time. For now, working with the token as a workaround, was fine.

Continuing adding the book service to the appliance

Next, I added these necessary commands to my booksservices.sh script, with the following content:
[in bold, I highlighted the changes]

#!/bin/bash
echo "**** Begin installing booksservices"

#Create Helm chart
echo "**** Create Helm chart"
cd /vagrant
cd helmcharts
rm -rf /vagrant/helmcharts/booksservice-chart/*
helm create booksservice-chart

rm -rf /vagrant/helmcharts/booksservice-chart/templates/*
cp /vagrant/yaml/*booksservice*.yaml /vagrant/helmcharts/booksservice-chart/templates

# Create Docker images
echo "**** Docker images"
cd /vagrant
cd applications
cd books_service_1.0
docker build -t booksservice:v1.0 .

cd ..
cd books_service_2.0
docker build -t booksservice:v2.0 .

# Wait 30 seconds
echo "**** Waiting 30 seconds ..."
sleep 30

# Create containerd images
echo "**** Containerd image booksservice:v1.0"
docker save booksservice:v1.0 -o /vagrant/docker_images/booksservice_v1.0.tar
sudo ctr -n=k8s.io images import /vagrant/docker_images/booksservice_v1.0.tar

echo "**** Containerd image booksservice:v2.0"
docker save booksservice:v2.0 -o /vagrant/docker_images/booksservice_v2.0.tar
sudo ctr -n=k8s.io images import /vagrant/docker_images/booksservice_v2.0.tar

# Wait 30 seconds
echo "**** Waiting 30 seconds ..."
sleep 30

#List containerd images
echo "**** List containerd images"
sudo k3s crictl images

# Install Helm chart
cd /vagrant
cd helmcharts
echo "**** Install Helm chart booksservice-chart"
helm install booksservice-release ./booksservice-chart

# Wait 2,5 minute
echo "**** Waiting 2,5 minute ..."
sleep 150

#List helm releases
echo "**** List helm releases"
helm list -d

#List pods
echo "**** List pods with namespace nl-amis-testing"
kubectl get pods --namespace nl-amis-testing

#List services
echo "**** List services with namespace nl-amis-testing"
kubectl get service --namespace nl-amis-testing

echo "**** Determine the IP of the k3s node"
nodeIP=$(kubectl get node ubuntu-bionic -o yaml | grep address: | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}")
echo "---$nodeIP---"

echo "**** Via socat forward local port 9010 to port 30010 on the k3s node ($nodeIP)"
socat tcp-listen:9010,fork tcp:$nodeIP:30010 &

echo "**** Via socat forward local port 9020 to port 30020 on the k3s node ($nodeIP)"
socat tcp-listen:9020,fork tcp:$nodeIP:30020 &

echo "**** Via socat forward local port 9110 to port 30110 on the k3s node ($nodeIP)"
socat tcp-listen:9110,fork tcp:$nodeIP:30110 &

# Wait 1 minute
echo "**** Waiting 1 minute ..."
sleep 60

echo "**** Add books"
curl --header "Content-Type: application/json" --request POST --data '{"id": 1, "title": "The Threat: How the FBI Protects America in the Age of Terror and Trump", "author": "Andrew G. McCabe", "type": "Hardcover", "price": 17.99, "numOfPages": 288, "language": "English", "isbn13": "978-1250207579"}' http://localhost:9110/books

curl --header "Content-Type: application/json" --request POST --data '{"id": 2, "title": "Becoming", "publishDate": "2018-11-13", "author": "Michelle Obama", "type": "Hardcover", "price": 17.88, "numOfPages": 448, "publisher": "Crown Publishing Group; First Edition edition", "language": "English", "isbn13": "978-1524763138"}' http://localhost:9110/books

echo "**** Get books"
curl http://localhost:9110/books

echo ""
echo "**** List the books in the database"
mysql -h 127.0.0.1 -uroot -ppassword -e "show databases;"
mysql -h 127.0.0.1 -uroot -ppassword -e "use test; select * from book;"

echo "**** End installing booksservices"

Then, I changed the content of Vagrantfile to:
[in bold, I highlighted the changes]

Vagrant.configure("2") do |config|
  config.vm.box = "my_ubuntu_k3s_mysql"
  
  config.vm.define "ubuntu_k3s_mysql_services" do |ubuntu_k3s_mysql_services|
  
    config.vm.network "forwarded_port",
      guest: 8001,
      host:  8001,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9010,
      host:  9010,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9020,
      host:  9020,
      auto_correct: true
      
    config.vm.network "forwarded_port",
      guest: 9110,
      host:  9110,
      auto_correct: true
      
    config.vm.provider "virtualbox" do |vb|
        vb.name = "Ubuntu k3s mysql services"
        vb.memory = "8192"
        vb.cpus = "1"
        
    args = []
    config.vm.provision "commands shell script", type: "shell",
        path: "scripts/commands.sh",
        args: args
        
    args = []
    config.vm.provision "docker shell script", type: "shell",
        path: "scripts/docker.sh",
        args: args
        
    args = []
    config.vm.provision "booksservices shell script", type: "shell",
        path: "scripts/booksservices.sh",
        args: args
        
    end
    
  end

end

In order to stop the running machine and destroy its resources, I used the following command on the Windows Command Prompt: vagrant destroy

With the following output:

ubuntu_k3s_mysql_services: Are you sure you want to destroy the ‘ubuntu_k3s_mysql_services’ VM? [y/N] y
==> ubuntu_k3s_mysql_services: Forcing shutdown of VM…
==> ubuntu_k3s_mysql_services: Destroying VM and associated drives…

From the subdirectory named env on my Windows laptop, I opened a Windows Command Prompt (cmd) and typed: vagrant up

With the following output (only showing the last part):


ubuntu_k3s_mysql_services: **** Get books
100   256    0     0  100   256      0    209  0:00:01  0:00:01 –:–:–   209
    ubuntu_k3s_mysql_services:
    ubuntu_k3s_mysql_services:
    ubuntu_k3s_mysql_services: %
    ubuntu_k3s_mysql_services:
    ubuntu_k3s_mysql_services: T
    ubuntu_k3s_mysql_services: o
    ubuntu_k3s_mysql_services: t
    ubuntu_k3s_mysql_services: a
    ubuntu_k3s_mysql_services: l
    ubuntu_k3s_mysql_services:
    ubuntu_k3s_mysql_services:
    ubuntu_k3s_mysql_services:   % Received % Xferd  Average Speed   Time    Time     Time  Current
    ubuntu_k3s_mysql_services:                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 –:–:– –:–:– –:–:–     0
    ubuntu_k3s_mysql_services: [{“id”:”1″,”title”:”The Threat: How the FBI Protects America in the Age of Terror and Trump”,”author”:”Andrew G. McCabe”,”type”:”Hardcover”,”price”:17.99,”numOfPages”:288,”language”:”English”,”isbn13″:”978-1250207579″},{“id”:”2″,”title”:”Becoming”,”author”:”Michelle Obama”,”type”:”Hardcover”,”price”:17.88,”numOfPages”:448,”language”:”English”,”isbn13″:”978-1524763138″}]
100   372    0   372    0     0    796      0 –:–:– –:–:– –:–:–   796
    ubuntu_k3s_mysql_services:
    ubuntu_k3s_mysql_services: **** List the books in the database
    ubuntu_k3s_mysql_services: mysql:
    ubuntu_k3s_mysql_services: [Warning] Using a password on the command line interface can be insecure.
    ubuntu_k3s_mysql_services: Database
    ubuntu_k3s_mysql_services: information_schema
    ubuntu_k3s_mysql_services: mysql
    ubuntu_k3s_mysql_services: performance_schema
    ubuntu_k3s_mysql_services: test
    ubuntu_k3s_mysql_services: mysql:
    ubuntu_k3s_mysql_services: [Warning] Using a password on the command line interface can be insecure.
    ubuntu_k3s_mysql_services: id       author  isbn13  language        num_of_pages    price   title   type
    ubuntu_k3s_mysql_services: 1        Andrew G. McCabe        978-1250207579  English 288     17.99   The Threat: How the FBI Protects America in the Age of Terror and Trump    Hardcover
    ubuntu_k3s_mysql_services: 2        Michelle Obama  978-1524763138  English 448     17.88   Becoming        Hardcover
    ubuntu_k3s_mysql_services: **** End installing booksservices

On my Windows laptop, from Postman I invoked a request named “GetAllBooksRequest” (with method “GET” and URL “http://locahost:9110/books”) and a response with “Status 200 OK” was shown:

As a final step I checked the contents of the book table.

I used vagrant ssh to connect into the running VM. Next, I used the following command on the Linux Command Prompt:

mysql -h 127.0.0.1 -uroot -ppassword -e "use test; select * from book;"

With the following output:

As you can see, both books that I added (via my booksservices.sh script), were returned from the catalog.

With these final checks I conclude this article. I shared with you the steps I took, to add mysql to my k3s demo environment and in order to test the environment, I also included the book service (RESTful Web Service Spring Boot application) that I created earlier and ran in Minikube.

With some minor changes to my existing script files I was able to get everything working. I showed you, how I was able to get my Docker built images working with the containerd, which is included and the default industry-standard container runtime in k3s.

I also showed you how I created my own prepackaged Vagrant Box from an existing virtual machine (a re-usable box).

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Next Post

Add tests for Angular and Node.js webapp in Azure Pipelines (part 3)

Facebook0TwitterLinkedinIn the previous article I showed how to build a webapp in the Azure pipeline. In this article I will explain to you how you run tests in your pipeline. I will show you how to do this for both Angular and Node.js. Adding tests frontend – Angular Adding tests […]
Azure Pipelines: How to build and test an Angular and Node.js webapp