Vagrant and Docker followed by Puppet to provision complex environments

The Docker container build process has some limitations. The Docker file can quickly become unwieldy. Besides, by and large the Docker file has us execution fairly low level instructions – somewhat against the trend of using declarations rather than scripts to specify what environments should look like. Additionally, I have to admit defeatusing instructions found at: there is no way to access files outside the Docker build context (other than through HTTP download) and there is no way to work with installation files and intermediate files without these files becoming part of the layers that make up the container being built. With large files this can be a serious issue.

To work around these limitations I have decided on a different workflow for producing my containers. Note: this approach also makes a better distinction (at least in my head) between the provisioning stage (where the container is initially put together) and the production stage where the container is actually running and doing its thing. With the combination of Docker and Vagrant, these two stages get easily mixed up.

The flow I am adopting is shown in the next figure. Simply put, Vagrant provides the VirtualBox Linux VM that acts as the Docker engine (1). It then provides the Docker container (2) – leveraging Docker to perform basic configuration of that container (3) – including the installation of Puppet (4). The detailed configuration of the container is done using Puppet after the Docker build has completed. During the Puppet provisioning, the container can leverage folder mappings from the host (both Vagrant host and Docker host) (5) – meaning that the limitation on build context and temporary files no longer apply. Anything Puppet can do can now be done – in a neat, declarative way – on the Docker Container (6).

 

In order to retain the results from the Puppet provisioning, it is probably a good idea to create a (custom, local) image from the container. When we want to run the container, we no longer care about build and provisioning, we simply want to use the result by running a fresh container based on the image we have created. This container does not need the build-time folder mappings to the local Puppet and staging file areas. Instead it could use different mappings as well as links to other containers that are not meaningful at build time.

This article provides details on this approach with the bare-minimum set of Vagrant, Docker and Puppet files – running on a Windows host and using Ubuntu as the Linux distro.

Note: apparently, Vagrant can run the Puppet provisioning in addition to the Docker provisioner. It seems like that could make my approach even more compact and elegant, but I have not found of way of getting that to work properly.

The sources used for this article are available on GitHub: https://github.com/lucasjellema/vagrant-docker-puppet

This is what the file system will look like after cloning the Git repository:

 

The assumption here is that Vagrant and VirtualBox have been installed on your host machine. Nothing else is required.

The first step is to open a command window, navigate to the home directory for this project (the folder that contains the Vagrant file) and execute vagrant up my-puppet-base-container:

 

The dockerhostvm is created in this case (because I am on Windows and it does not already exist).

The last steps in this action prepare the dockerhostvm for using Docker:

Then the container is built:

As part of this build step, the container is Puppet enabled (using instructions found at https://docs.puppetlabs.com/guides/install_puppet/install_debian_ubuntu.html), the /u01 directory is created and a shell-script run Puppet later on is copied to this directory. Then the container is completed.

 

At this point, we move from the Vagrant (Windows) host to the dockerhostvm – to start the container and execute the Puppet provisioning inside. Note: I have used vagrant global-status to find out the id for the dockerhostvm (07fadf2 in this case).

Using vagrant shh <id for dockerhostvm>  we connect into the VM that runs Docker. Using docker ps -a we get a list of containers – including the my-puppet-base-container. We need the Container Id for this container.

 

 

With docker start <container id>  we can start the container that was built by the Vagrant & Docker team. Finally the command docker exec -it container id bash “/u01/run2Puppet.sh” is used to execute the run2Puppet.sh script in the container, which in turn invokes Puppet to provision the environment: puppet apply –debug /u01/puppet/manifests/base.pp. The configuration described in the base.pp manifest is being applied/realized by Puppet.

When the actions by Puppet are complete, the container is provisioned and now it can be turned into an image (not: this is an optional step):

With this image, we can start new containers and start extending them if we like. This can be done both from within the dockerhostvm as well as from the Vagrant host. The latter is done using a second configuration inside the Vagrant file:

The command to start up the container: vagrant up my-complete-container.

And we can execute commands in containers based on this image from the Windows host like this using the command vagrant docker-run my-complete-container -t — bash:

The last section (cd /u01/app/oracle) proves that Puppet has indeed executed successfully: the directory /u01/app/oracle was defined declaratively in the base.pp Puppet manifest – and nowhere else:

 

 

Summary

This article shows an approach for the build workflow for Docker container using a combination of Vagrant, VirtualBox, Docker and Puppet. Rather than having Docker do the complete provisioning of the container – which has a number of limitations associated with it – I suggest we use Docker for the basic stuff and subsequently have Puppet do the complex stuff and final touches. This approach offers the benefit of using a declarative approach (for which many, many modules are already available) instead of a very script oriented way of working – labor intensive to create, hard to maintain and difficult to debug – Puppet offers a quite elegant way of configuring the desired environment. Additionally, this approach helps us work around the fact that the Docker build context cannot access files and folders on the host.

The steps from a bare Windows system to complex Docker containers spinning up are fairly straightforward, as was shown. Install Vagrant and VirtualBox and configure the Vagrantfile, Dockerfile and Puppet manifest. Quite simple really.

 

Resources

My first steps with Vagrant and Docker – https://technology.amis.nl/2015/08/22/first-steps-with-provisioning-of-docker-containers-using-vagrant-as-provider/

More advanced topics around Vagrant and Docker – https://technology.amis.nl/2015/08/25/vagrant-and-docker-next-and-advanced-steps-with-folders-ports-volumes-linking-and-more/

Quickstart a Web Development Stack Using Vagrant & Docker – https://www.smaato.com/quickstart-a-web-development-stack-using-vagrant-docker/

Docker Documentation – working with containers – https://docs.docker.com/userguide/usingdocker/

Dockerfile Tutorial – Building Docker Images for Containers – http://www.slashroot.in/dockerfile-tutorial-building-docker-images-for-containers

Slideshare: Taking Control of Chaos with Docker and Puppet – http://www.slideshare.net/PuppetLabs/docker-puppetcamp-london

Slideshare: Vagrant + Docker provider [+Puppet] – http://www.slideshare.net/ni_po/n-poggi-vagrantdocker

Slideshare: Vagrant + Docker – http://www.slideshare.net/3dgiordano/vagrant-docker

 

Articles on not being able to reference volumes, external folders, volumes-from etc. from the Docker Build Context:

there are many of them – I have read a large number. they all boil down to the conclusion: what we want to do, cannot be done!

How to mount host volumes into docker containers in Dockerfile during build