CHAPTER 6

Automating Your Infrastructure as Code

This chapter aims to explain how the Infrastructure as Code works, install Vagrant, how to work using it, create new virtual machines in an automated way using the Shell Script, and set up all Infrastructure we did before.

Structure

In this chapter, we will discuss the following topics:

• Introduction to Infrastructure as Code
• Installing Vagrant and creating virtual machines
• Setting up a LAMP server
• Setting up a Docker server

Objectives

After studying this unit, you should be able to:

• What is Vagrant and how it works
• Setup a Docker environment using Vagrant automation
• Setting up an Apache server
• Benefits of automation

Introduction to Infrastructure as Code

Infrastructure as Code is a technique used to document, version, and maintain the control of your infrastructure in a very similar way as you do with software development. Then, you can create a file describing the desired state of your infrastructure, deploy it on a test environment, validate your code, deploy on a staging environment, and if everything goes well, you can proceed to the production environment. It is possible using pipelines; therefore, you can use the same code for all the environment and change the server's sizes using the variables defined for each of the environments.

What is Vagrant and How It Works?

In the beginning of this book, you will learn how to setup the VirtualBox, download an ISO, create a VM, attach the ISO, and format and install the operating system. However, it is a recurrent task in the daily life of a DevOps engineer. Can you imagine yourself installing 100 VMs every day and destroying them after the working day to save money? It is insane. But, when we use Infrastructure as Code (IaC), this task becomes very easy to do, because you just need to do some clicks, and run some commands to create and destroy everything.

Vagrant is a tool that can be integrated with the Virtual Box. It is responsible to read your code, convert it into Virtual Box commands, and also run the code within the VM. You need not to execute all those steps learned in the first chapter. HashiCorp created Vagrant and they have an official repository of images that you can use to setup your environment. So, you do not need to go to the official website, download the ISO, and setup the machine.

Vagrant installing

You can download Vagrant from the following link:

https://www.vagrantup.com/downloads.html

I am using Windows, so I will download the MSI version.

Figure 6.1

The installation process is simple like any other software to be installed on Windows. You just need to open the MSI file and follow the process, Next, Next:

Figure 6.2

And then, click Finish with all the default options:

Figure 6.3

Now, since the installation is completed, let us learn the basics.

Usage

Firstly, create a folder to store your vagrant files as I did:

I created a folder, called VagrantChaper and I entered that folder. Right now, it is empty. To create your first code, type the following command:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant init

A 'Vagrantfile' has been placed in this directory. You are now

ready to 'vagrant up' your first virtual environment! Please read

the comments in the Vagrantfile as well as documentation on

'vagrantup.com' for more information on using Vagrant.

PS C:\Users\1511 MXTI\VagrantChapter>

The command vagrant init, creates a sample file for you with the common configuration for the most of the cases of VMs. Let's have a look in the file content:

PS C:\Users\1511 MXTI\VagrantChapter> cat .\Vagrantfile

# -*- mode: ruby -*-

# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure

# configures the configuration version (we support older styles for

# backwards compatibility). Please don't change it unless you know what

# you're doing.

Vagrant.configure("2") do |config|

# The most common configuration options are documented and commented below.

# For a complete reference, please see the online documentation at

# https://docs.vagrantup.com.

# Every Vagrant development environment requires a box. You can search for

# boxes at https://vagrantcloud.com/search.

config.vm.box = "base"

# Disable automatic box update checking. If you disable this, then

# boxes will only be checked for updates when the user runs

# 'vagrant box outdated'. This is not recommended.

# config.vm.box_check_update = false

# Create a forwarded port mapping which allows access to a specific port

# within the machine from a port on the host machine. In the example below,

# accessing "localhost:8080" will access port 80 on the guest machine.

# NOTE: This will enable public access to the opened port

# config.vm.network "forwarded_port", guest: 80, host: 8080

# Create a forwarded port mapping which allows access to a specific port

# within the machine from a port on the host machine and only allow access

# via 127.0.0.1 to disable public access

# config.vm.network "forwarded_port", guest: 80, host: 8080, host_ip: "127.0.0.1"

# Create a private network, which allows host-only access to the machine

# using a specific IP.

# config.vm.network "private_network", ip: "192.168.33.10"

# Create a public network, which generally matched to bridged network.

# Bridged networks make the machine appear as another physical device on

# your network.

# config.vm.network "public_network"

# Share an additional folder to the guest VM. The first argument is

# the path on the host to the actual folder. The second argument is

# the path on the guest to mount the folder. And the optional third

# argument is a set of non-required options.

# config.vm.synced_folder "../data", "/vagrant_data"

# Provider-specific configuration so you can fine-tune various

# backing providers for Vagrant. These expose provider-specific options.

# Example for VirtualBox:

#

# config.vm.provider "virtualbox" do |vb|

# # Display the VirtualBox GUI when booting the machine

# vb.gui = true

#

# # Customize the amount of memory on the VM:

# vb.memory = "1024"

# end

#

# View the documentation for the provider you are using for more

# information on available options.

# Enable provisioning with a shell script. Additional provisioners such as

# Ansible, Chef, Docker, Puppet and Salt are also available. Please see the

# documentation for more information about their specific syntax and use.

# config.vm.provision "shell", inline: <<-SHELL

# apt-get update

# apt-get install -y apache2

# SHELL

end

In the same folder, you run the following command:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant status

Current machine states:

default not created (virtualbox)

It means that we have one VM, called default, and it is not created yet. So, we will create one.

The sample file has some configurations and many comments giving a brief explanation of what the commands do. This file, with the default configuration, allows you to create just one VM. In this chapter, we will create two virtual machines, one with the LAMP server installed, and other with the Docker installed. I will modify that file and explain the differences between the example file and the new one:

# -*- mode: ruby -*-

# vi: set ft=ruby :

Vagrant.configure("2") do |config|

config.vm.box = "ubuntu/focal64"

config.vm.box_check_update = false

config.vm.provider "virtualbox" do |vb|

vb.memory = "1024"

end

config.vm.define "lamp" do |lamp|

lamp.vm.network "private_network", ip: "192.168.33.10"

lamp.vm.provision "shell", inline: <<-SHELL

apt-get clean

apt update -y

SHELL

end

config.vm.define "docker" do |docker|

docker.vm.network "private_network", ip: "192.168.33.11"

docker.vm.provision "shell", inline: <<-SHELL

apt-get clean

apt update -y

SHELL

end

config.vm.provision "shell", inline: <<-SHELL

apt-get clean

apt update -y

SHELL

End

The preceding file is my new and modified one. It is cleaner than the example one, and does more stuff. Let's run the vagrant status, to see if something has changed:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant status

Current machine states:

lamp      not created (virtualbox)

docker    not created (virtualbox)

Interestingly, now we have two virtual machines, one is called a lamp, and the other is called Docker. Both are not created yet. In the new file, I used a Vagrant resource called multi-machine. It allows you to create more than one machine per file. So, you can have as many machines as you want in the same file.

Now, let's analyze the file and what will happen before we create the VMs:

config.vm.box = "ubuntu/focal64"

config.vm.box_check_update = false

config.vm.provider "virtualbox" do |vb|

vb.memory = "1024"

end

This part with config.vm is the common configuration for all the virtual machines declared in the file. So, all the machines will be created with 1024 MB of RAM. The config.vm.box is the image which will be used in the machine. In our case, we will use the Ubuntu server, version 20.04:

config.vm.define "lamp" do |lamp|

lamp.vm.network "private_network", ip: "192.168.33.10"

lamp.vm.provision "shell", inline: <<-SHELL

apt-get clean

apt update -y

SHELL

end

In this part, we have a local configuration just for the machine lamp. It is specifically for the virtual machine, called lamp. Then, in this called, the VM will have the IP address 192.168.33.10, and the command which will be run in the provision of the VM, as in, inside this part: lamp.vm.provision. It also applies to the Docker VM:

config.vm.provision "shell", inline: <<-SHELL

apt-get clean

apt update -y

SHELL

In the final part, we have the config.vm.provision. Inside this block, you can define the command which will run in all the VMs, except lamp.vm.provision, which will run only in the lamp VM, and docker.vm.provision, which will run only in the Docker VM. Therefore, we have global configurations that will run in all VMs and local configurations that will run only in the specified VMs.

Up and running

With the file explained and everything configures to create the VMs, type the following command:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant up --provider=virtualbox

Bringing machine 'lamp' up with 'virtualbox' provider…

Bringing machine 'docker' up with 'virtualbox' provider…

==> lamp: Box 'ubuntu/focal64' could not be found. Attempting to find and install…

lamp: Box Provider: virtualbox

lamp: Box Version: >= 0

==> lamp: Loading metadata for box 'ubuntu/focal64'

lamp: URL: https://vagrantcloud.com/ubuntu/focal64

It will take a while because Vagrant will download the box:

==> lamp: Loading metadata for box 'ubuntu/focal64'

lamp: URL: https://vagrantcloud.com/ubuntu/focal64

==> lamp: Adding box 'ubuntu/focal64' (v20200409.0.0) for provider: virtualbox

lamp:Downloading: https://vagrantcloud.com/ubuntu/boxes/focal64/versions/20200409.0.0/providers/virtualbox.box

lamp: Download redirected to host: cloud-images.ubuntu.com

lamp: Progress: 11% (Rate: 1411k/s, Estimated time remaining: 0:06:09)

The box is a Cloud image, but with specific configurations to run on top of Vagrant. As you can see in the logs, Vagrant is downloading the image for VirtualBox. It is also possible to provision using Hyper-V or KVM.

Vagrant Cloud (http://vagrantcloud.com/) is the official repository to download the images. But, of course, is not the only one. You can also create your images using Packer (https://packer.io/). However, if you don’t, you can search there what images are available for you to create your VMs.

Figure 6.4

After the provision, if you want to check your local boxes, use the following command:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant box list

centos/7              (virtualbox, 1801.02)

fedora/28-cloud-base  (virtualbox, 20180425)

ubuntu/bionic64       (virtualbox, 20190807.0.0)

ubuntu/focal64        (virtualbox, 20200409.0.0)

ubuntu/xenial64       (virtualbox, 20180413.0.0)Status: Downloaded newer image for

If you have a fresh installation, you probably have only the Ubuntu/focal64, the one we downloaded:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant status

Current machine states:

lamp     running (virtualbox)

docker   running (virtualbox)

If everything went well with your provision, run the command vagrant status and it will show you the running VMs. If there is a machine that is not created yet, you can try to run the vagrant up –provider=virtualbox again, and see if it works. If it doesn’t, check your installation and your internet connection.

Suppose, you did not define any password or SSH key to connect to these VMs, then, how do we connect to them? Vagrant is also managing that for us. With the vagrant status, we can see the names of the VMs and use these names to connect using the command, vagrant ssh:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant ssh docker

Welcome to Ubuntu Focal Fossa (development branch) (GNU/Linux 5.4.0-21-generic x86_64)

* Documentation:  https://help.ubuntu.com

* Management:     https://landscape.canonical.com

* Support:        https://ubuntu.com/advantage

The VM was successfully provisioned, and as you can see with the Ubuntu Focal Fossa version, Vagrant also created as user, called vagrant, to connect to the VMs, which has pseudo permission. So, we can just change the user and run all the commands:

vagrant@ubuntu-focal:~$ sudo su -

root@ubuntu-focal:~#

Now, you can manage everything as we did in the previous chapters. But, let's see how we can provision a LAMP in an automated way.

Setting up a LAMP Server

You already know the commands, and then, we can basically put the commands in the provision section of vagrant file and provision the VM again:

# -*- mode: ruby -*-

# vi: set ft=ruby :

Vagrant.configure("2") do |config|

config.vm.box = "ubuntu/focal64"

config.vm.box_check_update = false

config.vm.provider "virtualbox" do |vb|

vb.memory = "1024"

end

config.vm.define "lamp" do |lamp|

lamp.vm.network "private_network", ip: "192.168.33.10"

lamp.vm.hostname = "lamp"

lamp.vm.provision "shell", inline: <<-SHELL

sudo apt-get clean

sudo apt update -y

sudo apt install apache2 mysql-server php7.4 -y

SHELL

end

config.vm.define "docker" do |docker|

docker.vm.network "private_network", ip: "192.168.33.11"

docker.vm.hostname = "docker"

docker.vm.provision "shell", inline: <<-SHELL

apt-get clean

apt update -y

SHELL

end

config.vm.provision "shell", inline: <<-SHELL

apt-get clean

apt update -y

SHELL

end

In this case, I changed the lamp.vm.provision, including the Shell Script to install PHP, Apache, and MySQL server. Once the VM is created, run the following command:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant provision lamp

==> lamp: Running provisioner: shell…

lamp: Running: inline script

lamp:

lamp: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

lamp: Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]

lamp: Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease

lamp: Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease

You can follow the provisioning process in your Terminal. After the installation finishes, you can login into the VM:

PS C:\Users\1511 MXTI\VagrantChapter>vagrant ssh lamp

Welcome to Ubuntu Focal Fossa (development branch) (GNU/Linux 5.4.0-21-generic x86_64)

Check the services running:

Check the IP address, the one defined in the Vagrant file:

vagrant@lamp:~$ ip a

3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000

link/ether 08:00:27:69:a5:81 brd ff:ff:ff:ff:ff:ff

inet 192.168.33.10/24 brd 192.168.33.255 scope global enp0s8

valid_lft forever preferred_lft forever

You can check the IP address in the browser and see Apache running:

Figure 6.5

Now, every time you want to provision a new LAMP server, you can directly copy the code, or include a new VM with the provision code and it will work.

Setting up the Docker Server

This process is not different from the first one. So, let's edit the Vagrant file again:

# -*- mode: ruby -*-

# vi: set ft=ruby :

Vagrant.configure("2") do |config|

config.vm.box = "ubuntu/bionic64"

config.vm.box_check_update = false

config.vm.provider "virtualbox" do |vb|

vb.memory = "1024"

end

config.vm.define "lamp" do |lamp|

lamp.vm.network "private_network", ip: "192.168.33.10"

lamp.vm.hostname = "lamp"

lamp.vm.provision "shell", inline: <<-SHELL

sudo apt-get clean

sudo apt update -y

sudo apt install apache2 mysql-server php7.4 -y

SHELL

end

config.vm.define "docker" do |docker|

docker.vm.box = "ubuntu/bionic64"

docker.vm.network "private_network", ip: "192.168.33.11"

docker.vm.hostname = "docker"

docker.vm.provision "shell", inline: <<-SHELL

apt clean

apt-get update

apt-get remove docker docker-engine docker.io containerd runc -y

apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

apt-get update -y

apt-get install docker-ce docker-ce-cli containerd.io -y

SHELL

end

config.vm.provision "shell", inline: <<-SHELL

apt-get clean

apt update -y

SHELL

end

Note that I changed the box just for the Docker VM because, in the Docker repo, the support for Focal Fosse was not available yet. But it is also interesting to see how we can create different VMs with different OS versions.

Now, within the provision part of the Docker VM, we have all the installation steps to setup the server. Then, you can start the machine again:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant up docker

Bringing machine 'docker' up with 'virtualbox' provider…

==> docker: Importing base box 'ubuntu/focal64'…

==> docker: Matching MAC address for NAT networking…

==> docker: Setting the name of the VM: VagrantChapter_docker_1586798805547_50415

==> docker: Fixed port collision for 22 => 2222. Now on port 2200.

Run the provision command:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant provision docker

==> docker: Running provisioner: shell…

docker: Running: inline script

docker: WARNING:

docker: apt

docker:

docker: does not have a stable CLI interface.

docker: Use with caution in scripts.

docker: Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]

Let's check if the installation occurred successfully:

PS C:\Users\1511 MXTI\VagrantChapter> vagrant ssh docker

Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-55-generic x86_64)

* Documentation:  https://help.ubuntu.com

* Management:     https://landscape.canonical.com

* Support:        https://ubuntu.com/advantageSuccessfully installed Jinja2-2.11.1 MarkupSafe-1.1.1 Werkzeug-1.0.1 click-7.1.1 flask-1.1.1 itsdangerous-1.1.0

Checking that the Docker is up and running. If you face some connection issues, it is important to check if the Host Network Manager on VirtualBox is enabled:

Figure 6.6

Check the column DHCP server if it is enabled, and if the IPv4 address is present in the network 192.168.33.0/24 as the following screenshot:

Figure 6.7

Conclusion

In this chapter, we saw the benefits of using a tool like Vagrant, which allows us to write the VM configurations like OS image, IP address, hostname, VM memory, etc. In the provision part, we could write a Shell Script and the script will run during the boot. If it doesn’t, or if you want to run the script again, you can execute the command vagrant provision. Vagrant allows you to use different technologies, like Ansible, Puppet, or Chef to create your provision scripts. I used the Shell Script as an example, but, feel free to test and create your own scripts. Another important thing we learned is the Vagrant file. In this file, you define everything, and you can share the content with your team, so that they will be available to contribute with you to develop your environment.