An alternative to the in-place upgrade strategy is the expand and contract strategy. This strategy has become popular of late, thanks to the self-service nature of on-demand infrastructures, such as cloud computing or virtualization pools. The ability to create new servers on demand from a large pool of available resources means that every deployment of an application can happen on brand new systems. This strategy avoids a host of issues, such as a build up of cruft on long-running systems, like the following:
- Configuration files that are no longer managed by Ansible being left behind
- Runaway processes consuming resources in the background
- Changes being made to the server manually by human beings without updating the Ansible playbooks
Starting fresh each time also removes the differences between an initial deployment and an upgrade. The same code path can be used, reducing the risk of surprises when upgrading an application. This type of installation can also make it extremely easy to roll back if the new version does not perform as expected. In addition to this, as new systems are created to replace old systems, the application does not need to go into a degraded state during the upgrade.
Let's re-approach our previous upgraded playbook with the expand and contract strategy. Our pattern will be to create new servers, deploy our application, verify our application, add new servers to the load balancer, and remove old servers from the load balancer. Let's start by creating new servers. For this example, we'll make use of an OpenStack compute cloud to launch new instances:
--- - name: Create new foo servers hosts: localhost tasks: - name: launch instances
os_server:
name: foo-appv{{ version }}-{{ item }}
image: foo-appv{{ version }}
flavor: 4
key_name: ansible-prod
security_groups: foo-app
auto_floating_ip: false
state: present
auth:
auth_url: https://me.openstack.blueboxgrid.com:5001/v2.0
username: jlk
password: FAKEPASSW0RD
project_name: mastery
register: launch
loop: "{{ range(1, 8 + 1, 1)|list }}"
In this task, we're looping over a count of 8, using the new loop with range syntax that was introduced in Ansible 2.5. Each loop in the item variable will be replaced by a number. This allows us to create eight new server instances with names based on the version of our application and the number of the loop. We're also assuming a prebuilt image to use, so that we do not need to do any further configuration on the instance. In order to use the servers in future plays, we need to add their details to the inventory. To accomplish this, we register the results of the run in the launch variable, which we'll use to create runtime inventory entries:
- name: add hosts
add_host:
name: "{{ item.openstack.name }}"
ansible_ssh_host: "{{ item.openstack.private_v4 }}"
groups: new-foo-app
with_items: launch.results
This task will create new inventory items with the same names as those of our server instance. To help Ansible know how to connect, we'll set ansible_ssh_host to the IP address that our cloud provider assigned to the instance (this is assuming that the address is reachable by the host running Ansible). Finally, we'll add the hosts to the new-foo-app group. As our launch variable comes from a task with a loop, we need to iterate over the results of that loop by accessing the results key. This allows us to loop over each launch action to access the data specific to that task.
Next, we'll operate on the servers to ensure that the new service is ready for use. We'll use wait_for again, just like we did earlier, as a part of a new play on our new-foo-app group:
- name: Ensure new app
hosts: new-foo-app
tasks:
- name: ensure healthy service
wait_for:
port: 80
Once they're all ready to go, we can reconfigure the load balancer to make use of our new servers. For the sake of simplicity, we will assume a template for the haproxy configuration that expects hosts in a new-foo-app group, and the end result will be a configuration that knows all about our new hosts and forgets about our old hosts. This means that we can simply call a template task on the load balancer system itself, rather than attempting to manipulate the running state of the balancer:
- name: Configure load balancer hosts: foo-lb tasks:
- name: haproxy config
template:
dest: /etc/haproxy/haproxy.cfg
src: templates/etc/haproxy/haproxy.cfg
- name: reload haproxy
service:
name: haproxy
state: reloaded
Once the new configuration file is in place, we can issue a reload of the haproxy service. This will parse the new configuration file and start a new listening process for new incoming connections. The existing connections will eventually close, and the old processes will terminate. All new connections will be routed to the new servers running our new application version.
This playbook can be extended to decommission the old version of the servers, or that action may happen at a different times when it has been decided that a rollback to the old version capability is no longer necessary.
The expand and contract strategy can involve more tasks, and even separate playbooks for creating a golden image set, but the benefits of a fresh infrastructure for every release far outweigh the extra tasks or added complexity of creation followed by deletion.