Creating persistent storage for storing the application database

We can now shift our attention to the application database, which is an essential supporting component of the todobackend application. If you are running in AWS, my typical recommendation would be, regardless of container orchestration platform, to use the Relational Database Service (RDS) as we have done throughout this book, however the application database requirement for the todobackend application provides an opportunity to demonstrate how you can support persistent storage using the Docker for AWS solution.

In addition to EFS-backed volumes, the Cloudstor volume plugin also supports relocatable Elastic Block Store (EBS) volumes. Relocatable means that the plugin will automatically relocate the currently assigned EBS volume for a container to another node in the event Docker Swarm determines it has to relocate a container from one node to another. What actually happens during relocation of the EBS volume depends on the scenario:

It is important to note that Docker only supports singular access to a relocatable EBS-backed volume—that is, there should only ever be a single container that reads/writes to the volume at any given time. If you require shared access to a volume, then you must create an EFS-backed shared volume.

Now, let's define a volume called data to store the todobackend database, and create a db service that will run MySQL and attach to the data volume:

version: '3.6'

networks:
  net:
    driver: overlay

volumes:
  public:
    driver: cloudstor:aws
    driver_opts:
      backing: shared
  data:
    driver: cloudstor:aws
    driver_opts: 
      backing: relocatable
      size: 10
      ebstype: gp2

services:
  app:
    image: 385605022855.dkr.ecr.us-east-1.amazonaws.com/docker-in-aws/todobackend
    ports:
      - target: 8000
        published: 80
    networks:
      - net
    volumes:
      - public:/public
    ...
    ...
  collectstatic:
    image: 385605022855.dkr.ecr.us-east-1.amazonaws.com/docker-in-aws/todobackend
    volumes:
      - public:/public
    ...
    ...
  db:
    image: mysql:5.7
    environment:
      MYSQL_DATABASE: todobackend
      MYSQL_USER: todo
      MYSQL_PASSWORD: password
      MYSQL_ROOT_PASSWORD: password
    networks:
      - net
    volumes:
      - data:/var/lib/mysql
    command:
      - --ignore-db-dir=lost+found
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role == manager

First, we create a volume called data and configure the driver as cloudstor:aws. In the driver options, we specify a backing of relocatable to create an EBS volume, specifying a size of 10 GB and an EBS type of gp2 (SSD) storage. We then define a new service called db that runs the official MySQL 5.7 image, attaching the db service to the previously defined net network and mounting the data volume at /var/lib/mysql, which is where MySQL stores its database. Note that because the Cloudstor plugin formats the mounted volume as ext4, a folder called lost+found is automatically created during the formatting process, which causes the MySQL container to abort as it thinks an existing database called lost+found is present.

To overcome this, we pass in a single flag called --ignore-db-dir that references the lost+found folder, which is passed to the MySQL image entrypoint and configures the MySQL daemon to ignore this folder.

Finally, we define a placement constraint that will force the db service to be deployed to the Swarm manager, which will allow us to test the relocatable features of the data volume by changing this placement constraint to a worker later on.

If you now deploy the stack and monitor the db service, you should observe that the service takes some time to come up while the data volume is initializing:

> docker stack deploy --with-registry-auth -c stack.yml todobackend
docker stack deploy --with-registry-auth -c stack.yml todobackend
Updating service todobackend_app (id: 28vrdqcsekdvoqcmxtum1eaoj)
Updating service todobackend_collectstatic (id: sowciy4i0zuikf93lmhi624iw)
Creating service todobackend_db
> docker service ps todobackend_db --format "{{ .Name }} ({{ .ID }}): {{ .CurrentState }}"
todobackend_db.1 (u4upsnirpucs): Preparing 35 seconds ago
> docker service ps todobackend_db --format "{{ .Name }} ({{ .ID }}): {{ .CurrentState }}"
todobackend_db.1 (u4upsnirpucs): Running 2 seconds ago

To verify, an EBS volume has actually been created, you can use the AWS CLI as follows:

> aws ec2 describe-volumes --filters Name=tag:CloudstorVolumeName,Values=* \
    --query "Volumes[*].{ID:VolumeId,Zone:AvailabilityZone,Attachment:Attachments,Tag:Tags}"
[
    {
        "ID": "vol-0db01995ba87433b3",
        "Zone": "us-east-1b",
        "Attachment": [
            {
                "AttachTime": "2018-07-20T09:58:16.000Z",
                "Device": "/dev/xvdf",
                "InstanceId": "i-0dc762f73f8ce4abf",
                "State": "attached",
                "VolumeId": "vol-0db01995ba87433b3",
                "DeleteOnTermination": false
            }
        ],
        "Tag": [
            {
                "Key": "CloudstorVolumeName",
                "Value": "todobackend_data"
            },
            {
                "Key": "StackID",
                "Value": "0825319e9d91a2fc0bf06d2139708b1a"
            }
        ]
    }
]

Note that EBS volumes created by the Cloudstor plugin are tagged with a key of CloudstorVolumeName and a value of the Docker Swarm volume name. In the preceding example, you can also see that the volume has been created in the us-east-1b availability zone.