Whenever you create a CodeBuild project, you must define how CodeBuild should test and build your application source code, and then publish application artifacts and/or Docker images. CodeBuild defines these tasks within a build specification, which provides the build instructions the CodeBuild agent should execute when running a build.
CodeBuild allows you to provide a build specification in several ways:
- Self-defined: CodeBuild looks for a file that is defined within the source repository of the project. By default, this is a file called buildspec.yml; however, you can also configure a custom file where your build specification is located.
- Preconfigured: When you create a CodeBuild project, you can define a build specification as part of your project setup.
- On demand: If you initiate a CodeBuild build job using the AWS CLI or SDK, you can override the preconfigured or self-defined build specification
In general, I recommend using the self-defined method, as it allows the repository owner (typically, your developers) to configure and maintain the specification independently of CodeBuild; this is the approach we will take.
The following example demonstrates adding a build specification to the todobackend repository, in a file called buildspec.yml:
version: 0.2
phases:
pre_build:
commands:
- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --storage-driver=overlay&
- timeout -t 15 sh -c "until docker info; do echo .; sleep 1; done"
- export BUILD_ID=$(echo $CODEBUILD_BUILD_ID | sed 's/^[^:]*://g')
- export APP_VERSION=$CODEBUILD_RESOLVED_SOURCE_VERSION.$BUILD_ID
- make login
build:
commands:
- make test
- make release
- make publish
post_build:
commands:
- make clean
- make logout
The build specification starts by specifying a version that must be included in every build specification, the most current version being 0.2, as of the writing of this book. Next, you define the phases sequence, which is required, defining the commands that CodeBuild will run during the various phases of the build. In the previous example, you define three phases:
- pre_build: Commands that CodeBuild will run before the build. Here, you can run commands such as logging into ECR, or any other commands that are required for your build to run successfully.
- build: These commands run your build steps.
- post_build: Commands that CodeBuild will run after your build. These typically involve clean up tasks, such as logging out of ECR and removing temporary files.
You can find more information about the CodeBuild build specifications at https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html.
During the pre_build stage, you perform the following actions:
- The first two commands are used to start the Docker daemon in your custom CodeBuild image; the nohup command starts the Docker daemon as a background task, while the timeout command is used to ensure the Docker daemon has started successfully, before attempting to continue.
- Export a BUILD_ID environment variable, which is used to add build information to the application version that will be generated for your build. This BUILD_ID value will be added to the application version tag that is attached to the Docker image that is built during the build phase, and therefore, it can only include characters that are compatible with Docker's tag format. The CodeBuild job ID is exposed to your build agent via the CODEBUILD_BUILD_ID environment variable, and has the format <project-name>:<job-id>, where <job-id> is a UUID value. The colon in the CodeBuild job ID is not supported in Docker tags; hence, you strip the <project-name>: portion of the job ID using a sed expression, leaving just the job ID value that will be included in the Docker tag.
- Export the APP_VERSION environment variable, which is used in the Makefile to define the application version that is tagged on the built Docker image. When you use CodeBuild with CodePipeline, it is important to understand that the source artifact presented to CodeBuild is actually a zipped version located in an S3 bucket that CodePipeline creates after cloning the source code from your source repository. CodePipeline does not include any Git metadata; therefore, the APP_VERSION directive in the todobackend Makefile - export APP_VERSION ?= $(shell git rev-parse --short HEAD - will fail, as the Git client will not have any Git metadata available. Luckily, the ?= syntax in GNU Make means to use the value of the aforementioned environment variable, if it is already defined in the environment. So, we can export APP_VERSION in the CodeBuild environment, and Make will just use the configured value, rather than run the Git commands. In the previous example, you construct the APP_VERSION from a variable called CODEBUILD_RESOLVED_SOURCE_VERSION, which is the full commit hash of the source repository, and is set by CodePipeline. You also append the BUILD_ID variable calculated in the previous command, which allows you to trace a specific Docker image build to a CodeBuild build job.
- Log in to ECR using the make login command included in the source repository.
Once the pre_build stage has completed, the build stage is straightforward, and simply executes the various build steps that we have executed manually so far in this book. The final post_build stage runs the make clean task to tear down the Docker Compose environment, and then removes any local ECR credentials by running the make logout command.
One important point to note is that the post_build stage always runs, even if the build stage fails. This means you should only reserve post_build tasks for actions that you would run regardless of whether the build passes or fails. For example, you might be tempted to run the make publish task as a post_build step; however, if you do this, and the previous build stage fails, CodeBuild will still attempt to run the make publish task, given that it is defined as a post_build step. Placing the make publish task as the final action in the build stage ensures that if make test or make release fails, the build stage will immediately exit with an error, bypassing the make publish action and proceeding to execute the cleanup tasks in the post_build step.
You can find out more about all of the CodeBuild phases, and whether they execute on success/failure, at https://docs.aws.amazon.com/codebuild/latest/userguide/view-build-details.html#view-build-details-phases.
The final step that you need to perform is to commit and push your changes to your Git repository, so that the newly created buildspec.yml file will be available when you configure CodePipeline and CodeBuild:
> git add -A
> git commit -a -m "Add build specification"
[master ab7ac16] Add build specification
1 file changed, 19 insertions(+)
create mode 100644 buildspec.yml
> git push
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 584 bytes | 584.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To github.com:docker-in-aws/todobackend.git
5fdbe62..ab7ac16 master -> master