Docker cache on Travis and Docker 1.12

// under travis mozilla-planet ellak-planet

Edits

2016-09-12 Edited to prevent caching of builds in pull requests.
2017-07-07 Edited to point to exact git commit b/c we don't use Travis for Snippets Service anymore.

I blogged before about building Docker images on Travis and suggested uploading images after successful test runs to Docker Hub and use them as Cache after downloading them in next Travis runs.

Travis upgraded recently to Docker version 1.12 (from 1.9) and since version 1.10 Docker features Content Addressability for layers. This change breaks caching and we need to implement a workaround using Travis cache.

Changes need to be made in .travis.yml:

Enable cache

cache:
  directories:
    - /home/travis/docker/

Start by requesting Travis to cache /home/travis/docker directory.

Load Docker images previously cached

before_install:
  - if [ -f ${DOCKER_CACHE_FILE} ]; then gunzip -c ${DOCKER_CACHE_FILE} | docker load; fi

This checks that ${DOCKER_CACHE_FILE} exists and then loads it in Docker while gunzip-ing it. For convenience and smaller lines I did setup ${DOCKER_CACHE_FILE} as environment variable

env:
  global:
    - DOCKER_CACHE_FILE=/home/travis/docker/cache.tar.gz

Save image to cache

After building the image and running the tests it's time to save the new Docker image to Travis cache.

I chose to save only when I'm building the master branch but all branches and PRs will still get the cache as Travis will make the cache directory available to all builds.

script:
  # Tests go here
  - if [[ ${TRAVIS_BRANCH} == "master" ]] && [[ ${TRAVIS_PULL_REQUEST} == "false" ]]; then mkdir -p $(dirname ${DOCKER_CACHE_FILE}) ; docker save $(docker history -q ${DOCKER_REPOSITORY}:${TRAVIS_COMMIT} | grep -v '<missing>') | gzip > ${DOCKER_CACHE_FILE}; fi

Note that we don't just docker save the resulting image. Instead we need to save all the intermediate layers explicitly. We get all the layers using docker history and we grep out all the <missing> images.

It's important that docker save happens in script step because right after this step Travis will save the contents of the cached directory.

That's all!

Note that Docker saving and loading, as well as Travis uploading and downloading of cache from S3 costs time. It may be faster to rebuilt your Docker image instead of caching it.