My Experience Getting Docker Images to Cache in AWS Codebuild using ECR

CodeBuild Logs

When it comes to speeding up Docker builds in AWS CodeBuild, caching is your best friend — or so I thought until I actually tried implementing it. My journey began with a guide from 2020, which you can read here. This guide introduced the ` — cache-from` flag and discussed using AWS ECR as a cache repository. Despite sounding promising, the cache manifest appeared empty.

Determined to resolve this, I found a 2023 article that went into greater detail about using AWS ECR for caching. You can read it here. It added valuable insights on configuring the --cache-to and --cache-fromcommands, which were essential to getting everything working properly for me.

Here’s the simplified pseudocode that captures the effective setup:

this.project = new Project(this, 'PrValidator', {
  environment: {
    buildImage: LinuxBuildImage.STANDARD_7_0,
    privileged: true,  // Necessary for Docker operations
  },

  buildSpec: BuildSpec.fromObject({
    version: '0.2',
    phases: {
      pre_build: {
        commands: [
          `aws ecr get-login-password | docker login --username AWS --password-stdin ${repositoryUri}`,
          `docker buildx create --use --name mybuilder --driver docker-container`,  // Crucial for enabling Buildx features
        ],
      },
      build: {
        commands: [
          `TAG="build-$CODEBUILD_BUILD_NUMBER"`,
          `docker buildx build --builder mybuilder --cache-from type=registry,ref=${repositoryUri}:cache --cache-to mode=max,image-manifest=true,oci-mediatypes=true,type=registry,ref=${repositoryUri}:cache --tag ${repositoryUri}:$TAG .`,  // Key for efficient caching
        ],
      },
    },
  });
);

Key Configurations Explained:

Privileged Mode: Essential for performing Docker operations within AWS CodeBuild. This setting allows Docker commands that require elevated permissions.

Docker Buildx Setup: The command docker buildx create --use --name mybuilder --driver docker-container initializes a Docker Buildx builder which is crucial for enabling advanced Docker features like build caching.

Caching Commands:

• --cache-from type=registry,ref=${repositoryUri}:cache Specifies where to pull the cache from, enhancing build speed by reusing previously computed results.
• --cache-to mode=max,image-manifest=true,oci-mediatypes=true,type=registry,ref=${repositoryUri}:cache Defines how to store the build cache, optimizing subsequent builds.

The real breakthrough in achieving efficient caching came from configuring Docker Buildx along with the precise setup of the --cache-from and --cache-to commands.

I hope my experience and the time spent figuring this out can help someone else facing similar challenges. So, if you’re trying to get Docker caching to work efficiently in CodeBuild, consider this approach — it could be just what you need!