For each build command in the Dockerfile, Docker will generate a new filesystem layer. An image is a combination of all filesystem layers created by the build commands of the Dockerfile. Each layer is mapped to a specific build command in the Dockerfile.

The size of each filesystem layer depends on the command executed (e.g., adding files, installing libraries, etc.). The final image size is the sum of the sizes of all its filesystem layers (including the deleted layers). Adding commands to delete files created in previous layers does not reduce the final image size, because each layer is immutable and retained in the image history.

For a new created image, Docker will execute all the build commands in the Dockerfile.

When building an already created image (assuming the cache was not cleared), Docker uses, for each instruction in the Dockerfile, the corresponding filesystem layer from its cache, except if:

• The instruction itself changes.
  
  
• Any file content that affects the build context changes.
  
  
• A parent layer was rebuilt (cache invalidation cascades down).
  
  

If any of the above conditions are met, Docker will invalidate the cache and re-run the instruction and all the instructions in the Dockerfile that are below it.

Allowing Docker to use cached layers when rebuilding images can significantly reduce build time.

Best practices: To minimize invalidating the cache, a build command that might create a new filesystem layer every time it's executed (e.g., update OS, library) should be placed, when possible, at the bottom of the Dockerfile. Another option is to place that command in a parent Dockerfile and use the generated image as a parent image for the descendant Dockerfiles. In such case case, the parent image can be re-built only when needed.

Best practices: When possible, multiple build commands should be combined in one single build command (using && operator).

Following best practices when designing Dockerfiles helps:

• speed up the creation of Docker images by leveraging the cache.
  
  
• reduce the size of Docker images.
  
  
• speed up copying/downloading Docker images by skipping layers that have already been copied/downloaded.
  
  

Let's execute again the build command. The output should be similar to the following:

Sending build context to Docker daemon  10.75kB
Step 1/6 : FROM ubuntu:latest
 ---> 6015f66923d7
Step 2/6 : RUN apt-get -y update
 ---> Using cache
 ---> b34542b1b9fb
Step 3/6 : RUN apt-get -y install curl
 ---> Using cache
 ---> 9e0f805a3972
Step 4/6 : RUN apt-get -y install nginx
 ---> Using cache
 ---> 01756a2038d9
Step 5/6 : RUN groupadd -r mtitek --gid=1001
 ---> Using cache
 ---> 41ca8f17b674
Step 6/6 : RUN useradd -r -g mtitek --uid=1001 mtitek
 ---> Using cache
 ---> 7b666fb64cbe
Successfully built 7b666fb64cbe
Successfully tagged ubuntu-nginx:latest

As you can see, Docker verify for each build step if there is a cached layer, and if it does exists, it will use it (see ---> Using cache).

Let's change the Dockerfile a bit (I will change the order of the instructions):

$ vi Dockerfile

FROM ubuntu:latest

RUN groupadd -r mtitek --gid=1001
RUN useradd -r -g mtitek --uid=1001 mtitek

RUN apt-get -y update

RUN apt-get -y install curl
RUN apt-get -y install nginx

Let's execute the build command. The output should be similar to the following:

Sending build context to Docker daemon  10.75kB
Step 1/6 : FROM ubuntu:latest
 ---> 6015f66923d7
Step 2/6 : RUN groupadd -r mtitek --gid=1001
 ---> Running in f7b327d63845
 ---> Removed intermediate container f7b327d63845
 ---> 4629c2db165c
Step 3/6 : RUN useradd -r -g mtitek --uid=1001 mtitek
 ---> Running in 01c98ebe495c
useradd warning: mtitek's uid 1001 is greater than SYS_UID_MAX 999
 ---> Removed intermediate container 01c98ebe495c
 ---> 2edc548cef67
Step 4/6 : RUN apt-get -y update
 ---> Running in 33ee26d60ba9
...
 ---> Removed intermediate container 33ee26d60ba9
 ---> 2ec87303ed4c
Step 5/6 : RUN apt-get -y install curl
 ---> Running in 99ff38439160
...
 ---> Removed intermediate container 99ff38439160
 ---> fc77c62a95d1
Step 6/6 : RUN apt-get -y install nginx
 ---> Running in fa1e572b2c54
...
 ---> Removed intermediate container fa1e572b2c54
 ---> fd3be710b6e8
Successfully built fd3be710b6e8
Successfully tagged ubuntu-nginx:latest

As you can notice, the instruction of the step 2 was changed (because of reordering) and hence Docker is building a new image layer. Docker will then execute the subsequent steps and won't use the local cache even if the instructions themselves didn't change.

Note: You can use the --no-cache option to instruct Docker not to use the cache from previous builds.

The example of the Dockerfile we used above is unoptimized:

$ vi Dockerfile

FROM ubuntu:latest

RUN apt-get -y update

RUN apt-get -y install curl
RUN apt-get -y install nginx

RUN groupadd -r mtitek --gid=1001
RUN useradd -r -g mtitek --uid=1001 mtitek

Let's inspect the layers of the ubuntu image (ubuntu:latest):

$ docker image inspect ubuntu:latest --format '{{json .RootFS.Layers}}' | jq

[
  "sha256:8901a649dd5a9284fa6206a08f3ba3b5a12fddbfd2f82c880e68cdb699d98bfb"
]

Let's inspect the layers of the new image (ubuntu-nginx:latest):

$ docker image inspect ubuntu-nginx:latest --format '{{json .RootFS.Layers}}' | jq

[
  "sha256:8901a649dd5a9284fa6206a08f3ba3b5a12fddbfd2f82c880e68cdb699d98bfb",
  "sha256:3f8c7084c7bbc943bd6afed212591874b303398bc7e0ea95c43e4033c9fd2d3a",
  "sha256:7bf6682df91a9076c66f394cac225db5b878b96de5041c494adc8e4acd5c0f16",
  "sha256:798623fc5905c87438699e37cc35fec5bff9bc5bb20d2d8e9752b47cad36ce8d",
  "sha256:f48e14b4afc200c8022b3e36d9f29924afaae221aeb1a1e9eb3d996a8587213c",
  "sha256:6468f3e1a65da2cab178429ce66b9c057e2a9ec3faeeeb1f199b772beb3321b5"
]

Note that the image 'ubuntu-nginx:latest' inherit all the layers of the parent image 'ubuntu:latest'.

The five new layers are the ones created by executing the build steps in the Dockerfile.

Let's check the size of the layers of the new image (ubuntu-nginx:latest):

$ docker image history ubuntu-nginx:latest

IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT
aa946f62cb3c   21 minutes ago   /bin/sh -c useradd -r -g mtitek --uid=1001 m…   24.6kB
4aebef1f834f   21 minutes ago   /bin/sh -c groupadd -r mtitek --gid=1001        24.6kB
6b9093ec8233   21 minutes ago   /bin/sh -c apt-get -y install nginx             7.82MB
157c3399ec8a   21 minutes ago   /bin/sh -c apt-get -y install curl              10.9MB
6751ea0c7fc3   21 minutes ago   /bin/sh -c apt-get -y update                    48.9MB
6015f66923d7   5 weeks ago      /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B
<missing>      5 weeks ago      /bin/sh -c #(nop) ADD file:ad85a9d7b0a74c214…   87.6MB
<missing>      5 weeks ago      /bin/sh -c #(nop)  LABEL org.opencontainers.…   0B
<missing>      5 weeks ago      /bin/sh -c #(nop)  LABEL org.opencontainers.…   0B
<missing>      5 weeks ago      /bin/sh -c #(nop)  ARG LAUNCHPAD_BUILD_ARCH     0B
<missing>      5 weeks ago      /bin/sh -c #(nop)  ARG RELEASE                  0B

Check the image size:

$ docker image ls ubuntu-nginx:latest

REPOSITORY     TAG       IMAGE ID       CREATED         SIZE
ubuntu-nginx   latest    aa946f62cb3c   3 minutes ago   226MB

Note that deleting temporary files in later layers won't reduce the image size. Let's demonstrate this by updating the Dockerfile above:

$ vi Dockerfile

FROM ubuntu:latest

RUN apt-get -y update

RUN apt-get -y install curl
RUN apt-get -y install nginx

RUN groupadd -r mtitek --gid=1001
RUN useradd -r -g mtitek --uid=1001 mtitek

RUN rm -rf /var/lib/apt/lists/*

Let's build the new Dockerfile:

$ DOCKER_BUILDKIT=0 docker build -t ubuntu-nginx:latest .

Sending build context to Docker daemon  10.75kB
Step 1/7 : FROM ubuntu:latest
 ---> 6015f66923d7
Step 2/7 : RUN apt-get -y update
 ---> Using cache
 ---> 6751ea0c7fc3
Step 3/7 : RUN apt-get -y install curl
 ---> Using cache
 ---> 157c3399ec8a
Step 4/7 : RUN apt-get -y install nginx
 ---> Using cache
 ---> 6b9093ec8233
Step 5/7 : RUN groupadd -r mtitek --gid=1001
 ---> Using cache
 ---> 4aebef1f834f
Step 6/7 : RUN useradd -r -g mtitek --uid=1001 mtitek
 ---> Using cache
 ---> aa946f62cb3c
Step 7/7 : RUN rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 93bccff9a921
Successfully built 93bccff9a921
Successfully tagged ubuntu-nginx:latest

Let's inspect the layers of the new image (ubuntu-nginx:latest):

$ docker image inspect ubuntu-nginx:latest --format '{{json .RootFS.Layers}}' | jq

[
  "sha256:8901a649dd5a9284fa6206a08f3ba3b5a12fddbfd2f82c880e68cdb699d98bfb",
  "sha256:3f8c7084c7bbc943bd6afed212591874b303398bc7e0ea95c43e4033c9fd2d3a",
  "sha256:7bf6682df91a9076c66f394cac225db5b878b96de5041c494adc8e4acd5c0f16",
  "sha256:798623fc5905c87438699e37cc35fec5bff9bc5bb20d2d8e9752b47cad36ce8d",
  "sha256:f48e14b4afc200c8022b3e36d9f29924afaae221aeb1a1e9eb3d996a8587213c",
  "sha256:6468f3e1a65da2cab178429ce66b9c057e2a9ec3faeeeb1f199b772beb3321b5",
  "sha256:deef6270e5c6912ae672d7f30d4e90c5160efcfa0ef43423474535e3bef125a9"
]

You can notice that a new layer (deef6270e...) was added for the command rm -rf /var/lib/apt/lists/*

Let's check the size of the layers of the new image (ubuntu-nginx:latest):

$ docker image history ubuntu-nginx:latest

IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT
93bccff9a921   16 minutes ago   /bin/sh -c rm -rf /var/lib/apt/lists/*          20.5kB
aa946f62cb3c   24 minutes ago   /bin/sh -c useradd -r -g mtitek --uid=1001 m…   24.6kB
4aebef1f834f   24 minutes ago   /bin/sh -c groupadd -r mtitek --gid=1001        24.6kB
6b9093ec8233   24 minutes ago   /bin/sh -c apt-get -y install nginx             7.82MB
157c3399ec8a   24 minutes ago   /bin/sh -c apt-get -y install curl              10.9MB
6751ea0c7fc3   24 minutes ago   /bin/sh -c apt-get -y update                    48.9MB
6015f66923d7   5 weeks ago      /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B
<missing>      5 weeks ago      /bin/sh -c #(nop) ADD file:ad85a9d7b0a74c214…   87.6MB
<missing>      5 weeks ago      /bin/sh -c #(nop)  LABEL org.opencontainers.…   0B
<missing>      5 weeks ago      /bin/sh -c #(nop)  LABEL org.opencontainers.…   0B
<missing>      5 weeks ago      /bin/sh -c #(nop)  ARG LAUNCHPAD_BUILD_ARCH     0B
<missing>      5 weeks ago      /bin/sh -c #(nop)  ARG RELEASE                  0B

Let's check the size of the new image:

$ docker image ls ubuntu-nginx:latest

REPOSITORY     TAG       IMAGE ID       CREATED         SIZE
ubuntu-nginx   latest    93bccff9a921   2 minutes ago   226MB

The image size remains the same because the sizes of the filesystem layers are additive.

Let's optimize a bit the Dockerfile we used above:

$ vi Dockerfile-optimized

FROM ubuntu:latest

RUN groupadd -r mtitek --gid=1001 && useradd -r -g mtitek --uid=1001 mtitek

RUN apt-get -y update && apt-get -y install curl && apt-get -y install nginx && rm -rf /var/lib/apt/lists/*

Note that I have organized the commands and chained some together using the && operator.
All chained commands will be executed, and they all need to succeed (exit with a return code of 0).
The && operator instructs Docker to execute the subsequent command in the chain only if the preceding command is successful.

Build the Dockerfile:

$ DOCKER_BUILDKIT=0 docker build -t ubuntu-nginx-optimized:latest -f Dockerfile-optimized .

Sending build context to Docker daemon  10.75kB
Step 1/3 : FROM ubuntu:latest
 ---> 6015f66923d7
Step 2/3 : RUN groupadd -r mtitek --gid=1001 && useradd -r -g mtitek --uid=1001 mtitek
 ---> Running in 3cb6ef9fb342
useradd warning: mtitek's uid 1001 is greater than SYS_UID_MAX 999
 ---> Removed intermediate container 3cb6ef9fb342
 ---> c6a8431a0584
Step 3/3 : RUN apt-get -y update && apt-get -y install curl && apt-get -y install nginx && rm -rf /var/lib/apt/lists/*
 ---> Running in 309918e8ba28
...
 ---> Removed intermediate container 309918e8ba28
 ---> 98fb588c7970
Successfully built 98fb588c7970
Successfully tagged ubuntu-nginx-optimized:latest

Let's inspect the layers of the new image:

$ docker image inspect ubuntu-nginx-optimized:latest --format '{{json .RootFS.Layers}}' | jq

[
  "sha256:8901a649dd5a9284fa6206a08f3ba3b5a12fddbfd2f82c880e68cdb699d98bfb",
  "sha256:75b8e003ac9658ae32fa701a2021cd0ae73bf82eaf82e6667d694e958c2ed837",
  "sha256:257b99b3df8dcd52ca57748d038ec4f53914352ce846fa5b49c5760d2b94b2b3"
]

First note that the build steps were reduced to only three steps. This result to only two new layers created for the new image instead of five with the non-optimized Dockerfile.

Note that the steps, in the Dockerfile, to create the mtitek group and user are placed before the installation instructions of curl and nginx. This is to show that the instructions that are more likely to generate the same exact result, no matter how many time they are executed, should be placed at the top of the Dockerfile. This will save time when building the image frequently.

Note also the instruction rm -rf /var/lib/apt/lists/*, which is useful to save disk space on the final image. The size of the image is now 141MB instead of 226MB.

$ docker image ls ubuntu-nginx-optimized:latest

REPOSITORY               TAG       IMAGE ID       CREATED         SIZE
ubuntu-nginx-optimized   latest    98fb588c7970   3 minutes ago   141MB

Let's check the size of the layers of the new image (ubuntu-nginx-optimized:latest):

$ docker image history ubuntu-nginx-optimized:latest

IMAGE          CREATED         CREATED BY                                      SIZE      COMMENT
98fb588c7970   6 minutes ago   /bin/sh -c apt-get -y update && apt-get -y i…   17.2MB
c6a8431a0584   6 minutes ago   /bin/sh -c groupadd -r mtitek --gid=1001 && …   41kB
6015f66923d7   5 weeks ago     /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B
<missing>      5 weeks ago     /bin/sh -c #(nop) ADD file:ad85a9d7b0a74c214…   87.6MB
<missing>      5 weeks ago     /bin/sh -c #(nop)  LABEL org.opencontainers.…   0B
<missing>      5 weeks ago     /bin/sh -c #(nop)  LABEL org.opencontainers.…   0B
<missing>      5 weeks ago     /bin/sh -c #(nop)  ARG LAUNCHPAD_BUILD_ARCH     0B
<missing>      5 weeks ago     /bin/sh -c #(nop)  ARG RELEASE                  0B