The process of containerization is ubiquitous. It has soaked in almost every branch of software development. It is therefore not surprising that 50% of vulnerable targets are attacked within one hour. That’s why it is good to know the basic rules of creating images for the containers. This article is willing to cover such best approaches to Dockerfile creation. It is going to be kept in the convention of cookbook/handbook. There won’t be much in-depth diving into how container technology works or each of the introduced tools. Rather some quick and sharp methods that can be introduced in your everyday development process. Some tips, which will be given here, are sometimes based on my own experience. If you disagree with anything or see a better way you can freely share that knowledge in the comment section or on Slack.

This article was written originally for Netguru.

Table of Contents

1. Approach
2. Steps
   1. Step 1
   2. Step 2
   3. Step 3
   4. Step 4
   5. Step 5
   6. Step 6
   7. Step 7
   8. Step 8
   9. Step 9
   10. Step 10
3. Summary
4. Extending sources

Approach

What we want to achieve is the most secure and lightweight image, which will keep its functionality. Let’s start defining from the end.

1. By functionality, I mean that the app from simple-web-app will be running in the container, and its endpoint will be working.
2. Lightweight image means faster builds and better security (smaller attack surface). We will obtain the size of each image from the output of the docker images command.
3. For security scanning I picked up trivy scanner. The amount of found vulnerabilities will be the measurement of security quality. Also, the SBOM will be generated by the Docker feature docker sbom for each image to measure its amount of dependencies and potential attack surface.

Our target is to obtain the lowest size of our initial image, the lowest amount of found vulnerabilities by the Trivy scanner and the lowest amount of dependencies in SBOM generated by docker sbom with functionality maintained.

The command used to build images is docker buildx build --tag <tag-name> . unless told it is different. The buildx in the command is making use of the BuildKit feature to create the image.

All code related to this article can be found here.

Steps

Step 1 - init Docker image

FROM python
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
ENTRYPOINT ["uvicorn"]
CMD ["main:app", "--host", "0.0.0.0", "--port", "80"]

For sure, it is not production-ready, although the output of running such an image is a “working” application. It has a lot of space to improve. We will try to cover it as much as we can. Let’s start with explaining every instruction from the Dockerfile in more detail:

• FROM python - initialize parent image as official python image from Docker Hub
• WORKDIR /app - set workdir to /app
• COPY requirements.txt . - copy the requirements.txt file from the local machine to the container filesystem
• RUN pip install -r requirements.txt - run command installing requirements inside the container
• COPY . . - copy all the files from the current location in the local machine to the container filesystem
• ENTRYPOINT ["uvicorn"] - entry point command that is starting an uvicorn web server
• CMD ["main:app", "--host", "0.0.0.0", "--port", "80"] - args to the entry point command which are telling to start a web server on 0.0.0.0:80 inside the container network (we need to map it f.e. while doing docker run)

Let’s init our quality table:

Step 2 - Parent Image

Let’s focus on the first instruction in our Dockerfile - FROM python. It is setting the parent image as python, with the latest version. Such a strategy is very error-prone:

• you don’t know what version of python you are downloading.
• with every minor version update of python you will download such a change with every build - your code may not be ready for that.
• you don’t know what distro is under your image.
• your level of control over this image is very low.

It may cause weird scenarios where one day your code works, and another it doesn’t despite the lack of changes in it (at least from your perspective). That’s why we want to have the highest level of control over our parent image.

#line below has changed
FROM python:3.11-slim-bullseye 
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
ENTRYPOINT ["uvicorn"]
CMD ["main:app", "--host", "0.0.0.0", "--port", "80"]

We added the newest stable version - 3.11 and the latest stable Debian distribution slim-bullseye. The decision to pick up slim over alpine was made because of the performance issues with the latter one.

From a security point of view, you should NOT use unofficial images. The official ones are curated and their quality standards are the responsibility of some serious entities (at least they are saying so). But… if you really need to use some random image, you MUST build it on your own. You need to possess the Dockerfile (and, of course, validate it) and run the docker build on your own. Do NOT use the already-built images. Mostly because you can’t say:

• what is inside, is there any malware code
• how this image has been built
• who is responsible for keeping all updates up-to-date
• is this image up-to-date with security updates

In short, it means you are giving away lots of the control to some random character from the Internet. It is the situation you want to avoid.

Let’s add another row to our table.

Step 3 - Caching

Docker is using sequential layer caching. To utilize this functionality, the dependencies import and installation are separated from the source code import.

FROM python:3.11-slim-bullseye

WORKDIR /app
#here we are importing and then installing deps
COPY requirements.txt ./
RUN pip install -r requirements.txt
#here we are importing our source code (generally all files from project for now)
COPY . .

ENTRYPOINT ["uvicorn"]
CMD ["main:app", "--host", "0.0.0.0", "--port", "80"]

Thanks to this Docker, with each change in the source code, will “be aware” that the change is only affecting the layer linked to the instruction (COPY . .) of copying the code. Because of that, it will start rebuilding the image from that step. The layer responsible for deps will be left untouched, as it is happening before the code import.

Another feature is caching your dependencies across different builds. With frequently changing dependencies, the current Dockerfile will try to download the previously downloaded packages in every run. It is the behavior we want to avoid, as these packages were already at least once downloaded. We don’t want to waste our time on the same thing twice.

To describe it more preciously:

Now, to raise some awareness in our dependency manager, we will use the dependencies cache mechanism:

FROM python:3.11-slim-bullseye

WORKDIR /app

COPY requirements.txt .
#change in the line below
RUN --mount=type=cache,target=/root/.cache \
 pip install -r requirements.txt

COPY . .

ENTRYPOINT ["uvicorn"]
CMD ["main:app", "--host", "0.0.0.0", "--port", "80"]

--mount=type=cache is pretty self-descriptive, we are dealing with cache, and thus we are using this type. target=/root/.cache - this is the directory, in which are dependencies will be stored ACROSS all builds. Thanks to this, we won’t experience re-downloading packages, which are already in use.

If you run the scenario described above you will notice a change in the behavior of the build run. The logs are now telling us

#8 1.919 Collecting fastapi==0.88.0
#8 1.920   Using cached fastapi-0.88.0-py3-none-any.whl (55 kB)

for package already downloaded, instead of

#8 2.088 Collecting fastapi==0.88.0
#8 2.153   Downloading fastapi-0.88.0-py3-none-any.whl (55 kB)
#8 2.164      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.5/55.5 kB 5.5 MB/s eta 0:00:00

when we were building the image without caching.

After this point, we are reverting requirements.txt to its initial state (without django).

And another one, just for statistics.

Step 4 - Multi-stage builds

Our Dockerfile is doing everything in one go: building dependencies and then launching the app. Because of that, it has common environment for both of these activities. With mutli-stage builds we can separate both environments and actions from each other. Thanks to this we can:

1. have more constant env over time for building deps
2. abandon some tools only needed while building deps in our final image, which leads to less sized image
3. better readability
4. more control over each env

FROM python:3.11-slim-bullseye AS base

FROM base AS builder
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache \
  pip install --no-warn-script-location -r requirements.txt

FROM python:3.11-slim-bullseye as final-image
WORKDIR /app
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY . .
ENV PYTHONPATH=/usr/local/lib/python3.11/site-packages \
    PATH="/usr/local/bin:${PATH}"
ENTRYPOINT ["uvicorn"]
CMD ["main:app", "--host", "0.0.0.0", "--port", "80"]

Whew, a little change, right? Let’s go through it. The first image is base and possesses the dependencies needed to create a python env. It is the place where you can f.e. define python envs or upgrade your dependency manager (pip). The second image is named builder and its responsibility is to build all project dependencies. We are installing all dependencies based on requirements.txt with pip as usual. --no-warn-script-location is just a cosmetic change to hide the error message related to the PATH environment variable. The third and final image is the moment when we are launching our app. But, contrary to the previous Dockerfiles, we are not installing here any dependencies. We can just copy already prepared libs and binaries from the previous image. Thanks to this we can drop dependencies related to the builder image like pip. We will make use of this in future steps.

Dockerfile seems to be larger now, but will it echo the actual size of the image?

The quality did not change, because our final image is still based on python:3.11-slim-bullseye, despite the fact it does not need to. Let’s make full use of the introduced multi-staging.

Step 5 - ‘Distroless’ image

Every container got its own OS. Usually, these OSs have lots of packages or binaries that are not needed for our app to run. In the end, we only need our source code and runtime dependencies. And here comes distroless image. They are images based on Debian with erased almost every program you would expect to find in the standard Linux distribution. Thanks to this they are very small, and this is what we are expecting from our image, right?

FROM python:3.11-slim-bullseye AS base

FROM base AS builder
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache \ 
  pip install --no-warn-script-location -r requirements.txt
#new image used below
FROM gcr.io/distroless/python3-debian11 as final-image
WORKDIR /app
COPY --from=builder /usr/local/bin/uvicorn /usr/local/bin
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
ENV PYTHONPATH=/usr/local/lib/python3.11/site-packages \
    PATH="/usr/local/bin:${PATH}"
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]

We didn’t need to change a lot, mostly because we already prepared for this change in the last step. Who would guess…? Let’s check how will it affect the quality of our final image.

We are progressing, that’s for sure. But still far from perfect. Let’s take a look at more security aspects of our image before a final clash with its size.

Step 6 - Build context

So far our command used to build the image has been docker buildx build --tag <tag-name> .. The last character of it

• . is our build context. It is telling Docker to take everything from our current directory and use it in the image build process. It can be a very dangerous approach, because sometimes we may have files, we don’t want to put into containers, like secrets, Excel files with data, random images, or .git history. In order to avoid it, we can follow two approaches.

1. Use a more precise build context. Right now we are copying everything from the build context into the container (COPY . .). To have more control over what we are taking, we should use the COPY method more explicitly - COPY ./src .. Thanks to a little refactor we have our source code in one directory, which can be explicitly copied into the image.
2. Use .dockerignore Philosophy is the same as with .gitignore - you are adding files, which should be ignored by the docker build context.

Let’s check how this tiny change will influence our image.

A little step further…

Step 7 - System packages

When running an app based on a more complex image than distroless or scratch you should keep your OS dependencies up-to-date. Theoretically, it is the responsibility of the image maintainer, but it is a good approach to do it by yourself.

RUN apt-get update && apt-get install -y --no-install-recommends && rm -rf /var/lib/apt/lists/*

--no-install-recommends is used here, because we don’t want to install any additional packages. /var/lib/apt/lists is the directory, where the package information cache is being stored. After the update, we don’t need it anymore so it can be deleted to reduce a little the size of our image.

Since we are using distroless image we even can’t trigger such a command, because our base image doesn’t have package manager anymore. But… we can care of the python dependencies manager and make it up-to-date.

FROM python:3.11-slim-bullseye AS base
#here we added updating pip
RUN pip install --upgrade pip

FROM base AS builder
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache \
  pip install --no-warn-script-location -r requirements.txt

FROM gcr.io/distroless/python3-debian11 as final-image
WORKDIR /app
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=builder /usr/local/bin/uvicorn /usr/local/bin/uvicorn
COPY /src .
ENV PYTHONPATH=/usr/local/lib/python3.11/site-packages \
    PATH="/usr/local/bin:${PATH}"
CMD ["/usr/local/bin/uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]

The added command is upgrading pip. For formalities, let’s add our new image to the table.

Step 8 - Non-root user

Default user being used by Docker while running instructions from Dockerfile is root. As we want to stick to the least privileged user, we do not want our app being run by root. Contrary to the building process, which can be done by root. This way, the user running the app will be able only to read our source code, not to manipulate it (without write access). In my opinion, it reduces the surface of potential harm caused in case of hijacking the container by some shady character.

Let’s check who currently is running our app.

# whoami 
root
# ls -la
total 16
drwxr-xr-x    1 root     root          4096 Dec 23 16:36 .
drwxr-xr-x    1 root     root          4096 Dec 23 16:36 ..
-rw-r--r--    1 root     root            21 Dec 23 16:39 main.py
# echo "print('Hello world')" > main.py
# python main.py
Hello world

As you may see in these logs, the user can add a new code and trigger it. It is the situation we need to avoid.

Usually, you could create a new user by example command:

RUN useradd -m python-user

and pin it to the Dockerfile entry point command just before running it with the instruction USER:

USER python-user
ENTRYPOINT ["uvicorn"]
CMD ["main:app", "--host", "0.0.0.0", "--port", "80"]

In distroless images, we only need to specify the tag :nonroot to run the app as a non-root user.

FROM python:3.11-slim-bullseye AS base
RUN pip install --upgrade pip

FROM base AS builder
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache \
  pip install --no-warn-script-location -r requirements.txt
# tag added below
FROM gcr.io/distroless/python3-debian11:nonroot as final-image
WORKDIR /app
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=builder /usr/local/bin/uvicorn /usr/local/bin/uvicorn
COPY /src .
ENV PYTHONPATH=/usr/local/lib/python3.11/site-packages \
    PATH="/usr/local/bin:${PATH}"
CMD ["/usr/local/bin/uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]

Now, the user triggering the app is not a root, but the source code belongs to the root. Thanks to this users can only read source code, not modify it. Thus, it can not insert any malicious code.

$ whoami
nonroot
$ ls -la
total 12
drwxr-xr-x    1 nonroot  nonroot       4096 Dec 23 16:47 .
drwxr-xr-x    1 root     root          4096 Dec 23 16:47 ..
-rw-r--r--    1 root     root           118 Dec 21 10:04 main.py
$ cat main.py   
from fastapi import FastAPI

app = FastAPI()


@app.get("/")
async def root():
    return {"message": "Hello World"}

$ echo "print('Hello world')" > main.py
sh: 5: cannot create main.py: Permission denied

It won’t influence our image size, but for sure it is a much improvement

Step 9 - Linting

Going through each of the steps with every new Dockerfile would be a very time-consuming and complex chore. Thus, we need automation of this process to some degree. Here comes the linting tool hadolint. It helps to ensure that Dockerfile is being written with aligning to best practices.

../linting/Dockerfile:2 DL3013 warning: Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>` or `pip install --requirement <requirements file>`
../linting/Dockerfile:2 DL3042 warning: Avoid use of cache directory with pip. Use `pip install --no-cache-dir <package>`
../linting/Dockerfile:7 DL3042 warning: Avoid use of cache directory with pip. Use `pip install --no-cache-dir <package>`

Linter is telling us that we got three issues with pip caching. All of them are false negatives. We will use the flag directly in the Dockerfile to suppress it.

FROM python:3.11-slim-bullseye AS base
# hadolint ignore=DL3013,DL3042
RUN pip install --upgrade pip

FROM base AS builder
WORKDIR /app
COPY requirements.txt .
# hadolint ignore=DL3042
RUN --mount=type=cache,target=/root/.cache \
  pip install --no-warn-script-location -r requirements.txt

FROM gcr.io/distroless/python3-debian11:nonroot as final-image
WORKDIR /app
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=builder /usr/local/bin/uvicorn /usr/local/bin/uvicorn
COPY /src .
ENV PYTHONPATH=/usr/local/lib/python3.11/site-packages \
    PATH="/usr/local/bin:${PATH}"
CMD ["/usr/local/bin/uvicorn", "main:app", "--host", "0.0.0.0", "--port", "80"]

Let’s check how aligning with the best practices influence quality of our image.

Step 10 - DockerSlim

Let’s recap our target. It is to build the most lightweight and secure image without breaking the functionality of the app inside. If we take a look at the SBOM of the previous image we can see, that there are many packages, we can not tell if we need them at the first sight. Is there a space to improve? The tool which can answer that question is DockerSlim.

Not only it can brush the image from useless deps, but it also can be used as a linter or to x-ray the image of its content. By default, it is creating Seccomp and AppArmor security profiles for the processed image. Generally, it is a wide tool, which, to fully understand, needs more time invested than this handbook.

Okay, let’s build the new image and use DockerSlim on it.

docker-slim build --target docker-slim:latest --expose 80

Port 80 is the only port used by our app. We need to expose it, so the docker-slim could check which packages are being “touched” by our app. Let’s check how is the output image presenting.

:nois:

Summary

Final table with each step.

We started from a 1.04 GB sized image with over 1k vulnerabilities and we went down to 19.4 MB and 0 vulns. The very last image is much closer to the production image than any other during this process. The quality increase is tremendous. During this journey, we learned a few tools, which can be used to automate the process of building the image, like hadolint or DockerSLim. We got familiar with the most common security approaches, like least privileged user, control over build context or Seccomp and AppArmor security profiles, which are lowering the potential harm caused by cyberattacks.

Extending sources

https://earthly.dev/blog/docker-slim/ - DockerSlim light intro
https://pythonspeed.com/ - blog about containerization and python efficiency
https://www.youtube.com/@DockerIo - Docker official channel
https://rootlesscontaine.rs/ - about rootless-mode https://dev.to/abdelino17/package-your-fastapi-application-with-distroless-docker-images-16k9 - multi-staging with Pipfile
https://www.merixstudio.com/blog/docker-multi-stage-builds-python-development/ - multi-staging with wheels
https://falco.org/ - runtime security monitoring of container