Modify a Clear Linux OS-based container image

This guide describes how to customize Clear Linux* OS-based container images on Docker Hub, which include popular applications and runtimes.

Overview

Most of these images utilize a Docker build feature called a multi-stage build to reduce image size while some use single-stage build Dockerfiles. An official base clearlinux image on Docker Hub is also available. To create a generic Clear Linux OS container image, see our guide.

Prerequisites

  • Set up a functional Docker environment as described in Docker*.

  • Download the Clear Linux OS microservice Dockerfile repo with the following command:

    git clone https://github.com/clearlinux/dockerfiles.git

  • Navigate to and operate from the cloned dockerfiles directory.

    cd dockerfiles/

Example 1: Add a bundle

In this example, we add wget to the clearlinux/redis Dockerfile.

  1. Enter swupd search wget to discover which Clear Linux OS bundle includes the software. The output should tell you that wget is available in the wget bundle.

  2. Open a an editor to modify the Dockerfile.

    $EDITOR redis/Dockerfile

  3. Append the wget bundle to the --bundles= parameter of the swupd os-install command.

  4. Run git diff.

    The output shows the edits made after adding wget in the clearlinux/redis Dockerfile.

    diff --git a/redis/Dockerfile b/redis/Dockerfile
index af977cb..b1effab 100644
--- a/redis/Dockerfile
+++ b/redis/Dockerfile
@@ -15,7 +15,7 @@ RUN source /os-release && \
    mkdir /install_root \
    && swupd os-install -V ${VERSION_ID} \
    --path /install_root --statedir /swupd-state \
 -    --bundles=redis-native,findutils,su-exec --no-boot-update
 +    --bundles=redis-native,findutils,su-exec,wget --no-boot-update

  5. Build the Dockerfile and apply a unique tag name. In this this example, we use wget_added and add proxies.

    docker build \
--no-cache \
--build-arg http_proxy=$http_proxy \
--build-arg https_proxy=$https_proxy \
--tag clearlinux/redis:wget_added \
redis/

  6. Run the Dockerfile with the wget –version command to verify that wget has been added to the image.

    docker run clearlinux/redis:wget_added wget --version

  7. The output shows:

    GNU Wget 1.20.3 built on linux-gnu.

-cares +digest -gpgme +https +ipv6 -iri +large-file -metalink +nls
-ntlm +opie -psl +ssl/openssl

Example 2: Change Clear Linux OS version (single-stage build)

This example shows how to rebuild single-stage containers against a specific OS version, <CL_VERSION>, by adding a new argument to the Docker build command line.

  1. Rebuild the clearlinux/machine-learning-ui. Add an extra build argument swupd_args="-m <CL_VERSION>"; in this case, the build version is 31110.

    1
2
3
4
5
6
7
    docker build \
--no-cache \
--build-arg http_proxy=$http_proxy \
--build-arg https_proxy=$https_proxy \
--build-arg swupd_args="-m 31110" \
--tag clearlinux/machine-learning-ui:31110 \
machine-learning-ui/

  2. Run the docker container image:

    docker run clearlinux/machine-learning-ui:31110 swupd info

  3. Sample output shows:

    Distribution:      Clear Linux OS
Installed version: 31110
Version URL:       https://cdn.download.clearlinux.org/update
Content URL:       https://cdn.download.clearlinux.org/update

Example 3: Change Clear Linux OS version (multi-stage build)

This example shows how to rebuild the cgit Dockerfile to use a specific Clear Linux OS version. The clearlinux/cgit Dockerfile has a multi-stage build with multiple layers: os-core, httpd, and cgit. This can be used as reference for building other multi-stage images with any number of layers.

Important

All upper layers of multi-stage Dockerfiles inherit the Clear Linux OS version from the base layer. Rebuild the all underlying base layers against the desired OS version. In this example, four base layers must be rebuilt.

First layer: os-core

  1. Rebuild the first layer, os-core. Add an extra build argument swupd_args="-m <CL_VERSION>"; in this case, the build version is 31110.

    1
2
3
4
5
6
7
    docker build \
--no-cache \
--build-arg http_proxy=$http_proxy \
--build-arg https_proxy=$https_proxy \
--build-arg swupd_args="-m 31110" \
--tag clearlinux/os-core:31110 \
os-core/

  2. Verify the version-specific image is available:

    docker images clearlinux/os-core:31110

Second layer: httpd

The next layer is clearlinux/httpd.

  1. Change the httpd/Dockerfile to use the version-specific os-core:31110 image that was previously built.

    $EDITOR httpd/Dockerfile

  2. Run git diff.

    The output shows a diff of a modified clearlinux/httpd Dockerfile that uses the previously built clearlinux/os-core:31110.

    diff --git a/httpd/Dockerfile b/httpd/Dockerfile
index 6b2a6bf..9df89e4 100644
--- a/httpd/Dockerfile
+++ b/httpd/Dockerfile
@@ -7,7 +7,7 @@ RUN swupd update --no-boot-update $swupd_args

# Grab os-release info from the minimal base image so
# that the new content matches the exact OS version
-COPY --from=clearlinux/os-core:latest /usr/lib/os-release /
+COPY --from=clearlinux/os-core:31110 /usr/lib/os-release /

# Install additional content in a target directory
# using the os version from the minimal base
@@ -26,7 +26,7 @@ COPY --from=clearlinux/os-core:latest / /
os_core_install/
RUN cd / && \
    find os_core_install | sed -e 's/os_core_install/install_root/' | xargs rm -d &> /dev/null || true

-FROM clearlinux/os-core:latest
+FROM clearlinux/os-core:31110

  3. Build Dockerfile.

    docker build \
--no-cache \
--build-arg http_proxy=$http_proxy \
--build-arg https_proxy=$https_proxy \
--tag clearlinux/httpd:31110 \
httpd/

Third layer: cgit

The next layer is clearlinux/cgit.

  1. Change the cgit/Dockerfile to use the desired OS version; in this case, the build version is 31110.

    $EDITOR cgit/Dockerfile

  2. Run git diff.

    The output shows:

    diff --git a/cgit/Dockerfile b/cgit/Dockerfile
index 9a3796d..59260fe 100644
--- a/cgit/Dockerfile
+++ b/cgit/Dockerfile
@@ -7,7 +7,7 @@ RUN swupd update --no-boot-update $swupd_args

# Grab os-release info from the minimal base image so
# that the new content matches the exact OS version
-COPY --from=clearlinux/httpd:latest /usr/lib/os-release /
+COPY --from=clearlinux/httpd:31110 /usr/lib/os-release /

# Install additional content in a target directory
# using the os version from the minimal base
@@ -22,11 +22,11 @@ RUN source /os-release && \
# file exists on different layers. To minimize docker
# image size, remove the overlapped files before copy.
RUN mkdir /os_core_install
-COPY --from=clearlinux/httpd:latest / /os_core_install/
+COPY --from=clearlinux/httpd:31110 / /os_core_install/
RUN cd / && \
    find os_core_install | sed -e 's/os_core_install/install_root/' | xargs rm -d &> /dev/null || true

-FROM clearlinux/httpd:latest
+FROM clearlinux/httpd:31110

  3. Build Dockerfile.

    docker build \
--no-cache \
--build-arg http_proxy=$http_proxy \
--build-arg https_proxy=$https_proxy \
--tag clearlinux/cgit:31110 \
cgit/

  4. Verify the installed OS version by noting the VERSION_ID value in the /usr/lib/os-release file in the container filesystem.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    docker run clearlinux/cgit:31110 cat /usr/lib/os-release
NAME="Clear Linux OS"
VERSION=1
ID=clear-linux-os
ID_LIKE=clear-linux-os
VERSION_ID=31110
PRETTY_NAME="Clear Linux OS"
ANSI_COLOR="1;35"
HOME_URL="https://clearlinux.org"
SUPPORT_URL="https://clearlinux.org"
BUG_REPORT_URL="mailto:dev@lists.clearlinux.org"
PRIVACY_POLICY_URL=http://www.intel.com/privacy

Example 4: Customize an application image at runtime

This section describes how to modify a published Clear Linux OS container at runtime. In this example, we add Tensorflow* into a clearlinux/python container. This approach can help accelerate the feature development process.

In this example, three separate console windows are used to easily interact inside and outside of the container.

First console: Start the container

  1. Launch the clearlinux/python container.

    docker run -it --rm clearlinux/python
Python 3.7.3 (default, Jun 17 2019, 00:47:04)
[GCC 9.1.1 20190616 gcc-9-branch@272336] on linux
Type "help", "copyright", "credits" or "license" for more information.

  2. Try to import Tensorflow inside the container using the command: import tensorflow as tf. The example below shows the expected error message because the Docker image does not yet include the Tensorflow module.

    >>> import tensorflow as tf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'tensorflow'
>>>

Second console: Add a bundle

  1. In another console, find the <Container_ID> of clearlinux/python launched. This example Container ID is d4ce9d526fa6.

    docker ps

  2. The output shows:

    CONTAINER ID   IMAGE               COMMAND   CREATED             STATUS              PORTS          NAMES
d4ce9d526fa6   clearlinux/python   python3   About a minute ago  Up About a minute                  amazing_villani

  3. Connect to the running clearlinux/python container.

    docker exec -it d4ce9d526fa6 /usr/bin/bash
root@d4ce9d526fa6/ #

  4. Use swupd to install the machine-learning-tensorflow bundle.

    root@d4ce9d526fa6/ # swupd bundle-add machine-learning-tensorflow
Loading required manifests...
Downloading packs (692.32 Mb) for:
- machine-learning-tensorflow
… …
...100%
Finishing packs extraction...
No extra files need to be downloaded
Installing bundle(s) files...
...100%
Calling post-update helper scripts.
Successfully installed 1 bundle

  5. After the machine-learning-tensorflow bundle is installed in the container, in the first console, import Tensorflow, which will be successful now. You could also save the updated container using the command docker commit <Container_ID>.

    >>> import tensorflow as tf
>>> tf.__version__
'1.13.1'

Third console: Save the modified container

  1. In a third console, save the container with a new tag. Our example uses the tag tensorflow_added to identify our modified container.

    docker commit d4ce9d526fa6 clearlinux/python:tensorflow_added

  2. Launch the modified container, and then import Tensorflow with success.

    docker run -it clearlinux/python:tensorflow_added
Python 3.7.3 (default, Jun 17 2019, 00:47:04)
[GCC 9.1.1 20190616 gcc-9-branch@272336] on linux
Type "help", "copyright", "credits" or "license" for more information.

    >>> import tensorflow as tf
>>> tf.__version__
'1.13.1'
>>>

Background

Multi-stage Dockerfiles contain more than one FROM directive. All of the multi-stage Clear Linux OS Dockerfiles share a common base layer called clearlinux/os-core:latest. All of the higher level layers inherit the Clear Linux OS version from this base layer.

For details on how we leveraged multi-stage Docker builds, see the article Minimizing Clear Linux OS container sizes.

  1. clearlinux/os-core is built once per day. It is a container containing a minimal Linux userspace.

  2. The target container image uses either clearlinux/os-core as a base layer or another container image clearlinux/ as a base layer.

  3. Bundle(s) containing the application are downloaded during the first stage of the build process using swupd.

  4. The final container image is a composition of its base layer and the specific feature layer, via FROM clearlinux/<base layer>:latest , such as: os-core, httpd, and via :command:`COPY --from=builder / install_root /. Using this method, the target container images are kept up to date without file duplication. For application-centric containers, os-core-update is excluded to improve size optimization.