Modify a Clear Linux OS-based container image¶
This guide describes how to customize Clear Linux* OS-based container images on Docker Hub, which include popular applications and runtimes.
Overview¶
Most of these images utilize a Docker build feature called a multi-stage build to reduce image size while some use single-stage build Dockerfiles. An official base clearlinux image on Docker Hub is also available. To create a generic Clear Linux OS container image, see our guide.
Prerequisites¶
Set up a functional Docker environment as described in Docker*.
Download the Clear Linux OS microservice Dockerfile repo with the following command:
git clone https://github.com/clearlinux/dockerfiles.git
Navigate to and operate from the cloned
dockerfiles
directory.cd dockerfiles/
Example 1: Add a bundle¶
In this example, we add wget to the clearlinux/redis Dockerfile.
Enter swupd search wget to discover which Clear Linux OS bundle includes the software. The output should tell you that wget is available in the wget bundle.
Open a an editor to modify the Dockerfile.
$EDITOR redis/Dockerfile
Append the wget bundle to the --bundles= parameter of the swupd os-install command.
Run git diff.
The output shows the edits made after adding wget in the clearlinux/redis Dockerfile.
diff --git a/redis/Dockerfile b/redis/Dockerfile index af977cb..b1effab 100644 --- a/redis/Dockerfile +++ b/redis/Dockerfile @@ -15,7 +15,7 @@ RUN source /os-release && \ mkdir /install_root \ && swupd os-install -V ${VERSION_ID} \ --path /install_root --statedir /swupd-state \ - --bundles=redis-native,findutils,su-exec --no-boot-update + --bundles=redis-native,findutils,su-exec,wget --no-boot-update
Build the Dockerfile and apply a unique tag name. In this this example, we use wget_added and add proxies.
docker build \ --no-cache \ --build-arg http_proxy=$http_proxy \ --build-arg https_proxy=$https_proxy \ --tag clearlinux/redis:wget_added \ redis/
Run the Dockerfile with the wget –version command to verify that wget has been added to the image.
docker run clearlinux/redis:wget_added wget --version
The output shows:
GNU Wget 1.20.3 built on linux-gnu. -cares +digest -gpgme +https +ipv6 -iri +large-file -metalink +nls -ntlm +opie -psl +ssl/openssl
Example 2: Change Clear Linux OS version (single-stage build)¶
This example shows how to rebuild single-stage containers against a specific
OS version, <CL_VERSION>
, by adding a new argument to the Docker build
command line.
Rebuild the
clearlinux/machine-learning-ui
. Add an extra build argument swupd_args="-m <CL_VERSION>"; in this case, the build version is 31110.1docker build \ 2--no-cache \ 3--build-arg http_proxy=$http_proxy \ 4--build-arg https_proxy=$https_proxy \ 5--build-arg swupd_args="-m 31110" \ 6--tag clearlinux/machine-learning-ui:31110 \ 7machine-learning-ui/
Run the docker container image:
docker run clearlinux/machine-learning-ui:31110 swupd info
Sample output shows:
Distribution: Clear Linux OS Installed version: 31110 Version URL: https://cdn.download.clearlinux.org/update Content URL: https://cdn.download.clearlinux.org/update
Example 3: Change Clear Linux OS version (multi-stage build)¶
This example shows how to rebuild the cgit Dockerfile to use a specific Clear Linux OS version. The clearlinux/cgit Dockerfile has a multi-stage build with multiple layers: os-core, httpd, and cgit. This can be used as reference for building other multi-stage images with any number of layers.
Important
All upper layers of multi-stage Dockerfiles inherit the Clear Linux OS version from the base layer. Rebuild the all underlying base layers against the desired OS version. In this example, four base layers must be rebuilt.
First layer: os-core¶
Rebuild the first layer, os-core. Add an extra build argument swupd_args="-m <CL_VERSION>"; in this case, the build version is 31110.
1docker build \ 2--no-cache \ 3--build-arg http_proxy=$http_proxy \ 4--build-arg https_proxy=$https_proxy \ 5--build-arg swupd_args="-m 31110" \ 6--tag clearlinux/os-core:31110 \ 7os-core/
Verify the version-specific image is available:
docker images clearlinux/os-core:31110
Second layer: httpd¶
The next layer is clearlinux/httpd
.
Change the
httpd/Dockerfile
to use the version-specific os-core:31110 image that was previously built.$EDITOR httpd/Dockerfile
Run git diff.
The output shows a diff of a modified
clearlinux/httpd
Dockerfile that uses the previously built clearlinux/os-core:31110.diff --git a/httpd/Dockerfile b/httpd/Dockerfile index 6b2a6bf..9df89e4 100644 --- a/httpd/Dockerfile +++ b/httpd/Dockerfile @@ -7,7 +7,7 @@ RUN swupd update --no-boot-update $swupd_args # Grab os-release info from the minimal base image so # that the new content matches the exact OS version -COPY --from=clearlinux/os-core:latest /usr/lib/os-release / +COPY --from=clearlinux/os-core:31110 /usr/lib/os-release / # Install additional content in a target directory # using the os version from the minimal base @@ -26,7 +26,7 @@ COPY --from=clearlinux/os-core:latest / / os_core_install/ RUN cd / && \ find os_core_install | sed -e 's/os_core_install/install_root/' | xargs rm -d &> /dev/null || true -FROM clearlinux/os-core:latest +FROM clearlinux/os-core:31110
Build Dockerfile.
docker build \ --no-cache \ --build-arg http_proxy=$http_proxy \ --build-arg https_proxy=$https_proxy \ --tag clearlinux/httpd:31110 \ httpd/
Third layer: cgit¶
The next layer is clearlinux/cgit
.
Change the
cgit/Dockerfile
to use the desired OS version; in this case, the build version is 31110.$EDITOR cgit/Dockerfile
Run git diff.
The output shows:
diff --git a/cgit/Dockerfile b/cgit/Dockerfile index 9a3796d..59260fe 100644 --- a/cgit/Dockerfile +++ b/cgit/Dockerfile @@ -7,7 +7,7 @@ RUN swupd update --no-boot-update $swupd_args # Grab os-release info from the minimal base image so # that the new content matches the exact OS version -COPY --from=clearlinux/httpd:latest /usr/lib/os-release / +COPY --from=clearlinux/httpd:31110 /usr/lib/os-release / # Install additional content in a target directory # using the os version from the minimal base @@ -22,11 +22,11 @@ RUN source /os-release && \ # file exists on different layers. To minimize docker # image size, remove the overlapped files before copy. RUN mkdir /os_core_install -COPY --from=clearlinux/httpd:latest / /os_core_install/ +COPY --from=clearlinux/httpd:31110 / /os_core_install/ RUN cd / && \ find os_core_install | sed -e 's/os_core_install/install_root/' | xargs rm -d &> /dev/null || true -FROM clearlinux/httpd:latest +FROM clearlinux/httpd:31110
Build Dockerfile.
docker build \ --no-cache \ --build-arg http_proxy=$http_proxy \ --build-arg https_proxy=$https_proxy \ --tag clearlinux/cgit:31110 \ cgit/
Verify the installed OS version by noting the VERSION_ID value in the
/usr/lib/os-release
file in the container filesystem.1docker run clearlinux/cgit:31110 cat /usr/lib/os-release 2NAME="Clear Linux OS" 3VERSION=1 4ID=clear-linux-os 5ID_LIKE=clear-linux-os 6VERSION_ID=31110 7PRETTY_NAME="Clear Linux OS" 8ANSI_COLOR="1;35" 9HOME_URL="https://clearlinux.org" 10SUPPORT_URL="https://clearlinux.org" 11BUG_REPORT_URL="mailto:dev@lists.clearlinux.org" 12PRIVACY_POLICY_URL=http://www.intel.com/privacy
Example 4: Customize an application image at runtime¶
This section describes how to modify a published Clear Linux OS container at runtime. In this example, we add Tensorflow* into a clearlinux/python container. This approach can help accelerate the feature development process.
In this example, three separate console windows are used to easily interact inside and outside of the container.
First console: Start the container¶
Launch the clearlinux/python container.
docker run -it --rm clearlinux/python Python 3.7.3 (default, Jun 17 2019, 00:47:04) [GCC 9.1.1 20190616 gcc-9-branch@272336] on linux Type "help", "copyright", "credits" or "license" for more information.
Try to import Tensorflow inside the container using the command: import tensorflow as tf. The example below shows the expected error message because the Docker image does not yet include the Tensorflow module.
>>> import tensorflow as tf Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'tensorflow' >>>
Second console: Add a bundle¶
In another console, find the <Container_ID> of clearlinux/python launched. This example Container ID is d4ce9d526fa6.
docker ps
The output shows:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d4ce9d526fa6 clearlinux/python python3 About a minute ago Up About a minute amazing_villani
Connect to the running clearlinux/python container.
docker exec -it d4ce9d526fa6 /usr/bin/bash root@d4ce9d526fa6/ #
Use swupd to install the machine-learning-tensorflow bundle.
root@d4ce9d526fa6/ # swupd bundle-add machine-learning-tensorflow Loading required manifests... Downloading packs (692.32 Mb) for: - machine-learning-tensorflow … … ...100% Finishing packs extraction... No extra files need to be downloaded Installing bundle(s) files... ...100% Calling post-update helper scripts. Successfully installed 1 bundle
After the machine-learning-tensorflow bundle is installed in the container, in the first console, import Tensorflow, which will be successful now. You could also save the updated container using the command docker commit <Container_ID>.
>>> import tensorflow as tf >>> tf.__version__ '1.13.1'
Third console: Save the modified container¶
In a third console, save the container with a new tag. Our example uses the tag tensorflow_added to identify our modified container.
docker commit d4ce9d526fa6 clearlinux/python:tensorflow_added
Launch the modified container, and then import Tensorflow with success.
docker run -it clearlinux/python:tensorflow_added Python 3.7.3 (default, Jun 17 2019, 00:47:04) [GCC 9.1.1 20190616 gcc-9-branch@272336] on linux Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf >>> tf.__version__ '1.13.1' >>>
Background¶
Multi-stage Dockerfiles contain more than one FROM directive. All of the multi-stage Clear Linux OS Dockerfiles share a common base layer called clearlinux/os-core:latest. All of the higher level layers inherit the Clear Linux OS version from this base layer.
For details on how we leveraged multi-stage Docker builds, see the article Minimizing Clear Linux OS container sizes.
clearlinux/os-core is built once per day. It is a container containing a minimal Linux userspace.
The target container image uses either clearlinux/os-core as a base layer or another container image clearlinux/ as a base layer.
Bundle(s) containing the application are downloaded during the first stage of the build process using swupd.
The final container image is a composition of its base layer and the specific feature layer, via FROM clearlinux/<base layer>:latest , such as: os-core, httpd, and via :command:`COPY --from=builder / install_root /. Using this method, the target container images are kept up to date without file duplication. For application-centric containers, os-core-update is excluded to improve size optimization.