Make taggers manifest functions (#2252)

* Make taggers and manifests functions

* Add changelog

CHANGELOG.md

@@ -8,6 +8,7 @@ All image manifests can be found in [the wiki](https://github.com/jupyter/docker
 Affected: all images.
 
 - **Non-breaking:** Add `conda` and `mamba` version taggers ([#2251](https://github.com/jupyter/docker-stacks/pull/2251)).
+- **Non-breaking:** Make taggers and manifests functions ([#2252](https://github.com/jupyter/docker-stacks/pull/2252)).
 
 ## 2025-02-21
 

docs/maintaining/tagging.md

@@ -1,7 +1,8 @@
 # Tagging and manifest creation
 
-The main purpose of the source code in [the `tagging` folder](https://github.com/jupyter/docker-stacks/tree/main/tagging) is to properly write tag files and manifests for single-platform images,
+The main purpose of the source code in [the `tagging` folder](https://github.com/jupyter/docker-stacks/tree/main/tagging) is to
-apply these tags, and merge single-platform images into one multi-arch image.
+properly write tags file, build history line and manifest for a single-platform image,
+apply these tags, and then merge single-platform images into one multi-arch image.
 
 ## What is a tag and a manifest
 
@@ -16,9 +17,9 @@ For example, we dump all `conda` packages with their versions into the manifest.
 
 - All images are organized in a hierarchical tree.
   More info on [image relationships](../using/selecting.md#image-relationships).
-- Classes inherit from `TaggerInterface` and `ManifestInterface` to generate tags and manifest pieces by running commands in Docker containers.
+- `TaggerInterface` and `ManifestInterface` are interfaces for functions to generate tags and manifest pieces by running commands in Docker containers.
 - Tags and manifests are reevaluated for each image in the hierarchy since values may change between parent and child images.
-- To tag an image and create its manifest, run `make hook/<somestack>` (e.g., `make hook/base-notebook`).
+- To tag an image and create its manifest and build history line, run `make hook/<somestack>` (e.g., `make hook/base-notebook`).
 
 ## Utils
 
@@ -46,42 +47,42 @@ The prefix of commit hash (namely, 12 letters) is used as an image tag to make i
 
 ### Tagger
 
-`Tagger` is a class that can be run inside a docker container to calculate a tag for an image.
+`Tagger` is a function that runs commands inside a docker container to calculate a tag for an image.
 
-All the taggers are inherited from `TaggerInterface`:
+All the taggers follow `TaggerInterface`:
 
 ```{literalinclude} ../../tagging/taggers/tagger_interface.py
 :language: py
-:start-at: class TaggerInterface
+:start-at: TaggerInterface
 ```
 
-So, the `tag_value(container)` method gets a docker container as an input and returns a tag.
+So, the `tagger(container)` gets a docker container as an input and returns a tag.
 
-`SHATagger` example:
+`commit_sha_tagger` example:
 
 ```{literalinclude} ../../tagging/taggers/sha.py
 :language: py
-:start-at: class SHATagger
+:start-at: def
 ```
 
-- `taggers/` subdirectory contains all the taggers.
+- `taggers/` subdirectory contains all taggers.
 - `apps/write_tags_file.py`, `apps/apply_tags.py`, and `apps/merge_tags.py` are Python executable used to write tags for an image, apply tags from a file, and create multi-arch images.
 
 ### Manifest
 
-All manifest classes except `BuildInfo` are inherited from `ManifestInterface`
+All manifest functions except `build_info_manifest` follow `ManifestInterface`
-and `markdown_piece(container)` method returns a piece of the build manifest.
+and `manifest(container)` method returns a piece of the build manifest.
 
 ```{literalinclude} ../../tagging/manifests/manifest_interface.py
 :language: py
-:start-at: class ManifestInterface
+:start-at: ManifestInterface
 ```
 
-`AptPackagesManifest` example:
+`apt_packages_manifest` example:
 
 ```{literalinclude} ../../tagging/manifests/apt_packages.py
 :language: py
-:start-at: class AptPackagesManifest
+:start-at: def
 ```
 
 - `quoted_output(container, cmd)` simply runs the command inside a container using `DockerRunner.exec_cmd` and wraps it to triple quotes to create a valid markdown piece.

tagging/apps/write_manifest.py

@@ -10,7 +10,7 @@ from tagging.apps.common_cli_arguments import common_arguments_parser
 from tagging.apps.config import Config
 from tagging.hierarchy.get_manifests import get_manifests
 from tagging.hierarchy.get_taggers import get_taggers
-from tagging.manifests.build_info import BuildInfo, BuildInfoConfig
+from tagging.manifests.build_info import BuildInfoConfig, build_info_manifest
 from tagging.utils.docker_runner import DockerRunner
 from tagging.utils.get_prefix import get_file_prefix, get_tag_prefix
 from tagging.utils.git_helper import GitHelper
@@ -27,7 +27,7 @@ def get_build_history_line(config: Config, filename: str, container: Container)
 
     taggers = get_taggers(config.image)
     tags_prefix = get_tag_prefix(config.variant)
-    all_tags = [tags_prefix + "-" + tagger.tag_value(container) for tagger in taggers]
+    all_tags = [tags_prefix + "-" + tagger(container) for tagger in taggers]
 
     date_column = f"`{BUILD_TIMESTAMP}`"
     image_column = MARKDOWN_LINE_BREAK.join(
@@ -64,7 +64,7 @@ def get_manifest(config: Config, commit_hash_tag: str, container: Container) ->
     LOGGER.info(f"Calculating manifest file for image: {config.image}")
 
     manifests = get_manifests(config.image)
-    manifest_names = [manifest.__class__.__name__ for manifest in manifests]
+    manifest_names = [manifest.__name__ for manifest in manifests]
     LOGGER.info(f"Using manifests: {manifest_names}")
 
     build_info_config = BuildInfoConfig(
@@ -77,8 +77,8 @@ def get_manifest(config: Config, commit_hash_tag: str, container: Container) ->
 
     markdown_pieces = [
         f"# Build manifest for image: {config.image}:{commit_hash_tag}",
-        BuildInfo.markdown_piece(build_info_config).get_str(),
+        build_info_manifest(build_info_config).get_str(),
-        *(manifest.markdown_piece(container).get_str() for manifest in manifests),
+        *(manifest(container).get_str() for manifest in manifests),
     ]
     markdown_content = "\n\n".join(markdown_pieces) + "\n"
 

tagging/apps/write_tags_file.py

@@ -20,8 +20,8 @@ def get_tags(config: Config) -> list[str]:
     tags = [f"{config.full_image()}:{tags_prefix}-latest"]
     with DockerRunner(config.full_image()) as container:
         for tagger in taggers:
-            tagger_name = tagger.__class__.__name__
+            tagger_name = tagger.__name__
-            tag_value = tagger.tag_value(container)
+            tag_value = tagger(container)
             LOGGER.info(
                 f"Calculated tag, tagger_name: {tagger_name} tag_value: {tag_value}"
             )

tagging/hierarchy/images_hierarchy.py

@@ -2,31 +2,17 @@
 # Distributed under the terms of the Modified BSD License.
 from dataclasses import dataclass, field
 
-from tagging.manifests.apt_packages import AptPackagesManifest
+from tagging.manifests.apt_packages import apt_packages_manifest
-from tagging.manifests.conda_environment import CondaEnvironmentManifest
+from tagging.manifests.conda_environment import conda_environment_manifest
-from tagging.manifests.julia_packages import JuliaPackagesManifest
+from tagging.manifests.julia_packages import julia_packages_manifest
 from tagging.manifests.manifest_interface import ManifestInterface
-from tagging.manifests.r_packages import RPackagesManifest
+from tagging.manifests.r_packages import r_packages_manifest
-from tagging.manifests.spark_info import SparkInfoManifest
+from tagging.manifests.spark_info import spark_info_manifest
-from tagging.taggers.date import DateTagger
+from tagging.taggers import versions
-from tagging.taggers.sha import SHATagger
+from tagging.taggers.date import date_tagger
+from tagging.taggers.sha import commit_sha_tagger
 from tagging.taggers.tagger_interface import TaggerInterface
-from tagging.taggers.ubuntu_version import UbuntuVersionTagger
+from tagging.taggers.ubuntu_version import ubuntu_version_tagger
-from tagging.taggers.versions import (
-    CondaVersionTagger,
-    JavaVersionTagger,
-    JuliaVersionTagger,
-    JupyterHubVersionTagger,
-    JupyterLabVersionTagger,
-    JupyterNotebookVersionTagger,
-    MambaVersionTagger,
-    PythonMajorMinorVersionTagger,
-    PythonVersionTagger,
-    PytorchVersionTagger,
-    RVersionTagger,
-    SparkVersionTagger,
-    TensorflowVersionTagger,
-)
 
 
 @dataclass
@@ -40,55 +26,55 @@ ALL_IMAGES = {
     "docker-stacks-foundation": ImageDescription(
         parent_image=None,
         taggers=[
-            SHATagger(),
+            commit_sha_tagger,
-            DateTagger(),
+            date_tagger,
-            UbuntuVersionTagger(),
+            ubuntu_version_tagger,
-            PythonMajorMinorVersionTagger(),
+            versions.python_major_minor_tagger,
-            PythonVersionTagger(),
+            versions.python_tagger,
-            MambaVersionTagger(),
+            versions.mamba_tagger,
-            CondaVersionTagger(),
+            versions.conda_tagger,
         ],
-        manifests=[CondaEnvironmentManifest(), AptPackagesManifest()],
+        manifests=[conda_environment_manifest, apt_packages_manifest],
     ),
     "base-notebook": ImageDescription(
         parent_image="docker-stacks-foundation",
         taggers=[
-            JupyterNotebookVersionTagger(),
+            versions.jupyter_notebook_tagger,
-            JupyterLabVersionTagger(),
+            versions.jupyter_lab_tagger,
-            JupyterHubVersionTagger(),
+            versions.jupyter_hub_tagger,
         ],
     ),
     "minimal-notebook": ImageDescription(parent_image="base-notebook"),
     "scipy-notebook": ImageDescription(parent_image="minimal-notebook"),
     "r-notebook": ImageDescription(
         parent_image="minimal-notebook",
-        taggers=[RVersionTagger()],
+        taggers=[versions.r_tagger],
-        manifests=[RPackagesManifest()],
+        manifests=[r_packages_manifest],
     ),
     "julia-notebook": ImageDescription(
         parent_image="minimal-notebook",
-        taggers=[JuliaVersionTagger()],
+        taggers=[versions.julia_tagger],
-        manifests=[JuliaPackagesManifest()],
+        manifests=[julia_packages_manifest],
     ),
     "tensorflow-notebook": ImageDescription(
-        parent_image="scipy-notebook", taggers=[TensorflowVersionTagger()]
+        parent_image="scipy-notebook", taggers=[versions.tensorflow_tagger]
     ),
     "pytorch-notebook": ImageDescription(
-        parent_image="scipy-notebook", taggers=[PytorchVersionTagger()]
+        parent_image="scipy-notebook", taggers=[versions.python_tagger]
     ),
     "datascience-notebook": ImageDescription(
         parent_image="scipy-notebook",
-        taggers=[RVersionTagger(), JuliaVersionTagger()],
+        taggers=[versions.r_tagger, versions.julia_tagger],
-        manifests=[RPackagesManifest(), JuliaPackagesManifest()],
+        manifests=[r_packages_manifest, julia_packages_manifest],
     ),
     "pyspark-notebook": ImageDescription(
         parent_image="scipy-notebook",
-        taggers=[SparkVersionTagger(), JavaVersionTagger()],
+        taggers=[versions.spark_tagger, versions.java_tagger],
-        manifests=[SparkInfoManifest()],
+        manifests=[spark_info_manifest],
     ),
     "all-spark-notebook": ImageDescription(
         parent_image="pyspark-notebook",
-        taggers=[RVersionTagger()],
+        taggers=[versions.r_tagger],
-        manifests=[RPackagesManifest()],
+        manifests=[r_packages_manifest],
     ),
 }

tagging/manifests/apt_packages.py

@@ -2,13 +2,11 @@
 # Distributed under the terms of the Modified BSD License.
 from docker.models.containers import Container
 
-from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
+from tagging.manifests.manifest_interface import MarkdownPiece
 from tagging.utils.quoted_output import quoted_output
 
 
-class AptPackagesManifest(ManifestInterface):
+def apt_packages_manifest(container: Container) -> MarkdownPiece:
-    @staticmethod
-    def markdown_piece(container: Container) -> MarkdownPiece:
     return MarkdownPiece(
         title="## Apt Packages",
         sections=[quoted_output(container, "apt list --installed")],

tagging/manifests/build_info.py

@@ -25,11 +25,8 @@ class BuildInfoConfig:
         return f"{self.registry}/{self.owner}/{self.image}"
 
 
-class BuildInfo:
+def build_info_manifest(config: BuildInfoConfig) -> MarkdownPiece:
     """BuildInfo doesn't fall under common interface, and we run it separately"""
-
-    @staticmethod
-    def markdown_piece(config: BuildInfoConfig) -> MarkdownPiece:
     commit_hash = GitHelper.commit_hash()
     commit_hash_tag = GitHelper.commit_hash_tag()
     commit_message = GitHelper.commit_message()

tagging/manifests/conda_environment.py

@@ -2,14 +2,12 @@
 # Distributed under the terms of the Modified BSD License.
 from docker.models.containers import Container
 
-from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
+from tagging.manifests.manifest_interface import MarkdownPiece
 from tagging.utils.docker_runner import DockerRunner
 from tagging.utils.quoted_output import quoted_output
 
 
-class CondaEnvironmentManifest(ManifestInterface):
+def conda_environment_manifest(container: Container) -> MarkdownPiece:
-    @staticmethod
-    def markdown_piece(container: Container) -> MarkdownPiece:
     return MarkdownPiece(
         title="## Python Packages",
         sections=[

tagging/manifests/julia_packages.py

@@ -2,13 +2,11 @@
 # Distributed under the terms of the Modified BSD License.
 from docker.models.containers import Container
 
-from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
+from tagging.manifests.manifest_interface import MarkdownPiece
 from tagging.utils.quoted_output import quoted_output
 
 
-class JuliaPackagesManifest(ManifestInterface):
+def julia_packages_manifest(container: Container) -> MarkdownPiece:
-    @staticmethod
-    def markdown_piece(container: Container) -> MarkdownPiece:
     return MarkdownPiece(
         title="## Julia Packages",
         sections=[

tagging/manifests/manifest_interface.py

@@ -1,5 +1,6 @@
 # Copyright (c) Jupyter Development Team.
 # Distributed under the terms of the Modified BSD License.
+from collections.abc import Callable
 from dataclasses import dataclass
 
 from docker.models.containers import Container
@@ -18,9 +19,4 @@ class MarkdownPiece:
         return "\n\n".join([self.title, *self.sections])
 
 
-class ManifestInterface:
+ManifestInterface = Callable[[Container], MarkdownPiece]
-    """Common interface for all manifests"""
-
-    @staticmethod
-    def markdown_piece(container: Container) -> MarkdownPiece:
-        raise NotImplementedError

tagging/manifests/r_packages.py

@@ -2,13 +2,11 @@
 # Distributed under the terms of the Modified BSD License.
 from docker.models.containers import Container
 
-from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
+from tagging.manifests.manifest_interface import MarkdownPiece
 from tagging.utils.quoted_output import quoted_output
 
 
-class RPackagesManifest(ManifestInterface):
+def r_packages_manifest(container: Container) -> MarkdownPiece:
-    @staticmethod
-    def markdown_piece(container: Container) -> MarkdownPiece:
     return MarkdownPiece(
         title="## R Packages",
         sections=[

tagging/manifests/spark_info.py

@@ -2,13 +2,11 @@
 # Distributed under the terms of the Modified BSD License.
 from docker.models.containers import Container
 
-from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
+from tagging.manifests.manifest_interface import MarkdownPiece
 from tagging.utils.quoted_output import quoted_output
 
 
-class SparkInfoManifest(ManifestInterface):
+def spark_info_manifest(container: Container) -> MarkdownPiece:
-    @staticmethod
-    def markdown_piece(container: Container) -> MarkdownPiece:
     return MarkdownPiece(
         title="## Apache Spark",
         sections=[

tagging/taggers/date.py

@@ -4,10 +4,6 @@ import datetime
 
 from docker.models.containers import Container
 
-from tagging.taggers.tagger_interface import TaggerInterface
 
-
+def date_tagger(container: Container) -> str:
-class DateTagger(TaggerInterface):
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return datetime.datetime.now(datetime.UTC).strftime("%Y-%m-%d")

tagging/taggers/sha.py

@@ -2,11 +2,8 @@
 # Distributed under the terms of the Modified BSD License.
 from docker.models.containers import Container
 
-from tagging.taggers.tagger_interface import TaggerInterface
 from tagging.utils.git_helper import GitHelper
 
 
-class SHATagger(TaggerInterface):
+def commit_sha_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return GitHelper.commit_hash_tag()

tagging/taggers/tagger_interface.py

@@ -1,11 +1,7 @@
 # Copyright (c) Jupyter Development Team.
 # Distributed under the terms of the Modified BSD License.
+from collections.abc import Callable
+
 from docker.models.containers import Container
 
-
+TaggerInterface = Callable[[Container], str]
-class TaggerInterface:
-    """Common interface for all taggers"""
-
-    @staticmethod
-    def tag_value(container: Container) -> str:
-        raise NotImplementedError

tagging/taggers/ubuntu_version.py

@@ -2,13 +2,10 @@
 # Distributed under the terms of the Modified BSD License.
 from docker.models.containers import Container
 
-from tagging.taggers.tagger_interface import TaggerInterface
 from tagging.utils.docker_runner import DockerRunner
 
 
-class UbuntuVersionTagger(TaggerInterface):
+def ubuntu_version_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     os_release = DockerRunner.exec_cmd(
         container,
         "cat /etc/os-release",

tagging/taggers/versions.py

@@ -2,7 +2,6 @@
 # Distributed under the terms of the Modified BSD License.
 from docker.models.containers import Container
 
-from tagging.taggers.tagger_interface import TaggerInterface
 from tagging.utils.docker_runner import DockerRunner
 
 
@@ -23,79 +22,55 @@ def _get_pip_package_version(container: Container, package: str) -> str:
     return version_line[len(PIP_VERSION_PREFIX) :]
 
 
-class PythonVersionTagger(TaggerInterface):
+def python_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "python-" + _get_program_version(container, "python").split()[1]
 
 
-class PythonMajorMinorVersionTagger(TaggerInterface):
+def python_major_minor_tagger(container: Container) -> str:
-    @staticmethod
+    full_version = python_tagger(container)
-    def tag_value(container: Container) -> str:
-        full_version = PythonVersionTagger.tag_value(container)
     return full_version[: full_version.rfind(".")]
 
 
-class MambaVersionTagger(TaggerInterface):
+def mamba_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "mamba-" + _get_program_version(container, "mamba")
 
 
-class CondaVersionTagger(TaggerInterface):
+def conda_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "conda-" + _get_program_version(container, "conda").split()[1]
 
 
-class JupyterNotebookVersionTagger(TaggerInterface):
+def jupyter_notebook_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "notebook-" + _get_program_version(container, "jupyter-notebook")
 
 
-class JupyterLabVersionTagger(TaggerInterface):
+def jupyter_lab_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "lab-" + _get_program_version(container, "jupyter-lab")
 
 
-class JupyterHubVersionTagger(TaggerInterface):
+def jupyter_hub_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "hub-" + _get_program_version(container, "jupyterhub")
 
 
-class RVersionTagger(TaggerInterface):
+def r_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "r-" + _get_program_version(container, "R").split()[2]
 
 
-class JuliaVersionTagger(TaggerInterface):
+def julia_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "julia-" + _get_program_version(container, "julia").split()[2]
 
 
-class TensorflowVersionTagger(TaggerInterface):
+def tensorflow_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     try:
         return "tensorflow-" + _get_pip_package_version(container, "tensorflow")
     except AssertionError:
         return "tensorflow-" + _get_pip_package_version(container, "tensorflow-cpu")
 
 
-class PytorchVersionTagger(TaggerInterface):
+def pytorch_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "pytorch-" + _get_pip_package_version(container, "torch").split("+")[0]
 
 
-class SparkVersionTagger(TaggerInterface):
+def spark_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     SPARK_VERSION_LINE_PREFIX = r"   /___/ .__/\_,_/_/ /_/\_\   version"
 
     spark_version = _get_program_version(container, "spark-submit")
@@ -108,7 +83,5 @@ class SparkVersionTagger(TaggerInterface):
     return "spark-" + version_line.split(" ")[-1]
 
 
-class JavaVersionTagger(TaggerInterface):
+def java_tagger(container: Container) -> str:
-    @staticmethod
-    def tag_value(container: Container) -> str:
     return "java-" + _get_program_version(container, "java").split()[1]