hazza/docker-stacks
1
0
Fork 0
mirror of https://github.com/jupyter/docker-stacks.git synced 2025-10-07 10:04:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

Make taggers manifest functions (#2252)

Browse Source 
* Make taggers and manifests functions

* Add changelog
This commit is contained in:
Ayaz Salikhov
2025-03-12 16:05:09 +00:00
committed by GitHub
parent 48b065050c
commit 2c1df4020a
17 changed files with 183 additions and 253 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@@ -8,6 +8,7 @@ All image manifests can be found in [the wiki](https://github.com/jupyter/docker
Affected: all images. Affected: all images.
- **Non-breaking:** Add `conda` and `mamba` version taggers ([#2251](https://github.com/jupyter/docker-stacks/pull/2251)). - **Non-breaking:** Add `conda` and `mamba` version taggers ([#2251](https://github.com/jupyter/docker-stacks/pull/2251)).
- **Non-breaking:** Make taggers and manifests functions ([#2252](https://github.com/jupyter/docker-stacks/pull/2252)).
## 2025-02-21 ## 2025-02-21

33
docs/maintaining/tagging.md
View File

@@ -1,7 +1,8 @@
# Tagging and manifest creation # Tagging and manifest creation
The main purpose of the source code in [the `tagging` folder](https://github.com/jupyter/docker-stacks/tree/main/tagging) is to properly write tag files and manifests for single-platform images, The main purpose of the source code in [the `tagging` folder](https://github.com/jupyter/docker-stacks/tree/main/tagging) is to
apply these tags, and merge single-platform images into one multi-arch image. properly write tags file, build history line and manifest for a single-platform image,
apply these tags, and then merge single-platform images into one multi-arch image.
## What is a tag and a manifest ## What is a tag and a manifest
@@ -16,9 +17,9 @@ For example, we dump all `conda` packages with their versions into the manifest.
- All images are organized in a hierarchical tree. - All images are organized in a hierarchical tree.
More info on [image relationships](../using/selecting.md#image-relationships). More info on [image relationships](../using/selecting.md#image-relationships).
- Classes inherit from `TaggerInterface` and `ManifestInterface` to generate tags and manifest pieces by running commands in Docker containers. - `TaggerInterface` and `ManifestInterface` are interfaces for functions to generate tags and manifest pieces by running commands in Docker containers.
- Tags and manifests are reevaluated for each image in the hierarchy since values may change between parent and child images. - Tags and manifests are reevaluated for each image in the hierarchy since values may change between parent and child images.
- To tag an image and create its manifest, run `make hook/<somestack>` (e.g., `make hook/base-notebook`). - To tag an image and create its manifest and build history line, run `make hook/<somestack>` (e.g., `make hook/base-notebook`).
## Utils ## Utils
@@ -46,42 +47,42 @@ The prefix of commit hash (namely, 12 letters) is used as an image tag to make i
### Tagger ### Tagger
`Tagger` is a class that can be run inside a docker container to calculate a tag for an image. `Tagger` is a function that runs commands inside a docker container to calculate a tag for an image.
All the taggers are inherited from `TaggerInterface`: All the taggers follow `TaggerInterface`:
```{literalinclude} ../../tagging/taggers/tagger_interface.py ```{literalinclude} ../../tagging/taggers/tagger_interface.py
:language: py :language: py
:start-at: class TaggerInterface :start-at: TaggerInterface
``` ```
So, the `tag_value(container)` method gets a docker container as an input and returns a tag. So, the `tagger(container)` gets a docker container as an input and returns a tag.
`SHATagger` example: `commit_sha_tagger` example:
```{literalinclude} ../../tagging/taggers/sha.py ```{literalinclude} ../../tagging/taggers/sha.py
:language: py :language: py
:start-at: class SHATagger :start-at: def
``` ```
- `taggers/` subdirectory contains all the taggers. - `taggers/` subdirectory contains all taggers.
- `apps/write_tags_file.py`, `apps/apply_tags.py`, and `apps/merge_tags.py` are Python executable used to write tags for an image, apply tags from a file, and create multi-arch images. - `apps/write_tags_file.py`, `apps/apply_tags.py`, and `apps/merge_tags.py` are Python executable used to write tags for an image, apply tags from a file, and create multi-arch images.
### Manifest ### Manifest
All manifest classes except `BuildInfo` are inherited from `ManifestInterface` All manifest functions except `build_info_manifest` follow `ManifestInterface`
and `markdown_piece(container)` method returns a piece of the build manifest. and `manifest(container)` method returns a piece of the build manifest.
```{literalinclude} ../../tagging/manifests/manifest_interface.py ```{literalinclude} ../../tagging/manifests/manifest_interface.py
:language: py :language: py
:start-at: class ManifestInterface :start-at: ManifestInterface
``` ```
`AptPackagesManifest` example: `apt_packages_manifest` example:
```{literalinclude} ../../tagging/manifests/apt_packages.py ```{literalinclude} ../../tagging/manifests/apt_packages.py
:language: py :language: py
:start-at: class AptPackagesManifest :start-at: def
``` ```
- `quoted_output(container, cmd)` simply runs the command inside a container using `DockerRunner.exec_cmd` and wraps it to triple quotes to create a valid markdown piece. - `quoted_output(container, cmd)` simply runs the command inside a container using `DockerRunner.exec_cmd` and wraps it to triple quotes to create a valid markdown piece.

10
tagging/apps/write_manifest.py
View File

@@ -10,7 +10,7 @@ from tagging.apps.common_cli_arguments import common_arguments_parser
from tagging.apps.config import Config from tagging.apps.config import Config
from tagging.hierarchy.get_manifests import get_manifests from tagging.hierarchy.get_manifests import get_manifests
from tagging.hierarchy.get_taggers import get_taggers from tagging.hierarchy.get_taggers import get_taggers
from tagging.manifests.build_info import BuildInfo, BuildInfoConfig from tagging.manifests.build_info import BuildInfoConfig, build_info_manifest
from tagging.utils.docker_runner import DockerRunner from tagging.utils.docker_runner import DockerRunner
from tagging.utils.get_prefix import get_file_prefix, get_tag_prefix from tagging.utils.get_prefix import get_file_prefix, get_tag_prefix
from tagging.utils.git_helper import GitHelper from tagging.utils.git_helper import GitHelper
@@ -27,7 +27,7 @@ def get_build_history_line(config: Config, filename: str, container: Container)
taggers = get_taggers(config.image) taggers = get_taggers(config.image)
tags_prefix = get_tag_prefix(config.variant) tags_prefix = get_tag_prefix(config.variant)
all_tags = [tags_prefix + "-" + tagger.tag_value(container) for tagger in taggers] all_tags = [tags_prefix + "-" + tagger(container) for tagger in taggers]
date_column = f"`{BUILD_TIMESTAMP}`" date_column = f"`{BUILD_TIMESTAMP}`"
image_column = MARKDOWN_LINE_BREAK.join( image_column = MARKDOWN_LINE_BREAK.join(
@@ -64,7 +64,7 @@ def get_manifest(config: Config, commit_hash_tag: str, container: Container) ->
LOGGER.info(f"Calculating manifest file for image: {config.image}") LOGGER.info(f"Calculating manifest file for image: {config.image}")
manifests = get_manifests(config.image) manifests = get_manifests(config.image)
manifest_names = [manifest.__class__.__name__ for manifest in manifests] manifest_names = [manifest.__name__ for manifest in manifests]
LOGGER.info(f"Using manifests: {manifest_names}") LOGGER.info(f"Using manifests: {manifest_names}")
build_info_config = BuildInfoConfig( build_info_config = BuildInfoConfig(
@@ -77,8 +77,8 @@ def get_manifest(config: Config, commit_hash_tag: str, container: Container) ->
markdown_pieces = [ markdown_pieces = [
f"# Build manifest for image: {config.image}:{commit_hash_tag}", f"# Build manifest for image: {config.image}:{commit_hash_tag}",
BuildInfo.markdown_piece(build_info_config).get_str(), build_info_manifest(build_info_config).get_str(),
*(manifest.markdown_piece(container).get_str() for manifest in manifests), *(manifest(container).get_str() for manifest in manifests),
] ]
markdown_content = "\n\n".join(markdown_pieces) + "\n" markdown_content = "\n\n".join(markdown_pieces) + "\n"

4
tagging/apps/write_tags_file.py
View File

@@ -20,8 +20,8 @@ def get_tags(config: Config) -> list[str]:
tags = [f"{config.full_image()}:{tags_prefix}-latest"] tags = [f"{config.full_image()}:{tags_prefix}-latest"]
with DockerRunner(config.full_image()) as container: with DockerRunner(config.full_image()) as container:
for tagger in taggers: for tagger in taggers:
tagger_name = tagger.__class__.__name__ tagger_name = tagger.__name__
tag_value = tagger.tag_value(container) tag_value = tagger(container)
LOGGER.info( LOGGER.info(
f"Calculated tag, tagger_name: {tagger_name} tag_value: {tag_value}" f"Calculated tag, tagger_name: {tagger_name} tag_value: {tag_value}"
) )

78
tagging/hierarchy/images_hierarchy.py
View File

@@ -2,31 +2,17 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from dataclasses import dataclass, field from dataclasses import dataclass, field
from tagging.manifests.apt_packages import AptPackagesManifest from tagging.manifests.apt_packages import apt_packages_manifest
from tagging.manifests.conda_environment import CondaEnvironmentManifest from tagging.manifests.conda_environment import conda_environment_manifest
from tagging.manifests.julia_packages import JuliaPackagesManifest from tagging.manifests.julia_packages import julia_packages_manifest
from tagging.manifests.manifest_interface import ManifestInterface from tagging.manifests.manifest_interface import ManifestInterface
from tagging.manifests.r_packages import RPackagesManifest from tagging.manifests.r_packages import r_packages_manifest
from tagging.manifests.spark_info import SparkInfoManifest from tagging.manifests.spark_info import spark_info_manifest
from tagging.taggers.date import DateTagger from tagging.taggers import versions
from tagging.taggers.sha import SHATagger from tagging.taggers.date import date_tagger
from tagging.taggers.sha import commit_sha_tagger
from tagging.taggers.tagger_interface import TaggerInterface from tagging.taggers.tagger_interface import TaggerInterface
from tagging.taggers.ubuntu_version import UbuntuVersionTagger from tagging.taggers.ubuntu_version import ubuntu_version_tagger
from tagging.taggers.versions import (
CondaVersionTagger,
JavaVersionTagger,
JuliaVersionTagger,
JupyterHubVersionTagger,
JupyterLabVersionTagger,
JupyterNotebookVersionTagger,
MambaVersionTagger,
PythonMajorMinorVersionTagger,
PythonVersionTagger,
PytorchVersionTagger,
RVersionTagger,
SparkVersionTagger,
TensorflowVersionTagger,
)
@dataclass @dataclass
@@ -40,55 +26,55 @@ ALL_IMAGES = {
"docker-stacks-foundation": ImageDescription( "docker-stacks-foundation": ImageDescription(
parent_image=None, parent_image=None,
taggers=[ taggers=[
SHATagger(), commit_sha_tagger,
DateTagger(), date_tagger,
UbuntuVersionTagger(), ubuntu_version_tagger,
PythonMajorMinorVersionTagger(), versions.python_major_minor_tagger,
PythonVersionTagger(), versions.python_tagger,
MambaVersionTagger(), versions.mamba_tagger,
CondaVersionTagger(), versions.conda_tagger,
], ],
manifests=[CondaEnvironmentManifest(), AptPackagesManifest()], manifests=[conda_environment_manifest, apt_packages_manifest],
), ),
"base-notebook": ImageDescription( "base-notebook": ImageDescription(
parent_image="docker-stacks-foundation", parent_image="docker-stacks-foundation",
taggers=[ taggers=[
JupyterNotebookVersionTagger(), versions.jupyter_notebook_tagger,
JupyterLabVersionTagger(), versions.jupyter_lab_tagger,
JupyterHubVersionTagger(), versions.jupyter_hub_tagger,
], ],
), ),
"minimal-notebook": ImageDescription(parent_image="base-notebook"), "minimal-notebook": ImageDescription(parent_image="base-notebook"),
"scipy-notebook": ImageDescription(parent_image="minimal-notebook"), "scipy-notebook": ImageDescription(parent_image="minimal-notebook"),
"r-notebook": ImageDescription( "r-notebook": ImageDescription(
parent_image="minimal-notebook", parent_image="minimal-notebook",
taggers=[RVersionTagger()], taggers=[versions.r_tagger],
manifests=[RPackagesManifest()], manifests=[r_packages_manifest],
), ),
"julia-notebook": ImageDescription( "julia-notebook": ImageDescription(
parent_image="minimal-notebook", parent_image="minimal-notebook",
taggers=[JuliaVersionTagger()], taggers=[versions.julia_tagger],
manifests=[JuliaPackagesManifest()], manifests=[julia_packages_manifest],
), ),
"tensorflow-notebook": ImageDescription( "tensorflow-notebook": ImageDescription(
parent_image="scipy-notebook", taggers=[TensorflowVersionTagger()] parent_image="scipy-notebook", taggers=[versions.tensorflow_tagger]
), ),
"pytorch-notebook": ImageDescription( "pytorch-notebook": ImageDescription(
parent_image="scipy-notebook", taggers=[PytorchVersionTagger()] parent_image="scipy-notebook", taggers=[versions.python_tagger]
), ),
"datascience-notebook": ImageDescription( "datascience-notebook": ImageDescription(
parent_image="scipy-notebook", parent_image="scipy-notebook",
taggers=[RVersionTagger(), JuliaVersionTagger()], taggers=[versions.r_tagger, versions.julia_tagger],
manifests=[RPackagesManifest(), JuliaPackagesManifest()], manifests=[r_packages_manifest, julia_packages_manifest],
), ),
"pyspark-notebook": ImageDescription( "pyspark-notebook": ImageDescription(
parent_image="scipy-notebook", parent_image="scipy-notebook",
taggers=[SparkVersionTagger(), JavaVersionTagger()], taggers=[versions.spark_tagger, versions.java_tagger],
manifests=[SparkInfoManifest()], manifests=[spark_info_manifest],
), ),
"all-spark-notebook": ImageDescription( "all-spark-notebook": ImageDescription(
parent_image="pyspark-notebook", parent_image="pyspark-notebook",
taggers=[RVersionTagger()], taggers=[versions.r_tagger],
manifests=[RPackagesManifest()], manifests=[r_packages_manifest],
), ),
} }

14
tagging/manifests/apt_packages.py
View File

@@ -2,14 +2,12 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from docker.models.containers import Container from docker.models.containers import Container
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece from tagging.manifests.manifest_interface import MarkdownPiece
from tagging.utils.quoted_output import quoted_output from tagging.utils.quoted_output import quoted_output
class AptPackagesManifest(ManifestInterface): def apt_packages_manifest(container: Container) -> MarkdownPiece:
@staticmethod return MarkdownPiece(
def markdown_piece(container: Container) -> MarkdownPiece: title="## Apt Packages",
return MarkdownPiece( sections=[quoted_output(container, "apt list --installed")],
title="## Apt Packages", )
sections=[quoted_output(container, "apt list --installed")],
)

51
tagging/manifests/build_info.py
View File

@@ -25,36 +25,33 @@ class BuildInfoConfig:
return f"{self.registry}/{self.owner}/{self.image}" return f"{self.registry}/{self.owner}/{self.image}"
class BuildInfo: def build_info_manifest(config: BuildInfoConfig) -> MarkdownPiece:
"""BuildInfo doesn't fall under common interface, and we run it separately""" """BuildInfo doesn't fall under common interface, and we run it separately"""
commit_hash = GitHelper.commit_hash()
commit_hash_tag = GitHelper.commit_hash_tag()
commit_message = GitHelper.commit_message()
@staticmethod # Unfortunately, `docker images` doesn't work when specifying `docker.io` as registry
def markdown_piece(config: BuildInfoConfig) -> MarkdownPiece: fixed_registry = config.registry + "/" if config.registry != "docker.io" else ""
commit_hash = GitHelper.commit_hash()
commit_hash_tag = GitHelper.commit_hash_tag()
commit_message = GitHelper.commit_message()
# Unfortunately, `docker images` doesn't work when specifying `docker.io` as registry image_size = docker[
fixed_registry = config.registry + "/" if config.registry != "docker.io" else "" "images",
f"{fixed_registry}{config.owner}/{config.image}:latest",
"--format",
"{{.Size}}",
]().rstrip()
image_size = docker[ build_info = textwrap.dedent(
"images", f"""\
f"{fixed_registry}{config.owner}/{config.image}:latest", - Build timestamp: {config.build_timestamp}
"--format", - Docker image: `{config.full_image()}:{commit_hash_tag}`
"{{.Size}}", - Docker image size: {image_size}
]().rstrip() - Git commit SHA: [{commit_hash}](https://github.com/{config.repository}/commit/{commit_hash})
- Git commit message:
build_info = textwrap.dedent( ```text
f"""\ {{message}}
- Build timestamp: {config.build_timestamp} ```"""
- Docker image: `{config.full_image()}:{commit_hash_tag}` ).format(message=commit_message)
- Docker image size: {image_size}
- Git commit SHA: [{commit_hash}](https://github.com/{config.repository}/commit/{commit_hash})
- Git commit message:
```text return MarkdownPiece(title="## Build Info", sections=[build_info])
{{message}}
```"""
).format(message=commit_message)
return MarkdownPiece(title="## Build Info", sections=[build_info])

24
tagging/manifests/conda_environment.py
View File

@@ -2,20 +2,18 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from docker.models.containers import Container from docker.models.containers import Container
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece from tagging.manifests.manifest_interface import MarkdownPiece
from tagging.utils.docker_runner import DockerRunner from tagging.utils.docker_runner import DockerRunner
from tagging.utils.quoted_output import quoted_output from tagging.utils.quoted_output import quoted_output
class CondaEnvironmentManifest(ManifestInterface): def conda_environment_manifest(container: Container) -> MarkdownPiece:
@staticmethod return MarkdownPiece(
def markdown_piece(container: Container) -> MarkdownPiece: title="## Python Packages",
return MarkdownPiece( sections=[
title="## Python Packages", DockerRunner.exec_cmd(container, "python --version"),
sections=[ quoted_output(container, "conda info"),
DockerRunner.exec_cmd(container, "python --version"), quoted_output(container, "mamba info"),
quoted_output(container, "conda info"), quoted_output(container, "mamba list"),
quoted_output(container, "mamba info"), ],
quoted_output(container, "mamba list"), )
],
)

24
tagging/manifests/julia_packages.py
View File

@@ -2,19 +2,17 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from docker.models.containers import Container from docker.models.containers import Container
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece from tagging.manifests.manifest_interface import MarkdownPiece
from tagging.utils.quoted_output import quoted_output from tagging.utils.quoted_output import quoted_output
class JuliaPackagesManifest(ManifestInterface): def julia_packages_manifest(container: Container) -> MarkdownPiece:
@staticmethod return MarkdownPiece(
def markdown_piece(container: Container) -> MarkdownPiece: title="## Julia Packages",
return MarkdownPiece( sections=[
title="## Julia Packages", quoted_output(
sections=[ container, "julia -E 'using InteractiveUtils; versioninfo()'"
quoted_output( ),
container, "julia -E 'using InteractiveUtils; versioninfo()'" quoted_output(container, "julia -E 'import Pkg; Pkg.status()'"),
), ],
quoted_output(container, "julia -E 'import Pkg; Pkg.status()'"), )
],
)

8
tagging/manifests/manifest_interface.py
View File

@@ -1,5 +1,6 @@
# Copyright (c) Jupyter Development Team. # Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from collections.abc import Callable
from dataclasses import dataclass from dataclasses import dataclass
from docker.models.containers import Container from docker.models.containers import Container
@@ -18,9 +19,4 @@ class MarkdownPiece:
return "\n\n".join([self.title, *self.sections]) return "\n\n".join([self.title, *self.sections])
class ManifestInterface: ManifestInterface = Callable[[Container], MarkdownPiece]
"""Common interface for all manifests"""
@staticmethod
def markdown_piece(container: Container) -> MarkdownPiece:
raise NotImplementedError

24
tagging/manifests/r_packages.py
View File

@@ -2,19 +2,17 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from docker.models.containers import Container from docker.models.containers import Container
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece from tagging.manifests.manifest_interface import MarkdownPiece
from tagging.utils.quoted_output import quoted_output from tagging.utils.quoted_output import quoted_output
class RPackagesManifest(ManifestInterface): def r_packages_manifest(container: Container) -> MarkdownPiece:
@staticmethod return MarkdownPiece(
def markdown_piece(container: Container) -> MarkdownPiece: title="## R Packages",
return MarkdownPiece( sections=[
title="## R Packages", quoted_output(container, "R --version"),
sections=[ quoted_output(
quoted_output(container, "R --version"), container, "R --silent -e 'installed.packages(.Library)[, c(1,3)]'"
quoted_output( ),
container, "R --silent -e 'installed.packages(.Library)[, c(1,3)]'" ],
), )
],
)

18
tagging/manifests/spark_info.py
View File

@@ -2,16 +2,14 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from docker.models.containers import Container from docker.models.containers import Container
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece from tagging.manifests.manifest_interface import MarkdownPiece
from tagging.utils.quoted_output import quoted_output from tagging.utils.quoted_output import quoted_output
class SparkInfoManifest(ManifestInterface): def spark_info_manifest(container: Container) -> MarkdownPiece:
@staticmethod return MarkdownPiece(
def markdown_piece(container: Container) -> MarkdownPiece: title="## Apache Spark",
return MarkdownPiece( sections=[
title="## Apache Spark", quoted_output(container, "/usr/local/spark/bin/spark-submit --version")
sections=[ ],
quoted_output(container, "/usr/local/spark/bin/spark-submit --version") )
],
)

8
tagging/taggers/date.py
View File

@@ -4,10 +4,6 @@ import datetime
from docker.models.containers import Container from docker.models.containers import Container
from tagging.taggers.tagger_interface import TaggerInterface
def date_tagger(container: Container) -> str:
class DateTagger(TaggerInterface): return datetime.datetime.now(datetime.UTC).strftime("%Y-%m-%d")
@staticmethod
def tag_value(container: Container) -> str:
return datetime.datetime.now(datetime.UTC).strftime("%Y-%m-%d")

7
tagging/taggers/sha.py
View File

@@ -2,11 +2,8 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from docker.models.containers import Container from docker.models.containers import Container
from tagging.taggers.tagger_interface import TaggerInterface
from tagging.utils.git_helper import GitHelper from tagging.utils.git_helper import GitHelper
class SHATagger(TaggerInterface): def commit_sha_tagger(container: Container) -> str:
@staticmethod return GitHelper.commit_hash_tag()
def tag_value(container: Container) -> str:
return GitHelper.commit_hash_tag()

10
tagging/taggers/tagger_interface.py
View File

@@ -1,11 +1,7 @@
# Copyright (c) Jupyter Development Team. # Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from collections.abc import Callable
from docker.models.containers import Container from docker.models.containers import Container
TaggerInterface = Callable[[Container], str]
class TaggerInterface:
"""Common interface for all taggers"""
@staticmethod
def tag_value(container: Container) -> str:
raise NotImplementedError

21
tagging/taggers/ubuntu_version.py
View File

@@ -2,18 +2,15 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from docker.models.containers import Container from docker.models.containers import Container
from tagging.taggers.tagger_interface import TaggerInterface
from tagging.utils.docker_runner import DockerRunner from tagging.utils.docker_runner import DockerRunner
class UbuntuVersionTagger(TaggerInterface): def ubuntu_version_tagger(container: Container) -> str:
@staticmethod os_release = DockerRunner.exec_cmd(
def tag_value(container: Container) -> str: container,
os_release = DockerRunner.exec_cmd( "cat /etc/os-release",
container, ).split("\n")
"cat /etc/os-release", for line in os_release:
).split("\n") if line.startswith("VERSION_ID"):
for line in os_release: return "ubuntu-" + line.split("=")[1].strip('"')
if line.startswith("VERSION_ID"): raise RuntimeError(f"did not find ubuntu version in: {os_release}")
return "ubuntu-" + line.split("=")[1].strip('"')
raise RuntimeError(f"did not find ubuntu version in: {os_release}")

101
tagging/taggers/versions.py
View File

@@ -2,7 +2,6 @@
# Distributed under the terms of the Modified BSD License. # Distributed under the terms of the Modified BSD License.
from docker.models.containers import Container from docker.models.containers import Container
from tagging.taggers.tagger_interface import TaggerInterface
from tagging.utils.docker_runner import DockerRunner from tagging.utils.docker_runner import DockerRunner
@@ -23,92 +22,66 @@ def _get_pip_package_version(container: Container, package: str) -> str:
return version_line[len(PIP_VERSION_PREFIX) :] return version_line[len(PIP_VERSION_PREFIX) :]
class PythonVersionTagger(TaggerInterface): def python_tagger(container: Container) -> str:
@staticmethod return "python-" + _get_program_version(container, "python").split()[1]
def tag_value(container: Container) -> str:
return "python-" + _get_program_version(container, "python").split()[1]
class PythonMajorMinorVersionTagger(TaggerInterface): def python_major_minor_tagger(container: Container) -> str:
@staticmethod full_version = python_tagger(container)
def tag_value(container: Container) -> str: return full_version[: full_version.rfind(".")]
full_version = PythonVersionTagger.tag_value(container)
return full_version[: full_version.rfind(".")]
class MambaVersionTagger(TaggerInterface): def mamba_tagger(container: Container) -> str:
@staticmethod return "mamba-" + _get_program_version(container, "mamba")
def tag_value(container: Container) -> str:
return "mamba-" + _get_program_version(container, "mamba")
class CondaVersionTagger(TaggerInterface): def conda_tagger(container: Container) -> str:
@staticmethod return "conda-" + _get_program_version(container, "conda").split()[1]
def tag_value(container: Container) -> str:
return "conda-" + _get_program_version(container, "conda").split()[1]
class JupyterNotebookVersionTagger(TaggerInterface): def jupyter_notebook_tagger(container: Container) -> str:
@staticmethod return "notebook-" + _get_program_version(container, "jupyter-notebook")
def tag_value(container: Container) -> str:
return "notebook-" + _get_program_version(container, "jupyter-notebook")
class JupyterLabVersionTagger(TaggerInterface): def jupyter_lab_tagger(container: Container) -> str:
@staticmethod return "lab-" + _get_program_version(container, "jupyter-lab")
def tag_value(container: Container) -> str:
return "lab-" + _get_program_version(container, "jupyter-lab")
class JupyterHubVersionTagger(TaggerInterface): def jupyter_hub_tagger(container: Container) -> str:
@staticmethod return "hub-" + _get_program_version(container, "jupyterhub")
def tag_value(container: Container) -> str:
return "hub-" + _get_program_version(container, "jupyterhub")
class RVersionTagger(TaggerInterface): def r_tagger(container: Container) -> str:
@staticmethod return "r-" + _get_program_version(container, "R").split()[2]
def tag_value(container: Container) -> str:
return "r-" + _get_program_version(container, "R").split()[2]
class JuliaVersionTagger(TaggerInterface): def julia_tagger(container: Container) -> str:
@staticmethod return "julia-" + _get_program_version(container, "julia").split()[2]
def tag_value(container: Container) -> str:
return "julia-" + _get_program_version(container, "julia").split()[2]
class TensorflowVersionTagger(TaggerInterface): def tensorflow_tagger(container: Container) -> str:
@staticmethod try:
def tag_value(container: Container) -> str: return "tensorflow-" + _get_pip_package_version(container, "tensorflow")
try: except AssertionError:
return "tensorflow-" + _get_pip_package_version(container, "tensorflow") return "tensorflow-" + _get_pip_package_version(container, "tensorflow-cpu")
except AssertionError:
return "tensorflow-" + _get_pip_package_version(container, "tensorflow-cpu")
class PytorchVersionTagger(TaggerInterface): def pytorch_tagger(container: Container) -> str:
@staticmethod return "pytorch-" + _get_pip_package_version(container, "torch").split("+")[0]
def tag_value(container: Container) -> str:
return "pytorch-" + _get_pip_package_version(container, "torch").split("+")[0]
class SparkVersionTagger(TaggerInterface): def spark_tagger(container: Container) -> str:
@staticmethod SPARK_VERSION_LINE_PREFIX = r" /___/ .__/\_,_/_/ /_/\_\ version"
def tag_value(container: Container) -> str:
SPARK_VERSION_LINE_PREFIX = r" /___/ .__/\_,_/_/ /_/\_\ version"
spark_version = _get_program_version(container, "spark-submit") spark_version = _get_program_version(container, "spark-submit")
version_line = next( version_line = next(
filter( filter(
lambda line: line.startswith(SPARK_VERSION_LINE_PREFIX), lambda line: line.startswith(SPARK_VERSION_LINE_PREFIX),
spark_version.split("\n"), spark_version.split("\n"),
)
) )
return "spark-" + version_line.split(" ")[-1] )
return "spark-" + version_line.split(" ")[-1]
class JavaVersionTagger(TaggerInterface): def java_tagger(container: Container) -> str:
@staticmethod return "java-" + _get_program_version(container, "java").split()[1]
def tag_value(container: Container) -> str:
return "java-" + _get_program_version(container, "java").split()[1]