mirror of
https://github.com/jupyter/docker-stacks.git
synced 2025-10-07 01:54:04 +00:00
Make taggers manifest functions (#2252)
* Make taggers and manifests functions * Add changelog
This commit is contained in:
@@ -8,6 +8,7 @@ All image manifests can be found in [the wiki](https://github.com/jupyter/docker
|
||||
Affected: all images.
|
||||
|
||||
- **Non-breaking:** Add `conda` and `mamba` version taggers ([#2251](https://github.com/jupyter/docker-stacks/pull/2251)).
|
||||
- **Non-breaking:** Make taggers and manifests functions ([#2252](https://github.com/jupyter/docker-stacks/pull/2252)).
|
||||
|
||||
## 2025-02-21
|
||||
|
||||
|
@@ -1,7 +1,8 @@
|
||||
# Tagging and manifest creation
|
||||
|
||||
The main purpose of the source code in [the `tagging` folder](https://github.com/jupyter/docker-stacks/tree/main/tagging) is to properly write tag files and manifests for single-platform images,
|
||||
apply these tags, and merge single-platform images into one multi-arch image.
|
||||
The main purpose of the source code in [the `tagging` folder](https://github.com/jupyter/docker-stacks/tree/main/tagging) is to
|
||||
properly write tags file, build history line and manifest for a single-platform image,
|
||||
apply these tags, and then merge single-platform images into one multi-arch image.
|
||||
|
||||
## What is a tag and a manifest
|
||||
|
||||
@@ -16,9 +17,9 @@ For example, we dump all `conda` packages with their versions into the manifest.
|
||||
|
||||
- All images are organized in a hierarchical tree.
|
||||
More info on [image relationships](../using/selecting.md#image-relationships).
|
||||
- Classes inherit from `TaggerInterface` and `ManifestInterface` to generate tags and manifest pieces by running commands in Docker containers.
|
||||
- `TaggerInterface` and `ManifestInterface` are interfaces for functions to generate tags and manifest pieces by running commands in Docker containers.
|
||||
- Tags and manifests are reevaluated for each image in the hierarchy since values may change between parent and child images.
|
||||
- To tag an image and create its manifest, run `make hook/<somestack>` (e.g., `make hook/base-notebook`).
|
||||
- To tag an image and create its manifest and build history line, run `make hook/<somestack>` (e.g., `make hook/base-notebook`).
|
||||
|
||||
## Utils
|
||||
|
||||
@@ -46,42 +47,42 @@ The prefix of commit hash (namely, 12 letters) is used as an image tag to make i
|
||||
|
||||
### Tagger
|
||||
|
||||
`Tagger` is a class that can be run inside a docker container to calculate a tag for an image.
|
||||
`Tagger` is a function that runs commands inside a docker container to calculate a tag for an image.
|
||||
|
||||
All the taggers are inherited from `TaggerInterface`:
|
||||
All the taggers follow `TaggerInterface`:
|
||||
|
||||
```{literalinclude} ../../tagging/taggers/tagger_interface.py
|
||||
:language: py
|
||||
:start-at: class TaggerInterface
|
||||
:start-at: TaggerInterface
|
||||
```
|
||||
|
||||
So, the `tag_value(container)` method gets a docker container as an input and returns a tag.
|
||||
So, the `tagger(container)` gets a docker container as an input and returns a tag.
|
||||
|
||||
`SHATagger` example:
|
||||
`commit_sha_tagger` example:
|
||||
|
||||
```{literalinclude} ../../tagging/taggers/sha.py
|
||||
:language: py
|
||||
:start-at: class SHATagger
|
||||
:start-at: def
|
||||
```
|
||||
|
||||
- `taggers/` subdirectory contains all the taggers.
|
||||
- `taggers/` subdirectory contains all taggers.
|
||||
- `apps/write_tags_file.py`, `apps/apply_tags.py`, and `apps/merge_tags.py` are Python executable used to write tags for an image, apply tags from a file, and create multi-arch images.
|
||||
|
||||
### Manifest
|
||||
|
||||
All manifest classes except `BuildInfo` are inherited from `ManifestInterface`
|
||||
and `markdown_piece(container)` method returns a piece of the build manifest.
|
||||
All manifest functions except `build_info_manifest` follow `ManifestInterface`
|
||||
and `manifest(container)` method returns a piece of the build manifest.
|
||||
|
||||
```{literalinclude} ../../tagging/manifests/manifest_interface.py
|
||||
:language: py
|
||||
:start-at: class ManifestInterface
|
||||
:start-at: ManifestInterface
|
||||
```
|
||||
|
||||
`AptPackagesManifest` example:
|
||||
`apt_packages_manifest` example:
|
||||
|
||||
```{literalinclude} ../../tagging/manifests/apt_packages.py
|
||||
:language: py
|
||||
:start-at: class AptPackagesManifest
|
||||
:start-at: def
|
||||
```
|
||||
|
||||
- `quoted_output(container, cmd)` simply runs the command inside a container using `DockerRunner.exec_cmd` and wraps it to triple quotes to create a valid markdown piece.
|
||||
|
@@ -10,7 +10,7 @@ from tagging.apps.common_cli_arguments import common_arguments_parser
|
||||
from tagging.apps.config import Config
|
||||
from tagging.hierarchy.get_manifests import get_manifests
|
||||
from tagging.hierarchy.get_taggers import get_taggers
|
||||
from tagging.manifests.build_info import BuildInfo, BuildInfoConfig
|
||||
from tagging.manifests.build_info import BuildInfoConfig, build_info_manifest
|
||||
from tagging.utils.docker_runner import DockerRunner
|
||||
from tagging.utils.get_prefix import get_file_prefix, get_tag_prefix
|
||||
from tagging.utils.git_helper import GitHelper
|
||||
@@ -27,7 +27,7 @@ def get_build_history_line(config: Config, filename: str, container: Container)
|
||||
|
||||
taggers = get_taggers(config.image)
|
||||
tags_prefix = get_tag_prefix(config.variant)
|
||||
all_tags = [tags_prefix + "-" + tagger.tag_value(container) for tagger in taggers]
|
||||
all_tags = [tags_prefix + "-" + tagger(container) for tagger in taggers]
|
||||
|
||||
date_column = f"`{BUILD_TIMESTAMP}`"
|
||||
image_column = MARKDOWN_LINE_BREAK.join(
|
||||
@@ -64,7 +64,7 @@ def get_manifest(config: Config, commit_hash_tag: str, container: Container) ->
|
||||
LOGGER.info(f"Calculating manifest file for image: {config.image}")
|
||||
|
||||
manifests = get_manifests(config.image)
|
||||
manifest_names = [manifest.__class__.__name__ for manifest in manifests]
|
||||
manifest_names = [manifest.__name__ for manifest in manifests]
|
||||
LOGGER.info(f"Using manifests: {manifest_names}")
|
||||
|
||||
build_info_config = BuildInfoConfig(
|
||||
@@ -77,8 +77,8 @@ def get_manifest(config: Config, commit_hash_tag: str, container: Container) ->
|
||||
|
||||
markdown_pieces = [
|
||||
f"# Build manifest for image: {config.image}:{commit_hash_tag}",
|
||||
BuildInfo.markdown_piece(build_info_config).get_str(),
|
||||
*(manifest.markdown_piece(container).get_str() for manifest in manifests),
|
||||
build_info_manifest(build_info_config).get_str(),
|
||||
*(manifest(container).get_str() for manifest in manifests),
|
||||
]
|
||||
markdown_content = "\n\n".join(markdown_pieces) + "\n"
|
||||
|
||||
|
@@ -20,8 +20,8 @@ def get_tags(config: Config) -> list[str]:
|
||||
tags = [f"{config.full_image()}:{tags_prefix}-latest"]
|
||||
with DockerRunner(config.full_image()) as container:
|
||||
for tagger in taggers:
|
||||
tagger_name = tagger.__class__.__name__
|
||||
tag_value = tagger.tag_value(container)
|
||||
tagger_name = tagger.__name__
|
||||
tag_value = tagger(container)
|
||||
LOGGER.info(
|
||||
f"Calculated tag, tagger_name: {tagger_name} tag_value: {tag_value}"
|
||||
)
|
||||
|
@@ -2,31 +2,17 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from tagging.manifests.apt_packages import AptPackagesManifest
|
||||
from tagging.manifests.conda_environment import CondaEnvironmentManifest
|
||||
from tagging.manifests.julia_packages import JuliaPackagesManifest
|
||||
from tagging.manifests.apt_packages import apt_packages_manifest
|
||||
from tagging.manifests.conda_environment import conda_environment_manifest
|
||||
from tagging.manifests.julia_packages import julia_packages_manifest
|
||||
from tagging.manifests.manifest_interface import ManifestInterface
|
||||
from tagging.manifests.r_packages import RPackagesManifest
|
||||
from tagging.manifests.spark_info import SparkInfoManifest
|
||||
from tagging.taggers.date import DateTagger
|
||||
from tagging.taggers.sha import SHATagger
|
||||
from tagging.manifests.r_packages import r_packages_manifest
|
||||
from tagging.manifests.spark_info import spark_info_manifest
|
||||
from tagging.taggers import versions
|
||||
from tagging.taggers.date import date_tagger
|
||||
from tagging.taggers.sha import commit_sha_tagger
|
||||
from tagging.taggers.tagger_interface import TaggerInterface
|
||||
from tagging.taggers.ubuntu_version import UbuntuVersionTagger
|
||||
from tagging.taggers.versions import (
|
||||
CondaVersionTagger,
|
||||
JavaVersionTagger,
|
||||
JuliaVersionTagger,
|
||||
JupyterHubVersionTagger,
|
||||
JupyterLabVersionTagger,
|
||||
JupyterNotebookVersionTagger,
|
||||
MambaVersionTagger,
|
||||
PythonMajorMinorVersionTagger,
|
||||
PythonVersionTagger,
|
||||
PytorchVersionTagger,
|
||||
RVersionTagger,
|
||||
SparkVersionTagger,
|
||||
TensorflowVersionTagger,
|
||||
)
|
||||
from tagging.taggers.ubuntu_version import ubuntu_version_tagger
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -40,55 +26,55 @@ ALL_IMAGES = {
|
||||
"docker-stacks-foundation": ImageDescription(
|
||||
parent_image=None,
|
||||
taggers=[
|
||||
SHATagger(),
|
||||
DateTagger(),
|
||||
UbuntuVersionTagger(),
|
||||
PythonMajorMinorVersionTagger(),
|
||||
PythonVersionTagger(),
|
||||
MambaVersionTagger(),
|
||||
CondaVersionTagger(),
|
||||
commit_sha_tagger,
|
||||
date_tagger,
|
||||
ubuntu_version_tagger,
|
||||
versions.python_major_minor_tagger,
|
||||
versions.python_tagger,
|
||||
versions.mamba_tagger,
|
||||
versions.conda_tagger,
|
||||
],
|
||||
manifests=[CondaEnvironmentManifest(), AptPackagesManifest()],
|
||||
manifests=[conda_environment_manifest, apt_packages_manifest],
|
||||
),
|
||||
"base-notebook": ImageDescription(
|
||||
parent_image="docker-stacks-foundation",
|
||||
taggers=[
|
||||
JupyterNotebookVersionTagger(),
|
||||
JupyterLabVersionTagger(),
|
||||
JupyterHubVersionTagger(),
|
||||
versions.jupyter_notebook_tagger,
|
||||
versions.jupyter_lab_tagger,
|
||||
versions.jupyter_hub_tagger,
|
||||
],
|
||||
),
|
||||
"minimal-notebook": ImageDescription(parent_image="base-notebook"),
|
||||
"scipy-notebook": ImageDescription(parent_image="minimal-notebook"),
|
||||
"r-notebook": ImageDescription(
|
||||
parent_image="minimal-notebook",
|
||||
taggers=[RVersionTagger()],
|
||||
manifests=[RPackagesManifest()],
|
||||
taggers=[versions.r_tagger],
|
||||
manifests=[r_packages_manifest],
|
||||
),
|
||||
"julia-notebook": ImageDescription(
|
||||
parent_image="minimal-notebook",
|
||||
taggers=[JuliaVersionTagger()],
|
||||
manifests=[JuliaPackagesManifest()],
|
||||
taggers=[versions.julia_tagger],
|
||||
manifests=[julia_packages_manifest],
|
||||
),
|
||||
"tensorflow-notebook": ImageDescription(
|
||||
parent_image="scipy-notebook", taggers=[TensorflowVersionTagger()]
|
||||
parent_image="scipy-notebook", taggers=[versions.tensorflow_tagger]
|
||||
),
|
||||
"pytorch-notebook": ImageDescription(
|
||||
parent_image="scipy-notebook", taggers=[PytorchVersionTagger()]
|
||||
parent_image="scipy-notebook", taggers=[versions.python_tagger]
|
||||
),
|
||||
"datascience-notebook": ImageDescription(
|
||||
parent_image="scipy-notebook",
|
||||
taggers=[RVersionTagger(), JuliaVersionTagger()],
|
||||
manifests=[RPackagesManifest(), JuliaPackagesManifest()],
|
||||
taggers=[versions.r_tagger, versions.julia_tagger],
|
||||
manifests=[r_packages_manifest, julia_packages_manifest],
|
||||
),
|
||||
"pyspark-notebook": ImageDescription(
|
||||
parent_image="scipy-notebook",
|
||||
taggers=[SparkVersionTagger(), JavaVersionTagger()],
|
||||
manifests=[SparkInfoManifest()],
|
||||
taggers=[versions.spark_tagger, versions.java_tagger],
|
||||
manifests=[spark_info_manifest],
|
||||
),
|
||||
"all-spark-notebook": ImageDescription(
|
||||
parent_image="pyspark-notebook",
|
||||
taggers=[RVersionTagger()],
|
||||
manifests=[RPackagesManifest()],
|
||||
taggers=[versions.r_tagger],
|
||||
manifests=[r_packages_manifest],
|
||||
),
|
||||
}
|
||||
|
@@ -2,14 +2,12 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
|
||||
from tagging.manifests.manifest_interface import MarkdownPiece
|
||||
from tagging.utils.quoted_output import quoted_output
|
||||
|
||||
|
||||
class AptPackagesManifest(ManifestInterface):
|
||||
@staticmethod
|
||||
def markdown_piece(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## Apt Packages",
|
||||
sections=[quoted_output(container, "apt list --installed")],
|
||||
)
|
||||
def apt_packages_manifest(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## Apt Packages",
|
||||
sections=[quoted_output(container, "apt list --installed")],
|
||||
)
|
||||
|
@@ -25,36 +25,33 @@ class BuildInfoConfig:
|
||||
return f"{self.registry}/{self.owner}/{self.image}"
|
||||
|
||||
|
||||
class BuildInfo:
|
||||
def build_info_manifest(config: BuildInfoConfig) -> MarkdownPiece:
|
||||
"""BuildInfo doesn't fall under common interface, and we run it separately"""
|
||||
commit_hash = GitHelper.commit_hash()
|
||||
commit_hash_tag = GitHelper.commit_hash_tag()
|
||||
commit_message = GitHelper.commit_message()
|
||||
|
||||
@staticmethod
|
||||
def markdown_piece(config: BuildInfoConfig) -> MarkdownPiece:
|
||||
commit_hash = GitHelper.commit_hash()
|
||||
commit_hash_tag = GitHelper.commit_hash_tag()
|
||||
commit_message = GitHelper.commit_message()
|
||||
# Unfortunately, `docker images` doesn't work when specifying `docker.io` as registry
|
||||
fixed_registry = config.registry + "/" if config.registry != "docker.io" else ""
|
||||
|
||||
# Unfortunately, `docker images` doesn't work when specifying `docker.io` as registry
|
||||
fixed_registry = config.registry + "/" if config.registry != "docker.io" else ""
|
||||
image_size = docker[
|
||||
"images",
|
||||
f"{fixed_registry}{config.owner}/{config.image}:latest",
|
||||
"--format",
|
||||
"{{.Size}}",
|
||||
]().rstrip()
|
||||
|
||||
image_size = docker[
|
||||
"images",
|
||||
f"{fixed_registry}{config.owner}/{config.image}:latest",
|
||||
"--format",
|
||||
"{{.Size}}",
|
||||
]().rstrip()
|
||||
build_info = textwrap.dedent(
|
||||
f"""\
|
||||
- Build timestamp: {config.build_timestamp}
|
||||
- Docker image: `{config.full_image()}:{commit_hash_tag}`
|
||||
- Docker image size: {image_size}
|
||||
- Git commit SHA: [{commit_hash}](https://github.com/{config.repository}/commit/{commit_hash})
|
||||
- Git commit message:
|
||||
|
||||
build_info = textwrap.dedent(
|
||||
f"""\
|
||||
- Build timestamp: {config.build_timestamp}
|
||||
- Docker image: `{config.full_image()}:{commit_hash_tag}`
|
||||
- Docker image size: {image_size}
|
||||
- Git commit SHA: [{commit_hash}](https://github.com/{config.repository}/commit/{commit_hash})
|
||||
- Git commit message:
|
||||
```text
|
||||
{{message}}
|
||||
```"""
|
||||
).format(message=commit_message)
|
||||
|
||||
```text
|
||||
{{message}}
|
||||
```"""
|
||||
).format(message=commit_message)
|
||||
|
||||
return MarkdownPiece(title="## Build Info", sections=[build_info])
|
||||
return MarkdownPiece(title="## Build Info", sections=[build_info])
|
||||
|
@@ -2,20 +2,18 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
|
||||
from tagging.manifests.manifest_interface import MarkdownPiece
|
||||
from tagging.utils.docker_runner import DockerRunner
|
||||
from tagging.utils.quoted_output import quoted_output
|
||||
|
||||
|
||||
class CondaEnvironmentManifest(ManifestInterface):
|
||||
@staticmethod
|
||||
def markdown_piece(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## Python Packages",
|
||||
sections=[
|
||||
DockerRunner.exec_cmd(container, "python --version"),
|
||||
quoted_output(container, "conda info"),
|
||||
quoted_output(container, "mamba info"),
|
||||
quoted_output(container, "mamba list"),
|
||||
],
|
||||
)
|
||||
def conda_environment_manifest(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## Python Packages",
|
||||
sections=[
|
||||
DockerRunner.exec_cmd(container, "python --version"),
|
||||
quoted_output(container, "conda info"),
|
||||
quoted_output(container, "mamba info"),
|
||||
quoted_output(container, "mamba list"),
|
||||
],
|
||||
)
|
||||
|
@@ -2,19 +2,17 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
|
||||
from tagging.manifests.manifest_interface import MarkdownPiece
|
||||
from tagging.utils.quoted_output import quoted_output
|
||||
|
||||
|
||||
class JuliaPackagesManifest(ManifestInterface):
|
||||
@staticmethod
|
||||
def markdown_piece(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## Julia Packages",
|
||||
sections=[
|
||||
quoted_output(
|
||||
container, "julia -E 'using InteractiveUtils; versioninfo()'"
|
||||
),
|
||||
quoted_output(container, "julia -E 'import Pkg; Pkg.status()'"),
|
||||
],
|
||||
)
|
||||
def julia_packages_manifest(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## Julia Packages",
|
||||
sections=[
|
||||
quoted_output(
|
||||
container, "julia -E 'using InteractiveUtils; versioninfo()'"
|
||||
),
|
||||
quoted_output(container, "julia -E 'import Pkg; Pkg.status()'"),
|
||||
],
|
||||
)
|
||||
|
@@ -1,5 +1,6 @@
|
||||
# Copyright (c) Jupyter Development Team.
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
|
||||
from docker.models.containers import Container
|
||||
@@ -18,9 +19,4 @@ class MarkdownPiece:
|
||||
return "\n\n".join([self.title, *self.sections])
|
||||
|
||||
|
||||
class ManifestInterface:
|
||||
"""Common interface for all manifests"""
|
||||
|
||||
@staticmethod
|
||||
def markdown_piece(container: Container) -> MarkdownPiece:
|
||||
raise NotImplementedError
|
||||
ManifestInterface = Callable[[Container], MarkdownPiece]
|
||||
|
@@ -2,19 +2,17 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
|
||||
from tagging.manifests.manifest_interface import MarkdownPiece
|
||||
from tagging.utils.quoted_output import quoted_output
|
||||
|
||||
|
||||
class RPackagesManifest(ManifestInterface):
|
||||
@staticmethod
|
||||
def markdown_piece(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## R Packages",
|
||||
sections=[
|
||||
quoted_output(container, "R --version"),
|
||||
quoted_output(
|
||||
container, "R --silent -e 'installed.packages(.Library)[, c(1,3)]'"
|
||||
),
|
||||
],
|
||||
)
|
||||
def r_packages_manifest(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## R Packages",
|
||||
sections=[
|
||||
quoted_output(container, "R --version"),
|
||||
quoted_output(
|
||||
container, "R --silent -e 'installed.packages(.Library)[, c(1,3)]'"
|
||||
),
|
||||
],
|
||||
)
|
||||
|
@@ -2,16 +2,14 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.manifests.manifest_interface import ManifestInterface, MarkdownPiece
|
||||
from tagging.manifests.manifest_interface import MarkdownPiece
|
||||
from tagging.utils.quoted_output import quoted_output
|
||||
|
||||
|
||||
class SparkInfoManifest(ManifestInterface):
|
||||
@staticmethod
|
||||
def markdown_piece(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## Apache Spark",
|
||||
sections=[
|
||||
quoted_output(container, "/usr/local/spark/bin/spark-submit --version")
|
||||
],
|
||||
)
|
||||
def spark_info_manifest(container: Container) -> MarkdownPiece:
|
||||
return MarkdownPiece(
|
||||
title="## Apache Spark",
|
||||
sections=[
|
||||
quoted_output(container, "/usr/local/spark/bin/spark-submit --version")
|
||||
],
|
||||
)
|
||||
|
@@ -4,10 +4,6 @@ import datetime
|
||||
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.taggers.tagger_interface import TaggerInterface
|
||||
|
||||
|
||||
class DateTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return datetime.datetime.now(datetime.UTC).strftime("%Y-%m-%d")
|
||||
def date_tagger(container: Container) -> str:
|
||||
return datetime.datetime.now(datetime.UTC).strftime("%Y-%m-%d")
|
||||
|
@@ -2,11 +2,8 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.taggers.tagger_interface import TaggerInterface
|
||||
from tagging.utils.git_helper import GitHelper
|
||||
|
||||
|
||||
class SHATagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return GitHelper.commit_hash_tag()
|
||||
def commit_sha_tagger(container: Container) -> str:
|
||||
return GitHelper.commit_hash_tag()
|
||||
|
@@ -1,11 +1,7 @@
|
||||
# Copyright (c) Jupyter Development Team.
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from collections.abc import Callable
|
||||
|
||||
from docker.models.containers import Container
|
||||
|
||||
|
||||
class TaggerInterface:
|
||||
"""Common interface for all taggers"""
|
||||
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
raise NotImplementedError
|
||||
TaggerInterface = Callable[[Container], str]
|
||||
|
@@ -2,18 +2,15 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.taggers.tagger_interface import TaggerInterface
|
||||
from tagging.utils.docker_runner import DockerRunner
|
||||
|
||||
|
||||
class UbuntuVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
os_release = DockerRunner.exec_cmd(
|
||||
container,
|
||||
"cat /etc/os-release",
|
||||
).split("\n")
|
||||
for line in os_release:
|
||||
if line.startswith("VERSION_ID"):
|
||||
return "ubuntu-" + line.split("=")[1].strip('"')
|
||||
raise RuntimeError(f"did not find ubuntu version in: {os_release}")
|
||||
def ubuntu_version_tagger(container: Container) -> str:
|
||||
os_release = DockerRunner.exec_cmd(
|
||||
container,
|
||||
"cat /etc/os-release",
|
||||
).split("\n")
|
||||
for line in os_release:
|
||||
if line.startswith("VERSION_ID"):
|
||||
return "ubuntu-" + line.split("=")[1].strip('"')
|
||||
raise RuntimeError(f"did not find ubuntu version in: {os_release}")
|
||||
|
@@ -2,7 +2,6 @@
|
||||
# Distributed under the terms of the Modified BSD License.
|
||||
from docker.models.containers import Container
|
||||
|
||||
from tagging.taggers.tagger_interface import TaggerInterface
|
||||
from tagging.utils.docker_runner import DockerRunner
|
||||
|
||||
|
||||
@@ -23,92 +22,66 @@ def _get_pip_package_version(container: Container, package: str) -> str:
|
||||
return version_line[len(PIP_VERSION_PREFIX) :]
|
||||
|
||||
|
||||
class PythonVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "python-" + _get_program_version(container, "python").split()[1]
|
||||
def python_tagger(container: Container) -> str:
|
||||
return "python-" + _get_program_version(container, "python").split()[1]
|
||||
|
||||
|
||||
class PythonMajorMinorVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
full_version = PythonVersionTagger.tag_value(container)
|
||||
return full_version[: full_version.rfind(".")]
|
||||
def python_major_minor_tagger(container: Container) -> str:
|
||||
full_version = python_tagger(container)
|
||||
return full_version[: full_version.rfind(".")]
|
||||
|
||||
|
||||
class MambaVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "mamba-" + _get_program_version(container, "mamba")
|
||||
def mamba_tagger(container: Container) -> str:
|
||||
return "mamba-" + _get_program_version(container, "mamba")
|
||||
|
||||
|
||||
class CondaVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "conda-" + _get_program_version(container, "conda").split()[1]
|
||||
def conda_tagger(container: Container) -> str:
|
||||
return "conda-" + _get_program_version(container, "conda").split()[1]
|
||||
|
||||
|
||||
class JupyterNotebookVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "notebook-" + _get_program_version(container, "jupyter-notebook")
|
||||
def jupyter_notebook_tagger(container: Container) -> str:
|
||||
return "notebook-" + _get_program_version(container, "jupyter-notebook")
|
||||
|
||||
|
||||
class JupyterLabVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "lab-" + _get_program_version(container, "jupyter-lab")
|
||||
def jupyter_lab_tagger(container: Container) -> str:
|
||||
return "lab-" + _get_program_version(container, "jupyter-lab")
|
||||
|
||||
|
||||
class JupyterHubVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "hub-" + _get_program_version(container, "jupyterhub")
|
||||
def jupyter_hub_tagger(container: Container) -> str:
|
||||
return "hub-" + _get_program_version(container, "jupyterhub")
|
||||
|
||||
|
||||
class RVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "r-" + _get_program_version(container, "R").split()[2]
|
||||
def r_tagger(container: Container) -> str:
|
||||
return "r-" + _get_program_version(container, "R").split()[2]
|
||||
|
||||
|
||||
class JuliaVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "julia-" + _get_program_version(container, "julia").split()[2]
|
||||
def julia_tagger(container: Container) -> str:
|
||||
return "julia-" + _get_program_version(container, "julia").split()[2]
|
||||
|
||||
|
||||
class TensorflowVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
try:
|
||||
return "tensorflow-" + _get_pip_package_version(container, "tensorflow")
|
||||
except AssertionError:
|
||||
return "tensorflow-" + _get_pip_package_version(container, "tensorflow-cpu")
|
||||
def tensorflow_tagger(container: Container) -> str:
|
||||
try:
|
||||
return "tensorflow-" + _get_pip_package_version(container, "tensorflow")
|
||||
except AssertionError:
|
||||
return "tensorflow-" + _get_pip_package_version(container, "tensorflow-cpu")
|
||||
|
||||
|
||||
class PytorchVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "pytorch-" + _get_pip_package_version(container, "torch").split("+")[0]
|
||||
def pytorch_tagger(container: Container) -> str:
|
||||
return "pytorch-" + _get_pip_package_version(container, "torch").split("+")[0]
|
||||
|
||||
|
||||
class SparkVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
SPARK_VERSION_LINE_PREFIX = r" /___/ .__/\_,_/_/ /_/\_\ version"
|
||||
def spark_tagger(container: Container) -> str:
|
||||
SPARK_VERSION_LINE_PREFIX = r" /___/ .__/\_,_/_/ /_/\_\ version"
|
||||
|
||||
spark_version = _get_program_version(container, "spark-submit")
|
||||
version_line = next(
|
||||
filter(
|
||||
lambda line: line.startswith(SPARK_VERSION_LINE_PREFIX),
|
||||
spark_version.split("\n"),
|
||||
)
|
||||
spark_version = _get_program_version(container, "spark-submit")
|
||||
version_line = next(
|
||||
filter(
|
||||
lambda line: line.startswith(SPARK_VERSION_LINE_PREFIX),
|
||||
spark_version.split("\n"),
|
||||
)
|
||||
return "spark-" + version_line.split(" ")[-1]
|
||||
)
|
||||
return "spark-" + version_line.split(" ")[-1]
|
||||
|
||||
|
||||
class JavaVersionTagger(TaggerInterface):
|
||||
@staticmethod
|
||||
def tag_value(container: Container) -> str:
|
||||
return "java-" + _get_program_version(container, "java").split()[1]
|
||||
def java_tagger(container: Container) -> str:
|
||||
return "java-" + _get_program_version(container, "java").split()[1]
|
||||
|
Reference in New Issue
Block a user