hazza/docker-stacks
1
0
Fork 0
mirror of https://github.com/jupyter/docker-stacks.git synced 2025-10-16 22:42:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into asalikhov/ubuntu_jammy

Browse Source
This commit is contained in:
Ayaz Salikhov
2022-07-09 16:50:05 +04:00
parent 061b2b2d4c 18b10e7f73
commit 103da29c0a
30 changed files with 127 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/conf.py
View File

@@ -99,7 +99,7 @@ linkcheck_ignore = [
]
linkcheck_allowed_redirects = {
r"https://results\.pre-commit\.ci/latest/github/jupyter/docker-stacks/master": r"https://results\.pre-commit\.ci/run/github/.*", # Latest master CI build
r"https://results\.pre-commit\.ci/latest/github/jupyter/docker-stacks/main": r"https://results\.pre-commit\.ci/run/github/.*", # Latest main CI build
r"https://github\.com/jupyter/docker-stacks/issues/new.*": r"https://github\.com/login.*", # GitHub wants user to be logon to use this features
r"https://github\.com/orgs/jupyter/teams/docker-image-maintainers/members": r"https://github\.com/login.*",
}

2
docs/contributing/issues.md
View File

@@ -5,7 +5,7 @@ Please review the following guidelines when reporting your problem.
- If you believe you’ve found a security vulnerability in any of the Jupyter projects included in Jupyter Docker Stacks images,
please report it to [security@ipython.org](mailto:security@ipython.org), **not in the issue trackers on GitHub**.
If you prefer to encrypt your security reports, you can use [this PGP public key](https://github.com/jupyter/jupyter.github.io/blob/master/assets/ipython_security.asc).
If you prefer to encrypt your security reports, you can use [this PGP public key](https://github.com/jupyter/jupyter.github.io/blob/HEAD/assets/ipython_security.asc).
- If you think your problem is unique to the Jupyter Docker Stacks images,
please search the [jupyter/docker-stacks issue tracker](https://github.com/jupyter/docker-stacks/issues)
to see if someone else has already reported the same problem.

12
docs/contributing/stacks.md
View File

@@ -80,12 +80,11 @@ The cookiecutter template comes with a `.github/workflows/docker.yml` file, whic
push:
branches:
- main
- master
paths-ignore:
- "*.md"
```
This will trigger the CI pipeline whenever you push to your `main` or `master` branch and when any Pull Requests are made to your repository.
This will trigger the CI pipeline whenever you push to your `main` branch and when any Pull Requests are made to your repository.
For more details on this configuration, visit the [GitHub actions documentation on triggers](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows).
2. Commit your changes and push to GitHub.
@@ -99,7 +98,7 @@ The cookiecutter template comes with a `.github/workflows/docker.yml` file, whic
## Configuring Docker Hub
Now, configure Docker Hub to build your stack image and push it to Docker Hub repository whenever
you merge a GitHub pull request to the master branch of your project.
you merge a GitHub pull request to the main branch of your project.
1. Visit [https://hub.docker.com/](https://hub.docker.com/) and log in.
2. Select the account or organization matching the one you entered when prompted with `stack_org` by the cookiecutter.
@@ -137,14 +136,13 @@ you merge a GitHub pull request to the master branch of your project.
Make edits to the Dockerfile in your project to add third-party libraries and configure Jupyter
applications.
Refer to the Dockerfiles for the core stacks (e.g., [jupyter/datascience-notebook](https://github.com/jupyter/docker-stacks/blob/master/datascience-notebook/Dockerfile))
Refer to the Dockerfiles for the core stacks (e.g., [jupyter/datascience-notebook](https://github.com/jupyter/docker-stacks/blob/main/datascience-notebook/Dockerfile))
to get a feel for what's possible and best practices.
[Submit pull requests](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)
to your project repository on GitHub.
Ensure your image builds correctly on GitHub actions before merging to
master or main.
Refer to Docker Hub to build your master or main branch that you can `docker pull`.
Ensure your image builds correctly on GitHub actions before merging to main branch.
Refer to Docker Hub to build your main branch that you can `docker pull`.
## Sharing Your Image

4
docs/contributing/tests.md
View File

@@ -4,7 +4,7 @@ We greatly appreciate pull requests that extend the automated tests that vet the
## How the Tests Work
A [GitHub Action workflow](https://github.com/jupyter/docker-stacks/blob/master/.github/workflows/docker.yml)
A [GitHub Action workflow](https://github.com/jupyter/docker-stacks/blob/main/.github/workflows/docker.yml)
runs tests against pull requests submitted to the `jupyter/docker-stacks` repository.
We use `pytest` module to run tests on the image.
@@ -18,7 +18,7 @@ If your test is located in `tests/<somestack>-notebook/`, it will be run against
```
Many tests make use of global [pytest fixtures](https://docs.pytest.org/en/latest/reference/fixtures.html)
defined in the [conftest.py](https://github.com/jupyter/docker-stacks/blob/master/tests/conftest.py) file.
defined in the [conftest.py](https://github.com/jupyter/docker-stacks/blob/main/tests/conftest.py) file.
## Unit tests

10
docs/maintaining/tasks.md
View File

@@ -10,10 +10,10 @@ To build new images and publish them to the Docker Hub registry, do the followin
```{note}
We think GitHub Actions are quite reliable, so please, investigate if some error occurs.
Building Docker images in PRs is exactly the same after merging to master, except there is an additional `push` step.
Building Docker images in PRs is exactly the same after merging to main, except there is an additional `push` step.
```
4. Try to avoid merging another PR to master until all pending builds are complete.
4. Try to avoid merging another PR to main branch until all pending builds are complete.
This way, you will know which commit might have broken the build and also have correct tags for moving tags (like `python` version).
## Updating the Ubuntu Base Image
@@ -34,10 +34,10 @@ In general, we do not add new core images and ask contributors to either create
When there's a new stack definition, do the following before merging the PR with the new stack:
1. Ensure the PR includes an update to the stack overview diagram
[in the documentation](https://github.com/jupyter/docker-stacks/blob/master/docs/using/selecting.md#image-relationships).
[in the documentation](https://github.com/jupyter/docker-stacks/blob/main/docs/using/selecting.md#image-relationships).
The image links to the [blockdiag source](http://interactive.blockdiag.com/) used to create it.
2. Ensure the PR updates the [Makefile](https://github.com/jupyter/docker-stacks/blob/master/Makefile), which is used to build the stacks in order on GitHub Actions.
3. Ensure necessary tags / manifests are added for the new image in the [tagging](https://github.com/jupyter/docker-stacks/tree/master/tagging) folder.
2. Ensure the PR updates the [Makefile](https://github.com/jupyter/docker-stacks/blob/main/Makefile), which is used to build the stacks in order on GitHub Actions.
3. Ensure necessary tags / manifests are added for the new image in the [tagging](https://github.com/jupyter/docker-stacks/tree/main/tagging) folder.
4. Create a new repository in the `jupyter` org on Docker Hub named after the stack folder in the
git repo.
5. Grant the `stacks` team permission to write to the repo.

6
docs/using/common.md
View File

@@ -133,7 +133,7 @@ or executables (`chmod +x`) to be run to the paths below:
- `/usr/local/bin/before-notebook.d/` - handled **after** all the standard options noted above are applied
and ran right before the notebook server launches
See the `run-hooks` function in the [`jupyter/base-notebook start.sh`](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/start.sh)
See the `run-hooks` function in the [`jupyter/base-notebook start.sh`](https://github.com/jupyter/docker-stacks/blob/main/base-notebook/start.sh)
script for execution details.
## SSL Certificates
@@ -166,10 +166,10 @@ The certificate file or PEM may contain one or more certificates (e.g., server,
For additional information about using SSL, see the following:
- The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples)
- The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/main/examples)
for information about how to use
[Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
- The [`jupyter_server_config.py`](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/jupyter_server_config.py)
- The [`jupyter_server_config.py`](https://github.com/jupyter/docker-stacks/blob/main/base-notebook/jupyter_server_config.py)
file for how this Docker image generates a self-signed certificate.
- The [Jupyter Server documentation](https://jupyter-server.readthedocs.io/en/latest/operators/public-server.html#securing-a-jupyter-server)
for best practices about securing a public notebook server in general.

14
docs/using/recipes.md
View File

@@ -29,7 +29,7 @@ Create a new Dockerfile like the one shown below.
```dockerfile
# Start from a core stack version
FROM jupyter/datascience-notebook:6b49f3337709
FROM jupyter/datascience-notebook:807999a41207
# Install in the default python3 environment
RUN pip install --quiet --no-cache-dir 'flake8==3.9.2' && \
fix-permissions "${CONDA_DIR}" && \
@@ -48,7 +48,7 @@ Next, create a new Dockerfile like the one shown below.
```dockerfile
# Start from a core stack version
FROM jupyter/datascience-notebook:6b49f3337709
FROM jupyter/datascience-notebook:807999a41207
# Install from requirements.txt file
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
RUN pip install --quiet --no-cache-dir --requirement /tmp/requirements.txt && \
@@ -60,7 +60,7 @@ For conda, the Dockerfile is similar:
```dockerfile
# Start from a core stack version
FROM jupyter/datascience-notebook:6b49f3337709
FROM jupyter/datascience-notebook:807999a41207
# Install from requirements.txt file
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
RUN mamba install --yes --file /tmp/requirements.txt && \
@@ -148,7 +148,7 @@ Ref: <https://github.com/jupyter/docker-stacks/issues/999>
## Let's Encrypt a Notebook server
See the README for a basic automation here
<https://github.com/jupyter/docker-stacks/tree/master/examples/make-deploy>
<https://github.com/jupyter/docker-stacks/tree/main/examples/make-deploy>
which includes steps for requesting and renewing a Let's Encrypt certificate.
Ref: <https://github.com/jupyter/docker-stacks/issues/78>
@@ -283,7 +283,7 @@ To use a specific version of JupyterHub, the version of `jupyterhub` in your ima
version in the Hub itself.
```dockerfile
FROM jupyter/base-notebook:6b49f3337709
FROM jupyter/base-notebook:807999a41207
RUN pip install --quiet --no-cache-dir jupyterhub==1.4.1 && \
fix-permissions "${CONDA_DIR}" && \
fix-permissions "/home/${NB_USER}"
@@ -474,7 +474,7 @@ For JupyterLab:
```bash
docker run -it --rm \
jupyter/base-notebook:6b49f3337709 \
jupyter/base-notebook:807999a41207 \
start.sh jupyter lab --LabApp.token=''
```
@@ -482,7 +482,7 @@ For jupyter classic:
```bash
docker run -it --rm \
jupyter/base-notebook:6b49f3337709 \
jupyter/base-notebook:807999a41207 \
start.sh jupyter notebook --NotebookApp.token=''
```

14
docs/using/running.md
View File

@@ -15,12 +15,12 @@ The following are some common patterns.
**Example 1:**
This command pulls the `jupyter/scipy-notebook` image tagged `6b49f3337709` from Docker Hub if it is not already present on the local host.
This command pulls the `jupyter/scipy-notebook` image tagged `807999a41207` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888.
The server logs appear in the terminal and include a URL to the notebook server.
```bash
docker run -it -p 8888:8888 jupyter/scipy-notebook:6b49f3337709
docker run -it -p 8888:8888 jupyter/scipy-notebook:807999a41207
# Entered start.sh with args: jupyter lab
@@ -39,7 +39,7 @@ Pressing `Ctrl-C` twice shuts down the notebook server but leaves the container
# list containers
docker ps -a
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
# 221331c047c4 jupyter/scipy-notebook:6b49f3337709 "tini -g -- start-no…" 11 seconds ago Exited (0) 8 seconds ago cranky_benz
# 221331c047c4 jupyter/scipy-notebook:807999a41207 "tini -g -- start-no…" 11 seconds ago Exited (0) 8 seconds ago cranky_benz
# start the stopped container
docker start -a 221331c047c4
@@ -53,12 +53,12 @@ docker rm 221331c047c4
**Example 2:**
This command pulls the `jupyter/r-notebook` image tagged `6b49f3337709` from Docker Hub if it is not already present on the local host.
This command pulls the `jupyter/r-notebook` image tagged `807999a41207` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Notebook server and exposes the server on host port 10000.
The server logs appear in the terminal and include a URL to the notebook server, but with the internal container port (8888) instead of the correct host port (10000).
```bash
docker run -it --rm -p 10000:8888 -v "${PWD}":/home/jovyan/work jupyter/r-notebook:6b49f3337709
docker run -it --rm -p 10000:8888 -v "${PWD}":/home/jovyan/work jupyter/r-notebook:807999a41207
```
Pressing `Ctrl-C` twice shuts down the notebook server and immediately destroys the Docker container.
@@ -130,7 +130,7 @@ subuidSize=$(( $(podman info --format "{{ range .Host.IDMappings.UIDMap }}+{{.Si
subgidSize=$(( $(podman info --format "{{ range .Host.IDMappings.GIDMap }}+{{.Size }}{{end }}" ) - 1 ))
```
This command pulls the `docker.io/jupyter/r-notebook` image tagged `6b49f3337709` from Docker Hub if it is not already present on the local host.
This command pulls the `docker.io/jupyter/r-notebook` image tagged `807999a41207` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Server and exposes the server on host port 10000.
The server logs appear in the terminal and include a URL to the notebook server, but with the internal container port (8888) instead of the correct host port (10000).
@@ -139,7 +139,7 @@ podman run -it --rm -p 10000:8888 \
-v "${PWD}":/home/jovyan/work --user $uid:$gid \
--uidmap $uid:0:1 --uidmap 0:1:$uid --uidmap $(($uid+1)):$(($uid+1)):$(($subuidSize-$uid)) \
--gidmap $gid:0:1 --gidmap 0:1:$gid --gidmap $(($gid+1)):$(($gid+1)):$(($subgidSize-$gid)) \
docker.io/jupyter/r-notebook:6b49f3337709
docker.io/jupyter/r-notebook:807999a41207
```
```{warning}

32
docs/using/selecting.md
View File

@@ -18,8 +18,8 @@ The following sections describe these images, including their contents, relation
### jupyter/base-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/base-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/base-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/base-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/base-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/base-notebook/tags/)
`jupyter/base-notebook` is a small image supporting the [options common across all core stacks](common.md).
@@ -40,8 +40,8 @@ It is the basis for all other stacks and contains:
### jupyter/minimal-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/minimal-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/minimal-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/minimal-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/minimal-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/minimal-notebook/tags/)
`jupyter/minimal-notebook` adds command-line tools useful when working in Jupyter applications.
@@ -56,8 +56,8 @@ It contains:
### jupyter/r-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/r-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/r-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/r-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/r-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/r-notebook/tags/)
`jupyter/r-notebook` includes popular packages from the R ecosystem listed below:
@@ -87,8 +87,8 @@ It contains:
### jupyter/scipy-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/scipy-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/scipy-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/scipy-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/scipy-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/scipy-notebook/tags/)
`jupyter/scipy-notebook` includes popular packages from the scientific Python ecosystem.
@@ -129,8 +129,8 @@ It contains:
### jupyter/tensorflow-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/tensorflow-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/tensorflow-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/tensorflow-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/tensorflow-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/tensorflow-notebook/tags/)
`jupyter/tensorflow-notebook` includes popular Python deep learning libraries.
@@ -140,8 +140,8 @@ It contains:
### jupyter/datascience-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/datascience-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/datascience-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/datascience-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/datascience-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/datascience-notebook/tags/)
`jupyter/datascience-notebook` includes libraries for data analysis from the Julia, Python, and R
@@ -159,8 +159,8 @@ communities.
### jupyter/pyspark-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/pyspark-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/pyspark-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/pyspark-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/pyspark-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/pyspark-notebook/tags/)
`jupyter/pyspark-notebook` includes Python support for Apache Spark.
@@ -171,8 +171,8 @@ communities.
### jupyter/all-spark-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/all-spark-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/all-spark-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/all-spark-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/all-spark-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/all-spark-notebook/tags/)
`jupyter/all-spark-notebook` includes Python and R support for Apache Spark.

38
docs/using/specifics.md
View File

@@ -41,37 +41,43 @@ ipython profile create
You can build a `pyspark-notebook` image (and also the downstream `all-spark-notebook` image) with a different version of Spark by overriding the default value of the following arguments at build time.
- Spark distribution is defined by the combination of the Spark and the Hadoop version and verified by the package checksum,
- Spark distribution is defined by the combination of Spark, Hadoop and Scala versions and verified by the package checksum,
see [Download Apache Spark](https://spark.apache.org/downloads.html) and the [archive repo](https://archive.apache.org/dist/spark/) for more information.
- `spark_version`: The Spark version to install (`3.0.0`).
- `hadoop_version`: The Hadoop version (`3.2`).
- `spark_checksum`: The package checksum (`BFE4540...`).
- Spark can run with different OpenJDK versions.
- `openjdk_version`: The version of (JRE headless) the OpenJDK distribution (`11`), see [Ubuntu packages](https://packages.ubuntu.com/search?keywords=openjdk).
For example here is how to build a `pyspark-notebook` image with Spark `2.4.7`, Hadoop `2.7` and OpenJDK `8`.
- `spark_version`: The Spark version to install (`3.3.0`).
- `hadoop_version`: The Hadoop version (`3.2`).
- `scala_version`: The Scala version (`2.13`).
- `spark_checksum`: The package checksum (`BFE4540...`).
- `openjdk_version`: The version of the OpenJDK (JRE headless) distribution (`17`).
- This version needs to match the version supported by the Spark distribution used above.
- See [Spark Overview](https://spark.apache.org/docs/latest/#downloading) and [Ubuntu packages](https://packages.ubuntu.com/search?keywords=openjdk).
- Starting with _Spark >= 3.2_ the distribution file contains Scala version, hence building older Spark will not work.
- Building older version requires modification to the Dockerfile or using it's older version of the Dockerfile.
For example here is how to build a `pyspark-notebook` image with Spark `3.2.0`, Hadoop `3.2` and OpenJDK `11`.
```bash
# From the root of the project
# Build the image with different arguments
docker build --rm --force-rm \
-t jupyter/pyspark-notebook:spark-2.4.7 ./pyspark-notebook \
--build-arg spark_version=2.4.7 \
--build-arg hadoop_version=2.7 \
--build-arg spark_checksum=0F5455672045F6110B030CE343C049855B7BA86C0ECB5E39A075FF9D093C7F648DA55DED12E72FFE65D84C32DCD5418A6D764F2D6295A3F894A4286CC80EF478 \
--build-arg openjdk_version=8
-t jupyter/pyspark-notebook:spark-3.2.0 ./pyspark-notebook \
--build-arg spark_version=3.2.0 \
--build-arg hadoop_version=3.2 \
--build-arg spark_checksum=707DDE035926A50B75E53FCA72CADA519F3239B14A96546911CB4916A58DCF69A1D2BFDD2C7DD5899324DBD82B6EEAB9797A7B4ABF86736FFCA4C26D0E0BF0EE \
--build-arg openjdk_version=11
# Check the newly built image
docker run -it --rm jupyter/pyspark-notebook:spark-2.4.7 pyspark --version
docker run -it --rm jupyter/pyspark-notebook:spark-3.2.0 pyspark --version
# Welcome to
# ____ __
# / __/__ ___ _____/ /__
# _\ \/ _ \/ _ `/ __/ '_/
# /___/ .__/\_,_/_/ /_/\_\ version 2.4.7
# /___/ .__/\_,_/_/ /_/\_\ version 3.2.0
# /_/
#
# Using Scala version 2.11.12, OpenJDK 64-Bit Server VM, 1.8.0_275
# Using Scala version 2.13.5, OpenJDK 64-Bit Server VM, 11.0.15
```
### Usage Examples