hazza/docker-stacks
1
0
Fork 0
mirror of https://github.com/jupyter/docker-stacks.git synced 2025-10-07 18:14:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into asalikhov/ubuntu_jammy

Browse Source
This commit is contained in:
Ayaz Salikhov
2022-07-09 16:50:05 +04:00
parent 061b2b2d4c 18b10e7f73
commit 103da29c0a
30 changed files with 127 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/using/common.md
View File

@@ -133,7 +133,7 @@ or executables (`chmod +x`) to be run to the paths below:
- `/usr/local/bin/before-notebook.d/` - handled **after** all the standard options noted above are applied
and ran right before the notebook server launches
See the `run-hooks` function in the [`jupyter/base-notebook start.sh`](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/start.sh)
See the `run-hooks` function in the [`jupyter/base-notebook start.sh`](https://github.com/jupyter/docker-stacks/blob/main/base-notebook/start.sh)
script for execution details.
## SSL Certificates
@@ -166,10 +166,10 @@ The certificate file or PEM may contain one or more certificates (e.g., server,
For additional information about using SSL, see the following:
- The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples)
- The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/main/examples)
for information about how to use
[Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
- The [`jupyter_server_config.py`](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/jupyter_server_config.py)
- The [`jupyter_server_config.py`](https://github.com/jupyter/docker-stacks/blob/main/base-notebook/jupyter_server_config.py)
file for how this Docker image generates a self-signed certificate.
- The [Jupyter Server documentation](https://jupyter-server.readthedocs.io/en/latest/operators/public-server.html#securing-a-jupyter-server)
for best practices about securing a public notebook server in general.

14
docs/using/recipes.md
View File

@@ -29,7 +29,7 @@ Create a new Dockerfile like the one shown below.
```dockerfile
# Start from a core stack version
FROM jupyter/datascience-notebook:6b49f3337709
FROM jupyter/datascience-notebook:807999a41207
# Install in the default python3 environment
RUN pip install --quiet --no-cache-dir 'flake8==3.9.2' && \
fix-permissions "${CONDA_DIR}" && \
@@ -48,7 +48,7 @@ Next, create a new Dockerfile like the one shown below.
```dockerfile
# Start from a core stack version
FROM jupyter/datascience-notebook:6b49f3337709
FROM jupyter/datascience-notebook:807999a41207
# Install from requirements.txt file
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
RUN pip install --quiet --no-cache-dir --requirement /tmp/requirements.txt && \
@@ -60,7 +60,7 @@ For conda, the Dockerfile is similar:
```dockerfile
# Start from a core stack version
FROM jupyter/datascience-notebook:6b49f3337709
FROM jupyter/datascience-notebook:807999a41207
# Install from requirements.txt file
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
RUN mamba install --yes --file /tmp/requirements.txt && \
@@ -148,7 +148,7 @@ Ref: <https://github.com/jupyter/docker-stacks/issues/999>
## Let's Encrypt a Notebook server
See the README for a basic automation here
<https://github.com/jupyter/docker-stacks/tree/master/examples/make-deploy>
<https://github.com/jupyter/docker-stacks/tree/main/examples/make-deploy>
which includes steps for requesting and renewing a Let's Encrypt certificate.
Ref: <https://github.com/jupyter/docker-stacks/issues/78>
@@ -283,7 +283,7 @@ To use a specific version of JupyterHub, the version of `jupyterhub` in your ima
version in the Hub itself.
```dockerfile
FROM jupyter/base-notebook:6b49f3337709
FROM jupyter/base-notebook:807999a41207
RUN pip install --quiet --no-cache-dir jupyterhub==1.4.1 && \
fix-permissions "${CONDA_DIR}" && \
fix-permissions "/home/${NB_USER}"
@@ -474,7 +474,7 @@ For JupyterLab:
```bash
docker run -it --rm \
jupyter/base-notebook:6b49f3337709 \
jupyter/base-notebook:807999a41207 \
start.sh jupyter lab --LabApp.token=''
```
@@ -482,7 +482,7 @@ For jupyter classic:
```bash
docker run -it --rm \
jupyter/base-notebook:6b49f3337709 \
jupyter/base-notebook:807999a41207 \
start.sh jupyter notebook --NotebookApp.token=''
```

14
docs/using/running.md
View File

@@ -15,12 +15,12 @@ The following are some common patterns.
**Example 1:**
This command pulls the `jupyter/scipy-notebook` image tagged `6b49f3337709` from Docker Hub if it is not already present on the local host.
This command pulls the `jupyter/scipy-notebook` image tagged `807999a41207` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888.
The server logs appear in the terminal and include a URL to the notebook server.
```bash
docker run -it -p 8888:8888 jupyter/scipy-notebook:6b49f3337709
docker run -it -p 8888:8888 jupyter/scipy-notebook:807999a41207
# Entered start.sh with args: jupyter lab
@@ -39,7 +39,7 @@ Pressing `Ctrl-C` twice shuts down the notebook server but leaves the container
# list containers
docker ps -a
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
# 221331c047c4 jupyter/scipy-notebook:6b49f3337709 "tini -g -- start-no…" 11 seconds ago Exited (0) 8 seconds ago cranky_benz
# 221331c047c4 jupyter/scipy-notebook:807999a41207 "tini -g -- start-no…" 11 seconds ago Exited (0) 8 seconds ago cranky_benz
# start the stopped container
docker start -a 221331c047c4
@@ -53,12 +53,12 @@ docker rm 221331c047c4
**Example 2:**
This command pulls the `jupyter/r-notebook` image tagged `6b49f3337709` from Docker Hub if it is not already present on the local host.
This command pulls the `jupyter/r-notebook` image tagged `807999a41207` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Notebook server and exposes the server on host port 10000.
The server logs appear in the terminal and include a URL to the notebook server, but with the internal container port (8888) instead of the correct host port (10000).
```bash
docker run -it --rm -p 10000:8888 -v "${PWD}":/home/jovyan/work jupyter/r-notebook:6b49f3337709
docker run -it --rm -p 10000:8888 -v "${PWD}":/home/jovyan/work jupyter/r-notebook:807999a41207
```
Pressing `Ctrl-C` twice shuts down the notebook server and immediately destroys the Docker container.
@@ -130,7 +130,7 @@ subuidSize=$(( $(podman info --format "{{ range .Host.IDMappings.UIDMap }}+{{.Si
subgidSize=$(( $(podman info --format "{{ range .Host.IDMappings.GIDMap }}+{{.Size }}{{end }}" ) - 1 ))
```
This command pulls the `docker.io/jupyter/r-notebook` image tagged `6b49f3337709` from Docker Hub if it is not already present on the local host.
This command pulls the `docker.io/jupyter/r-notebook` image tagged `807999a41207` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Server and exposes the server on host port 10000.
The server logs appear in the terminal and include a URL to the notebook server, but with the internal container port (8888) instead of the correct host port (10000).
@@ -139,7 +139,7 @@ podman run -it --rm -p 10000:8888 \
-v "${PWD}":/home/jovyan/work --user $uid:$gid \
--uidmap $uid:0:1 --uidmap 0:1:$uid --uidmap $(($uid+1)):$(($uid+1)):$(($subuidSize-$uid)) \
--gidmap $gid:0:1 --gidmap 0:1:$gid --gidmap $(($gid+1)):$(($gid+1)):$(($subgidSize-$gid)) \
docker.io/jupyter/r-notebook:6b49f3337709
docker.io/jupyter/r-notebook:807999a41207
```
```{warning}

32
docs/using/selecting.md
View File

@@ -18,8 +18,8 @@ The following sections describe these images, including their contents, relation
### jupyter/base-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/base-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/base-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/base-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/base-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/base-notebook/tags/)
`jupyter/base-notebook` is a small image supporting the [options common across all core stacks](common.md).
@@ -40,8 +40,8 @@ It is the basis for all other stacks and contains:
### jupyter/minimal-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/minimal-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/minimal-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/minimal-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/minimal-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/minimal-notebook/tags/)
`jupyter/minimal-notebook` adds command-line tools useful when working in Jupyter applications.
@@ -56,8 +56,8 @@ It contains:
### jupyter/r-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/r-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/r-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/r-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/r-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/r-notebook/tags/)
`jupyter/r-notebook` includes popular packages from the R ecosystem listed below:
@@ -87,8 +87,8 @@ It contains:
### jupyter/scipy-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/scipy-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/scipy-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/scipy-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/scipy-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/scipy-notebook/tags/)
`jupyter/scipy-notebook` includes popular packages from the scientific Python ecosystem.
@@ -129,8 +129,8 @@ It contains:
### jupyter/tensorflow-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/tensorflow-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/tensorflow-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/tensorflow-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/tensorflow-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/tensorflow-notebook/tags/)
`jupyter/tensorflow-notebook` includes popular Python deep learning libraries.
@@ -140,8 +140,8 @@ It contains:
### jupyter/datascience-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/datascience-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/datascience-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/datascience-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/datascience-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/datascience-notebook/tags/)
`jupyter/datascience-notebook` includes libraries for data analysis from the Julia, Python, and R
@@ -159,8 +159,8 @@ communities.
### jupyter/pyspark-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/pyspark-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/pyspark-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/pyspark-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/pyspark-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/pyspark-notebook/tags/)
`jupyter/pyspark-notebook` includes Python support for Apache Spark.
@@ -171,8 +171,8 @@ communities.
### jupyter/all-spark-notebook
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/master/all-spark-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/all-spark-notebook/Dockerfile) |
[Source on GitHub](https://github.com/jupyter/docker-stacks/tree/main/all-spark-notebook) |
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/main/all-spark-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/all-spark-notebook/tags/)
`jupyter/all-spark-notebook` includes Python and R support for Apache Spark.

38
docs/using/specifics.md
View File

@@ -41,37 +41,43 @@ ipython profile create
You can build a `pyspark-notebook` image (and also the downstream `all-spark-notebook` image) with a different version of Spark by overriding the default value of the following arguments at build time.
- Spark distribution is defined by the combination of the Spark and the Hadoop version and verified by the package checksum,
- Spark distribution is defined by the combination of Spark, Hadoop and Scala versions and verified by the package checksum,
see [Download Apache Spark](https://spark.apache.org/downloads.html) and the [archive repo](https://archive.apache.org/dist/spark/) for more information.
- `spark_version`: The Spark version to install (`3.0.0`).
- `hadoop_version`: The Hadoop version (`3.2`).
- `spark_checksum`: The package checksum (`BFE4540...`).
- Spark can run with different OpenJDK versions.
- `openjdk_version`: The version of (JRE headless) the OpenJDK distribution (`11`), see [Ubuntu packages](https://packages.ubuntu.com/search?keywords=openjdk).
For example here is how to build a `pyspark-notebook` image with Spark `2.4.7`, Hadoop `2.7` and OpenJDK `8`.
- `spark_version`: The Spark version to install (`3.3.0`).
- `hadoop_version`: The Hadoop version (`3.2`).
- `scala_version`: The Scala version (`2.13`).
- `spark_checksum`: The package checksum (`BFE4540...`).
- `openjdk_version`: The version of the OpenJDK (JRE headless) distribution (`17`).
- This version needs to match the version supported by the Spark distribution used above.
- See [Spark Overview](https://spark.apache.org/docs/latest/#downloading) and [Ubuntu packages](https://packages.ubuntu.com/search?keywords=openjdk).
- Starting with _Spark >= 3.2_ the distribution file contains Scala version, hence building older Spark will not work.
- Building older version requires modification to the Dockerfile or using it's older version of the Dockerfile.
For example here is how to build a `pyspark-notebook` image with Spark `3.2.0`, Hadoop `3.2` and OpenJDK `11`.
```bash
# From the root of the project
# Build the image with different arguments
docker build --rm --force-rm \
-t jupyter/pyspark-notebook:spark-2.4.7 ./pyspark-notebook \
--build-arg spark_version=2.4.7 \
--build-arg hadoop_version=2.7 \
--build-arg spark_checksum=0F5455672045F6110B030CE343C049855B7BA86C0ECB5E39A075FF9D093C7F648DA55DED12E72FFE65D84C32DCD5418A6D764F2D6295A3F894A4286CC80EF478 \
--build-arg openjdk_version=8
-t jupyter/pyspark-notebook:spark-3.2.0 ./pyspark-notebook \
--build-arg spark_version=3.2.0 \
--build-arg hadoop_version=3.2 \
--build-arg spark_checksum=707DDE035926A50B75E53FCA72CADA519F3239B14A96546911CB4916A58DCF69A1D2BFDD2C7DD5899324DBD82B6EEAB9797A7B4ABF86736FFCA4C26D0E0BF0EE \
--build-arg openjdk_version=11
# Check the newly built image
docker run -it --rm jupyter/pyspark-notebook:spark-2.4.7 pyspark --version
docker run -it --rm jupyter/pyspark-notebook:spark-3.2.0 pyspark --version
# Welcome to
# ____ __
# / __/__ ___ _____/ /__
# _\ \/ _ \/ _ `/ __/ '_/
# /___/ .__/\_,_/_/ /_/\_\ version 2.4.7
# /___/ .__/\_,_/_/ /_/\_\ version 3.2.0
# /_/
#
# Using Scala version 2.11.12, OpenJDK 64-Bit Server VM, 1.8.0_275
# Using Scala version 2.13.5, OpenJDK 64-Bit Server VM, 11.0.15
```
### Usage Examples