hazza/docker-stacks
1
0
Fork 0
mirror of https://github.com/jupyter/docker-stacks.git synced 2025-10-10 03:23:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
51ab78f4a99b46dd91a72f727235f98328b8a5aa
docker-stacks/docs/using.md
Peter Parente 51ab78f4a9 Finish single container examples
2018-03-25 17:51:20 -04:00

18 KiB
Raw Blame History

Users Guide

Using one of the Jupyter Docker Stacks requires two choices:

  1. Which Docker image you wish to use
  2. How you wish to start Docker containers from that image

This section provides details about the available images and runtimes.

Selecting an Image

Core Stacks

The Jupyter team maintains a set of Docker image definitions in the https://github.com/jupyter/docker-stacks GitHub repository. The following sections describe these images including their contents, relationships, and versioning strategy.

jupyter/base-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/base-notebook is a small image supporting the options common across all core stacks. It is the basis for all other stacks.

  • Minimally-functional Jupyter Notebook server (e.g., no pandoc for saving notebooks as PDFs)
  • Miniconda Python 3.x
  • No preinstalled scientific computing packages
  • Unprivileged user jovyan (uid=1000, configurable, see options) in group users (gid=100) with ownership over the /home/jovyan and /opt/conda paths
  • tini as the container entrypoint and a start-notebook.sh script as the default command
  • A start-singleuser.sh script useful for launching containers in JupyterHub
  • A start.sh script useful for running alternative commands in the container (e.g. ipython, jupyter kernelgateway, jupyter lab)
  • Options for a self-signed HTTPS certificate and passwordless sudo

jupyter/minimal-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/minimal-notebook adds command line tools useful when working in Jupyter applications.

jupyter/r-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/r-notebook includes popular packages from the R ecosystem.

jupyter/scipy-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/scipy-notebook includes popular packages from the scientific Python ecosystem.

jupyter/tensorflow-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/tensorflow-notebook includes popular Python deep learning libraries.

  • Everything in jupyter/scipy-notebook and its ancestor images
  • tensorflow and keras machine learning libraries

jupyter/datascience-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/datascience-notebook includes libraries for data analysis from the Julia, Python, and R communities.

  • Everything in the jupyter/scipy-notebook and jupyter/r-notebook images, and their ancestor images
  • The Julia compiler and base environment
  • IJulia to support Julia code in Jupyter notebooks
  • HDF5, Gadfly, and RDatasets packages

jupyter/pyspark-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/pyspark-notebook includes Python support for Apache Spark, optionally on Mesos.

  • Everything in jupyter/scipy-notebook and its ancestor images
  • Apache Spark with Hadoop binaries
  • Mesos client libraries

jupyter/all-spark-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/all-spark-notebook includes Python, R, and Scala support for Apache Spark, optionally on Mesos.

Image Relationships

The following diagram depicts the build dependency tree of the core images. (i.e., the FROM statements in their Dockerfiles). Any given image inherits the complete content of all ancestor images pointing to it.

Image inheritance diagram

Builds

Pull requests to the jupyter/docker-stacks repository trigger builds of all images on Travis CI. These images are for testing purposes only and are not saved for use. When pull requests merge to master, all images rebuild on Docker Cloud and become available to docker pull from Docker Hub.

Versioning

The latest tag in each Docker Hub repository tracks the master branch HEAD reference on GitHub. latest is a moving target, by definition, and will have backward-incompatible changes regularly.

Every image on Docker Hub also receives a 12-character tag which corresponds with the git commit SHA that triggered the image build. You can inspect the state of the jupyter/docker-stacks repository for that commit to review the definition of the image (e.g., images with tag 7c45ec67c8 were built from https://github.com/jupyter/docker-stacks/tree/7c45ec67c8e7).

You must refer to git-SHA image tags when stability and reproducibility are important in your work. (e.g. FROM jupyter/scipy-notebook:7c45ec67c8e7, docker run -it --rm jupyter/scipy-notebook:7c45ec67c8e7). You should only use latest when a one-off container instance is acceptable (e.g., you want to briefly try a new library in a notebook).

Community Stacks

The core stacks are just a tiny sample of what's possible when combining Jupyter with other technologies. We encourage members of the Jupyter community to create their own stacks based on the core images and link them below.

Nothing here yet! You can be the first!

See the contributing guide for information about how to create your own Jupyter Docker Stack.

Running a Container

Using the Docker CLI

You can launch a local Docker container from the Jupyter Docker Stacks using the Docker command line interface. There are numerous ways to configure containers using the CLI. The following are a couple common patterns.

Example 1 This command pulls the jupyter/scipy-notebook image tagged 2c80cf3537ca from Docker Hub if it is not already present on the local host. It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888. The server logs appear in the terminal and include a URL to the notebook server.

docker run -p 8888:8888 jupyter/scipy-notebook:2c80cf3537ca

Executing the command: jupyter notebook
[I 15:33:00.567 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
[W 15:33:01.084 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 15:33:01.150 NotebookApp] JupyterLab alpha preview extension loaded from /opt/conda/lib/python3.6/site-packages/jupyterlab
[I 15:33:01.150 NotebookApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 15:33:01.155 NotebookApp] Serving notebooks from local directory: /home/jovyan
[I 15:33:01.156 NotebookApp] 0 active kernels
[I 15:33:01.156 NotebookApp] The Jupyter Notebook is running at:
[I 15:33:01.157 NotebookApp] http://[all ip addresses on your system]:8888/?token=112bb073331f1460b73768c76dffb2f87ac1d4ca7870d46a
[I 15:33:01.157 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 15:33:01.160 NotebookApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=112bb073331f1460b73768c76dffb2f87ac1d4ca7870d46a

Pressing Ctrl-C shuts down the notebook server but leaves the container intact on disk for later restart or permanent deletion using commands like the following:

# list containers
docker ps -a
CONTAINER ID        IMAGE                   COMMAND                  CREATED    STATUS                      PORTS               NAMES
d67fe77f1a84        jupyter/base-notebook   "tini -- start-noteb…"   44 seconds ago    Exited (0) 39 seconds ago                       cocky_mirzakhani

# start the stopped container
docker start -a d67fe77f1a84
Executing the command: jupyter notebook
[W 16:45:02.020 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
...

# remove the stopped container
docker rm d67fe77f1a84
d67fe77f1a84

Example 2 This command pulls the jupyter/r-notebook image tagged e5c5a7d3e52d from Docker Hub if it is not already present on the local host. It then starts a container running a Jupyter Notebook server and exposes the server on host port 10000. The server logs appear in the terminal and include a URL to the notebook server, but with the internal container port (8888) instead of the the correct host port (10000).

docker run --rm -p 10000:8888 -v "$PWD":/home/jovyan/work jupyter/r-notebook:e5c5a7d3e52d

Executing the command: jupyter notebook
[I 19:31:09.573 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
[W 19:31:11.930 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 19:31:12.085 NotebookApp] JupyterLab alpha preview extension loaded from /opt/conda/lib/python3.6/site-packages/jupyterlab
[I 19:31:12.086 NotebookApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 19:31:12.117 NotebookApp] Serving notebooks from local directory: /home/jovyan
[I 19:31:12.117 NotebookApp] 0 active kernels
[I 19:31:12.118 NotebookApp] The Jupyter Notebook is running at:
[I 19:31:12.119 NotebookApp] http://[all ip addresses on your system]:8888/?token=3b8dce890cb65570fb0d9c4a41ae067f7604873bd604f5ac
[I 19:31:12.120 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 19:31:12.122 NotebookApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=3b8dce890cb65570fb0d9c4a41ae067f7604873bd604f5ac

Pressing Ctrl-C shuts down the notebook server and immediately destroys the Docker container. Files written to ~/work in the container remain touched. Any other changes made in the container are lost.

Example 3 This command pulls the jupyter/all-spark-notebook image currently tagged latest from Docker Hub if an image tagged latest is not already present on the local host. It then starts a container named notebook running a JupyterLab server and exposes the server on a randomly selected port.

docker run -d -P --name notebook jupyter/all-spark-notebook

The assigned port and notebook server token are visible using other Docker commands.

# get the random host port assigned to the container port 8888
docker port notebook 8888
0.0.0.0:32769

# get the notebook token from the logs
docker logs --tail 3 notebook
    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=15914ca95f495075c0aa7d0e060f1a78b6d94f70ea373b00

Together, the URL to visit on the host machine to access the server in this case is http://localhost:32769?token=15914ca95f495075c0aa7d0e060f1a78b6d94f70ea373b00.

The container runs in the background until stopped and/or removed by additional Docker commands.

# stop the container
docker stop notebook
notebook

# remove the container permanently
docker rm notebook
notebook

Using Binder

Binder is a service that allows you to create and share custom computing environments for projects in version control. You can use any of the Jupyter Docker Stack images as a basis for a Binder-compatible Dockerfile. See the docker-stacks example and Using a Dockerfile sections in the Binder documentation for instructions.

Using JupyterHub

You can configure JupyterHub to launcher Docker containers from the Jupyter Docker Stacks images. If you've been following the Zero to JupyterHub with Kubernetes guide, see the Use an existing Docker image section for details. If you have a custom JupyterHub deployment, see the Picking or building a Docker image instructions for the dockerspawner instead.

Using Other Tools and Services

You can use the Jupyter Docker Stacks with any Docker-compatible technology (e.g., Docker Compose, docker-py, your favorite cloud container service). See the documentation of the tool, library, or service for details about how to reference, configure, and launch containers from these images.