Files
docker-stacks/docs/using/selecting.md
Yuvi Panda 39582f7099 Install Pluto.jl and jupyter-pluto-proxy (#1929)
* Install Pluto.jl and jupyter-pluto-proxy

[Pluto.jl](https://plutojl.org/) is an alternative reactive notebook
frontend focused specifically on Julia. I think shipping this
by default in the julia-enabled images helps serve the Julia
community better, particularly when used with JupyterHub.

For context, I am working with the Julia users of the
[Jupyter Meets the Earth](https://jupytearth.org/) project, and
trying to understand how to best serve their needs on a JupyterHub.
We currently maintain a massive image that 'has everything', but
I'm trying to instead work upstream wherever possible so everyone
working in these subfields can benefit. Meeting Julia users where
they are at seems a useful path forward here.

* Add note about Pluto.jl to selecting.md

* Default to replacing - with _ in package imports

* Add jupyter-pluto-proxy to package import mapping

* Add Pluto.jl to datascience-notebook image

* Add test for pluto proxy starting correctly

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test_packages.py

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2023-07-05 18:48:36 +04:00

19 KiB
Raw Blame History

Selecting an Image

Using one of the Jupyter Docker Stacks requires two choices:

  1. Which Docker image you wish to use
  2. How you wish to start Docker containers from that image

This section provides details about the first.

Core Stacks

The Jupyter team maintains a set of Docker image definitions in the https://github.com/jupyter/docker-stacks GitHub repository. The following sections describe these images, including their contents, relationships, and versioning strategy.

jupyter/docker-stacks-foundation

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/docker-stacks-foundation is a small image supporting a majority of options common across all core stacks. It is the basis for all other stacks on which Jupyter-related applications can be built (e.g., kernel-based containers, nbclient applications, etc.). As such, it does not contain application-level software like Jupyter Notebook server, Jupyter Lab or Jupyter Hub.

It contains:

  • Package managers
    • conda: "cross-platform, language-agnostic binary package manager".
    • mamba: "reimplementation of the conda package manager in C++". We use this package manager by default when installing packages.
  • Unprivileged user jovyan (uid=1000, configurable, see options in the common features section of this documentation) in group users (gid=100) with ownership over the /home/jovyan and /opt/conda paths
  • tini as the container entry point
  • A start.sh script as the default command - useful for running alternative commands in the container as applications are added (e.g. ipython, jupyter kernelgateway, jupyter lab)
  • Options for a passwordless sudo
  • No preinstalled scientific computing packages

jupyter/base-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/base-notebook adds base Jupyter server applications like Notebook, Jupyter Lab and Jupyter Hub and serves as the basis for all other stacks besides jupyter/docker-stacks-foundation.

It contains:

  • Everything in jupyter/docker-stacks-foundation
  • Minimally functional Jupyter Notebook server (e.g., no LaTeX support for saving notebooks as PDFs)
  • notebook, jupyterhub and jupyterlab packages
  • A start-notebook.sh script as the default command
  • A start-singleuser.sh script useful for launching containers in JupyterHub
  • Options for a self-signed HTTPS certificate

jupyter/minimal-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/minimal-notebook adds command-line tools useful when working in Jupyter applications.

It contains:

  • Everything in jupyter/base-notebook
  • Common useful utilities like git, nano (actually nano-tiny), tzdata, unzip and vi (actually vim-tiny),
  • TeX Live for notebook document conversion

jupyter/r-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/r-notebook includes popular packages from the R ecosystem listed below:

jupyter/julia-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/julia-notebook includes popular packages from the Julia ecosystem listed below:

jupyter/scipy-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/scipy-notebook includes popular packages from the scientific Python ecosystem.

jupyter/tensorflow-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/tensorflow-notebook includes popular Python deep learning libraries.

  • Everything in jupyter/scipy-notebook and its ancestor images
  • tensorflow machine learning library

jupyter/datascience-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/datascience-notebook includes libraries for data analysis from the Julia, Python, and R communities.

  • Everything in the jupyter/scipy-notebook and jupyter/r-notebook images and their ancestor images
  • rpy2 package
  • The Julia compiler and base environment
  • IJulia to support Julia code in Jupyter notebooks
  • Pluto.jl reactive Julia notebook interface, made accessible with jupyter-pluto-proxy
  • HDF5 package

jupyter/pyspark-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/pyspark-notebook includes Python support for Apache Spark.

  • Everything in jupyter/scipy-notebook and its ancestor images
  • Apache Spark with Hadoop binaries
  • pyarrow library

jupyter/all-spark-notebook

Source on GitHub | Dockerfile commit history | Docker Hub image tags

jupyter/all-spark-notebook includes Python and R support for Apache Spark.

  • Everything in jupyter/pyspark-notebook and its ancestor images
  • IRKernel to support R code in Jupyter notebooks
  • rcurl, sparklyr, ggplot2 packages

Image Relationships

The following diagram depicts the build dependency tree of the core images. (i.e., the FROM statements in their Dockerfiles). Any given image inherits the complete content of all ancestor images pointing to it.

Image inheritance
diagram

Builds

Every Monday and whenever a pull request is merged, images are rebuilt and pushed to the public container registry.

Versioning via image tags

Whenever a docker image is pushed to the container registry, it is tagged with:

  • a latest tag
  • a 12-character git commit SHA like 1ffe43816ba9
  • a date formatted like 2023-01-30
  • OS version like ubuntu-22.04
  • a set of software version tags like python-3.10.8 and lab-3.5.3
- Tags before `2022-07-05` were sometimes incorrect. Please, do not rely on them.
- Single-platform images have either `aarch64` or `x86_64` tag prefixes, for example, `jupyter/base-notebook:aarch64-python-3.10.5`

For stability and reproducibility, you should either reference a date formatted tag from a date before the current date (in UTC time) or a git commit SHA older than the latest git commit SHA in the default branch of the jupyter/docker-stacks GitHub repository.

Community Stacks

The core stacks are but a tiny sample of what's possible when combining Jupyter with other technologies. We encourage members of the Jupyter community to create their own stacks based on the core images and link them below. See the contributing guide for information about how to create your own Jupyter Docker Stack.

Flavor Binder Description
csharp bb More than 200 Jupyter Notebooks with example C# code
education bb nbgrader and RISE on top of the datascience-notebook image
ihaskell bb Based on IHaskell. Includes popular packages and example notebooks
java bb IJava kernel on top of the minimal-notebook image
sage bb sagemath kernel on top of the minimal-notebook image
cgspatial bb Major geospatial Python & R libraries on top of the datascience-notebook image
kotlin bb Kotlin kernel for Jupyter/IPython on top of the base-notebook image
transformers bb Transformers and NLP libraries such as Tensorflow, Keras, Jax and PyTorch
scraper bb Scraper tools (selenium, chromedriver, beatifulsoup4, requests) on minimal-notebook image
almond bb Scala kernel for Jupyter using Almond on top of the base-notebook image

GPU accelerated notebooks

Flavor Description
GPU-Jupyter Power of your NVIDIA GPU and GPU calculations using Tensorflow and Pytorch in collaborative notebooks. This is done by generating a Dockerfile that consists of the nvidia/cuda base image, the well-maintained docker-stacks that is integrated as a submodule and GPU-able libraries like Tensorflow, Keras and PyTorch on top of it.
PRP-GPU PRP (Pacific Research Platform) maintained registry for jupyter stack based on NVIDIA CUDA-enabled image. Added the PRP image with Pytorch and some other python packages and GUI Desktop notebook based on https://github.com/jupyterhub/jupyter-remote-desktop-proxy.
b-data GPU accelerated, multi-arch (linux/amd64, linux/arm64/v8) docker images for R, Python and Julia. Derived from nvidia/cuda devel-flavored images, including TensortRT and TensorRT plugin libraries. With code-server next to JupyterLab. Just Python no Conda/Mamba.