diff --git a/Makefile b/Makefile index 973cc179..94b413a2 100644 --- a/Makefile +++ b/Makefile @@ -13,9 +13,7 @@ ALL_STACKS:=minimal-kernel \ pyspark-notebook \ all-spark-notebook -ALL_SINGLEUSERS:=$(shell echo $(ALL_STACKS) | sed "s/ /\n/g" | grep notebook | sed s/notebook/singleuser/g) - -ALL_IMAGES:=$(ALL_STACKS) $(ALL_SINGLEUSERS) +ALL_IMAGES:=$(ALL_STACKS) GIT_MASTER_HEAD_SHA:=$(shell git rev-parse --short=12 --verify HEAD) @@ -30,9 +28,6 @@ help: build/%: DARGS?= -build/%-singleuser: build/%-notebook - ./internal/build-singleuser $(OWNER)/$*-notebook $(OWNER)/$*-singleuser - build/%: docker build $(DARGS) --rm --force-rm -t $(OWNER)/$(notdir $@):latest ./$(notdir $@) diff --git a/README.md b/README.md index f7526faf..ed2d2fc6 100644 --- a/README.md +++ b/README.md @@ -25,15 +25,13 @@ If this is your first time using Docker or any of the Jupyter projects, do the f Here's a diagram of the `FROM` relationships between all of the images defined in this project: -[![Image inheritance diagram](internal/inherit-diagram.png)](http://interactive.blockdiag.com/?compression=deflate&src=eJyNjrEOgkAQRHu-4kLt9UaCkc7e0hizwELWW_bI3VGg8d8FCxMR0XLfzLxszrYwJUGtbpFS1hFKgEBWVKpa64IDCsmQlFhBx-FcWQmerjjE65Fn6siQI6dxiTmBbC7oPeHusM_iUxI9G3qrGhJqgLXYgLm1ZvUiBp0gj6ppZ9z5gtp-gWhPUjN2Ht2o-KyXEGCgKAX-5BPZ3D_uy_XHtO19C84ssolmbgLM-h_6pro_AAFCq-Y) +[![Image inheritance diagram](internal/inherit-diagram.png)](http://interactive.blockdiag.com/?compression=deflate&src=eJyFzDELwjAQhuG9vyJ0trtYKnZzdxSRS3OVM9dcSOJQxf9u41BoEVzf77nTLJ01BDf1KpSSQOgSJBKnGuUlpACU6mkx2MOD07UXlyI9cZq3ubfqzKCRm9KgJnC7O8ZIeDgd2_JSF19R7dVAjgbgyklCLWI3c7EYHHJ-tTb5Lnbkx7lktSzZGEgwVXQdLuSvf-Gv8GP0EOzCrVt2wFyt5fsDQc9zBA) ## Stacks, Tags, Versioning, and Progress Starting with [git commit SHA 9bd33dcc8688](https://github.com/jupyter/docker-stacks/tree/9bd33dcc8688): * Every folder here on GitHub has an equivalent `jupyter/` on Docker Hub. -* Each `*-notebook` stack also has a corresponding `*-singleuser` stack, - for use with [JupyterHub](https://jupyterhub.readthedocs.org). * The `latest` tag in each Docker Hub repository tracks the `master` branch `HEAD` reference on GitHub. * Any 12-character image tag on Docker Hub refers to a git commit SHA here on GitHub. See the [Docker build history wiki page](https://github.com/jupyter/docker-stacks/wiki/Docker-build-history) for a table of build details. * Stack contents (e.g., new library versions) will be updated upon request via PRs against this project. @@ -42,7 +40,8 @@ Starting with [git commit SHA 9bd33dcc8688](https://github.com/jupyter/docker-st ## Other Tips -* `tini -- start-notebook.sh` is the default Docker entrypoint-plus-command in every notebook stack. If you plan to modify it any way, be sure to check the *Notebook Options* section of your stack's README to understand the consequences. +* `tini -- start-notebook.sh` is the default Docker entrypoint-plus-command in every notebook stack. If you plan to modify it in any way, be sure to check the *Notebook Options* section of your stack's README to understand the consequences. +* Every notebook stack is compatible with [JupyterHub](https://jupyterhub.readthedocs.org) 0.5. When running with JupyterHub, you must override the Docker run command to point to the [start-singleuser.sh](minimal-notebook/start-singleuser.sh) script, which starts a single-user instance of the Notebook server. See each stack's README for instructions on running with JupyterHub. * Check the [Docker recipes wiki page](https://github.com/jupyter/docker-stacks/wiki/Docker-Recipes) attached to this project for information about extending and deploying the Docker images defined here. Add to the wiki if you have relevant information. ## Maintainer Workflow @@ -76,4 +75,4 @@ This will take time as the entire set of stacks will rebuild. 1. Create a new repo in the `jupyter` org on Docker Hub named after the stack folder in the git repo. 2. Grant the `stacks` team permission to write to the repo. -3. Copy/paste the short and long descriptions from one of the other docker-stacks repos on Docker Hub. Modify the appropriate values. \ No newline at end of file +3. Copy/paste the short and long descriptions from one of the other docker-stacks repos on Docker Hub. Modify the appropriate values. diff --git a/all-spark-notebook/README.md b/all-spark-notebook/README.md index 431efdfe..ef7e030e 100644 --- a/all-spark-notebook/README.md +++ b/all-spark-notebook/README.md @@ -14,6 +14,7 @@ * Mesos client 0.22 binary that can communicate with a Mesos master * Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda` * [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../minimal-notebook/start-notebook.sh) as the default command +* A [start-singleuser.sh](../minimal-notebook/start-singleuser.sh) script for use as an alternate command that runs a single-user instance of the Notebook server, as required by [JupyterHub](#JupyterHub) * Options for HTTPS, password auth, and passwordless `sudo` @@ -253,9 +254,14 @@ The commands `ipython`, `python`, `pip`, `easy_install`, and `conda` (among othe ## JupyterHub -To use this stack with [JupyterHub](https://jupyterhub.readthedocs.org) and [DockerSpawner](https://github.com/jupyter/dockerspawner), -set +[JupyterHub](https://jupyterhub.readthedocs.org) requires a single-user instance of the Jupyter Notebook server per user. To use this stack with JupyterHub and [DockerSpawner](https://github.com/jupyter/dockerspawner), you must specify the container image name and override the default container run command in your `jupyterhub_config.py`: ```python -c.DockerSpawner.container_image = 'jupyter/all-spark-singleuser' +# Spawn user containers from this image +c.DockerSpawner.container_image = 'jupyter/all-spark-notebook' + +# Have the Spawner override the Docker run command +c.DockerSpawner.extra_create_kwargs.update({ + 'command': '/usr/local/bin/start-singleuser.sh' +}) ``` diff --git a/datascience-notebook/README.md b/datascience-notebook/README.md index beeb2c14..ae067669 100644 --- a/datascience-notebook/README.md +++ b/datascience-notebook/README.md @@ -12,6 +12,7 @@ * Julia v0.3.x with Gadfly, RDatasets and HDF5 pre-installed * Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda` * [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../minimal-notebook/start-notebook.sh) as the default command +* A [start-singleuser.sh](../minimal-notebook/start-singleuser.sh) script for use as an alternate command that runs a single-user instance of the Notebook server, as required by [JupyterHub](#JupyterHub) * Options for HTTPS, password auth, and passwordless `sudo` ## Basic Use @@ -62,9 +63,14 @@ The commands `ipython`, `python`, `pip`, `easy_install`, and `conda` (among othe ## JupyterHub -To use this stack with [JupyterHub](https://jupyterhub.readthedocs.org) and [DockerSpawner](https://github.com/jupyter/dockerspawner), -set +[JupyterHub](https://jupyterhub.readthedocs.org) requires a single-user instance of the Jupyter Notebook server per user. To use this stack with JupyterHub and [DockerSpawner](https://github.com/jupyter/dockerspawner), you must specify the container image name and override the default container run command in your `jupyterhub_config.py`: ```python -c.DockerSpawner.container_image = 'jupyter/datascience-singleuser' +# Spawn user containers from this image +c.DockerSpawner.container_image = 'jupyter/datascience-notebook' + +# Have the Spawner override the Docker run command +c.DockerSpawner.extra_create_kwargs.update({ + 'command': '/usr/local/bin/start-singleuser.sh' +}) ``` diff --git a/internal/build-singleuser b/internal/build-singleuser deleted file mode 100755 index c1db4878..00000000 --- a/internal/build-singleuser +++ /dev/null @@ -1,30 +0,0 @@ -#!/usr/bin/env bash -# Build a singleuser image for JupyterHub from a given stack -# Usage: -# ./build-singleuser jupyter/scipy-notebook my-singleuser-image - -set -e - -V="0.2.0" -# get dockerspawner -if [[ ! -d dockerspawner ]]; then - tgz="dockerspawner.tar.gz" - wget -N https://github.com/jupyter/dockerspawner/archive/$V.tar.gz -O $tgz - echo "8af14608ad30df4dfcced7c7be508d336d485d9c31af965dc84c5e882e19dfe4 *$tgz" | shasum -a 256 -c - || (echo "shasum mismatch $(shasum -a 256 $tgz)" && exit -1) - tar -xzf $tgz - mv dockerspawner-$V dockerspawner -fi - -base="$1" -name="$2" -if [[ -z "$base" || -z "$name" ]]; then - echo "Usage: build-singleuser [base image] [destination image]" >&2 - exit 1 -fi -echo "building $name from $base" - -singleuser=dockerspawner/singleuser -# copy single-user stack -sed -i "s@^FROM.*\$@FROM $base@" "$singleuser/Dockerfile" -docker build -t "$name" "$singleuser" -echo "built $name" diff --git a/internal/inherit-diagram.png b/internal/inherit-diagram.png index b74a2cf9..dd29befc 100644 Binary files a/internal/inherit-diagram.png and b/internal/inherit-diagram.png differ diff --git a/minimal-notebook/Dockerfile b/minimal-notebook/Dockerfile index 6dc35b24..16ee4555 100644 --- a/minimal-notebook/Dockerfile +++ b/minimal-notebook/Dockerfile @@ -82,6 +82,9 @@ RUN conda install --quiet --yes \ terminado \ && conda clean -tipsy +# Install JupyterHub to get the jupyterhub-singleuser startup script +RUN pip install 'jupyterhub==0.5' + USER root # Configure container startup as root @@ -91,7 +94,10 @@ ENTRYPOINT ["tini", "--"] CMD ["start-notebook.sh"] # Add local files as late as possible to avoid cache busting +# Start notebook server COPY start-notebook.sh /usr/local/bin/ +# Start single-user notebook server for use with JupyterHub +COPY start-singleuser.sh /usr/local/bin/ COPY jupyter_notebook_config.py /home/$NB_USER/.jupyter/ RUN chown -R $NB_USER:users /home/$NB_USER/.jupyter diff --git a/minimal-notebook/README.md b/minimal-notebook/README.md index 9bc34de9..bf350228 100644 --- a/minimal-notebook/README.md +++ b/minimal-notebook/README.md @@ -9,6 +9,7 @@ * No preinstalled scientific computing packages * Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda` * [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](./start-notebook.sh) as the default command +* A [start-singleuser.sh](../minimal-notebook/start-singleuser.sh) script for use as an alternate command that runs a single-user instance of the Notebook server, as required by [JupyterHub](#JupyterHub) * Options for HTTPS, password auth, and passwordless `sudo` ## Basic Use @@ -47,9 +48,14 @@ The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/en ## JupyterHub -To use this stack with [JupyterHub](https://jupyterhub.readthedocs.org) and [DockerSpawner](https://github.com/jupyter/dockerspawner), -set +[JupyterHub](https://jupyterhub.readthedocs.org) requires a single-user instance of the Jupyter Notebook server per user. To use this stack with JupyterHub and [DockerSpawner](https://github.com/jupyter/dockerspawner), you must specify the container image name and override the default container run command in your `jupyterhub_config.py`: ```python -c.DockerSpawner.container_image = 'jupyter/minimal-singleuser' +# Spawn user containers from this image +c.DockerSpawner.container_image = 'jupyter/minimal-notebook' + +# Have the Spawner override the Docker run command +c.DockerSpawner.extra_create_kwargs.update({ + 'command': '/usr/local/bin/start-singleuser.sh' +}) ``` diff --git a/minimal-notebook/start-singleuser.sh b/minimal-notebook/start-singleuser.sh new file mode 100755 index 00000000..3658464e --- /dev/null +++ b/minimal-notebook/start-singleuser.sh @@ -0,0 +1,19 @@ +#!/bin/bash +set -e + +notebook_arg="" +if [ -n "${NOTEBOOK_DIR:+x}" ] +then + notebook_arg="--notebook-dir=${NOTEBOOK_DIR}" +fi + +exec jupyterhub-singleuser \ + --port=8888 \ + --ip=0.0.0.0 \ + --user=$JPY_USER \ + --cookie-name=$JPY_COOKIE_NAME \ + --base-url=$JPY_BASE_URL \ + --hub-prefix=$JPY_HUB_PREFIX \ + --hub-api-url=$JPY_HUB_API_URL \ + ${notebook_arg} \ + $@ diff --git a/pyspark-notebook/README.md b/pyspark-notebook/README.md index 834201de..6985a38d 100644 --- a/pyspark-notebook/README.md +++ b/pyspark-notebook/README.md @@ -11,6 +11,7 @@ * Mesos client 0.22 binary that can communicate with a Mesos master * Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda` * [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../minimal-notebook/start-notebook.sh) as the default command +* A [start-singleuser.sh](../minimal-notebook/start-singleuser.sh) script for use as an alternate command that runs a single-user instance of the Notebook server, as required by [JupyterHub](#JupyterHub) * Options for HTTPS, password auth, and passwordless `sudo` ## Basic Use @@ -136,9 +137,14 @@ The commands `ipython`, `python`, `pip`, `easy_install`, and `conda` (among othe ## JupyterHub -To use this stack with [JupyterHub](https://jupyterhub.readthedocs.org) and [DockerSpawner](https://github.com/jupyter/dockerspawner), -set +[JupyterHub](https://jupyterhub.readthedocs.org) requires a single-user instance of the Jupyter Notebook server per user. To use this stack with JupyterHub and [DockerSpawner](https://github.com/jupyter/dockerspawner), you must specify the container image name and override the default container run command in your `jupyterhub_config.py`: ```python -c.DockerSpawner.container_image = 'jupyter/pyspark-singleuser' +# Spawn user containers from this image +c.DockerSpawner.container_image = 'jupyter/pyspark-notebook' + +# Have the Spawner override the Docker run command +c.DockerSpawner.extra_create_kwargs.update({ + 'command': '/usr/local/bin/start-singleuser.sh' +}) ``` diff --git a/r-notebook/README.md b/r-notebook/README.md index 28850cd8..eaaf8278 100644 --- a/r-notebook/README.md +++ b/r-notebook/README.md @@ -9,6 +9,7 @@ * plyr, devtools, dplyr, ggplot2, tidyr, shiny, rmarkdown, forecast, stringr, rsqlite, reshape2, nycflights13, caret, rcurl, and randomforest pre-installed * Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda` * [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../minimal-notebook/start-notebook.sh) as the default command +* A [start-singleuser.sh](../minimal-notebook/start-singleuser.sh) script for use as an alternate command that runs a single-user instance of the Notebook server, as required by [JupyterHub](#JupyterHub) * Options for HTTPS, password auth, and passwordless `sudo` ## Basic Use @@ -43,9 +44,14 @@ You may customize the execution of the Docker container and the Notebook server ## JupyterHub -To use this stack with [JupyterHub](https://jupyterhub.readthedocs.org) and [DockerSpawner](https://github.com/jupyter/dockerspawner), -set +[JupyterHub](https://jupyterhub.readthedocs.org) requires a single-user instance of the Jupyter Notebook server per user. To use this stack with JupyterHub and [DockerSpawner](https://github.com/jupyter/dockerspawner), you must specify the container image name and override the default container run command in your `jupyterhub_config.py`: ```python -c.DockerSpawner.container_image = 'jupyter/r-singleuser' +# Spawn user containers from this image +c.DockerSpawner.container_image = 'jupyter/r-notebook' + +# Have the Spawner override the Docker run command +c.DockerSpawner.extra_create_kwargs.update({ + 'command': '/usr/local/bin/start-singleuser.sh' +}) ``` diff --git a/scipy-notebook/README.md b/scipy-notebook/README.md index 23b1b5c6..518489fd 100644 --- a/scipy-notebook/README.md +++ b/scipy-notebook/README.md @@ -9,6 +9,7 @@ * pandas, matplotlib, scipy, seaborn, scikit-learn, scikit-image, sympy, cython, patsy, statsmodel, cloudpickle, dill, numba, bokeh pre-installed * Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda` * [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../minimal-notebook/start-notebook.sh) as the default command +* A [start-singleuser.sh](../minimal-notebook/start-singleuser.sh) script for use as an alternate command that runs a single-user instance of the Notebook server, as required by [JupyterHub](#JupyterHub) * Options for HTTPS, password auth, and passwordless `sudo` ## Basic Use @@ -59,9 +60,14 @@ The commands `ipython`, `python`, `pip`, `easy_install`, and `conda` (among othe ## JupyterHub -To use this stack with [JupyterHub](https://jupyterhub.readthedocs.org) and [DockerSpawner](https://github.com/jupyter/dockerspawner), -set +[JupyterHub](https://jupyterhub.readthedocs.org) requires a single-user instance of the Jupyter Notebook server per user. To use this stack with JupyterHub and [DockerSpawner](https://github.com/jupyter/dockerspawner), you must specify the container image name and override the default container run command in your `jupyterhub_config.py`: ```python -c.DockerSpawner.container_image = 'jupyter/scipy-singleuser' +# Spawn user containers from this image +c.DockerSpawner.container_image = 'jupyter/scipy-notebook' + +# Have the Spawner override the Docker run command +c.DockerSpawner.extra_create_kwargs.update({ + 'command': '/usr/local/bin/start-singleuser.sh' +}) ```