mirror of
https://github.com/jupyter/docker-stacks.git
synced 2025-10-08 18:44:06 +00:00
Make RTD the doc source of truth
This commit is contained in:
Peter Parente
committed by
Peter Parente
Peter Parente
parent
6fd049e1ae
commit
2b33e23bb7
137
README.md
137
README.md
@@ -1,134 +1,27 @@
|
||||
# docker-stacks
|
||||
# Jupyter Docker Stacks
|
||||
|
||||
[](https://travis-ci.org/jupyter/docker-stacks)
|
||||
[](https://gitter.im/jupyter/jupyter?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
|
||||
Jupyter Docker Stacks are a set of ready-to-run Docker images containing Jupyter applications and interactive computing tools.
|
||||
|
||||
Opinionated stacks of ready-to-run Jupyter applications in Docker.
|
||||
Quick Start
|
||||
-----------
|
||||
|
||||
## Quick Start
|
||||
The two examples below may help you get started if you [have Docker installed](https://docs.docker.com/install/) know [which Docker image](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html) you want to use, and want to launch a single Jupyter Notebook server in a container.
|
||||
|
||||
If you're familiar with Docker, have it configured, and know exactly what you'd like to run, one of these commands should get you up and running:
|
||||
The [User Guide on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/) describes additional uses and features in detail.
|
||||
|
||||
```
|
||||
# Run an ephemeral Jupyter Notebook server in a Docker container in the terminal foreground.
|
||||
# Note that any work saved in the container will be lost when it is destroyed with this config.
|
||||
# -ti: pseudo-TTY+STDIN open.
|
||||
# -rm: remove the container on exit.
|
||||
# -p: publish port to the host
|
||||
docker run -ti --rm -p 8888:8888 jupyter/<your desired stack>:<git-sha-tag>
|
||||
**Example 1:** This command pulls the `jupyter/scipy-notebook` image tagged `2c80cf3537ca` from Docker Hub if it is not already present on the local host. It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888. The server logs appear in the terminal and include a URL to the notebook server. The container remains intact for restart after notebook server exit.::
|
||||
|
||||
# Run a Jupyter Notebook server in a Docker container in the terminal foreground.
|
||||
# Any files written to ~/work in the container will be saved to the current working
|
||||
# directory on the host.
|
||||
docker run -ti --rm -p 8888:8888 -v "$PWD":/home/jovyan/work jupyter/<your desired stack>:<git-sha-tag>
|
||||
docker run -p 8888:8888 jupyter/scipy-notebook:2c80cf3537ca
|
||||
|
||||
# Run an ephemeral Jupyter Notebook server in a Docker container in the background.
|
||||
# Note that any work saved in the container will be lost when it is destroyed with this config.
|
||||
# -d: detach, run container in background.
|
||||
# -P: Publish all exposed ports to random ports
|
||||
docker run -d -P jupyter/<your desired stack>:<git-sha-tag>
|
||||
```
|
||||
**Example 2:** This command pulls the `jupyter/r-notebook` image tagged `e5c5a7d3e52d` from Docker Hub if it is not already present on the local host. It then starts an *ephemeral* container running a Jupyter Notebook server and exposes the server on host port 10000. The command mounts the current working directory on the host as `/home/jovyan/work` in the container. Docker destroys the container after notebook server exit, but any files written to `~/work` in the container remain intact on the host.::
|
||||
|
||||
## Getting Started
|
||||
docker run --rm -p 10000:8888 -v "$PWD":/home/jovyan/work jupyter/r-notebook:e5c5a7d3e52d
|
||||
|
||||
If this is your first time using Docker or any of the Jupyter projects, do the following to get started.
|
||||
Contributing
|
||||
------------
|
||||
|
||||
1. [Install Docker](https://docs.docker.com/installation/) on your host of choice.
|
||||
2. Open the README in one of the folders in this git repository.
|
||||
3. Follow the README for that stack.
|
||||
Please see the [Contributor Guide on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/) for information about how to contribute package updates, recipes features, tests, and community maintained stacks.
|
||||
|
||||
## Visual Overview
|
||||
Resources
|
||||
---------
|
||||
|
||||
Here's a diagram of the `FROM` relationships between all of the images defined in this project:
|
||||
|
||||
[](http://interactive.blockdiag.com/?compression=deflate&src=eJyFzTEPgjAQhuHdX9Gws5sQjGzujsaYKxzmQrlr2msMGv-71K0srO_3XGud9NNA8DSfgzESCFlBSdi0xkvQAKTNugw4QnL6GIU10hvX-Zh7Z24OLLq2SjaxpvP10lX35vCf6pOxELFmUbQiUz4oQhYzMc3gCrRt2cWe_FKosmSjyFHC6OS1AwdQWCtyj7sfh523_BI9hKlQ25YdOFdv5fcH0kiEMA)
|
||||
|
||||
[Click here for a commented build history of each image, with references to tag/SHA values.](https://github.com/jupyter/docker-stacks/wiki/Docker-build-history)
|
||||
|
||||
The following are quick-links to READMEs about each image and their Docker image tags on Docker Cloud:
|
||||
|
||||
* base-notebook: [README](https://github.com/jupyter/docker-stacks/tree/master/base-notebook), [SHA list](https://hub.docker.com/r/jupyter/base-notebook/tags/)
|
||||
* minimal-notebook: [README](https://github.com/jupyter/docker-stacks/tree/master/minimal-notebook), [SHA list](https://hub.docker.com/r/jupyter/minimal-notebook/tags/)
|
||||
* scipy-notebook: [README](https://github.com/jupyter/docker-stacks/tree/master/scipy-notebook), [SHA list](https://hub.docker.com/r/jupyter/scipy-notebook/tags/)
|
||||
* r-notebook: [README](https://github.com/jupyter/docker-stacks/tree/master/r-notebook), [SHA list](https://hub.docker.com/r/jupyter/r-notebook/tags/)
|
||||
* tensorflow-notebook: [README](https://github.com/jupyter/docker-stacks/tree/master/tensorflow-notebook), [SHA list](https://hub.docker.com/r/jupyter/tensorflow-notebook/tags/)
|
||||
* datascience-notebook: [README](https://github.com/jupyter/docker-stacks/tree/master/datascience-notebook), [SHA list](https://hub.docker.com/r/jupyter/datascience-notebook/tags/)
|
||||
* pyspark-notebook: [README](https://github.com/jupyter/docker-stacks/tree/master/pyspark-notebook), [SHA list](https://hub.docker.com/r/jupyter/pyspark-notebook/tags/)
|
||||
* all-spark-notebook: [README](https://github.com/jupyter/docker-stacks/tree/master/all-spark-notebook), [SHA list](https://hub.docker.com/r/jupyter/all-spark-notebook/tags/)
|
||||
|
||||
## Stacks, Tags, Versioning, and Progress
|
||||
|
||||
Starting with [git commit SHA 9bd33dcc8688](https://github.com/jupyter/docker-stacks/tree/9bd33dcc8688):
|
||||
|
||||
* Nearly every folder here on GitHub has an equivalent `jupyter/<stack name>` on Docker Hub (e.g., all-spark-notebook → jupyter/all-spark-notebook).
|
||||
* The `latest` tag in each Docker Hub repository tracks the `master` branch `HEAD` reference on GitHub.
|
||||
This is a moving target and will make backward-incompatible changes regularly.
|
||||
* Any 12-character image tag on Docker Hub refers to a git commit SHA here on GitHub. See the [Docker build history wiki page](https://github.com/jupyter/docker-stacks/wiki/Docker-build-history) for a table of build details.
|
||||
* Stack contents (e.g., new library versions) will be updated upon request via PRs against this project.
|
||||
* Users looking for reproducibility or stability should always refer to specific git SHA tagged images in their work, not `latest`.
|
||||
* For legacy reasons, there are two additional tags named `3.2` and `4.0` on Docker Hub which point to images prior to our versioning scheme switch.
|
||||
|
||||
## Other Tips and Known Issues
|
||||
|
||||
- If you haven't already, pin your image to a tag, e.g. `FROM jupyter/scipy-notebook:7c45ec67c8e7`.
|
||||
`latest` is a moving target which can change in backward-incompatible ways as packages and operating systems are updated.
|
||||
* Python 2.x was [removed from all images](https://github.com/jupyter/docker-stacks/pull/433) on August 10th, 2017, starting in tag `cc9feab481f7`. If you wish to continue using Python 2.x, pin to tag `82b978b3ceeb`.
|
||||
* `tini -- start-notebook.sh` is the default Docker entrypoint-plus-command in every notebook stack. If you plan to modify it in any way, be sure to check the *Notebook Options* section of your stack's README to understand the consequences.
|
||||
* Every notebook stack is compatible with [JupyterHub](https://jupyterhub.readthedocs.io) 0.5 or higher. When running with JupyterHub, you must override the Docker run command to point to the [start-singleuser.sh](base-notebook/start-singleuser.sh) script, which starts a single-user instance of the Notebook server. See each stack's README for instructions on running with JupyterHub.
|
||||
* Check the [Docker recipes wiki page](https://github.com/jupyter/docker-stacks/wiki/Docker-Recipes) attached to this project for information about extending and deploying the Docker images defined here. Add to the wiki if you have relevant information.
|
||||
* The pyspark-notebook and all-spark-notebook stacks will fail to submit Spark jobs to a Mesos cluster when run on Mac OSX due to https://github.com/docker/for-mac/issues/68.
|
||||
|
||||
## Maintainer Workflow
|
||||
|
||||
**To build new images on Docker Cloud and publish them to the Docker Hub registry, do the following:**
|
||||
|
||||
1. Make sure Travis is green for a PR.
|
||||
2. Merge the PR.
|
||||
3. Monitor the Docker Cloud build status for each of the stacks, starting with [jupyter/base-notebook](https://cloud.docker.com/app/jupyter/repository/docker/jupyter/base-notebook/general) and ending with [jupyter/all-spark-notebook](https://cloud.docker.com/app/jupyter/repository/docker/jupyter/all-spark-notebook/general).
|
||||
* See the stack hierarchy diagram for the current, complete build order.
|
||||
4. Manually click the retry button next to any build that fails to resume that build and any dependent builds.
|
||||
5. Avoid merging another PR to master until all outstanding builds complete.
|
||||
* There's no way at present to propagate the git SHA to build through the Docker Cloud build trigger API. Every build trigger works off of master HEAD.
|
||||
|
||||
**When there's a security fix in the Ubuntu base image, do the following in place of the last command:**
|
||||
|
||||
Update the `ubuntu:16.04` SHA in the most-base images (e.g., base-notebook). Submit it as a regular PR and go through the build process. Expect the build to take a while to complete: every image layer will rebuild.
|
||||
|
||||
**When there's a new stack definition, do the following before merging the PR with the new stack:**
|
||||
|
||||
1. Ensure the PR includes an update to the stack overview diagram in the top-level README.
|
||||
* The source of the diagram is included in the alt-text of the image. Visit that URL to make edits.
|
||||
2. Ensure the PR updates the Makefile which is used to build the stacks in order on Travis CI.
|
||||
3. Create a new repoistory in the `jupyter` org on Docker Cloud named after the stack folder in the git repo.
|
||||
4. Grant the `stacks` team permission to write to the repo.
|
||||
5. Click *Builds* and then *Configure Automated Builds* for the repository.
|
||||
6. Select `jupyter/docker-stacks` as the source repository.
|
||||
7. Choose *Build on Docker Cloud's infrastructure using a Small node* unless you have reason to believe a bigger host is required.
|
||||
8. Update the *Build Context* in the default build rule to be `/<name-of-the-stack>`.
|
||||
9. Toggle *Autobuild* to disabled unless the stack is a new root stack (e.g., like `jupyter/base-notebook`).
|
||||
10. If the new stack depends on the build of another stack in the hierarchy:
|
||||
1. Hit *Save* and then click *Configure Automated Builds*.
|
||||
2. At the very bottom, add a build trigger named *Stack hierarchy trigger*.
|
||||
3. Copy the build trigger URL.
|
||||
4. Visit the parent repository *Builds* page and click *Configure Automated Builds*.
|
||||
5. Add the URL you copied to the *NEXT_BUILD_TRIGGERS* environment variable comma separated list of URLs, creating that environment variable if it does not already exist.
|
||||
6. Hit *Save*.
|
||||
11. If the new stack should trigger other dependent builds:
|
||||
1. Add an environment variable named *NEXT_BUILD_TRIGGERS*.
|
||||
2. Copy the build trigger URLs from the dependent builds into the *NEXT_BUILD_TRIGGERS* comma separated list of URLs.
|
||||
3. Hit *Save*.
|
||||
12. Adjust other *NEXT_BUILD_TRIGGERS* values as needed so that the build order matches that in the stack hierarchy diagram.
|
||||
|
||||
**When there's a new maintainer, do the following:**
|
||||
|
||||
1. Visit https://cloud.docker.com/app/jupyter/team/stacks/users
|
||||
2. Add the new maintainer user name.
|
||||
|
||||
**If automated builds have got you down, do the following:**
|
||||
|
||||
1. Clone this repository.
|
||||
2. Check out the git SHA you want to build and publish.
|
||||
3. `docker login` with your Docker Hub/Cloud credentials.
|
||||
4. Run `make retry/release-all`.
|
||||
|
||||
When `make retry/release-all` successfully pushes the last of its images to Docker Hub (currently `jupyter/all-spark-notebook`), Docker Hub invokes [the webhook](https://github.com/jupyter/docker-stacks/blob/master/internal/docker-stacks-webhook/) which updates the [Docker build history](https://github.com/jupyter/docker-stacks/wiki/Docker-build-history) wiki page.
|
||||
|
||||
@@ -1,307 +1,9 @@
|
||||
  [](https://microbadger.com/images/jupyter/all-spark-notebook "jupyter/all-spark-notebook image metadata")
|
||||
|
||||
|
||||
# Jupyter Notebook Python, Scala, R, Spark, Mesos Stack
|
||||
|
||||
## What it Gives You
|
||||
Please visit the documentation site for help using and contributing to this image and others.
|
||||
|
||||
* Jupyter Notebook 5.2.x
|
||||
* Conda Python 3.x environment
|
||||
* Conda R 3.3.x environment
|
||||
* Scala 2.11.x
|
||||
* pyspark, pandas, matplotlib, scipy, seaborn, scikit-learn pre-installed for Python
|
||||
* ggplot2, rcurl preinstalled for R
|
||||
* Spark 2.2.0 with Hadoop 2.7 for use in local mode or to connect to a cluster of Spark workers
|
||||
* Mesos client 1.2 binary that can communicate with a Mesos master
|
||||
* spylon-kernel
|
||||
* Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda`
|
||||
* [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../base-notebook/start-notebook.sh) as the default command
|
||||
* `/usr/local/bin/start-notebook.d` directory for custom init scripts that you can add in derived images
|
||||
* A [start-singleuser.sh](../base-notebook/start-singleuser.sh) script useful for running a single-user instance of the Notebook server, as required by JupyterHub
|
||||
* A [start.sh](../base-notebook/start.sh) script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
|
||||
* Options for a self-signed HTTPS certificate and passwordless `sudo`
|
||||
|
||||
## Basic Use
|
||||
|
||||
The following command starts a container with the Notebook server listening for HTTP connections on port 8888 with a randomly generated authentication token configured.
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/all-spark-notebook
|
||||
```
|
||||
|
||||
Take note of the authentication token included in the notebook startup log messages. Include it in the URL you visit to access the Notebook server or enter it in the Notebook login form.
|
||||
|
||||
## Using Spark Local Mode
|
||||
|
||||
This configuration is nice for using Spark on small, local data.
|
||||
|
||||
### In a Python Notebook
|
||||
|
||||
0. Run the container as shown above.
|
||||
1. Open a Python 2 or 3 notebook.
|
||||
2. Create a `SparkContext` configured for local mode.
|
||||
|
||||
For example, the first few cells in a notebook might read:
|
||||
|
||||
```python
|
||||
import pyspark
|
||||
sc = pyspark.SparkContext('local[*]')
|
||||
|
||||
# do something to prove it works
|
||||
rdd = sc.parallelize(range(1000))
|
||||
rdd.takeSample(False, 5)
|
||||
```
|
||||
|
||||
### In a R Notebook
|
||||
|
||||
0. Run the container as shown above.
|
||||
1. Open a R notebook.
|
||||
2. Initialize a `sparkR` session for local mode.
|
||||
|
||||
For example, the first few cells in a R notebook might read:
|
||||
|
||||
```
|
||||
library(SparkR)
|
||||
|
||||
as <- sparkR.session("local[*]")
|
||||
|
||||
# do something to prove it works
|
||||
df <- as.DataFrame(iris)
|
||||
head(filter(df, df$Petal_Width > 0.2))
|
||||
```
|
||||
|
||||
### In an Apache Toree - Scala Notebook
|
||||
|
||||
0. Run the container as shown above.
|
||||
1. Open an Apache Toree - Scala notebook.
|
||||
2. Use the pre-configured `SparkContext` in variable `sc`.
|
||||
|
||||
For example:
|
||||
|
||||
```
|
||||
val rdd = sc.parallelize(0 to 999)
|
||||
rdd.takeSample(false, 5)
|
||||
```
|
||||
|
||||
### In spylon-kernel - Scala Notebook
|
||||
|
||||
0. Run the container as shown above.
|
||||
1. Open a spylon-kernel notebook
|
||||
2. Lazily instantiate the sparkcontext by just running any cell without magics
|
||||
|
||||
For example
|
||||
|
||||
```
|
||||
val rdd = sc.parallelize(0 to 999)
|
||||
rdd.takeSample(false, 5)
|
||||
```
|
||||
|
||||
## Connecting to a Spark Cluster on Mesos
|
||||
|
||||
This configuration allows your compute cluster to scale with your data.
|
||||
|
||||
0. [Deploy Spark on Mesos](http://spark.apache.org/docs/latest/running-on-mesos.html).
|
||||
1. Configure each slave with [the `--no-switch_user` flag](https://open.mesosphere.com/reference/mesos-slave/) or create the `jovyan` user on every slave node.
|
||||
2. Run the Docker container with `--net=host` in a location that is network addressable by all of your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).)
|
||||
* NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details.
|
||||
3. Follow the language specific instructions below.
|
||||
|
||||
### In a Python Notebook
|
||||
|
||||
0. Open a Python 2 or 3 notebook.
|
||||
1. Create a `SparkConf` instance in a new notebook pointing to your Mesos master node (or Zookeeper instance) and Spark binary package location.
|
||||
2. Create a `SparkContext` using this configuration.
|
||||
|
||||
For example, the first few cells in a Python 3 notebook might read:
|
||||
|
||||
```python
|
||||
import os
|
||||
# make sure pyspark tells workers to use python3 not 2 if both are installed
|
||||
os.environ['PYSPARK_PYTHON'] = '/usr/bin/python3'
|
||||
|
||||
import pyspark
|
||||
conf = pyspark.SparkConf()
|
||||
|
||||
# point to mesos master or zookeeper entry (e.g., zk://10.10.10.10:2181/mesos)
|
||||
conf.setMaster("mesos://10.10.10.10:5050")
|
||||
# point to spark binary package in HDFS or on local filesystem on all slave
|
||||
# nodes (e.g., file:///opt/spark/spark-2.2.0-bin-hadoop2.7.tgz)
|
||||
conf.set("spark.executor.uri", "hdfs://10.10.10.10/spark/spark-2.2.0-bin-hadoop2.7.tgz")
|
||||
# set other options as desired
|
||||
conf.set("spark.executor.memory", "8g")
|
||||
conf.set("spark.core.connection.ack.wait.timeout", "1200")
|
||||
|
||||
# create the context
|
||||
sc = pyspark.SparkContext(conf=conf)
|
||||
|
||||
# do something to prove it works
|
||||
rdd = sc.parallelize(range(100000000))
|
||||
rdd.sumApprox(3)
|
||||
```
|
||||
|
||||
To use Python 2 in the notebook and on the workers, change the `PYSPARK_PYTHON` environment variable to point to the location of the Python 2.x interpreter binary. If you leave this environment variable unset, it defaults to `python`.
|
||||
|
||||
Of course, all of this can be hidden in an [IPython kernel startup script](http://ipython.org/ipython-doc/stable/development/config.html?highlight=startup#startup-files), but "explicit is better than implicit." :)
|
||||
|
||||
### In a R Notebook
|
||||
|
||||
0. Run the container as shown above.
|
||||
1. Open a R notebook.
|
||||
2. Initialize `sparkR` Mesos master node (or Zookeeper instance) and Spark binary package location.
|
||||
3. Initialize `sparkRSQL`.
|
||||
|
||||
For example, the first few cells in a R notebook might read:
|
||||
|
||||
```
|
||||
library(SparkR)
|
||||
|
||||
# point to mesos master or zookeeper entry (e.g., zk://10.10.10.10:2181/mesos)\
|
||||
# as the first argument
|
||||
# point to spark binary package in HDFS or on local filesystem on all slave
|
||||
# nodes (e.g., file:///opt/spark/spark-2.2.0-bin-hadoop2.7.tgz) in sparkEnvir
|
||||
# set other options in sparkEnvir
|
||||
sc <- sparkR.session("mesos://10.10.10.10:5050", sparkEnvir=list(
|
||||
spark.executor.uri="hdfs://10.10.10.10/spark/spark-2.2.0-bin-hadoop2.7.tgz",
|
||||
spark.executor.memory="8g"
|
||||
)
|
||||
)
|
||||
|
||||
# do something to prove it works
|
||||
data(iris)
|
||||
df <- as.DataFrame(iris)
|
||||
head(filter(df, df$Petal_Width > 0.2))
|
||||
```
|
||||
|
||||
### In an Apache Toree - Scala Notebook
|
||||
|
||||
0. Open a terminal via *New -> Terminal* in the notebook interface.
|
||||
1. Add information about your cluster to the `SPARK_OPTS` environment variable when running the container.
|
||||
2. Open an Apache Toree - Scala notebook.
|
||||
3. Use the pre-configured `SparkContext` in variable `sc` or `SparkSession` in variable `spark`.
|
||||
|
||||
The Apache Toree kernel automatically creates a `SparkContext` when it starts based on configuration information from its command line arguments and environment variables. You can pass information about your Mesos cluster via the `SPARK_OPTS` environment variable when you spawn a container.
|
||||
|
||||
For instance, to pass information about a Mesos master, Spark binary location in HDFS, and an executor options, you could start the container like so:
|
||||
|
||||
`docker run -d -p 8888:8888 -e SPARK_OPTS '--master=mesos://10.10.10.10:5050 \
|
||||
--spark.executor.uri=hdfs://10.10.10.10/spark/spark-2.2.0-bin-hadoop2.7.tgz \
|
||||
--spark.executor.memory=8g' jupyter/all-spark-notebook`
|
||||
|
||||
Note that this is the same information expressed in a notebook in the Python case above. Once the kernel spec has your cluster information, you can test your cluster in an Apache Toree notebook like so:
|
||||
|
||||
```
|
||||
// should print the value of --master in the kernel spec
|
||||
println(sc.master)
|
||||
|
||||
// do something to prove it works
|
||||
val rdd = sc.parallelize(0 to 99999999)
|
||||
rdd.sum()
|
||||
```
|
||||
## Connecting to a Spark Cluster on Standalone Mode
|
||||
|
||||
Connection to Spark Cluster on Standalone Mode requires the following set of steps:
|
||||
|
||||
0. Verify that the docker image (check the Dockerfile) and the Spark Cluster which is being deployed, run the same version of Spark.
|
||||
1. [Deploy Spark on Standalone Mode](http://spark.apache.org/docs/latest/spark-standalone.html).
|
||||
2. Run the Docker container with `--net=host` in a location that is network addressable by all of your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).)
|
||||
* NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details.
|
||||
3. The language specific instructions are almost same as mentioned above for Mesos, only the master url would now be something like spark://10.10.10.10:7077
|
||||
|
||||
## Notebook Options
|
||||
|
||||
The Docker container executes a [`start-notebook.sh` script](../base-notebook/start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
|
||||
|
||||
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed ([how-to](http://jupyter-notebook.readthedocs.io/en/latest/public_server.html#preparing-a-hashed-password)) instead of the default token, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/all-spark-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
|
||||
```
|
||||
|
||||
For example, to set the base URL of the notebook server, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/all-spark-notebook start-notebook.sh --NotebookApp.base_url=/some/path
|
||||
```
|
||||
|
||||
For example, to disable all authentication mechanisms (not a recommended practice):
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/all-spark-notebook start-notebook.sh --NotebookApp.token=''
|
||||
```
|
||||
|
||||
You can sidestep the `start-notebook.sh` script and run your own commands in the container. See the *Alternative Commands* section later in this document for more information.
|
||||
|
||||
## Docker Options
|
||||
|
||||
You may customize the execution of the Docker container and the command it is running with the following optional arguments.
|
||||
|
||||
* `-e GEN_CERT=yes` - Generates a self-signed SSL certificate and configures Jupyter Notebook to use it to accept encrypted HTTPS connections.
|
||||
* `-e NB_UID=1000` - Specify the uid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the user id.)
|
||||
* `-e NB_GID=100` - Specify the gid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the group id.)
|
||||
* `-e GRANT_SUDO=yes` - Gives the `jovyan` user passwordless `sudo` capability. Useful for installing OS packages. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adding `jovyan` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
|
||||
* `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
|
||||
|
||||
## SSL Certificates
|
||||
|
||||
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt`:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder:/etc/ssl/notebook \
|
||||
jupyter/all-spark-notebook start-notebook.sh \
|
||||
--NotebookApp.keyfile=/etc/ssl/notebook/notebook.key
|
||||
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
|
||||
```
|
||||
|
||||
Alternatively, you may mount a single PEM file containing both the key and certificate. For example:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \
|
||||
jupyter/all-spark-notebook start-notebook.sh \
|
||||
--NotebookApp.certfile=/etc/ssl/notebook.pem
|
||||
```
|
||||
|
||||
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
|
||||
|
||||
For additional information about using SSL, see the following:
|
||||
|
||||
* The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
|
||||
* The [jupyter_notebook_config.py](jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
|
||||
* The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#using-ssl-for-encrypted-communication) for best practices about running a public notebook server in general, most of which are encoded in this image.
|
||||
|
||||
|
||||
## Conda Environments
|
||||
|
||||
The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/envs.html) resides in `/opt/conda`.
|
||||
|
||||
The commands `jupyter`, `ipython`, `python`, `pip`, and `conda` (among others) are available in both environments. For convenience, you can install packages into either environment regardless of what environment is currently active using commands like the following:
|
||||
|
||||
```
|
||||
# install a package into the default (python 3.x) environment
|
||||
pip install some-package
|
||||
conda install some-package
|
||||
```
|
||||
|
||||
|
||||
## Alternative Commands
|
||||
|
||||
### start.sh
|
||||
|
||||
The `start.sh` script supports the same features as the default `start-notebook.sh` script (e.g., `GRANT_SUDO`), but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following:
|
||||
|
||||
```
|
||||
docker run -it --rm jupyter/all-spark-notebook start.sh ipython
|
||||
```
|
||||
|
||||
Or, to run JupyterLab instead of the classic notebook, run the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/all-spark-notebook start.sh jupyter lab
|
||||
```
|
||||
|
||||
This script is particularly useful when you derive a new Dockerfile from this image and install additional Jupyter applications with subcommands like `jupyter console`, `jupyter kernelgateway`, etc.
|
||||
|
||||
### Others
|
||||
|
||||
You can bypass the provided scripts and specify your an arbitrary start command. If you do, keep in mind that certain features documented above will not function (e.g., `GRANT_SUDO`).
|
||||
* [Jupyter Docker Stacks on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
|
||||
* [Selecting an Image :: Core Stacks :: jupyter/all-spark-notebook](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-all-spark-notebook)
|
||||
* [Image Specifics :: Apache Spark](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/specifics.html#apache-spark)
|
||||
|
||||
@@ -2,134 +2,7 @@
|
||||
|
||||
# Base Jupyter Notebook Stack
|
||||
|
||||
Small base image for defining your own stack
|
||||
|
||||
## What it Gives You
|
||||
|
||||
* Minimally-functional Jupyter Notebook 5.2.x (e.g., no pandoc for document conversion)
|
||||
* Miniconda Python 3.x
|
||||
* No preinstalled scientific computing packages
|
||||
* Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda`
|
||||
* [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](./start-notebook.sh) as the default command
|
||||
* A [start-singleuser.sh](./start-singleuser.sh) script useful for running a single-user instance of the Notebook server, as required by JupyterHub
|
||||
* A [start.sh](./start.sh) script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
|
||||
* Options for a self-signed HTTPS certificate and passwordless `sudo`
|
||||
|
||||
## Basic Use
|
||||
|
||||
The following command starts a container with the Notebook server listening for HTTP connections on port 8888 with a randomly generated authentication token configured.
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/base-notebook
|
||||
```
|
||||
|
||||
Take note of the authentication token included in the notebook startup log messages. Include it in the URL you visit to access the Notebook server or enter it in the Notebook login form.
|
||||
|
||||
## Notebook Options
|
||||
|
||||
The Docker container executes a [`start-notebook.sh` script](./start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
|
||||
|
||||
You can launch [JupyterLab](https://github.com/jupyterlab/jupyterlab) by setting `JUPYTER_ENABLE_LAB`:
|
||||
|
||||
```
|
||||
docker run -it --rm -e JUPYTER_ENABLE_LAB=1 --rm -p 8888:8888 jupyter/base-notebook
|
||||
```
|
||||
|
||||
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/base-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
|
||||
```
|
||||
|
||||
For example, to set the base URL of the notebook server, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/base-notebook start-notebook.sh --NotebookApp.base_url=/some/path
|
||||
```
|
||||
|
||||
For example, to disable all authentication mechanisms (not a recommended practice):
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/base-notebook start-notebook.sh --NotebookApp.token=''
|
||||
```
|
||||
|
||||
You can sidestep the `start-notebook.sh` script and run your own commands in the container. See the *Alternative Commands* section later in this document for more information.
|
||||
|
||||
## Docker Options
|
||||
|
||||
You may customize the execution of the Docker container and the command it is running with the following optional arguments.
|
||||
|
||||
* `-e GEN_CERT=yes` - Generates a self-signed SSL certificate and configures Jupyter Notebook to use it to accept encrypted HTTPS connections.
|
||||
* `-e NB_UID=1000` - Specify the uid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the user id.)
|
||||
* `-e NB_GID=100` - Specify the gid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the group id.)
|
||||
* `-e GRANT_SUDO=yes` - Gives the `jovyan` user passwordless `sudo` capability. Useful for installing OS packages. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adding `jovyan` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
|
||||
* `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
|
||||
* `--group-add users` - use this argument if you are also specifying
|
||||
a specific user id to launch the container (`-u 5000`), rather than launching the container as root and relying on NB_UID and NB_GID to set the user and group.
|
||||
|
||||
|
||||
## SSL Certificates
|
||||
|
||||
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt`:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder:/etc/ssl/notebook \
|
||||
jupyter/base-notebook start-notebook.sh \
|
||||
--NotebookApp.keyfile=/etc/ssl/notebook/notebook.key
|
||||
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
|
||||
```
|
||||
|
||||
Alternatively, you may mount a single PEM file containing both the key and certificate. For example:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \
|
||||
jupyter/base-notebook start-notebook.sh \
|
||||
--NotebookApp.certfile=/etc/ssl/notebook.pem
|
||||
```
|
||||
|
||||
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
|
||||
|
||||
For additional information about using SSL, see the following:
|
||||
|
||||
* The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
|
||||
* The [jupyter_notebook_config.py](jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
|
||||
* The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#using-ssl-for-encrypted-communication) for best practices about running a public notebook server in general, most of which are encoded in this image.
|
||||
|
||||
|
||||
## Conda Environments
|
||||
|
||||
The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/envs.html) resides in `/opt/conda`.
|
||||
|
||||
The commands `jupyter`, `ipython`, `python`, `pip`, and `conda` (among others) are available in both environments. For convenience, you can install packages into either environment regardless of what environment is currently active using commands like the following:
|
||||
|
||||
```
|
||||
# install a package into the default (python 3.x) environment
|
||||
pip install some-package
|
||||
conda install some-package
|
||||
```
|
||||
|
||||
|
||||
## Alternative Commands
|
||||
|
||||
### start.sh
|
||||
|
||||
The `start.sh` script supports the same features as the default `start-notebook.sh` script (e.g., `GRANT_SUDO`), but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following:
|
||||
|
||||
```
|
||||
docker run -it --rm jupyter/base-notebook start.sh ipython
|
||||
```
|
||||
|
||||
Or, to run JupyterLab instead of the classic notebook, run the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/base-notebook start.sh jupyter lab
|
||||
```
|
||||
|
||||
This script is particularly useful when you derive a new Dockerfile from this image and install additional Jupyter applications with subcommands like `jupyter console`, `jupyter kernelgateway`, etc.
|
||||
|
||||
### Others
|
||||
|
||||
You can bypass the provided scripts and specify your an arbitrary start command. If you do, keep in mind that certain features documented above will not function (e.g., `GRANT_SUDO`).
|
||||
Please visit the documentation site for help using and contributing to this image and others.
|
||||
|
||||
* [Jupyter Docker Stacks on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
|
||||
* [Selecting an Image :: Core Stacks :: jupyter/base-notebook](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-base-notebook)
|
||||
|
||||
@@ -2,128 +2,7 @@
|
||||
|
||||
# Jupyter Notebook Data Science Stack
|
||||
|
||||
## What it Gives You
|
||||
Please visit the documentation site for help using and contributing to this image and others.
|
||||
|
||||
* Jupyter Notebook 5.2.x
|
||||
* Conda Python 3.x environment
|
||||
* pandas, matplotlib, scipy, seaborn, scikit-learn, scikit-image, sympy, cython, patsy, statsmodel, cloudpickle, dill, numba, bokeh pre-installed
|
||||
* Conda R v3.3.x and channel
|
||||
* plyr, devtools, shiny, rmarkdown, forecast, rsqlite, reshape2, nycflights13, caret, rcurl, and randomforest pre-installed
|
||||
* The [tidyverse](https://github.com/tidyverse/tidyverse) R packages are also installed, including ggplot2, dplyr, tidyr, readr, purrr, tibble, stringr, lubridate, and broom
|
||||
* Julia v0.6.x with Gadfly, RDatasets and HDF5 pre-installed
|
||||
* Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda`
|
||||
* [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../base-notebook/start-notebook.sh) as the default command
|
||||
* `/usr/local/bin/start-notebook.d` directory for custom init scripts that you can add in derived images
|
||||
* A [start-singleuser.sh](../base-notebook/start-singleuser.sh) script useful for running a single-user instance of the Notebook server, as required by JupyterHub
|
||||
* A [start.sh](../base-notebook/start.sh) script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
|
||||
* Options for a self-signed HTTPS certificate and passwordless `sudo`
|
||||
|
||||
## Basic Use
|
||||
|
||||
The following command starts a container with the Notebook server listening for HTTP connections on port 8888 with a randomly generated authentication token configured.
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/datascience-notebook
|
||||
```
|
||||
|
||||
Take note of the authentication token included in the notebook startup log messages. Include it in the URL you visit to access the Notebook server or enter it in the Notebook login form.
|
||||
|
||||
## Notebook Options
|
||||
|
||||
The Docker container executes a [`start-notebook.sh` script](../base-notebook/start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
|
||||
|
||||
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/datascience-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
|
||||
```
|
||||
|
||||
For example, to set the base URL of the notebook server, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/datascience-notebook start-notebook.sh --NotebookApp.base_url=/some/path
|
||||
```
|
||||
|
||||
For example, to disable all authentication mechanisms (not a recommended practice):
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/datascience-notebook start-notebook.sh --NotebookApp.token=''
|
||||
```
|
||||
|
||||
You can sidestep the `start-notebook.sh` script and run your own commands in the container. See the *Alternative Commands* section later in this document for more information.
|
||||
|
||||
## Docker Options
|
||||
|
||||
You may customize the execution of the Docker container and the command it is running with the following optional arguments.
|
||||
|
||||
* `-e GEN_CERT=yes` - Generates a self-signed SSL certificate and configures Jupyter Notebook to use it to accept encrypted HTTPS connections.
|
||||
* `-e NB_UID=1000` - Specify the uid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the user id.)
|
||||
* `-e NB_GID=100` - Specify the gid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the group id.)
|
||||
* `-e GRANT_SUDO=yes` - Gives the `jovyan` user passwordless `sudo` capability. Useful for installing OS packages. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adding `jovyan` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
|
||||
* `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
|
||||
|
||||
## SSL Certificates
|
||||
|
||||
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt`:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder:/etc/ssl/notebook \
|
||||
jupyter/datascience-notebook start-notebook.sh \
|
||||
--NotebookApp.keyfile=/etc/ssl/notebook/notebook.key
|
||||
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
|
||||
```
|
||||
|
||||
Alternatively, you may mount a single PEM file containing both the key and certificate. For example:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \
|
||||
jupyter/datascience-notebook start-notebook.sh \
|
||||
--NotebookApp.certfile=/etc/ssl/notebook.pem
|
||||
```
|
||||
|
||||
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
|
||||
|
||||
For additional information about using SSL, see the following:
|
||||
|
||||
* The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
|
||||
* The [jupyter_notebook_config.py](jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
|
||||
* The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#using-ssl-for-encrypted-communication) for best practices about running a public notebook server in general, most of which are encoded in this image.
|
||||
|
||||
|
||||
## Conda Environments
|
||||
|
||||
The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/envs.html) resides in `/opt/conda`.
|
||||
|
||||
The commands `jupyter`, `ipython`, `python`, `pip`, and `conda` (among others) are available in both environments. For convenience, you can install packages into either environment regardless of what environment is currently active using commands like the following:
|
||||
|
||||
```
|
||||
# install a package into the default (python 3.x) environment
|
||||
pip install some-package
|
||||
conda install some-package
|
||||
```
|
||||
|
||||
|
||||
## Alternative Commands
|
||||
|
||||
|
||||
### start.sh
|
||||
|
||||
The `start.sh` script supports the same features as the default `start-notebook.sh` script (e.g., `GRANT_SUDO`), but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following:
|
||||
|
||||
```
|
||||
docker run -it --rm jupyter/datascience-notebook start.sh ipython
|
||||
```
|
||||
|
||||
Or, to run JupyterLab instead of the classic notebook, run the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/datascience-notebook start.sh jupyter lab
|
||||
```
|
||||
|
||||
This script is particularly useful when you derive a new Dockerfile from this image and install additional Jupyter applications with subcommands like `jupyter console`, `jupyter kernelgateway`, etc.
|
||||
|
||||
### Others
|
||||
|
||||
You can bypass the provided scripts and specify your an arbitrary start command. If you do, keep in mind that certain features documented above will not function (e.g., `GRANT_SUDO`).
|
||||
* [Jupyter Docker Stacks on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
|
||||
* [Selecting an Image :: Core Stacks :: jupyter/datascience-notebook](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-datascience-notebook)
|
||||
|
||||
@@ -43,6 +43,12 @@ Table of Contents
|
||||
contributing/tests
|
||||
contributing/stacks
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Maintainer Guide
|
||||
|
||||
maintaining/tasks
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Getting Help
|
||||
|
||||
57
docs/maintaining/tasks.md
Normal file
57
docs/maintaining/tasks.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# Maintainer Playbook
|
||||
|
||||
## Merging Pull Requests
|
||||
|
||||
To build new images on Docker Cloud and publish them to the Docker Hub registry, do the following:
|
||||
|
||||
1. Make sure Travis is green for a PR.
|
||||
2. Merge the PR.
|
||||
3. Monitor the Docker Cloud build status for each of the stacks, starting with [jupyter/base-notebook](https://cloud.docker.com/app/jupyter/repository/docker/jupyter/base-notebook/general) and ending with [jupyter/all-spark-notebook](https://cloud.docker.com/app/jupyter/repository/docker/jupyter/all-spark-notebook/general). See the [stack hierarchy diagram](../using/selecting.html#image-relationships) for the current, complete build order.
|
||||
4. Manually click the retry button next to any build that fails to resume that build and any dependent builds.
|
||||
5. Try to avoid merging another PR to master until all outstanding builds complete. There's no way at present to propagate the git SHA to build through the Docker Cloud build trigger API. Every build trigger works off of master HEAD.
|
||||
|
||||
## Updating the Ubuntu Base Image
|
||||
|
||||
When there's a security fix in the Ubuntu base image or after some time passes, it's a good idea to update the pinned SHA in the [jupyter/base-notebook Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile). Submit it as a regular PR and go through the build process. Expect the build to take a while to complete: every image layer will rebuild.
|
||||
|
||||
## Adding a New Core Image to Docker Cloud
|
||||
|
||||
When there's a new stack definition, do the following before merging the PR with the new stack:
|
||||
|
||||
1. Ensure the PR includes an update to the stack overview diagram [in the documentation](https://github.com/jupyter/docker-stacks/blob/master/docs/using/selecting.md#image-relationships). The image links to the [blockdiag source](http://interactive.blockdiag.com/) used to create it.
|
||||
2. Ensure the PR updates the Makefile which is used to build the stacks in order on Travis CI.
|
||||
3. Create a new repository in the `jupyter` org on Docker Cloud named after the stack folder in the git repo.
|
||||
4. Grant the `stacks` team permission to write to the repo.
|
||||
5. Click *Builds* and then *Configure Automated Builds* for the repository.
|
||||
6. Select `jupyter/docker-stacks` as the source repository.
|
||||
7. Choose *Build on Docker Cloud's infrastructure using a Small node* unless you have reason to believe a bigger host is required.
|
||||
8. Update the *Build Context* in the default build rule to be `/<name-of-the-stack>`.
|
||||
9. Toggle *Autobuild* to disabled unless the stack is a new root stack (e.g., like `jupyter/base-notebook`).
|
||||
10. If the new stack depends on the build of another stack in the hierarchy:
|
||||
1. Hit *Save* and then click *Configure Automated Builds*.
|
||||
2. At the very bottom, add a build trigger named *Stack hierarchy trigger*.
|
||||
3. Copy the build trigger URL.
|
||||
4. Visit the parent repository *Builds* page and click *Configure Automated Builds*.
|
||||
5. Add the URL you copied to the *NEXT_BUILD_TRIGGERS* environment variable comma separated list of URLs, creating that environment variable if it does not already exist.
|
||||
6. Hit *Save*.
|
||||
11. If the new stack should trigger other dependent builds:
|
||||
1. Add an environment variable named *NEXT_BUILD_TRIGGERS*.
|
||||
2. Copy the build trigger URLs from the dependent builds into the *NEXT_BUILD_TRIGGERS* comma separated list of URLs.
|
||||
3. Hit *Save*.
|
||||
12. Adjust other *NEXT_BUILD_TRIGGERS* values as needed so that the build order matches that in the stack hierarchy diagram.
|
||||
|
||||
## Adding a New Maintainer Account
|
||||
|
||||
1. Visit https://cloud.docker.com/app/jupyter/team/stacks/users
|
||||
2. Add the maintainer's Docker Cloud username.
|
||||
3. Visit https://github.com/orgs/jupyter/teams/docker-image-maintainers/members
|
||||
4. Add the maintainer's GitHub username.
|
||||
|
||||
## Pushing a Build Manually
|
||||
|
||||
If automated builds on Docker Cloud have got you down, do the following to push a build manually:
|
||||
|
||||
1. Clone this repository.
|
||||
2. Check out the git SHA you want to build and publish.
|
||||
3. `docker login` with your Docker Hub/Cloud credentials.
|
||||
4. Run `make retry/release-all`.
|
||||
@@ -2,127 +2,7 @@
|
||||
|
||||
# Minimal Jupyter Notebook Stack
|
||||
|
||||
Small image for working in the notebook and installing your own libraries
|
||||
Please visit the documentation site for help using and contributing to this image and others.
|
||||
|
||||
## What it Gives You
|
||||
|
||||
* Fully-functional Jupyter Notebook 5.2.x
|
||||
* Miniconda Python 3.x
|
||||
* No preinstalled scientific computing packages
|
||||
* Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda`
|
||||
* [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../base-notebook/start-notebook.sh) as the default command
|
||||
* A [start-singleuser.sh](../base-notebook/start-singleuser.sh) script useful for running a single-user instance of the Notebook server, as required by JupyterHub
|
||||
* `/usr/local/bin/start-notebook.d` directory for custom init scripts that you can add in derived images
|
||||
* A [start.sh](../base-notebook/start.sh) script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
|
||||
* Options for a self-signed HTTPS certificate and passwordless `sudo`
|
||||
|
||||
## Basic Use
|
||||
|
||||
The following command starts a container with the Notebook server listening for HTTP connections on port 8888 with a randomly generated authentication token configured.
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/minimal-notebook
|
||||
```
|
||||
|
||||
Take note of the authentication token included in the notebook startup log messages. Include it in the URL you visit to access the Notebook server or enter it in the Notebook login form.
|
||||
|
||||
## Notebook Options
|
||||
|
||||
The Docker container executes a [`start-notebook.sh` script](../base-notebook/start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
|
||||
|
||||
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/minimal-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
|
||||
```
|
||||
|
||||
For example, to set the base URL of the notebook server, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/minimal-notebook start-notebook.sh --NotebookApp.base_url=/some/path
|
||||
```
|
||||
|
||||
For example, to disable all authentication mechanisms (not a recommended practice):
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/minimal-notebook start-notebook.sh --NotebookApp.token=''
|
||||
```
|
||||
|
||||
You can sidestep the `start-notebook.sh` script and run your own commands in the container. See the *Alternative Commands* section later in this document for more information.
|
||||
|
||||
|
||||
## Docker Options
|
||||
|
||||
You may customize the execution of the Docker container and the Notebook server it contains with the following optional arguments.
|
||||
|
||||
* `-e GEN_CERT=yes` - Generates a self-signed SSL certificate and configures Jupyter Notebook to use it to accept encrypted HTTPS connections.
|
||||
* `-e NB_UID=1000` - Specify the uid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the user id.)
|
||||
* `-e NB_GID=100` - Specify the gid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the group id.)
|
||||
* `-e GRANT_SUDO=yes` - Gives the `jovyan` user passwordless `sudo` capability. Useful for installing OS packages. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adding `jovyan` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
|
||||
* `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
|
||||
|
||||
## SSL Certificates
|
||||
|
||||
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt`:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder:/etc/ssl/notebook \
|
||||
jupyter/minimal-notebook start-notebook.sh \
|
||||
--NotebookApp.keyfile=/etc/ssl/notebook/notebook.key
|
||||
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
|
||||
```
|
||||
|
||||
Alternatively, you may mount a single PEM file containing both the key and certificate. For example:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \
|
||||
jupyter/minimal-notebook start-notebook.sh \
|
||||
--NotebookApp.certfile=/etc/ssl/notebook.pem
|
||||
```
|
||||
|
||||
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
|
||||
|
||||
For additional information about using SSL, see the following:
|
||||
|
||||
* The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
|
||||
* The [jupyter_notebook_config.py](jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
|
||||
* The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#using-ssl-for-encrypted-communication) for best practices about running a public notebook server in general, most of which are encoded in this image.
|
||||
|
||||
|
||||
## Conda Environments
|
||||
|
||||
The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/envs.html) resides in `/opt/conda`.
|
||||
|
||||
The commands `jupyter`, `ipython`, `python`, `pip`, and `conda` (among others) are available in both environments. For convenience, you can install packages into either environment regardless of what environment is currently active using commands like the following:
|
||||
|
||||
```
|
||||
# install a package into the default (python 3.x) environment
|
||||
pip install some-package
|
||||
conda install some-package
|
||||
```
|
||||
|
||||
|
||||
## Alternative Commands
|
||||
|
||||
|
||||
### start.sh
|
||||
|
||||
The `start.sh` script supports the same features as the default `start-notebook.sh` script (e.g., `GRANT_SUDO`), but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following:
|
||||
|
||||
```
|
||||
docker run -it --rm jupyter/minimal-notebook start.sh ipython
|
||||
```
|
||||
|
||||
Or, to run JupyterLab instead of the classic notebook, run the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/minimal-notebook start.sh jupyter lab
|
||||
```
|
||||
|
||||
This script is particularly useful when you derive a new Dockerfile from this image and install additional Jupyter applications with subcommands like `jupyter console`, `jupyter kernelgateway`, etc.
|
||||
|
||||
### Others
|
||||
|
||||
You can bypass the provided scripts and specify your an arbitrary start command. If you do, keep in mind that certain features documented above will not function (e.g., `GRANT_SUDO`).
|
||||
* [Jupyter Docker Stacks on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
|
||||
* [Selecting an Image :: Core Stacks :: jupyter/minimal-notebook](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-minimal-notebook)
|
||||
|
||||
@@ -2,199 +2,8 @@
|
||||
|
||||
# Jupyter Notebook Python, Spark, Mesos Stack
|
||||
|
||||
## What it Gives You
|
||||
Please visit the documentation site for help using and contributing to this image and others.
|
||||
|
||||
* Jupyter Notebook 5.2.x
|
||||
* Conda Python 3.x environment
|
||||
* pyspark, pandas, matplotlib, scipy, seaborn, scikit-learn pre-installed
|
||||
* Spark 2.2.0 with Hadoop 2.7 for use in local mode or to connect to a cluster of Spark workers
|
||||
* Mesos client 1.2 binary that can communicate with a Mesos master
|
||||
* Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda`
|
||||
* [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../base-notebook/start-notebook.sh) as the default command
|
||||
* A [start-singleuser.sh](../base-notebook/start-singleuser.sh) script useful for running a single-user instance of the Notebook server, as required by JupyterHub
|
||||
* `/usr/local/bin/start-notebook.d` directory for custom init scripts that you can add in derived images
|
||||
* A [start.sh](../base-notebook/start.sh) script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
|
||||
* Options for a self-signed HTTPS certificate and passwordless `sudo`
|
||||
|
||||
## Basic Use
|
||||
|
||||
The following command starts a container with the Notebook server listening for HTTP connections on port 8888 with a randomly generated authentication token configured.
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/pyspark-notebook
|
||||
```
|
||||
|
||||
Take note of the authentication token included in the notebook startup log messages. Include it in the URL you visit to access the Notebook server or enter it in the Notebook login form.
|
||||
|
||||
## Using Spark Local Mode
|
||||
|
||||
This configuration is nice for using Spark on small, local data.
|
||||
|
||||
0. Run the container as shown above.
|
||||
2. Open a Python 2 or 3 notebook.
|
||||
3. Create a `SparkContext` configured for local mode.
|
||||
|
||||
For example, the first few cells in the notebook might read:
|
||||
|
||||
```python
|
||||
import pyspark
|
||||
sc = pyspark.SparkContext('local[*]')
|
||||
|
||||
# do something to prove it works
|
||||
rdd = sc.parallelize(range(1000))
|
||||
rdd.takeSample(False, 5)
|
||||
```
|
||||
|
||||
## Connecting to a Spark Cluster on Mesos
|
||||
|
||||
This configuration allows your compute cluster to scale with your data.
|
||||
|
||||
0. [Deploy Spark on Mesos](http://spark.apache.org/docs/latest/running-on-mesos.html).
|
||||
1. Configure each slave with [the `--no-switch_user` flag](https://open.mesosphere.com/reference/mesos-slave/) or create the `jovyan` user on every slave node.
|
||||
2. Ensure Python 2.x and/or 3.x and any Python libraries you wish to use in your Spark lambda functions are installed on your Spark workers.
|
||||
3. Run the Docker container with `--net=host` in a location that is network addressable by all of your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).)
|
||||
* NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details.
|
||||
4. Open a Python 2 or 3 notebook.
|
||||
5. Create a `SparkConf` instance in a new notebook pointing to your Mesos master node (or Zookeeper instance) and Spark binary package location.
|
||||
6. Create a `SparkContext` using this configuration.
|
||||
|
||||
For example, the first few cells in a Python 3 notebook might read:
|
||||
|
||||
```python
|
||||
import os
|
||||
# make sure pyspark tells workers to use python3 not 2 if both are installed
|
||||
os.environ['PYSPARK_PYTHON'] = '/usr/bin/python3'
|
||||
|
||||
import pyspark
|
||||
conf = pyspark.SparkConf()
|
||||
|
||||
# point to mesos master or zookeeper entry (e.g., zk://10.10.10.10:2181/mesos)
|
||||
conf.setMaster("mesos://10.10.10.10:5050")
|
||||
# point to spark binary package in HDFS or on local filesystem on all slave
|
||||
# nodes (e.g., file:///opt/spark/spark-2.2.0-bin-hadoop2.7.tgz)
|
||||
conf.set("spark.executor.uri", "hdfs://10.122.193.209/spark/spark-2.2.0-bin-hadoop2.7.tgz")
|
||||
# set other options as desired
|
||||
conf.set("spark.executor.memory", "8g")
|
||||
conf.set("spark.core.connection.ack.wait.timeout", "1200")
|
||||
|
||||
# create the context
|
||||
sc = pyspark.SparkContext(conf=conf)
|
||||
|
||||
# do something to prove it works
|
||||
rdd = sc.parallelize(range(100000000))
|
||||
rdd.sumApprox(3)
|
||||
```
|
||||
|
||||
To use Python 2 in the notebook and on the workers, change the `PYSPARK_PYTHON` environment variable to point to the location of the Python 2.x interpreter binary. If you leave this environment variable unset, it defaults to `python`.
|
||||
|
||||
Of course, all of this can be hidden in an [IPython kernel startup script](http://ipython.org/ipython-doc/stable/development/config.html?highlight=startup#startup-files), but "explicit is better than implicit." :)
|
||||
|
||||
## Connecting to a Spark Cluster on Standalone Mode
|
||||
|
||||
Connection to Spark Cluster on Standalone Mode requires the following set of steps:
|
||||
|
||||
0. Verify that the docker image (check the Dockerfile) and the Spark Cluster which is being deployed, run the same version of Spark.
|
||||
1. [Deploy Spark on Standalone Mode](http://spark.apache.org/docs/latest/spark-standalone.html).
|
||||
2. Run the Docker container with `--net=host` in a location that is network addressable by all of your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).)
|
||||
* NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details.
|
||||
3. The language specific instructions are almost same as mentioned above for Mesos, only the master url would now be something like spark://10.10.10.10:7077
|
||||
|
||||
## Notebook Options
|
||||
|
||||
The Docker container executes a [`start-notebook.sh` script](../base-notebook/start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
|
||||
|
||||
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/pyspark-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
|
||||
```
|
||||
|
||||
For example, to set the base URL of the notebook server, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/pyspark-notebook start-notebook.sh --NotebookApp.base_url=/some/path
|
||||
```
|
||||
|
||||
For example, to disable all authentication mechanisms (not a recommended practice):
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/pyspark-notebook start-notebook.sh --NotebookApp.token=''
|
||||
```
|
||||
|
||||
You can sidestep the `start-notebook.sh` script and run your own commands in the container. See the *Alternative Commands* section later in this document for more information.
|
||||
|
||||
## Docker Options
|
||||
|
||||
You may customize the execution of the Docker container and the command it is running with the following optional arguments.
|
||||
|
||||
* `-e GEN_CERT=yes` - Generates a self-signed SSL certificate and configures Jupyter Notebook to use it to accept encrypted HTTPS connections.
|
||||
* `-e NB_UID=1000` - Specify the uid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the user id.)
|
||||
* `-e NB_GID=100` - Specify the gid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the group id.)
|
||||
* `-e GRANT_SUDO=yes` - Gives the `jovyan` user passwordless `sudo` capability. Useful for installing OS packages. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adding `jovyan` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
|
||||
* `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
|
||||
|
||||
## SSL Certificates
|
||||
|
||||
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt`:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder:/etc/ssl/notebook \
|
||||
jupyter/pyspark-notebook start-notebook.sh \
|
||||
--NotebookApp.keyfile=/etc/ssl/notebook/notebook.key
|
||||
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
|
||||
```
|
||||
|
||||
Alternatively, you may mount a single PEM file containing both the key and certificate. For example:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \
|
||||
jupyter/pyspark-notebook start-notebook.sh \
|
||||
--NotebookApp.certfile=/etc/ssl/notebook.pem
|
||||
```
|
||||
|
||||
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
|
||||
|
||||
For additional information about using SSL, see the following:
|
||||
|
||||
* The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
|
||||
* The [jupyter_notebook_config.py](jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
|
||||
* The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#using-ssl-for-encrypted-communication) for best practices about running a public notebook server in general, most of which are encoded in this image.
|
||||
|
||||
|
||||
## Conda Environments
|
||||
|
||||
The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/envs.html) resides in `/opt/conda`.
|
||||
|
||||
The commands `jupyter`, `ipython`, `python`, `pip`, and `conda` (among others) are available in both environments. For convenience, you can install packages into either environment regardless of what environment is currently active using commands like the following:
|
||||
|
||||
```
|
||||
# install a package into the default (python 3.x) environment
|
||||
pip install some-package
|
||||
conda install some-package
|
||||
```
|
||||
|
||||
|
||||
## Alternative Commands
|
||||
|
||||
|
||||
### start.sh
|
||||
|
||||
The `start.sh` script supports the same features as the default `start-notebook.sh` script (e.g., `GRANT_SUDO`), but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following:
|
||||
|
||||
```
|
||||
docker run -it --rm jupyter/pyspark-notebook start.sh ipython
|
||||
```
|
||||
|
||||
Or, to run JupyterLab instead of the classic notebook, run the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/pyspark-notebook start.sh jupyter lab
|
||||
```
|
||||
|
||||
This script is particularly useful when you derive a new Dockerfile from this image and install additional Jupyter applications with subcommands like `jupyter console`, `jupyter kernelgateway`, etc.
|
||||
|
||||
### Others
|
||||
|
||||
You can bypass the provided scripts and specify your an arbitrary start command. If you do, keep in mind that certain features documented above will not function (e.g., `GRANT_SUDO`).
|
||||
* [Jupyter Docker Stacks on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
|
||||
* [Selecting an Image :: Core Stacks :: jupyter/pyspark-notebook](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-pyspark-notebook)
|
||||
* [Image Specifics :: Apache Spark](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/specifics.html#apache-spark)
|
||||
|
||||
@@ -2,112 +2,5 @@
|
||||
|
||||
# Jupyter Notebook R Stack
|
||||
|
||||
## What it Gives You
|
||||
|
||||
* Jupyter Notebook 5.2.x
|
||||
* Conda R v3.3.x and channel
|
||||
* plyr, devtools, shiny, rmarkdown, forecast, rsqlite, reshape2, nycflights13, caret, rcurl, and randomforest pre-installed
|
||||
* The [tidyverse](https://github.com/tidyverse/tidyverse) R packages are also installed, including ggplot2, dplyr, tidyr, readr, purrr, tibble, stringr, lubridate, and broom
|
||||
* Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda`
|
||||
* [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../base-notebook/start-notebook.sh) as the default command
|
||||
* `/usr/local/bin/start-notebook.d` directory for custom init scripts that you can add in derived images
|
||||
* A [start-singleuser.sh](../base-notebook/start-singleuser.sh) script useful for running a single-user instance of the Notebook server, as required by JupyterHub
|
||||
* A [start.sh](../base-notebook/start.sh) script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
|
||||
* Options for a self-signed HTTPS certificate and passwordless `sudo`
|
||||
|
||||
## Basic Use
|
||||
|
||||
The following command starts a container with the Notebook server listening for HTTP connections on port 8888 with a randomly generated authentication token configured.
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/r-notebook
|
||||
```
|
||||
|
||||
Take note of the authentication token included in the notebook startup log messages. Include it in the URL you visit to access the Notebook server or enter it in the Notebook login form.
|
||||
|
||||
## Notebook Options
|
||||
|
||||
The Docker container executes a [`start-notebook.sh` script](../base-notebook/start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
|
||||
|
||||
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/r-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
|
||||
```
|
||||
|
||||
For example, to set the base URL of the notebook server, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/r-notebook start-notebook.sh --NotebookApp.base_url=/some/path
|
||||
```
|
||||
|
||||
For example, to disable all authentication mechanisms (not a recommended practice):
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/r-notebook start-notebook.sh --NotebookApp.token=''
|
||||
```
|
||||
|
||||
You can sidestep the `start-notebook.sh` script and run your own commands in the container. See the *Alternative Commands* section later in this document for more information.
|
||||
|
||||
## Docker Options
|
||||
|
||||
You may customize the execution of the Docker container and the command it is running with the following optional arguments.
|
||||
|
||||
* `-e GEN_CERT=yes` - Generates a self-signed SSL certificate and configures Jupyter Notebook to use it to accept encrypted HTTPS connections.
|
||||
* `-e NB_UID=1000` - Specify the uid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the user id.)
|
||||
* `-e NB_GID=100` - Specify the gid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the group id.)
|
||||
* `-e GRANT_SUDO=yes` - Gives the `jovyan` user passwordless `sudo` capability. Useful for installing OS packages. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adding `jovyan` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
|
||||
* `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
|
||||
|
||||
## SSL Certificates
|
||||
|
||||
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt`:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder:/etc/ssl/notebook \
|
||||
jupyter/r-notebook start-notebook.sh \
|
||||
--NotebookApp.keyfile=/etc/ssl/notebook/notebook.key
|
||||
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
|
||||
```
|
||||
|
||||
Alternatively, you may mount a single PEM file containing both the key and certificate. For example:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \
|
||||
jupyter/r-notebook start-notebook.sh \
|
||||
--NotebookApp.certfile=/etc/ssl/notebook.pem
|
||||
```
|
||||
|
||||
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
|
||||
|
||||
For additional information about using SSL, see the following:
|
||||
|
||||
* The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
|
||||
* The [jupyter_notebook_config.py](jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
|
||||
* The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#using-ssl-for-encrypted-communication) for best practices about running a public notebook server in general, most of which are encoded in this image.
|
||||
|
||||
|
||||
## Alternative Commands
|
||||
|
||||
|
||||
### start.sh
|
||||
|
||||
The `start.sh` script supports the same features as the default `start-notebook.sh` script (e.g., `GRANT_SUDO`), but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following:
|
||||
|
||||
```
|
||||
docker run -it --rm jupyter/r-notebook start.sh ipython
|
||||
```
|
||||
|
||||
Or, to run JupyterLab instead of the classic notebook, run the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/r-notebook start.sh jupyter lab
|
||||
```
|
||||
|
||||
This script is particularly useful when you derive a new Dockerfile from this image and install additional Jupyter applications with subcommands like `jupyter console`, `jupyter kernelgateway`, etc.
|
||||
|
||||
### Others
|
||||
|
||||
You can bypass the provided scripts and specify your an arbitrary start command. If you do, keep in mind that certain features documented above will not function (e.g., `GRANT_SUDO`).
|
||||
* [Jupyter Docker Stacks on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
|
||||
* [Selecting an Image :: Core Stacks :: jupyter/r-notebook](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-r-notebook)
|
||||
|
||||
@@ -2,125 +2,7 @@
|
||||
|
||||
# Jupyter Notebook Scientific Python Stack
|
||||
|
||||
## What it Gives You
|
||||
Please visit the documentation site for help using and contributing to this image and others.
|
||||
|
||||
* Jupyter Notebook 5.2.x
|
||||
* Conda Python 3.x environment
|
||||
* pandas, matplotlib, scipy, seaborn, scikit-learn, scikit-image, sympy, cython, patsy, statsmodel, cloudpickle, dill, numba, bokeh, vincent, beautifulsoup, xlrd pre-installed
|
||||
* Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda`
|
||||
* [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../base-notebook/start-notebook.sh) as the default command
|
||||
* `/usr/local/bin/start-notebook.d` directory for custom init scripts that you can add in derived images
|
||||
* A [start-singleuser.sh](../base-notebook/start-singleuser.sh) script useful for running a single-user instance of the Notebook server, as required by JupyterHub
|
||||
* A [start.sh](../base-notebook/start.sh) script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
|
||||
* Options for HTTPS, password auth, and passwordless `sudo`
|
||||
|
||||
## Basic Use
|
||||
|
||||
The following command starts a container with the Notebook server listening for HTTP connections on port 8888 with a randomly generated authentication token configured.
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/scipy-notebook
|
||||
```
|
||||
|
||||
Take note of the authentication token included in the notebook startup log messages. Include it in the URL you visit to access the Notebook server or enter it in the Notebook login form.
|
||||
|
||||
## Notebook Options
|
||||
|
||||
The Docker container executes a [`start-notebook.sh` script](../base-notebook/start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
|
||||
|
||||
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/scipy-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
|
||||
```
|
||||
|
||||
For example, to set the base URL of the notebook server, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/scipy-notebook start-notebook.sh --NotebookApp.base_url=/some/path
|
||||
```
|
||||
|
||||
For example, to disable all authentication mechanisms (not a recommended practice):
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/scipy-notebook start-notebook.sh --NotebookApp.token=''
|
||||
```
|
||||
|
||||
You can sidestep the `start-notebook.sh` script and run your own commands in the container. See the *Alternative Commands* section later in this document for more information.
|
||||
|
||||
|
||||
## Docker Options
|
||||
|
||||
You may customize the execution of the Docker container and the Notebook server it contains with the following optional arguments.
|
||||
|
||||
* `-e GEN_CERT=yes` - Generates a self-signed SSL certificate and configures Jupyter Notebook to use it to accept encrypted HTTPS connections.
|
||||
* `-e NB_UID=1000` - Specify the uid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the user id.)
|
||||
* `-e NB_GID=100` - Specify the gid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the group id.)
|
||||
* `-e GRANT_SUDO=yes` - Gives the `jovyan` user passwordless `sudo` capability. Useful for installing OS packages. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adding `jovyan` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
|
||||
* `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
|
||||
|
||||
## SSL Certificates
|
||||
|
||||
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt`:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder:/etc/ssl/notebook \
|
||||
jupyter/scipy-notebook start-notebook.sh \
|
||||
--NotebookApp.keyfile=/etc/ssl/notebook/notebook.key
|
||||
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
|
||||
```
|
||||
|
||||
Alternatively, you may mount a single PEM file containing both the key and certificate. For example:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \
|
||||
jupyter/scipy-notebook start-notebook.sh \
|
||||
--NotebookApp.certfile=/etc/ssl/notebook.pem
|
||||
```
|
||||
|
||||
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
|
||||
|
||||
For additional information about using SSL, see the following:
|
||||
|
||||
* The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
|
||||
* The [jupyter_notebook_config.py](jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
|
||||
* The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#using-ssl-for-encrypted-communication) for best practices about running a public notebook server in general, most of which are encoded in this image.
|
||||
|
||||
|
||||
## Conda Environments
|
||||
|
||||
The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/envs.html) resides in `/opt/conda`.
|
||||
|
||||
The commands `jupyter`, `ipython`, `python`, `pip`, and `conda` (among others) are available in both environments. For convenience, you can install packages into either environment regardless of what environment is currently active using commands like the following:
|
||||
|
||||
```
|
||||
# install a package into the default (python 3.x) environment
|
||||
pip install some-package
|
||||
conda install some-package
|
||||
```
|
||||
|
||||
|
||||
## Alternative Commands
|
||||
|
||||
|
||||
### start.sh
|
||||
|
||||
The `start.sh` script supports the same features as the default `start-notebook.sh` script (e.g., `GRANT_SUDO`), but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following:
|
||||
|
||||
```
|
||||
docker run -it --rm jupyter/scipy-notebook start.sh ipython
|
||||
```
|
||||
|
||||
Or, to run JupyterLab instead of the classic notebook, run the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/scipy-notebook start.sh jupyter lab
|
||||
```
|
||||
|
||||
This script is particularly useful when you derive a new Dockerfile from this image and install additional Jupyter applications with subcommands like `jupyter console`, `jupyter kernelgateway`, etc.
|
||||
|
||||
### Others
|
||||
|
||||
You can bypass the provided scripts and specify your an arbitrary start command. If you do, keep in mind that certain features documented above will not function (e.g., `GRANT_SUDO`).
|
||||
* [Jupyter Docker Stacks on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
|
||||
* [Selecting an Image :: Core Stacks :: jupyter/scipy-notebook](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-scipy-notebook)
|
||||
|
||||
@@ -1,150 +1,9 @@
|
||||
  [](https://microbadger.com/images/jupyter/tensorflow-notebook "jupyter/tensorflow-notebook image metadata")
|
||||
|
||||
# Jupyter Notebook Scientific Python Stack + Tensorflow
|
||||
# Jupyter Notebook Deep Learning Stack
|
||||
|
||||
## What it Gives You
|
||||
Please visit the documentation site for help using and contributing to this image and others.
|
||||
|
||||
* Everything in [Scipy Notebook](https://github.com/jupyter/docker-stacks/tree/master/scipy-notebook)
|
||||
* Tensorflow and Keras for Python 3.x (without GPU support)
|
||||
|
||||
## Basic Use
|
||||
|
||||
The following command starts a container with the Notebook server listening for HTTP connections on port 8888 with a randomly generated authentication token configured.
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/tensorflow-notebook
|
||||
```
|
||||
|
||||
Take note of the authentication token included in the notebook startup log messages. Include it in the URL you visit to access the Notebook server or enter it in the Notebook login form.
|
||||
|
||||
## Tensorflow Machine Mode
|
||||
|
||||
Tensorflow can use single machine, or distributed mode.
|
||||
|
||||
Single machine mode:
|
||||
|
||||
```
|
||||
import tensorflow as tf
|
||||
|
||||
hello = tf.Variable('Hello World!')
|
||||
|
||||
sess = tf.Session()
|
||||
init = tf.global_variables_initializer()
|
||||
|
||||
sess.run(init)
|
||||
sess.run(hello)
|
||||
```
|
||||
|
||||
Distributed mode:
|
||||
|
||||
```
|
||||
import tensorflow as tf
|
||||
|
||||
hello = tf.Variable('Hello Distributed World!')
|
||||
|
||||
server = tf.train.Server.create_local_server()
|
||||
sess = tf.Session(server.target)
|
||||
init = tf.global_variables_initializer()
|
||||
|
||||
sess.run(init)
|
||||
sess.run(hello)
|
||||
```
|
||||
|
||||
## Notebook Options
|
||||
|
||||
The Docker container executes a [`start-notebook.sh` script](../base-notebook/start-notebook.sh) script by default. The `start-notebook.sh` script handles the `NB_UID`, `NB_GID` and `GRANT_SUDO` features documented in the next section, and then executes the `jupyter notebook`.
|
||||
|
||||
You can pass [Jupyter command line options](https://jupyter.readthedocs.io/en/latest/projects/jupyter-command.html) through the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/tensorflow-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
|
||||
```
|
||||
|
||||
For example, to set the base URL of the notebook server, run the following:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/tensorflow-notebook start-notebook.sh --NotebookApp.base_url=/some/path
|
||||
```
|
||||
|
||||
For example, to disable all authentication mechanisms (not a recommended practice):
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 jupyter/tensorflow-notebook start-notebook.sh --NotebookApp.token=''
|
||||
```
|
||||
|
||||
You can sidestep the `start-notebook.sh` script and run your own commands in the container. See the *Alternative Commands* section later in this document for more information.
|
||||
|
||||
## Docker Options
|
||||
|
||||
You may customize the execution of the Docker container and the command it is running with the following optional arguments.
|
||||
|
||||
* `-e GEN_CERT=yes` - Generates a self-signed SSL certificate and configures Jupyter Notebook to use it to accept encrypted HTTPS connections.
|
||||
* `-e NB_UID=1000` - Specify the uid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the user id.)
|
||||
* `-e NB_GID=100` - Specify the gid of the `jovyan` user. Useful to mount host volumes with specific file ownership. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adjusting the group id.)
|
||||
* `-e GRANT_SUDO=yes` - Gives the `jovyan` user passwordless `sudo` capability. Useful for installing OS packages. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su jovyan` after adding `jovyan` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
|
||||
* `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
|
||||
|
||||
## SSL Certificates
|
||||
|
||||
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt`:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder:/etc/ssl/notebook \
|
||||
jupyter/tensorflow-notebook start-notebook.sh \
|
||||
--NotebookApp.keyfile=/etc/ssl/notebook/notebook.key
|
||||
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
|
||||
```
|
||||
|
||||
Alternatively, you may mount a single PEM file containing both the key and certificate. For example:
|
||||
|
||||
```
|
||||
docker run -d -p 8888:8888 \
|
||||
-v /some/host/folder/notebook.pem:/etc/ssl/notebook.pem \
|
||||
jupyter/tensorflow-notebook start-notebook.sh \
|
||||
--NotebookApp.certfile=/etc/ssl/notebook.pem
|
||||
```
|
||||
|
||||
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
|
||||
|
||||
For additional information about using SSL, see the following:
|
||||
|
||||
* The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
|
||||
* The [jupyter_notebook_config.py](jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
|
||||
* The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#using-ssl-for-encrypted-communication) for best practices about running a public notebook server in general, most of which are encoded in this image.
|
||||
|
||||
|
||||
## Conda Environments
|
||||
|
||||
The default Python 3.x [Conda environment](http://conda.pydata.org/docs/using/envs.html) resides in `/opt/conda`.
|
||||
|
||||
The commands `jupyter`, `ipython`, `python`, `pip`, and `conda` (among others) are available in both environments. For convenience, you can install packages into either environment regardless of what environment is currently active using commands like the following:
|
||||
|
||||
```
|
||||
# install a package into the default (python 3.x) environment
|
||||
pip install some-package
|
||||
conda install some-package
|
||||
```
|
||||
|
||||
|
||||
## Alternative Commands
|
||||
|
||||
### start.sh
|
||||
|
||||
The `start.sh` script supports the same features as the default `start-notebook.sh` script (e.g., `GRANT_SUDO`), but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following:
|
||||
|
||||
```
|
||||
docker run -it --rm jupyter/tensorflow-notebook start.sh ipython
|
||||
```
|
||||
|
||||
Or, to run JupyterLab instead of the classic notebook, run the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -p 8888:8888 jupyter/tensorflow-notebook start.sh jupyter lab
|
||||
```
|
||||
|
||||
This script is particularly useful when you derive a new Dockerfile from this image and install additional Jupyter applications with subcommands like `jupyter console`, `jupyter kernelgateway`, etc.
|
||||
|
||||
### Others
|
||||
|
||||
You can bypass the provided scripts and specify your an arbitrary start command. If you do, keep in mind that certain features documented above will not function (e.g., `GRANT_SUDO`).
|
||||
* [Jupyter Docker Stacks on ReadTheDocs](http://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
|
||||
* [Selecting an Image :: Core Stacks :: jupyter/tensorflow-notebook](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-tensorflow-notebook)
|
||||
* [Image Specifics :: Tensorflow](http://jupyter-docker-stacks.readthedocs.io/en/latest/using/specifics.html#tensorflow)
|
||||
|
||||
Reference in New Issue
Block a user