hazza/docker-stacks
1
0
Fork 0
mirror of https://github.com/jupyter/docker-stacks.git synced 2025-10-08 10:34:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Limit markdown line length to 200

Browse Source
This commit is contained in:
Ayaz Salikhov
2021-06-21 12:59:21 +03:00
parent 34992fea4c
commit 334381dfe2
22 changed files with 454 additions and 293 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.github/ISSUE_TEMPLATE/feature_request.md vendored
View File

@@ -20,4 +20,7 @@ Example: Add a package [altair](https://altair-viz.github.io).
**How does this change will affect users?** **How does this change will affect users?**
Example: Altair is a declarative statistical visualization library for Python, based on Vega and Vega-Lite, and the source is available on GitHub. With Altair, you can spend more time understanding your data and its meaning. Altair’s API is simple, friendly and consistent and built on top of the powerful Vega-Lite visualization grammar. This elegant simplicity produces beautiful and effective visualizations with a minimal amount of code. Example: Altair is a declarative statistical visualization library for Python, based on Vega and Vega-Lite, and the source is available on GitHub.
With Altair, you can spend more time understanding your data and its meaning.
Altair’s API is simple, friendly and consistent and built on top of the powerful Vega-Lite visualization grammar.
This elegant simplicity produces beautiful and effective visualizations with a minimal amount of code.

2
.markdownlint.yaml
View File

@@ -4,4 +4,4 @@ default: true
# MD013/line-length - Line length # MD013/line-length - Line length
MD013: MD013:
# Number of characters # Number of characters
line_length: 1000 line_length: 200

78
README.md
View File

@@ -12,44 +12,45 @@ containing Jupyter applications and interactive computing tools.
## Quick Start ## Quick Start
You can try a You can try a [relatively recent build of the jupyter/base-notebook image on mybinder.org](https://mybinder.org/v2/gh/jupyter/docker-stacks/master?filepath=README.ipynb)
[relatively recent build of the jupyter/base-notebook image on mybinder.org](https://mybinder.org/v2/gh/jupyter/docker-stacks/master?filepath=README.ipynb) by simply clicking the preceding link.
by simply clicking the preceding link. The image used in binder was last updated on 22 May 2021. The image used in binder was last updated on 22 May 2021.
Otherwise, the two examples below may help you get started if Otherwise, the two examples below may help you get started if you [have Docker installed](https://docs.docker.com/install/),
you [have Docker installed](https://docs.docker.com/install/) know know [which Docker image](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html) you want to use,
[which Docker image](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html) you and want to launch a single Jupyter Notebook server in a container.
want to use, and want to launch a single Jupyter Notebook server in a container.
The [User Guide on ReadTheDocs](https://jupyter-docker-stacks.readthedocs.io/) describes additional The [User Guide on ReadTheDocs](https://jupyter-docker-stacks.readthedocs.io/) describes additional uses and features in detail.
uses and features in detail.
**Example 1:** This command pulls the `jupyter/scipy-notebook` image tagged `33add21fab64` from **Example 1:** This command pulls the `jupyter/scipy-notebook` image tagged `33add21fab64` from Docker Hub if it is not already present on the local host.
Docker Hub if it is not already present on the local host. It then starts a container running a It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888.
Jupyter Notebook server and exposes the server on host port 8888. The server logs appear in the The server logs appear in the terminal.
terminal. Visiting `http://<hostname>:8888/?token=<token>` in a browser loads the Jupyter Notebook Visiting `http://<hostname>:8888/?token=<token>` in a browser loads the Jupyter Notebook dashboard page,
dashboard page, where `hostname` is the name of the computer running docker and `token` is the where `hostname` is the name of the computer running docker and `token` is the secret token printed in the console.
secret token printed in the console. The container remains intact for restart after the notebook The container remains intact for restart after the notebook server exits.
server exits.
```bash
docker run -p 8888:8888 jupyter/scipy-notebook:33add21fab64 docker run -p 8888:8888 jupyter/scipy-notebook:33add21fab64
```
**Example 2:** This command performs the same operations as **Example 1**, but it exposes the server **Example 2:** This command performs the same operations as **Example 1**, but it exposes the server on host port 10000 instead of port 8888.
on host port 10000 instead of port 8888. Visiting `http://<hostname>:10000/?token=<token>` in a Visiting `http://<hostname>:10000/?token=<token>` in a browser loads JupyterLab,
browser loads JupyterLab, where `hostname` is the name of the computer running docker and `token` is where `hostname` is the name of the computer running docker and `token` is the secret token printed in the console.
the secret token printed in the console.::
```bash
docker run -p 10000:8888 jupyter/scipy-notebook:33add21fab64 docker run -p 10000:8888 jupyter/scipy-notebook:33add21fab64
```
**Example 3:** This command pulls the `jupyter/datascience-notebook` image tagged `33add21fab64` **Example 3:** This command pulls the `jupyter/datascience-notebook` image tagged `33add21fab64` from Docker Hub if it is not already present on the local host.
from Docker Hub if it is not already present on the local host. It then starts an _ephemeral_ It then starts an _ephemeral_ container running a Jupyter Notebook server and exposes the server on host port 10000.
container running a Jupyter Notebook server and exposes the server on host port 10000. The command The command mounts the current working directory on the host as `/home/jovyan/work` in the container.
mounts the current working directory on the host as `/home/jovyan/work` in the container. The server The server logs appear in the terminal.
logs appear in the terminal. Visiting `http://<hostname>:10000/?token=<token>` in a browser loads Visiting `http://<hostname>:10000/?token=<token>` in a browser loads JupyterLab,
JupyterLab, where `hostname` is the name of the computer running docker and `token` is the secret where `hostname` is the name of the computer running docker and `token` is the secret token printed in the console.
token printed in the console. Docker destroys the container after notebook server exit, but any Docker destroys the container after notebook server exit, but any files written to `~/work` in the container remain intact on the host.
files written to `~/work` in the container remain intact on the host.
```bash
docker run --rm -p 10000:8888 -e JUPYTER_ENABLE_LAB=yes -v "${PWD}":/home/jovyan/work jupyter/datascience-notebook:33add21fab64 docker run --rm -p 10000:8888 -e JUPYTER_ENABLE_LAB=yes -v "${PWD}":/home/jovyan/work jupyter/datascience-notebook:33add21fab64
```
## Contributing ## Contributing
@@ -59,13 +60,12 @@ maintained stacks.
## Maintainer Help Wanted ## Maintainer Help Wanted
We value all positive contributions to the Docker stacks project, from We value all positive contributions to the Docker stacks project,
[bug reports](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/issues.html) to from [bug reports](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/issues.html)
[pull requests](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/packages.html) to [pull requests](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/packages.html)
to to [translations](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/translations.html)
[translations](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/translations.html) to help answering questions.
to help answering questions. We'd also like to invite members of the community to help with two We'd also like to invite members of the community to help with two maintainer activities:
maintainer activities:
- Issue triage: Reading and providing a first response to issues, labeling issues appropriately, - Issue triage: Reading and providing a first response to issues, labeling issues appropriately,
redirecting cross-project questions to Jupyter Discourse redirecting cross-project questions to Jupyter Discourse
@@ -73,9 +73,8 @@ maintainer activities:
to improve the contribution, deciding if the contribution should take another form (e.g., a recipe to improve the contribution, deciding if the contribution should take another form (e.g., a recipe
instead of a permanent change to the images) instead of a permanent change to the images)
Anyone in the community can jump in and help with these activities at any time. We will happily Anyone in the community can jump in and help with these activities at any time
grant additional permissions (e.g., ability to merge PRs) to anyone who shows an on-going interest We will happily grant additional permissions (e.g., ability to merge PRs) to anyone who shows an on-going interest in working on the project.
in working on the project.
## Jupyter Notebook Deprecation Notice ## Jupyter Notebook Deprecation Notice
@@ -85,7 +84,8 @@ more information is available in the [documentation](https://jupyter-docker-stac
At some point, JupyterLab will become the default for all of the Jupyter Docker stack images, however a new environment variable will be introduced to switch back to Jupyter Notebook if needed. At some point, JupyterLab will become the default for all of the Jupyter Docker stack images, however a new environment variable will be introduced to switch back to Jupyter Notebook if needed.
After the change of default, and according to the Jupyter Notebook project status and its compatibility with JupyterLab, these Docker images may remove the classic Jupyter Notebook interface altogether in favor of another _classic-like_ UI built atop JupyterLab. After the change of default, and according to the Jupyter Notebook project status and its compatibility with JupyterLab,
these Docker images may remove the classic Jupyter Notebook interface altogether in favor of another _classic-like_ UI built atop JupyterLab.
This change is tracked in the issue [#1217](https://github.com/jupyter/docker-stacks/issues/1217), please check its content for more information. This change is tracked in the issue [#1217](https://github.com/jupyter/docker-stacks/issues/1217), please check its content for more information.

9
docs/contributing/features.md
View File

@@ -37,15 +37,14 @@ Roughly speaking, we evaluate new features based on the following criteria:
If there's agreement that the feature belongs in one or more of the core stacks: If there's agreement that the feature belongs in one or more of the core stacks:
1. Implement the feature in a local clone of the `jupyter/docker-stacks` project. 1. Implement the feature in a local clone of the `jupyter/docker-stacks` project.
2. Please build the image locally before submitting a pull request. Building the image locally 2. Please build the image locally before submitting a pull request
shortens the debugging cycle by taking some load off GitHub Actions, which graciously provide Building the image locally shortens the debugging cycle by taking some load off GitHub Actions, which graciously provide free build services for open source projects like this one.
free build services for open source projects like this one. If you use `make`, call: If you use `make`, call:
```bash ```bash
make build/somestack-notebook make build/somestack-notebook
``` ```
3. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) 3. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)(PR) with your changes.
(PR) with your changes.
4. Watch for GitHub to report a build success or failure for your PR on GitHub. 4. Watch for GitHub to report a build success or failure for your PR on GitHub.
5. Discuss changes with the maintainers and address any build issues. 5. Discuss changes with the maintainers and address any build issues.

34
docs/contributing/issues.md
View File

@@ -1,25 +1,17 @@
# Project Issues # Project Issues
We appreciate your taking the time to report an issue you encountered using the We appreciate your taking the time to report an issue you encountered using the Jupyter Docker Stacks.
Jupyter Docker Stacks. Please review the following guidelines when reporting Please review the following guidelines when reporting your problem.
your problem.
- If you believe you’ve found a security vulnerability in any of the Jupyter - If you believe you’ve found a security vulnerability in any of the Jupyter projects included in Jupyter Docker Stacks images,
projects included in Jupyter Docker Stacks images, please report it to please report it to [security@ipython.org](mailto:security@iypthon.org), not in the issue trackers on GitHub.
[security@ipython.org](mailto:security@iypthon.org), not in the issue trackers If you prefer to encrypt your security reports, you can use [this PGP public key](https://github.com/jupyter/jupyter.github.io/blob/master/assets/ipython_security.asc).
on GitHub. If you prefer to encrypt your security reports, you can use [this
PGP public
key](https://github.com/jupyter/jupyter.github.io/blob/master/assets/ipython_security.asc).
- If you think your problem is unique to the Jupyter Docker Stacks images, - If you think your problem is unique to the Jupyter Docker Stacks images,
please search the [jupyter/docker-stacks issue please search the [jupyter/docker-stacks issue tracker](https://github.com/jupyter/docker-stacks/issues)
tracker](https://github.com/jupyter/docker-stacks/issues) to see if someone to see if someone else has already reported the same problem.
else has already reported the same problem. If not, please open a [new If not, please open a [new issue](https://github.com/jupyter/docker-stacks/issues/new) and provide all of the information requested in the issue template.
issue](https://github.com/jupyter/docker-stacks/issues/new) and provide all of - If the issue you're seeing is with one of the open source libraries included in the Docker images and is reproducible outside the images,
the information requested in the issue template. please file a bug with the appropriate open source project.
- If the issue you're seeing is with one of the open source libraries included - If you have a general question about how to use the Jupyter Docker Stacks in your environment,
in the Docker images and is reproducible outside the images, please file a bug in conjunction with other tools, with customizations, and so on,
with the appropriate open source project. please post your question on the [Jupyter Discourse site](https://discourse.jupyter.org).
- If you have a general question about how to use the Jupyter Docker Stacks in
your environment, in conjunction with other tools, with customizations, and so
on, please post your question on the [Jupyter Discourse
site](https://discourse.jupyter.org).

4
docs/contributing/lint.md
View File

@@ -51,7 +51,9 @@ The following rules are ignored by default for all images in the `.hadolint.yaml
For other rules, the preferred way to do it is to flag ignored rules in the `Dockerfile`. For other rules, the preferred way to do it is to flag ignored rules in the `Dockerfile`.
> It is also possible to ignore rules by using a special comment directly above the Dockerfile instruction you want to make an exception for. Ignore rule comments look like `# hadolint ignore=DL3001,SC1081`. For example: > It is also possible to ignore rules by using a special comment directly above the Dockerfile instruction you want to make an exception for.
> Ignore rule comments look like `# hadolint ignore=DL3001,SC1081`.
> For example:
```dockerfile ```dockerfile
FROM ubuntu FROM ubuntu

21
docs/contributing/packages.md
View File

@@ -9,13 +9,13 @@ Please follow the process below to update a package version:
1. Locate the Dockerfile containing the library you wish to update (e.g., 1. Locate the Dockerfile containing the library you wish to update (e.g.,
[base-notebook/Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile), [base-notebook/Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile),
[scipy-notebook/Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/scipy-notebook/Dockerfile)) [scipy-notebook/Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/scipy-notebook/Dockerfile))
2. Adjust the version number for the package. We prefer to pin the major and minor version number of 2. Adjust the version number for the package.
packages so as to minimize rebuild side-effects when users submit pull requests (PRs). For We prefer to pin the major and minor version number of packages so as to minimize rebuild side-effects when users submit pull requests (PRs).
example, you'll find the Jupyter Notebook package, `notebook`, installed using conda with For example, you'll find the Jupyter Notebook package, `notebook`, installed using conda with
`notebook=5.4.*`. `notebook=5.4.*`.
3. Please build the image locally before submitting a pull request. Building the image locally 3. Please build the image locally before submitting a pull request.
shortens the debugging cycle by taking some load off GitHub Actions, which graciously provide Building the image locally shortens the debugging cycle by taking some load off GitHub Actions, which graciously provide free build services for open source projects like this one.
free build services for open source projects like this one. If you use `make`, call: If you use `make`, call:
```bash ```bash
make build/somestack-notebook make build/somestack-notebook
@@ -24,13 +24,14 @@ Please follow the process below to update a package version:
4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) 4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)
(PR) with your changes. (PR) with your changes.
5. Watch for GitHub to report a build success or failure for your PR on GitHub. 5. Watch for GitHub to report a build success or failure for your PR on GitHub.
6. Discuss changes with the maintainers and address any build issues. Version conflicts are the most 6. Discuss changes with the maintainers and address any build issues.
common problem. You may need to upgrade additional packages to fix build failures. Version conflicts are the most common problem.
You may need to upgrade additional packages to fix build failures.
## Notes ## Notes
In order to help identifying packages that can be updated you can use the following helper tool. It In order to help identifying packages that can be updated you can use the following helper tool.
will list all the packages installed in the `Dockerfile` that can be updated -- dependencies are It will list all the packages installed in the `Dockerfile` that can be updated -- dependencies are
filtered to focus only on requested packages. filtered to focus only on requested packages.
```bash ```bash

6
docs/contributing/recipes.md
View File

@@ -1,8 +1,10 @@
# New Recipes # New Recipes
We welcome contributions of [recipes](../using/recipes.md), short examples of using, configuring, or extending the Docker Stacks, for inclusion in the documentation site. Follow the process below to add a new recipe: We welcome contributions of [recipes](../using/recipes.md), short examples of using, configuring, or extending the Docker Stacks, for inclusion in the documentation site.
Follow the process below to add a new recipe:
1. Open the `docs/using/recipes.md` source file. 1. Open the `docs/using/recipes.md` source file.
2. Add a second-level Markdown heading naming your recipe at the bottom of the file (e.g., `## Add the RISE extension`) 2. Add a second-level Markdown heading naming your recipe at the bottom of the file (e.g., `## Add the RISE extension`)
3. Write the body of your recipe under the heading, including whatever command line, Dockerfile, links, etc. you need. 3. Write the body of your recipe under the heading, including whatever command line, Dockerfile, links, etc. you need.
4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) (PR) with your changes. Maintainers will respond and work with you to address any formatting or content issues. 4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) (PR) with your changes.
Maintainers will respond and work with you to address any formatting or content issues.

58
docs/contributing/stacks.md
View File

@@ -1,19 +1,17 @@
# Community Stacks # Community Stacks
We love to see the community create and share new Jupyter Docker images. We've put together a We love to see the community create and share new Jupyter Docker images.
[cookiecutter project](https://github.com/jupyter/cookiecutter-docker-stacks) and the documentation We've put together a [cookiecutter project](https://github.com/jupyter/cookiecutter-docker-stacks)
below to help you get started defining, building, and sharing your Jupyter environments in Docker. and the documentation below to help you get started defining, building, and sharing your Jupyter environments in Docker.
Following these steps will: Following these steps will:
1. Setup a project on GitHub containing a Dockerfile based on either the `jupyter/base-notebook` or 1. Setup a project on GitHub containing a Dockerfile based on either the `jupyter/base-notebook` or `jupyter/minimal-notebook` image.
`jupyter/minimal-notebook` image. 2. Configure GitHub Actions to build and test your image when users submit pull requests to your repository.
2. Configure GitHub Actions to build and test your image when users submit pull requests to your
repository.
3. Configure Docker Hub to build and host your images for others to use. 3. Configure Docker Hub to build and host your images for others to use.
4. Update the [list of community stacks](../using/selecting.md#community-stacks) in this documentation to include your image. 4. Update the [list of community stacks](../using/selecting.md#community-stacks) in this documentation to include your image.
This approach mirrors how we build and share the core stack images. Feel free to follow it or pave This approach mirrors how we build and share the core stack images.
your own path using alternative services and build tools. Feel free to follow it or pave your own path using alternative services and build tools.
## Creating a Project ## Creating a Project
@@ -23,31 +21,27 @@ First, install [cookiecutter](https://github.com/cookiecutter/cookiecutter) usin
pip install cookiecutter # or conda install cookiecutter pip install cookiecutter # or conda install cookiecutter
``` ```
Run the cookiecutter command pointing to the Run the cookiecutter command pointing to the [jupyter/cookiecutter-docker-stacks](https://github.com/jupyter/cookiecutter-docker-stacks) project on GitHub.
[jupyter/cookiecutter-docker-stacks](https://github.com/jupyter/cookiecutter-docker-stacks) project
on GitHub.
```bash ```bash
cookiecutter https://github.com/jupyter/cookiecutter-docker-stacks.git cookiecutter https://github.com/jupyter/cookiecutter-docker-stacks.git
``` ```
Enter a name for your new stack image. This will serve as both the git repository name and the part Enter a name for your new stack image.
of the Docker image name after the slash. This will serve as both the git repository name and the part of the Docker image name after the slash.
```text ```text
stack_name [my-jupyter-stack]: stack_name [my-jupyter-stack]:
``` ```
Enter the user or organization name under which this stack will reside on Docker Hub. You Enter the user or organization name under which this stack will reside on Docker Hub.
must have access to manage this Docker Hub organization to push images here and set up automated You must have access to manage this Docker Hub organization to push images here and set up automated builds.
builds.
```text ```text
stack_org [my-project]: stack_org [my-project]:
``` ```
Select an image from the jupyter/docker-stacks project that will serve as the base for your new Select an image from the jupyter/docker-stacks project that will serve as the base for your new image.
image.
```text ```text
stack_base_image [jupyter/base-notebook]: stack_base_image [jupyter/base-notebook]:
@@ -90,13 +84,15 @@ The cookiecutter template comes with a `.github/workflows/docker.yml` file, whic
- "*.md" - "*.md"
``` ```
This will trigger the CI pipeline whenever you push to your `main` or `master` branch and when any Pull Requests are made to your repository. For more details on this configuration, visit the [GitHub actions documentation on triggers](https://docs.github.com/en/actions/reference/events-that-trigger-workflows). This will trigger the CI pipeline whenever you push to your `main` or `master` branch and when any Pull Requests are made to your repository.
For more details on this configuration, visit the [GitHub actions documentation on triggers](https://docs.github.com/en/actions/reference/events-that-trigger-workflows).
2. Commit your changes and push to GitHub. 2. Commit your changes and push to GitHub.
3. Head back to your repository and click on the **Actions** tab. 3. Head back to your repository and click on the **Actions** tab.
![GitHub actions tab screenshot](../_static/github-actions-tab.png) ![GitHub actions tab screenshot](../_static/github-actions-tab.png)
From there, you can click on the workflows on the left-hand side of the screen. From there, you can click on the workflows on the left-hand side of the screen.
4. In the next screen, you will be able to see information about the workflow run and duration. If you click again on the button with the workflow name, you will see the logs for the workflow steps. 4. In the next screen, you will be able to see information about the workflow run and duration.
If you click again on the button with the workflow name, you will see the logs for the workflow steps.
![Github actions workflow run screenshot](../_static/github-actions-workflow.png) ![Github actions workflow run screenshot](../_static/github-actions-workflow.png)
## Configuring Docker Hub ## Configuring Docker Hub
@@ -121,25 +117,29 @@ you merge a GitHub pull request to the master branch of your project.
![Docker account Security settings screenshot](../_static/docker-org-security.png) ![Docker account Security settings screenshot](../_static/docker-org-security.png)
11. Enter a meaningful name for your token and click on **Create** 11. Enter a meaningful name for your token and click on **Create**
![Docker account create new token screenshot](../_static/docker-org-create-token.png) ![Docker account create new token screenshot](../_static/docker-org-create-token.png)
12. Copy the personal access token displayed on the next screen. **Note that you will not be able to see it again after you close the pop-up window**. 12. Copy the personal access token displayed on the next screen.
**Note that you will not be able to see it again after you close the pop-up window**.
13. Head back to your GitHub repository and click on the **Settings tab**. 13. Head back to your GitHub repository and click on the **Settings tab**.
![Github repository settings tab screenshot](../_static/github-create-secrets.png) ![Github repository settings tab screenshot](../_static/github-create-secrets.png)
14. Click on the **Secrets** section and then on the **New repository secret** button on the top right corner (see image above). 14. Click on the **Secrets** section and then on the **New repository secret** button on the top right corner (see image above).
15. Create a **DOCKERHUB_TOKEN** secret and paste the Personal Access Token from DockerHub in the **value** field. 15. Create a **DOCKERHUB_TOKEN** secret and paste the Personal Access Token from DockerHub in the **value** field.
![GitHub create secret token screenshot](../_static/github-secret-token.png) ![GitHub create secret token screenshot](../_static/github-secret-token.png)
16. Repeat the above step but creating a **DOCKERHUB_USERNAME** and replacing the _value_ field with your DockerHub username. Once you have completed these steps, your repository secrets section should look something like this: 16. Repeat the above step but creating a **DOCKERHUB_USERNAME** and replacing the _value_ field with your DockerHub username.
Once you have completed these steps, your repository secrets section should look something like this:
![GitHub repository secrets created screenshot](../_static/github-secrets-completed.png) ![GitHub repository secrets created screenshot](../_static/github-secrets-completed.png)
## Defining Your Image ## Defining Your Image
Make edits to the Dockerfile in your project to add third-party libraries and configure Jupyter Make edits to the Dockerfile in your project to add third-party libraries and configure Jupyter
applications. Refer to the Dockerfiles for the core stacks (e.g., applications.
[jupyter/datascience-notebook](https://github.com/jupyter/docker-stacks/blob/master/datascience-notebook/Dockerfile)) Refer to the Dockerfiles for the core stacks (e.g., [jupyter/datascience-notebook](https://github.com/jupyter/docker-stacks/blob/master/datascience-notebook/Dockerfile))
to get a feel for what's possible and best practices. to get a feel for what's possible and best practices.
[Submit pull requests](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) [Submit pull requests](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)
to your project repository on GitHub. Ensure your image builds correctly on GitHub actions before merging to to your project repository on GitHub.
master or main. Refer to Docker Hub to build your master or main branch that you can `docker pull`. Ensure your image builds correctly on GitHub actions before merging to
master or main.
Refer to Docker Hub to build your master or main branch that you can `docker pull`.
## Sharing Your Image ## Sharing Your Image
@@ -149,5 +149,5 @@ following:
1. Clone the [jupyter/docker-stacks](https://github.com/jupyter/docker-stacks) GitHub repository. 1. Clone the [jupyter/docker-stacks](https://github.com/jupyter/docker-stacks) GitHub repository.
2. Open the `docs/using/selecting.md` source file and locate the **Community Stacks** section. 2. Open the `docs/using/selecting.md` source file and locate the **Community Stacks** section.
3. Add a bullet with a link to your project and a short description of what your Docker image contains. 3. Add a bullet with a link to your project and a short description of what your Docker image contains.
4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request) 4. [Submit a pull request](https://github.com/PointCloudLibrary/pcl/wiki/A-step-by-step-guide-on-preparing-and-submitting-a-pull-request)(PR) with your changes.
(PR) with your changes. Maintainers will respond and work with you to address any formatting or content issues. Maintainers will respond and work with you to address any formatting or content issues.

15
docs/contributing/tests.md
View File

@@ -6,13 +6,13 @@ of the Docker images.
## How the Tests Work ## How the Tests Work
GitHub executes `make build-test-all` against pull requests submitted to the `jupyter/docker-stacks` GitHub executes `make build-test-all` against pull requests submitted to the `jupyter/docker-stacks`
repository. This `make` command builds every docker image. After building each image, the `make` repository.
command executes `pytest` to run both image-specific tests like those in This `make` command builds every docker image.
After building each image, the `make` command executes `pytest` to run both image-specific tests like those in
[base-notebook/test/](https://github.com/jupyter/docker-stacks/tree/master/base-notebook/test) and [base-notebook/test/](https://github.com/jupyter/docker-stacks/tree/master/base-notebook/test) and
common tests defined in [test/](https://github.com/jupyter/docker-stacks/tree/master/test). Both common tests defined in [test/](https://github.com/jupyter/docker-stacks/tree/master/test).
kinds of tests make use of global [pytest fixtures](https://docs.pytest.org/en/latest/reference/fixtures.html) Both kinds of tests make use of global [pytest fixtures](https://docs.pytest.org/en/latest/reference/fixtures.html)
defined in the [conftest.py](https://github.com/jupyter/docker-stacks/blob/master/conftest.py) file defined in the [conftest.py](https://github.com/jupyter/docker-stacks/blob/master/conftest.py) file at the root of the projects.
at the root of the projects.
## Contributing New Tests ## Contributing New Tests
@@ -22,7 +22,8 @@ Please follow the process below to add new tests:
[test/](https://github.com/jupyter/docker-stacks/tree/master/test) or create a new module. [test/](https://github.com/jupyter/docker-stacks/tree/master/test) or create a new module.
2. If your test should run against a single image, add your test code to one of the modules in 2. If your test should run against a single image, add your test code to one of the modules in
`some-notebook/test/` or create a new module. `some-notebook/test/` or create a new module.
3. Build one or more images you intend to test and run the tests locally. If you use `make`, call: 3. Build one or more images you intend to test and run the tests locally.
If you use `make`, call:
```bash ```bash
make build/somestack-notebook make build/somestack-notebook

6
docs/contributing/translations.md
View File

@@ -1,7 +1,9 @@
# Doc Translations # Doc Translations
We are delighted when members of the Jupyter community want to help translate these documentation pages to other languages. If you're interested, please visit links below below to join our team on [Transifex](https://transifex.com) and to start creating, reviewing, and updating translations of the Jupyter Docker Stacks documentation. We are delighted when members of the Jupyter community want to help translate these documentation pages to other languages.
If you're interested, please visit links below below to join our team on [Transifex](https://transifex.com) and to start creating, reviewing, and updating translations of the Jupyter Docker Stacks documentation.
1. Follow the steps documented on the [Getting Started as a Translator](https://docs.transifex.com/getting-started-1/translators) page. 1. Follow the steps documented on the [Getting Started as a Translator](https://docs.transifex.com/getting-started-1/translators) page.
2. Look for _jupyter-docker-stacks_ when prompted to choose a translation team. Alternatively, visit <https://www.transifex.com/project-jupyter/jupyter-docker-stacks-1> after creating your account and request to join the project. 2. Look for _jupyter-docker-stacks_ when prompted to choose a translation team.
Alternatively, visit <https://www.transifex.com/project-jupyter/jupyter-docker-stacks-1> after creating your account and request to join the project.
3. See [Translating with the Web Editor](https://docs.transifex.com/translation/translating-with-the-web-editor) in the Transifex documentation. 3. See [Translating with the Web Editor](https://docs.transifex.com/translation/translating-with-the-web-editor) in the Transifex documentation.

25
docs/index.rst
View File

@@ -1,7 +1,8 @@
Jupyter Docker Stacks Jupyter Docker Stacks
===================== =====================
Jupyter Docker Stacks are a set of ready-to-run Docker images containing Jupyter applications and interactive computing tools. You can use a stack image to do any of the following (and more): Jupyter Docker Stacks are a set of ready-to-run Docker images containing Jupyter applications and interactive computing tools.
You can use a stack image to do any of the following (and more):
* Start a personal Jupyter Notebook server in a local Docker container * Start a personal Jupyter Notebook server in a local Docker container
* Run JupyterLab servers for a team using JupyterHub * Run JupyterLab servers for a team using JupyterHub
@@ -10,20 +11,34 @@ Jupyter Docker Stacks are a set of ready-to-run Docker images containing Jupyter
Quick Start Quick Start
----------- -----------
You can try a `recent build of the jupyter/base-notebook image on mybinder.org <https://mybinder.org/v2/gh/jupyter/docker-stacks/master?filepath=README.ipynb>`_ by simply clicking the preceding link. Otherwise, three examples below may help you get started if you `have Docker installed <https://docs.docker.com/install/>`_, know :doc:`which Docker image <using/selecting>` you want to use, and want to launch a single Jupyter Notebook server in a container. You can try a `recent build of the jupyter/base-notebook image on mybinder.org <https://mybinder.org/v2/gh/jupyter/docker-stacks/master?filepath=README.ipynb>`_ by simply clicking the preceding link.
Otherwise, three examples below may help you get started if you `have Docker installed <https://docs.docker.com/install/>`_, know :doc:`which Docker image <using/selecting>` you want to use, and want to launch a single Jupyter Notebook server in a container.
The other pages in this documentation describe additional uses and features in detail. The other pages in this documentation describe additional uses and features in detail.
**Example 1:** This command pulls the ``jupyter/scipy-notebook`` image tagged ``33add21fab64`` from Docker Hub if it is not already present on the local host. It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888. The server logs appear in the terminal. Visiting ``http://<hostname>:8888/?token=<token>`` in a browser loads the Jupyter Notebook dashboard page, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console. The container remains intact for restart after the notebook server exits.:: **Example 1:** This command pulls the ``jupyter/scipy-notebook`` image tagged ``33add21fab64`` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888.
The server logs appear in the terminal.
Visiting ``http://<hostname>:8888/?token=<token>`` in a browser loads the Jupyter Notebook dashboard page, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console.
The container remains intact for restart after the notebook server exits.:
.. code-block:: bash
docker run -p 8888:8888 jupyter/scipy-notebook:33add21fab64 docker run -p 8888:8888 jupyter/scipy-notebook:33add21fab64
**Example 2:** This command performs the same operations as **Example 1**, but it exposes the server on host port 10000 instead of port 8888. Visiting ``http://<hostname>:10000/?token=<token>`` in a browser loads Jupyter Notebook server, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console.:: **Example 2:** This command performs the same operations as **Example 1**, but it exposes the server on host port 10000 instead of port 8888.
Visiting ``http://<hostname>:10000/?token=<token>`` in a browser loads Jupyter Notebook server, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console.:
.. code-block:: bash
docker run -p 10000:8888 jupyter/scipy-notebook:33add21fab64 docker run -p 10000:8888 jupyter/scipy-notebook:33add21fab64
**Example 3:** This command pulls the ``jupyter/datascience-notebook`` image tagged ``33add21fab64`` from Docker Hub if it is not already present on the local host. It then starts an *ephemeral* container running a Jupyter Notebook server and exposes the server on host port 10000. The command mounts the current working directory on the host as ``/home/jovyan/work`` in the container. The server logs appear in the terminal. Visiting ``http://<hostname>:10000/lab?token=<token>`` in a browser loads JupyterLab, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console. Docker destroys the container after notebook server exit, but any files written to ``~/work`` in the container remain intact on the host.:: **Example 3:** This command pulls the ``jupyter/datascience-notebook`` image tagged ``33add21fab64`` from Docker Hub if it is not already present on the local host.
It then starts an *ephemeral* container running a Jupyter Notebook server and exposes the server on host port 10000.
The command mounts the current working directory on the host as ``/home/jovyan/work`` in the container.
The server logs appear in the terminal.
Visiting ``http://<hostname>:10000/lab?token=<token>`` in a browser loads JupyterLab, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console.
Docker destroys the container after notebook server exit, but any files written to ``~/work`` in the container remain intact on the host.:
.. code-block:: bash
docker run --rm -p 10000:8888 -e JUPYTER_ENABLE_LAB=yes -v "${PWD}":/home/jovyan/work jupyter/datascience-notebook:33add21fab64 docker run --rm -p 10000:8888 -e JUPYTER_ENABLE_LAB=yes -v "${PWD}":/home/jovyan/work jupyter/datascience-notebook:33add21fab64
Table of Contents Table of Contents

11
docs/maintaining/tasks.md
View File

@@ -9,16 +9,15 @@ To build new images and publish them to the Docker Hub registry, do the followin
3. Monitor the merge commit GitHub Actions status. 3. Monitor the merge commit GitHub Actions status.
**Note**: we think, GitHub Actions are quite reliable, so please, investigate, if some error occurs. **Note**: we think, GitHub Actions are quite reliable, so please, investigate, if some error occurs.
The process of building docker images in PRs is exactly the same after merging to master, except there is an additional `push` step. The process of building docker images in PRs is exactly the same after merging to master, except there is an additional `push` step.
4. Try to avoid merging another PR to master until all pending builds complete. This way you will know which commit 4. Try to avoid merging another PR to master until all pending builds complete.
might have broken the build and also have correct tags for moving tags (like `python` version). This way you will know which commit might have broken the build and also have correct tags for moving tags (like `python` version).
## Updating the Ubuntu Base Image ## Updating the Ubuntu Base Image
When there's a security fix in the Ubuntu base image or after some time passes, it's a good idea to When there's a security fix in the Ubuntu base image or after some time passes, it's a good idea to update the pinned SHA in the
update the pinned SHA in the
[jupyter/base-notebook Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile). [jupyter/base-notebook Dockerfile](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile).
Submit it as a regular PR and go through the build process. Expect the build to take a while to Submit it as a regular PR and go through the build process.
complete: every image layer will rebuild. Expect the build to take a while to complete: every image layer will rebuild.
## Adding a New Core Image to Docker Hub ## Adding a New Core Image to Docker Hub

98
docs/using/common.md
View File

@@ -1,12 +1,15 @@
# Common Features # Common Features
A container launched from any Jupyter Docker Stacks image runs a Jupyter Notebook server by default. The container does so by executing a `start-notebook.sh` script. This script configures the internal container environment and then runs `jupyter notebook`, passing it any command line arguments received. A container launched from any Jupyter Docker Stacks image runs a Jupyter Notebook server by default.
The container does so by executing a `start-notebook.sh` script.
This script configures the internal container environment and then runs `jupyter notebook`, passing it any command line arguments received.
This page describes the options supported by the startup script as well as how to bypass it to run alternative commands. This page describes the options supported by the startup script as well as how to bypass it to run alternative commands.
## Notebook Options ## Notebook Options
You can pass [Jupyter command line options](https://jupyter-notebook.readthedocs.io/en/stable/config.html#options) to the `start-notebook.sh` script when launching the container. For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, you can run the following: You can pass [Jupyter command line options](https://jupyter-notebook.readthedocs.io/en/stable/config.html#options) to the `start-notebook.sh` script when launching the container.
For example, to secure the Notebook server with a custom password hashed using `IPython.lib.passwd()` instead of the default token, you can run the following:
```bash ```bash
docker run -d -p 8888:8888 jupyter/base-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e' docker run -d -p 8888:8888 jupyter/base-notebook start-notebook.sh --NotebookApp.password='sha1:74ba40f8a388:c913541b7ee99d15d5ed31d4226bf7838f83a50e'
@@ -21,21 +24,60 @@ docker run -d -p 8888:8888 jupyter/base-notebook start-notebook.sh --NotebookApp
## Docker Options ## Docker Options
You may instruct the `start-notebook.sh` script to customize the container environment before launching You may instruct the `start-notebook.sh` script to customize the container environment before launching
the notebook server. You do so by passing arguments to the `docker run` command. the notebook server.
You do so by passing arguments to the `docker run` command.
- `-e NB_USER=jovyan` - Instructs the startup script to change the default container username from `jovyan` to the provided value. Causes the script to rename the `jovyan` user home folder. For this option to take effect, you must run the container with `--user root`, set the working directory `-w /home/${NB_USER}` and set the environment variable `-e CHOWN_HOME=yes` (see below for detail). This feature is useful when mounting host volumes with specific home folder. - `-e NB_USER=jovyan` - Instructs the startup script to change the default container username from `jovyan` to the provided value.
- `-e NB_UID=1000` - Instructs the startup script to switch the numeric user ID of `${NB_USER}` to the given value. This feature is useful when mounting host volumes with specific owner permissions. For this option to take effect, you must run the container with `--user root`. (The startup script will `su ${NB_USER}` after adjusting the user ID.) You might consider using modern Docker options `--user` and `--group-add` instead. See the last bullet below for details. Causes the script to rename the `jovyan` user home folder.
- `-e NB_GID=100` - Instructs the startup script to change the primary group of`${NB_USER}` to `${NB_GID}` (the new group is added with a name of `${NB_GROUP}` if it is defined, otherwise the group is named `${NB_USER}`). This feature is useful when mounting host volumes with specific group permissions. For this option to take effect, you must run the container with `--user root`. (The startup script will `su ${NB_USER}` after adjusting the group ID.) You might consider using modern Docker options `--user` and `--group-add` instead. See the last bullet below for details. The user is added to supplemental group `users` (gid 100) in order to allow write access to the home directory and `/opt/conda`. If you override the user/group logic, ensure the user stays in group `users` if you want them to be able to modify files in the image. For this option to take effect, you must run the container with `--user root`, set the working directory `-w /home/${NB_USER}` and set the environment variable `-e CHOWN_HOME=yes` (see below for detail).
- `-e NB_GROUP=<name>` - The name used for `${NB_GID}`, which defaults to `${NB_USER}`. This is only used if `${NB_GID}` is specified and completely optional: there is only cosmetic effect. This feature is useful when mounting host volumes with specific home folder.
- `-e NB_UMASK=<umask>` - Configures Jupyter to use a different umask value from default, i.e. `022`. For example, if setting umask to `002`, new files will be readable and writable by group members instead of just writable by the owner. Wikipedia has a good article about [umask](https://en.wikipedia.org/wiki/Umask). Feel free to read it in order to choose the value that better fits your needs. Default value should fit most situations. Note that `NB_UMASK` when set only applies to the Jupyter process itself - you cannot use it to set a umask for additional files created during run-hooks e.g. via `pip` or `conda` - if you need to set a umask for these you must set `umask` for each command. - `-e NB_UID=1000` - Instructs the startup script to switch the numeric user ID of `${NB_USER}` to the given value.
- `-e CHOWN_HOME=yes` - Instructs the startup script to change the `${NB_USER}` home directory owner and group to the current value of `${NB_UID}` and `${NB_GID}`. This change will take effect even if the user home directory is mounted from the host using `-v` as described below. The change is **not** applied recursively by default. You can change modify the `chown` behavior by setting `CHOWN_HOME_OPTS` (e.g., `-e CHOWN_HOME_OPTS='-R'`). This feature is useful when mounting host volumes with specific owner permissions.
- `-e CHOWN_EXTRA="<some dir>,<some other dir>"` - Instructs the startup script to change the owner and group of each comma-separated container directory to the current value of `${NB_UID}` and `${NB_GID}`. The change is **not** applied recursively by default. You can change modify the `chown` behavior by setting `CHOWN_EXTRA_OPTS` (e.g., `-e CHOWN_EXTRA_OPTS='-R'`). For this option to take effect, you must run the container with `--user root`.
- `-e GRANT_SUDO=yes` - Instructs the startup script to grant the `NB_USER` user passwordless `sudo` capability. You do **not** need this option to allow the user to `conda` or `pip` install additional packages. This option is useful, however, when you wish to give `${NB_USER}` the ability to install OS packages with `apt` or modify other root-owned files in the container. For this option to take effect, you must run the container with `--user root`. (The `start-notebook.sh` script will `su ${NB_USER}` after adding `${NB_USER}` to sudoers.) **You should only enable `sudo` if you trust the user or if the container is running on an isolated host.** (The startup script will `su ${NB_USER}` after adjusting the user ID.)
You might consider using modern Docker options `--user` and `--group-add` instead.
See the last bullet below for details.
- `-e NB_GID=100` - Instructs the startup script to change the primary group of`${NB_USER}` to `${NB_GID}`
(the new group is added with a name of `${NB_GROUP}` if it is defined, otherwise the group is named `${NB_USER}`).
This feature is useful when mounting host volumes with specific group permissions.
For this option to take effect, you must run the container with `--user root`.
(The startup script will `su ${NB_USER}` after adjusting the group ID.)
You might consider using modern Docker options `--user` and `--group-add` instead.
See the last bullet below for details.
The user is added to supplemental group `users` (gid 100) in order to allow write access to the home directory and `/opt/conda`.
If you override the user/group logic, ensure the user stays in group `users` if you want them to be able to modify files in the image.
- `-e NB_GROUP=<name>` - The name used for `${NB_GID}`, which defaults to `${NB_USER}`.
This is only used if `${NB_GID}` is specified and completely optional: there is only cosmetic effect.
- `-e NB_UMASK=<umask>` - Configures Jupyter to use a different umask value from default, i.e. `022`.
For example, if setting umask to `002`, new files will be readable and writable by group members instead of just writable by the owner.
Wikipedia has a good article about [umask](https://en.wikipedia.org/wiki/Umask).
Feel free to read it in order to choose the value that better fits your needs.
Default value should fit most situations.
Note that `NB_UMASK` when set only applies to the Jupyter process itself - you cannot use it to set a umask for additional files created during run-hooks
e.g. via `pip` or `conda` - if you need to set a umask for these you must set `umask` for each command.
- `-e CHOWN_HOME=yes` - Instructs the startup script to change the `${NB_USER}` home directory owner and group to the current value of `${NB_UID}` and `${NB_GID}`.
This change will take effect even if the user home directory is mounted from the host using `-v` as described below.
The change is **not** applied recursively by default.
You can change modify the `chown` behavior by setting `CHOWN_HOME_OPTS` (e.g., `-e CHOWN_HOME_OPTS='-R'`).
- `-e CHOWN_EXTRA="<some dir>,<some other dir>"` - Instructs the startup script to change the owner and group of each comma-separated container directory to the current value of `${NB_UID}` and `${NB_GID}`.
The change is **not** applied recursively by default.
You can change modify the `chown` behavior by setting `CHOWN_EXTRA_OPTS` (e.g., `-e CHOWN_EXTRA_OPTS='-R'`).
- `-e GRANT_SUDO=yes` - Instructs the startup script to grant the `NB_USER` user passwordless `sudo` capability.
You do **not** need this option to allow the user to `conda` or `pip` install additional packages.
This option is useful, however, when you wish to give `${NB_USER}` the ability to install OS packages with `apt` or modify other root-owned files in the container.
For this option to take effect, you must run the container with `--user root`.
(The `start-notebook.sh` script will `su ${NB_USER}` after adding `${NB_USER}` to sudoers.)
**You should only enable `sudo` if you trust the user or if the container is running on an isolated host.**
- `-e GEN_CERT=yes` - Instructs the startup script to generates a self-signed SSL certificate and configure Jupyter Notebook to use it to accept encrypted HTTPS connections. - `-e GEN_CERT=yes` - Instructs the startup script to generates a self-signed SSL certificate and configure Jupyter Notebook to use it to accept encrypted HTTPS connections.
- `-e JUPYTER_ENABLE_LAB=yes` - Instructs the startup script to run `jupyter lab` instead of the default `jupyter notebook` command. Useful in container orchestration environments where setting environment variables is easier than change command line parameters. - `-e JUPYTER_ENABLE_LAB=yes` - Instructs the startup script to run `jupyter lab` instead of the default `jupyter notebook` command.
- `-e RESTARTABLE=yes` - Runs Jupyter in a loop so that quitting Jupyter does not cause the container to exit. This may be useful when you need to install extensions that require restarting Jupyter. Useful in container orchestration environments where setting environment variables is easier than change command line parameters.
- `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container. Useful when you want to preserve notebooks and other work even after the container is destroyed. **You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).** - `-e RESTARTABLE=yes` - Runs Jupyter in a loop so that quitting Jupyter does not cause the container to exit.
- `--user 5000 --group-add users` - Launches the container with a specific user ID and adds that user to the `users` group so that it can modify files in the default home directory and `/opt/conda`. You can use these arguments as alternatives to setting `${NB_UID}` and `${NB_GID}`. This may be useful when you need to install extensions that require restarting Jupyter.
- `-v /some/host/folder/for/work:/home/jovyan/work` - Mounts a host machine directory as folder in the container.
Useful when you want to preserve notebooks and other work even after the container is destroyed.
**You must grant the within-container notebook user or group (`NB_UID` or `NB_GID`) write access to the host directory (e.g., `sudo chown 1000 /some/host/folder/for/work`).**
- `--user 5000 --group-add users` - Launches the container with a specific user ID and adds that user to the `users` group so that it can modify files in the default home directory and `/opt/conda`.
You can use these arguments as alternatives to setting `${NB_UID}` and `${NB_GID}`.
## Startup Hooks ## Startup Hooks
@@ -52,7 +94,8 @@ script for execution details.
## SSL Certificates ## SSL Certificates
You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections. For example, to mount a host folder containing a `notebook.key` and `notebook.crt` and use them, you might run the following: You may mount SSL key and certificate files into a container and configure Jupyter Notebook to use them to accept HTTPS connections.
For example, to mount a host folder containing a `notebook.key` and `notebook.crt` and use them, you might run the following:
```bash ```bash
docker run -d -p 8888:8888 \ docker run -d -p 8888:8888 \
@@ -62,7 +105,8 @@ docker run -d -p 8888:8888 \
--NotebookApp.certfile=/etc/ssl/notebook/notebook.crt --NotebookApp.certfile=/etc/ssl/notebook/notebook.crt
``` ```
Alternatively, you may mount a single PEM file containing both the key and certificate. For example: Alternatively, you may mount a single PEM file containing both the key and certificate.
For example:
```bash ```bash
docker run -d -p 8888:8888 \ docker run -d -p 8888:8888 \
@@ -71,11 +115,13 @@ docker run -d -p 8888:8888 \
--NotebookApp.certfile=/etc/ssl/notebook.pem --NotebookApp.certfile=/etc/ssl/notebook.pem
``` ```
In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file. The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root). In either case, Jupyter Notebook expects the key and certificate to be a base64 encoded text file.
The certificate file or PEM may contain one or more certificates (e.g., server, intermediate, and root).
For additional information about using SSL, see the following: For additional information about using SSL, see the following:
- The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use [Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain. - The [docker-stacks/examples](https://github.com/jupyter/docker-stacks/tree/master/examples) for information about how to use
[Let's Encrypt](https://letsencrypt.org/) certificates when you run these stacks on a publicly visible domain.
- The [jupyter_notebook_config.py](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate. - The [jupyter_notebook_config.py](https://github.com/jupyter/docker-stacks/blob/master/base-notebook/jupyter_notebook_config.py) file for how this Docker image generates a self-signed certificate.
- The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#securing-a-notebook-server) for best practices about securing a public notebook server in general. - The [Jupyter Notebook documentation](https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#securing-a-notebook-server) for best practices about securing a public notebook server in general.
@@ -83,7 +129,9 @@ For additional information about using SSL, see the following:
### start.sh ### start.sh
The `start-notebook.sh` script actually inherits most of its option handling capability from a more generic `start.sh` script. The `start.sh` script supports all of the features described above, but allows you to specify an arbitrary command to execute. For example, to run the text-based `ipython` console in a container, do the following: The `start-notebook.sh` script actually inherits most of its option handling capability from a more generic `start.sh` script.
The `start.sh` script supports all of the features described above, but allows you to specify an arbitrary command to execute.
For example, to run the text-based `ipython` console in a container, do the following:
```bash ```bash
docker run -it --rm jupyter/base-notebook start.sh ipython docker run -it --rm jupyter/base-notebook start.sh ipython
@@ -99,13 +147,17 @@ This script is particularly useful when you derive a new Dockerfile from this im
### Others ### Others
You can bypass the provided scripts and specify an arbitrary start command. If you do, keep in mind that features supported by the `start.sh` script and its kin will not function (e.g., `GRANT_SUDO`). You can bypass the provided scripts and specify an arbitrary start command.
If you do, keep in mind that features supported by the `start.sh` script and its kin will not function (e.g., `GRANT_SUDO`).
## Conda Environments ## Conda Environments
The default Python 3.x [Conda environment](https://conda.io/projects/conda/en/latest/user-guide/concepts/environments.html) resides in `/opt/conda`. The `/opt/conda/bin` directory is part of the default `jovyan` user's `${PATH}`. That directory is also whitelisted for use in `sudo` commands by the `start.sh` script. The default Python 3.x [Conda environment](https://conda.io/projects/conda/en/latest/user-guide/concepts/environments.html) resides in `/opt/conda`.
The `/opt/conda/bin` directory is part of the default `jovyan` user's `${PATH}`.
That directory is also whitelisted for use in `sudo` commands by the `start.sh` script.
The `jovyan` user has full read/write access to the `/opt/conda` directory. You can use either `pip`, `conda` or `mamba` to install new packages without any additional permissions. The `jovyan` user has full read/write access to the `/opt/conda` directory.
You can use either `pip`, `conda` or `mamba` to install new packages without any additional permissions.
```bash ```bash
# install a package into the default (python 3.x) environment and cleanup after the installation # install a package into the default (python 3.x) environment and cleanup after the installation

76
docs/using/recipes.md
View File

@@ -1,19 +1,16 @@
# Contributed Recipes # Contributed Recipes
Users sometimes share interesting ways of using the Jupyter Docker Stacks. We encourage users to Users sometimes share interesting ways of using the Jupyter Docker Stacks.
[contribute these recipes](../contributing/recipes.md) to the documentation in case they prove We encourage users to [contribute these recipes](../contributing/recipes.md) to the documentation in case they prove
useful to other members of the community by submitting a pull request to `docs/using/recipes.md`. useful to other members of the community by submitting a pull request to `docs/using/recipes.md`.
The sections below capture this knowledge. The sections below capture this knowledge.
## Using `sudo` within a container ## Using `sudo` within a container
Password authentication is disabled for the `NB_USER` (e.g., `jovyan`). This choice was made to Password authentication is disabled for the `NB_USER` (e.g., `jovyan`).
avoid distributing images with a weak default password that users ~might~ will forget to change This choice was made to avoid distributing images with a weak default password that users ~might~ will forget to change before running a container on a publicly accessible host.
before running a container on a publicly accessible host.
You can grant the within-container `NB_USER` passwordless `sudo` access by adding You can grant the within-container `NB_USER` passwordless `sudo` access by adding `-e GRANT_SUDO=yes` and `--user root` to your Docker command line or appropriate container orchestrator config.
`-e GRANT_SUDO=yes` and `--user root` to your Docker command line or appropriate container
orchestrator config.
For example: For example:
@@ -21,8 +18,8 @@ For example:
docker run -it -e GRANT_SUDO=yes --user root jupyter/minimal-notebook docker run -it -e GRANT_SUDO=yes --user root jupyter/minimal-notebook
``` ```
**You should only enable `sudo` if you trust the user and/or if the container is running on an **You should only enable `sudo` if you trust the user and/or if the container is running on an isolated host.**
isolated host.** See [Docker security documentation](https://docs.docker.com/engine/security/userns-remap/) for more information about running containers as `root`. See [Docker security documentation](https://docs.docker.com/engine/security/userns-remap/) for more information about running containers as `root`.
## Using `pip install` or `conda install` in a Child Docker image ## Using `pip install` or `conda install` in a Child Docker image
@@ -44,7 +41,8 @@ docker build --rm -t jupyter/my-datascience-notebook .
``` ```
To use a requirements.txt file, first create your `requirements.txt` file with the listing of To use a requirements.txt file, first create your `requirements.txt` file with the listing of
packages desired. Next, create a new Dockerfile like the one shown below. packages desired.
Next, create a new Dockerfile like the one shown below.
```dockerfile ```dockerfile
# Start from a core stack version # Start from a core stack version
@@ -73,9 +71,8 @@ Ref: [docker-stacks/commit/79169618d571506304934a7b29039085e77db78c](https://git
## Add a Python 2.x environment ## Add a Python 2.x environment
Python 2.x was removed from all images on August 10th, 2017, starting in tag `cc9feab481f7`. You can Python 2.x was removed from all images on August 10th, 2017, starting in tag `cc9feab481f7`.
add a Python 2.x environment by defining your own Dockerfile inheriting from one of the images like You can add a Python 2.x environment by defining your own Dockerfile inheriting from one of the images like so:
so:
```dockerfile ```dockerfile
# Choose your desired base image # Choose your desired base image
@@ -150,7 +147,8 @@ Run jupyterlab using a command such as
## Dask JupyterLab Extension ## Dask JupyterLab Extension
[Dask JupyterLab Extension](https://github.com/dask/dask-labextension) provides a JupyterLab extension to manage Dask clusters, as well as embed Dask's dashboard plots directly into JupyterLab panes. Create the Dockerfile as: [Dask JupyterLab Extension](https://github.com/dask/dask-labextension) provides a JupyterLab extension to manage Dask clusters, as well as embed Dask's dashboard plots directly into JupyterLab panes.
Create the Dockerfile as:
```dockerfile ```dockerfile
# Start from a core stack version # Start from a core stack version
@@ -208,8 +206,8 @@ Credit: [Paolo D.](https://github.com/pdonorio) based on
## xgboost ## xgboost
You need to install conda's gcc for Python xgboost to work properly. Otherwise, you'll get an You need to install conda's gcc for Python xgboost to work properly.
exception about libgomp.so.1 missing GOMP_4.0. Otherwise, you'll get an exception about libgomp.so.1 missing GOMP_4.0.
```bash ```bash
conda install --quiet --yes gcc && \ conda install --quiet --yes gcc && \
@@ -233,25 +231,25 @@ Sometimes it is useful to run the Jupyter instance behind a nginx proxy, for ins
- you may have many different services in addition to Jupyter running on the same server, and want - you may have many different services in addition to Jupyter running on the same server, and want
to nginx to help improve server performance in manage the connections to nginx to help improve server performance in manage the connections
Here is a [quick example NGINX configuration](https://gist.github.com/cboettig/8643341bd3c93b62b5c2) Here is a [quick example NGINX configuration](https://gist.github.com/cboettig/8643341bd3c93b62b5c2) to get started.
to get started. You'll need a server, a `.crt` and `.key` file for your server, and `docker` & You'll need a server, a `.crt` and `.key` file for your server, and `docker` & `docker-compose` installed.
`docker-compose` installed. Then just download the files at that gist and run `docker-compose up -d` Then just download the files at that gist and run `docker-compose up -d` to test it out.
to test it out. Customize the `nginx.conf` file to set the desired paths and add other services. Customize the `nginx.conf` file to set the desired paths and add other services.
## Host volume mounts and notebook errors ## Host volume mounts and notebook errors
If you are mounting a host directory as `/home/jovyan/work` in your container and you receive If you are mounting a host directory as `/home/jovyan/work` in your container and you receive
permission errors or connection errors when you create a notebook, be sure that the `jovyan` user permission errors or connection errors when you create a notebook, be sure that the `jovyan` user
(UID=1000 by default) has read/write access to the directory on the host. Alternatively, specify the (UID=1000 by default) has read/write access to the directory on the host.
UID of the `jovyan` user on container startup using the `-e NB_UID` option described in the Alternatively, specify the UID of the `jovyan` user on container startup using the `-e NB_UID` option described in the
[Common Features, Docker Options section](../using/common.html#Docker-Options) [Common Features, Docker Options section](../using/common.html#Docker-Options)
Ref: <https://github.com/jupyter/docker-stacks/issues/199> Ref: <https://github.com/jupyter/docker-stacks/issues/199>
## Manpage installation ## Manpage installation
Most containers, including our Ubuntu base image, ship without manpages installed to save space. You Most containers, including our Ubuntu base image, ship without manpages installed to save space.
can use the following dockerfile to inherit from one of our images to enable manpages: You can use the following dockerfile to inherit from one of our images to enable manpages:
```dockerfile ```dockerfile
# Choose your desired base image # Choose your desired base image
@@ -467,21 +465,29 @@ RUN pip install --quiet --no-cache-dir jupyter_dashboards faker && \
USER root USER root
# Ensure we overwrite the kernel config so that toree connects to cluster # Ensure we overwrite the kernel config so that toree connects to cluster
RUN jupyter toree install --sys-prefix --spark_opts="--master yarn --deploy-mode client --driver-memory 512m --executor-memory 512m --executor-cores 1 --driver-java-options -Dhdp.version=2.5.3.0-37 --conf spark.hadoop.yarn.timeline-service.enabled=false" RUN jupyter toree install --sys-prefix --spark_opts="\
--master yarn
--deploy-mode client
--driver-memory 512m
--executor-memory 512m
--executor-cores 1
--driver-java-options
-Dhdp.version=2.5.3.0-37
--conf spark.hadoop.yarn.timeline-service.enabled=false
"
USER ${NB_UID} USER ${NB_UID}
``` ```
Credit: [britishbadger](https://github.com/britishbadger) from Credit: [britishbadger](https://github.com/britishbadger) from [docker-stacks/issues/369](https://github.com/jupyter/docker-stacks/issues/369)
[docker-stacks/issues/369](https://github.com/jupyter/docker-stacks/issues/369)
## Run Jupyter Notebook/Lab inside an already secured environment (i.e., with no token) ## Run Jupyter Notebook/Lab inside an already secured environment (i.e., with no token)
(Adapted from [issue 728](https://github.com/jupyter/docker-stacks/issues/728)) (Adapted from [issue 728](https://github.com/jupyter/docker-stacks/issues/728))
The default security is very good. There are use cases, encouraged by containers, where the jupyter The default security is very good.
container and the system it runs within, lie inside the security boundary. In these use cases it is There are use cases, encouraged by containers, where the jupyter container and the system it runs within, lie inside the security boundary.
convenient to launch the server without a password or token. In this case, you should use the In these use cases it is convenient to launch the server without a password or token.
`start.sh` script to launch the server with no token: In this case, you should use the `start.sh` script to launch the server with no token:
For jupyterlab: For jupyterlab:
@@ -517,7 +523,8 @@ Ref: <https://github.com/jupyter/docker-stacks/issues/675>
## Enable auto-sklearn notebooks ## Enable auto-sklearn notebooks
Using `auto-sklearn` requires `swig`, which the other notebook images lack, so it cant be experimented with. Also, there is no Conda package for `auto-sklearn`. Using `auto-sklearn` requires `swig`, which the other notebook images lack, so it cant be experimented with.
Also, there is no Conda package for `auto-sklearn`.
```dockerfile ```dockerfile
ARG BASE_CONTAINER=jupyter/scipy-notebook ARG BASE_CONTAINER=jupyter/scipy-notebook
@@ -539,7 +546,8 @@ RUN pip install --quiet --no-cache-dir auto-sklearn && \
## Enable Delta Lake in Spark notebooks ## Enable Delta Lake in Spark notebooks
Please note that the [Delta Lake](https://delta.io/) packages are only available for Spark version > `3.0`. By adding the properties to `spark-defaults.conf`, the user no longer needs to enable Delta support in each notebook. Please note that the [Delta Lake](https://delta.io/) packages are only available for Spark version > `3.0`.
By adding the properties to `spark-defaults.conf`, the user no longer needs to enable Delta support in each notebook.
```dockerfile ```dockerfile
FROM jupyter/pyspark-notebook:latest FROM jupyter/pyspark-notebook:latest

36
docs/using/running.md
View File

@@ -9,9 +9,13 @@ This section provides details about the second.
## Using the Docker CLI ## Using the Docker CLI
You can launch a local Docker container from the Jupyter Docker Stacks using the [Docker command line interface](https://docs.docker.com/engine/reference/commandline/cli/). There are numerous ways to configure containers using the CLI. The following are some common patterns. You can launch a local Docker container from the Jupyter Docker Stacks using the [Docker command line interface](https://docs.docker.com/engine/reference/commandline/cli/).
There are numerous ways to configure containers using the CLI.
The following are some common patterns.
**Example 1** This command pulls the `jupyter/scipy-notebook` image tagged `33add21fab64` from Docker Hub if it is not already present on the local host. It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888. The server logs appear in the terminal and include a URL to the notebook server. **Example 1** This command pulls the `jupyter/scipy-notebook` image tagged `33add21fab64` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Notebook server and exposes the server on host port 8888.
The server logs appear in the terminal and include a URL to the notebook server.
```bash ```bash
$ docker run -p 8888:8888 jupyter/scipy-notebook:33add21fab64 $ docker run -p 8888:8888 jupyter/scipy-notebook:33add21fab64
@@ -52,7 +56,9 @@ $ docker rm d67fe77f1a84
d67fe77f1a84 d67fe77f1a84
``` ```
**Example 2** This command pulls the `jupyter/r-notebook` image tagged `33add21fab64` from Docker Hub if it is not already present on the local host. It then starts a container running a Jupyter Notebook server and exposes the server on host port 10000. The server logs appear in the terminal and include a URL to the notebook server, but with the internal container port (8888) instead of the the correct host port (10000). **Example 2** This command pulls the `jupyter/r-notebook` image tagged `33add21fab64` from Docker Hub if it is not already present on the local host.
It then starts a container running a Jupyter Notebook server and exposes the server on host port 10000.
The server logs appear in the terminal and include a URL to the notebook server, but with the internal container port (8888) instead of the the correct host port (10000).
```bash ```bash
$ docker run --rm -p 10000:8888 -v "${PWD}":/home/jovyan/work jupyter/r-notebook:33add21fab64 $ docker run --rm -p 10000:8888 -v "${PWD}":/home/jovyan/work jupyter/r-notebook:33add21fab64
@@ -74,9 +80,12 @@ Executing the command: jupyter notebook
http://localhost:8888/?token=3b8dce890cb65570fb0d9c4a41ae067f7604873bd604f5ac http://localhost:8888/?token=3b8dce890cb65570fb0d9c4a41ae067f7604873bd604f5ac
``` ```
Pressing `Ctrl-C` shuts down the notebook server and immediately destroys the Docker container. Files written to `~/work` in the container remain touched. Any other changes made in the container are lost. Pressing `Ctrl-C` shuts down the notebook server and immediately destroys the Docker container.
Files written to `~/work` in the container remain touched.
Any other changes made in the container are lost.
**Example 3** This command pulls the `jupyter/all-spark-notebook` image currently tagged `latest` from Docker Hub if an image tagged `latest` is not already present on the local host. It then starts a container named `notebook` running a JupyterLab server and exposes the server on a randomly selected port. **Example 3** This command pulls the `jupyter/all-spark-notebook` image currently tagged `latest` from Docker Hub if an image tagged `latest` is not already present on the local host.
It then starts a container named `notebook` running a JupyterLab server and exposes the server on a randomly selected port.
```bash ```bash
docker run -d -P --name notebook jupyter/all-spark-notebook docker run -d -P --name notebook jupyter/all-spark-notebook
@@ -112,12 +121,23 @@ notebook
## Using Binder ## Using Binder
[Binder](https://mybinder.org/) is a service that allows you to create and share custom computing environments for projects in version control. You can use any of the Jupyter Docker Stacks images as a basis for a Binder-compatible Dockerfile. See the [docker-stacks example](https://mybinder.readthedocs.io/en/latest/sample_repos.html#using-a-docker-image-from-the-jupyter-docker-stacks-repository) and [Using a Dockerfile](https://mybinder.readthedocs.io/en/latest/tutorials/dockerfile.html) sections in the [Binder documentation](https://mybinder.readthedocs.io/en/latest/index.html) for instructions. [Binder](https://mybinder.org/) is a service that allows you to create and share custom computing environments for projects in version control.
You can use any of the Jupyter Docker Stacks images as a basis for a Binder-compatible Dockerfile.
See the
[docker-stacks example](https://mybinder.readthedocs.io/en/latest/sample_repos.html#using-a-docker-image-from-the-jupyter-docker-stacks-repository) and
[Using a Dockerfile](https://mybinder.readthedocs.io/en/latest/tutorials/dockerfile.html) sections in the
[Binder documentation](https://mybinder.readthedocs.io/en/latest/index.html) for instructions.
## Using JupyterHub ## Using JupyterHub
You can configure JupyterHub to launcher Docker containers from the Jupyter Docker Stacks images. If you've been following the [Zero to JupyterHub with Kubernetes](https://zero-to-jupyterhub.readthedocs.io/en/latest/) guide, see the [Use an existing Docker image](https://zero-to-jupyterhub.readthedocs.io/en/latest/jupyterhub/customizing/user-environment.html#choose-and-use-an-existing-docker-image) section for details. If you have a custom JupyterHub deployment, see the [Picking or building a Docker image](https://github.com/jupyterhub/dockerspawner#picking-or-building-a-docker-image) instructions for the [dockerspawner](https://github.com/jupyterhub/dockerspawner) instead. You can configure JupyterHub to launcher Docker containers from the Jupyter Docker Stacks images.
If you've been following the [Zero to JupyterHub with Kubernetes](https://zero-to-jupyterhub.readthedocs.io/en/latest/) guide,
see the [Use an existing Docker image](https://zero-to-jupyterhub.readthedocs.io/en/latest/jupyterhub/customizing/user-environment.html#choose-and-use-an-existing-docker-image) section for details.
If you have a custom JupyterHub deployment, see the [Picking or building a Docker image](https://github.com/jupyterhub/dockerspawner#picking-or-building-a-docker-image)
instructions for the [dockerspawner](https://github.com/jupyterhub/dockerspawner) instead.
## Using Other Tools and Services ## Using Other Tools and Services
You can use the Jupyter Docker Stacks with any Docker-compatible technology (e.g., [Docker Compose](https://docs.docker.com/compose/), [docker-py](https://github.com/docker/docker-py), your favorite cloud container service). See the documentation of the tool, library, or service for details about how to reference, configure, and launch containers from these images. You can use the Jupyter Docker Stacks with any Docker-compatible technology
(e.g., [Docker Compose](https://docs.docker.com/compose/), [docker-py](https://github.com/docker/docker-py), your favorite cloud container service).
See the documentation of the tool, library, or service for details about how to reference, configure, and launch containers from these images.

87
docs/using/selecting.md
View File

@@ -13,10 +13,8 @@ This section provides details about the first.
## Core Stacks ## Core Stacks
The Jupyter team maintains a set of Docker image definitions in the The Jupyter team maintains a set of Docker image definitions in the <https://github.com/jupyter/docker-stacks> GitHub repository.
<https://github.com/jupyter/docker-stacks> GitHub The following sections describe these images including their contents, relationships, and versioning strategy.
repository. The following sections describe these images including their contents, relationships,
and versioning strategy.
### jupyter/base-notebook ### jupyter/base-notebook
@@ -24,8 +22,8 @@ and versioning strategy.
[Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/base-notebook/Dockerfile) | [Dockerfile commit history](https://github.com/jupyter/docker-stacks/commits/master/base-notebook/Dockerfile) |
[Docker Hub image tags](https://hub.docker.com/r/jupyter/base-notebook/tags/) [Docker Hub image tags](https://hub.docker.com/r/jupyter/base-notebook/tags/)
`jupyter/base-notebook` is a small image supporting the `jupyter/base-notebook` is a small image supporting the [options common across all core stacks](common.md).
[options common across all core stacks](common.md). It is the basis for all other stacks. It is the basis for all other stacks.
- Minimally-functional Jupyter Notebook server (e.g., no LaTeX support for saving notebooks as PDFs) - Minimally-functional Jupyter Notebook server (e.g., no LaTeX support for saving notebooks as PDFs)
- [Miniforge](https://github.com/conda-forge/miniforge) Python 3.x in `/opt/conda` with two package managers - [Miniforge](https://github.com/conda-forge/miniforge) Python 3.x in `/opt/conda` with two package managers
@@ -37,8 +35,7 @@ and versioning strategy.
with ownership over the `/home/jovyan` and `/opt/conda` paths with ownership over the `/home/jovyan` and `/opt/conda` paths
- `tini` as the container entrypoint and a `start-notebook.sh` script as the default command - `tini` as the container entrypoint and a `start-notebook.sh` script as the default command
- A `start-singleuser.sh` script useful for launching containers in JupyterHub - A `start-singleuser.sh` script useful for launching containers in JupyterHub
- A `start.sh` script useful for running alternative commands in the container (e.g. `ipython`, - A `start.sh` script useful for running alternative commands in the container (e.g. `ipython`, `jupyter kernelgateway`, `jupyter lab`)
`jupyter kernelgateway`, `jupyter lab`)
- Options for a self-signed HTTPS certificate and passwordless sudo - Options for a self-signed HTTPS certificate and passwordless sudo
### jupyter/minimal-notebook ### jupyter/minimal-notebook
@@ -188,29 +185,26 @@ communities.
### Image Relationships ### Image Relationships
The following diagram depicts the build dependency tree of the core images. (i.e., the `FROM` The following diagram depicts the build dependency tree of the core images. (i.e., the `FROM` statements in their Dockerfiles).
statements in their Dockerfiles). Any given image inherits the complete content of all ancestor Any given image inherits the complete content of all ancestor images pointing to it.
images pointing to it.
[![Image inheritance [![Image inheritance
diagram](../images/inherit.svg)](http://interactive.blockdiag.com/?compression=deflate&src=eJyFzTEPgjAQhuHdX9Gws5sQjGzujsaYKxzmQrlr2msMGv-71K0srO_3XGud9NNA8DSfgzESCFlBSdi0xkvQAKTNugw4QnL6GIU10hvX-Zh7Z24OLLq2SjaxpvP10lX35vCf6pOxELFmUbQiUz4oQhYzMc3gCrRt2cWe_FKosmSjyFHC6OS1AwdQWCtyj7sfh523_BI9hKlQ25YdOFdv5fcH0kiEMA) diagram](../images/inherit.svg)](http://interactive.blockdiag.com/?compression=deflate&src=eJyFzTEPgjAQhuHdX9Gws5sQjGzujsaYKxzmQrlr2msMGv-71K0srO_3XGud9NNA8DSfgzESCFlBSdi0xkvQAKTNugw4QnL6GIU10hvX-Zh7Z24OLLq2SjaxpvP10lX35vCf6pOxELFmUbQiUz4oQhYzMc3gCrRt2cWe_FKosmSjyFHC6OS1AwdQWCtyj7sfh523_BI9hKlQ25YdOFdv5fcH0kiEMA)
### Builds ### Builds
Pull requests to the `jupyter/docker-stacks` repository trigger builds of all images on GitHub Pull requests to the `jupyter/docker-stacks` repository trigger builds of all images on GitHub Actions.
Actions. These images are for testing purposes only and are not saved for further use. When pull requests These images are for testing purposes only and are not saved for further use.
merge to master, all images rebuild on Docker Hub and become available to `docker pull` from When pull requests merge to master, all images rebuild on Docker Hub and become available to `docker pull` from Docker Hub.
Docker Hub.
### Versioning ### Versioning
The `latest` tag in each Docker Hub repository tracks the master branch `HEAD` reference on GitHub. The `latest` tag in each Docker Hub repository tracks the master branch `HEAD` reference on GitHub.
`latest` is a moving target, by definition, and will have backward-incompatible changes regularly. `latest` is a moving target, by definition, and will have backward-incompatible changes regularly.
Every image on Docker Hub also receives a 12-character tag which corresponds with the git commit SHA Every image on Docker Hub also receives a 12-character tag which corresponds with the git commit SHA that triggered the image build.
that triggered the image build. You can inspect the state of the `jupyter/docker-stacks` repository You can inspect the state of the `jupyter/docker-stacks` repository for that commit to review the definition of the image
for that commit to review the definition of the image (e.g., images with tag `33add21fab64` were built (e.g., images with tag `33add21fab64` were built from <https://github.com/jupyter/docker-stacks/tree/33add21fab64>.
from <https://github.com/jupyter/docker-stacks/tree/33add21fab64>.
You must refer to git-SHA image tags when stability and reproducibility are important in your work. You must refer to git-SHA image tags when stability and reproducibility are important in your work.
(e.g. `FROM jupyter/scipy-notebook:33add21fab64`, `docker run -it --rm jupyter/scipy-notebook:33add21fab64`). (e.g. `FROM jupyter/scipy-notebook:33add21fab64`, `docker run -it --rm jupyter/scipy-notebook:33add21fab64`).
@@ -220,18 +214,17 @@ You should only use `latest` when a one-off container instance is acceptable
## Community Stacks ## Community Stacks
The core stacks are just a tiny sample of what's possible when combining Jupyter with other The core stacks are just a tiny sample of what's possible when combining Jupyter with other
technologies. We encourage members of the Jupyter community to create their own stacks based on the technologies.
We encourage members of the Jupyter community to create their own stacks based on the
core images and link them below. core images and link them below.
- [csharp-notebook is a community Jupyter Docker Stack image. Try C# in Jupyter Notebooks](https://github.com/tlinnet/csharp-notebook). - [csharp-notebook is a community Jupyter Docker Stack image. Try C# in Jupyter Notebooks](https://github.com/tlinnet/csharp-notebook).
The image includes more than 200 Jupyter Notebooks with example C# code and can readily be tried The image includes more than 200 Jupyter Notebooks with example C# code and can readily be tried online via mybinder.org.
online via mybinder.org. Click here to launch Try it on [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tlinnet/csharp-notebook/master).
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tlinnet/csharp-notebook/master).
- [education-notebook is a community Jupyter Docker Stack image](https://github.com/umsi-mads/education-notebook). - [education-notebook is a community Jupyter Docker Stack image](https://github.com/umsi-mads/education-notebook).
The image includes nbgrader and RISE on top of the datascience-notebook image. Click here to The image includes nbgrader and RISE on top of the datascience-notebook image.
launch it on Try it on [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/umsi-mads/education-notebook/master).
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/umsi-mads/education-notebook/master).
- **crosscompass/ihaskell-notebook** - **crosscompass/ihaskell-notebook**
@@ -242,36 +235,34 @@ core images and link them below.
`crosscompass/ihaskell-notebook` is based on [IHaskell](https://github.com/gibiansky/IHaskell). `crosscompass/ihaskell-notebook` is based on [IHaskell](https://github.com/gibiansky/IHaskell).
Includes popular packages and example notebooks. Includes popular packages and example notebooks.
Try it on Try it on [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jamesdbrock/learn-you-a-haskell-notebook/master?urlpath=lab/tree/ihaskell_examples/ihaskell/IHaskell.ipynb)
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jamesdbrock/learn-you-a-haskell-notebook/master?urlpath=lab/tree/ihaskell_examples/ihaskell/IHaskell.ipynb)
- [java-notebook is a community Jupyter Docker Stack image](https://github.com/jbindinga/java-notebook). - [java-notebook is a community Jupyter Docker Stack image](https://github.com/jbindinga/java-notebook).
The image includes [IJava](https://github.com/SpencerPark/IJava) kernel on top of the The image includes [IJava](https://github.com/SpencerPark/IJava) kernel on top of the minimal-notebook image.
minimal-notebook image. Click here to launch it on Try it on [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jbindinga/java-notebook/master).
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jbindinga/java-notebook/master).
- [sage-notebook](https://github.com/sharpTrick/sage-notebook) is a community Jupyter Docker Stack - [sage-notebook](https://github.com/sharpTrick/sage-notebook)
image with the [sagemath](https://www.sagemath.org) kernel on top of the minimal-notebook image. Click is a community Jupyter Docker Stack image with the [sagemath](https://www.sagemath.org) kernel on top of the minimal-notebook image.
here to launch it on Try it on [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/sharpTrick/sage-notebook/master).
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/sharpTrick/sage-notebook/master).
- [GPU-Jupyter](https://github.com/iot-salzburg/gpu-jupyter/): Leverage Jupyter Notebooks with the - [GPU-Jupyter](https://github.com/iot-salzburg/gpu-jupyter/): Leverage Jupyter Notebooks with the
power of your NVIDIA GPU and perform GPU calculations using Tensorflow and Pytorch in power of your NVIDIA GPU and perform GPU calculations using Tensorflow and Pytorch in collaborative notebooks.
collaborative notebooks. This is done by generating a Dockerfile, that consists of the This is done by generating a Dockerfile, that consists of the **nvidia/cuda** base image,
**nvidia/cuda** base image, the well-maintained **docker-stacks** that is integrated as submodule the well-maintained **docker-stacks** that is integrated as submodule and
and GPU-able libraries like **Tensorflow**, **Keras** and **PyTorch** on top of it. GPU-able libraries like **Tensorflow**, **Keras** and **PyTorch** on top of it.
- [PRP GPU Jupyter repo](https://gitlab.nautilus.optiputer.net/prp/jupyter-stack/-/tree/prp) and [Registry](https://gitlab.nautilus.optiputer.net/prp/jupyter-stack/container_registry): PRP (Pacific Research Platform) maintained registry for jupyter stack based on NVIDIA CUDA-enabled image. Added the PRP image with Pytorch and some other python packages, and GUI Desktop notebook based on <https://github.com/jupyterhub/jupyter-remote-desktop-proxy>. - [PRP GPU Jupyter repo](https://gitlab.nautilus.optiputer.net/prp/jupyter-stack/-/tree/prp) and [Registry](https://gitlab.nautilus.optiputer.net/prp/jupyter-stack/container_registry)
PRP (Pacific Research Platform) maintained registry for jupyter stack based on NVIDIA CUDA-enabled image.
Added the PRP image with Pytorch and some other python packages, and GUI Desktop notebook based on <https://github.com/jupyterhub/jupyter-remote-desktop-proxy>.
- [cgspatial-notebook](https://github.com/SCiO-systems/cgspatial-notebook) is a community Jupyter - [cgspatial-notebook](https://github.com/SCiO-systems/cgspatial-notebook) is a community Jupyter Docker Stack image.
Docker Stack image. The image includes major geospatial Python & R libraries on top of the The image includes major geospatial Python & R libraries on top of the datascience-notebook image.
datascience-notebook image. Try it on Try it on [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/SCiO-systems/cgspatial-notebook/master)
binder:[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/SCiO-systems/cgspatial-notebook/master)
- [kotlin-notebook](https://github.com/knonm/kotlin-notebook) is a community Jupyter - [kotlin-notebook](https://github.com/knonm/kotlin-notebook) is a community Jupyter Docker Stack image.
Docker Stack image. The image includes [Kotlin kernel for Jupyter/IPython](https://github.com/Kotlin/kotlin-jupyter) on top of the The image includes [Kotlin kernel for Jupyter/IPython](https://github.com/Kotlin/kotlin-jupyter) on top of the
`base-notebook` image. Try it on `base-notebook` image.
Binder: [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/knonm/kotlin-notebook/main) Try it on [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/knonm/kotlin-notebook/main)
See the [contributing guide](../contributing/stacks.md) for information about how to create your own See the [contributing guide](../contributing/stacks.md) for information about how to create your own
Jupyter Docker Stack. Jupyter Docker Stack.

22
docs/using/specifics.md
View File

@@ -6,13 +6,18 @@ This page provides details about features specific to one or more images.
### Specific Docker Image Options ### Specific Docker Image Options
- `-p 4040:4040` - The `jupyter/pyspark-notebook` and `jupyter/all-spark-notebook` images open [SparkUI (Spark Monitoring and Instrumentation UI)](https://spark.apache.org/docs/latest/monitoring.html) at default port `4040`, this option map `4040` port inside docker container to `4040` port on host machine . Note every new spark context that is created is put onto an incrementing port (ie. 4040, 4041, 4042, etc.), and it might be necessary to open multiple ports. For example: `docker run -d -p 8888:8888 -p 4040:4040 -p 4041:4041 jupyter/pyspark-notebook`. - `-p 4040:4040` - The `jupyter/pyspark-notebook` and `jupyter/all-spark-notebook` images open
[SparkUI (Spark Monitoring and Instrumentation UI)](https://spark.apache.org/docs/latest/monitoring.html) at default port `4040`,
this option map `4040` port inside docker container to `4040` port on host machine.
Note every new spark context that is created is put onto an incrementing port (ie. 4040, 4041, 4042, etc.), and it might be necessary to open multiple ports.
For example: `docker run -d -p 8888:8888 -p 4040:4040 -p 4041:4041 jupyter/pyspark-notebook`.
### Build an Image with a Different Version of Spark ### Build an Image with a Different Version of Spark
You can build a `pyspark-notebook` image (and also the downstream `all-spark-notebook` image) with a different version of Spark by overriding the default value of the following arguments at build time. You can build a `pyspark-notebook` image (and also the downstream `all-spark-notebook` image) with a different version of Spark by overriding the default value of the following arguments at build time.
- Spark distribution is defined by the combination of the Spark and the Hadoop version and verified by the package checksum, see [Download Apache Spark](https://spark.apache.org/downloads.html) and the [archive repo](https://archive.apache.org/dist/spark/) for more information. - Spark distribution is defined by the combination of the Spark and the Hadoop version and verified by the package checksum,
see [Download Apache Spark](https://spark.apache.org/downloads.html) and the [archive repo](https://archive.apache.org/dist/spark/) for more information.
- `spark_version`: The Spark version to install (`3.0.0`). - `spark_version`: The Spark version to install (`3.0.0`).
- `hadoop_version`: The Hadoop version (`3.2`). - `hadoop_version`: The Hadoop version (`3.2`).
- `spark_checksum`: The package checksum (`BFE4540...`). - `spark_checksum`: The package checksum (`BFE4540...`).
@@ -46,7 +51,8 @@ docker run -it --rm jupyter/pyspark-notebook:spark-2.4.7 pyspark --version
### Usage Examples ### Usage Examples
The `jupyter/pyspark-notebook` and `jupyter/all-spark-notebook` images support the use of [Apache Spark](https://spark.apache.org/) in Python, R, and Scala notebooks. The following sections provide some examples of how to get started using them. The `jupyter/pyspark-notebook` and `jupyter/all-spark-notebook` images support the use of [Apache Spark](https://spark.apache.org/) in Python, R, and Scala notebooks.
The following sections provide some examples of how to get started using them.
#### Using Spark Local Mode #### Using Spark Local Mode
@@ -133,16 +139,18 @@ Connection to Spark Cluster on **[Standalone Mode](https://spark.apache.org/docs
deployed, run the same version of Spark. deployed, run the same version of Spark.
1. [Deploy Spark in Standalone Mode](https://spark.apache.org/docs/latest/spark-standalone.html). 1. [Deploy Spark in Standalone Mode](https://spark.apache.org/docs/latest/spark-standalone.html).
2. Run the Docker container with `--net=host` in a location that is network addressable by all of 2. Run the Docker container with `--net=host` in a location that is network addressable by all of
your Spark workers. (This is a [Spark networking your Spark workers.
requirement](https://spark.apache.org/docs/latest/cluster-overview.html#components).) (This is a [Spark networking requirement](https://spark.apache.org/docs/latest/cluster-overview.html#components).)
- NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`. See <https://github.com/jupyter/docker-stacks/issues/64> for details. - NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`.
See <https://github.com/jupyter/docker-stacks/issues/64> for details.
**Note**: In the following examples we are using the Spark master URL `spark://master:7077` that shall be replaced by the URL of the Spark master. **Note**: In the following examples we are using the Spark master URL `spark://master:7077` that shall be replaced by the URL of the Spark master.
##### Standalone Mode in Python ##### Standalone Mode in Python
The **same Python version** needs to be used on the notebook (where the driver is located) and on the Spark workers. The **same Python version** needs to be used on the notebook (where the driver is located) and on the Spark workers.
The python version used at driver and worker side can be adjusted by setting the environment variables `PYSPARK_PYTHON` and / or `PYSPARK_DRIVER_PYTHON`, see [Spark Configuration][spark-conf] for more information. The python version used at driver and worker side can be adjusted by setting the environment variables `PYSPARK_PYTHON` and / or `PYSPARK_DRIVER_PYTHON`,
see [Spark Configuration][spark-conf] for more information.
```python ```python
from pyspark.sql import SparkSession from pyspark.sql import SparkSession

24
examples/docker-compose/README.md
View File

@@ -38,7 +38,8 @@ notebook/down.sh
### How do I specify which docker-stack notebook image to deploy? ### How do I specify which docker-stack notebook image to deploy?
You can customize the docker-stack notebook image to deploy by modifying the `notebook/Dockerfile`. For example, you can build and deploy a `jupyter/all-spark-notebook` by modifying the Dockerfile like so: You can customize the docker-stack notebook image to deploy by modifying the `notebook/Dockerfile`.
For example, you can build and deploy a `jupyter/all-spark-notebook` by modifying the Dockerfile like so:
```dockerfile ```dockerfile
FROM jupyter/all-spark-notebook:33add21fab64 FROM jupyter/all-spark-notebook:33add21fab64
@@ -85,7 +86,9 @@ NAME=your-notebook PORT=9001 WORK_VOLUME=our-work notebook/up.sh
### How do I run over HTTPS? ### How do I run over HTTPS?
To run the notebook server with a self-signed certificate, pass the `--secure` option to the `up.sh` script. You must also provide a password, which will be used to secure the notebook server. You can specify the password by setting the `PASSWORD` environment variable, or by passing it to the `up.sh` script. To run the notebook server with a self-signed certificate, pass the `--secure` option to the `up.sh` script.
You must also provide a password, which will be used to secure the notebook server.
You can specify the password by setting the `PASSWORD` environment variable, or by passing it to the `up.sh` script.
```bash ```bash
PASSWORD=a_secret notebook/up.sh --secure PASSWORD=a_secret notebook/up.sh --secure
@@ -98,7 +101,8 @@ notebook/up.sh --secure --password a_secret
Sure. If you want to secure access to publicly addressable notebook containers, you can generate a free certificate using the [Let's Encrypt](https://letsencrypt.org) service. Sure. If you want to secure access to publicly addressable notebook containers, you can generate a free certificate using the [Let's Encrypt](https://letsencrypt.org) service.
This example includes the `bin/letsencrypt.sh` script, which runs the `letsencrypt` client to create a full-chain certificate and private key, and stores them in a Docker volume. _Note:_ The script hard codes several `letsencrypt` options, one of which automatically agrees to the Let's Encrypt Terms of Service. This example includes the `bin/letsencrypt.sh` script, which runs the `letsencrypt` client to create a full-chain certificate and private key, and stores them in a Docker volume.
_Note:_ The script hard codes several `letsencrypt` options, one of which automatically agrees to the Let's Encrypt Terms of Service.
The following command will create a certificate chain and store it in a Docker volume named `mydomain-secrets`. The following command will create a certificate chain and store it in a Docker volume named `mydomain-secrets`.
@@ -108,7 +112,8 @@ FQDN=host.mydomain.com EMAIL=myemail@somewhere.com \
bin/letsencrypt.sh bin/letsencrypt.sh
``` ```
Now run `up.sh` with the `--letsencrypt` option. You must also provide the name of the secrets volume and a password. Now run `up.sh` with the `--letsencrypt` option.
You must also provide the name of the secrets volume and a password.
```bash ```bash
PASSWORD=a_secret SECRETS_VOLUME=mydomain-secrets notebook/up.sh --letsencrypt PASSWORD=a_secret SECRETS_VOLUME=mydomain-secrets notebook/up.sh --letsencrypt
@@ -117,7 +122,9 @@ PASSWORD=a_secret SECRETS_VOLUME=mydomain-secrets notebook/up.sh --letsencrypt
notebook/up.sh --letsencrypt --password a_secret --secrets mydomain-secrets notebook/up.sh --letsencrypt --password a_secret --secrets mydomain-secrets
``` ```
Be aware that Let's Encrypt has a pretty [low rate limit per domain](https://community.letsencrypt.org/t/public-beta-rate-limits/4772/3) at the moment. You can avoid exhausting your limit by testing against the Let's Encrypt staging servers. To hit their staging servers, set the environment variable `CERT_SERVER=--staging`. Be aware that Let's Encrypt has a pretty [low rate limit per domain](https://community.letsencrypt.org/t/public-beta-rate-limits/4772/3) at the moment.
You can avoid exhausting your limit by testing against the Let's Encrypt staging servers.
To hit their staging servers, set the environment variable `CERT_SERVER=--staging`.
```bash ```bash
FQDN=host.mydomain.com EMAIL=myemail@somewhere.com \ FQDN=host.mydomain.com EMAIL=myemail@somewhere.com \
@@ -125,11 +132,14 @@ FQDN=host.mydomain.com EMAIL=myemail@somewhere.com \
bin/letsencrypt.sh bin/letsencrypt.sh
``` ```
Also, be aware that Let's Encrypt certificates are short lived (90 days). If you need them for a longer period of time, you'll need to manually setup a cron job to run the renewal steps. (You can reuse the command above.) Also, be aware that Let's Encrypt certificates are short lived (90 days).
If you need them for a longer period of time, you'll need to manually setup a cron job to run the renewal steps.
(You can reuse the command above.)
### Can I deploy to any Docker Machine host? ### Can I deploy to any Docker Machine host?
Yes, you should be able to deploy to any Docker Machine-controlled host. To make it easier to get up and running, this example includes scripts to provision Docker Machines to VirtualBox and IBM SoftLayer, but more scripts are welcome! Yes, you should be able to deploy to any Docker Machine-controlled host.
To make it easier to get up and running, this example includes scripts to provision Docker Machines to VirtualBox and IBM SoftLayer, but more scripts are welcome!
To create a Docker machine using a VirtualBox VM on local desktop: To create a Docker machine using a VirtualBox VM on local desktop:

25
examples/make-deploy/README.md
View File

@@ -48,7 +48,8 @@ make notebook NAME=your-notebook PORT=9001 WORK_VOLUME=our-work
### How do I run over HTTPS? ### How do I run over HTTPS?
Instead of `make notebook`, run `make self-signed-notebook PASSWORD=your_desired_password`. This target gives you a notebook with a self-signed certificate. Instead of `make notebook`, run `make self-signed-notebook PASSWORD=your_desired_password`.
This target gives you a notebook with a self-signed certificate.
### That self-signed certificate is a pain. Let's Encrypt? ### That self-signed certificate is a pain. Let's Encrypt?
@@ -59,15 +60,21 @@ make letsencrypt FQDN=host.mydomain.com EMAIL=myemail@somewhere.com
make letsencrypt-notebook make letsencrypt-notebook
``` ```
The first command creates a Docker volume named after the notebook container with a `-secrets` suffix. It then runs the `letsencrypt` client with a slew of options (one of which has you automatically agreeing to the Let's Encrypt Terms of Service, see the Makefile). The second command mounts the secrets volume and configures Jupyter to use the full-chain certificate and private key. The first command creates a Docker volume named after the notebook container with a `-secrets` suffix.
It then runs the `letsencrypt` client with a slew of options (one of which has you automatically agreeing to the Let's Encrypt Terms of Service, see the Makefile).
The second command mounts the secrets volume and configures Jupyter to use the full-chain certificate and private key.
Be aware: Let's Encrypt has a pretty [low rate limit per domain](https://community.letsencrypt.org/t/public-beta-rate-limits/4772/3) at the moment. You can avoid exhausting your limit by testing against the Let's Encrypt staging servers. To hit their staging servers, set the environment variable `CERT_SERVER=--staging`. Be aware: Let's Encrypt has a pretty [low rate limit per domain](https://community.letsencrypt.org/t/public-beta-rate-limits/4772/3) at the moment.
You can avoid exhausting your limit by testing against the Let's Encrypt staging servers.
To hit their staging servers, set the environment variable `CERT_SERVER=--staging`.
```bash ```bash
make letsencrypt FQDN=host.mydomain.com EMAIL=myemail@somewhere.com CERT_SERVER=--staging make letsencrypt FQDN=host.mydomain.com EMAIL=myemail@somewhere.com CERT_SERVER=--staging
``` ```
Also, keep in mind Let's Encrypt certificates are short lived: 90 days at the moment. You'll need to manually setup a cron job to run the renewal steps at the moment. (You can reuse the first command above.) Also, keep in mind Let's Encrypt certificates are short lived: 90 days at the moment.
You'll need to manually setup a cron job to run the renewal steps at the moment.
(You can reuse the first command above.)
### My pip/conda/apt-get installs disappear every time I restart the container. Can I make them permanent? ### My pip/conda/apt-get installs disappear every time I restart the container. Can I make them permanent?
@@ -86,11 +93,14 @@ make image DOCKER_ARGS=--pull
make notebook make notebook
``` ```
The first line pulls the latest version of the Docker image used in the local Dockerfile. Then it rebuilds the local Docker image containing any customizations you may have added to it. The second line kills your currently running notebook container, and starts a fresh one using the new image. The first line pulls the latest version of the Docker image used in the local Dockerfile.
Then it rebuilds the local Docker image containing any customizations you may have added to it.
The second line kills your currently running notebook container, and starts a fresh one using the new image.
### Can I run on another VM provider other than VirtualBox? ### Can I run on another VM provider other than VirtualBox?
Yes. As an example, there's a `softlayer.makefile` included in this repo as an example. You would use it like so: Yes. As an example, there's a `softlayer.makefile` included in this repo as an example.
You would use it like so:
```bash ```bash
make softlayer-vm NAME=myhost \ make softlayer-vm NAME=myhost \
@@ -112,7 +122,8 @@ If you'd like to add support for another docker-machine driver, use the `softlay
### Uh ... make? ### Uh ... make?
Yes, sorry Windows users. It got the job done for a simple example. We can certainly accept other deployment mechanism examples in the parent folder or in other repos. Yes, sorry Windows users. It got the job done for a simple example.
We can certainly accept other deployment mechanism examples in the parent folder or in other repos.
### Are there any other options? ### Are there any other options?

48
examples/openshift/README.md
View File

@@ -2,15 +2,23 @@
This example provides templates for deploying the Jupyter Project docker-stacks images to OpenShift. This example provides templates for deploying the Jupyter Project docker-stacks images to OpenShift.
## Prerequsites ## Prerequisites
Any OpenShift 3 environment. The templates were tested with OpenShift 3.7. It is believed they should work with at least OpenShift 3.6 or later. Any OpenShift 3 environment.
The templates were tested with OpenShift 3.7.
It is believed they should work with at least OpenShift 3.6 or later.
Do be aware that the Jupyter Project docker-stacks images are very large. The OpenShift environment you are using must provide sufficient quota on the per user space for images and the file system for running containers. If the quota is too small, the pulling of the images to a node in the OpenShift cluster when deploying them, will fail due to lack of space. Even if the image is able to be run, if the quota is only just larger than the space required for the image, you will not be able to install many packages into the container before running out of space. Do be aware that the Jupyter Project docker-stacks images are very large.
The OpenShift environment you are using must provide sufficient quota on the per user space for images and the file system for running containers.
If the quota is too small, the pulling of the images to a node in the OpenShift cluster when deploying them, will fail due to lack of space.
Even if the image is able to be run, if the quota is only just larger than the space required for the image, you will not be able to install many packages into the container before running out of space.
OpenShift Online, the public hosted version of OpenShift from Red Hat has a quota of only 3GB for the image and container file system. As a result, only the `minimal-notebook` can be started and there is little space remaining to install additional packages. Although OpenShift Online is suitable for demonstrating these templates work, what you can do in that environment will be limited due to the size of the images. OpenShift Online, the public hosted version of OpenShift from Red Hat has a quota of only 3GB for the image and container file system.
As a result, only the `minimal-notebook` can be started and there is little space remaining to install additional packages.
Although OpenShift Online is suitable for demonstrating these templates work, what you can do in that environment will be limited due to the size of the images.
If you want to experiment with using Jupyter Notebooks in an OpenShift environment, you should instead use [Minishift](https://www.openshift.org/minishift/). Minishift provides you the ability to run OpenShift in a virtual machine on your own local computer. If you want to experiment with using Jupyter Notebooks in an OpenShift environment, you should instead use [Minishift](https://www.openshift.org/minishift/).
Minishift provides you the ability to run OpenShift in a virtual machine on your own local computer.
## Loading the Templates ## Loading the Templates
@@ -22,7 +30,8 @@ oc create -f https://raw.githubusercontent.com/jupyter-on-openshift/docker-stack
This should create the `jupyter-notebook` template This should create the `jupyter-notebook` template
The template can be used from the command line using the `oc new-app` command, or from the OpenShift web console by selecting _Add to Project_. This `README` is only going to explain deploying from the command line. The template can be used from the command line using the `oc new-app` command, or from the OpenShift web console by selecting _Add to Project_.
This `README` is only going to explain deploying from the command line.
## Deploying a Notebook ## Deploying a Notebook
@@ -56,7 +65,8 @@ The output will be similar to:
Run 'oc status' to view your app. Run 'oc status' to view your app.
``` ```
When no template parameters are provided, the name of the deployed notebook will be `notebook`. The image used will be: When no template parameters are provided, the name of the deployed notebook will be `notebook`.
The image used will be:
```lang-none ```lang-none
jupyter/minimal-notebook:latest jupyter/minimal-notebook:latest
@@ -129,7 +139,10 @@ Setting the environment variable will trigger a new deployment and the Jupyter L
## Adding Persistent Storage ## Adding Persistent Storage
You can upload notebooks and other files using the web interface of the notebook. Any uploaded files or changes you make to them will be lost when the notebook instance is restarted. If you want to save your work, you need to add persistent storage to the notebook. To add persistent storage run: You can upload notebooks and other files using the web interface of the notebook.
Any uploaded files or changes you make to them will be lost when the notebook instance is restarted.
If you want to save your work, you need to add persistent storage to the notebook.
To add persistent storage run:
```bash ```bash
oc set volume dc/mynotebook --add \ oc set volume dc/mynotebook --add \
@@ -162,7 +175,8 @@ If you are using a persistent volume, you can also create a configuration file a
This will be merged at the end of the configuration from the config map. This will be merged at the end of the configuration from the config map.
Because the configuration is Python code, ensure any indenting is correct. Any errors in the configuration file will cause the notebook to fail when starting. Because the configuration is Python code, ensure any indenting is correct.
Any errors in the configuration file will cause the notebook to fail when starting.
If the error is in the config map, edit it again to fix it and trigger a new deployment if necessary by running: If the error is in the config map, edit it again to fix it and trigger a new deployment if necessary by running:
@@ -182,7 +196,9 @@ Then run:
oc debug dc/mynotebook oc debug dc/mynotebook
``` ```
to run the notebook in debug mode. This will provide you with an interactive terminal session inside a running container, but the notebook will not have been started. Edit the configuration file in the volume to fix any errors and exit the terminal session. to run the notebook in debug mode.
This will provide you with an interactive terminal session inside a running container, but the notebook will not have been started.
Edit the configuration file in the volume to fix any errors and exit the terminal session.
Start up the notebook again. Start up the notebook again.
@@ -192,7 +208,8 @@ oc scale dc/mynotebook --replicas 1
## Changing the Notebook Password ## Changing the Notebook Password
The password for the notebook is supplied as a template parameter, or if not supplied will be automatically generated by the template. It will be passed into the container through an environment variable. The password for the notebook is supplied as a template parameter, or if not supplied will be automatically generated by the template.
It will be passed into the container through an environment variable.
If you want to change the password, you can do so by editing the environment variable on the deployment configuration. If you want to change the password, you can do so by editing the environment variable on the deployment configuration.
@@ -206,9 +223,11 @@ If using a persistent volume, you could instead setup a password in the file `/h
## Deploying from a Custom Image ## Deploying from a Custom Image
If you want to deploy a custom variant of the Jupyter Project docker-stacks images, you can replace the image name with that of your own. If the image is not stored on Docker Hub, but some other public image registry, prefix the name of the image with the image registry host details. If you want to deploy a custom variant of the Jupyter Project docker-stacks images, you can replace the image name with that of your own.
If the image is not stored on Docker Hub, but some other public image registry, prefix the name of the image with the image registry host details.
If the image is in your OpenShift project, because you imported the image into OpenShift, or used the docker build strategy of OpenShift to build a derived custom image, you can use the name of the image stream for the image name, including any image tag if necessary. If the image is in your OpenShift project, because you imported the image into OpenShift, or used the docker build strategy of OpenShift to build a derived custom image,
you can use the name of the image stream for the image name, including any image tag if necessary.
This can be illustrated by first importing an image into the OpenShift project. This can be illustrated by first importing an image into the OpenShift project.
@@ -225,4 +244,5 @@ oc new-app --template jupyter-notebook \
--param NOTEBOOK_PASSWORD=mypassword --param NOTEBOOK_PASSWORD=mypassword
``` ```
Importing an image into OpenShift before deploying it means that when a notebook is started, the image need only be pulled from the internal OpenShift image registry rather than Docker Hub for each deployment. Because the images are so large, this can speed up deployments when the image hasn't previously been deployed to a node in the OpenShift cluster. Importing an image into OpenShift before deploying it means that when a notebook is started, the image need only be pulled from the internal OpenShift image registry rather than Docker Hub for each deployment.
Because the images are so large, this can speed up deployments when the image hasn't previously been deployed to a node in the OpenShift cluster.

57
examples/source-to-image/README.md
View File

@@ -1,8 +1,12 @@
# Custom Jupyter Notebook images # Custom Jupyter Notebook images
This example provides scripts for building custom Jupyter Notebook images containing notebooks, data files, and with Python packages required by the notebooks already installed. The scripts provided work with the Source-to-Image tool and you can create the images from the command line on your own computer. Templates are also provided to enable running builds in OpenShift, as well as deploying the resulting image to OpenShift to make it available. This example provides scripts for building custom Jupyter Notebook images containing notebooks, data files, and with Python packages required by the notebooks already installed.
The scripts provided work with the Source-to-Image tool and you can create the images from the command line on your own computer.
Templates are also provided to enable running builds in OpenShift, as well as deploying the resulting image to OpenShift to make it available.
The build scripts, when used with the Source-to-Image tool, provide similar capabilities to `repo2docker`. When builds are run under OpenShift with the supplied templates, it provides similar capabilities to `mybinder.org`, but where notebook instances are deployed in your existing OpenShift project and JupyterHub is not required. The build scripts, when used with the Source-to-Image tool, provide similar capabilities to `repo2docker`.
When builds are run under OpenShift with the supplied templates, it provides similar capabilities to `mybinder.org`,
but where notebook instances are deployed in your existing OpenShift project and JupyterHub is not required.
For separate examples of using JupyterHub with OpenShift, see the project: For separate examples of using JupyterHub with OpenShift, see the project:
@@ -10,13 +14,16 @@ For separate examples of using JupyterHub with OpenShift, see the project:
## Source-to-Image Project ## Source-to-Image Project
Source-to-Image (S2I) is an open source project which provides a tool for creating container images. It works by taking a base image, injecting additional source code or files into a running container created from the base image, and running a builder script in the container to process the source code or files to prepare the new image. Source-to-Image (S2I) is an open source project which provides a tool for creating container images.
It works by taking a base image, injecting additional source code or files into a running container created from the base image,
and running a builder script in the container to process the source code or files to prepare the new image.
Details on the S2I tool, and executable binaries for Linux, macOS and Windows, can be found on GitHub at: Details on the S2I tool, and executable binaries for Linux, macOS and Windows, can be found on GitHub at:
- <https://github.com/openshift/source-to-image> - <https://github.com/openshift/source-to-image>
The tool is standalone, and can be used on any system which provides a docker daemon for running containers. To provide an end-to-end capability to build and deploy applications in containers, support for S2I is also integrated into container platforms such as OpenShift. The tool is standalone, and can be used on any system which provides a docker daemon for running containers.
To provide an end-to-end capability to build and deploy applications in containers, support for S2I is also integrated into container platforms such as OpenShift.
## Getting Started with S2I ## Getting Started with S2I
@@ -31,7 +38,9 @@ s2i build \
notebook-examples notebook-examples
``` ```
This example command will pull down the Git repository <https://github.com/jupyter/notebook> and build the image `notebook-examples` using the files contained in the `docs/source/examples/Notebook` directory of that Git repository. The base image which the files will be combined with is `jupyter/minimal-notebook:latest`, but you can specify any of the Jupyter Project `docker-stacks` images as the base image. This example command will pull down the Git repository <https://github.com/jupyter/notebook>
and build the image `notebook-examples` using the files contained in the `docs/source/examples/Notebook` directory of that Git repository.
The base image which the files will be combined with is `jupyter/minimal-notebook:latest`, but you can specify any of the Jupyter Project `docker-stacks` images as the base image.
The resulting image from running the command can be seen by running `docker images` command: The resulting image from running the command can be seen by running `docker images` command:
@@ -66,11 +75,15 @@ Open your browser on the URL displayed, and you will find the notebooks from the
## The S2I Builder Scripts ## The S2I Builder Scripts
Normally when using S2I, the base image would be S2I enabled and contain the builder scripts needed to prepare the image and define how the application in the image should be run. As the Jupyter Project `docker-stacks` images are not S2I enabled (although they could be), in the above example the `--scripts-url` option has been used to specify that the example builder scripts contained in this directory of this Git repository should be used. Normally when using S2I, the base image would be S2I enabled and contain the builder scripts needed to prepare the image and define how the application in the image should be run.
As the Jupyter Project `docker-stacks` images are not S2I enabled (although they could be),
in the above example the `--scripts-url` option has been used to specify that the example builder scripts contained in this directory of this Git repository should be used.
Using the `--scripts-url` option, the builder scripts can be hosted on any HTTP server, or you could also use builder scripts local to your computer file using an appropriate `file://` format URI argument to `--scripts-url`. Using the `--scripts-url` option, the builder scripts can be hosted on any HTTP server,
or you could also use builder scripts local to your computer file using an appropriate `file://` format URI argument to `--scripts-url`.
The builder scripts in this directory of this repository are `assemble` and `run` and are provided as examples of what can be done. You can use the scripts as is, or create your own. The builder scripts in this directory of this repository are `assemble` and `run` and are provided as examples of what can be done.
You can use the scripts as is, or create your own.
The supplied `assemble` script performs a few key steps. The supplied `assemble` script performs a few key steps.
@@ -97,7 +110,8 @@ fi
This determines whether a `environment.yml` or `requirements.txt` file exists with the files and if so, runs the appropriate package management tool to install any Python packages listed in those files. This determines whether a `environment.yml` or `requirements.txt` file exists with the files and if so, runs the appropriate package management tool to install any Python packages listed in those files.
This means that so long as a set of notebook files provides one of these files listing what Python packages they need, those packages will be automatically installed into the image so they are available when the image is run. This means that so long as a set of notebook files provides one of these files listing what Python packages they need,
those packages will be automatically installed into the image so they are available when the image is run.
A final step is: A final step is:
@@ -106,9 +120,14 @@ fix-permissions "${CONDA_DIR}"
fix-permissions "/home/${NB_USER}" fix-permissions "/home/${NB_USER}"
``` ```
This fixes up permissions on any new files created by the build. This is necessary to ensure that when the image is run, you can still install additional files. This is important for when an image is run in `sudo` mode, or it is hosted in a more secure container platform such as Kubernetes/OpenShift where it will be run as a set user ID that isn't known in advance. This fixes up permissions on any new files created by the build.
This is necessary to ensure that when the image is run, you can still install additional files.
This is important for when an image is run in `sudo` mode, or it is hosted in a more secure container platform such as Kubernetes/OpenShift where it will be run as a set user ID that isn't known in advance.
As long as you preserve the first and last set of steps, you can do whatever you want in the `assemble` script to install packages, create files etc. Do be aware though that S2I builds do not run as `root` and so you cannot install additional system packages. If you need to install additional system packages, use a `Dockerfile` and normal `docker build` to first create a new custom base image from the Jupyter Project `docker-stacks` images, with the extra system packages, and then use that image with the S2I build to combine your notebooks and have Python packages installed. As long as you preserve the first and last set of steps, you can do whatever you want in the `assemble` script to install packages, create files etc.
Do be aware though that S2I builds do not run as `root` and so you cannot install additional system packages.
If you need to install additional system packages, use a `Dockerfile` and normal `docker build` to first create a new custom base image from the Jupyter Project `docker-stacks` images,
with the extra system packages, and then use that image with the S2I build to combine your notebooks and have Python packages installed.
The `run` script in this directory is very simple and just runs the notebook application. The `run` script in this directory is very simple and just runs the notebook application.
@@ -118,7 +137,9 @@ exec start-notebook.sh "$@"
## Integration with OpenShift ## Integration with OpenShift
The OpenShift platform provides integrated support for S2I type builds. Templates are provided for using the S2I build mechanism with the scripts in this directory. To load the templates run: The OpenShift platform provides integrated support for S2I type builds.
Templates are provided for using the S2I build mechanism with the scripts in this directory.
To load the templates run:
```bash ```bash
oc create -f https://raw.githubusercontent.com/jupyter/docker-stacks/master/examples/source-to-image/templates.json oc create -f https://raw.githubusercontent.com/jupyter/docker-stacks/master/examples/source-to-image/templates.json
@@ -131,7 +152,8 @@ jupyter-notebook-builder
jupyter-notebook-quickstart jupyter-notebook-quickstart
``` ```
The templates can be used from the OpenShift web console or command line. This `README` is only going to explain deploying from the command line. The templates can be used from the OpenShift web console or command line.
This `README` is only going to explain deploying from the command line.
To use the OpenShift command line to build into an image, and deploy, the set of notebooks used above, run: To use the OpenShift command line to build into an image, and deploy, the set of notebooks used above, run:
@@ -144,9 +166,11 @@ oc new-app --template jupyter-notebook-quickstart \
--param NOTEBOOK_PASSWORD=mypassword --param NOTEBOOK_PASSWORD=mypassword
``` ```
You can provide a password using the `NOTEBOOK_PASSWORD` parameter. If you don't set that parameter, a password will be generated, with it being displayed by the `oc new-app` command. You can provide a password using the `NOTEBOOK_PASSWORD` parameter.
If you don't set that parameter, a password will be generated, with it being displayed by the `oc new-app` command.
Once the image has been built, it will be deployed. To see the hostname for accessing the notebook, run `oc get routes`. Once the image has been built, it will be deployed.
To see the hostname for accessing the notebook, run `oc get routes`.
```lang-none ```lang-none
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
@@ -155,6 +179,7 @@ notebook-examples notebook-examples-jupyter.abcd.pro-us-east-1.openshiftapps.c
As the deployment will use a secure connection, the URL for accessing the notebook in this case would be <https://notebook-examples-jupyter.abcd.pro-us-east-1.openshiftapps.com>. As the deployment will use a secure connection, the URL for accessing the notebook in this case would be <https://notebook-examples-jupyter.abcd.pro-us-east-1.openshiftapps.com>.
If you only want to build an image but not deploy it, you can use the `jupyter-notebook-builder` template. You can then deploy it using the `jupyter-notebook` template provided with the [openshift](../openshift) examples directory. If you only want to build an image but not deploy it, you can use the `jupyter-notebook-builder` template.
You can then deploy it using the `jupyter-notebook` template provided with the [openshift](../openshift) examples directory.
See the `openshift` examples directory for further information on customizing configuration for a Jupyter Notebook deployment and deleting a deployment. See the `openshift` examples directory for further information on customizing configuration for a Jupyter Notebook deployment and deleting a deployment.