hazza/docker-stacks
1
0
Fork 0
mirror of https://github.com/jupyter/docker-stacks.git synced 2025-10-07 18:14:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Limit markdown line length to 200

Browse Source
This commit is contained in:
Ayaz Salikhov
2021-06-21 12:59:21 +03:00
parent 34992fea4c
commit 334381dfe2
22 changed files with 454 additions and 293 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
examples/source-to-image/README.md
View File

@@ -1,8 +1,12 @@
# Custom Jupyter Notebook images
This example provides scripts for building custom Jupyter Notebook images containing notebooks, data files, and with Python packages required by the notebooks already installed. The scripts provided work with the Source-to-Image tool and you can create the images from the command line on your own computer. Templates are also provided to enable running builds in OpenShift, as well as deploying the resulting image to OpenShift to make it available.
This example provides scripts for building custom Jupyter Notebook images containing notebooks, data files, and with Python packages required by the notebooks already installed.
The scripts provided work with the Source-to-Image tool and you can create the images from the command line on your own computer.
Templates are also provided to enable running builds in OpenShift, as well as deploying the resulting image to OpenShift to make it available.
The build scripts, when used with the Source-to-Image tool, provide similar capabilities to `repo2docker`. When builds are run under OpenShift with the supplied templates, it provides similar capabilities to `mybinder.org`, but where notebook instances are deployed in your existing OpenShift project and JupyterHub is not required.
The build scripts, when used with the Source-to-Image tool, provide similar capabilities to `repo2docker`.
When builds are run under OpenShift with the supplied templates, it provides similar capabilities to `mybinder.org`,
but where notebook instances are deployed in your existing OpenShift project and JupyterHub is not required.
For separate examples of using JupyterHub with OpenShift, see the project:
@@ -10,13 +14,16 @@ For separate examples of using JupyterHub with OpenShift, see the project:
## Source-to-Image Project
Source-to-Image (S2I) is an open source project which provides a tool for creating container images. It works by taking a base image, injecting additional source code or files into a running container created from the base image, and running a builder script in the container to process the source code or files to prepare the new image.
Source-to-Image (S2I) is an open source project which provides a tool for creating container images.
It works by taking a base image, injecting additional source code or files into a running container created from the base image,
and running a builder script in the container to process the source code or files to prepare the new image.
Details on the S2I tool, and executable binaries for Linux, macOS and Windows, can be found on GitHub at:
- <https://github.com/openshift/source-to-image>
The tool is standalone, and can be used on any system which provides a docker daemon for running containers. To provide an end-to-end capability to build and deploy applications in containers, support for S2I is also integrated into container platforms such as OpenShift.
The tool is standalone, and can be used on any system which provides a docker daemon for running containers.
To provide an end-to-end capability to build and deploy applications in containers, support for S2I is also integrated into container platforms such as OpenShift.
## Getting Started with S2I
@@ -31,7 +38,9 @@ s2i build \
notebook-examples
```
This example command will pull down the Git repository <https://github.com/jupyter/notebook> and build the image `notebook-examples` using the files contained in the `docs/source/examples/Notebook` directory of that Git repository. The base image which the files will be combined with is `jupyter/minimal-notebook:latest`, but you can specify any of the Jupyter Project `docker-stacks` images as the base image.
This example command will pull down the Git repository <https://github.com/jupyter/notebook>
and build the image `notebook-examples` using the files contained in the `docs/source/examples/Notebook` directory of that Git repository.
The base image which the files will be combined with is `jupyter/minimal-notebook:latest`, but you can specify any of the Jupyter Project `docker-stacks` images as the base image.
The resulting image from running the command can be seen by running `docker images` command:
@@ -66,11 +75,15 @@ Open your browser on the URL displayed, and you will find the notebooks from the
## The S2I Builder Scripts
Normally when using S2I, the base image would be S2I enabled and contain the builder scripts needed to prepare the image and define how the application in the image should be run. As the Jupyter Project `docker-stacks` images are not S2I enabled (although they could be), in the above example the `--scripts-url` option has been used to specify that the example builder scripts contained in this directory of this Git repository should be used.
Normally when using S2I, the base image would be S2I enabled and contain the builder scripts needed to prepare the image and define how the application in the image should be run.
As the Jupyter Project `docker-stacks` images are not S2I enabled (although they could be),
in the above example the `--scripts-url` option has been used to specify that the example builder scripts contained in this directory of this Git repository should be used.
Using the `--scripts-url` option, the builder scripts can be hosted on any HTTP server, or you could also use builder scripts local to your computer file using an appropriate `file://` format URI argument to `--scripts-url`.
Using the `--scripts-url` option, the builder scripts can be hosted on any HTTP server,
or you could also use builder scripts local to your computer file using an appropriate `file://` format URI argument to `--scripts-url`.
The builder scripts in this directory of this repository are `assemble` and `run` and are provided as examples of what can be done. You can use the scripts as is, or create your own.
The builder scripts in this directory of this repository are `assemble` and `run` and are provided as examples of what can be done.
You can use the scripts as is, or create your own.
The supplied `assemble` script performs a few key steps.
@@ -97,7 +110,8 @@ fi
This determines whether a `environment.yml` or `requirements.txt` file exists with the files and if so, runs the appropriate package management tool to install any Python packages listed in those files.
This means that so long as a set of notebook files provides one of these files listing what Python packages they need, those packages will be automatically installed into the image so they are available when the image is run.
This means that so long as a set of notebook files provides one of these files listing what Python packages they need,
those packages will be automatically installed into the image so they are available when the image is run.
A final step is:
@@ -106,9 +120,14 @@ fix-permissions "${CONDA_DIR}"
fix-permissions "/home/${NB_USER}"
```
This fixes up permissions on any new files created by the build. This is necessary to ensure that when the image is run, you can still install additional files. This is important for when an image is run in `sudo` mode, or it is hosted in a more secure container platform such as Kubernetes/OpenShift where it will be run as a set user ID that isn't known in advance.
This fixes up permissions on any new files created by the build.
This is necessary to ensure that when the image is run, you can still install additional files.
This is important for when an image is run in `sudo` mode, or it is hosted in a more secure container platform such as Kubernetes/OpenShift where it will be run as a set user ID that isn't known in advance.
As long as you preserve the first and last set of steps, you can do whatever you want in the `assemble` script to install packages, create files etc. Do be aware though that S2I builds do not run as `root` and so you cannot install additional system packages. If you need to install additional system packages, use a `Dockerfile` and normal `docker build` to first create a new custom base image from the Jupyter Project `docker-stacks` images, with the extra system packages, and then use that image with the S2I build to combine your notebooks and have Python packages installed.
As long as you preserve the first and last set of steps, you can do whatever you want in the `assemble` script to install packages, create files etc.
Do be aware though that S2I builds do not run as `root` and so you cannot install additional system packages.
If you need to install additional system packages, use a `Dockerfile` and normal `docker build` to first create a new custom base image from the Jupyter Project `docker-stacks` images,
with the extra system packages, and then use that image with the S2I build to combine your notebooks and have Python packages installed.
The `run` script in this directory is very simple and just runs the notebook application.
@@ -118,7 +137,9 @@ exec start-notebook.sh "$@"
## Integration with OpenShift
The OpenShift platform provides integrated support for S2I type builds. Templates are provided for using the S2I build mechanism with the scripts in this directory. To load the templates run:
The OpenShift platform provides integrated support for S2I type builds.
Templates are provided for using the S2I build mechanism with the scripts in this directory.
To load the templates run:
```bash
oc create -f https://raw.githubusercontent.com/jupyter/docker-stacks/master/examples/source-to-image/templates.json
@@ -131,7 +152,8 @@ jupyter-notebook-builder
jupyter-notebook-quickstart
```
The templates can be used from the OpenShift web console or command line. This `README` is only going to explain deploying from the command line.
The templates can be used from the OpenShift web console or command line.
This `README` is only going to explain deploying from the command line.
To use the OpenShift command line to build into an image, and deploy, the set of notebooks used above, run:
@@ -144,9 +166,11 @@ oc new-app --template jupyter-notebook-quickstart \
--param NOTEBOOK_PASSWORD=mypassword
```
You can provide a password using the `NOTEBOOK_PASSWORD` parameter. If you don't set that parameter, a password will be generated, with it being displayed by the `oc new-app` command.
You can provide a password using the `NOTEBOOK_PASSWORD` parameter.
If you don't set that parameter, a password will be generated, with it being displayed by the `oc new-app` command.
Once the image has been built, it will be deployed. To see the hostname for accessing the notebook, run `oc get routes`.
Once the image has been built, it will be deployed.
To see the hostname for accessing the notebook, run `oc get routes`.
```lang-none
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
@@ -155,6 +179,7 @@ notebook-examples notebook-examples-jupyter.abcd.pro-us-east-1.openshiftapps.c
As the deployment will use a secure connection, the URL for accessing the notebook in this case would be <https://notebook-examples-jupyter.abcd.pro-us-east-1.openshiftapps.com>.
If you only want to build an image but not deploy it, you can use the `jupyter-notebook-builder` template. You can then deploy it using the `jupyter-notebook` template provided with the [openshift](../openshift) examples directory.
If you only want to build an image but not deploy it, you can use the `jupyter-notebook-builder` template.
You can then deploy it using the `jupyter-notebook` template provided with the [openshift](../openshift) examples directory.
See the `openshift` examples directory for further information on customizing configuration for a Jupyter Notebook deployment and deleting a deployment.