diff --git a/all-spark-notebook/Dockerfile b/all-spark-notebook/Dockerfile index ce624c9b..d421e8ad 100644 --- a/all-spark-notebook/Dockerfile +++ b/all-spark-notebook/Dockerfile @@ -27,8 +27,7 @@ RUN conda install --quiet --yes \ 'r-ggplot2=3.3*' \ 'r-irkernel=1.1*' \ 'r-rcurl=1.98*' \ - 'r-sparklyr=1.6*' \ - && \ + 'r-sparklyr=1.6*' && \ conda clean --all -f -y && \ fix-permissions "${CONDA_DIR}" && \ fix-permissions "/home/${NB_USER}" diff --git a/datascience-notebook/Dockerfile b/datascience-notebook/Dockerfile index 2e26cc49..2eab99b8 100644 --- a/datascience-notebook/Dockerfile +++ b/datascience-notebook/Dockerfile @@ -67,10 +67,13 @@ RUN conda install --quiet --yes \ 'r-randomforest=4.6*' \ 'r-rcurl=1.98*' \ 'r-rmarkdown=2.7*' \ + 'r-rodbc=1.3*' \ 'r-rsqlite=2.2*' \ 'r-shiny=1.6*' \ + 'r-tidymodels=0.1*' \ 'r-tidyverse=1.3*' \ - 'rpy2=3.4*' && \ + 'rpy2=3.4*' \ + 'unixodbc=2.3.*' && \ conda clean --all -f -y && \ fix-permissions "${CONDA_DIR}" && \ fix-permissions "/home/${NB_USER}" diff --git a/docs/index.rst b/docs/index.rst index 3feaa654..7ff13525 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -10,7 +10,7 @@ Jupyter Docker Stacks are a set of ready-to-run Docker images containing Jupyter Quick Start ----------- -You can try a `recent build of the jupyter/base-notebook image on mybinder.org `_ by simply clicking the preceding link. Otherwise, the two examples below may help you get started if you `have Docker installed `_, know :doc:`which Docker image ` you want to use, and want to launch a single Jupyter Notebook server in a container. +You can try a `recent build of the jupyter/base-notebook image on mybinder.org `_ by simply clicking the preceding link. Otherwise, three examples below may help you get started if you `have Docker installed `_, know :doc:`which Docker image ` you want to use, and want to launch a single Jupyter Notebook server in a container. The other pages in this documentation describe additional uses and features in detail. @@ -18,11 +18,11 @@ The other pages in this documentation describe additional uses and features in d docker run -p 8888:8888 jupyter/scipy-notebook:33add21fab64 -**Example 2:** This command performs the same operations as **Example 1**, but it exposes the server on host port 10000 instead of port 8888. Visiting ``http://:10000/?token=`` in a browser loads JupyterLab, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console.:: +**Example 2:** This command performs the same operations as **Example 1**, but it exposes the server on host port 10000 instead of port 8888. Visiting ``http://:10000/?token=`` in a browser loads Jupyter Notebook server, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console.:: docker run -p 10000:8888 jupyter/scipy-notebook:33add21fab64 -**Example 3:** This command pulls the ``jupyter/datascience-notebook`` image tagged ``33add21fab64`` from Docker Hub if it is not already present on the local host. It then starts an *ephemeral* container running a Jupyter Notebook server and exposes the server on host port 10000. The command mounts the current working directory on the host as ``/home/jovyan/work`` in the container. The server logs appear in the terminal. Visiting ``http://:10000/?token=`` in a browser loads JupyterLab, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console. Docker destroys the container after notebook server exit, but any files written to ``~/work`` in the container remain intact on the host.:: +**Example 3:** This command pulls the ``jupyter/datascience-notebook`` image tagged ``33add21fab64`` from Docker Hub if it is not already present on the local host. It then starts an *ephemeral* container running a Jupyter Notebook server and exposes the server on host port 10000. The command mounts the current working directory on the host as ``/home/jovyan/work`` in the container. The server logs appear in the terminal. Visiting ``http://:10000/lab?token=`` in a browser loads JupyterLab, where ``hostname`` is the name of the computer running docker and ``token`` is the secret token printed in the console. Docker destroys the container after notebook server exit, but any files written to ``~/work`` in the container remain intact on the host.:: docker run --rm -p 10000:8888 -e JUPYTER_ENABLE_LAB=yes -v "$PWD":/home/jovyan/work jupyter/datascience-notebook:33add21fab64 diff --git a/docs/using/selecting.md b/docs/using/selecting.md index 0bfdc3bd..a06baef6 100644 --- a/docs/using/selecting.md +++ b/docs/using/selecting.md @@ -31,6 +31,7 @@ and versioning strategy. - [Miniforge](https://github.com/conda-forge/miniforge) Python 3.x in `/opt/conda` with two package managers - [conda](https://github.com/conda/conda): "cross-platform, language-agnostic binary package manager". - [mamba](https://github.com/mamba-org/mamba): "reimplementation of the conda package manager in C++". +- `notebook`, `jupyterhub` and `jupyterlab` packages - No preinstalled scientific computing packages - Unprivileged user `jovyan` (`uid=1000`, configurable, see options) in group `users` (`gid=100`) with ownership over the `/home/jovyan` and `/opt/conda` paths @@ -52,7 +53,7 @@ and versioning strategy. - [TeX Live](https://www.tug.org/texlive/) for notebook document conversion - [git](https://git-scm.com/), [vi](https://www.vim.org) (actually `vim-tiny`), - [nano](https://www.nano-editor.org/) (actually `nano-tiny`), tzdata, and unzip + [nano](https://www.nano-editor.org/) (actually `nano-tiny`), `tzdata`, and `unzip` ### jupyter/r-notebook @@ -65,17 +66,25 @@ and versioning strategy. - Everything in `jupyter/minimal-notebook` and its ancestor images - The [R](https://www.r-project.org/) interpreter and base environment - [IRKernel](https://irkernel.github.io/) to support R code in Jupyter notebooks -- [tidyverse](https://www.tidyverse.org/) packages from - [conda-forge](https://conda-forge.org/feedstock-outputs/index.html) -- [devtools](https://cran.r-project.org/web/packages/devtools/index.html), - [shiny](https://shiny.rstudio.com/), [rmarkdown](https://rmarkdown.rstudio.com), - [forecast](https://cran.r-project.org/web/packages/forecast/forecast.pdf), - [rsqlite](https://cran.r-project.org/web/packages/RSQLite/index.html), +- [tidyverse](https://www.tidyverse.org/) + packages from [conda-forge](https://conda-forge.org/feedstock-outputs/index.html) +- [caret](https://topepo.github.io/caret/index.html), + [crayon](https://cran.r-project.org/web/packages/crayon/index.html), + [devtools](https://cran.r-project.org/web/packages/devtools/index.html), + [forecast](https://cran.r-project.org/web/packages/forecast/index.html), + [hexbin](https://cran.r-project.org/web/packages/hexbin/index.html), + [htmltools](https://cran.r-project.org/web/packages/htmltools/index.html), + [htmlwidgets](https://www.htmlwidgets.org), [nycflights13](https://cran.r-project.org/web/packages/nycflights13/index.html), - [caret](https://topepo.github.io/caret/index.html), [tidymodels](https://www.tidymodels.org/), - [rcurl](https://cran.r-project.org/web/packages/RCurl/index.html), and - [randomforest](https://cran.r-project.org/web/packages/randomForest/randomForest.pdf) packages - from [conda-forge](https://conda-forge.org/feedstock-outputs/index.html) + [randomforest](https://cran.r-project.org/web/packages/randomForest/index.html), + [rcurl](https://cran.r-project.org/web/packages/RCurl/index.html), + [rmarkdown](https://rmarkdown.rstudio.com), + [rodbc](https://cran.r-project.org/web/packages/RODBC/index.html), + [rsqlite](https://cran.r-project.org/web/packages/RSQLite/index.html), + [shiny](https://shiny.rstudio.com/), + [tidymodels](https://www.tidymodels.org/), + [unixodbc](http://www.unixodbc.org) + packages from [conda-forge](https://conda-forge.org/feedstock-outputs/index.html) ### jupyter/scipy-notebook @@ -86,25 +95,38 @@ and versioning strategy. `jupyter/scipy-notebook` includes popular packages from the scientific Python ecosystem. - Everything in `jupyter/minimal-notebook` and its ancestor images -- [dask](https://dask.org/), [pandas](https://pandas.pydata.org/), - [numexpr](https://github.com/pydata/numexpr), [matplotlib](https://matplotlib.org/), - [scipy](https://www.scipy.org/), [seaborn](https://seaborn.pydata.org/), - [scikit-learn](https://scikit-learn.org/stable/), [scikit-image](https://scikit-image.org), - [sympy](https://www.sympy.org/en/index.html), [cython](https://cython.org), - [patsy](https://patsy.readthedocs.io/en/latest/), - [statsmodel](https://www.statsmodels.org/stable/index.html), +- [beautifulsoup4](https://www.crummy.com/software/BeautifulSoup/), + [bokeh](https://docs.bokeh.org/en/latest/), + [bottleneck](https://bottleneck.readthedocs.io/en/latest/), [cloudpickle](https://github.com/cloudpipe/cloudpickle), - [dill](https://pypi.org/project/dill/), [numba](https://numba.pydata.org/), - [bokeh](https://docs.bokeh.org/en/latest/), [sqlalchemy](https://www.sqlalchemy.org/), - [hdf5](https://www.h5py.org), [vincent](https://vincent.readthedocs.io/en/latest/), - [beautifulsoup](https://www.crummy.com/software/BeautifulSoup/), + [conda-forge::blas=\*=openblas](https://www.openblas.net), + [cython](https://cython.org), + [dask](https://dask.org/), + [dill](https://pypi.org/project/dill/), + [h5py](https://www.h5py.org), + [matplotlib-base](https://matplotlib.org/), + [numba](https://numba.pydata.org/), + [numexpr](https://github.com/pydata/numexpr), + [pandas](https://pandas.pydata.org/), + [patsy](https://patsy.readthedocs.io/en/latest/), [protobuf](https://developers.google.com/protocol-buffers/docs/pythontutorial), - [xlrd](https://www.python-excel.org), [bottleneck](https://bottleneck.readthedocs.io/en/latest/), - and [pytables](https://www.pytables.org/) packages -- [ipywidgets](https://ipywidgets.readthedocs.io/en/stable/) and - [ipympl](https://github.com/matplotlib/ipympl) for interactive visualizations and - plots in Python notebooks -- [Facets](https://github.com/PAIR-code/facets) for visualizing machine learning datasets + [pytables](https://www.pytables.org/), + [scikit-image](https://scikit-image.org), + [scikit-learn](https://scikit-learn.org/stable/), + [scipy](https://www.scipy.org/), + [seaborn](https://seaborn.pydata.org/), + [sqlalchemy](https://www.sqlalchemy.org/), + [statsmodel](https://www.statsmodels.org/stable/index.html), + [sympy](https://www.sympy.org/en/index.html), + [vincent](https://vincent.readthedocs.io/en/latest/), + [widgetsnbextension](https://ipywidgets.readthedocs.io/en/latest/user_install.html#installing-in-classic-jupyter-notebook), + [xlrd](https://www.python-excel.org) + packages +- [ipympl](https://github.com/matplotlib/ipympl) and + [ipywidgets](https://ipywidgets.readthedocs.io/en/stable/) + for interactive visualizations and plots in Python notebooks +- [Facets](https://github.com/PAIR-code/facets) + for visualizing machine learning datasets ### jupyter/tensorflow-notebook @@ -115,8 +137,7 @@ and versioning strategy. `jupyter/tensorflow-notebook` includes popular Python deep learning libraries. - Everything in `jupyter/scipy-notebook` and its ancestor images -- [tensorflow](https://www.tensorflow.org/) and [keras](https://keras.io/) machine learning - libraries +- [tensorflow](https://www.tensorflow.org/) machine learning library ### jupyter/datascience-notebook @@ -129,10 +150,13 @@ communities. - Everything in the `jupyter/scipy-notebook` and `jupyter/r-notebook` images, and their ancestor images +- [rpy2](https://rpy2.github.io/doc/latest/html/index.html) package - The [Julia](https://julialang.org/) compiler and base environment - [IJulia](https://github.com/JuliaLang/IJulia.jl) to support Julia code in Jupyter notebooks -- [HDF5](https://github.com/JuliaIO/HDF5.jl), [Gadfly](https://gadflyjl.org/stable/), and - [RDatasets](https://github.com/JuliaStats/RDatasets.jl) packages +- [HDF5](https://github.com/JuliaIO/HDF5.jl), + [Gadfly](https://gadflyjl.org/stable/), + [RDatasets](https://github.com/JuliaStats/RDatasets.jl) + packages ### jupyter/pyspark-notebook @@ -144,6 +168,7 @@ communities. - Everything in `jupyter/scipy-notebook` and its ancestor images - [Apache Spark](https://spark.apache.org/) with Hadoop binaries +- [pyarrow](https://arrow.apache.org/docs/python/) library ### jupyter/all-spark-notebook @@ -155,11 +180,11 @@ communities. - Everything in `jupyter/pyspark-notebook` and its ancestor images - [IRKernel](https://irkernel.github.io/) to support R code in Jupyter notebooks -- [Apache Toree](https://toree.apache.org/) and - [spylon-kernel](https://github.com/vericast/spylon-kernel) to support Scala code in Jupyter - notebooks -- [ggplot2](https://ggplot2.tidyverse.org), [sparklyr](https://spark.rstudio.com), and - [rcurl](https://cran.r-project.org/web/packages/RCurl/index.html) packages +- [rcurl](https://cran.r-project.org/web/packages/RCurl/index.html), + [sparklyr](https://spark.rstudio.com), + [ggplot2](https://ggplot2.tidyverse.org) + packages +- [spylon-kernel](https://github.com/vericast/spylon-kernel) to support Scala code in Jupyter notebooks ### Image Relationships @@ -173,7 +198,7 @@ diagram](../images/inherit.svg)](http://interactive.blockdiag.com/?compression=d ### Builds Pull requests to the `jupyter/docker-stacks` repository trigger builds of all images on GitHub -Actions. These images are for testing purposes only and are not saved for use. When pull requests +Actions. These images are for testing purposes only and are not saved for further use. When pull requests merge to master, all images rebuild on Docker Hub and become available to `docker pull` from Docker Hub. @@ -185,14 +210,12 @@ The `latest` tag in each Docker Hub repository tracks the master branch `HEAD` r Every image on Docker Hub also receives a 12-character tag which corresponds with the git commit SHA that triggered the image build. You can inspect the state of the `jupyter/docker-stacks` repository for that commit to review the definition of the image (e.g., images with tag `33add21fab64` were built -from -. +from . You must refer to git-SHA image tags when stability and reproducibility are important in your work. -(e.g. `FROM jupyter/scipy-notebook:33add21fab64`, -`docker run -it --rm jupyter/scipy-notebook:33add21fab64`). You should only use `latest` when a -one-off container instance is acceptable (e.g., you want to briefly try a new library in a -notebook). +(e.g. `FROM jupyter/scipy-notebook:33add21fab64`, `docker run -it --rm jupyter/scipy-notebook:33add21fab64`). +You should only use `latest` when a one-off container instance is acceptable +(e.g., you want to briefly try a new library in a notebook). ## Community Stacks diff --git a/r-notebook/Dockerfile b/r-notebook/Dockerfile index e6da2fd3..8d314006 100644 --- a/r-notebook/Dockerfile +++ b/r-notebook/Dockerfile @@ -23,7 +23,7 @@ RUN ln -s /bin/tar /bin/gtar USER $NB_UID -# R packages +# R packages including IRKernel which gets installed globally. RUN conda install --quiet --yes \ 'r-base=4.0.3' \ 'r-caret=6.*' \ @@ -41,10 +41,9 @@ RUN conda install --quiet --yes \ 'r-rodbc=1.3*' \ 'r-rsqlite=2.2*' \ 'r-shiny=1.6*' \ - 'r-tidyverse=1.3*' \ - 'unixodbc=2.3.*' \ 'r-tidymodels=0.1*' \ - && \ + 'r-tidyverse=1.3*' \ + 'unixodbc=2.3.*' && \ conda clean --all -f -y && \ fix-permissions "${CONDA_DIR}" && \ fix-permissions "/home/${NB_USER}" diff --git a/scipy-notebook/Dockerfile b/scipy-notebook/Dockerfile index 6c3adda6..e29c8ffe 100644 --- a/scipy-notebook/Dockerfile +++ b/scipy-notebook/Dockerfile @@ -17,16 +17,16 @@ USER $NB_UID # Install Python 3 packages RUN conda install --quiet --yes \ 'beautifulsoup4=4.9.*' \ - 'conda-forge::blas=*=openblas' \ 'bokeh=2.3.*' \ 'bottleneck=1.3.*' \ 'cloudpickle=1.6.*' \ + 'conda-forge::blas=*=openblas' \ 'cython=0.29.*' \ 'dask=2021.4.*' \ 'dill=0.3.*' \ 'h5py=3.2.*' \ - 'ipywidgets=7.6.*' \ 'ipympl=0.7.*'\ + 'ipywidgets=7.6.*' \ 'matplotlib-base=3.4.*' \ 'numba=0.53.*' \ 'numexpr=2.7.*' \