From 5c2618f27a5ff26625f5f89543ad276540795cf1 Mon Sep 17 00:00:00 2001 From: Mladen Kolovic Date: Fri, 18 Mar 2016 22:42:45 -0400 Subject: [PATCH] Add HDF5 support for Julia * create `.juliarc.jl` start up script to point Julia to conda libraries, specifically the hdf5 libraries installed through the h5py package in the scipy stack * add the HDF5 Julia package Update README.md to document HDF5 addition Add -F option to force config load Add the -F flag to julia expression execution to load .juliarc.jl before execution. --- all-spark-notebook/README.md | 4 +++- datascience-notebook/Dockerfile | 6 +++++- datascience-notebook/README.md | 4 +++- minimal-kernel/README.md | 2 ++ minimal-notebook/README.md | 2 ++ pyspark-notebook/README.md | 2 ++ r-notebook/README.md | 2 ++ scipy-notebook/README.md | 2 ++ 8 files changed, 21 insertions(+), 3 deletions(-) diff --git a/all-spark-notebook/README.md b/all-spark-notebook/README.md index 8c808275..4b86b1ae 100644 --- a/all-spark-notebook/README.md +++ b/all-spark-notebook/README.md @@ -1,3 +1,5 @@ +![docker pulls](https://img.shields.io/docker/pulls/jupyter/all-spark-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/all-spark-notebook.svg) + # Jupyter Notebook Python, Scala, R, Spark, Mesos Stack ## What it Gives You @@ -195,7 +197,7 @@ rdd.sum() Connection to Spark Cluster on Standalone Mode requires the following set of steps: -0. Verify that the docker image (check the Dockerfile) and the Spark Cluster which is being deployed, run the same version of Spark. +0. Verify that the docker image (check the Dockerfile) and the Spark Cluster which is being deployed, run the same version of Spark. 1. [Deploy Spark on Standalone Mode](http://spark.apache.org/docs/latest/spark-standalone.html). 2. Run the Docker container with `--net=host` in a location that is network addressable by all of your Spark workers. (This is a [Spark networking requirement](http://spark.apache.org/docs/latest/cluster-overview.html#components).) * NOTE: When using `--net=host`, you must also use the flags `--pid=host -e TINI_SUBREAPER=true`. See https://github.com/jupyter/docker-stacks/issues/64 for details. diff --git a/datascience-notebook/Dockerfile b/datascience-notebook/Dockerfile index ed0fb2de..f2b81237 100644 --- a/datascience-notebook/Dockerfile +++ b/datascience-notebook/Dockerfile @@ -52,4 +52,8 @@ RUN julia -e 'Pkg.add("IJulia")' && \ mv /home/$NB_USER/.local/share/jupyter/kernels/* $CONDA_DIR/share/jupyter/kernels/ && \ chmod -R go+rx $CONDA_DIR/share/jupyter && \ rm -rf /home/$NB_USER/.local/share -RUN julia -e 'Pkg.add("Gadfly")' && julia -e 'Pkg.add("RDatasets")' + +# Show Julia where conda libraries are +# Add essential packages +RUN echo 'push!(Sys.DL_LOAD_PATH, "/opt/conda/lib")' > /home/$NB_USER/.juliarc.jl && \ + julia -e 'Pkg.add("Gadfly")' && julia -e 'Pkg.add("RDatasets")' && julia -F -e 'Pkg.add("HDF5")' diff --git a/datascience-notebook/README.md b/datascience-notebook/README.md index 3d680eab..f26c21fe 100644 --- a/datascience-notebook/README.md +++ b/datascience-notebook/README.md @@ -1,3 +1,5 @@ +![docker pulls](https://img.shields.io/docker/pulls/jupyter/datascience-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/datascience-notebook.svg) + # Jupyter Notebook Data Science Stack ## What it Gives You @@ -7,7 +9,7 @@ * pandas, matplotlib, scipy, seaborn, scikit-learn, scikit-image, sympy, cython, patsy, statsmodel, cloudpickle, dill, numba, bokeh pre-installed * Conda R v3.2.x and channel * plyr, devtools, dplyr, ggplot2, tidyr, shiny, rmarkdown, forecast, stringr, rsqlite, reshape2, nycflights13, caret, rcurl, and randomforest pre-installed -* Julia v0.3.x with Gadfly and RDatasets pre-installed +* Julia v0.3.x with Gadfly, RDatasets and HDF5 pre-installed * Unprivileged user `jovyan` (uid=1000, configurable, see options) in group `users` (gid=100) with ownership over `/home/jovyan` and `/opt/conda` * [tini](https://github.com/krallin/tini) as the container entrypoint and [start-notebook.sh](../minimal-notebook/start-notebook.sh) as the default command * Options for HTTPS, password auth, and passwordless `sudo` diff --git a/minimal-kernel/README.md b/minimal-kernel/README.md index 25ceeb4e..809b0171 100644 --- a/minimal-kernel/README.md +++ b/minimal-kernel/README.md @@ -1,3 +1,5 @@ +![docker pulls](https://img.shields.io/docker/pulls/jupyter/minimal-kernel.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/minimal-kernel.svg) + # Kernel Gateway Stack ## What it Gives You diff --git a/minimal-notebook/README.md b/minimal-notebook/README.md index 27a8220e..c31dd9bc 100644 --- a/minimal-notebook/README.md +++ b/minimal-notebook/README.md @@ -1,3 +1,5 @@ +![docker pulls](https://img.shields.io/docker/pulls/jupyter/minimal-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/minimal-notebook.svg) + # Minimal Jupyter Notebook Stack ## What it Gives You diff --git a/pyspark-notebook/README.md b/pyspark-notebook/README.md index de2108bb..a252f44e 100644 --- a/pyspark-notebook/README.md +++ b/pyspark-notebook/README.md @@ -1,3 +1,5 @@ +![docker pulls](https://img.shields.io/docker/pulls/jupyter/pyspark-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/pyspark-notebook.svg) + # Jupyter Notebook Python, Spark, Mesos Stack ## What it Gives You diff --git a/r-notebook/README.md b/r-notebook/README.md index fbff77aa..7ccf4c18 100644 --- a/r-notebook/README.md +++ b/r-notebook/README.md @@ -1,3 +1,5 @@ +![docker pulls](https://img.shields.io/docker/pulls/jupyter/r-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/r-notebook.svg) + # Jupyter Notebook R Stack ## What it Gives You diff --git a/scipy-notebook/README.md b/scipy-notebook/README.md index 925dd32c..6503e9d5 100644 --- a/scipy-notebook/README.md +++ b/scipy-notebook/README.md @@ -1,3 +1,5 @@ +![docker pulls](https://img.shields.io/docker/pulls/jupyter/scipy-notebook.svg) ![docker stars](https://img.shields.io/docker/stars/jupyter/scipy-notebook.svg) + # Jupyter Notebook Scientific Python Stack ## What it Gives You