Commit Graph

32 Commits

Author SHA1 Message Date
John Kirkham
ca3f00a6c8 pyspark-notebook/Dockerfile: Note to get Mesos for jessie. 2016-03-05 15:12:24 -05:00
John Kirkham
4711a6ce75 pyspark-notebook/Dockerfile: Clean out conda. 2016-03-04 18:29:31 -05:00
John Kirkham
2a51eb9139 pyspark-notebook/Dockerfile: Tell conda yes. 2016-03-04 17:31:47 -05:00
Peter Parente
3fd3850bdc Bump python libs to match other images
(c) Copyright IBM Corp. 2016
2016-02-13 20:49:11 -05:00
Peter Parente
d25f33a5fb Fix py4j version and path
(c) Copyright IBM Corp. 2016
2016-02-04 21:29:36 -05:00
Peter Parente
83f9f834bc Update README Spark version mentions
(c) Copyright IBM Corp. 2016
2016-02-03 13:29:21 -05:00
Peter Parente
154c4d64d2 Bump Spark dependencies
* Use Spark 1.6.0
* Use Apache Toree (new name/home of ibm-et/spark-kernel)
* Add missing pub key for binary
* Add SHA check for spark package download

(c) Copyright IBM Corp. 2016
2016-02-03 13:29:21 -05:00
Peter Parente
232d6fc465 Use jq instead of sed
(c) Copyright IBM Corp. 2016
2016-02-02 10:47:57 -05:00
Peter Parente
3e27c990e9 Remove os.environ step from spark READMEs
(c) Copyright IBM Corp. 2016
2016-02-02 09:06:54 -05:00
Peter Parente
155fdea583 Set PYSPARK_HOME in python2 kernelspec
* Install python2 spec to CONDA_DIR instead of system prefix too

(c) Copyright IBM Corp. 2016
2016-02-02 09:03:30 -05:00
Peter Parente
90a3d2ef7d Note bump to notebook 4.1.x
(c) Copyright IBM Corp. 2016
2016-01-17 11:27:34 -05:00
Peter Parente
9aae44ef66 Use updated LICENSE.md file
Also fix copyright headers on various files
2016-01-17 11:27:10 -05:00
Peter Parente
6d5cd67528 Make jovyan the default user for the docker cmd
* Switch to jovyan at the end of every Dockerfile
* Document --user root requirement for NB_UID and GRANT_SUDO flags

(c) Copyright IBM Corp. 2015
2015-12-27 21:05:48 -05:00
Justin Tyberg
e53b288e6e Bump to pandas 0.17.
(c) Copyright IBM Corp. 2015
2015-12-04 17:47:12 -05:00
Peter Parente
ebda307c7c Doc --net=host requirements for su
(c) Copyright IBM Corp. 2015
2015-11-20 20:09:40 -05:00
Alexander Lewitt
2a6c7a59ed pyspark readme, spark ui support.
adding instructions for how to open an extra port in order to view spark ui monitoring status.
2015-11-10 09:55:12 -05:00
Peter Parente
744c903854 Merge pull request #50 from parente/doc-cmd-args
Remove trivial env vars, doc command line args
2015-10-24 22:36:03 -04:00
Kai Wang
817550198f update for Spark 1.5.1 2015-10-21 15:22:57 +00:00
Peter Parente
ad99f8c74f Remove trivial env vars, doc command line args
* Remove PORT and INTERFACE env vars which can conflict with other systems (e.g., Mesos)
* Document command line pass-through to start-notebook.sh

(c) Copyright IBM Corp. 2015
2015-10-19 09:36:28 -04:00
Peter Parente
488aa7ba64 Remove 3.x vs 4.x legacy info from README
Fixes #48

(c) Copyright IBM Corp. 2015
2015-10-19 09:05:10 -04:00
Peter Parente
43e1914065 Add doc about conda environments
Contribution (c) Copyright IBM Corp. 2015
2015-10-07 22:31:27 -04:00
Peter Parente
1f282e8a27 Document Mesos user flag requirements 2015-10-07 22:05:45 -04:00
Peter Parente
6e7c1574a6 Note tini is in 4.0 and up images 2015-09-23 20:17:33 -04:00
Peter Parente
d425a3a502 Mention tini and start-notebook.sh in READMEs
Contribution (c) Copyright IBM Corp. 2015
2015-09-22 20:51:42 -04:00
Peter Parente
678b64e9e4 Swap tini for supervisord
* Pass $@ args to start-notebook.sh
* Set tini as entrypoint, but keep start-notebook.sh as easily overridable CMD
* su to jovyan user within start-notebook.sh script

Contribution (c) Copyright IBM Corp. 2015
2015-09-22 20:51:18 -04:00
Peter Parente
d51afb34f7 Explicitly document jovyan user 2015-09-12 20:39:31 -04:00
Peter Parente
69c5c35f7f Run pyspark, all-spark, scipy, r conda installs as jovyan
… but install kernel specs global to avoid problems when switching NB_UID

Contribution (c) Copyright IBM Corp. 2015
2015-09-11 00:36:40 -04:00
Peter Parente
701212dbf1 Explicitly set USER root in every Dockerfile
Contribution (c) Copyright IBM Corp. 2015
2015-08-29 21:50:04 -04:00
Peter Parente
c4616560cf Make subimages compatible with late user creation
* Always remain as root during install
* Put kernel specs in system path, not user home
* Create user work directory at startup
* Note this is in 4.0 and up images, not 3.2

Contribution (c) Copyright IBM Corp. 2015
2015-08-28 23:43:57 -04:00
Peter Parente
16db7cfe55 Bump master to Jupyter Notebook 4.0
* Merge 4.0.x branch
* Rejigger Dockerfiles to build from minimal-notebook:latest
2015-08-25 20:45:04 -04:00
Peter Parente
e1c0c75c9c Upgrade to Jupyter Notebook
* Change minimal-notebook to install notebook=4.0*
* Change other Dockerfile to point to 4.0 Docker Hub tag (to be built)
* Change config and pem file paths for jupyter
* Install ipywidgets in all containers that have a Python stack
* Update all READMEs to describe v3.2 and v4.0 since Docker Hub only shows one README for all tags

Contribution (c) Copyright IBM Corp. 2015
2015-08-23 08:55:09 -04:00
Peter Parente
871c0edc16 Spark 1.4.1 and Mesos 0.22, for Python 2 and 3
Plus README explaining how to use pyspark in local mode and with a
Mesos cluster.

(c) Copyright IBM Corp. 2015
2015-08-12 12:10:09 -04:00