hazza/docker-stacks
1
0
Fork 0
mirror of https://github.com/jupyter/docker-stacks.git synced 2025-10-11 20:12:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,507 Commits 4 Branches 0 Tags
b68ff7f67fa3e02c638a4d082bc2502cc8ba3112
Commit Graph

116 Commits

Author SHA1 Message Date
romainx
9529e3dffe Turn off ipython low-level output capture and forward 2022-01-07 17:53:27 +01:00
Darek
df72530d28 Upgrading Spark to 3.2.0 2021-10-20 13:42:25 +00:00
Ayaz Salikhov
a6d0ed456e Merge branch 'master' into asalikhov/automatic_conda_versioning 2021-07-09 16:52:31 +03:00
Ayaz Salikhov
3c9e62efce Introduce owner to Dockerfiles to make it easy to test locally 2021-07-08 17:26:56 +03:00
Ayaz Salikhov
bfb8cc7d50 Allow conda to automatically deduce package versions 2021-06-20 22:13:15 +03:00
Ayaz Salikhov
4befdd5948 Merge branch 'master' into asalikhov/use_mamba 2021-06-07 23:05:26 +03:00
Ayaz Salikhov
396024a4dd Merge pull request #1335 from mathbunnyru/asalikhov/unify_bash_variables 
Unify bash variables usage and add quotes where needed
 2021-06-07 23:00:11 +03:00
Darek
3b5b735bb6 Upgrading spark_version->3.1.2 2021-06-04 15:02:52 +00:00
Ayaz Salikhov
6ab40f0002 Use mamba instead of conda in spark images 2021-06-01 20:30:30 +03:00
Ayaz Salikhov
0999f1a36f Unify bash variables usage and add quotes where needed 2021-05-26 13:06:10 +03:00
Ayaz Salikhov
fbceaa9892 Fixes 2021-05-22 14:01:01 +03:00
romainx
25d4876efe Regular update 2021-05-04 18:36:03 +02:00
Ayaz Salikhov
b5d6f04e43 Delete unused hadolint ignore 2021-05-02 23:28:53 +03:00
Ayaz Salikhov
0fff54f3f2 Install spark from archive.apache.org to be able to use old versions 2021-05-02 23:28:32 +03:00
Antony Neu
5b0b25d158 Fix config formatting 2021-03-20 14:19:55 +01:00
Bjørn-Andre Skaar
0192175d05 Spark 3.1.1 upgrade 2021-03-04 08:27:14 +01:00
romainx
753abff645 Regular update 2021-02-28 06:56:36 +01:00
Aaron Cody
ffea0d5c94 bumped spark version to 3.0.2 as 3.0.1 is no longer available for download 2021-02-23 16:27:49 -08:00
romainx
1dd95bad2c Fix spark installation for Java 11 and Arrow 2020-12-13 11:58:46 +01:00
romainx
1dca39182b Improve spark installation 
Spark installation improved by sourcing the `spark-config.sh` in the `before-notebook.d` hook that is run by `start.sh`. It permits to add automatically the right Py4J dependency version in the `PYTHONPATH`. So it is not needed anymore to set this variable at build time.

Documentation describing the installation of a custom Spark version modified to remove this step. Also updated to install the latest `2.x` Spark version.

`test_pyspark` fixed (was always OK before that).
 2020-11-24 20:40:06 +01:00
romainx
0f2a7473c4 Regular update 2020-11-05 20:55:18 +01:00
romainx
4271b09b33 rollback pyarrow update 2020-10-20 12:10:36 +02:00
romainx
32090f6011 Update docker stack 2020-11-20 
The following changes have been made.

- `base-notebook`
  - `Changed`: Bump Ubuntu
  - `Changed`: Bump conda
  - `Fixed`: Add missing `apt-get clean`
- `minimal-notebook`
  - `Removed`: `jed` editor see #1174 (partial)
  - `Deprecated`: `emacs` editor (in the documentation and in the image) see #1174 (partial)
- `scipy-notebook`
  - `Fixed`: Add missing `apt-get clean`
  - `Changed`: Bump `dask`
  - `Changed`: Bump `protobuf`
- `r-notebook`
  - `Changed`: Bump `r-base`
  - `Changed`: Bump `r-rmarkdown`
  - `Removed`: The description of `tidyverse` packages because it's not the place to do that and it will always be obsolete.
- `datascience-notebook`
  - `Changed`: Bump `r-base` to `4.0.x` see #1102 (partial)
  - `Removed`: `plyr` package because it's retired fixes #1103
  - `Removed`: `r-reshape2` package because it's retired fixes #1103
  - `Changed`: Bump `r-rmarkdown`
  - `Changed`: Bump Julia
- `tensorflow-notebook`
  - `Changed`: Bump `tensorflow`
- `pyspark-notebook`
  - `Fixed`: Add missing `apt-get clean`
  - `Changed`: Bump `pyarrow`
- `all-spark-notebook`
  - `Fixed`: Add missing `apt-get clean`
  - `Changed`: Bump `r-base` to `4.0.x` see #1102 (partial)
  - `Changed`: Bump `r-sparklyr`
 2020-10-20 11:36:34 +02:00
romainx
325dd5b3e2 Fix build (hope) 
- miniconda versions and arguments
- remove useless hd5 install (dependency of h5py)
- pin the version of pyarrow
 2020-09-19 17:27:47 +02:00
romainx
384acda330 Spark 3.0.1 -> fixes #1156 2020-09-10 15:25:00 +02:00
romainx
c288e77acb Fix debug line commented 2020-08-16 17:03:10 +02:00
romainx
8669d6e79b Resolves #1131: Allow alternative Spark version 
Allow to build `pyspark-notebook` image with an alternative Spark version.

- Define arguments for Spark installation
- Add a note in "Image Specifics" explaining how to build an image with an alternative Spark version
- Remove Toree documentation from "Image Specifics" since its support has been droped in #1115
 2020-08-15 20:19:35 +02:00
Darek
568708d279 Upgrading Spark to 3.0, removing Toree 2020-06-19 00:52:51 +00:00
Romain
5e6645d137 Ignore DL3006 and DL3008 by default 2020-06-01 06:23:44 +02:00
Romain
2ce0b49fb5 Final review 2020-05-30 05:44:53 +02:00
Romain
593698985f Fixes 2020-05-29 21:27:18 +02:00
Romain
7b48e43b74 Fix hadolint deviations 2020-05-29 19:33:24 +02:00
Peter Parente
bbbabd22a3 Update comments to remove mesos references 2020-05-25 12:25:47 -04:00
Travis CI
44a7d70805 Remove mesos lib from pyspark-notebook 2020-05-24 14:18:47 -04:00
Peter Parente
18bb3a4b8b Merge branch 'master' into fix-pyspark 2020-02-15 20:18:46 -05:00
Peter Parente
4a8b58a41b Test payspark import 2020-02-15 19:08:40 -05:00
Peter Parente
3aa61f94c2 Split SPARK_HOME definition from other env vars 2020-02-15 18:53:51 -05:00
romainx
4333c7cc14 fix as_json param 2020-02-13 13:02:37 +01:00
romainx
7f7be5707c spark mirror improvement 2020-02-13 12:00:41 +01:00
romainx
8b3ce5cfa6 Change spark mirror 2020-02-13 11:21:54 +01:00
romainx
45d51e3b42 Bump to spark 2.4.5 + minor improvements 2020-02-11 21:30:47 +01:00
Manny Cato
1089ae349c Address updating spark version to fix wget error 2019-09-10 16:25:27 -07:00
Peter Parente
411ec857bb Update to Spark 2.4.3 2019-05-11 19:27:53 -04:00
echowhisky
167c686011 Merge branch 'master' into issue-861 
Resolved conflicts between local branch and updates to the upstream
master.
 2019-05-06 18:46:51 +00:00
echowhisky
40c5c07b0a added -f flag to all conda clean commands 
This commit adds the additional `-f` force command to all uses of `conda
clean --all` through the repo. Size should be smaller, but still testing
if anything breaks. See issue #861.
 2019-05-04 19:11:32 +00:00
echowhisky
1f8311a7aa changed -tipsy to --all -y across all files 
The last commit was only for the base-notebook's Dockerfile. For this,
all the files in the repo were grepped through and changed.
 2019-05-04 18:53:49 +00:00
Peter Parente
825166612c Update to Spark 2.4.2 2019-04-25 10:06:50 -07:00
Peter Parente
32240517eb Update checksum for spark 2.4.1 
https://www.apache.org/dist/spark/spark-2.4.1/spark-2.4.1-bin-hadoop2.7.tgz.sha512
 2019-04-10 09:30:59 -04:00
Peter Parente
8cea451451 Update to Spark 2.4.1 
The mirror we prefer no longer has 2.4.0
 2019-04-10 00:05:55 -04:00
Tim Ryan
3fc28b67e0 Remove apt-get clean command. Unecessary on official Ubuntu images. 2018-12-17 10:40:58 -05:00