Commit Graph

5 Commits

Author SHA1 Message Date
Ayaz Salikhov
30a1d17085 Add a way to easily test units 2021-06-26 18:40:50 +03:00
romainx
1dca39182b Improve spark installation
Spark installation improved by sourcing the `spark-config.sh` in the `before-notebook.d` hook that is run by `start.sh`. It permits to add automatically the right Py4J dependency version in the `PYTHONPATH`. So it is not needed anymore to set this variable at build time.

Documentation describing the installation of a custom Spark version modified to remove this step. Also updated to install the latest `2.x` Spark version.

`test_pyspark` fixed (was always OK before that).
2020-11-24 20:40:06 +01:00
Ayaz Salikhov
94cd03d6b9 Fix python codestyle 2020-07-04 18:30:53 +02:00
Peter Parente
4a8b58a41b Test payspark import 2020-02-15 19:08:40 -05:00
romainx
45d51e3b42 Bump to spark 2.4.5 + minor improvements 2020-02-11 21:30:47 +01:00