Commit Graph

179 Commits

Author SHA1 Message Date
Ayaz Salikhov
833e235281 Implement SparkInfoManifest 2021-04-25 12:40:48 +03:00
Ayaz Salikhov
1839a4934d Implement RPackagesManifest 2021-04-25 12:28:29 +03:00
Ayaz Salikhov
2f8934e5ac Remove unused run_hook parts 2021-04-25 12:22:00 +03:00
Ayaz Salikhov
f52987a959 Delete run_hook parts already implemented in python 2021-04-25 11:54:46 +03:00
Ayaz Salikhov
9991d6438e Delete index update from run_hook 2021-04-24 20:38:44 +03:00
Ayaz Salikhov
74fe3361d7 Merge branch 'master' into asalikhov/python_tagging 2021-04-18 11:31:23 +03:00
Ayaz Salikhov
a8b7682fd6 Add newlines 2021-04-17 18:20:37 +03:00
Ayaz Salikhov
f8d21532ee Remove docker tag from run_hook 2021-03-22 13:36:32 +03:00
romainx
d0e066e116 Regular update 2021-03-17 2021-03-17 22:13:19 +01:00
Romain
d113a601db Merge pull request #1198 from romainx/fix_spark_java11
Fix spark installation for Java 11 and Arrow
2020-12-26 17:22:44 +01:00
romainx
daff4686e8 Fix .dockerignore by adding default hooks + test dirs 2020-12-20 05:47:12 +01:00
romainx
1dd95bad2c Fix spark installation for Java 11 and Arrow 2020-12-13 11:58:46 +01:00
romainx
86523f0619 Regular update + improvements 2020-12-01 22:05:21 +01:00
romainx
32090f6011 Update docker stack 2020-11-20
The following changes have been made.

- `base-notebook`
  - `Changed`: Bump Ubuntu
  - `Changed`: Bump conda
  - `Fixed`: Add missing `apt-get clean`
- `minimal-notebook`
  - `Removed`: `jed` editor see #1174 (partial)
  - `Deprecated`: `emacs` editor (in the documentation and in the image) see #1174 (partial)
- `scipy-notebook`
  - `Fixed`: Add missing `apt-get clean`
  - `Changed`: Bump `dask`
  - `Changed`: Bump `protobuf`
- `r-notebook`
  - `Changed`: Bump `r-base`
  - `Changed`: Bump `r-rmarkdown`
  - `Removed`: The description of `tidyverse` packages because it's not the place to do that and it will always be obsolete.
- `datascience-notebook`
  - `Changed`: Bump `r-base` to `4.0.x` see #1102 (partial)
  - `Removed`: `plyr` package because it's retired fixes #1103
  - `Removed`: `r-reshape2` package because it's retired fixes #1103
  - `Changed`: Bump `r-rmarkdown`
  - `Changed`: Bump Julia
- `tensorflow-notebook`
  - `Changed`: Bump `tensorflow`
- `pyspark-notebook`
  - `Fixed`: Add missing `apt-get clean`
  - `Changed`: Bump `pyarrow`
- `all-spark-notebook`
  - `Fixed`: Add missing `apt-get clean`
  - `Changed`: Bump `r-base` to `4.0.x` see #1102 (partial)
  - `Changed`: Bump `r-sparklyr`
2020-10-20 11:36:34 +02:00
Ayaz Salikhov
530e2b3834 Merge branch 'master' into asalikhov/py_codestyle 2020-08-16 20:02:40 +03:00
Peter Parente
1031477369 [ci skip] Update READMEs visible on Docker Hub 2020-08-08 15:09:02 -04:00
Peter Parente
5758b59314 Fix SOURCE_COMMIT -> GITHUB_SHA references 2020-08-01 00:12:07 -04:00
Peter Parente
d2cb805740 Clean up hook debugs, leftovers 2020-07-30 07:50:23 -04:00
Peter Parente
34e73d4f6b Refactor wiki hooks for Actions 2020-07-30 07:49:00 -04:00
Ayaz Salikhov
e28a197b41 Fix 2020-07-04 20:33:55 +02:00
Ayaz Salikhov
b1e4688d5c Add automatic checks 2020-07-04 19:06:21 +02:00
Darek
568708d279 Upgrading Spark to 3.0, removing Toree 2020-06-19 00:52:51 +00:00
Romain
5e6645d137 Ignore DL3006 and DL3008 by default 2020-06-01 06:23:44 +02:00
Romain
7b48e43b74 Fix hadolint deviations 2020-05-29 19:33:24 +02:00
Romain
c83024c950 Add spark notebook tests and change examples
* Test added for all kernels
* Same examples as provided in the documentation (`specifics.md`)
* Used the same use case for all examples: sum of the first 100 whole numbers

Note: I've not automatically tested `local_sparklyr.ipynb` since it creates by default the `metastore_db` dir and the `derby.log` file in the working directory. Since I mount it in `RO` it's not working. I'm struggling to set it elsewhere...
2020-05-29 06:54:46 +02:00
Peter Parente
ba6723edf1 Merge branch 'master' into remove-mesos 2020-05-27 16:05:31 -04:00
Ayaz Salikhov
9096270368 Print image size in manifest 2020-05-26 15:07:25 +02:00
Travis CI
44a7d70805 Remove mesos lib from pyspark-notebook 2020-05-24 14:18:47 -04:00
romainx
d3ef6d89b2 Regular update 2020 W18
* Bump `base-notebook`
	* OS version
	* `jupyterlab`
* Bump `scipy-notebook`
	* `beautifulsoup4`
	* `cloudpickle`
	* `dask`
	* `numba`
	* `xlrd`: version specified (it was not the case)
* Bump `datascience-notebook`
	* `r-devtools`
	* `r-forecast`
	* Julia version
	* Test added to check if Julia is correctly installed (`julia --version`)
	* Note: A new version of `r-base` is out `4.0.0` however it cannot be installed yet since dependencies with other packages cannot be resolved.
* Bump `r-notebook`
	* Same as `datascience-notebook` except Julia.
* Bump `all-spark-notebook`
	* `r-sparklyr`
2020-04-29 09:16:28 +02:00
romainx
d458afffaf Big upgrade
- Bump root container version
- Bump  packages of
  - scipy-notebook
  - r-notebook
  - datascience-notebook
- add -y flag to jupyter lab commands in scipy-notebook
- Fix a bug in check outdated packages when packages installed through pip
2020-04-04 23:04:45 +02:00
romainx
03ec0a592d Bump version of R packages 2020-01-30 19:01:44 +01:00
Peter Parente
171ba82a30 Fix ssh key format
New Docker Hub UI loses newlines in the env
var settings. Loss of new lines leads ssh-add
to prompt and fail when loading the key.

Base64 encode and decode the key to workaround
the limitation.
2020-01-29 09:37:04 -05:00
Richard Darst
8bfffcaa3f Sort package installation lines alphabetically for easier comparison 2019-08-06 10:44:10 +03:00
Richard Darst
fed5b9ac62 Update versions in all-spark-notebook 2019-08-04 02:15:30 +03:00
echowhisky
40c5c07b0a added -f flag to all conda clean commands
This commit adds the additional `-f` force command to all uses of `conda
clean --all` through the repo. Size should be smaller, but still testing
if anything breaks. See issue #861.
2019-05-04 19:11:32 +00:00
echowhisky
1f8311a7aa changed -tipsy to --all -y across all files
The last commit was only for the base-notebook's Dockerfile. For this,
all the files in the repo were grepped through and changed.
2019-05-04 18:53:49 +00:00
Chia-liang Kao
a657b782bd Move tzdata to minimal-notebook and document it 2019-03-25 16:04:04 +08:00
Peter Parente
f3da6c276a [ci skip] stderr when no tty 2019-03-18 22:32:25 -04:00
Peter Parente
657f8a095a [ci skip] Prefer SPARK_HOME 2019-03-17 10:17:06 -04:00
Peter Parente
4abdeee523 [ci skip] Fix spark version check
Not in PATH
2019-03-17 10:05:48 -04:00
Peter Parente
76853d93ed [ci skip] Escape those backticks 2019-03-16 22:38:40 -04:00
Peter Parente
4667cffc41 [ci skip] List full image name and tag 2019-03-16 22:31:10 -04:00
Peter Parente
3339b63abf [ci skip] Add row to build index after every image push 2019-03-16 22:03:59 -04:00
Peter Parente
243da296a1 [ci skip] Add language versions, conda info, fix spark info 2019-03-16 22:03:54 -04:00
Peter Parente
10c58da405 [ci skip] Add package manifest support
Generates manifest pages and pushes them
to the docker-stacks GitHub wiki.
Does not update a global index of all builds yet.
2019-03-15 16:08:27 -04:00
Peter Parente
c12adebb32 Merge pull request #782 from parente/daozzg-master
Update all R libs in all-spark-notebook for Spark 2.4
2018-12-29 10:34:11 -05:00
Peter Parente
8b0c000794 Update R packages 2018-12-29 00:05:19 -05:00
Tim Ryan
3fc28b67e0 Remove apt-get clean command. Unecessary on official Ubuntu images. 2018-12-17 10:40:58 -05:00
Alan
06446856b6 Update Dockerfile 2018-12-07 09:50:44 +08:00
Alan
2a25febf80 Update Dockerfile 2018-12-07 09:43:03 +08:00