Commit Graph

9 Commits

Author SHA1 Message Date
Bjørn Jørgensen
52a999a554 Upgrade Apache Spark to 3.5.0 (#1995)
* 1.

* add note for pandas version

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update images/pyspark-notebook/Dockerfile

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2023-09-16 20:56:10 +04:00
Ayaz Salikhov
c44f6e9476 Remove extra line in some test files 2023-07-03 00:35:00 +04:00
Ayaz Salikhov
2343fdba34 Add run_command script in tests (#1931)
* Add run_command script in tests

* Fix

* Add timeout back
2023-07-03 00:34:06 +04:00
Bjørn Jørgensen
df5d516e34 Pin pandas to version 1.5.3 to all spark images (#1925)
* 1.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add note

* typo

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove test

* >=1.5.3 and <2.0.0

* update test

* Update pyspark-notebook/Dockerfile

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update pyspark-notebook/Dockerfile

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

* move test to file

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add pandas to EXCLUDED_PACKAGES

* add 1.5.3,<2.0.0 and sort list

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add '

* "pandas[version='>"

* Rename test_pandas_version.py to unit_pandas_version.py

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2023-06-27 15:42:18 +04:00
Darek
3b1ddfa670 Removing no_warnings 2022-06-21 15:18:33 -04:00
Darek
aeb940220f Spark->3.3,Hadoop->3,Scala->2.13,Java->17 2022-06-16 14:53:37 -04:00
Ayaz Salikhov
0aa95c71ad Make tests a module and make imports absolute 2022-04-01 01:26:37 +01:00
Ayaz Salikhov
2f201a4b81 Run pre-commit tools: isort, requirements-txt-fixer 2022-03-16 14:48:23 +02:00
Ayaz Salikhov
6d04f39de6 Run tests on all children images 2022-02-17 16:34:15 +00:00