* add scala version choise
* add ; \ fi
* change checksum and removed default scala version
* remove RUN
* add { } and remove old code
* remove 3 duplicated lines.
* Add the commint as a comment
* Add back #Fix
* Rename downloadeds as spark.tgz
* Fix doc
* Update specifics.md
* New fix
* Fix wget
* Remove make link to spark
* Set full path to /usr/local/spark
* Change /usr/local/spark to ${SPARK_HOME}
* fix RUN with if
* Remove empty lines
* Update Dockerfile
* Update Dockerfile
* Update Dockerfile
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
* Remove scala
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove scala from web
* Remove scala from specifics
* Remove scala and spylon
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Delta 1.2.1:
Fix an issue with loading error messages in --packages mode. Previous release had a bug that resulted in user getting NullPointerException instead of proper error message when using Delta Lake with --packages mode either in pyspark or spark-shell (Fix, Test)
Fix incorrect exception type thrown in some Python APIs. A bug caused pyspark to throw incorrect type of exceptions instead of expected AnalysisException. This issue is fixed. See issue #1086 for more details.
Fix for S3 multi-cluster mode configuration. A bug in the S3 multi-cluster mode caused --conf to not work for certain configuration parameters. This issue is fixed by having these configuration parameters begin with spark. See the updated documentation.
Make the GCS LogStore configuration simpler by automatically deriving the LogStore implementation class config spark.delta.logStore.gs.impl from the scheme in the table path. See the updated documentation.
Make SetAccumulator thread safe. SetAccumulator used by Merge was not thread safe and might cause executor heartbeat failures in rare cases. This was fixed by using a synchronized set.
* Add exmaple 4 that demonstrates how to run the container
image docker.io/jupyter/r-notebook:6b49f3337709 with rootless podman.
Some text from example 2 was reused in example 4.
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
