hazza/docker-stacks
1
0
Fork 0
mirror of https://github.com/jupyter/docker-stacks.git synced 2025-10-18 23:42:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add build arg for delta lake version

Browse Source 
Co-authored-by: Romain <romainx@users.noreply.github.com>
This commit is contained in:
Tony
2021-03-27 10:09:14 +01:00
committed by GitHub
parent fa8cea32f6
commit 74fe49040b
1 changed files with 3 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/using/recipes.md
View File

@@ -535,9 +535,11 @@ Please note that the Delta lake packages are only available for Spark version >
```dockerfile ```dockerfile
FROM jupyter/pyspark-notebook:latest FROM jupyter/pyspark-notebook:latest
ARG delta-core_version="0.8.0"
USER root USER root
RUN echo 'spark.jars.packages io.delta:delta-core_2.12:0.8.0' >> $SPARK_HOME/conf/spark-defaults.conf && \ RUN echo "spark.jars.packages io.delta:delta-core_2.12:${delta-core_version}" >> $SPARK_HOME/conf/spark-defaults.conf && \
echo 'spark.sql.extensions io.delta.sql.DeltaSparkSessionExtension' >> $SPARK_HOME/conf/spark-defaults.conf && \ echo 'spark.sql.extensions io.delta.sql.DeltaSparkSessionExtension' >> $SPARK_HOME/conf/spark-defaults.conf && \
echo 'spark.sql.catalog.spark_catalog org.apache.spark.sql.delta.catalog.DeltaCatalog' >> $SPARK_HOME/conf/spark-defaults.conf echo 'spark.sql.catalog.spark_catalog org.apache.spark.sql.delta.catalog.DeltaCatalog' >> $SPARK_HOME/conf/spark-defaults.conf