From 99d306450edf23e0d2dba4c26d361d2d8127912c Mon Sep 17 00:00:00 2001 From: Yuvi Panda Date: Wed, 29 Nov 2023 13:56:28 -0800 Subject: [PATCH] Specify multiple architectures for Julia to precompile to (#2044) * Specify multiple architectures for Julia to precompile to For amd64 (x86_64), we should specify what specific targets the precompilation should be done for. If we don't specify it, it's *only* done for the target of the host doing the compilation. When the container runs on a host that's still x86_64, but a *different* generation of CPU than what the build host was, the precompilation is useless and Julia takes a long long time to start up. This specific multitarget comes from https://docs.julialang.org/en/v1/devdocs/sysimg/#Specifying-multiple-system-image-targets, and is the same set of options that the official Julia x86_64 build is compiled with. If the architecture the container runs on is different, precompilation may still have to be re-done on first startup - but this *should* catch most of the issues. h/t to https://discourse.julialang.org/t/is-it-possible-to-make-precompilation-portable-for-docker-images-built-with-a-different-cpu/95913 which helped point me towards `JULIA_CPU_TARGET`. Fixes https://github.com/jupyter/docker-stacks/issues/2015 for more information * Fix bash syntax issue * Add JULIA_CPU_TARGET for aarch64 as well - Don't need `export` as this is only used within this script - Steal from upstream what should be setup for aarch64 * Re-add export for JULIA_CPU_TARGET Quietens pre-commit --- .../setup-scripts/setup-julia-packages.bash | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/images/minimal-notebook/setup-scripts/setup-julia-packages.bash b/images/minimal-notebook/setup-scripts/setup-julia-packages.bash index 16adb428..fa1421e9 100755 --- a/images/minimal-notebook/setup-scripts/setup-julia-packages.bash +++ b/images/minimal-notebook/setup-scripts/setup-julia-packages.bash @@ -5,6 +5,28 @@ set -exuo pipefail # - The JULIA_PKGDIR environment variable is set # - Julia is already set up, with the setup-julia.bash command + +# If we don't specify what CPUs the precompilation should be done for, it's +# *only* done for the target of the host doing the compilation. When the +# container runs on a host that's the same architecture, but a *different* +# generation of CPU than what the build host was, the precompilation is useless +# and Julia takes a long long time to start up. This specific multitarget comes +# from https://github.com/JuliaCI/julia-buildkite/blob/70bde73f6cb17d4381b62236fc2d96b1c7acbba7/utilities/build_envs.sh#L20-L76, +# and may need to be updated as new CPU generations come out. +# If the architecture the container runs on is different, +# precompilation may still have to be re-done on first startup - but this +# *should* catch most of the issues. See +# https://github.com/jupyter/docker-stacks/issues/2015 for more information +if [ "$(uname -m)" == "x86_64" ]; then + # See https://github.com/JuliaCI/julia-buildkite/blob/70bde73f6cb17d4381b62236fc2d96b1c7acbba7/utilities/build_envs.sh#L24 + # for an explanation of these options + export JULIA_CPU_TARGET="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)" +elif [ "$(uname -m)" == "aarch64" ]; then + # See https://github.com/JuliaCI/julia-buildkite/blob/70bde73f6cb17d4381b62236fc2d96b1c7acbba7/utilities/build_envs.sh#L54 + # for an explanation of these options + export JULIA_CPU_TARGET="generic;cortex-a57;thunderx2t99;carmel" +fi + # Install base Julia packages julia -e ' import Pkg;