Commit Graph

176 Commits

Author SHA1 Message Date
Ayaz Salikhov
27dfcdb326 Fix broken link 2025-02-23 20:43:42 +00:00
Ayaz Salikhov
123d215128 Refactor tagging: create functions, better logs and names, textwrap.dedent (#2239) 2025-02-23 20:41:04 +00:00
Ayaz Salikhov
6c47a89558 Use textwrap.dedent() to create build info 2025-02-23 19:17:09 +00:00
Ayaz Salikhov
92a029158c Add MarkdownPiece class to make manifest pieces better structured (#2238) 2025-02-23 19:15:45 +00:00
Ayaz Salikhov
4ee4f2b906 Remove explicit utf-8 from decode() as it's default (#2237) 2025-02-22 14:15:53 +00:00
Ayaz Salikhov
8fc97cb36b Add tagging config to pass params easier (#2234)
* Add tagging config to pass params easier

* Shorter function signatures
2025-02-22 01:05:54 +00:00
Ayaz Salikhov
18e09a7872 Clarify container.remove() behaviour (#2235) 2025-02-22 00:46:48 +00:00
Ayaz Salikhov
1bbab5c80d Add more logs in tagging 2025-02-21 22:24:59 +00:00
Ayaz Salikhov
d9b6333f13 Write correct manifests in forks 2025-02-21 22:23:51 +00:00
Ayaz Salikhov
80f4426b8e Unify common arguments in tagging/apps (#2230) 2025-02-21 16:28:36 +00:00
Ayaz Salikhov
0c483ad1c1 Add copyright and improve tagging docs 2025-02-21 12:56:28 +00:00
Ayaz Salikhov
e815fde31b Better tagging directory structure (#2228) 2025-02-21 12:48:18 +00:00
Ayaz Salikhov
355813e360 Fix some typing comments 2025-02-21 01:56:40 +00:00
Ayaz Salikhov
e796b11ee1 Properly generate wiki home page in forks (#2226)
* Properly generate wiki home page in forks

* Better description
2025-02-20 00:01:55 +00:00
Fabian Rost
332aac5ba7 mkdir parents in update_wiki.py (#2225) 2025-02-19 18:07:14 +00:00
Ayaz Salikhov
8dddb5207c Use consistent order in metioning images 2025-02-18 17:30:19 +00:00
Ayaz Salikhov
b2246417aa Do not use deprecated datetime.utcnow() 2025-02-18 15:27:07 +00:00
Ayaz Salikhov
80b71625f0 Use Python 3.12 for internal code (#2222)
* Use Python 3.12 for internal code

* Add changelog entry
2025-02-18 15:26:06 +00:00
Ayaz Salikhov
e57047801c Improve handling common parser arguments (#2221) 2025-02-18 11:06:10 +00:00
Ayaz Salikhov
0b16cf8dbf Simplify unify_aarch64 2025-01-26 20:53:33 +00:00
Ayaz Salikhov
7e4e18403a Create all necessary directories during wiki update 2025-01-06 18:27:47 +05:00
Ayaz Salikhov
a6ef176ff3 Make commits a url on wiki home page 2024-12-31 21:16:33 +05:00
Ayaz Salikhov
ad4d4c6e6e Add yearly total stats to update_wiki 2024-12-31 20:41:30 +05:00
Ayaz Salikhov
a61c9ae8a7 Improve commits counting: inly count merge commits, calc between midnights UTC 2024-12-31 20:41:25 +05:00
Ayaz Salikhov
d812d68234 Allow no new manfests or build history lines files if special flag is present during update_wiki 2024-12-30 22:15:19 +05:00
Ayaz Salikhov
af0493a305 Add number of commits to wiki home page 2024-12-30 22:10:14 +05:00
Ayaz Salikhov
67dd54966c Fix args order when running python modules 2024-12-29 07:31:25 +05:00
Ayaz Salikhov
a41b6e80ce Add some statistics to wiki page 2024-12-29 06:51:06 +05:00
Ayaz Salikhov
c175251fdb [FAST_BUILD] Run mamba info without --quiet and put conda info in build manifest files (#2191) 2024-12-29 05:38:15 +05:00
Ayaz Salikhov
209a7464f4 Group build manifests by year 2024-12-29 02:36:27 +05:00
Ayaz Salikhov
2ba3651be1 Put manifest files by to year/year-month/ dirs 2024-12-29 01:30:11 +05:00
Ayaz Salikhov
c4cb04ec37 Make Spark scripts more robust: support preview versions and Spark 4 output 2024-10-19 19:08:04 +01:00
ChristofKaufmann
b9553a8e5d Add cuda12 variant of tensorflow-notebook (#2100)
* Add cuda12 variant for tensorflow-notebook

* Reduce size of CPU version of tensorflow-notebook

* Try to fix tests

* Update docs/using/selecting.md

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

* Update images/tensorflow-notebook/cuda12/Dockerfile

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

* Update tests/docker-stacks-foundation/test_packages.py

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

* Remove obsolete XLA_FLAGS env var

* Install CUDA and cuDNN using pip instead of mamba

* Fix pre-commit shell checks

* Change tensorflow variant name from cuda12 to cuda

* Update selecting.md

* Update selecting.md

---------

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2024-03-26 01:12:59 +00:00
Johanna Reiml
eccda243c3 Add pytorch-notebook image variants with cuda 11 and 12 (x86_64 versions only) (#2091)
* feat: build cuda variants of pytorch

* feat: build with variant tag

* style: remove unused import

* refactor: rename get_prefix params

(cherry picked from commit 12b50af258c2f331d4100fb63fd41ad1a30acb1d)

* revert: drop ROOT_CONTAINER addition from Makefile

(cherry picked from commit f42314513df2855957a05c6ba0c748d2df26d7b0)

* style: use consistent three empty lines in Makefile

(cherry picked from commit 446b45aab37a37720462b5df305ce96b139cf67a)

* refactor: add default value for parent-image

(cherry picked from commit 32955cec99c7202f0ce50647dfc61ec98f57f741)

* revert: use original workflow structure

(cherry picked from commit 68c6744513636ec93d14f9bd0bbd123907efd13b)

* refactor: use single build image step

(cherry picked from commit 5f1ac0aeedcb5969a6d4b2a5bc939817378ab55d)

* fix: run merge tags regardless of repository owner

(cherry picked from commit 3fce366a98adc5db0d127f28ddf3157d13297a0f)

* refactor: build cuda12 instead of cuda tag

(cherry picked from commit 217144ecd322356376f04efb92792a20b4380177)

* docs: add note about CUDA tags to documentation

* refactor: add default value for variant in build-test-upload

* refactor: swap ordering of cuda11/cuda12 variants

* refactor: remove optional str type in arg parser

* fix: add proper env variables to CUDA Dockerfiles

* fix: remove CUDA build for aarch64

* fix: use latest NVIDIA documentation link

* fix: skip aarch64 tags file for CUDA variants

---------

Co-authored-by: zynaa <7562909-zynaa@users.noreply.gitlab.com>
2024-02-24 10:17:38 +00:00
Ayaz Salikhov
bf33945b9e Do not bloat spark image with ENV variables (#2081)
* Do not bloat spark image with ENV variables

* Remove HadoopVersionTagger
2024-01-17 13:34:33 +04:00
Ayaz Salikhov
d599a7aeca Improve get_manifest_timestamp function 2024-01-14 15:41:00 +04:00
Ayaz Salikhov
4d2c81cd88 Update update_wiki.py: fix for pattern download, build timestamps, additional checks 2024-01-14 14:27:33 +04:00
Ayaz Salikhov
c294e9e2d9 Automatically install latest spark version (#2075)
* Automatically install latest pyspark version

* Better text

* Do not use shutil to keep behaviour

* Make setup_script cwd independent

* Use _get_program_version to calculate spark version

* Update setup_spark.py reqs

* Update setup_spark.py

* Add info about HADOOP_VERSION

* Add customization back

* Better text

* Specify build args when they are actually needed

* Better text

* Better code

* Better code

* Better text

* Get rid of warning

* Improve code

* Remove information about checksum

* Better text
2024-01-07 10:01:23 +04:00
Ayaz Salikhov
b6104850fa Automatically install latest julia version (#2046)
* Automatically install latest julia version

* Better text

* Fix

* Fix

* Update setup-julia.bash

* Install plumbum

* Better docs

* Better docs

* Use subprocess.check_call instead of plumbum

* Do not use dash in python filename

* Remove plumbum from the image

* Remove jq from the image

* Remove setup-julia.bash file

* Fix file name

* Fix docstring
2023-12-05 00:19:01 +01:00
Thijs Walcarius
278dd76837 Add jupyter/pytorch-notebook (#1936)
* Add PyTorch image

* Fix linting errors

* Fix link to pytorch website

* Address review comments

* Fix PytorchVersionTagger

* Remove "+cpu" suffix from pytorch version tag

* Update selecting.md

* Rename pytorch-notebook/.dockerignore to images/pytorch-notebook/.dockerignore

* Rename pytorch-notebook/Dockerfile to images/pytorch-notebook/Dockerfile

* Rename pytorch-notebook/README.md to images/pytorch-notebook/README.md

* Add pytorch-notebook to registry-overviews

* Add registry to pytorch image

* Use Quay.io

* Remove incorrect link

* Update action.yml

* Update docker.yml

* Remove information about Docker Hub, because this image won't be uploaded to Docker Hub

* Update docker.yml

* Update action.yml

* Add pytorch-notebook to registry-move.yml

---------

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@gmail.com>
2023-11-22 11:56:45 +01:00
Ayaz Salikhov
0a75e3d3a6 Minor improvements 2023-11-21 13:17:33 +01:00
Ayaz Salikhov
809672790e Fix usages of hist_lines_dir and manifests_dir 2023-11-19 13:42:50 +01:00
Ayaz Salikhov
69e5b1d5c2 Fix arg names for --hist-lines-dir and --manifests-dir 2023-11-19 13:34:18 +01:00
Ayaz Salikhov
d8c60bc42c Fix more grammar issues 2023-11-19 12:16:19 +01:00
Ayaz Salikhov
d03229331a Fix some grammar issues 2023-11-19 02:09:26 +01:00
Ayaz Salikhov
188ff1eb1b Update README.md 2023-11-19 00:34:13 +01:00
Ayaz Salikhov
cd2875a0da [FAST_BUILD] Remove latest tag in workflow to make local development easier (#2034)
* Remove latest tag in workflow to make local development easier

* Fix style

* Update docker-tag-push.yml

* Update docker-tag-push.yml
2023-11-17 23:01:33 +01:00
Ayaz Salikhov
49150068fc Add asserts to make sure wiki is always updated 2023-11-07 14:37:39 +01:00
Ayaz Salikhov
716683512e Fix table style on the home page 2023-11-07 14:35:37 +01:00
Ayaz Salikhov
3badd8f21c Improve text in tagging files 2023-11-07 00:38:23 +01:00