Commit Graph

157 Commits

Author SHA1 Message Date
Ayaz Salikhov
0b16cf8dbf Simplify unify_aarch64 2025-01-26 20:53:33 +00:00
Ayaz Salikhov
7e4e18403a Create all necessary directories during wiki update 2025-01-06 18:27:47 +05:00
Ayaz Salikhov
a6ef176ff3 Make commits a url on wiki home page 2024-12-31 21:16:33 +05:00
Ayaz Salikhov
ad4d4c6e6e Add yearly total stats to update_wiki 2024-12-31 20:41:30 +05:00
Ayaz Salikhov
a61c9ae8a7 Improve commits counting: inly count merge commits, calc between midnights UTC 2024-12-31 20:41:25 +05:00
Ayaz Salikhov
d812d68234 Allow no new manfests or build history lines files if special flag is present during update_wiki 2024-12-30 22:15:19 +05:00
Ayaz Salikhov
af0493a305 Add number of commits to wiki home page 2024-12-30 22:10:14 +05:00
Ayaz Salikhov
67dd54966c Fix args order when running python modules 2024-12-29 07:31:25 +05:00
Ayaz Salikhov
a41b6e80ce Add some statistics to wiki page 2024-12-29 06:51:06 +05:00
Ayaz Salikhov
c175251fdb [FAST_BUILD] Run mamba info without --quiet and put conda info in build manifest files (#2191) 2024-12-29 05:38:15 +05:00
Ayaz Salikhov
209a7464f4 Group build manifests by year 2024-12-29 02:36:27 +05:00
Ayaz Salikhov
2ba3651be1 Put manifest files by to year/year-month/ dirs 2024-12-29 01:30:11 +05:00
Ayaz Salikhov
c4cb04ec37 Make Spark scripts more robust: support preview versions and Spark 4 output 2024-10-19 19:08:04 +01:00
ChristofKaufmann
b9553a8e5d Add cuda12 variant of tensorflow-notebook (#2100)
* Add cuda12 variant for tensorflow-notebook

* Reduce size of CPU version of tensorflow-notebook

* Try to fix tests

* Update docs/using/selecting.md

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

* Update images/tensorflow-notebook/cuda12/Dockerfile

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

* Update tests/docker-stacks-foundation/test_packages.py

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>

* Remove obsolete XLA_FLAGS env var

* Install CUDA and cuDNN using pip instead of mamba

* Fix pre-commit shell checks

* Change tensorflow variant name from cuda12 to cuda

* Update selecting.md

* Update selecting.md

---------

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2024-03-26 01:12:59 +00:00
Johanna Reiml
eccda243c3 Add pytorch-notebook image variants with cuda 11 and 12 (x86_64 versions only) (#2091)
* feat: build cuda variants of pytorch

* feat: build with variant tag

* style: remove unused import

* refactor: rename get_prefix params

(cherry picked from commit 12b50af258c2f331d4100fb63fd41ad1a30acb1d)

* revert: drop ROOT_CONTAINER addition from Makefile

(cherry picked from commit f42314513df2855957a05c6ba0c748d2df26d7b0)

* style: use consistent three empty lines in Makefile

(cherry picked from commit 446b45aab37a37720462b5df305ce96b139cf67a)

* refactor: add default value for parent-image

(cherry picked from commit 32955cec99c7202f0ce50647dfc61ec98f57f741)

* revert: use original workflow structure

(cherry picked from commit 68c6744513636ec93d14f9bd0bbd123907efd13b)

* refactor: use single build image step

(cherry picked from commit 5f1ac0aeedcb5969a6d4b2a5bc939817378ab55d)

* fix: run merge tags regardless of repository owner

(cherry picked from commit 3fce366a98adc5db0d127f28ddf3157d13297a0f)

* refactor: build cuda12 instead of cuda tag

(cherry picked from commit 217144ecd322356376f04efb92792a20b4380177)

* docs: add note about CUDA tags to documentation

* refactor: add default value for variant in build-test-upload

* refactor: swap ordering of cuda11/cuda12 variants

* refactor: remove optional str type in arg parser

* fix: add proper env variables to CUDA Dockerfiles

* fix: remove CUDA build for aarch64

* fix: use latest NVIDIA documentation link

* fix: skip aarch64 tags file for CUDA variants

---------

Co-authored-by: zynaa <7562909-zynaa@users.noreply.gitlab.com>
2024-02-24 10:17:38 +00:00
Ayaz Salikhov
bf33945b9e Do not bloat spark image with ENV variables (#2081)
* Do not bloat spark image with ENV variables

* Remove HadoopVersionTagger
2024-01-17 13:34:33 +04:00
Ayaz Salikhov
d599a7aeca Improve get_manifest_timestamp function 2024-01-14 15:41:00 +04:00
Ayaz Salikhov
4d2c81cd88 Update update_wiki.py: fix for pattern download, build timestamps, additional checks 2024-01-14 14:27:33 +04:00
Ayaz Salikhov
c294e9e2d9 Automatically install latest spark version (#2075)
* Automatically install latest pyspark version

* Better text

* Do not use shutil to keep behaviour

* Make setup_script cwd independent

* Use _get_program_version to calculate spark version

* Update setup_spark.py reqs

* Update setup_spark.py

* Add info about HADOOP_VERSION

* Add customization back

* Better text

* Specify build args when they are actually needed

* Better text

* Better code

* Better code

* Better text

* Get rid of warning

* Improve code

* Remove information about checksum

* Better text
2024-01-07 10:01:23 +04:00
Ayaz Salikhov
b6104850fa Automatically install latest julia version (#2046)
* Automatically install latest julia version

* Better text

* Fix

* Fix

* Update setup-julia.bash

* Install plumbum

* Better docs

* Better docs

* Use subprocess.check_call instead of plumbum

* Do not use dash in python filename

* Remove plumbum from the image

* Remove jq from the image

* Remove setup-julia.bash file

* Fix file name

* Fix docstring
2023-12-05 00:19:01 +01:00
Thijs Walcarius
278dd76837 Add jupyter/pytorch-notebook (#1936)
* Add PyTorch image

* Fix linting errors

* Fix link to pytorch website

* Address review comments

* Fix PytorchVersionTagger

* Remove "+cpu" suffix from pytorch version tag

* Update selecting.md

* Rename pytorch-notebook/.dockerignore to images/pytorch-notebook/.dockerignore

* Rename pytorch-notebook/Dockerfile to images/pytorch-notebook/Dockerfile

* Rename pytorch-notebook/README.md to images/pytorch-notebook/README.md

* Add pytorch-notebook to registry-overviews

* Add registry to pytorch image

* Use Quay.io

* Remove incorrect link

* Update action.yml

* Update docker.yml

* Remove information about Docker Hub, because this image won't be uploaded to Docker Hub

* Update docker.yml

* Update action.yml

* Add pytorch-notebook to registry-move.yml

---------

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@gmail.com>
2023-11-22 11:56:45 +01:00
Ayaz Salikhov
0a75e3d3a6 Minor improvements 2023-11-21 13:17:33 +01:00
Ayaz Salikhov
809672790e Fix usages of hist_lines_dir and manifests_dir 2023-11-19 13:42:50 +01:00
Ayaz Salikhov
69e5b1d5c2 Fix arg names for --hist-lines-dir and --manifests-dir 2023-11-19 13:34:18 +01:00
Ayaz Salikhov
d8c60bc42c Fix more grammar issues 2023-11-19 12:16:19 +01:00
Ayaz Salikhov
d03229331a Fix some grammar issues 2023-11-19 02:09:26 +01:00
Ayaz Salikhov
188ff1eb1b Update README.md 2023-11-19 00:34:13 +01:00
Ayaz Salikhov
cd2875a0da [FAST_BUILD] Remove latest tag in workflow to make local development easier (#2034)
* Remove latest tag in workflow to make local development easier

* Fix style

* Update docker-tag-push.yml

* Update docker-tag-push.yml
2023-11-17 23:01:33 +01:00
Ayaz Salikhov
49150068fc Add asserts to make sure wiki is always updated 2023-11-07 14:37:39 +01:00
Ayaz Salikhov
716683512e Fix table style on the home page 2023-11-07 14:35:37 +01:00
Ayaz Salikhov
3badd8f21c Improve text in tagging files 2023-11-07 00:38:23 +01:00
Ayaz Salikhov
87afdf9689 Better logs 2023-11-06 20:49:36 +01:00
Ayaz Salikhov
e0c2997d6e Better logs 2023-11-06 20:48:54 +01:00
Ayaz Salikhov
9ab127bccb Put monthly-files in its own folder in docs 2023-11-06 20:47:21 +01:00
Ayaz Salikhov
3d87ce2006 Create missing dir for manifest month 2023-11-06 20:40:44 +01:00
Ayaz Salikhov
b97d0d6d91 Actually put manifests in monthly folders, not daily 2023-11-06 20:35:30 +01:00
Ayaz Salikhov
5a2225e126 Improve logs during manifests handle 2023-11-06 20:29:25 +01:00
Ayaz Salikhov
74dfec85fb Remove old manifests and put everything in folders (#2026) 2023-11-06 16:02:08 +01:00
Ayaz Salikhov
5f31a1fdba Simpler month_line 2023-11-06 00:59:04 +01:00
Ayaz Salikhov
8521680eb2 Simpler wiki home page table 2023-11-06 00:55:25 +01:00
Ayaz Salikhov
156cc18dd8 Improve update_wiki.py 2023-11-06 00:48:41 +01:00
Ayaz Salikhov
3f5cc4fdf2 Refactor wiki page to put everything in monthly files 2023-11-06 00:26:54 +01:00
Ayaz Salikhov
641e13e933 Add command which was run to the markdown piece 2023-11-05 20:40:04 +01:00
Ayaz Salikhov
2401290ab3 Make manifests and manifests.py itself more beautiful 2023-11-05 20:30:41 +01:00
Ayaz Salikhov
a73ba5a76d Specify code blocks in manifests 2023-11-05 14:43:00 +01:00
Ayaz Salikhov
61e271032d Make manifests more beautiful 2023-11-03 10:12:29 +01:00
Ayaz Salikhov
b331741540 Start new sentences on a new line in markdown 2023-10-21 20:11:49 +02:00
Ayaz Salikhov
542eae692c Write registry to a file with all tags 2023-10-21 00:11:03 +02:00
Ayaz Salikhov
00a6728161 Move from Docker Hub to quay.io (#2010)
* Move from Docker Hub to quay.io

* Fix http->https

* Update registry-overviews

* Remove Docker Hub name
2023-10-20 22:31:45 +02:00
Ayaz Salikhov
848a826747 Fix build manifest image size and improve name 2023-10-20 03:45:03 +02:00