Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] Add benchmark for Ray Data + Trainer #37624

Merged
merged 21 commits into from
Jul 25, 2023
Merged

[Data] Add benchmark for Ray Data + Trainer #37624

Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b98e1a6
initial benchmark code
scottjlee Jul 20, 2023
9635e22
add into release tests
scottjlee Jul 21, 2023
f103a51
lint
scottjlee Jul 21, 2023
f6c2391
update release test pathing
scottjlee Jul 21, 2023
0e9c354
relative pathing
scottjlee Jul 21, 2023
ec75d20
Merge branch 'master' into 0720-benchmark-train
scottjlee Jul 21, 2023
2baba01
move working dir to air_tests instead
scottjlee Jul 21, 2023
3391671
add 4/16 node tests
scottjlee Jul 21, 2023
2093c62
Merge branch 'master' into 0720-benchmark-train
scottjlee Jul 21, 2023
bd8f866
update benchmark utils
scottjlee Jul 21, 2023
6b498ee
update script params
scottjlee Jul 21, 2023
0a80e2f
fix broken cluster env/compute directories and relative import with u…
scottjlee Jul 21, 2023
e8df198
test import paths
scottjlee Jul 22, 2023
f749365
move utils to benchmark.py
scottjlee Jul 22, 2023
8b4697b
try single node
scottjlee Jul 24, 2023
2da2175
Merge branch 'master' into 0720-benchmark-train
scottjlee Jul 24, 2023
2366d9c
revert refactor
scottjlee Jul 24, 2023
82ce8c6
lint
scottjlee Jul 24, 2023
fbac655
Merge branch 'master' into 0720-benchmark-train
scottjlee Jul 24, 2023
a1416eb
resource yamls
scottjlee Jul 25, 2023
25e6f76
yaml path
scottjlee Jul 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
yaml path 
Signed-off-by: Scott Lee <[email protected]>
  • Loading branch information
@scottjlee
scottjlee committed Jul 25, 2023
commit 25e6f76e33b59a22b5fb69219a59c3c770512c6b
4 changes: 2 additions & 2 deletions release/release_tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6206,7 +6206,7 @@
byod:
type: gpu
cluster_env: app_config.yaml
cluster_compute: ../../air_tests/air_benchmarks/mlperf-train/compute_gpu_4x4_aws.yaml
cluster_compute: ../../air_tests/air_benchmarks/compute_gpu_4x4_aws.yaml

run:
timeout: 600
Expand All @@ -6219,7 +6219,7 @@
frequency: manual
cluster:
cluster_env: app_config.yaml
cluster_compute: ../../air_tests/air_benchmarks/mlperf-train/compute_gpu_4x4_gce.yaml
cluster_compute: ../../air_tests/air_benchmarks/compute_gpu_4x4_gce.yaml

- name: read_tfrecords_benchmark_single_node
group: data-tests
Expand Down
Loading