Add perf metrics for 2.24.0 #45708
Merged
Add perf metrics for 2.24.0 #45708
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
khluu
commented
Jun 4, 2024
khluu
commented
Signed-off-by: kevin <[email protected]>
jjyao
approved these changes
Jun 4, 2024
|
only json files. will just merge it. |
chungen04
pushed a commit
to chungen04/ray
that referenced
this pull request
Jun 18, 2024
``` REGRESSION 9.59%: 1_1_async_actor_calls_with_args_async (THROUGHPUT) regresses from 2512.820463128068 to 2271.9010969950546 in microbenchmark.json REGRESSION 6.74%: multi_client_put_gigabytes (THROUGHPUT) regresses from 38.09272404390616 to 35.52392832333356 in microbenchmark.json REGRESSION 6.37%: single_client_get_object_containing_10k_refs (THROUGHPUT) regresses from 13.30751068234607 to 12.459623096174152 in microbenchmark.json REGRESSION 5.44%: 1_1_async_actor_calls_async (THROUGHPUT) regresses from 3485.371511081248 to 3295.7761919502855 in microbenchmark.json REGRESSION 4.23%: single_client_put_gigabytes (THROUGHPUT) regresses from 20.46923471565212 to 19.60264129638186 in microbenchmark.json REGRESSION 4.09%: tasks_per_second (THROUGHPUT) regresses from 372.1993782750883 to 356.96608310545133 in benchmarks/many_nodes.json REGRESSION 3.17%: client__1_1_actor_calls_async (THROUGHPUT) regresses from 987.583992857757 to 956.234692691667 in microbenchmark.json REGRESSION 3.09%: 1_1_async_actor_calls_sync (THROUGHPUT) regresses from 1347.8868250169457 to 1306.2185283312654 in microbenchmark.json REGRESSION 3.04%: client__1_1_actor_calls_sync (THROUGHPUT) regresses from 527.6097359140301 to 511.54682034013 in microbenchmark.json REGRESSION 2.94%: client__1_1_actor_calls_concurrent (THROUGHPUT) regresses from 993.4252979888793 to 964.227785207118 in microbenchmark.json REGRESSION 2.25%: n_n_actor_calls_async (THROUGHPUT) regresses from 27321.88100222539 to 26706.0034223919 in microbenchmark.json REGRESSION 2.05%: client__get_calls (THROUGHPUT) regresses from 1122.3334401198258 to 1099.299059786817 in microbenchmark.json REGRESSION 1.45%: single_client_tasks_and_get_batch (THROUGHPUT) regresses from 7.730134993824188 to 7.618329296876913 in microbenchmark.json REGRESSION 0.76%: client__put_calls (THROUGHPUT) regresses from 799.7415900241213 to 793.6471952322032 in microbenchmark.json REGRESSION 0.64%: client__put_gigabytes (THROUGHPUT) regresses from 0.13117998295265082 to 0.13033848458516117 in microbenchmark.json REGRESSION 0.59%: n_n_async_actor_calls_async (THROUGHPUT) regresses from 23164.85974320361 to 23029.099137990735 in microbenchmark.json REGRESSION 0.46%: 1_1_actor_calls_async (THROUGHPUT) regresses from 8766.429278436297 to 8726.401857123754 in microbenchmark.json REGRESSION 0.25%: client__tasks_and_put_batch (THROUGHPUT) regresses from 11191.448976686781 to 11163.342583763659 in microbenchmark.json REGRESSION 0.10%: n_n_actor_calls_with_arg_async (THROUGHPUT) regresses from 2672.287170984248 to 2669.6470548189563 in microbenchmark.json REGRESSION 263.43%: dashboard_p95_latency_ms (LATENCY) regresses from 13.367 to 48.58 in benchmarks/many_nodes.json REGRESSION 26.19%: dashboard_p95_latency_ms (LATENCY) regresses from 2641.416 to 3333.28 in benchmarks/many_actors.json REGRESSION 11.55%: dashboard_p95_latency_ms (LATENCY) regresses from 3265.071 to 3642.242 in benchmarks/many_tasks.json REGRESSION 11.50%: dashboard_p99_latency_ms (LATENCY) regresses from 176.812 to 197.137 in benchmarks/many_nodes.json REGRESSION 6.69%: stage_4_spread (LATENCY) regresses from 0.4547737011485014 to 0.4851944753960548 in stress_tests/stress_test_many_tasks.json REGRESSION 5.77%: time_to_broadcast_1073741824_bytes_to_50_nodes (LATENCY) regresses from 15.855233231 to 16.769440987999985 in scalability/object_store.json REGRESSION 5.31%: avg_pg_remove_time_ms (LATENCY) regresses from 0.8566281216197303 to 0.9020739549548232 in stress_tests/stress_test_placement_group.json REGRESSION 3.71%: dashboard_p50_latency_ms (LATENCY) regresses from 3.367 to 3.492 in benchmarks/many_pgs.json REGRESSION 3.33%: dashboard_p50_latency_ms (LATENCY) regresses from 3.752 to 3.877 in benchmarks/many_nodes.json REGRESSION 3.22%: dashboard_p99_latency_ms (LATENCY) regresses from 4407.973 to 4549.74 in benchmarks/many_tasks.json REGRESSION 3.15%: 1000000_queued_time (LATENCY) regresses from 186.46388957200003 to 192.34304552800003 in scalability/single_node.json REGRESSION 1.22%: avg_iteration_time (LATENCY) regresses from 1.0421651029586791 to 1.0548744773864747 in stress_tests/stress_test_dead_actors.json REGRESSION 0.74%: avg_pg_create_time_ms (LATENCY) regresses from 0.9066000405417517 to 0.9132710030025517 in stress_tests/stress_test_placement_group.json ``` Signed-off-by: kevin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.