Skip to content

Commit

Permalink
Always run baseline benchmark first, even with --only
Browse files Browse the repository at this point in the history
Summary: Otherwise, we won't end up calculating some metrics because we mistakenly think that a baseline is lacking.

Reviewed By: xuzhao9

Differential Revision: D58503025

fbshipit-source-id: 17e614794f29c07a62ab55380057905bb8eb8a60
  • Loading branch information
int3 authored and facebook-github-bot committed Jun 14, 2024
1 parent b91caad commit 95b4202
Showing 1 changed file with 9 additions and 9 deletions.
18 changes: 9 additions & 9 deletions torchbenchmark/util/triton_op.py
Original file line number Diff line number Diff line change
Expand Up @@ -526,15 +526,15 @@ def run(
if self.name in ENABLED_BENCHMARKS
else []
)
# Run the baseline first, if baseline exists
baseline_name = (
BASELINE_BENCHMARKS[self.name]
if self.name in BASELINE_BENCHMARKS
else None
)
if baseline_name and baseline_name in benchmarks:
benchmarks.remove(baseline_name)
benchmarks.insert(0, baseline_name)
# Run the baseline first, if baseline exists
baseline_name = (
BASELINE_BENCHMARKS[self.name]
if self.name in BASELINE_BENCHMARKS
else None
)
if baseline_name and baseline_name in benchmarks:
benchmarks.remove(baseline_name)
benchmarks.insert(0, baseline_name)

# get metrics for for each registered benchmark
def _reduce_benchmarks(acc, bm_name: str):
Expand Down

0 comments on commit 95b4202

Please sign in to comment.