Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Order of "types" for generic benchmark is off #64

Closed
dvdplm opened this issue Sep 30, 2024 · 1 comment
Closed

Order of "types" for generic benchmark is off #64

dvdplm opened this issue Sep 30, 2024 · 1 comment

Comments

@dvdplm
Copy link

dvdplm commented Sep 30, 2024

When setting up a generic benchmark using the types macro argument I noticed that the order is not respected when the benchmark runs and the types in question are generic.

With an invocation like so:

#[divan::bench(types = [U256, U1024, U2048, U4096])
fn gcd<T>(bencher: Bencher<'_, '_>) {
  // SNIP
}

…I would expect the benchmark to run over the types in the order listed (U256, U1024, U2048, U4096). Instead, what I see is:

divan              fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ benches                       │               │               │               │         │
   ╰─ gcd                        │               │               │               │         │
      ├─ Uint<16>  199.8 ns      │ 3.569 µs      │ 223.7 ns      │ 228.2 ns      │ 273839  │ 2190712
      ├─ Uint<32>  274.9 ns      │ 4.593 µs      │ 322.7 ns      │ 327.9 ns      │ 320700  │ 1282800
      ├─ Uint<4>   140.3 ns      │ 1.21 µs       │ 155.9 ns      │ 157.5 ns      │ 273881  │ 4382096
      ╰─ Uint<64>  468.4 ns      │ 4.889 µs      │ 520.7 ns      │ 531 ns        │ 168361  │ 673444

What seems to be happening is that the proc macro resolves the types into Uint<const L: usize> and then orders them lexicographically (?) such that the "4" in "Uint<4>" ends up after "Uint<16>".

Is this something that is easy to fix? I think the least surprising behaviour would be to respect the order entered, e.g. types = [Vec<u8>, ZedType, AbcType] would run using first Vec<u8>, then ZedType and lastly AbcType.

@nvzqz
Copy link
Owner

nvzqz commented Nov 1, 2024

This will be fixed in the next release soon. Thanks for pointing it out!

@nvzqz nvzqz closed this as completed in 5212444 Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants