Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add vitest benchmark #583

Merged
merged 11 commits into from
Feb 3, 2024
Merged

Add vitest benchmark #583

merged 11 commits into from
Feb 3, 2024

Conversation

Casheeew
Copy link
Collaborator

First step to #87 . Adds some benchmarks for high bandwidth functions.
However, vitest benchmark is still experimental, so not sure about maintainers' thoughts.

Copy link

github-actions bot commented Jan 28, 2024

✔️ No visual differences introduced by this PR.

View Playwright Report (note: open the "playwright-report" artifact)

@djahandarie
Copy link
Collaborator

Very interesting, thanks! Maybe we could also consider using something like https://codspeed.io/blog/vitest-bench-performance-regressions ? (It seems to be free for OSS projects and looks more like the desired experience we would want on PRs. But I haven't looked at other potential solutions, it just came up from basic Google searching)

@Casheeew Casheeew marked this pull request as ready for review February 2, 2024 03:26
@Casheeew Casheeew requested a review from a team as a code owner February 2, 2024 03:26
@Casheeew
Copy link
Collaborator Author

Casheeew commented Feb 2, 2024

Can you setup the action in the CI?

@djahandarie
Copy link
Collaborator

@Scrub1492 I did the setup on the Codspeed side, and added the CODSPEED_TOKEN within our CI. Would you be able to add the contents of the action here: https://docs.codspeed.io/ci/github-actions to your PR?

@Casheeew
Copy link
Collaborator Author

Casheeew commented Feb 3, 2024

weird, it was working locally

@Casheeew
Copy link
Collaborator Author

Casheeew commented Feb 3, 2024

Does anyone have any idea what's breaking? not too familiar with github workflows 😅

@toasted-nutbread
Copy link

Benchmarking is an experimental feature.
Breaking changes might not follow SemVer, please pin Vitest's version when using it.

Something from the CI logs. Based on this, if you get this working, we may want to do two things:

  1. Update the vitest dependencies manually to the latest version (see Bump the minor group with 17 updates #612) and pin them.
  2. Test to make sure it works with the latest version.

@djahandarie
Copy link
Collaborator

https://github.com/CodSpeedHQ/codspeed-node/tree/main/packages/vitest-plugin:

The CodSpeed plugin is only compatible with [email protected] and above.

I believe this branch is still on vitest 1.1.1 because it's not on top of #612

@toasted-nutbread toasted-nutbread mentioned this pull request Feb 3, 2024
themoeway-bot
themoeway-bot previously approved these changes Feb 3, 2024
@Casheeew Casheeew mentioned this pull request Feb 3, 2024
Copy link

codspeed-hq bot commented Feb 3, 2024

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕 1 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

  • transformations (235.4 ms)

@djahandarie djahandarie added this pull request to the merge queue Feb 3, 2024
Merged via the queue into themoeway:master with commit 3fc7362 Feb 3, 2024
7 checks passed
@Casheeew Casheeew deleted the bench branch February 3, 2024 08:56
@djahandarie djahandarie added the kind/meta The issue or PR is meta label Feb 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/meta The issue or PR is meta
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants