Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate e2e Repro Specs #43505

Open
18 of 20 tasks
Tracked by #43320
iethree opened this issue Jun 3, 2024 · 1 comment
Open
18 of 20 tasks
Tracked by #43320

Consolidate e2e Repro Specs #43505

iethree opened this issue Jun 3, 2024 · 1 comment

Comments

@iethree
Copy link
Contributor

iethree commented Jun 3, 2024

There is significant (4-6 second) setup overhead in separate test files in CI. combining dozens of reproduction specs that all have 1 or 2 tests, provides several minutes of CI savings in many cases

We should consolidate all of these, even when there are only a couple offenders to eliminate the pattern from the codebase and to keep it from continuing.

Tasks

  1. .CI & Tests .Team/AdminWebapp no-backport
    iethree
  2. .CI & Tests .Team/AdminWebapp no-backport
    iethree
  3. .CI & Tests .Team/AdminWebapp no-backport
    iethree
  4. .CI & Tests .Team/AdminWebapp no-backport
    iethree
  5. .CI & Tests .Team/AdminWebapp no-backport
    iethree
  6. .CI & Tests .Team/AdminWebapp no-backport
    iethree
  7. .CI & Tests .Team/AdminWebapp no-backport
    iethree
  8. .CI & Tests .Team/AdminWebapp no-backport
    iethree
  9. .Team/AdminWebapp no-backport
    iethree
  10. .CI & Tests no-backport
    iethree
  11. .Team/DevEx no-backport
    iethree
  12. .CI & Tests .Team/DevEx no-backport
    iethree
  13. .CI & Tests .Team/DevEx no-backport
    iethree
  14. .Team/DevEx no-backport
    iethree
  15. .Team/DevEx no-backport
    iethree
  16. .Team/DevEx no-backport
    iethree
  17. .CI & Tests .Team/DevEx backport
    iethree
  18. .CI & Tests .Team/DevEx no-backport
    iethree
@iethree
Copy link
Contributor Author

iethree commented Jul 16, 2024

One down side to this project is that some developers find working with large files to be rather cumbersome.

One alternative approach suggested by @romeovs is keeping test files small, but running them via imports into larger files, which should solve the performance problem.

see discussion and proof of concept PR:

One open question:

  • Do most developers find large files hard to work with?

One blocker:

  • we need a linter that can ensure that all spec files are imported into runner files so that we don't end up with orphan spec files that never get run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant