Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve full-content tests and possibly package them up as a testing tool for blocks #40773

Open
talldan opened this issue May 3, 2022 · 0 comments
Labels
Developer Experience Ideas about improving block and theme developer experience [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Enhancement A suggestion for improvement.

Comments

@talldan
Copy link
Contributor

talldan commented May 3, 2022

What problem does this address?

The full-content tests and associated fixtures can be really useful for making sure blocks work as expected when making changes.

But they're undeniably hard to use. Even after reading the README file they can be difficult to understand and fix.

What is your proposed solution?

I think there's a series of improvements that can be made. I'll try to describe some of the problems and potential solutions. It'd be good to gather more of these examples in comments.

Problem: The tests are in a folder separate from the code that affects them. A lot of contributors don't know they exist at first (and are surprised when they fail).
Solution: Co-locate the fixture and snapshot files with the blocks in the block library. Use a glob to configure where the test should look for blocks.
Solution: Consider a naming convention where deprecations and their fixtures and snapshots are easily matched up.

Problem: The input fixture files for the tests (fixtures/blocks/block_name.html) are in the same place as the output for the tests. It's hard to know what each file does and how to regenerate the output files.
Solution: Store the input and the output separately. Consider using Jest snapshots for the output, or if that's not possible mimic the way jest snapshots work.

Problem: It's hard to create an accurate html fixture for block deprecations.
Solution: It might be possible to make a tool that produces the html based on an attributes object.

Problem: Every time a block's save function is changed, every output file needs to be regenerated, even those for deprecations.
Solution: Snapshots become easier to update using Jest's snapshot system.
Solution: Deprecations should all upgrade to the current block output, so maybe multiple files aren't needed.
Solution: Reconsider what needs to be tested. Does the PEG parser need to be tested in this way? It's output directly feeds into the block parser, so this could be duplication. Are there assertions that don't use snapshots that could be written? For example, is there a better way to test block supports features, which are very similar across multiple blocks. Another example is testing for isValid: true without using a fixture, which is important for deprecations.

Problem: A lot of third-party blocks could also do with a canonical way to test blocks
Solution: Move this tool to a reusable place (e.g. scripts or a different package) and package it up in a way that can be easily integrated into other projects.

@talldan talldan added [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. Developer Experience Ideas about improving block and theme developer experience labels May 3, 2022
@jordesign jordesign added the [Type] Enhancement A suggestion for improvement. label Sep 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Developer Experience Ideas about improving block and theme developer experience [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Enhancement A suggestion for improvement.
Projects
None yet
Development

No branches or pull requests

2 participants