Improve full-content tests and possibly package them up as a testing tool for blocks #40773
Labels
Developer Experience
Ideas about improving block and theme developer experience
[Type] Automated Testing
Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests.
[Type] Enhancement
A suggestion for improvement.
What problem does this address?
The full-content tests and associated fixtures can be really useful for making sure blocks work as expected when making changes.
But they're undeniably hard to use. Even after reading the README file they can be difficult to understand and fix.
What is your proposed solution?
I think there's a series of improvements that can be made. I'll try to describe some of the problems and potential solutions. It'd be good to gather more of these examples in comments.
Problem: The tests are in a folder separate from the code that affects them. A lot of contributors don't know they exist at first (and are surprised when they fail).
Solution: Co-locate the fixture and snapshot files with the blocks in the block library. Use a glob to configure where the test should look for blocks.
Solution: Consider a naming convention where deprecations and their fixtures and snapshots are easily matched up.
Problem: The input fixture files for the tests (fixtures/blocks/block_name.html) are in the same place as the output for the tests. It's hard to know what each file does and how to regenerate the output files.
Solution: Store the input and the output separately. Consider using Jest snapshots for the output, or if that's not possible mimic the way jest snapshots work.
Problem: It's hard to create an accurate html fixture for block deprecations.
Solution: It might be possible to make a tool that produces the html based on an attributes object.
Problem: Every time a block's
save
function is changed, every output file needs to be regenerated, even those for deprecations.Solution: Snapshots become easier to update using Jest's snapshot system.
Solution: Deprecations should all upgrade to the current block output, so maybe multiple files aren't needed.
Solution: Reconsider what needs to be tested. Does the PEG parser need to be tested in this way? It's output directly feeds into the block parser, so this could be duplication. Are there assertions that don't use snapshots that could be written? For example, is there a better way to test block supports features, which are very similar across multiple blocks. Another example is testing for
isValid: true
without using a fixture, which is important for deprecations.Problem: A lot of third-party blocks could also do with a canonical way to test blocks
Solution: Move this tool to a reusable place (e.g.
scripts
or a different package) and package it up in a way that can be easily integrated into other projects.The text was updated successfully, but these errors were encountered: