Optimize build commands & building the plugin js #20874

d-claassen · 2023-11-24T13:57:10Z

Context

Introduces build responsibility (and Yarn commands) for all separate packages in the monorepo.
Makes the building of the packages & the plugin js use the built versions of the other monorepo packages (via the main key in each package.json).
Builds the plugin js in a hierarchical order, where the lowest packages in the monorepo chain are build first when running $ yarn build and packages on the same hierarchical level are built in parallel.
Introduces better reliance on the NODE_ENV variable for production/development builds.
Follow-up: Move build tools for the plugin itself (packages/js) from the root package.json to its own package.json, including other files specifically for building the plugin that now live in the root.
Follow-up: Optimize building of all projects with a build-cache, where unchanged packages don't need a re-build.

Summary

This PR can be summarized in the following changelog entry:

Optimizes development processes for building JavaScript.

Relevant technical choices:

Replaced the Grunt alias shell:build-ui-library with generic shell:build-packages, powered by Lerna.
Introduced separate build commands for all packages.
- Introduced or improved PostCSS and Babel configuration for building of individual packages.
- Introduced or improved ESLint and Jest configuration to support (usage of) built packages in the monorepo. Used existing package specific eslintrc files, or added to the monorepo .eslintrc file for packages without their own file (following current practices).
Made the $ yarn build command in the root also build all packages, with the help of Lerna.
Updated grunt-eslint in yoastseo to align the included ESLint version with other packages (now all on v7.23.0)
Updates Jest and @wordpress/scripts to align the included Jest version across packages (now all on v27.5.1)
Updates the CI pipeline so the JS tests build the packages before running the tests: these tests now use the built version of the monorepo packages.
Updates existing snapshots:
- Packages are no longer applying styled-components or other packages, and are therefore ending up with different class names.
- Updated version of Jest and ESLint come with new errors or stricter assertions.
Adds to the Jest config a yoastseo specific module name mapper to ensure import .. from "yoastseo" in the package itself are resolved to the src files, and not to the build files.
I've disabled the tests for the UI library. There's a whole story behind it, but the current tool will be deprecated in v7.6 and fully replaced in v8. In the mean time, it's bringing old sub-dependencies complicating the state of our monorepo's node_modules structure. Since we're investigating updates to the Storybook, I've decided to jump ahead of the future deprecation.

Test instructions

Test instructions for the acceptance test before the PR gets merged

This PR can be acceptance tested by following these steps:

Check that CI continues to succeed
Run $ yarn lint in the root of the project.
Run $ yarn build in the root of the project.
Run $ yarn test in the root of the project.
Run $ yarn workspace @yoast/ui-library storybook in the root of the project.
Run $ yarn why jest to see a ~~single~~ two installments (27 & 29) of Jest in the node_modules, preventing conflicts throughout the monorepo.
Smoke test of the full sidebar/metabox, and settings pages.

Relevant test scenarios

Changes should be tested with the browser console open
Changes should be tested on different posts/pages/taxonomies/custom post types/custom taxonomies
Changes should be tested on different editors (Block/Classic/Elementor/other)
Changes should be tested on different browsers
Changes should be tested on multisite

Test instructions for QA when the code is in the RC

QA should use the same steps as above.

Impact check

This PR affects the following parts of the plugin, which may require extra testing:

All the JS, but it should not be none of the JS.

UI changes

This PR changes the UI in the plugin. I have added the 'UI change' label to this PR.

Other environments

This PR also affects Shopify. I have added a changelog entry starting with [shopify-seo], added test instructions for Shopify and attached the Shopify label to this PR.

Documentation

I have written documentation for this change.

Quality assurance

I have tested this code to the best of my abilities.
During testing, I had activated all plugins that Yoast SEO provides integrations for.
I have added unit tests to verify the code works as intended.
If any part of the code is behind a feature flag, my test instructions also cover cases where the feature flag is switched off.
I have written this PR in accordance with my team's definition of done.
I have checked that the base branch is correctly set.

Innovation

No innovation project is applicable for this PR.
This PR falls under an innovation project. I have attached the innovation label.
I have added my hours to the WBSO document.

Fixes #

packages/js/package.json

github-actions · 2024-01-04T16:18:08Z

@d-claassen Please be aware that following packages have been abandoned and are not actively maintained anymore:

Package name	Path
yoast-components	packages/yoast-components
@yoast/babel-preset	packages/babel-preset
@yoast/components	packages/components
@yoast/e2e-tests	packages/e2e-tests
@yoast/helpers	packages/helpers
@yoast/jest-preset	packages/jest-preset
@yoast/style-guide	packages/style-guide

Please consider using the other packages instead.

packages/components/package.json

coveralls · 2024-01-09T14:19:08Z

Pull Request Test Coverage Report for Build bbc42abf07c4b5d013194c0c400ca96d0b9a4946

Details

1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.07%) to 53.244%

Totals
Change from base Build 14a551f319c28c9acdff064bc91beec9741b1b8a:	0.07%
Covered Lines:	29059
Relevant Lines:	54895

💛 - Coveralls

d-claassen · 2024-01-17T13:52:05Z

After an extensive CR session with @igorschoester, we found these steps still need to be taken:

Don't bluntly ignore unresolved imports for all @yoast/* packages, but specify the packages from the monorepo explicitly.
Add dependencies on new tools in the package.json file for each package. So add Babel, presets, plugins, etc.
Split up the yarn build script of the yoastseo package into separate smaller scripts

igorschoester

CR 🏗️ ACC ✅

packages/eslint/default.yml

We already have Lerna installed to run linting and tests on all monorepo packages. Lerna has the option to run commands in the packages in a topological order, this means it'll follow the dependency tree from the bottom upwards to the start. So if package A depends on package B, Lerna will first build B before it builds A. This ensures that the build for package A uses the built version of package B. This will build all packages individually, like `@yoast/ui-library`, so removes the need for separate build commands for these packages. It also builds `yoastseo` and `@yoast/analysis-report`. For those packages however, the `main` configuration in the `package.json` file does not point to the build directory. So for these packages, the source files will remain to be used. The old Grunt alias `build-ui-library` is preserved for backward compatibility. It now utilizes the `yarn workspace <pkgName> <cmd>` command to allow more flexibility in the location of the package within the project.

Build sourcemap for analysis-report in development

- Remove unneeded try-catch. This new lint error appeared. It seemed best to just resolve it. - Ignore missing dangling commas in tests - Ignore import/no-unresolved for the premium configuration - Use `jest.spyOn` to replace missing global `spyOn` - Fix paper comparison with partial expectation When an expectation in asynchronous test fails, the `done` callback is never called and the test hangs until the timeout. Calling the `done` callback in the `try..catch` block ensures the test will finish as soon as possible. - Fixes tests that were hitting caches - Disable inclusive language config test - Ensure all `yoastseo` references load from src Without this line, certain `import .. from "yoastseo"` statements would load components from the `build/` directory when it exists. - Clear cache in tests - Remove old `grunt publish` from `yoastseo` Explain fuzzy Jest config for moduleNameMapper

Packages might depend on other packages. Because those packages now have their `main` property pointing to the build version, those build versions need to exist to make the tests pass. This doesn't build the plugin js itself. It has dependencies on javascript from WordProof which would have be built with composer prefixing. That would only lengthen the builds. But the plugin JavaScript isn't needed, because it is the top of the foodchain and will never be a dependency for any of the other packages.

- ESLint to ignore new standard "build" folder - Specify packages to allow unresolved imports from - Align babel-core and jest versions in the monorepo

igorschoester

CR && AT ✅ (once again)

Awaiting merge freeze (22.3-RC1 creation) before merging

d-claassen added innovation Innovative issue. Relating to performance, memory or data-flow. changelog: non-user-facing Needs to be included in the 'Non-userfacing' category in the changelog Shopify This PR impacts Shopify. labels Nov 24, 2023

igorschoester reviewed Nov 29, 2023

View reviewed changes

packages/js/package.json Show resolved Hide resolved

diedexx reviewed Jan 5, 2024

View reviewed changes

packages/components/package.json Outdated Show resolved Hide resolved

d-claassen marked this pull request as ready for review January 9, 2024 15:59

d-claassen requested a review from a team as a code owner January 9, 2024 15:59

d-claassen changed the title ~~[WIP] Optimize build commands & building the plugin js~~ Optimize build commands & building the plugin js Jan 11, 2024

igorschoester self-assigned this Jan 16, 2024

d-claassen force-pushed the d-claassen/optimize-build-commands branch 2 times, most recently from 3e5f4dc to 4c5076c Compare February 28, 2024 10:06

igorschoester requested changes Feb 29, 2024

View reviewed changes

packages/eslint/default.yml Outdated Show resolved Hide resolved

igorschoester assigned d-claassen and unassigned igorschoester Feb 29, 2024

manuelaugustin added the technical-debt label Feb 29, 2024

d-claassen added 12 commits March 1, 2024 09:05

Build @yoast/helpers separately

17bfc9a

Build @yoast/feature-flag

a9c37a6

Build @yoast/style-guide separately

54f484d

Build @yoast/components

17e0fa2

Build @yoast/analysis-report

b5b2280

Build sourcemap for analysis-report in development

Build @yoast/ui-library

9944c50

Build @yoast/replacement-variable-editor separately

fcd90ec

Build @yoast/search-metadata-preview separately

8ff0051

Build @yoast/social-metadata-forms separately

2e54775

Build @yoast/social-metadata-preview separately

5d36540

d-claassen added 5 commits March 1, 2024 09:05

Build @yoast/wordpress-seo

74dde26

Clean yoast-components dependencies

84ac1d7

Update tooling across monorepo

dd7d07a

- ESLint to ignore new standard "build" folder - Specify packages to allow unresolved imports from - Align babel-core and jest versions in the monorepo

Add missing Storyshot snapshots

d682985

d-claassen force-pushed the d-claassen/optimize-build-commands branch from 4c5076c to d682985 Compare March 1, 2024 08:28

d-claassen requested a review from igorschoester March 1, 2024 08:29

igorschoester assigned igorschoester and unassigned d-claassen Mar 4, 2024

igorschoester approved these changes Mar 4, 2024

View reviewed changes

igorschoester added this to the 22.4 milestone Mar 5, 2024

igorschoester merged commit a9632c1 into trunk Mar 5, 2024
19 checks passed

igorschoester deleted the d-claassen/optimize-build-commands branch March 5, 2024 14:08

igorschoester mentioned this pull request Mar 5, 2024

Fix GH actions' JS tests when not needing to run yoastseo tests #21195

Merged

18 tasks

leonidasmi mentioned this pull request Mar 6, 2024

Add cross-env before using NODE_ENV #21199

Merged

18 tasks

igorschoester mentioned this pull request Mar 7, 2024

Upgrade UI library dependencies #20944

Merged

18 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimize build commands & building the plugin js #20874

Optimize build commands & building the plugin js #20874

d-claassen commented Nov 24, 2023 •

edited by igorschoester

Loading

github-actions bot commented Jan 4, 2024 •

edited

Loading

coveralls commented Jan 9, 2024 •

edited

Loading

d-claassen commented Jan 17, 2024 •

edited

Loading

igorschoester left a comment

igorschoester left a comment

Optimize build commands & building the plugin js #20874

Optimize build commands & building the plugin js #20874

Conversation

d-claassen commented Nov 24, 2023 • edited by igorschoester Loading

Context

Summary

Relevant technical choices:

Test instructions

Test instructions for the acceptance test before the PR gets merged

Relevant test scenarios

Test instructions for QA when the code is in the RC

Impact check

UI changes

Other environments

Documentation

Quality assurance

Innovation

github-actions bot commented Jan 4, 2024 • edited Loading

coveralls commented Jan 9, 2024 • edited Loading

Pull Request Test Coverage Report for Build bbc42abf07c4b5d013194c0c400ca96d0b9a4946

Details

💛 - Coveralls

d-claassen commented Jan 17, 2024 • edited Loading

igorschoester left a comment

Choose a reason for hiding this comment

igorschoester left a comment

Choose a reason for hiding this comment

d-claassen commented Nov 24, 2023 •

edited by igorschoester

Loading

github-actions bot commented Jan 4, 2024 •

edited

Loading

coveralls commented Jan 9, 2024 •

edited

Loading

d-claassen commented Jan 17, 2024 •

edited

Loading