Block library: Try to use Babel plugins to inline block.json metadata

[gziolo]

Description

It's based on RFC: #13693.

We discussed it with @youknowriad earlier today. I think it looks very promising, the idea is that we inline block.json on the client to keep everything backward compatible with Drupal, Calypso, mobile
it also allows us to completely remove the existing script which generated server-side rendered definitions:
https://github.com/WordPress/gutenberg/blob/master/test/integration/full-content/server-registered.json

Testing

block.json is used for deprecated Text Columns block. To ensure it still works you can paste the following HTML code in the code editor mode:

<!-- wp:text-columns -->
<div class="wp-block-text-columns alignundefined columns-2"><div class="wp-block-column"><p>Column 1</p></div><div class="wp-block-column"><p>Column 2</p></div></div>
<!-- /wp:text-columns -->

Future

Moving forward we can add a few improvements by creating our own plugin:

• make it aware of translations
• structure generate code in a way which allows dead code elimination for Gutenberg and WordPress
• to do the previous, only ship metadata on the client if server-side registration isn’t used

In Gutenberg, we want to register all blocks on the server with PHP and don’t repeat it on the client. For the mobile app, there is a need to have those definitions to be loaded in JS as of today and maybe through network-based API later. Standalone Gutenberg which Riad works on would benefit from purely client-side block definitions loading. I'm fully aware, it’s all confusing 😅 With the proposed trick we can continue shipping npm modules which work out of the box even when blocks are partially defined in the JSON file. At the same time, we can introduce an env variable like process.env.BLOCK_METADATA_SOURCE (similar to process.env.GUTENBERG_PHASE) and don’t include code behind this flag in JS bundle when its value is set to server through some webpack magic. To better illustrate this idea, I hand-crafted example output which custom Babel plugin could generate:

// input
import metadata from './block.json';

{
	"name": "core/text-columns",
	"icon": "columns",
	"category": "layout",
	"attributes": {
		"content": {
			"type": "array",
			"source": "query",
			"selector": "p",
			"query": {
				"children": {
					"type": "string",
					"source": "html"
				}
			},
			"default": [ { }, { } ]
		},
		"columns": {
			"type": "number",
			"default": 2
		},
		"width": {
			"type": "string"
		}
	},
	"supports": {
		"inserter": false
	}
}

// output
var metadata = {
  name: "core/text-columns",
};
if (process.env.BLOCK_METADATA_SOURCE !== 'server') {
	metadata = Object.assign({}, {
		icon: "columns",
		category: "layout",
		attributes: {
			content: {
				type: "array",
				source: "query",
				selector: "p",
				query: {
					children: {
						type: "string",
						source: "html"
					}
				},
				"default": [{}, {}]
			},
			columns: {
				type: "number",
				"default": 2
			},
			width: {
				type: "string"
			}
		},
		supports: {
			inserter: false
		}
	}, metadata);
}
exports.metadata = metadata;

[gziolo]

It gets transpiled into:

[youknowriad]

Is this something that can be useful for our default preset?

[gziolo]

Yes, I think it would land there eventually once proved that we like this approach 👍

By the way, this plugin still depends on Babel 6. Not sure it's an issue, just noting.

[youknowriad]

I feel like I have a lot to say about these :), can we start with something very small like the "category"?

[youknowriad]

I know this PR is more about the way javascript consumes this file rather than about the content of the file directly and that's why I suggest to keep it to the minimum for the moment.

[gziolo]

I don't care what we include, I opted for everything that server is able to expose to the client:

https://github.com/WordPress/wordpress-develop/blob/689ba4eec66b8d735c57940303ea0d49657f4cb3/src/wp-admin/includes/post.php#L2206

[youknowriad]

Agreed it's not very important for now (to filter or not) but the shape of some of these properties might need to be changed later.

[gziolo]

supports removed, should I remove anything else?

[youknowriad]

I guess the idea later is to have a feature flag to set metadata to undefined in Core and Gutenberg when these are provided by the server right?

[gziolo]

Yes, exactly. That would be optimal to use dead-code elimination. I'm not quite sure how it would work with deprecations which depend on attributes though :)

Gallery block is a great example:
https://github.com/WordPress/gutenberg/blob/master/packages/block-library/src/gallery/index.js#L21-L71
https://github.com/WordPress/gutenberg/blob/master/packages/block-library/src/gallery/index.js#L229
https://github.com/WordPress/gutenberg/blob/master/packages/block-library/src/gallery/index.js#L289
https://github.com/WordPress/gutenberg/blob/master/packages/block-library/src/gallery/index.js#L324-L334

[aduth]

In Gutenberg, we want to register all blocks on the server with PHP and don’t repeat it on the client. For the mobile app, there is a need to have those definitions to be loaded in JS as of today and maybe through network-based API later. Standalone Gutenberg which Riad works on would benefit from purely client-side block definitions loading.

So, the contents of block.json would be bundled with the JavaScript. But in the context of WordPress, the server would register the block using the same block.json, largely to the effect that the registration can become "known" PHP-side. And we still bootstrap this into the client, which takes precedent over the bundled metadata (though likely it is the same).

Am I understanding correctly?

[gziolo]

So, the contents of block.json would be bundled with the JavaScript. But in the context of WordPress, the server would register the block using the same block.json, largely to the effect that the registration can become "known" PHP-side. And we still bootstrap this into the client, which takes precedent over the bundled metadata (though likely it is the same).

In overall, this is exactly what I'm envisioning with one small remark. Once this data is available on the server and is discoverable through API, we should always serve it from the server. Therefore the need to have it bundled into the client no longer makes sense. To avoid code duplication and make the bundle size smaller, we can remove this metadata from the client in a WordPress context.

[aduth]

Therefore the need to have it bundled into the client no longer makes sense. To avoid code duplication and make the bundle size smaller, we can remove this metadata from the client in a WordPress context.

Gotcha. That would be good; the one thing which stood out to me is that we'd have the data be duplicated. I suppose the reason for doing it this way is to allow for a more iterative conversion to a system where blocks become known by fetching data from an API, rather than explicitly registering as we do today in edit-post? It's still unclear (and perhaps yet-unsolved, I'm unsure) how the JavaScript implementation of the block would become known to the editor in this system. Or am I mistaken in thinking on this process of how blocks become known to the editor? It seems like it could be the necessary path for dynamically loading new blocks via e.g. some of the inserter concepts for installing new blocks.

[gziolo]

I suppose the reason for doing it this way is to allow for a more iterative conversion to a system where blocks become known by fetching data from an API, rather than explicitly registering as we do today in edit-post?

Yes, that is exactly what would happen at some point. Server and client would have the same list of all blocks expressed with metadata format. That would make it possible to automatically register blocks exposed by the server rather than doing it explicitly as of today. However, it's still unclear to me how we would connect metadata with block settings which can be expressed only with JavaScript. On the other hand, things like edit implementation aren't necessary until the block is inserted into the editor so it doesn't have to be registered immediately. This makes it more complicated but at the same time creates very interesting opportunities to load have logic only when it's actually needed.

It's still unclear (and perhaps yet-unsolved, I'm unsure) how the JavaScript implementation of the block would become known to the editor in this system. Or am I mistaken in thinking on this process of how blocks become known to the editor? It seems like it could be the necessary path for dynamically loading new blocks via e.g. some of the inserter concepts for installing new blocks.

This is one of the possible flows we could implement:

• from the Inserter you call API which returns the list of block metadatas from WordPress.org Block Directory
• based on that you somehow display them in the inserter (name, title, description, category, keywords is all you need probably)
• when you select the block, behind the scenes it gets installed on the server, the same way as if you would go to the Plugins page
• placeholder block is inserted into the editor
• once it’s installed, JS and CSS gets loaded into the editor, placeholder blocks gets transformed into this newly installed block

[gziolo]

@youknowriad @aduth - do you feel like we can land this PR as is and continue with the following roadmap:

• use inline JSON babel plugin as the first step with one block (this PR)
• update more client-side blocks to work with this plugin
• replace inline JSON Babel plugin with our own version which supports i18n
• add more fields to block.json
• update Babel plugin to bundle block.json metadata only when not using server-side registration
• figure out the way to register all static blocks on the server with some custom Gutenberg specific loader based on the block.json

Some tasks can be done in parallel which might make the development process more convenient.

[ghost]

very very thanks and good luck

[gziolo]

Can we merge it as is and work on follow-up PRs?