Add collections to build #94
Conversation
| return { | ||
| statusCode: 404, | ||
| error: new Error('Not Found'), | ||
| collectionInfo: additionalURLs.size ? { additionalURLs } : undefined, |
There was a problem hiding this comment.
This is a little bit of a hack to get around a chicken-or-egg problem.
In our blog example, we find a pages/$tag.astro collection. Great, so that generates /tag/index.html? Not so fast! It generates /tag/movie/index.html and /tag/television/index.html! But we don’t know what it generates before we load it. But we have to load it using its generated URLs. Ack!
In this scenario, we load /tag, which will 404. But because it’s a collection, it will return 404 but with the loaded data we need to complete the build step. We only got that 404 response because that collection loaded up its data, looked through everything, and at the end determined our request /tag didn’t match the expected format /tag/:tag. But since it had to do work to deliver that 404, it returned what it figured out up till that point so we know how to proceed.
Anyway, the hacky bit is returning collectionInfo from our runtime.load() response regardless of statusCode. But this was the least-disruptive way I could think to do this, without making a runtime.loadCollection() or some API that lives completely outside runtime.load()
|
|
||
| /** Recursively build collection URLs */ | ||
| async function loadCollection(url: string): Promise<LoadResult | undefined> { | ||
| if (builtURLs.has(url)) return; // this stops us from recursively building the same pages over and over |
There was a problem hiding this comment.
This is the thing that keeps the hack from being too bad: in our process of collecting URLs, we keep track of ones we’ve built already and only build new ones.
Let’s look at the worst offender here:
- Pass 1: load
pages/$tag.astro, get back['/tag/movie/', '/tag/television']. These are both new, so we start up a 2nd pass. - Pass 2:
/tag/movie->['/tag/movie/1', '/tag/movie/2']//tag/television->['/tag/television/1']. These are new as well, so we move to a 3rd pass. - Pass 3:
/tag/movie/1->['/tag/movie/1, '/tag/movie/2']//tag/movie/2->['/tag/movie/1, '/tag/movie/2']//tag/television/1->['/tag/television/1']. All these are repeats, so we’re done.
That last step is where we could be rendering the same pages over-and-over again. The builtURLs only crawls new URLs.
Any collection can be fully built in 2 or at most 3 passes (3 passes if the first request was a “miss” as it was with /tag missing parameters). While this can probably be improved, 3 passes is still far more efficient than p * n passes (p = all possible parameter values × n = number of pages per parameter values).
1ee21cf
to
088a4bd
Compare
| const pages = await allPages(pageRoot); | ||
|
|
||
| try { | ||
| await Promise.all( |
There was a problem hiding this comment.
With the addition of collections, pages now build in parallel rather than serially. This could have a big impact on build times.
|
lgtm |
Changes
Adds collections to the build, so they expand to multiple
.htmlfiles.Please refer to the comments for thoughts.
Testing
Docs