Pitfalls and surprises in data fetching

[jcreighton]

I'd love to hear from the Relay / React Data team on some of the pitfalls / surprises / lessons learned from working with Concurrent APIs. Outside of being a maintainer of a data fetching / data caching library, as a frontend engineer, I think a first concern would be what React 18+ means for data fetching.

[devknoll]

Additionally, I think one historically canonical example of using Suspense was to fetch data seemingly granularly on the client (by colocating a fetch for data inside the component that wants it), but it's easy to see that without also using something like GraphQL or the upcoming React Server Components that can batch requests for an entire tree, it's easy to end up in a situation with multiple request waterfalls and potentially subpar performance.

I think it would be useful to clarify if this is still a suggested/recommended pattern, and/or what people should do instead.

[josephsavona]

We're very happy to share lessons learned and thoughts on our experiences using Suspense for data-fetching - but I think that answer could go on for a while. Is there a specific area that you're most interested in? Are you more interested in what React 18 means as a product engineer or more about the lessons we've learned that are relevant to library authors?

[jcreighton]

I'm thinking about it more at a library author. Of course, we at Apollo Client are wondering what we can learn from your team. We also know we'll get questions (and currently do!) from users about data fetching in Strict Mode. We want to make our useQuery and useMutation safe for StrictMode, for example, but we'll also want to provide Best Practices for our users. Like in the case of being at to set a fetchPolicy that calls the network-only, I assume in StrictMode there's the consideration that that call could hit the network several times - we want to be prepared to help users navigate those potential pitfalls.

[jcreighton]

I know it's a large question so maybe we could list out some learnings and then branch new discussions off of those?

[josephsavona]

Thanks for clarifying - see responses below!

[josephsavona]

There are a few main concerns that come to mind in terms of data-management and concurrent rendering:

• Data changes during rendering
• Lazy-fetching and concurrent rendering
• Lazy-fetching and offscreen
• Prefetching and concurrent rendering

I'm going to create separate responses for these (will add one at a time as time permits).

[josephsavona]

Data Changes During Rendering

This scenario primarily applies to libraries like Relay or Apollo that maintain a "store" of data that is accessed by — and kept consistent across — multiple components or roots. In synchronous rendering it was generally not possible for data in the store to change during the course of a single render: render functions should be "pure", so if rendering occurs synchronously no other code can have run to modify the store. Note that libraries are generally expected to maintain this invariant too - APIs meant to be called during render should also be pure-ish: for example Relay's useFragment() reads data from the store but does not modify the store.

However, concurrent rendering changes this: concurrent rendering does not complete synchronously so it's possible for other code to run in between rendering of components. This can cause "tearing": when different parts of the screen show inconsistent data. An example of how this can occur:

1. The store says the user's name is "Joe"
2. A screen starts rendering, and the root component reads the name as "Joe"
3. In between units of rendering, something modifies the store and sets the user's name is "Joseph"
4. Rendering resumes and renders a child component which reads the name as "Joseph"
5. React finally commits the rendered UI atomically, showing the inconsistent names to the user (Joe/Joseph).

The issue is step 3, which modifies the store concurrently with rendering in a way that couldn't happen before, when rendering was synchronous. Again, note that this can only happen when concurrent rendering is enabled via new features.

There are a few main solutions:

1. Prevent the store from being modified during render (delay publishes while rendering is ongoing)
2. Prevent modifying the version of the store that the render is reading from (ie vending some sort of immutable snapshot for the render, while the main store can advance with new data)
3. Allow modifications to occur but detect them after the fact and fix them.

Relay's Approach

Relay currently uses strategy #3: we maintain a "version" counter in the store that increments every time data is published, and we remember the version of the store that each component read from during render. On commit (useEffect) we check if the store version is the same as during render. If not we re-read the data to see if the data the component used has changed - if so we re trigger a re-render. The useMutableSource hook works similarly and was developed after we created our implementation. One tradeoff with useMutableSource is that React understands the semantics and can fall back to a synchronous re-render if data changes. The upside is that this avoids tearing, but the downside is that this comes at the cost of delaying when the screen is updated.

We also have a good idea of what it would look like to support #2 in the data layer, but it would require some additional hooks in React. Basically, the data layer would need to understand how long a snapshot should be kept around, know when to reconcile different branches of the store together, etc. This is something we expect to continue exploring going forward.

Recommended Pattern

We recommend that existing data libraries start by adopting useMutableSource in order to avoid tearing during concurrent rendering.

[Andarist]

Thank you for the detailed, in-depth, comment.

You present 3 possible approaches and that the one you have chosen has been later implemented as useMutableSource (from what i understand those are not 100% identical but close enough). Do you believe there is even a place for the other two solutions to be implemented by anyone? All lead to the same result - just subtly differently, so I imagine that the answer is: no, useMutableSource is enough.

[josephsavona]

Do you believe there is even a place for the other two solutions to be implemented by anyone? All lead to the same result - just subtly differently, so I imagine that the answer is: no, useMutableSource is enough.

We think there's potentially room for option 2, where a data layer can provide an immutable snapshot for the span of a render, while still allowing concurrent modifications to the store. This requires more research and our sense is that we'd need additional hooks from React to make this viable to be implemented, or implemented efficiently enough. Specifically, there is no hook for when a render pass is cancelled, so there isn't a good way to know the lifetime of a snapshot that the store vends out.

So while folks in the community are welcome to try this approach as an experiment, we recommend that data libraries looking for a reliable solution to achieve concurrent compatibility today use useMutableSource().

[markerikson]

This actually brings up an interesting point for potential Redux usage.

In React-Redux v6, we switched from individual store subscriptions per component, to passing down the entire state value via new context from the root of the tree. One of the goals was to ensure a consistent state value across the tree to avoid any potential tearing. (This actually turned out to be awful for performance reasons, but that's a separate topic.)

However, it turned out that we had users who were relying on being able to see the state updating in the middle of a render pass, because they were dynamically adding slice reducers in constructors or componentWillMount:

reduxjs/react-redux#1150

The store was updated synchronously before any mapState functions ran, and the newly mounting components expected to be able to read from store.someNewSlice immediately. In v6, we were still operating off the "previous" state before the updated reducer addition, and so dynamically-loaded components began breaking since that data was no longer visible right away. So, we unintentionally broke that use case.

I'm not sure there's exactly a good solution here, but this discussion sounds very relevant.

[josephsavona]

Lazy-Fetching and Concurrent Rendering

First, note that fetching data during render isn't ideal from a performance perspective. As we discussed in our recent blog post, lazy fetching delays the point at which an application can start fetching data, which then blocks further rendering. See the post for more details about alternative approaches that we're using internally.

However, in some situations it's just easier to lazily fetch data during rendering. As a library developer trying to support this pattern in React 18, though, we have to handle the possibility that a component may render multiple times before it commits - or it may not commit! So libraries should ensure that each render doesn't trigger a duplicate fetch, while also cancelling fetches that are no longer needed. Libraries should not require developers to handle these cases. More on how to achieve this below.

As an example, consider an application with two components: a parent that fetches data (the user's profile photo URL), and then a child that shows the profile photo. The parent suspends while fetching the data, the child suspends while the photo is loading. With concurrent rendering, this would occur as follows:

1. Parent renders, calls useProfilePhoto() or similar. The data is not available so the hook fetches it and suspends.
2. React renders the nearest Suspense fallback
3. The fetch completes, so React tries re-rendering
4. Parent renders, calls useProfilePhoto(). The data is available so it returns and the component renders.
5. Child renders, tries to access the photo but it isn't loaded yet, so it suspends.
6. React continues rendering the nearest Suspense fallback.
7. The image loads, so React tries re-rendering
8. The parent and child both render w/o suspending.
9. React commits the final view.

Note that the parent component renders 3 times in this scenario, the child renders 2 times, and each commits once. But it's also possible that the user navigated to different content before this sequence completed, in which case the parent/child may have rendered some number of times without ever committing!

Relay's Approach

In Relay we handle these cases with a custom cache implementation that de-dupes fetches over a short period. The useLazyLoadQuery() hook creates a cache key for the query/variables/fetchPolicy combination and then checks if a cache entry exists. If it does the hook either returns, throws, or suspends based on the state of the cache entry. If there's no entry yet it checks whether the query can be fulfilled and populates a new entry - either returning from cache, resolving from cache while fetching in the background, or fetching and suspending. Notably, this cache entry will be marked as temporary until some component that needs it commits - and unused temporary entries are purged from the cache after a short-ish interval (30s iirc) so that future renders don't accidentally use stale data.

Recommended Pattern

Our experience using Suspense for data-fetching in Relay and elsewhere informed the creation of the new <Cache> API, to make it easier to handle these cases in data-fetching libraries. The challenge for libraries is that they don't know exactly how long to cache each fetch - we don't want to fetch each time a component re-renders, but we also don't want to cache an entry so long that an unrelated fetch later in a session uses stale data. But React knows the appropriate lifetime for such cache entries, and <Cache> effectively implements the complex logic I described from Relay's implementation with more precise control over the cache expiration.

In terms of a recommendation for library authors, <Cache> is relatively new and we don't have a lot of experience using it in production. We anticipate trying to use <Cache> with Relay soon but this is definitely less stable than useMutableSource. So at the moment we recommend experimenting with <Cache> but be prepared to potentially need to make adjustments as the API changes during the RC. Also note that <Cache> isn't meant for storing things like individual entries in a normalized database (or individual records in a GraphQL response) - it's really about de-duplicating coarse-grained side-effects during render such as HTTP requests. See #25 for more details about the Cache API.