
<p>When building complex applications today, Apollo Client users typically reach for <code>useQuery</code> to get data to their components. This is no surprise as this is considered a <a href="https://www.apollographql.com/docs/react/data/operation-best-practices#query-only-the-data-you-need-where-you-need-it">best practice</a> by the Apollo Client documentation. However, this can cause challenges while trying to minimize over-fetching while maintaining a clear separation of concern.</p>



<p>During my time at <a href="https://redshelf.com/" data-type="link" data-id="https://redshelf.com/" target="_blank" rel="noreferrer noopener">RedShelf</a>, a leader in the Higher EdTech space, I’ve been an advocate for Apollo’s best practices including “query only the data you need, where you need it.” However, this led to over-fetching because we naturally pushed our queries (a.k.a. network requests) as far down the component tree as possible, in most cases usually to the individual components.</p>



<p>Enter the power of the <a href="https://www.apollographql.com/docs/react/data/fragments/#usefragment">useFragment</a> hook and <a href="https://www.apollographql.com/docs/react/data/fragments#colocating-fragments">colocated fragments</a>. Combining these two techniques have enabled us to separate accountability from responsibility, remaining aligned with Apollo’s best practice while optimizing our client application’s fetching strategies along the way.</p>



<h2 class="wp-block-heading"><a href="#exploring-colocated-fragments" aria-hidden="true" class="aal_anchor" id="exploring-colocated-fragments"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Exploring Colocated Fragments</h2>



<p>Colocated fragments are not a new concept and, in fact, they <a href="https://relay.dev/docs/tutorial/fragments-1/">are one of the distinguishing features of Relay</a>. Essentially, colocated fragments move the responsibility of declaring data needs from parents to children further decoupling the two. Although a step in the right direction, they still require parent components to provide fragmented data through React props, or components that have defined their own data needs to manually access the Apollo Client cache.</p>



<p>However, with the introduction of the <code>useFragment</code> hook, we can now leverage the benefits of colocated fragments with the simplicity of accessing the fragmented data via a streamlined hook.</p>



<h2 class="wp-block-heading"><a href="#understanding-apollos-usefragment-hook" aria-hidden="true" class="aal_anchor" id="understanding-apollos-usefragment-hook"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Understanding Apollo&#8217;s <code>useFragment</code> Hook</h2>



<p>A clear separation between accountability and responsibility in data fetching is important. While components should be responsible for declaring the data they need, they shouldn&#8217;t necessarily be accountable for fetching it. This is where the <code>useFragment</code> hook shines.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="472" src="https://wp.apollographql.com/wp-content/uploads/2024/04/fragment-architecture1-1024x472.png" alt="A diagram showing a parent component and child components.  The child components each contain a useFragment call with fragments named ProductBrand, ProductImage, and ProductDetails.  The parent component contains a useQuery call with a query that spreads each of the aforementioned fragments.  The parent component is annotated with &quot;Accountable for fetching data&quot; and the child components are annotated with &quot;Responsible for declaring data needs&quot;" class="wp-image-23693" srcset="https://wp.apollographql.com/wp-content/uploads/2024/04/fragment-architecture1-1024x472.png 1024w, https://wp.apollographql.com/wp-content/uploads/2024/04/fragment-architecture1-300x138.png 300w, https://wp.apollographql.com/wp-content/uploads/2024/04/fragment-architecture1-768x354.png 768w, https://wp.apollographql.com/wp-content/uploads/2024/04/fragment-architecture1-1536x708.png 1536w, https://wp.apollographql.com/wp-content/uploads/2024/04/fragment-architecture1-2048x943.png 2048w, https://wp.apollographql.com/wp-content/uploads/2024/04/fragment-architecture1-1200x553.png 1200w, https://wp.apollographql.com/wp-content/uploads/2024/04/fragment-architecture1-1980x912.png 1980w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /></figure>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p>This article mainly focuses on mitigating over-fetching and improving code maintainability, but the <code>useFragment</code> hook provides other performance gains, too, like handling re-renders more efficiently. Read more about it in <a href="https://www.apollographql.com/blog/introducing-apollo-clients-nonreactive-directive-and-usefragment-hook">this Apollo blog post</a>.</p>
</blockquote>



<p>Introduced in <a href="https://github.com/apollographql/apollo-client/releases/tag/v3.7.0">Apollo Client 3.7</a> and stabilized in <a href="https://github.com/apollographql/apollo-client/releases/tag/v3.8.0">3.8</a>, the <code>useFragment</code> hook enables components to directly access fragment data from the normalized cache.</p>



<p>This not only helped our team stay true to Apollo’s best practice of querying only necessary data, but also helped solve some of the over-fetching we were doing by following the best practice in the first place.</p>



<h2 class="wp-block-heading"><a href="#integration-into-the-teams-applications" aria-hidden="true" class="aal_anchor" id="integration-into-the-teams-applications"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Integration into the Team&#8217;s Applications</h2>



<p>The first place we leveraged colocated fragments and the <code>useFragment</code> hook was to refactor a “Product Details” drawer within one of our applications where over-fetching had become a major problem. When the drawer mounted, it sent 9 network requests(!) &#8211; one for each component &#8211; because they were responsible for declaring AND accountable for fetching their data.</p>



<p>By associating colocated fragments and replacing <code>useQuery</code> with <code>useFragment</code> in every component, we were able to reduce the network requests count from 9 to 2 within the drawer. The only thing that kept us from only needing a single network request was some of the data required a network-only fetch policy.</p>



<p>Since then, we’ve propagated this pattern throughout our client applications, reducing over-fetching without compromising data requirements. This technique has been truly transformative for our team, improving the performance of our applications and enhancing the maintainability of our codebases.</p>



<h2 class="wp-block-heading"><a href="#lessons-learned" aria-hidden="true" class="aal_anchor" id="lessons-learned"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Lessons Learned</h2>



<p>While challenges are inevitable when adopting new patterns, we couldn’t have made the progress we did in the timeframe we did without the help and support from the <a href="https://events097286.typeform.com/joincommunity">Apollo Supergraph Champions community</a>.</p>



<p>For example, while replacing the <code>useQuery</code> hooks in each component with the <code>useFragment</code> hook, we weren’t receiving the fragmented data we were expecting. After trying a few approaches, we reached out to the Apollo community channel and had a great conversation with <a href="https://www.linkedin.com/in/adamluikart/">Adam Luikart</a>, a Principal Software Engineer at Indeed, who helped us realize we needed to pass the same exact variables we passed to <code>useQuery</code> to the <code>useFragment</code> hook or it won’t have a valid way to locate the data in the cache.</p>



<p>Thanks to that conversation, our team gained a deeper understanding of the inner workings of the Apollo Client cache and a working example that led to the eventual buy-in and integration of colocated fragments and the <code>useFragment</code> hook into our pattern library.</p>



<h2 class="wp-block-heading"><a href="#takeaways" aria-hidden="true" class="aal_anchor" id="takeaways"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Takeaways</h2>



<p>Apollo’s maxim of “query only the data you need, where you need it” is a great starting point.&nbsp; But certain component structures can result in more network calls than is needed. When we used the new <code>useFragment</code> hook in Apollo Client, though, we were able to stay true to the principle behind that best practice while reducing the number of fetches in our app.</p>



<p>By embracing this pattern, our team effectively separated the responsibility and accountability for data requirements.&nbsp; This helped us control the total demand on our network and enhance the end user experience without compromising our architecture.</p>


Optimizing Data Fetching with Apollo Client: Leveraging useFragment and Colocated Fragments


<p>Modern apps are chatty—they require a lot of data, and thus make a lot of requests to underlying services to fulfill those needs. Layering GraphQL over your services solves this issue, since it encapsulates multiple requests into a single operation, avoiding the cost of multiple round trips.</p>



<p>While GraphQL encourages this style of data fetching, React encourages developers to split up data fetching logic to be colocated with the components that use it. This pattern is extremely useful, but can lead to many of the same issues of an app making too many network requests for a server to handle at scale. The idea of batching client operations can be applied to GraphQL clients to solve many of these issues, but sometimes batching can hurt more than help.</p>



<p>To batch, or not to batch, that is the question.</p>



<p>By the end of this post, you should be able to answer the following questions about batching client operations with Apollo:</p>



<ul class="wp-block-list"><li>How does batching work?</li><li>What are the tradeoffs with batching?</li><li>Is batching necessary?</li><li>Can batching be done manually?</li><li>Can I fix automatic batching?</li></ul>



<h2 class="wp-block-heading" id="5853"><a href="#how-does-batching-work" aria-hidden="true" class="aal_anchor" id="how-does-batching-work"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>How does batching work?</h2>



<p>Batching is the process of taking a group of requests, combining them into one, and making a single request with the same data that all of the other queries would have made. This is usually done with a timing threshold. For example, with a threshold of 50ms, if a query is made from a component, instead of making the query immediately, the client waits 50ms. If any other queries are requested in that 50ms, all of those additional queries are requested at the same time, rather than separately.</p>



<p>In GraphQL apps, batching usually takes one of two forms. The first form takes all operations and combines them into a single operation using the alias feature in GraphQL. This approach is not recommended, however, since this removes the ease of tracking metrics on a per-operation basis and adds additional complexity to the client.</p>



<p>Instead, we recommend sending an array of operations to a GraphQL server, having the server recognize the request as an array of operations instead of a single one, and handle each operation separately. This method still requires only a single round-trip, while retaining the ability to track single operation performance. Apollo Client handles batching like this using&nbsp;<code>apollo-link-batch-http</code>. It’s also worth noting that this method requires the server to be able to receive an array of operations as well as single operations. Luckily,&nbsp;<code>apollo-server</code>&nbsp;supports this out of the box.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="546" src="https://apollographql.wpengine.com/wp-content/uploads/2020/03/1_IUrPIx21p74CvO2tTKabkA-1024x546.png" alt="" class="wp-image-678" srcset="https://wp.apollographql.com/wp-content/uploads/2020/03/1_IUrPIx21p74CvO2tTKabkA-1024x546.png 1024w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_IUrPIx21p74CvO2tTKabkA-300x160.png 300w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_IUrPIx21p74CvO2tTKabkA-768x410.png 768w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_IUrPIx21p74CvO2tTKabkA-1200x640.png 1200w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_IUrPIx21p74CvO2tTKabkA.png 1500w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /><figcaption>How Apollo batches operations with an array</figcaption></figure>



<p>For a deeper dive into how batching works with Apollo, check out&nbsp;<a target="_blank" rel="noreferrer noopener" href="https://blog.apollographql.com/query-batching-in-apollo-63acfd859862">this post</a>&nbsp;introducing batching in an earlier version of Apollo Client. Even though some of the implementation details have changed, the concepts are still relevant today.</p>



<h2 class="wp-block-heading" id="a63a"><a href="#what-are-the-tradeoffs-with-batching" aria-hidden="true" class="aal_anchor" id="what-are-the-tradeoffs-with-batching"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>What are the tradeoffs with batching?</h2>



<p>Batching may sound like the perfect solution to some network performance issues on the client, but it’s far from perfect. Request batching is prone to slow loading times on the client. For example, if five requests from separate components are made to the GraphQL endpoint, and one of the five requests take a long time, the client won’t get&nbsp;<em>any</em>&nbsp;of the results back until&nbsp;<em>all</em>&nbsp;operations have been resolved. In other words, batched operations are always as slow as the slowest operation in the batch.</p>



<p>Since GraphQL is often used to aggregate data from multiple data sources, this can be problematic. If one of the underlying services is slower than the rest, the effects may be felt across the whole client app.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="400" height="227" src="https://apollographql.wpengine.com/wp-content/uploads/2020/03/1__1VoQ49cbUEPHL7RPsl0og.gif" alt="" class="wp-image-677"/></figure>



<p>Additionally, batching makes it much harder to debug network traffic. In most scenarios, if a specific operation was taking a long time to resolve, it would be obvious. The component making the query would be stuck in a loading state, and the rest of the app would function normally. Additionally, when using browser debugging tools, slow network requests can usually be found easily. Batching queries makes both of these debugging methods more difficult.</p>



<h2 class="wp-block-heading" id="f94c"><a href="#is-batching-necessary" aria-hidden="true" class="aal_anchor" id="is-batching-necessary"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Is batching necessary?</h2>



<p>Before looking at solutions to the issues that batching introduces, it’s important to question whether batching is even needed in the first place. Since enabling batching is as easy as replacing one&nbsp;<code>apollo-link</code>&nbsp;with another, it&#8217;s tempting to introduce batching at the onset of a project, without any real evidence that batching is needed. In fact, because of its drawbacks, we don’t recommend batching unless performance issues are still present after all of the following steps have been taken:</p>



<ul class="wp-block-list"><li><a href="https://www.apollographql.com/docs/guides/performance.html#automatic-persisted-queries" target="_blank" rel="noreferrer noopener">Use automatic persisted queries</a>&nbsp;(APQ) to reduce the request body size.</li><li><a href="https://www.apollographql.com/docs/guides/performance.html#get" target="_blank" rel="noreferrer noopener">Cache requests with a CDN</a>&nbsp;to prevent operations from ever reaching the GraphQL endpoint.</li><li>Enable partial query caching with&nbsp;<a href="https://www.apollographql.com/docs/apollo-server/features/data-sources.html" target="_blank" rel="noreferrer noopener">Apollo Server</a>&nbsp;to make operation resolution significantly quicker in most cases.</li><li>If possible, use HTTP/2 on the server (with Node.js 10), which allows multiplexing of requests, effectively batching them.</li></ul>



<h2 class="wp-block-heading" id="cff2"><a href="#can-batching-be-done-manually" aria-hidden="true" class="aal_anchor" id="can-batching-be-done-manually"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Can batching be done manually?</h2>



<p>Yes! Batching can be done manually. Often, for the cases where batching may still be a necessity, inexpensive operations can be manually batched together to prevent unnecessary requests. In GraphQL, this is done by combining smaller queries into one larger one. For example, if there was a page with four content blocks on it, rather than having each block fetch its own data, a&nbsp;<code>container</code>&nbsp;could fetch the data and pass it to the components manually. This is conceptually similar to the first implementation of batching described in the first section.</p>



<p>This may sound counterintuitive to the patterns that have been established, like colocating queries with the components that use their response, but there are ways around this.</p>



<p>This isn’t suggesting to write one large GraphQL query at the container-level. Instead, write queries normally, next to the components that use them. When you’re ready to optimize a section of an app, convert those queries to&nbsp;<a href="https://www.apollographql.com/docs/react/advanced/fragments.html" target="_blank" rel="noreferrer noopener">fragments</a>&nbsp;and export them from the component file. You can then import these fragments in the container, let the container make the single, large query, and pass the fragment results back to the children. Using container components in this way can even allow you to control loading and error states at the container-level, rather than in each component.</p>



<p>Take a look at this CodeSandbox for a full example of this in action:</p>



<figure class="wp-block-embed-wordpress wp-block-embed is-type-wp-embed is-provider-codesandbox"><div class="wp-block-embed__wrapper">
<iframe loading="lazy" class="wp-embedded-content" sandbox="allow-scripts" security="restricted" title="Manual Query Batching" width="580" height="870" src="https://codesandbox.io/embed/wnyxmy8kv7#?secret=5eT4FyMOe1" data-secret="5eT4FyMOe1"></iframe>
</div></figure>



<p>Even manual batching has issues, though. Since manually batched operations are much larger,&nbsp;<strong>their ability to take advantage of whole-query caching is reduced</strong>. Whole-query cache TTLs are based on the field in an operation with the shortest TTL. Increasing the number of fields in an operation increases the chances that a field that can’t be cached for a long time is included, reducing the ability to cache the whole operation. For more on whole-query caching, and how these TTLs are calculated,&nbsp;<a href="https://www.apollographql.com/docs/engine/proxy/guides.html#cache-hints" target="_blank" rel="noreferrer noopener">read this doc</a>.</p>



<h2 class="wp-block-heading" id="dae7"><a href="#can-i-fix-automatic-batching" aria-hidden="true" class="aal_anchor" id="can-i-fix-automatic-batching"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Can I fix automatic batching?</h2>



<p>There is no silver bullet for batching. If batching is enabled, there is always the potential for portions of the batch to run slower, and thus hold up the remaining portions of the batch. Sometimes, however, the trouble of manually batching operations outweighs the benefits. Manually batching may be too complicated, or too large of a refactor to reasonably undertake.</p>



<p>Some of the issues around batching can be solved by manually&nbsp;<em>debatching</em>&nbsp;expensive operations–that is, allowing most operations to be batched like normal, but preventing batching for ones that are known to cause issues. Doing this requires a few steps:</p>



<ol class="wp-block-list"><li>Build components as usual, with colocated queries.</li><li>Identify the most expensive operations using a tool like&nbsp;<a href="https://www.apollographql.com/engine" target="_blank" rel="noreferrer noopener">Apollo Engine</a>&nbsp;(these are not necessarily just the largest queries).</li><li>Mark expensive operations on the client using an operation’s&nbsp;<code>context</code>. You can set the context by specifying a prop on your&nbsp;<code>Query</code>&nbsp;component.</li><li>Use&nbsp;<code>&lt;a href=&quot;https://www.apollographql.com/docs/link/composition.html#directional&quot; target=&quot;_blank&quot; rel=&quot;noreferrer noopener&quot;&gt;split&lt;/a&gt;</code>&nbsp;to switch between&nbsp;<code>apollo-link-http</code>&nbsp;or&nbsp;<code>apollo-link-batch-http</code>&nbsp;depending on the context of the operation.</li></ol>



<p>Take a look at this CodeSandbox for a full example of this in action:</p>



<figure class="wp-block-embed-wordpress wp-block-embed is-type-wp-embed is-provider-codesandbox"><div class="wp-block-embed__wrapper">
<iframe loading="lazy" class="wp-embedded-content" sandbox="allow-scripts" security="restricted" title="Link switching for batches (blog)" width="580" height="870" src="https://codesandbox.io/embed/14xk74wk0q#?secret=S2SdbHI0bf" data-secret="S2SdbHI0bf"></iframe>
</div></figure>



<h2 class="wp-block-heading" id="a1d1"><a href="#conclusion" aria-hidden="true" class="aal_anchor" id="conclusion"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Conclusion</h2>



<p>Batching is a tricky topic. There are plenty of reasons to use some form of client request batching, but many times these solutions just cause more problems than they solve.</p>



<p>Hopefully, armed with this information, you can feel confident when making a decision about how to boost performance on the client.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="480" height="359" src="https://apollographql.wpengine.com/wp-content/uploads/2020/03/1_z1jNMuLYJ-eOi_TGIjkjNw.gif" alt="" class="wp-image-676"/></figure>



<hr class="wp-block-separator"/>



<p>I hope you’ll also&nbsp;<a href="https://summit.graphql.com/" target="_blank" rel="noreferrer noopener">join us at the 3rd annual GraphQL Summit on Nov 7–8 in San Francisco</a>. With over 800 attendees expected, it’s the largest GraphQL developer event in the world. Tickets are selling fast, so&nbsp;<a href="https://www.eventbrite.com/e/graphql-summit-2018-tickets-46601841362" target="_blank" rel="noreferrer noopener">register today</a>&nbsp;to reserve your spot!</p>


Batching Client GraphQL Queries


<p>You don’t need anything fancy to load some data from a GraphQL server. All you need for a basic client is a POST request that sends a query down to the server and gets a result back. However, GraphQL queries include a lot of useful information that can be used to make your application faster and more efficient. This is why we have smart client implementations, such as Apollo Client and Relay, that take advantage of GraphQL’s query structure to do useful things.</p>



<p>Smart GraphQL clients can make interaction with the server more efficient by reducing the number of roundtrips to fetch data. A common way to reduce server roundtrips is through caching, because the fastest way to load something is to already have it.&nbsp;But it turns out that caching isn’t the only thing that can make server communication more efficient.</p>



<h3 class="wp-block-heading" id="5bdb"><a href="#what-is-query-batching" aria-hidden="true" class="aal_anchor" id="what-is-query-batching"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>What is query batching?</h3>



<p>GraphQL query batching means sending multiple queries to the server in one request, which can have some pretty significant benefits:</p>



<ol class="wp-block-list"><li>Reducing the total number of server roundtrips, which can incur significant overhead in HTTP 1.x.</li><li>When queries are sent in one request, you can use&nbsp;<a href="https://github.com/facebook/dataloader" target="_blank" rel="noreferrer noopener">DataLoader</a>&nbsp;to batch loading the actual data from your backend API or database.</li></ol>



<p>When you load your UI on the client, it might fire several queries in a short period of time to put together its initial state. One simple strategy to improve this without changing any of the UI code is to batch together requests made within a small time interval. This way, all of the data can be loaded in one roundtrip, without any extra effort.</p>



<p>While&nbsp;<a href="https://www.youtube.com/watch?v=xwz1aXkJLLA" target="_blank" rel="noreferrer noopener">updating the UI for Meteor Galaxy to use Apollo Client</a>, we realized how desirable it is to have query batching in a production app. So, we implemented automatic query batching in Apollo Client.</p>



<p>In this post, we’ll first take a look at how we can use batching and then we’ll open up the box and see how it is implemented. We’ll also consider some next steps we can take to make batching even more awesome in GraphQL.</p>



<h2 class="wp-block-heading" id="7db1"><a href="#using-batching" aria-hidden="true" class="aal_anchor" id="using-batching"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Using batching</h2>



<p>We wanted ensure that an application developer using Apollo Client wouldn’t have to do anything extra to get batching to happen; you should get it “for free”. For example, say you have two UI components that each fire off a couple of GraphQL queries with Apollo Client:</p>



<pre class="wp-block-preformatted">// these queries may be loaded from two different places within your<br>// code.client.query({ query: query1 })<br>client.query({ query: query2 })</pre>



<p>Batching is turned off in Apollo Client by default. To turn it on, all you have to do is set the&nbsp;<em>shouldBatch</em>&nbsp;option in the constructor:</p>



<pre class="wp-block-preformatted">const client = new ApolloClient({<br> // ... other options ...<br> shouldBatch: true,<br>});</pre>



<p>And that’s it! Now the two queries above will be sent in one request.</p>



<h3 class="wp-block-heading" id="edf9"><a href="#batching-using-time-intervals" aria-hidden="true" class="aal_anchor" id="batching-using-time-intervals"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Batching using time intervals</h3>



<p>The query batcher operates on “ticks” — it checks a queue every 10 milliseconds to see if there are any pending queries. If there are multiple queries in the queue, they are combined into one server request. So, as long as the two queries above are fired within the same 10-millisecond tick of the batcher, they’ll be sent as one request, like so:</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="795" height="374" src="https://apollographql.wpengine.com/wp-content/uploads/2020/03/1_STFomheClBDuyNolDUIWWw.png" alt="" class="wp-image-272" srcset="https://wp.apollographql.com/wp-content/uploads/2020/03/1_STFomheClBDuyNolDUIWWw.png 795w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_STFomheClBDuyNolDUIWWw-300x141.png 300w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_STFomheClBDuyNolDUIWWw-768x361.png 768w" sizes="auto, (max-width: 795px) 100vw, 795px" /><figcaption>Query batcher queue</figcaption></figure>



<p>Notice that your application code can remain completely oblivious to this batching. Other than providing an additional option to the&nbsp;<em>ApolloClient&nbsp;</em>constructor, you don’t have to change anything else about your application to start getting the benefits of batching.&nbsp;<a href="http://docs.apollostack.com/apollo-client/network.html#query-batching" target="_blank" rel="noreferrer noopener">Check out the docs</a>&nbsp;for a slightly closer look at the semantics of batching and how you can use it with custom network interface implementations.</p>



<h2 class="wp-block-heading" id="baff"><a href="#a-look-inside" aria-hidden="true" class="aal_anchor" id="a-look-inside"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>A look inside</h2>



<p>Let’s take a look at how this works. Here’s how the pieces fit together at a high level:</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="689" height="513" src="https://apollographql.wpengine.com/wp-content/uploads/2020/03/1_yDls-FaZwIAw-tFJ2-xboA.png" alt="" class="wp-image-271" srcset="https://wp.apollographql.com/wp-content/uploads/2020/03/1_yDls-FaZwIAw-tFJ2-xboA.png 689w, https://wp.apollographql.com/wp-content/uploads/2020/03/1_yDls-FaZwIAw-tFJ2-xboA-300x223.png 300w" sizes="auto, (max-width: 689px) 100vw, 689px" /><figcaption>Batching architecture</figcaption></figure>



<p>Let’s consider a case where we fetch two queries, one after another. Both of the queries will go through the&nbsp;<em>QueryManager</em>&nbsp;and will be placed into the&nbsp;<em>QueryBatcher</em>’s queue. Then, during the next tick of the&nbsp;<em>QueryBatcher</em>, these queries will be placed into an array and passed along to an instance of the&nbsp;<em>BatchedNetworkInterface</em>. Once the results for both of the queries have returned from the server, the&nbsp;<em>QueryBatcher&nbsp;</em>will resolve the promises issued for both of the queries.</p>



<p>The important bit here isn’t actually the&nbsp;<em>QueryBatcher</em>: that just acts as a consumer on a queue. Rather, it is the implementation of a&nbsp;<em>BatchedNetworkInterface</em>&nbsp;that batches together multiple requests in a way that works with any spec-compliant GraphQL server.</p>



<h2 class="wp-block-heading" id="2886"><a href="#batching-with-no-special-server-support" aria-hidden="true" class="aal_anchor" id="batching-with-no-special-server-support"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Batching with no special server support</h2>



<p>All GraphQL servers expose a way for us to submit a single query and receive a single result in return. However, there is little support to submit multiple queries and receive results for each of them in a single roundtrip. So, we need to utilize a method that can turn multiple queries into a single query, submit it to the server, receive a single result and then unpack it into results for each of the queries initially submitted. This is all stuff that the default network interface in Apollo Client does for you.</p>



<p>It does this by using a pretty simple concept called&nbsp;<em>query merging</em>. It involves taking multiple queries and putting them all under a single root query. As a simple example, say we’re telling the&nbsp;<em>BatchedNetworkInterface</em>&nbsp;to batch these two queries:</p>



<pre class="wp-block-preformatted">query firstQuery {<br>   author {<br>    firstName<br>    lastName<br>  }<br>}query secondQuery {<br>  fortuneCookie<br>}</pre>



<p>Then, we can imagine a simple approach to merging that produces the following query:</p>



<pre class="wp-block-preformatted">query ___composed {<br>  author {<br>    firstName<br>    lastName<br>  }  fortuneCookie<br>}</pre>



<p>However, the devil is in the details. We’ve picked nice, non-conflicting names in our two example queries, but we have no guarantee of that being the case when we are attempting to merge arbitrary queries. So, we turn to a nice feature of GraphQL: aliasing. Basically, this allows us to refer to a field with a different name and the server will use this name when returning the result. For example, given the following query:</p>



<pre class="wp-block-preformatted"><br>query ___composed {<br>  aliasName: author {<br>    firstName<br>    lastName<br>  }<br>}</pre>



<p>The server will return a result that looks like the following:</p>



<pre class="wp-block-preformatted">{<br>  “aliasName”: {<br>    “firstName”: “John”,<br>    “lastName”: “Smith”<br>  }<br>}</pre>



<p>Using this feature of GraphQL, we can step through the AST of a query and rename all of the top-level fields, inline fragments and named fragments in a way that makes sure that they will never conflict once merged. We also rename variables since two queries can definitely refer to a variable that takes on different values in each query (e.g. a variable like&nbsp;<em>id)</em>. When we apply this query merging to the queries we just mentioned, we get the following composed query:</p>



<pre class="wp-block-preformatted">query ___composed {<br>  ___firstQuery___requestIndex_0___fieldIndex_0: author {<br>    firstName<br>    lastName<br>  }  ___secondQuery___requestIndex_1___fieldIndex_0: fortuneCookie<br>}</pre>



<p>This query is then sent down to the server and we get a result that looks like this:</p>



<pre class="wp-block-preformatted"><br>{<br> “___firstQuery___requestIndex_0___fieldIndex_0”: {<br>   “firstName”: “John”,<br>   “lastName”: “Smith”<br> }, “___secondQuery___requestIndex_1___fieldIndex_0”: “No snowflake in an avalanche ever feels responsible”,<br>}</pre>



<p>This does look a bit ugly, but your frontend code will never see any of this. The network interface will automatically unpack this result into the results you’d expect for the queries you originally submitted:</p>



<pre class="wp-block-preformatted"><br>{<br>  “author”: {<br>    “firstName”: “John”,<br>    “lastName”: “Smith”<br>  }<br>}{<br>  “fortuneCookie”: “No snowflake in an avalanche ever feels responsible”<br>}<br></pre>



<p>The aliasing allows us to establish a one-to-one relationship between the fields returned by the server and the fields in the query originally submitted, allowing the network interface to unpack the result correctly.</p>



<p>This style of query merging can work with&nbsp;<em>any&nbsp;</em>GraphQL server since it only uses what is available in the GraphQL specification. The merged queries aren’t the prettiest, but Apollo Client handles this for you and allows your UI to render with a single roundtrip.</p>



<h2 class="wp-block-heading" id="1bce"><a href="#transport-level-batching" aria-hidden="true" class="aal_anchor" id="transport-level-batching"><svg aria-hidden="true" class="aal_svg" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Transport-level batching</h2>



<p>Although query merging is very effective, it has a few downsides. Primarily, if you’re trying to debug stuff from the server’s point of view, you’ll see queries that look very different from what you initially fired on the client, which could make it harder to debug your queries. One potential solution is described by Lee Byron, one of the creators of GraphQL,&nbsp;<a href="https://medium.com/apollo-stack/new-features-in-graphql-batch-defer-stream-live-and-subscribe-7585d0c28b07#.7amluhe5g" target="_blank" rel="noreferrer noopener">in his talk about new GraphQL features</a>: batching at the network transport level.</p>



<p>In most current GraphQL servers, requests are sent in the following form:</p>



<pre class="wp-block-preformatted">{<br>  "query": “&lt; query string goes here &gt;”,<br>  "variables": {<br>    &lt;variable values go here&gt;<br>  }<br>}</pre>



<p>The GraphQL server then resolves the query string and returns a single result.</p>



<p>Instead, imagine we submitted a request that looked like this:</p>



<pre class="wp-block-preformatted">[<br>  {<br>    query: &lt; query 0 &gt;,<br>    variables: &lt; variables for query 0 &gt;,<br>  },<br>  {<br>    query: &lt; query 1 &gt;,<br>    variables: &lt; variables for query 1 &gt;,<br>  },<br>  {<br>    query: &lt; query n &gt;<br>    variables: &lt; variables for query n &gt;,<br>  }<br>]</pre>



<p>And the server would return a response that looks like:</p>



<pre class="wp-block-preformatted">[<br>  &lt;result for query 0&gt;,<br>  &lt;result for query 1&gt;,<br>  ...<br>  &lt;result for query n&gt;<br>]</pre>



<p>This is another way of accomplishing what we are currently doing with query merging: fetching the results for multiple queries in a single roundtrip. However, this is much better, since the queries you see on the server will look&nbsp;<em>exactly&nbsp;</em>as they do on the client.</p>



<p>Transport-level batching is easier to debug than query merging, but it requires additional server support. The server has to know how to process an array of queries and respond with an array of results. We will soon be implementing support for this in&nbsp;<a href="https://github.com/apollostack/apollo-server" target="_blank" rel="noreferrer noopener">Apollo Server</a>, and hopefully the GraphQL community can work together to come up with a standard approach that can be used with all GraphQL server implementations.</p>



<p>In the meantime, just passing an additional option to the Apollo Client constructor will let you load your UI in a single roundtrip rather than a dozen, without needing to put in any effort to manually merge queries using fragments. Then, once transport-level batching is built into most GraphQL servers, you’ll be able to reap these benefits while still getting nice-looking queries in your server logs.</p>



<p>Want to try this out?&nbsp;<a href="http://docs.apollostack.com/apollo-client/index.html" target="_blank" rel="noreferrer noopener">Check out Apollo Client.</a>&nbsp;And follow us on Medium for more GraphQL and Apollo content!</p>


Backend

Optimizing Data Fetching with Apollo Client: Leveraging useFragment and Colocated Fragments

Batching Client GraphQL Queries

Query batching in Apollo

Backend

Optimizing Data Fetching with Apollo Client: Leveraging useFragment and Colocated Fragments

Batching Client GraphQL Queries

Query batching in Apollo

Stay in our orbit