Registering a data loader

4. Registering a data loader

10m

Overview

It's time to introduce data loaders to our application.

In this lesson, we will:

Implement and register our data loader

Adding the data loader

We have a new method on our ListingService class that's responsible for fetching amenities data for multiple listings. However, it won't be enough to just update our Listing.amenities datafetcher to use this method.

Imagine we did something like the following, replacing the call to amenitiesRequest with multipleAmenitiesRequest.

datafetchers/ListingDataFetcher

@DgsData(parentType = "Listing")
public List<Amenity> amenities(DgsDataFetchingEnvironment dfe) throws IOException {
    ListingModel listing = dfe.getSource();
    String id = listing.getId();
    Map<String, Boolean> localContext = dfe.getLocalContext();
    if (localContext.get("hasAmenityData")) {
        return listing.getAmenities();
    }
-   return listingService.amenitiesRequest(id);
+   return listingService.multipleAmenitiesRequest(id);
}

Though this change would utilize our new endpoint, it wouldn't actually be enough to make our query more performant. The Listing.amenities datafetcher, which is called for every listing we request amenities for, will still trigger one network request per listing. In other words, we're still missing the logic that can batch together the keys we need to fetch data for between datafetcher executions.

REQUEST FOR LISTING-1 AMENITIES:
GET /amenities/listings?ids=listing-1

REQUEST FOR LISTING-2 AMENITIES:
GET /amenities/listings?ids=listing-2

...etc.

In effect, we'd be doing the same thing as before—calling a REST endpoint for each listing. No performance gains here!

This is where our data loader class comes in.

Data loaders step-by-step

Here's how it will all work together.

We'll create a data loader class, denoting it as such with the DGS @DgsDataLoader annotation.
We'll update our class so that it implements the BatchLoader interface.
We'll give our class a load method. This method is responsible for gathering all of the necessary keys that data needs to be fetched for (those will be the IDs for every listing in the query).
We'll update our datafetcher method for the Listing.amenities field. Rather than calling our ListingService methods directly, it will delegate responsibility to the data loader to gather up, and provide amenity data for, all of the listings in our query. All at once, too!

We'll tackle the first two steps in this lesson, and bring everything home in the next. Let's get started!

Step 1 - Creating the data loader class

We'll start by jumping into our code and creating a new directory, dataloaders, to live alongside datafetchers, datasources, and models.

📂 com.example.listings
┣ 📂 datafetchers
┣ 📂 dataloaders
┣ 📂 datasources
┣ 📂 models
┣ 📄 ListingApplication
┗ 📄 WebConfiguration

Inside of the dataloaders directory, create a class called AmenityDataLoader.

dataloaders/AmenityDataLoader.java

package com.example.listings.dataloaders;

public class AmenityDataLoader {
  // TODO
}

Adding the `@DgsDataLoader` annotation

To be registered in our application as a data loader, we'll bring in some new imports.

dataloaders/AmenityDataLoader

import com.example.listings.datasources.ListingService;
import com.example.listings.generated.types.Amenity;
import com.netflix.graphql.dgs.DgsDataLoader;
import org.dataloader.BatchLoader;
import org.springframework.beans.factory.annotation.Autowired;
import java.io.IOException;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;

Next, we'll mark our class as an official data loader using the @DgsDataLoader annotation. We'll provide it with a name property we can use to identify this data loader elsewhere.

dataloaders/AmenityDataLoader.java

@DgsDataLoader(name = "amenities")
public class AmenityDataLoader {
  // TODO
}

The `BatchLoader` interface

To act as a true data loader, our class needs to implement an interface that handles batch loading. In this course, we'll use BatchLoader from the dataloader library. BatchLoader is a utility that accepts a list of keys, and loads the corresponding values.

The BatchLoader interface takes in two parameters:

The type of data for each identifier the batch loader will collect
The type of data that the batch loader is expected to return for each identifier

Let's go ahead and update our class to implement this interface.

For this particular data loader, the BatchLoader will collect some number of listing IDs of type String; and return a List of Amenity types for each. (Remember, each listing can have more than one amenity, which is why we get back a List<Amenity> type for each listing we request!)

dataloaders/AmenityDataLoader.java

public class AmenityDataLoader implements BatchLoader<String, List<Amenity>> {
  // TODO
}

Step 2 - Adding the load method

Next, we'll provide the class' load method. This method has a very specific signature.

The load method signature

public CompletionStage<List<SomeJavaClass>> load(List<DataTypeOfIdentifiers> listOfIdentifiers) {
  // logic to fetch data by identifiers
}

It needs to accept a List of keys, and return a CompletionStage type, which accepts a List of whatever class the batch loader resolves to.

Let's apply these one by one to see how they come together.

First, we'll write our load method and give it a List<String> parameter called listingIds.

dataloaders/AmenityDataLoader.java

public void load(List<String> listingIds) {
  // TODO
}

Next, let's update the return type. To comply with the BatchLoader interface, our method needs to return a CompletionStage type, which accepts a type variable.

public CompletionStage<> load(List<String> listingIds) {
  // TODO
}

Note: We use the CompletionStage interface when working with asynchronous actions. We'll use CompletableFuture, a class that implements the CompletionStage interface, shortly.

For the type variable, we'll pass the type that we expect our ListingService class' multipleAmenitiesRequest method to return—namely, a List of List<Amenity> types (one list of amenities for each listing)! So, we can update our load method's return type so that CompletionStage accepts a type variable of List<List<Amenity>>.

Here's what that looks like.

public CompletionStage<List<List<Amenity>>> load(List<String> listingIds) {
  // TODO
}

Finally, we need to make the call to our ListingService class method multipleAmenitiesRequest, using the listingIds parameter, to actually get our amenity data.

But we can't just call the method here, and return the results. Our load method's signature indicates that it returns a CompletionStage type.

To satisfy this, we'll use CompletableFuture, a class that implements the CompletionStage interface, and call one of its methods: supplyAsync. This method accepts a function, which is where we'll actually call the multipleAmenitiesRequest method, passing in listingIds.

public CompletionStage<List<List<Amenity>>> load(List<String> listingIds) {
  return CompletableFuture.supplyAsync(() -> listingService.multipleAmenitiesRequest(listingIds));
}

This call can result in a thrown exception, so we'll wrap our code in a try/catch.

public CompletionStage<List<List<Amenity>>> load(List<String> listingIds) {
  return CompletableFuture.supplyAsync(() -> {
      try {
          return listingService.multipleAmenitiesRequest(listingIds);
      } catch (IOException e) {
          throw new RuntimeException(e);
      }
  });
}

Note: Be sure to return the results from calling listingService.multipleAmenitiesRequest within the try block!

Our AmenityDataLoader class is nearly complete. For our load method to be valid, we need to apply the @Override annotation (this overrides the BatchLoader interface's implementation). We also need to provide an instance of the ListingService for our data loader to work with. Note that we're using the Spring @Autowired annotation to take advantage of dependency injection. This means that ListingService will automatically be injected when we create an instance of the AmenityDataLoader class.

@DgsDataLoader(name = "amenities")
public class AmenityDataLoader implements BatchLoader<String, Amenity> {

  @Autowired
  ListingService listingService;

  @Override
  public CompletionStage<List<List<Amenity>>> load(List<String> listingIds) {
      return CompletableFuture.supplyAsync(() -> {
          try {
              return listingService.multipleAmenitiesRequest(listingIds);
          } catch (IOException e) {
              throw new RuntimeException(e);
          }
      });
  }
}

And that's everything we need for the data loader to do its job!

Practice

Which of the following are required to register a new DGS data loader?

Implementing the DataLoader interface.Annotating the class with @DgsDataLoader.Defining the data loader method within the datafetcher class that uses it.Defining a load method with a specific signature.

Key takeaways

To register a data loader in our app, we need to complete three steps:
- The class needs to have the @DgsDataLoader annotation applied, with a name provided
- The class needs to implement an interface that supports batch loading. In this course, we've used the BatchLoader interface.
- The class needs a load function that follows a very specific signature

Up next

One last step, and our data loader will do the rest of the heavy lifting for us. In the next lesson, we'll finally apply our data loader inside of our datafetcher.

Share your questions and comments about this lesson

This course is currently in

beta

. Your feedback helps us improve! If you're stuck or confused, let us know and we'll help you out. All comments are public and must follow the Apollo Code of Conduct. Note that comments that have been resolved or addressed may be removed.

You'll need a GitHub account to post below. Don't have one? Post in our Odyssey forum instead.

Overview

Adding the data loader

Data loaders step-by-step

Step 1 - Creating the data loader class

Adding the @DgsDataLoader annotation

The BatchLoader interface

Step 2 - Adding the load method

Practice

Key takeaways

Up next

Share your questions and comments about this lesson

Adding the `@DgsDataLoader` annotation

The `BatchLoader` interface