DataLoader batch loading bug (MyBatis, JavaEE)

Viewed 220

I have graphql in my production, and it's forbidden to share the code. I'm using graphql-java-servlet, as ORM I use MyBatis.

<dependency>
     <groupId>com.graphql-java-kickstart</groupId>
     <artifactId>graphql-java-servlet</artifactId>
     <version>9.2.0</version>
</dependency>

After enabling BatchLoading I found that graphQL DataLoader confuses places of batch-requested Entities. Also it's easy to check if take a look at futureCacheMap, you will find that Key(Id) And Value(Entity) have different ids.

After debugging, I didn't find the way how graphQL resolves Entities after batchLoading (1000 psc). So I decided that I should have some ordering, so I implemented it, but it didn't resolve the issue.

For example: I Have Parent.class which have Childs inside.

class Parent {
    private Long id;
    private List<Long> childsIds;
}

I have ChildDataloader

private BatchLoader<Long, Child> buildBatchLoader() {
    return list -> > CompletableFuture.supplyAsync(() -> childService.findByIds(list));                
    }

private DataLoader<Long, Child> buildDataLoader(BatchLoader batchLoader) {
    DataLoaderOptions options = DataLoaderOptions.newOptions();
    options.setMaxBatchSize(1000);
    return new DataLoader<Long, Child>(batchLoader, options);
    }
}

I have ChildsFetcher where I call dataLoader.loadMany()

public class ChildsFetcher implements DataFetcher<CompletableFuture<List<Child>>>{

    private static final String PK_FIELD_NAME = "childsIds";

    @Override
    public CompletableFuture<List<LoadDefinitionDTO>> get(DataFetchingEnvironment environment) {
        GraphQLContext context = environment.getContext();
        DataLoaderRegistry dataLoaderRegistry = context.getDataLoaderRegistry().orElseThrow(
                () -> new DalException("there was no dataLoaderRegistry in context", Response.Status.INTERNAL_SERVER_ERROR)
        );
        List<Long> childsIds = getParentFieldValue(environment, PK_FIELD_NAME , List.class);

        DataLoader<Long, Child> childDataLoader = dataLoaderRegistry.getDataLoader("childDataLoader");
        return childDataLoader.loadMany(childsIds)
    }
}

For Example I have 2 Parents with 3 child each.

parents: [
 {
   "id": 1
   "childIds": {1,3,5}
 },
 {
   "id": 2
   "childIds": {2,4,6}
 }
]

As a result in fetcher I will have 2 requests:

  1. childDataLoader.loadMany({1,3,5})

  2. childDataLoader.loadMany({2,4,6})

In Dataloader it will be only one (as expected), but look at the order of the ids (I cannot take control on it):

childService.findByIds({1,3,5,2,4,6})

And in output I will receive:

"data": {
    "parents": [
      { 
        "id": 1,
        "childs": [
          {
            "id": 1,
          },
          {
            "id": 2,
           },
          {
            "id": 3,
           }
       },
      { 
        "id": 2,
        "childs": [
          {
            "id": 4,
          },
          {
            "id": 5,
           },
          {
            "id": 6,
           }
       }
      ]
     }
   ]
}
1 Answers

Order of your answer have to be the same as order of the request, If ORM have sorting the sql response, just resort it back in DataLoader after you get response from ORM, for example :

 private BatchLoader<Long, Child> buildBatchLoader() {
   return list -> CompletableFuture.supplyAsync(() -> 
                   childService.findByIds(list).stream()
                     .sorted(Comparator.comparingLong(entity -> 
                             list.indexOf(entity.getId())))
                     .collect(Collectors.toList()));
 };   
    
         
                     
Related