How to disable L1 Session caching with Spring Data JPA + Hibernate + PostgresSQL

user_1431656 0
Viewed 2504

I have the following Spring Data JPA Repository

public interface FooRepository extends JpaRepository<Foo, String> {

  @QueryHints(
      value = {
        @QueryHint(name = HINT_FETCH_SIZE, value = "1000"),
        @QueryHint(name = HINT_CACHEABLE, value = "false"),
        @QueryHint(name = HINT_FLUSH_MODE, value = "ALWAYS"),
        @QueryHint(name = HINT_CACHE_MODE, value = "IGNORE"),
        @QueryHint(name = HINT_READONLY, value = "true")
      })
  Stream<Foo> findAll();
}

called in the following method as follow

@Transactional
public void doSomething() {
  AtomicInteger counter = new AtomicInteger();

  try(Stream<Foo> stream = fooRepository.findAll()) {
    stream.forEach(foo -> {
      int i = counter.incrementAndGet();
      logger.info(() -> "" + i);
    });
  }
}

When running this code having millions of Fooentities, this exact code throws an OutOfMemoryError. Looking at the heap dump after it crashes, I see there's a very high amount of MutableEntityEntry, Foo and EntityEntryContext$ManagedEntityImpl. All three have exact same count. On top of that, there's exactly twice that count of EntityKey. For example, I have 40k of each of the first 3 and 80k of EntityKey in the heap dump.

To make this work, I tried without success to manualy flush, clear and garbage collect as follow

@Transactional // org.springframework.transaction.annotation.Transactional
public void doSomething() {
  entityManager.joinTransaction(); // properly injected through Spring DI
  AtomicInteger counter = new AtomicInteger();

  try(Stream<Foo> stream = fooRepository.findAll()) {
    stream.forEach(foo -> {
      int i = counter.incrementAndGet();
      if (i % 100 == 0) {
        fooRepository.flush();
        entityManager.clear();
        System.gc();
        logger.info(() -> "flush, clear, gc");
      }
      logger.info(() -> "" + i);
    });
 }

As no reference are kept in my code to any foo entities streamed and looking to the objects in the heap dump once the error is thrown, I'm suspecting the issue is in the L1 Session cache from Hibernate even if there's a QueryHint desactivating the cache (from my understanding). It feels like only HINT_FETCH_SIZE is working in the given QueryHints on my method and I have no idea why.

FYI, I'm not using Spring Boot at all in my project. So I have the following beans in my SpringConfiguration to configure Spring Data JPA:

@Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory()
    throws MalformedURLException {
  HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
  vendorAdapter.setDatabase(Database.POSTGRESQL);
  vendorAdapter.setGenerateDdl(false);

  LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
  factory.setJpaVendorAdapter(vendorAdapter);
  factory.setPackagesToScan(getClass().getPackage().getName());
  factory.setDataSource(dataSource());
  Properties jpaProperties = new Properties();
  jpaProperties.setProperty(
      "hibernate.physical_naming_strategy",
      "my.domain.hibernate.SnakeCasePhysicalNamingStrategy");
  jpaProperties.setProperty("hibernate.dialect", "org.hibernate.dialect.PostgreSQL10Dialect");
  factory.setJpaProperties(jpaProperties);

  return factory;
}

@Bean
public EntityManager entityManager() throws MalformedURLException {
  return entityManagerFactory().getObject().createEntityManager();
}

@Bean
public PlatformTransactionManager transactionManager() throws MalformedURLException {
  JpaTransactionManager txManager = new JpaTransactionManager();
  txManager.setEntityManagerFactory(entityManagerFactory().getObject());

  return txManager;
}

Here's the version of each

  • Spring 5.2.13.RELEASE
  • Spring Data JPA 2.3.7.RELEASE
  • Hibernate 5.4.28.Final
  • PostgreSQL 13.1 (on alpine using Docker)
spring-data-jpa jpa postgresql hibernate java
2 Answers

I think you should try using DTO projections instead of entities: you are not modifying these objects so there would be no need for Hibernate to keep track of the entity state.

This article explains a few strategies that could help you.

Hibernate also has the concept of stateless session but I've not tried it and I don't think it is exposed via the JPA API.

user_3134000

Finally found the issue which was the way the entityManager was injected in my class. Instead of adding a bean in my SpringConfiguration for it and inject it through the constructor, you must use @PersistenceContext on the field declaration in your class.

Here's the working code:

@PersistenceContext
private EntityManager entityManager;

[...]

@Transactional // org.springframework.transaction.annotation.Transactional
public void doSomething() {
  entityManager.joinTransaction();
  AtomicInteger counter = new AtomicInteger();

  try(Stream<Foo> stream = fooRepository.findAll()) {
    stream.forEach(foo -> {
      int i = counter.incrementAndGet();
      if (i % 100 == 0) {
        entityManager.flush();
        entityManager.clear();
        logger.info(() -> "flush then clear);
      }
      logger.info(() -> "" + i);
    });
 }

So doing entityManager.clear() will properly clear the L1 Session Cache as it's explained here

user_14316560
Related