LazyInitializationException after JOIN FETCH

user_18737324 0
Viewed 232

We have a Spring Boot / JPA application that has an entity which specifies a conversion from and to different units of measure. This conversion is specific for each product we work with and therefore this entity has a composite key with 3 attributes: FromUnitOfMeasure, ToUnitOfMeasure and Product.

@Getter
@Setter
@EqualsAndHashCode(of = "unitOfMeasureConversionByMaterialCompositeKey")
@NoArgsConstructor
@RequiredArgsConstructor
@Entity
public class UnitOfMeasureConversionByMaterial {

    @EmbeddedId
    @NonNull
    private UnitOfMeasureConversionByMaterialCompositeKey unitOfMeasureConversionByMaterialCompositeKey;
    
    @Data
    @NoArgsConstructor
    @AllArgsConstructor
    @Embeddable
    @EqualsAndHashCode
    public static class UnitOfMeasureConversionByMaterialCompositeKey implements Serializable {

        @ManyToOne(fetch = FetchType.LAZY)
        @NonNull
        private Product product;

        @ManyToOne(fetch = FetchType.LAZY)
        @NonNull
        private UnitOfMeasure fromUnitOfMeasure;

        @ManyToOne(fetch = FetchType.LAZY)
        @NonNull
        private UnitOfMeasure toUnitOfMeasure;

    }
    
    private Float conversionValue;

UnitOfMeasure class:

@Getter
@Setter
@ToString(of="id")
@EqualsAndHashCode(of = "id")
@AllArgsConstructor
@NoArgsConstructor
@Entity
public class UnitOfMeasure implements Serializable {

    @Id
    private String id;

    private String description;
    
    public UnitOfMeasure(String id) {
        this.id = id;
    }
    
}

The entire logic runs on an async thread, so no open session is available (equivalent to spring.jpa.open-in-view=false).

The @ManyToOne fields are Lazy, so we employ a JOIN FETCH query in our unit conversion repository to get both the conversions and @ManyToOne fields (thus following the best practice described at https://vladmihalcea.com/the-best-way-to-handle-the-lazyinitializationexception/)

@Query("SELECT cup FROM UnitOfMeasureConversionByMaterial cbm "
            + "LEFT JOIN FETCH cbm.unitOfMeasureConversionByMaterialCompositeKey.product p "
            + "LEFT JOIN FETCH cbm.unitOfMeasureConversionByMaterialCompositeKey.fromUnitOfMeasure fum "
            + "LEFT JOIN FETCH cbm.unitOfMeasureConversionByMaterialCompositeKey.toUnitOfMeasure tum")
List<UnitOfMeasureConversionByMaterial> customFindAllJoinProductsAndUom();

The application then proceeds to iterate all conversion entities and populate a map which indexes conversions by product, fromUnitOfMeasure and targetUnitOfMeasure:

protected Map<Product,Map<UnitOfMeasure,Map<UnitOfMeasure,Float>>> indexedConversionMap = new HashMap<>();

Here is the part of the code where I insert values in the map. The code runs successfully for the first 24.500 conversions, but then throws a 'org.hibernate.LazyInitializationException: could not initialize proxy [test.domain.UnitOfMeasure#kg] - no Session' exception when populating the final level of the map (putting the float conversion value for a given ToUnitOfMeasure).

for (UnitOfMeasureConversionByMaterial unitOfMeasureConversionByMaterial : conversionsExtractedWithJoinFetchQuery) {
            Product product = unitOfMeasureConversionByMaterial.getProduct();
            UnitOfMeasure fromUnitOfMeasure = unitOfMeasureConversionByMaterial.getFromUnitOfMeasure ();
            UnitOfMeasure toUnitOfMeasure = unitOfMeasureConversionByMaterial.getToUnitOfMeasure ();
            Float conversion = unitOfMeasureConversionByMaterial.getConversionValue();            
            
            if (!unitOfMeasureProjection.indexedConversionMap.containsKey(product)) {
                unitOfMeasureProjection.indexedConversionMap.put(product, new HashMap<>());
            }
            if (!unitOfMeasureProjection.indexedConversionMap.get(product).containsKey(fromUnitOfMeasure)) {
                unitOfMeasureProjection.indexedConversionMap.get(product).put(fromUnitOfMeasure, new HashMap<>());
            }
            
            unitOfMeasureProjection.indexedConversionMap.get(product).get(fromUnitOfMeasure).put(
                    toUnitOfMeasure, conversion); // ERROR happens here
        }

The error message is:

org.hibernate.LazyInitializationException: could not initialize proxy [test.domain.UnitOfMeasure#kg] - no Session
    at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:175)
    at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:315)
    at org.hibernate.proxy.pojo.bytebuddy.ByteBuddyInterceptor.intercept(ByteBuddyInterceptor.java:45)
    at org.hibernate.proxy.ProxyConfiguration$InterceptorDispatcher.intercept(ProxyConfiguration.java:95)
    at test.domain.unitofmeasureconversionbymaterial.UnitOfMeasure$HibernateProxy$2LNVM7uD.hashCode(Unknown Source)
    at java.base/java.util.HashMap.hash(HashMap.java:338)
    at java.base/java.util.HashMap.put(HashMap.java:610)
    at test.UnitOfMeasureProjectionFactory.getUnitOfMeasureProjectionWithConversions(UnitOfMeasureProjectionFactory.java:104)
    at test.ConversionService.executeConversion(ConversionService.java:326)
    at test.executeConversion$$FastClassBySpringCGLIB$$a7815e11.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
    at test.ConversionService$$EnhancerBySpringCGLIB$$77fac61.executeConversion(<generated>)
    at test.job.executeAsJob(ConversionJob.java:36)
    at test.job.executeAsJob(ConversionJob.java:20)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)

Some additional information :

  • I have have tried to delombok Equals and HashCode at the UnitOfMeasure entity
    • They work normally and only depend on the String id attribute
  • spring.jpa.open-in-view=true is not an option as we are starting a different thread. Performance is also key on this application so we must not count on lazy fetching
  • An overarching @Transactional annotation in our service is also not an option as hundreds of thousands of records are being saved in batches. We have memory constraints (application and database) which would be violated by a single-transaction operation

What I do not understand is:

  • Why does the code run with no issues for the first 24.500 conversions and only then the error shows up
  • Why even when we extract conversions with JOIN FETCH for all @ManyToOne attributes (which in turn have simple String ids) does Hibernate try to access the database to fetch the information

What could be going wrong? Any ideas as to how to debug this exception?

Edit 1:

Here's the SQL for the custom Query:

    select
        unitofmeas0_.product_id as product_4_31_0_,
        unitofmeas0_.to_unit_of_measure_id as unit_of_3_31_0_,
        unitofmeas0_.from_unit_of_measure_id as unit_of_2_31_0_,
        product1_.id as id1_97_1_,
        unitofmeas2_.id as id1_129_2_,
        unitofmeas3_.id as id1_129_3_,
        unitofmeas0_.conversion_value as conversi1_31_0_,
        product1_.active as active2_97_1_,
        product1_.descontinuation_date as desconti3_97_1_,
        product1_.introduction_date as introduc4_97_1_,
        product1_.description as descript5_97_1_,
        product1_.ean as ean6_97_1_,
        product1_.operational_model as operatio7_97_1_,
        product1_.ncm as ncm8_97_1_,
        product1_.standard_unit_of_measure_id as standard9_97_1_,
        product1_.transfer_unit_of_measure_id as transfer10_97_1_,
        product1_.sales_unit_of_measure_id as sales_un11_97_1_,
        unitofmeas2_.description as descript2_129_2_,
        unitofmeas3_.description as descript2_129_3_ 
    from
        unit_of_measure_conversion_by_material unitofmeas0_ 
    left outer join
        product product1_ 
            on unitofmeas0_.product_id=product_.id 
    left outer join
        unit_of_measure unitofmeas2_ 
            on unitofmeas0_.from_unit_of_measure_id=unitofmeas2_.id 
    left outer join
        unit_of_measure unitofmeas3_ 
            on unitofmeas0_.to_unit_of_measure_id=unitofmeas3_.id

We have other relationships to UnitOfMeasure in the Product entity, but we believe that is not the problem since we tried to JOIN FETCH the Product fields with the following Query and the error persists.

@Query("SELECT cup FROM UnitOfMeasureConversionByMaterial cbm "
            + "LEFT JOIN FETCH cbm.unitOfMeasureConversionByMaterialCompositeKey.product p "
            + "LEFT JOIN FETCH cbm.unitOfMeasureConversionByMaterialCompositeKey.fromUnitOfMeasure fum "
            + "LEFT JOIN FETCH cbm.unitOfMeasureConversionByMaterialCompositeKey.toUnitOfMeasure tum "
            + "LEFT JOIN FETCH p.standardUnitOfMeasure stum "
            + "LEFT JOIN FETCH p.salesUnitOfMeasure saum "
            + "LEFT JOIN FETCH p.transferUnitOfMeasure tum")
    List<UnitOfMeasureConversionByMaterial> customFindAllJoinProductsAndUom();

The SQL:

    select
        unitofmeas0_.product_id as product_4_31_0_,
        unitofmeas0_.to_unit_of_measure_id as unit_of_3_31_0_,
        unitofmeas0_.from_unit_of_measure_id as unit_of_2_31_0_,
        product1_.id as id1_97_1_,
        unitofmeas2_.id as id1_129_2_,
        unitofmeas3_.id as id1_129_3_,
        unitofmeas4_.id as id1_129_4_,
        unitofmeas5_.id as id1_129_5_,
        unitofmeas6_.id as id1_129_6_,
        unitofmeas0_.conversion_value as conversi1_31_0_,
        product1_.active as active2_97_1_,
        product1_.descontinuation_date as desconti3_97_1_,
        product1_.introduction_date as introduc4_97_1_,
        product1_.description as descript5_97_1_,
        product1_.ean as ean6_97_1_,
        product1_.operational_model as operatio7_97_1_,
        product1_.ncm as ncm8_97_1_,
        product1_.standard_unit_of_measure_id as standard9_97_1_,
        product1_.transfer_unit_of_measure_id as transfer10_97_1_,
        product1_.sales_unit_of_measure_id as sales_un11_97_1_,
        unitofmeas2_.description as descript2_129_2_,
        unitofmeas3_.description as descript2_129_3_,
        unitofmeas4_.description as descript2_129_4_,
        unitofmeas5_.description as descript2_129_5_,
        unitofmeas6_.description as descript2_129_6_ 
    from
        unit_of_measure_conversion_by_material unitofmeas0_ 
    left outer join
        product product1_ 
            on unitofmeas0_.product_id=product1_.id 
    left outer join
        unit_of_measure unitofmeas2_ 
            on unitofmeas0_.from_unit_of_measure_id=unitofmeas2_.id 
    left outer join
        unit_of_measure unitofmeas3_ 
            on unitofmeas0_.to_unit_of_measure_id=unitofmeas3_.id 
    left outer join
        unit_of_measure unitofmeas4_ 
            on product1_.standard_unit_of_measure_id=unitofmeas4_.id 
    left outer join
        unit_of_measure unitofmeas5_ 
            on product1_.sales_unit_of_measure_id=unitofmeas5_.id 
    left outer join
        unit_of_measure unitofmeas6_ 
            on product1_.transfer_unit_of_measure_id_id=unitofmeas6_.id

Edit 2:

We also tried to separete the method calls in the following way to analyze the problem:

for (UnitOfMeasureConversionByMaterial unitOfMeasureConversionByMaterial : conversionsExtractedWithJoinFetchQuery) {
    unitOfMeasureConversionByMaterial.getFromUnitOfMeasure().getId();
    unitOfMeasureConversionByMaterial.getFromUnitOfMeasure().hashCode();
    unitOfMeasureConversionByMaterial.getToUnitOfMeasure().getId();
    unitOfMeasureConversionByMaterial.getToUnitOfMeasure().hashCode();
}

Everything works the intended way until we try to call the "hashCode()" method in "toUnitOfMeasure" in the fifth line. The "unitOfMeasureConversionByMaterial.getToUnitOfMeasure()" part works correctly, but as soon as we try to call the "hashCode()" method the code generates the LazyInitializationException. It doesn't even execute the first line of the method.

jpa hibernate spring java
1 Answers

Why does the code run with no issues for the first 24.500 conversions and only then the error shows up

HashMap uses not only hashCode() method but equals() method too. equals() is used to get a value from a map, when a bucket has more than one value. I didn't look into actual implementation of HashMap, it can use equals(), for example, during increasing memory of the conatiner.

Also, for example, toString() method of persistent object can be used somewhere. For example during logging of an error.

Keep in mind, that Hibernate doesn't look into methods of persistent class. So if any method like toString() is invoked, Hibernate tries to load all lazy associations, because they can be used in the method.

How to debug

  1. Temporary open Hibernate session for all part of the code.
  2. Enable SQL logging
  3. Try to set breakpoint after 24500 conversions and debug the code Keep in mind that debugger can invoke toString() method and cause to load lazy associations.

Some notices

Overriding hashCode() and equals() methods of persistent classes is a bad idea. Using Lombok for that is a very very bad idea.

Using persistent classes as keys of a map is a bas idea too.

Using composite keys is not convenient almost in all cases.

user_34051710
Related