Batching insert/update using JPA and Hibernate does not align with Postgresql logs

Viewed 100

I have been trying to adjust my batching config in Spring Boot and It had worked pretty well until I verified postgres logs.

Here is my application.yaml config:

spring:
  jpa:
    properties:
      hibernate:
        order_inserts: true
        order_updates: true
        batch_versioned_data: true
        jdbc:
          batch_size: ${JDBC_BATCH_SIZE:10}

Here is my model:

public class MyEntity {

  @Id
  @GeneratedValue(
      strategy = GenerationType.SEQUENCE,
      generator = "my_entity_id_seq")
  @GenericGenerator(
      name = "my_entity_id_seq",
      strategy = "com.vladmihalcea.hibernate.id.BatchSequenceGenerator",
      parameters = {
        @org.hibernate.annotations.Parameter(
            name = "sequence",
            value = "my_entity_id_seq"),
        @org.hibernate.annotations.Parameter(name = "fetch_size", value = "30")
      })
  private Long id;
}

Result using data-proxy trying to insert 35 entities (simplified log):

Name:Batch-Logger, Connection:4, Time:1, Success:True
Type:Prepared, Batch:False, QuerySize:1, BatchSize:0
Query:["WITH RECURSIVE t(n, level_num) AS (SELECT nextval ('my_entity_id_seq') AS n, 1 AS level_num UNION ALL SELECT nextval ('my_entity_id_seq') AS n, level_num + 1 AS level_num  FROM t  WHERE level_num < ?) SELECT n FROM t"]
Params:[(30)]

Name:Batch-Logger, Connection:4, Time:0, Success:True
Type:Prepared, Batch:False, QuerySize:1, BatchSize:0
Query:["WITH RECURSIVE t(n, level_num) AS (SELECT nextval ('my_entity_id_seq') AS n, 1 AS level_num UNION ALL SELECT nextval ('my_entity_id_seq') AS n, level_num + 1 AS level_num  FROM t  WHERE level_num < ?) SELECT n FROM t"]
Params:[(30)]

Name:Batch-Logger, Connection:4, Time:3, Success:True
Type:Prepared, Batch:True, QuerySize:1, BatchSize:10

Name:Batch-Logger, Connection:4, Time:1, Success:True
Type:Prepared, Batch:True, QuerySize:1, BatchSize:10

Name:Batch-Logger, Connection:4, Time:1, Success:True
Type:Prepared, Batch:True, QuerySize:1, BatchSize:10

Name:Batch-Logger, Connection:4, Time:1, Success:True
Type:Prepared, Batch:True, QuerySize:1, BatchSize:5

Showing:

  • 2 round trip for the IDs (30x2)
  • 4 batching (3x10 and 1x5).

Postgres logs simplified for better visibility:

[...]
values 
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16),
($17, $18, $19, $20, $21, $22, $23, $24),
($25, $26, $27, $28, $29, $30, $31, $32),
($33, $34, $35, $36, $37, $38, $39, $40),
($41, $42, $43, $44, $45, $46, $47, $48),
($49, $50, $51, $52, $53, $54, $55, $56),
($57, $58, $59, $60, $61, $62, $63, $64)

LOG:  execute <unnamed>: insert into public.my_table 
[...]
values 
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16)

LOG:  execute <unnamed>: insert into public.my_table 
[...]
values 
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16),
($17, $18, $19, $20, $21, $22, $23, $24),
($25, $26, $27, $28, $29, $30, $31, $32),
($33, $34, $35, $36, $37, $38, $39, $40),
($41, $42, $43, $44, $45, $46, $47, $48),
($49, $50, $51, $52, $53, $54, $55, $56),
($57, $58, $59, $60, $61, $62, $63, $64)

LOG:  execute <unnamed>: insert into public.my_table 
[...]
values 
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16)

LOG:  execute <unnamed>: insert into public.my_table 
[...]
values 
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16),
($17, $18, $19, $20, $21, $22, $23, $24),
($25, $26, $27, $28, $29, $30, $31, $32),
($33, $34, $35, $36, $37, $38, $39, $40),
($41, $42, $43, $44, $45, $46, $47, $48),
($49, $50, $51, $52, $53, $54, $55, $56),
($57, $58, $59, $60, $61, $62, $63, $64)

LOG:  execute <unnamed>: insert into public.my_table 
[...]
values 
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16)

LOG:  execute <unnamed>: insert into public.my_table 
[...]
values 
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16),
($17, $18, $19, $20, $21, $22, $23, $24),
($25, $26, $27, $28, $29, $30, $31, $32)

LOG:  execute <unnamed>: insert into public.my_table 
[...]
values 
($1, $2, $3, $4, $5, $6, $7, $8)

Showing:

  • 1 batch: 8
  • 2 batch: 2
  • 3 batch: 8
  • 4 batch: 2
  • 5 batch: 8
  • 6 batch: 2
  • 7 batch: 4
  • 8 batch: 1

I was pretty surprise with the result ? Does anyone had the same result ?

Jonathan.

0 Answers
Related