I have been trying to adjust my batching config in Spring Boot and It had worked pretty well until I verified postgres logs.
Here is my application.yaml config:
spring:
jpa:
properties:
hibernate:
order_inserts: true
order_updates: true
batch_versioned_data: true
jdbc:
batch_size: ${JDBC_BATCH_SIZE:10}
Here is my model:
public class MyEntity {
@Id
@GeneratedValue(
strategy = GenerationType.SEQUENCE,
generator = "my_entity_id_seq")
@GenericGenerator(
name = "my_entity_id_seq",
strategy = "com.vladmihalcea.hibernate.id.BatchSequenceGenerator",
parameters = {
@org.hibernate.annotations.Parameter(
name = "sequence",
value = "my_entity_id_seq"),
@org.hibernate.annotations.Parameter(name = "fetch_size", value = "30")
})
private Long id;
}
Result using data-proxy trying to insert 35 entities (simplified log):
Name:Batch-Logger, Connection:4, Time:1, Success:True
Type:Prepared, Batch:False, QuerySize:1, BatchSize:0
Query:["WITH RECURSIVE t(n, level_num) AS (SELECT nextval ('my_entity_id_seq') AS n, 1 AS level_num UNION ALL SELECT nextval ('my_entity_id_seq') AS n, level_num + 1 AS level_num FROM t WHERE level_num < ?) SELECT n FROM t"]
Params:[(30)]
Name:Batch-Logger, Connection:4, Time:0, Success:True
Type:Prepared, Batch:False, QuerySize:1, BatchSize:0
Query:["WITH RECURSIVE t(n, level_num) AS (SELECT nextval ('my_entity_id_seq') AS n, 1 AS level_num UNION ALL SELECT nextval ('my_entity_id_seq') AS n, level_num + 1 AS level_num FROM t WHERE level_num < ?) SELECT n FROM t"]
Params:[(30)]
Name:Batch-Logger, Connection:4, Time:3, Success:True
Type:Prepared, Batch:True, QuerySize:1, BatchSize:10
Name:Batch-Logger, Connection:4, Time:1, Success:True
Type:Prepared, Batch:True, QuerySize:1, BatchSize:10
Name:Batch-Logger, Connection:4, Time:1, Success:True
Type:Prepared, Batch:True, QuerySize:1, BatchSize:10
Name:Batch-Logger, Connection:4, Time:1, Success:True
Type:Prepared, Batch:True, QuerySize:1, BatchSize:5
Showing:
- 2 round trip for the IDs (30x2)
- 4 batching (3x10 and 1x5).
Postgres logs simplified for better visibility:
[...]
values
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16),
($17, $18, $19, $20, $21, $22, $23, $24),
($25, $26, $27, $28, $29, $30, $31, $32),
($33, $34, $35, $36, $37, $38, $39, $40),
($41, $42, $43, $44, $45, $46, $47, $48),
($49, $50, $51, $52, $53, $54, $55, $56),
($57, $58, $59, $60, $61, $62, $63, $64)
LOG: execute <unnamed>: insert into public.my_table
[...]
values
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16)
LOG: execute <unnamed>: insert into public.my_table
[...]
values
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16),
($17, $18, $19, $20, $21, $22, $23, $24),
($25, $26, $27, $28, $29, $30, $31, $32),
($33, $34, $35, $36, $37, $38, $39, $40),
($41, $42, $43, $44, $45, $46, $47, $48),
($49, $50, $51, $52, $53, $54, $55, $56),
($57, $58, $59, $60, $61, $62, $63, $64)
LOG: execute <unnamed>: insert into public.my_table
[...]
values
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16)
LOG: execute <unnamed>: insert into public.my_table
[...]
values
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16),
($17, $18, $19, $20, $21, $22, $23, $24),
($25, $26, $27, $28, $29, $30, $31, $32),
($33, $34, $35, $36, $37, $38, $39, $40),
($41, $42, $43, $44, $45, $46, $47, $48),
($49, $50, $51, $52, $53, $54, $55, $56),
($57, $58, $59, $60, $61, $62, $63, $64)
LOG: execute <unnamed>: insert into public.my_table
[...]
values
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16)
LOG: execute <unnamed>: insert into public.my_table
[...]
values
($1, $2, $3, $4, $5, $6, $7, $8),
($9, $10, $11, $12, $13, $14, $15, $16),
($17, $18, $19, $20, $21, $22, $23, $24),
($25, $26, $27, $28, $29, $30, $31, $32)
LOG: execute <unnamed>: insert into public.my_table
[...]
values
($1, $2, $3, $4, $5, $6, $7, $8)
Showing:
- 1 batch: 8
- 2 batch: 2
- 3 batch: 8
- 4 batch: 2
- 5 batch: 8
- 6 batch: 2
- 7 batch: 4
- 8 batch: 1
I was pretty surprise with the result ? Does anyone had the same result ?
Jonathan.