Error in deserializing pojo object with Jooq

user_9614332 0
Viewed 32

I am facing a strange, at least for me, deserialization problem using Jooq. I have a POJO, a pretty simple one (with some custom Deserilizers but not related to this problem):

@Builder
@Value
@Jacksonized
@JsonIgnoreProperties(ignoreUnknown = true)
public class AlertVector implements Serializable {

    @JsonProperty(ERROR_ID)
    @Column(name = ALERTS_ERROR_ID_COLUMN, nullable=false)
    private long alertVectorId;

    /**
     * The GTM instant of the Alert log.
     */
    @JsonProperty(ERROR_TIMESTAMP)
    @Column(name = ALERTS_ERROR_TIMESTAMP_COLUMN)
    private Instant alertVectorTimestamp;
    
    /**
     * Machine Serial Number
     */
    @JsonProperty(MACHINE_SERIAL_NUMBER)
    @Column(name = ALERTS_MACHINE_SERIAL_NUMBER_COLUMN)
    @Builder.Default private int machineSerialNumber = -1;

    /**
     * Status of the record 
     * (i.e. the data are sent or not to ES)
     */
    @JsonIgnore
    @Column(name = ALERTS_RECORD_STATUS_COLUMN)
    private int recordStatus;

        /**
     * Array of Safety Faults bits
     */
    @JsonDeserialize(using = ByteArrayDeserializer.class)
    @JsonSerialize(using = ByteArraySerializer.class)
    @JsonProperty(SAFETY_FAULTS_NAME)
    @Column(name = ALERTS_SAFETY_FAULTS_COLUMN)
    private byte[] safetyFaults;

    /**
     * Array of Errors bits
     */
    @JsonDeserialize(using = ByteArrayDeserializer.class)
    @JsonSerialize(using = ByteArraySerializer.class)
    @JsonProperty(ERRORS_NAME)
    @Column(name = ALERTS_ERRORS_COLUMN)
    private byte[] errors;
    
    /**
     * Array of Warnings bits
     */
    @JsonDeserialize(using = ByteArrayDeserializer.class)
    @JsonSerialize(using = ByteArraySerializer.class)
    @JsonProperty(WARNINGS_NAME)
    @Column(name = ALERTS_WARNINGS_COLUMN)
    private byte[] warnings;
}

I build a test database with H2 to write some Junit tests. The test databases is fed with 24 records of data with:

URI alertURI = getClass().getClassLoader().getResource("alerts.csv").toURI();
String strAlerts = Files.readString(Path.of(alertURI), StandardCharsets.US_ASCII);
Result<AlertsRecord> alertsRecordsResult = context.fetchFromCSV(strAlerts).into(it.fox.mysql.tables.Alerts.ALERTS);
context.insertInto(ALERTS).columns(ALERTS.fields()).valuesOfRecords(alertsRecordsResult).execute();

I have checked that the data are correctly ingested in the DB like I expect reading back a record and visually checking it:

context.select(ALERTS.asterisk()).from(ALERTS).where(ALERTS.ERRORID.eq(1661367543364l)).fetch()

"+-------------+--------------------+------------+-------------------+----------------------------------------+----------------------------+----------------------------+
|      errorId|errorTimestamp      |recordStatus|machineSerialNumber|safetyFaults                            |errors                      |warnings                    |
+-------------+--------------------+------------+-------------------+----------------------------------------+----------------------------+----------------------------+
|1661367543364|2022-08-24T18:59:03Z|           0|                  5|AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA|AAAAAAAAAAAAAAAAAAAAAAAAAAA=|AAAAAAAAAAAAAAAxAAAAAAAAAAA=|
+-------------+--------------------+------------+-------------------+----------------------------------------+----------------------------+----------------------------+

The strange thing I found is that, even if the data are correct in the DataBase, the deserialization of my POJO is wrong:

context.select().from(ALERTS).where(ALERTS.ERRORID.eq(1661367543364l)).fetchOneInto(AlertVector.class);

The returned POJO has the filed machineSerialNumber and recordStatus inverted:

AlertVector@111 "AlertVector(alertVectorId=1661367536365, alertVectorTimestamp=2022-08-24T18:58:56Z, machineSerialNumber=0, recordStatus=5, safetyFaults=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], errors=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 49], warnings=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])"

If I modify the POJO "inverting" the position of the two variables like this:

/**
 * Status of the record 
 * (i.e. the data are sent or not to ES)
 */
@JsonIgnore
@Column(name = ALERTS_RECORD_STATUS_COLUMN)
private int recordStatus;

/**
 * Machine Serial Number
 */
@JsonProperty(MACHINE_SERIAL_NUMBER)
@Column(name = ALERTS_MACHINE_SERIAL_NUMBER_COLUMN)
@Builder.Default private int machineSerialNumber = -1;

The same code give me a correct AlertVector POJO.

I build the Jooq classes from XML and I have noted that I wrote the definition of recordStatus before the definition of machineSerialNumber:

        <column>
            <table_catalog></table_catalog>
            <table_schema>plc_data</table_schema>
            <table_name>alerts</table_name>
            <column_name>recordStatus</column_name>
            <data_type>int(11)</data_type>
            <character_maximum_length>0</character_maximum_length>
            <numeric_precision>10</numeric_precision>
            <numeric_scale>0</numeric_scale>
            <ordinal_position>3</ordinal_position>
            <is_nullable>true</is_nullable>
            <comment></comment>
        </column>
        <column>
            <table_catalog></table_catalog>
            <table_schema>plc_data</table_schema>
            <table_name>alerts</table_name>
            <column_name>machineSerialNumber</column_name>
            <data_type>int(11)</data_type>
            <character_maximum_length>0</character_maximum_length>
            <numeric_precision>10</numeric_precision>
            <numeric_scale>0</numeric_scale>
            <ordinal_position>4</ordinal_position>
            <is_nullable>true</is_nullable>
            <column_default>NULL</column_default>
            <comment></comment>
        </column>

Could someone explain me the behaviour here? The XML column definition MUST be in the very same order of the one in POJO class?

Thanks, S.

jooq-codegen jooq deserialization java
1 Answers

DefaultRecordMapper vs DefaultConverterProvider

The XML column definition MUST be in the very same order of the one in POJO class?

It's obviously recommended that you match the order of columns from the database also in generated code, because otherwise, you'll run into problems like these.

But it isn't strictly required, nor is your problem strictly related to code generation. It's rather caused by the behaviour of the DefaultRecordMapper. You say:

I have a POJO, a pretty simple one

But it isn't simple. You're mixing Jackson and this @Builder annotation stuff, which both compete for DefaultRecordMapper behaviour. You're probably thinking that Jackson should kick in, mapping database column names to attributes, but that's not the case. The Jackson mapping logic is implemented in the ConverterProvider default implementation, when mapping an individual JSON value to a POJO. In your case, that doesn't apply.

DefaultRecordMapper and constructors

Instead, the record is mapped to the POJO by jOOQ's DefaultRecordMapper, and your @Builder and @Value annotations probably produce a constructor with fields in the wrong order, given your SELECT * doesn't match your XML column definition. But Java constructors, like any Java methods, don't retain their parameter names, so jOOQ can only map by parameter index, which doesn't match in your case.

The real problem: Usage of explicit SELECT * in jOOQ

So, the main problem here is that you're explicitly using the asterisk() (or *), effectively delegating the projection to the effective database schema, rather than using the default projection by jOOQ, which corresponds to that of the generated code. The asterisk() was added explicitly to allow for using this feature, for whatever reason. There's hardly any reason why users should use asterisk(), unless they want this from jOOQ. Better just write:

context.selectFrom(ALERTS).where(ALERTS.ERRORID.eq(1661367543364l)).fetch()

This is also documented in the manual, or the Javadoc.

Whenever jOOQ generates an asterisk (explicitly, or because jOOQ doesn't know the exact projection), the column order, and the column set are defined by the database server, not jOOQ. If you're using generated code, this may lead to problems as there might be a different column order than expected, as well as too many or too few columns might be projected.

user_5217990
Related