Kafka JDBC sink connector for a existing table with different column names?

user_1945064 0
Viewed 450

I am trying to create a jdbc sink connector which reads from a topic and puts data in an existing table. The problem is it is not 1:1 mapping with the table. Rather all the column names are different to what i have in the message

Below is how my message in the topic looks like:

{
 "dealNo":"6666777",
"exception": 
{"code":12345E
 "desc": couldnot process the issue,  
},
}

Database table maps like this: ERROR_TB

id: auto generated sequence
srcId : "APP" (need to pass default value) 
factory : dealNo (from json)
body : here i need to pass the entire message
description : exception (from json)

Can someone please guide me how to apply transformations for this? i am new to it and not sure how it can be achieved

confluent-platform apache-kafka-connect apache-kafka
1 Answers

Hopefully, you figured this out a while ago, but for anyone in the future since I had the same question... it seems simple enough from looking at this web page https://docs.confluent.io/platform/current/connect/transforms/replacefield.html#replacefield

You just have to specify these properties when setting up the JDBC connector.

Rename a Field This configuration snippet shows how to use ReplaceField with the renames property.

"transforms": "RenameField",

"transforms.RenameField.type": "org.apache.kafka.connect.transforms.ReplaceField$Value",

"transforms.RenameField.renames": "foo:c1,bar:c2"

This replaces the foo and bar field names with c1 and c2.

Before: {"foo":22,"bar":"baz"}

After: {"c1":22,"c2":"baz"}

user_33089070
Related