I develop a Kafka connector which simply creates messages for each line in a file retrieved from an external API. It works nicely but now I try to consume the messages and I have two strange bytes at the beginning of each value. I can reproduce the problem with the console consumer and with my kafka stream processor.
�168410002,OpenX Market,459980962,OpenX_Bidder_Order_merkur_bidder_800x250,313115722,OpenX_Bidder_ANY_LI_merkur_800x250_550,106800839362,OpenX_Bidder_Creative_merkur_800x250_2,10
The source files are good and even printlns before creating the SourceRecord don't show these two bytes. I used a struct with one field before and now use a simple String schema but I still have the same problem:
def convert(line: String, ...) = {
...
val record = new SourceRecord(
Partition.sole(partition),
offset.forConnectApi,
topic,
Schema.STRING_SCHEMA,
line
)
...
So in the above code, if I add println(line) no strange chars are shown.