I'm used to crafting ProducerRecord instances and schlepping them off to my cluster using
// build up properties
// build KakfaProducer with properties
ProducerRecord<String,String> reccord = new ProducerRecord<String,String>("myTopic","woohoo");
producer.send( reccord);
this all depends on having the org.apache.kafka.* packages available.
A different 3rd-party application that we can write custom code for in Java8 and Groovy does NOT have org.apache.kafka.* packages available so must craft messages by-hand.
Is there a way(s) to capture what gets sent over the wire for analysis? Perhaps a ByteArrayOutputStream then loading what gets written to the buffer into a bytearray for viewing. Perhaps javax.interceptors.* could be useful.
TIA,
Still-learning Steve
<!-- update 2022-06-26T00:10:10.000Z -->
This is all running on a Windoze 10 box to which I have Admin access but cannot add any software to i.e. wireshark. So I can run pktmon and set a filter for port 9002 which the kafka cluster is running on. I also have a PowerShell panel open running kafka-console-producer where I can manually send messages, and a corresponding PowerShell panel open running kafka-console-consumer where I can see the messages appearing. Between all this plus the Java producer and Java consumer there simply must be a way to capture the serialized data on the wire.
<!-- update 2022-06-27T05:20:10.000Z -->
Two of the proffered solutions are spot-on -- except both rely on wireshark, which I mentioned I cannot load. Current approach is to create a ServerSocket aimed at port 9092 and an InputStream and try to bind a listener on the port and stream the bytes to an array for inspection. Fingers crossed.
<!-- update 2022-06-30T00:40:10.000Z -->
A variation of this kinda worked - I shut down my instance of Kafka and started a simple server on port 9092 that simply printed the stream to console. Sending a message from kafka-console-producer caused this to be printed
0x0 0x0 0x0 0x1a 0x0 0x12 0x0 0x2 0x0 0x0 0x0 0x0 0x0 0x10 0x50 0x61 0x73 0x73 0x70 0x6f 0x72 0x74 0x50 0x72 0x6f 0x64 0x75 0x63 0x65 0x72
which works out to
64-bit integer == 26 == total number of bytes following
32-bit integer ???
32-bit integer ???
64-bit integer == 16 == number of bytes in following string
PassportProducer
but no topic, key, or value turns up. Which leads me to think there's some back-and-forth between source and sink. I've tried attaching my server on the same port as the kafka service but get a "port already in use" errmsg. Can't two apps share a port if one only listens?