How to configure Kafka environment from different frameworks of Js and Go to ship JSON

Kumarsatyam4324 0
Viewed 32

I am building a couple of micro-services and a logging micro-service along with it.

The logging micro-service code will come after the other micro-services have sent their logs to kafka. Logging as a micro-service has to consume all the log data which is sent from JS-based micro-services and I have to receive the JSON in GO.
Is there any other way without using a parser? (Just like g-RPC changes the whole data to binary for faster transportation and is understandable for every environment.)

I have very less knowledge about how the different environments work together when using a message broker.

apache-kafka logging go microservices javascript
2 Answers
Vlad Goldman0

I'm not sure I fully understand what you're asking.

Kafka stores bytes, and there's clients in both JS or Golang. Both languages/environments need to have available libraries for serialization and deserialization. Regarding golang, you don't need a struct to read JSON.

like g-RPC changes the whole data to binary for faster transportation and is understandable for every environment

You can use Protobuf in Kafka topics.

Alternatively, you can use tools like Elasticsearch or Splunk to consume any Kafka event and index the fields after deserializing those records without needing any Go consumer.

But yes, a parser will be needed somewhere to index your data into a searchable format.

OneCricketeer0
Related