I am attempting to send data to a Kafka topic via python as a remote producer. My script reports no exceptions, but nothing shows up in Kafka. I'm able to get a response from the brokers (all on one separate host) using the consumer method. Looking through the forums I saw to make sure and flush the write cache, but no luck there. Script is below:
from kafka import KafkaProducer, KafkaConsumer
from json import dumps
producer = KafkaProducer(bootstrap_servers='192.168.1.100:9093', value_serializer=lambda x: dumps(x).encode('utf-8'))
producer.send('home-sensehat-temperature',{"timestamp": "2020-08-12 23:31:19.102347", "temperature": 127.6969})
producer.flush()
consumer=KafkaConsumer(bootstrap_servers='192.168.1.100:9093')
print(consumer.topics())
The response I get from consumer.topics() is:
{'home-sensehat-temperature', 'home-camera-path', 'home-sensehat-humidity', 'home-sensehat-pressure'}
So this implies I can make a good connection to the brokers.
I tried digging through the kafka broker logs but couldn't find anything. Any help would be greatly appreciated!