what is the difference between kafka ProducerRecord and KeyedMessage

Viewed 5743

I'm measuring the kafka producer producer performance. Currently I've met two clients with bit different configuration and usage:

Common:

def buildKafkaConfig(hosts: String, port: Int): Properties = {
  val props = new Properties()    
  props.put("metadata.broker.list", brokers)
  props.put("serializer.class", "kafka.serializer.StringEncoder")
  props.put("producer.type", "async") 
  props.put("request.required.acks", "0")
  props.put("queue.buffering.max.ms", "5000")
  props.put("queue.buffering.max.messages", "2000")
  props.put("batch.num.messages", "300")
  props
}

First Client:

"org.apache.kafka" % "kafka_2.11" % "0.8.2.2" 

Usage:

val kafkaConfig = KafkaUtils.buildKafkaConfig("kafkahost", 9092)
val producer = new Producer[String, String](new ProducerConfig(kafkaConfig))

// ... somewhere in code 
producer.send(new KeyedMessage[String, String]("my-topic", data))

Second Client:

"org.apache.kafka" % "kafka-clients" % "0.8.2.2"

Usage:

val kafkaConfig = KafkaUtils.buildKafkaConfig("kafkahost", 9092)
val producer = new KafkaProducer[String, String](kafkaConfig)
// ... somewhere in code 
producer.send(new ProducerRecord[String, String]("my-topic", data))

My questions are:

  • What is the difference between 2 clients?
  • Which properties should I configure, take into account to achieve optimal, high heavy writes performance, for high scale application?
1 Answers
Related