I know that the spark-cassandra connector comes with its own default loadbalancing policy implementation(DefaultLoadBalancingPolicy). How can I go about implementing my own custom LoadBalancing class? I want to have the application use the WhiteListRoundRobin policy. What steps would I need to take? I'm still a newbie in working with spark and Cassandra and I would appreciate any guidance in this. Thanks
1 Answers
You can look into implementation of LocalNodeFirstLoadBalancingPolicy - basically you need to create (if it doesn't exist) a class inherited from LoadBalancingPolicy, and implement your required logic for load balancing.
Then you need to create a class implementing CassandraConnectionFactory that will configure Cassandra session with required load balancing implementation. The simplest way is to take the code of DefaultConnectionFactory, but instead of using LocalNodeFirstLoadBalancingPolicy, specify your load balancing class.
And then you specify that connection factory class name in the spark.cassandra.connection.factory configuration property.
Related