Run cassandra rebuild in parallel

user_3904501 0
Viewed 47

Am running nodetool rebuild, there is a table having 400 sstables on one node from where streaming is happening. Only one file is being streamed at a time, is there any way to parallelize this operation so that multiple sstables can be streamed in parallel rather than sequential file streaming.

Mode: NORMAL
Rebuild f151a460-9bdd-11ec-82be-c9782f58af4d
    /10.1.1.1
        Sending 362 files, 856967791730 bytes total. Already sent 217 files, 516836789972 bytes total
            /CassandraData/data/keyspace/tableA-bcb7f6b460bf3dfabeea64cf6b01c3fe/mc-204328-big-Data.db 8932078253/35051713491 bytes(25%) sent to idx:0/10.2.2.2

Tried by increasing (doubled i.e. 400mb from 200mb) the streaming throughput but its not helping much i.e. its taking same time to stream the same amount of data with increased throughput values as well. Throughput is not helping, looking for a way to parallelize streaming operation at table level.

cassandra
1 Answers

It isn't possible to increase the number of streaming threads. In any case, there are several factors which affect the speed of the streaming, not just network throughput. The type of disks as well as the data model have a significant impact on how quick the JVM can serialise the data to stream as well as how quick it can cleanup the heap (GC).

I see that you've already tried to increase the streaming throughput. Note that you'll need to increase it for both the sending and receiving nodes (and really, all nodes) otherwise, the stream will only be as fast as the slowest node. Cheers!

user_42695350
Related