Should a Cassandra cluster return consistent results across nodes?

user_1769704 0
Viewed 34

I have a test Cassandra cluster (2.2.9) with 3 docker containers and all on the same docker network, and according to node tool, they all appear to be healthy and aware of each other. However, when I perform the following query on each of the nodes, I get three very different results...

select count(*) from my_table;

 count
-------
  2997


select count(*) from my_table;

 count
-------
     0


select count(*) from my_table;

 count
-------
   500

The correct answer should be the first one - 2997.

Firstly, is this expected behavior? (I assume no, but I'm not really sure how a full table scan is processed?). Secondly, what are the possible things to look for in order to correct this?

cassandra
2 Answers

ok, so I ran the following command, and it seemed to fix the problem:

# nodetool repair -full

I now have the expected number return from all the nodes.

user_17697040

Yes this is expected behavior (different results from different nodes) if they are not in sync. Over the period of time data in nodes deviate from each other due to some nodes are not able to get the writes. To overcome this it is expected to run repair regularly so that nodes sync with each other. So as you mentioned in your answer that running repair solved your problem. On side note, running COUNT without partition key on Cassandra is not a good practice.

user_29308120
Related