Proper Consistency Level to read 'everything'

Question

I'm creating a sync program to periodically copy our Cassandra data into another database. The database I'm copying from only gets INSERTs - data is never UPDATEd or DELETEd. I would like to address Cassandra's eventual consistency model in two ways:

1 - Each sync scan overlaps the last by a certain time span. For example, if the scan happens every hour, then each scan looks an hour and a half backwards. The data contains a unique key, so reading the same record in more than one scan is not an issue.

2 - I use a Consistency level of ALL to ensure that I'm scanning all of the nodes on the cluster for the data.

Is ALL the best Consistency for this situation? I just need to see a record on any node, I don't care if it appears on any other nodes. But I don't want to miss any INSERTed records either. But I also don't want to experience timeouts or performance issues because Cassandra is waiting for multiple nodes to see that record.

To complicate this a bit more, this Cassandra network is made up of 6 clusters in different geographic locations. I am only querying one. My assumption is that the overlap mentioned in #1 will eventually catch up records that exist on other clusters.

The query I'm doing is like this:

SELECT ... FROM transactions WHERE userid=:userid AND transactiondate>:(lastscan-overlap)

Where userid is the partioning key and transactiondate is a clustering column. The list of userId's is sourced elsewhere.

user_1054558 · Answer

I use a Consistency level of All to ensure that I'm scanning all of the nodes on the cluster for the data

So consistency ALL has more to do with the number of data replicas read than it does with the number of nodes contacted. If you have a replication factor (RF) of 3 and query a single row at ALL, then Cassandra will hash your partition key to figure out the three nodes responsible for that row, contact all 3 nodes, and wait for all 3 to respond.

I just need to see a record on one node

So I think you'd be fine with LOCAL_ONE, in this regard.

The only possible advantage of using ALL, is that it actually does help to enforce data consistency by triggering a read repair 100% of the time. So if eventual consistency is a concern, that's a "plus." But *_ONE is definitely faster.

The CL documentation talks a lot about 'stale data', but I am interested in 'new data'

In your case, I don't see stale data as a possibility, so you should be ok there. The issue that you would face instead, is in the event that one or more replicas failed during the write operation, querying at LOCAL_ONE may or may not get you the only replica that actually exists. So your data wouldn't be stale vs. new, it'd be exists vs. does not exist. One point I talk about in the linked answer, is that perhaps writing at a higher consistency level and reading at LOCAL_ONE might work for your use case.

A few years ago, I wrote an answer about the different consistency levels, which you might find helpful in this case: If lower consistency level is good then why we need to have a higher consistency(QUORUM,ALL) level in Cassandra?

Proper Consistency Level to read 'everything'

1 Answers