What is current recommended max storage space for Cassandra 2021?

user_4082810 0
Viewed 276

After many years in development, there have been introduced many features for Cassandra NoSQL. However there is little info about hardware and architecture with real world examples.

On the internet (and on stackoverflow) there are some voices to keep storage below 1 TB. However almost every time, it is not explained is it 1TB "active disk space" (so with x3 replication it would be about 3TB total) or is it 1TB "raw disk space". Moreover it is not explained why you should not exceed such 1TB (why not to have 2TB or 5TB?).

Furthermore I wonder has it changed over last years? Is it possible to store 50+TB on single machine without big impact on performance. It is generally problematic to have multiple machines with small storage space (eg. 1TB per machine).

I wonder about 3 things:

  1. What is maximum possible storage space (hdd/ssd) for single Cassandra machine (server)?
  2. What is recommended storage space (hdd/ssd) for single Cassandra machine (server)?
  3. Is there any solution to store 50+ TB per single NoSQL machine (with/without Cassandra)?
nosql amazon-web-services storage cassandra
2 Answers

The answer heavily depends on your requirements - is it always-on, randomly-accessed data, or some kind of archive, where only portion of data is actively accessed, and much more. So it's hard to say without such information. You also need to take into account maintenance operations - how much time it will take to add a new node to cluster, or remove it - right now on 10Tb it may take 3-5 days (4.0 has improvements that help to decrease that time).

General recommendation is to have 1-2Tb per machine (size on disk, compressed, taking into account replication). Depending on the access patterns, and version of Cassandra (like, coming soon 4.0 that has a lot of improvements & optimizations), that could be increased to something like 2-4Tb/machine. But that needs to be tested carefully. I would recommend to use something like NoSQLBench to model your schema & test access patterns.

I have seen setups with 10-12Tb (my colleagues have seen even more) of data per machine (size on disk, compressed data), but in most cases that were systems when big part of the data was rarely accessed, and kept mostly for compliance.

Alex_Ott_186270

It depends on how well your access patterns align with your Compaction Strategy. For the most part, size-tiered and leveled compaction are general purpose compaction strategies and its difficult to align access patterns to them. They depend heavily on compaction process to optimize for reads.

You can achieve highest density with time-window compaction and time-series data since read and write access patterns are aligned to a timewindow (best case).

In either case, increasing density and lowering the amount of nodes reduces operational complexity, but increases operational event complexity. As nodes become dense its challenging to move data around. Adding new nodes, rebuilding nodes, or even backups can become high risk.

user_31203450
Related