What is the purpose of BloomFilter in Cassandra?

user_5804509 0
Viewed 41

In datastax documentation it states (link):

A structure stored in memory that checks if row data exists in the memtable before accessing SSTables on disk

In glossary it is mentioned(link):

An off-heap structure associated with each SSTable that checks if any data for the requested row exists in the SSTable before doing any disk I/O.

Bloom filter checks for data requested for read presence in SSTable or memtable or it checks memtable and then SSTable?

cassandra
2 Answers

When you read data from Cassandra, memtable is always checked. But SSTables are also checked because memtable may have old data (for example, because of the network delays, etc.). With memtable it's easy to check. But with SSTables it's more complex - there could be dozens of SSTables on the disk, and if we hit each of them, then read operations will be slow. And Bloom Filter helps here - it allows to avoid reading of SSTables where specific partition key doesn't exist.

You can check effect of the bloom filter on your table (here is an example):

  • figure out the current number of SSTables using the nodetool tablestats command
  • use nodetool tablehistograms to find what is the number of SSTables read on the typical operations (min/max/50%/95%/...)
Alex_Ott_186270

Just to echo Alex's answer, bloom filters come in handy when there are many SSTable files on disk for a single table on a single node. Reads from the disk are slow (relative to reads from memory), so the idea is that querying the bloom filters help to narrow the scope of which SSTable files actually need to be read.

Also, the bloom filter is a probabilistic structure, capable of providing one of two answers in the read path:

  • This SSTable file may contain the requested data.
  • This SSTable file definitely does not contain the requested data.
user_10545580
Related