RocksDB: range query on numbers

collimarco 0
Viewed 45

Is it possible to use RocksDB efficiently for range queries on numbers?

For example if I have billions of tuples (price, product_id) can I use RocksDB to retrieve all products that have 10 <= price <= 100? Or it can't be used for that?

I am confused because I can't find any specific docs about number keys and range queries. However I also read that RocksDB is used as a database engine for many DBMS and that suggests that it's possible to query it efficiently for this case.

What is the recommended way to organize the above tuples in a key-value store like RocksDB in order to get arbitrary ranges (not known in advance)?

What kind of keys would you use? What type of queries would you use?

key-value-store rocksdb
1 Answers

Yes, rocksdb supports efficient range queries [even for arbitrary ranges that are not known in advance]

range queries.

https://github.com/facebook/rocksdb/wiki/Prefix-Seek

number keys

There are no docs on how to model your data like that - if you don't know how to model that already you shouldn't be using rocksdb in the first place as it is too low level

What is the recommended way to organize the above tuples in a key-value store like RocksDB in order to get arbitrary ranges (not known in advance)?

In your example - it is creating an index on price to lookup the product id

So you would encode the price as a byte array and use that as the key and then the product id as a byte array as the value

Example format

key => value
priceIndex:<price>#<productId> => <productId>

Then you will

  1. Create an iterator
  2. Seek to the lower bound of your price [priceIndex:10 in this case]
  3. Set upper bound on the options [priceIndex:100 in this case]
  4. Loop over until iterator is valid

This will give you all the key value pairs that are in the range - which in your case would be all the price, product id tuples that are within the price range

Care must be taken since many products can have the same price and rocksdb keys are unique - so you can suffix the price with the product id as well to make the key unique

Asad Awadia0
Related