I am currently designing redis-based cache (first time using redis) and need an advice.
Requirement:
- It is required to cache limited number of business entities in redis. For example: Products, Customers
- Each entity is stored in a several tables inside relational database. For example: ProductsTableA, ProductsTableB, CustomersTableA, CustomersTableB, CustomersTableC, CommonTableA, CommonTableB
- Not every table above contains an entity key. One table can store data for several entities (for example CommonTableA is used to populate both Product and Customer entities).
- Fairly complex business logic is required to build an entity.
- Cache is populated on-demand (if not found in redis -> query from db -> run business logic -> populate redis -> return to client)
- Data is mainly read-only
- Once in several hours/days periodic jobs runs that populates/updates db with data imported from external system (can be large volume).
Task: How to do effective cache eviction when new set of data is uploaded to db (see item 7. above)?
Issues/complications: Due to complexity of business logic/db schema, and amount of data updated at once I cannot map every updated/inserted/deleted record to cache key and do key-by-key eviction. On the other hand flushing entire redis db is not optimal as well.
Proposed approach: I picked single entity type as a "unit of eviction" if data in at least one of underlying tables is changed. For example if one or more rows in ProductsTableA were updated all redis keys like "product:XXX" should be removed.
Questions:
- Is there a way to add metadata/label to key/value pair in Redis? In my case I want to tag cache entries with table names so later and can bulk-expire all values that are related to specific tables.
- If answer to 1. is "no" I have an option of creating "metadata cache" in redis myself (using list?) where key will be table name and values are entity keys that related to this table. However this will require some mechanism to keep "main" and "metadata" caches in sync so I'd rather avoid it.
- Is there a better way to solve this problem? :)
Thank you!