Write-through cache Redis

user_917467 0
Viewed 5311

I have been poundering on how to reliably implement a write-through caching mechanism to store realtime data.

Basically what we need is this: 

Save data to Redis -> Save to database (underlying)  
Read data from Redis <- Read from database in case unavailable in cache

The resources online to help in the implementation of this caching strategy seem scarce.

The problem is:

1) No built-in transaction possibility between Redis and the database (Mongo in my case).
2) No transactions mean that writes to the underlying database are unreliable.

The most straightforward way I see how this can be implemented is by using a broker like Kafka and putting messages on a persistent queue to be processed later.
Therefore Kafka would be the responsible entity for reliable processing.

Another way would be by having a custom implementation in a scheduler that checks the Redis database for dirty records. On first thought there seem to be some tradeoffs to this approach and I would like not having to go this road if possible.

I am looking on some options on how this can be implemented otherwise.
Or whether this is in fact the most viable approach.

apache-kafka redis caching java
2 Answers

So better approach than is as u mentioned above is to use kafka and consumer which will store data to mongo. But read about it delivery guarantee, as i remember exactly once is guaranteed in kafka streams only (between two topics), in your case your database should be idempotent because u get at least once guarantee. And don't forget to turn AOF on with Redis, not to loose data. And don't forget that in this case u get eventual consistency in db with all the consequences.

user_80141450

On review I will use MongoDB as a single datastore without Redis at all.
Premature optimization is evil I guess.

Anyhow, I can add additional architecture afterwards after benchmarking.
Plans to refactor towards a cache shouldn't be too hard.

Scaling is additional concern so I shouldn't be bothered with that during development right now.

Accepted @Ipave answer, going with a single datastore for the moment.

user_9174670
Related