Active -Active Cross-Region Redis Replication

user_4259125 0
Viewed 3205

AWS introduces Cross-Region Replication this year for an Active - Passive setup (Global DataStore). That means that there is a primary Redis-Cluster for Read/Write and a Secondary cluster for Reads.

In my case, we want to use Active - Active Redis in different regions. An initial thought is to manually synchronize the Redis clusters with Kinesis streams and Lambda functions get triggered whenever there is a write and update the other region, e.g. when a write happens in region A, a Lambda updates the Redis in Region B and vice versa. This allows to write in both regions.

Is it possible to get Active Active working with Global Datastore?

or would a Global Datastore work just as well as the manual solution (in terms of latency) when write operations always go to the same region where the primary Redis cluster is deployed??

Any feedback is welcome

aws-elasticache redis replication amazon-web-services
2 Answers

Unfortunately at this time there is no AWS Solution for master-master within ElastiCache.

Here are some of the solutions you could do:

  • Within your application use write endpoint and read endpoint for Redis, the write endpoint would target the primary write global datastore, the read would target the local read datastore
  • Write to a local region resource (such as DynamoDB), have Lambda trigger and write to the primary write global datastore.
  • Write to a local region queue/stream such as SQS or Kinesis Data Streams (which supports a Lambda consumer) and have Lambda consume and write.

Any of the solutions will have a degree of latency between, option 1 does a direct write whereas the other 2 allow the process to happen in the background. The decision here would be based on your application (can it wait for the write to be done, or dos it need to happen in the flow).

Additionally if this is for writing a cache, could you just use cache priming instead? By this I mean generating a flattened cache via a script to ensure your redis cluster is always up to date.

user_134609330

How about connecting the different regions in the same Transit VPC so that you can access the read/write endpoint from any (connected) region?

https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/elasticache-vpc-accessing.html#elasticache-vpc-accessing-different-vpc-different-region

Not tested yet, I am currently trying to implement this, but the corresponding CloudFormation template from AWS Console is broken.

However, the solution should still do the trick, of course with an accepted lag, but at least there should not be additional work to maintain the local region resource, Lambda, etc.

How does this sound?

user_149901830
Related