I am running Redis on docker with 1 master, 2 replica and 3 sentinels. Once I stop the master one of the replica is elected and the setup works fine. But when I start the master again it doesn't sync and remains with the old data. It is simply in useless idle state.
docker-compose for redis:
redis-cache:
image: "bitnami/redis"
container_name: redis-cache
environment:
- REDIS_REPLICATION_MODE=master
- ALLOW_EMPTY_PASSWORD=yes
ports:
- "6379"
redis-replica:
image: "bitnami/redis"
environment:
- REDIS_REPLICATION_MODE=replica
- REDIS_MASTER_HOST=redis-cache
- ALLOW_EMPTY_PASSWORD=yes
ports:
- "6379"
depends_on:
- redis-master
redis-sentinel:
image: "bitnami/redis-sentinel"
environment:
- REDIS_MASTER_SET=mymaster
- REDIS_MASTER_HOST=redis-cache
- ALLOW_EMPTY_PASSWORD=yes
- REDIS_SENTINEL_DOWN_AFTER_MILLISECONDS=1000
- REDIS_SENTINEL_FAILOVER_TIMEOUT=1000
depends_on:
- redis-cache
- redis-replica
ports:
- "26379"
Sentinel log:
redis-sentinel 15:56:27.89
redis-sentinel 15:56:27.89 Welcome to the Bitnami redis-sentinel container
redis-sentinel 15:56:27.90 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-redis-sentinel
redis-sentinel 15:56:27.90 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-redis-sentinel/issues
redis-sentinel 15:56:27.90
redis-sentinel 15:56:27.90 INFO ==> ** Starting Redis sentinel setup **
redis-sentinel 15:56:27.99 INFO ==> Initializing Redis Sentinel...
redis-sentinel 15:56:27.99 INFO ==> Configuring Redis Sentinel...
redis-sentinel 15:56:28.04 INFO ==> ** Redis sentinel setup finished! **
redis-sentinel 15:56:28.05 INFO ==> ** Starting Redis Sentinel **
1:X 08 Feb 2022 15:56:28.063 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:X 08 Feb 2022 15:56:28.063 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=1, just started
1:X 08 Feb 2022 15:56:28.063 # Configuration loaded
1:X 08 Feb 2022 15:56:28.064 * monotonic clock: POSIX clock_gettime
1:X 08 Feb 2022 15:56:28.064 * Running mode=sentinel, port=26379.
1:X 08 Feb 2022 15:56:28.081 # Sentinel ID is 69530661f89cbd9e23b4b426e64ea443b50d1491
1:X 08 Feb 2022 15:56:28.081 # +monitor master mymaster 172.28.0.3 6379 quorum 2
1:X 08 Feb 2022 15:56:28.082 * +slave slave 172.28.0.4:6379 172.28.0.4 6379 @ mymaster 172.28.0.3 6379
1:X 08 Feb 2022 15:56:28.094 * +slave slave 172.28.0.5:6379 172.28.0.5 6379 @ mymaster 172.28.0.3 6379
1:X 08 Feb 2022 15:56:30.233 * +sentinel sentinel 61b105c9d928e63afe35f537e7622a4177c46675 172.28.0.8 26379 @ mymaster 172.28.0.3 6379
1:X 08 Feb 2022 15:56:30.642 * +sentinel sentinel 5581bab954496d8c41b8cbf34fd8aa9761b839ec 172.28.0.9 26379 @ mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:03:41.802 # +sdown master mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:03:41.848 # +new-epoch 1
1:X 08 Feb 2022 16:03:41.857 # +vote-for-leader 5581bab954496d8c41b8cbf34fd8aa9761b839ec 1
1:X 08 Feb 2022 16:03:41.861 # +odown master mymaster 172.28.0.3 6379 #quorum 3/2
1:X 08 Feb 2022 16:03:41.861 # Next failover delay: I will not start a failover before Tue Feb 8 16:03:44 2022
1:X 08 Feb 2022 16:03:43.034 # +sdown slave 172.28.0.4:6379 172.28.0.4 6379 @ mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:03:43.034 # +sdown slave 172.28.0.5:6379 172.28.0.5 6379 @ mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:03:44.286 # +new-epoch 2
1:X 08 Feb 2022 16:03:44.295 # +vote-for-leader 5581bab954496d8c41b8cbf34fd8aa9761b839ec 2
1:X 08 Feb 2022 16:03:44.301 # Next failover delay: I will not start a failover before Tue Feb 8 16:03:46 2022
1:X 08 Feb 2022 16:03:46.584 # +new-epoch 3
1:X 08 Feb 2022 16:03:46.595 # +vote-for-leader 61b105c9d928e63afe35f537e7622a4177c46675 3
1:X 08 Feb 2022 16:03:46.595 # Next failover delay: I will not start a failover before Tue Feb 8 16:03:48 2022
1:X 08 Feb 2022 16:03:48.838 # +new-epoch 4
1:X 08 Feb 2022 16:03:48.848 # +vote-for-leader 61b105c9d928e63afe35f537e7622a4177c46675 4
1:X 08 Feb 2022 16:03:48.887 # Next failover delay: I will not start a failover before Tue Feb 8 16:03:51 2022
1:X 08 Feb 2022 16:03:51.060 # +new-epoch 5
1:X 08 Feb 2022 16:03:51.060 # +try-failover master mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:03:51.071 # +vote-for-leader 69530661f89cbd9e23b4b426e64ea443b50d1491 5
1:X 08 Feb 2022 16:03:51.102 # 61b105c9d928e63afe35f537e7622a4177c46675 voted for 69530661f89cbd9e23b4b426e64ea443b50d1491 5
1:X 08 Feb 2022 16:03:51.109 # 5581bab954496d8c41b8cbf34fd8aa9761b839ec voted for 69530661f89cbd9e23b4b426e64ea443b50d1491 5
1:X 08 Feb 2022 16:03:51.142 # +elected-leader master mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:03:51.142 # +failover-state-select-slave master mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:03:51.205 # -failover-abort-no-good-slave master mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:03:51.296 # Next failover delay: I will not start a failover before Tue Feb 8 16:03:53 2022
1:X 08 Feb 2022 16:03:51.296 # -sdown slave 172.28.0.4:6379 172.28.0.4 6379 @ mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:04:12.829 # +tilt #tilt mode entered
1:X 08 Feb 2022 16:04:31.249 # +new-epoch 6
1:X 08 Feb 2022 16:04:31.258 # +vote-for-leader 5581bab954496d8c41b8cbf34fd8aa9761b839ec 6
1:X 08 Feb 2022 16:04:36.404 # +config-update-from sentinel 5581bab954496d8c41b8cbf34fd8aa9761b839ec 172.28.0.9 26379 @ mymaster 172.28.0.3 6379
1:X 08 Feb 2022 16:04:36.404 # +switch-master mymaster 172.28.0.3 6379 172.28.0.4 6379
1:X 08 Feb 2022 16:04:36.404 * +slave slave 172.28.0.5:6379 172.28.0.5 6379 @ mymaster 172.28.0.4 6379
1:X 08 Feb 2022 16:04:46.427 # +tilt #tilt mode entered
1:X 08 Feb 2022 16:05:16.474 # -tilt #tilt mode exited
1:X 08 Feb 2022 17:26:52.607 # +tilt #tilt mode entered
1:X 09 Feb 2022 02:02:42.355 # +tilt #tilt mode entered
1:X 09 Feb 2022 02:03:04.936 # +tilt #tilt mode entered
1:X 09 Feb 2022 04:47:32.477 # +tilt #tilt mode entered
1:X 09 Feb 2022 04:48:02.510 # -tilt #tilt mode exited
I am not able to figure out what configuration is missing.