mongoose connection to Replica Sets always fails

user_9693461 0
Viewed 405

I am having some trouble wrapping head around how to connect to a replica set - I need to .watch() collection events, which requires a replica set - see: MongoDB watch() to observe change in Database with NodeJS and Mongoose ( I have followed this answer as well, does not work - see bottom of this question )

Here's my setup:

  • nodejs express api + mongoose
  • my main 'admin' mongo db ( named DEV )
  • 3 mongo nodes for replica setup

I have used this snippet to setup the 3 mongo nodes with docker-compose:

version: "3"

services:
  mongo0:
    hostname: mongo0
    container_name: mongo0
    image: mongo:latest
    expose:
      - 30000
    ports:
      - 30000:30000
    volumes:
      - mongodb_repl_data1:/data/db
    restart: always
    command: "--bind_ip_all --replSet rs0 --port 30000"
  mongo1:
    hostname: mongo1
    container_name: mongo1
    image: mongo:latest
    expose:
      - 30001
    ports:
      - 30001:30001
    volumes:
      - mongodb_repl_data2:/data/db
    restart: always
    command: "--bind_ip_all --replSet rs0 --port 30001"
  mongo2:
    hostname: mongo2
    container_name: mongo2
    image: mongo:latest
    expose:
      - 30002
    ports:
      - 30002:30002
    volumes:
      - mongodb_repl_data3:/data/db
    restart: always
    command: "--bind_ip_all --replSet rs0 --port 30002"

volumes:
  mongodb_repl_data1:
  mongodb_repl_data2:
  mongodb_repl_data3:

which runs fine and I can docker exec -it <container> mongo --port <port> to enter the mongo shell on all of the containers.

I have my main 'admin' mongo db locally at port 27017.

I am trying to connect to them like this:

await mongoose.connect("mongodb://localhost:30000,localhost:30001,localhost:30002,localhost:27017/DEV?replicaSet=rs0", {
   useNewUrlParser: true,
   useUnifiedTopology: true
})

When I run, it hangs on this connect(), then throwing:

Failed to inititate mongo connection:  MongooseServerSelectionError: getaddrinfo ENOTFOUND mongo0
    at NativeConnection.Connection.openUri (/Users/zac/Projects/score/app/application/server/node_modules/mongoose/lib/connection.js:821:32)
    at /Users/zac/Projects/score/app/application/server/node_modules/mongoose/lib/index.js:342:10
    at /Users/zac/Projects/score/app/application/server/node_modules/mongoose/lib/helpers/promiseOrCallback.js:31:5
    at new Promise (<anonymous>)
    at promiseOrCallback (/Users/zac/Projects/score/app/application/server/node_modules/mongoose/lib/helpers/promiseOrCallback.js:30:10)
    at Mongoose.connect (/Users/zac/Projects/score/app/application/server/node_modules/mongoose/lib/index.js:341:10)
    at Object.exports.default (/Users/zac/Projects/score/app/application/server/dist/submodules/connectDb.js:13:34)
    at start (/Users/zac/Projects/score/app/application/server/dist/index.js:31:34)
    at Object.<anonymous> (/Users/zac/Projects/score/app/application/server/dist/index.js:47:1)
    at Module._compile (internal/modules/cjs/loader.js:1075:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1096:10)
    at Module.load (internal/modules/cjs/loader.js:940:32)
    at Function.Module._load (internal/modules/cjs/loader.js:781:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at internal/main/run_main_module.js:17:47 {
  reason: TopologyDescription {
    type: 'ReplicaSetNoPrimary',
    setName: 'rs0',
    maxSetVersion: 1,
    maxElectionId: 7fffffff0000000000000001,
    servers: Map(3) {
      'mongo0:30000' => [ServerDescription],
      'mongo1:30001' => [ServerDescription],
      'mongo2:30002' => [ServerDescription]
    },
    stale: false,
    compatible: true,
    compatibilityError: null,
    logicalSessionTimeoutMinutes: null,
    heartbeatFrequencyMS: 10000,
    localThresholdMS: 15,
    commonWireVersion: 9
  }
}

in the mongo nodes, I did do this:

config={"_id":"rs0","members":[{"_id":0,"host":"mongo0:30000"},{"_id":1,"host":"mongo1:30001"},{"_id":2,"host":"mongo2:30002"}]}

rs.initiate(config);

which returns success:

{
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1609794399, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1609794399, 1)
}

and if I do rs.status():

rs0:SECONDARY> rs.status()
{
    "set" : "rs0",
    "date" : ISODate("2021-01-04T21:06:53.009Z"),
    "myState" : 1,
    "term" : NumberLong(1),
    "syncSourceHost" : "",
    "syncSourceId" : -1,
    "heartbeatIntervalMillis" : NumberLong(2000),
    "majorityVoteCount" : 2,
    "writeMajorityCount" : 2,
    "votingMembersCount" : 3,
    "writableVotingMembersCount" : 3,
    "optimes" : {
        "lastCommittedOpTime" : {
            "ts" : Timestamp(1609794412, 1),
            "t" : NumberLong(1)
        },
        "lastCommittedWallTime" : ISODate("2021-01-04T21:06:52.570Z"),
        "readConcernMajorityOpTime" : {
            "ts" : Timestamp(1609794412, 1),
            "t" : NumberLong(1)
        },
        "readConcernMajorityWallTime" : ISODate("2021-01-04T21:06:52.570Z"),
        "appliedOpTime" : {
            "ts" : Timestamp(1609794412, 1),
            "t" : NumberLong(1)
        },
        "durableOpTime" : {
            "ts" : Timestamp(1609794412, 1),
            "t" : NumberLong(1)
        },
        "lastAppliedWallTime" : ISODate("2021-01-04T21:06:52.570Z"),
        "lastDurableWallTime" : ISODate("2021-01-04T21:06:52.570Z")
    },
    "lastStableRecoveryTimestamp" : Timestamp(1609794411, 3),
    "electionCandidateMetrics" : {
        "lastElectionReason" : "electionTimeout",
        "lastElectionDate" : ISODate("2021-01-04T21:06:51.027Z"),
        "electionTerm" : NumberLong(1),
        "lastCommittedOpTimeAtElection" : {
            "ts" : Timestamp(0, 0),
            "t" : NumberLong(-1)
        },
        "lastSeenOpTimeAtElection" : {
            "ts" : Timestamp(1609794399, 1),
            "t" : NumberLong(-1)
        },
        "numVotesNeeded" : 2,
        "priorityAtElection" : 1,
        "electionTimeoutMillis" : NumberLong(10000),
        "numCatchUpOps" : NumberLong(0),
        "newTermStartDate" : ISODate("2021-01-04T21:06:51.051Z"),
        "wMajorityWriteAvailabilityDate" : ISODate("2021-01-04T21:06:52.545Z")
    },
    "members" : [
        {
            "_id" : 0,
            "name" : "mongo0:30000",
            "health" : 1,
            "state" : 1,
            "stateStr" : "PRIMARY",
            "uptime" : 1485,
            "optime" : {
                "ts" : Timestamp(1609794412, 1),
                "t" : NumberLong(1)
            },
            "optimeDate" : ISODate("2021-01-04T21:06:52Z"),
            "syncSourceHost" : "",
            "syncSourceId" : -1,
            "infoMessage" : "",
            "electionTime" : Timestamp(1609794411, 1),
            "electionDate" : ISODate("2021-01-04T21:06:51Z"),
            "configVersion" : 1,
            "configTerm" : 1,
            "self" : true,
            "lastHeartbeatMessage" : ""
        },
        {
            "_id" : 1,
            "name" : "mongo1:30001",
            "health" : 1,
            "state" : 2,
            "stateStr" : "SECONDARY",
            "uptime" : 13,
            "optime" : {
                "ts" : Timestamp(1609794399, 1),
                "t" : NumberLong(-1)
            },
            "optimeDurable" : {
                "ts" : Timestamp(1609794399, 1),
                "t" : NumberLong(-1)
            },
            "optimeDate" : ISODate("2021-01-04T21:06:39Z"),
            "optimeDurableDate" : ISODate("2021-01-04T21:06:39Z"),
            "lastHeartbeat" : ISODate("2021-01-04T21:06:51.038Z"),
            "lastHeartbeatRecv" : ISODate("2021-01-04T21:06:52.549Z"),
            "pingMs" : NumberLong(0),
            "lastHeartbeatMessage" : "",
            "syncSourceHost" : "",
            "syncSourceId" : -1,
            "infoMessage" : "",
            "configVersion" : 1,
            "configTerm" : 0
        },
        {
            "_id" : 2,
            "name" : "mongo2:30002",
            "health" : 1,
            "state" : 2,
            "stateStr" : "SECONDARY",
            "uptime" : 13,
            "optime" : {
                "ts" : Timestamp(1609794399, 1),
                "t" : NumberLong(-1)
            },
            "optimeDurable" : {
                "ts" : Timestamp(1609794399, 1),
                "t" : NumberLong(-1)
            },
            "optimeDate" : ISODate("2021-01-04T21:06:39Z"),
            "optimeDurableDate" : ISODate("2021-01-04T21:06:39Z"),
            "lastHeartbeat" : ISODate("2021-01-04T21:06:51.040Z"),
            "lastHeartbeatRecv" : ISODate("2021-01-04T21:06:52.551Z"),
            "pingMs" : NumberLong(0),
            "lastHeartbeatMessage" : "",
            "syncSourceHost" : "",
            "syncSourceId" : -1,
            "infoMessage" : "",
            "configVersion" : 1,
            "configTerm" : 0
        }
    ],
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1609794412, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1609794412, 1)
}

Without Docker, I have tried to use mongodb-topology-manager like so: this was provided in: MongoDB watch() to observe change in Database with NodeJS and Mongoose

import { ReplSet } from 'mongodb-topology-manager';

export default async () => {
    // Starts a 3-node replica set on ports 31000, 31001, 31002, replica set
    // name is "rs0".
    const replSet = new ReplSet('mongod', [{ options: { port: 31000, dbpath: `${__dirname}/data/db/31000`, bind_ip: 'localhost' } },
        { options: { port: 31000, dbpath: `${__dirname}/data/db/31001`, bind_ip: 'localhost' } },
        { options: { port: 31001, dbpath: `${__dirname}/data/db/31001`, bind_ip: 'localhost' } },
        { options: { port: 31002, dbpath: `${__dirname}/data/db/31002`, bind_ip: 'localhost' } }
    ], { replSet: 'rs0' });

    try {
        console.log("rep set discovering...");
        await replSet.discover()

        console.log("rep set purging...");
        await replSet.purge();

        console.log("rep set starting...");
        await replSet.start();

        console.log("rep set done.");
    } catch (err) {
        console.error("Failed to start replica sets:", err);
    }
}

but if I run this, it always hangs on replSet.start() and never does anything, just hangs forever. The db paths exist and the ports are not used else where.

rep set discovering...
rep set purging...
rep set starting...

Simply put, how do I correctly start a mongodb repl set and connect to it via mongoose? If I need to have the repl sets locally ( not in Docker ), how do I do that with my one local mongodb instance ( so I can access them with just command $ mongo or $ mongod, not through docker )?

Any help will be greatly appreciated

replicaset mongoose docker node.js mongodb
2 Answers

I had the same issue and has been trying to find a solution for the past week. But the solution I have found is very trivial.

config={"_id":"rs0","members":[{"_id":0,"host":"mongo0:30000"},{"_id":1,"host":"mongo1:30001"},{"_id":2,"host":"mongo2:30002"}]}

I have changed the "mongo0", etc. in the host to my internal IP (ie. 192.168.1.5 and so on). It seems to have solved the issue.

user_150893430

There is a way to automate the configuration by deploying another docker just to execute a script whose apply that configuration. So I can create a single instance un replicaset mode with this files:

  • docker-compose.yml
version: "3"
services:
  mongo-setup:
    image: mongo
    restart: on-failure
    volumes:
      - ./scripts:/scripts
    entrypoint: [ "/scripts/setup-replicaset.sh" ]
    depends_on:
      - mongodb01
  mongodb01:
    image: "mongo"
    ports:
      - "27017:27017"
    volumes:
      - mongo-data:/data/db
    command: "--bind_ip_all --replSet rs0"
volumes:
 mongo-data:
    driver: local
  • script/setup-replicaset.sh
#!/bin/bash

MONGODB1=mongodb01

echo "**********************************************" ${MONGODB1}
echo "Waiting for startup.."
until curl http://${MONGODB1}:27017/serverStatus\?text\=1 2>&1 | grep uptime | head -1; do
  echo '.'
  sleep 1
done

echo SETUP.sh time now: `date +"%T" `
mongo --host ${MONGODB1}:27017 <<EOF
var cfg = {
    "_id": "rs0",
    "protocolVersion": 1,
    "version": 1,
    "members": [
        {
            "_id": 0,
            "host": "${MONGODB1}:27017",
            "priority": 2
        }
    ],settings: {chainingAllowed: true}
};
rs.initiate(cfg, { force: true });
rs.reconfig(cfg, { force: true });
rs.slaveOk();
db.getMongo().setReadPref('nearest');
db.getMongo().setSlaveOk();
EOF

After run this docker-compose you can connect with mongoose normally through localhost.

user_112996010
Related