Replace Cassandra seed node after hardware failure on same machine?

patrick_73804 0
Viewed 109

I have two VMs, each hosting a Cassandra seed node. I wipe one VM to simulate a hardware failure. I reinstall Cassandra and add replace_address_first_boot the end of the cassandra-env.sh file (with my real ipaddress)

 JVM_OPTS="$JVM_OPTS -Dcassandra.replace_address_first_boot=1.1.1.1"

nodetool status on the other machine lists this machine as DN

cassandra.yaml displays the default value of auto_bootstrap: true

Yet, Cassandra startup fails with:

ERROR [main] 2022-02-24 01:43:36,546 CassandraDaemon.java:803 - Exception encountered during startup
java.lang.RuntimeException: Replacing a node without bootstrapping risks invalidating consistency guarantees as the expected data may not be present until repair is run. To perform this operation, please restart with -Dcassandra.allow_unsafe_replace=true

How can I bootstrap the new seed machine even though it has the same ipaddress as the old seed machine without using an unsafe flag?

cassandra-3.0 cassandra
1 Answers

Unfortunately, seed nodes don't stream upon joining. My advice would be to remove its IP from the seeds in the cassandra.yaml. Then upon joining, it should be fine.

If you still want it to be a seed node, you can always edit its seeds property with its own IP and restart once bootstrapping is complete.

Otherwise, if you can get the replacement node to start (without bootstrapping), you can always run a nodetool repair to stream the data. The drawback here though, is that the node will try to serve requests during this process, so you may want to run nodetool disablebinary on it to prevent that (and of course re-enable it when done).

user_10545580
Related