I would like to know the cause for this error. org.apache.ignite.spi.IgniteSpiException: Node with the same ID was found in node IDs history

Viewed 3661

I am trying to use apache ignite for clustering. But, there are some errors.

I tried to fix issue for long time. but, i couldn't.

I installed v2.8.1 rpm package on two centos 7 box and use default config. I will thank for your help.

IGNITE_HOME is /usr/share/apache-ignite

config file: ${IGNITE_HOME}/config/default-config.xml:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd">
    <bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration"/>
</beans>


<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd">
    <bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="cacheConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="session-cache"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="backups" value="1"/>
                    <property name="onheapCacheEnabled" value="true"/>
                    <property name="evictionPolicy">
                    <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
                        <property name="maxSize" value="10000"/>
                    </bean>
                    </property>
                </bean>
            </list>
        </property>

        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                <value>192.168.146.181:47500..47509</value>
                                <value>192.168.146.182:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

logs:

[15:47:08,140][WARNING][main][G] Ignite work directory is not provided, automatically resolved to: /usr/share/apache-ignite/work
[15:47:08,343][INFO][main][IgniteKernal]

>>>    __________  ________________
>>>   /  _/ ___/ |/ /  _/_  __/ __/
>>>  _/ // (7 7    // /  / / / _/
>>> /___/\___/_/|_/___/ /_/ /___/
>>>
>>> ver. 2.8.1#20200521-sha1:86422096
>>> 2020 Copyright(C) Apache Software Foundation
>>>
>>> Ignite documentation: http://ignite.apache.org

[15:47:08,354][INFO][main][IgniteKernal] Config URL: file:/usr/share/apache-ignite/config/default-config.xml
[15:47:08,384][INFO][main][IgniteKernal] IgniteConfiguration [igniteInstanceName=null, pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8, stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, igfsPoolSize=4, dataStreamerPoolSize=8, utilityCachePoolSize=8, utilityCacheKeepAliveTime=60000, p2pPoolSize=2, qryPoolSize=8, sqlQryHistSize=1000, dfltQryTimeout=0, igniteHome=/usr/share/apache-ignite, igniteWorkDir=/usr/share/apache-ignite/work, mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@6f94fa3e, nodeId=e37eb335-ecaf-4550-8c68-80407b9befc6, marsh=BinaryMarshaller [], marshLocJobs=false, daemon=false, p2pEnabled=false, netTimeout=5000, netCompressionLevel=1, sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=10000, metricsUpdateFreq=2000, metricsExpTime=9223372036854775807, discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0, marsh=null, reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false], segPlc=STOP, segResolveAttempts=2, waitForSegOnStart=true, allResolversPassReq=true, segChkFreq=10000, commSpi=TcpCommunicationSpi [connectGate=null, connPlc=org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$FirstConnectionPolicy@5ffead27, chConnPlc=null, enableForcibleNodeKill=false, enableTroubleshootingLog=false, locAddr=null, locHost=null, locPort=47100, locPortRange=100, shmemPort=-1, directBuf=true, directSndBuf=false, idleConnTimeout=600000, connTimeout=5000, maxConnTimeout=600000, reconCnt=10, sockSndBuf=32768, sockRcvBuf=32768, msgQueueLimit=0, slowClientQueueLimit=0, nioSrvr=null, shmemSrv=null, usePairedConnections=false, connectionsPerNode=1, tcpNoDelay=true, filterReachableAddresses=false, ackSndThreshold=32, unackedMsgsBufSize=0, sockWriteTimeout=2000, boundTcpPort=-1, boundTcpShmemPort=-1, selectorsCnt=4, selectorSpins=0, addrRslvr=null, ctxInitLatch=java.util.concurrent.CountDownLatch@6356695f[Count = 1], stopping=false, metricsLsnr=null], evtSpi=org.apache.ignite.spi.eventstorage.NoopEventStorageSpi@4f18837a, colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [], indexingSpi=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@3e58a80e, addrRslvr=null, encryptionSpi=org.apache.ignite.spi.encryption.noop.NoopEncryptionSpi@4fb61f4a, clientMode=false, rebalanceThreadPoolSize=4, rebalanceTimeout=10000, rebalanceBatchesPrefetchCnt=3, rebalanceThrottle=0, rebalanceBatchSize=524288, txCfg=TransactionConfiguration [txSerEnabled=false, dfltIsolation=REPEATABLE_READ, dfltConcurrency=PESSIMISTIC, dfltTxTimeout=0, txTimeoutOnPartitionMapExchange=0, deadlockTimeout=10000, pessimisticTxLogSize=0, pessimisticTxLogLinger=10000, tmLookupClsName=null, txManagerFactory=null, useJtaSync=false], cacheSanityCheckEnabled=true, discoStartupDelay=60000, deployMode=SHARED, p2pMissedCacheSize=100, locHost=null, timeSrvPortBase=31100, timeSrvPortRange=100, failureDetectionTimeout=10000, sysWorkerBlockedTimeout=null, clientFailureDetectionTimeout=30000, metricsLogFreq=60000, hadoopCfg=null, connectorCfg=ConnectorConfiguration [jettyPath=null, host=null, port=11211, noDelay=true, directBuf=false, sndBufSize=32768, rcvBufSize=32768, idleQryCurTimeout=600000, idleQryCurCheckFreq=60000, sndQueueLimit=0, selectorCnt=4, idleTimeout=7000, sslEnabled=false, sslClientAuth=false, sslCtxFactory=null, sslFactory=null, portRange=100, threadPoolSize=8, msgInterceptor=null], odbcCfg=null, warmupClos=null, atomicCfg=AtomicConfiguration [seqReserveSize=1000, cacheMode=PARTITIONED, backups=1, aff=null, grpName=null], classLdr=null, sslCtxFactory=null, platformCfg=null, binaryCfg=null, memCfg=null, pstCfg=null, dsCfg=DataStorageConfiguration [sysRegionInitSize=41943040, sysRegionMaxSize=104857600, pageSize=0, concLvl=0, dfltDataRegConf=DataRegionConfiguration [name=default, maxSize=377496371, initSize=268435456, swapPath=null, pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100, metricsEnabled=false, metricsSubIntervalCount=5, metricsRateTimeInterval=60000, persistenceEnabled=false, checkpointPageBufSize=0, lazyMemoryAllocation=true], dataRegions=null, storagePath=null, checkpointFreq=180000, lockWaitTime=10000, checkpointThreads=4, checkpointWriteOrder=SEQUENTIAL, walHistSize=20, maxWalArchiveSize=1073741824, walSegments=10, walSegmentSize=67108864, walPath=db/wal, walArchivePath=db/wal/archive, metricsEnabled=false, walMode=LOG_ONLY, walTlbSize=131072, walBuffSize=0, walFlushFreq=2000, walFsyncDelay=1000, walRecordIterBuffSize=67108864, alwaysWriteFullPages=false, fileIOFactory=org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory@5158b42f, metricsSubIntervalCnt=5, metricsRateTimeInterval=60000, walAutoArchiveAfterInactivity=-1, writeThrottlingEnabled=false, walCompactionEnabled=false, walCompactionLevel=1, checkpointReadLockTimeout=null, walPageCompression=DISABLED, walPageCompressionLevel=null], activeOnStart=true, autoActivation=true, longQryWarnTimeout=3000, sqlConnCfg=null, cliConnCfg=ClientConnectorConfiguration [host=null, port=10800, portRange=100, sockSndBufSize=0, sockRcvBufSize=0, tcpNoDelay=true, maxOpenCursorsPerConn=128, threadPoolSize=8, idleTimeout=0, handshakeTimeout=10000, jdbcEnabled=true, odbcEnabled=true, thinCliEnabled=true, sslEnabled=false, useIgniteSslCtxFactory=true, sslClientAuth=false, sslCtxFactory=null, thinCliCfg=ThinClientConfiguration [maxActiveTxPerConn=100]], mvccVacuumThreadCnt=2, mvccVacuumFreq=5000, authEnabled=false, failureHnd=null, commFailureRslvr=null]
[15:47:08,384][INFO][main][IgniteKernal] Daemon mode: off
[15:47:08,385][INFO][main][IgniteKernal] OS: Linux 3.10.0-957.el7.x86_64 amd64
[15:47:08,385][INFO][main][IgniteKernal] OS user: ignite
[15:47:28,407][INFO][main][IgniteKernal] PID: 11238
[15:47:28,407][INFO][main][IgniteKernal] Language runtime: Java Platform API Specification ver. 1.8
[15:47:28,407][INFO][main][IgniteKernal] VM information: OpenJDK Runtime Environment 1.8.0_222-ea-b03 Oracle Corporation OpenJDK 64-Bit Server VM 25.222-b03
[15:47:28,409][INFO][main][IgniteKernal] VM total memory: 0.39GB
[15:47:28,409][INFO][main][IgniteKernal] Remote Management [restart: on, REST: on, JMX (remote: on, port: 49134, auth: off, ssl: off)]
[15:47:28,409][INFO][main][IgniteKernal] Logger: JavaLogger [quiet=true, config=null]
[15:47:28,409][INFO][main][IgniteKernal] IGNITE_HOME=/usr/share/apache-ignite
[15:47:28,409][INFO][main][IgniteKernal] VM arguments: [-XX:+AggressiveOpts, -Dfile.encoding=UTF-8, -DIGNITE_QUIET=true, -DIGNITE_SUCCESS_FILE=/usr/share/apache-ignite/work/ignite_success_68f2fe3c-b44e-47de-8cc2-a7ce03f478b2, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.port=49134, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -DIGNITE_HOME=/usr/share/apache-ignite, -DIGNITE_PROG_NAME=./bin/ignite.sh]
IGNITE_HOME=/usr/share/apache-ignite, -DIGNITE_PROG_NAME=./bin/ignite.sh]
[15:47:28,409][INFO][main][IgniteKernal] System cache's DataRegion size is configured to 40 MB. Use DataStorageConfiguration.systemRegionInitialSize property to change the setting.
[15:47:28,410][INFO][main][IgniteKernal] Configured caches [in 'sysMemPlc' dataRegion: ['ignite-sys-cache']]
[15:47:28,410][WARNING][main][IgniteKernal] Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed environments.
[15:47:28,413][INFO][main][IgniteKernal] 3-rd party licenses can be found at: /usr/share/apache-ignite/libs/licenses
[15:47:28,414][WARNING][pub-#22][GridDiagnostic] Initial heap size is 30MB (should be no less than 512MB, use -Xms512m -Xmx512m).
[15:47:28,503][INFO][main][IgnitePluginProcessor] Configured plugins:
[15:47:28,503][INFO][main][IgnitePluginProcessor]   ^-- None
[15:47:28,503][INFO][main][IgnitePluginProcessor]
[15:47:28,508][INFO][main][FailureProcessor] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]]
[15:47:28,813][INFO][main][TcpCommunicationSpi] Successfully bound communication NIO server to TCP port [port=47100, locHost=0.0.0.0/0.0.0.0, selectorsCnt=4, selectorSpins=0, pairedConn=false]
[15:47:48,839][WARNING][main][TcpCommunicationSpi] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.
[15:47:48,860][WARNING][main][NoopCheckpointSpi] Checkpoints are disabled (to enable configure any GridCheckpointSpi implementation)
[15:47:48,882][WARNING][main][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival).
[15:47:48,887][INFO][main][IgniteKernal] Security status [authentication=off, tls/ssl=off]
[15:47:49,058][INFO][main][TcpDiscoverySpi] Successfully bound to TCP port [port=47500, localHost=0.0.0.0/0.0.0.0, locNodeId=e37eb335-ecaf-4550-8c68-80407b9befc6]
[15:47:49,079][WARNING][main][IgniteCacheDatabaseSharedManager] DataRegionConfiguration.maxWalArchiveSize instead DataRegionConfiguration.walHistorySize would be used for removing old archive wal files
[15:47:49,089][INFO][main][IgniteCacheDatabaseSharedManager] Configured data regions initialized successfully [total=3]
[15:47:49,100][INFO][main][PartitionsEvictManager] Evict partition permits=2
[15:47:49,154][WARNING][main][IgniteH2Indexing] Serialization of Java objects in H2 was enabled.
[15:47:49,342][INFO][main][ClientListenerProcessor] Client connector processor has started on TCP port 10800
[15:47:49,395][INFO][main][GridTcpRestProtocol] Command protocol successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211]
[15:47:49,445][INFO][main][IgniteKernal] Non-loopback local IPs: 192.168.146.182, fe80:0:0:0:d874:3cb2:fe4b:3309%eth0
[15:47:49,445][INFO][main][IgniteKernal] Enabled local MACs: 00155DE03C34
[15:48:09,496][INFO][main][TcpDiscoverySpi] Connection check threshold is calculated: 10000
[15:48:09,501][WARNING][main][TcpDiscoveryMulticastIpFinder] TcpDiscoveryMulticastIpFinder has no pre-configured addresses (it is recommended in production to specify at least one address in TcpDiscoveryMulticastIpFinder.getAddresses() configuration property)
[15:48:20,739][WARNING][main][TcpDiscoverySpi] Timed out waiting for message delivery receipt (most probably, the reason is in long GC pauses on remote node; consider tuning GC and increasing 'ackTimeout' configuration property). Will retry to send message with increased timeout [currentTimeout=9959, rmtAddr=vm-pri/192.168.146.181:47500, rmtPort=47500]
[15:48:25,783][WARNING][main][TcpDiscoverySpi] Node has not been connected to topology and will repeat join process. Check remote nodes logs for possible error messages. Note that large topology may require significant time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration property if getting this message on the starting nodes [networkTimeout=5000]
[15:48:30,805][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/192.168.146.181, rmtPort=60741]
[15:48:30,814][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.146.181, rmtPort=60741]
[15:48:30,815][INFO][tcp-disco-sock-reader-[]-#5][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/192.168.146.181:60741, rmtPort=60741]
[15:48:30,818][INFO][tcp-disco-sock-reader-[]-#5][TcpDiscoverySpi] Received ping request from the remote node [rmtNodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:60741, rmtPort=60741]
[15:48:30,819][INFO][tcp-disco-sock-reader-[]-#5][TcpDiscoverySpi] Finished writing ping response [rmtNodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:60741, rmtPort=60741]
[15:48:30,819][INFO][tcp-disco-sock-reader-[]-#5][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/192.168.146.181:60741, rmtPort=60741
[15:48:30,881][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/192.168.146.181, rmtPort=58817]
[15:48:30,881][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.146.181, rmtPort=58817]
[15:48:30,882][INFO][tcp-disco-sock-reader-[]-#6][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/192.168.146.181:58817, rmtPort=58817]
[15:48:30,884][INFO][tcp-disco-sock-reader-[0ab07b94 192.168.146.181:58817]-#6][TcpDiscoverySpi] Initialized connection with remote server node [nodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:58817]
[15:48:37,043][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/192.168.146.181, rmtPort=41854]
[15:48:37,043][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.146.181, rmtPort=41854]
[15:48:37,043][INFO][tcp-disco-sock-reader-[]-#7][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/192.168.146.181:41854, rmtPort=41854]
[15:48:37,044][INFO][tcp-disco-sock-reader-[]-#7][TcpDiscoverySpi] Received ping request from the remote node [rmtNodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:41854, rmtPort=41854]
[15:48:37,044][INFO][tcp-disco-sock-reader-[]-#7][TcpDiscoverySpi] Finished writing ping response [rmtNodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:41854, rmtPort=41854]
[15:48:37,044][INFO][tcp-disco-sock-reader-[]-#7][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/192.168.146.181:41854, rmtPort=41854
[15:48:50,918][INFO][tcp-disco-sock-reader-[0ab07b94 192.168.146.181:58817]-#6][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/192.168.146.181:58817, rmtPort=58817
[15:48:50,923][INFO][tcp-disco-msg-worker-[]-#2][TcpDiscoverySpi] New next node [newNext=TcpDiscoveryNode [id=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, consistentId=0:0:0:0:0:0:0:1%lo,127.0.0.1,192.168.146.181:47500, addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.146.181], sockAddrs=HashSet [/192.168.146.181:47500, /0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1591598910893, loc=false, ver=2.8.1#20200521-sha1:86422096, isClient=false]]
[15:48:53,266][WARNING][main][TcpDiscoverySpi] Timed out waiting for message delivery receipt (most probably, the reason is in long GC pauses on remote node; consider tuning GC and increasing 'ackTimeout' configuration property). Will retry to send message with increased timeout [currentTimeout=9997, rmtAddr=vm-pri/192.168.146.181:47500, rmtPort=47500]
[15:49:00,939][WARNING][tcp-disco-msg-worker-[]-#2][TcpDiscoverySpi] Timed out waiting for message delivery receipt (most probably, the reason is in long GC pauses on remote node; consider tuning GC and increasing 'ackTimeout' configuration property). Will retry to send message with increased timeout [currentTimeout=9999, rmtAddr=/192.168.146.181:47500, rmtPort=47500]
[15:49:00,982][WARNING][tcp-disco-msg-worker-[]-#2][TcpDiscoverySpi] Local node has detected failed nodes and started cluster-wide procedure. To speed up failure detection please see 'Failure Detection' section under javadoc for 'TcpDiscoverySpi'
[15:49:01,920][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/192.168.146.181, rmtPort=58961]
[15:49:01,920][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.146.181, rmtPort=58961]
[15:49:01,921][INFO][tcp-disco-sock-reader-[]-#8][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/192.168.146.181:58961, rmtPort=58961]
[15:49:01,922][INFO][tcp-disco-sock-reader-[]-#8][TcpDiscoverySpi] Received ping request from the remote node [rmtNodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:58961, rmtPort=58961]
[15:49:01,922][INFO][tcp-disco-sock-reader-[]-#8][TcpDiscoverySpi] Finished writing ping response [rmtNodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:58961, rmtPort=58961]
[15:49:01,922][INFO][tcp-disco-sock-reader-[]-#8][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/192.168.146.181:58961, rmtPort=58961
[15:49:03,278][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/192.168.146.181, rmtPort=38519]
[15:49:03,278][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.146.181, rmtPort=38519]
[15:49:03,278][INFO][tcp-disco-sock-reader-[]-#9][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/192.168.146.181:38519, rmtPort=38519]
[15:49:03,279][INFO][tcp-disco-sock-reader-[]-#9][TcpDiscoverySpi] Received ping request from the remote node [rmtNodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:38519, rmtPort=38519]
[15:49:03,279][INFO][tcp-disco-sock-reader-[]-#9][TcpDiscoverySpi] Finished writing ping response [rmtNodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:38519, rmtPort=38519]
[15:49:03,280][INFO][tcp-disco-sock-reader-[]-#9][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/192.168.146.181:38519, rmtPort=38519
[15:49:03,283][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/192.168.146.181, rmtPort=40757]
[15:49:03,283][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.146.181, rmtPort=40757]
[15:49:03,283][INFO][tcp-disco-sock-reader-[]-#10][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/192.168.146.181:40757, rmtPort=40757]
[15:49:03,285][INFO][tcp-disco-sock-reader-[0ab07b94 192.168.146.181:40757]-#10][TcpDiscoverySpi] Initialized connection with remote server node [nodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:40757]
[15:49:03,293][SEVERE][main][IgniteKernal] Failed to start manager: GridManagerAdapter [enabled=true, name=o.a.i.i.managers.discovery.GridDiscoveryManager]
class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@6f815e7f], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
        at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:302)
        at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943)
        at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960)
        at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2045)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
        at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
        at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1035)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:921)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:820)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:659)
        at org.apache.ignite.Ignition.start(Ignition.java:346)
        at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:300)
Caused by: class org.apache.ignite.spi.IgniteSpiException: Node with the same ID was found in node IDs history or existing node in topology has the same ID (fix configuration and restart local node) [localNode=TcpDiscoveryNode [id=e37eb335-ecaf-4550-8c68-80407b9befc6, consistentId=0:0:0:0:0:0:0:1%lo,127.0.0.1,192.168.146.182:47500, addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.146.182], sockAddrs=HashSet [/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, vm-sec/192.168.146.182:47500], discPort=47500, order=0, intOrder=2, lastExchangeTime=1591598889496, loc=true, ver=2.8.1#20200521-sha1:86422096, isClient=false], existingNode=e37eb335-ecaf-4550-8c68-80407b9befc6]
        at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.duplicateIdError(TcpDiscoverySpi.java:1975)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1112)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427)
        at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099)
        at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:299)
        ... 13 more
[15:49:03,296][SEVERE][main][IgniteKernal] Got exception while starting (will rollback startup routine).
class org.apache.ignite.IgniteCheckedException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
        at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1965)
        at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2045)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703)
        at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
        at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1035)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:921)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:820)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:659)
        at org.apache.ignite.Ignition.start(Ignition.java:346)
        at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:300)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@6f815e7f], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=5, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
        at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:302)
        at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943)
        at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960)
        ... 11 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Node with the same ID was found in node IDs history or existing node in topology has the same ID (fix configuration and restart local node) [localNode=TcpDiscoveryNode [id=e37eb335-ecaf-4550-8c68-80407b9befc6, consistentId=0:0:0:0:0:0:0:1%lo,127.0.0.1,192.168.146.182:47500, addrs=ArrayList [0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.146.182], sockAddrs=HashSet [/0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, vm-sec/192.168.146.182:47500], discPort=47500, order=0, intOrder=2, lastExchangeTime=1591598889496, loc=true, ver=2.8.1#20200521-sha1:86422096, isClient=false], existingNode=e37eb335-ecaf-4550-8c68-80407b9befc6]
        at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.duplicateIdError(TcpDiscoverySpi.java:1975)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1112)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427)
        at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099)
        at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:299)
        ... 13 more
[15:49:03,299][INFO][tcp-disco-sock-reader-[0ab07b94 192.168.146.181:40757]-#10][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/192.168.146.181:40757, rmtPort=40757
[15:49:03,303][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery accepted incoming connection [rmtAddr=/192.168.146.181, rmtPort=52412]
[15:49:03,304][INFO][tcp-disco-srvr-[:47500]-#3][TcpDiscoverySpi] TCP discovery spawning a new thread for connection [rmtAddr=/192.168.146.181, rmtPort=52412]
[15:49:03,305][INFO][tcp-disco-sock-reader-[]-#11][TcpDiscoverySpi] Started serving remote node connection [rmtAddr=/192.168.146.181:52412, rmtPort=52412]
[15:49:03,308][INFO][tcp-disco-sock-reader-[0ab07b94 192.168.146.181:52412]-#11][TcpDiscoverySpi] Initialized connection with remote server node [nodeId=0ab07b94-1f5b-48bb-987e-b07b1aa4ff9d, rmtAddr=/192.168.146.181:52412]
[15:49:03,311][INFO][tcp-disco-sock-reader-[0ab07b94 192.168.146.181:52412]-#11][TcpDiscoverySpi] Finished serving remote node connection [rmtAddr=/192.168.146.181:52412, rmtPort=52412
[15:49:03,313][INFO][main][GridTcpRestProtocol] Command protocol successfully stopped: TCP binary
[15:49:03,380][INFO][main][IgniteKernal]

>>> +---------------------------------------------------------------------------------+
>>> Ignite ver. 2.8.1#20200521-sha1:864220966caa4157c4fee8a1bc85171623963604 stopped OK
>>> +---------------------------------------------------------------------------------+
>>> Grid uptime: 00:01:55.730
1 Answers

This means you have 2 nodes with same id UUID. This is weird. Do you get this error every time?

Otherwise, it seems that you have some network issue which prevents node from forming a topology, instead they ghost each other by being around but never finish join process.

I recommend switching to static IP discovery and putting just two addresses there: 192.168.146.181, 192.168.146.182.

Related