I cannot deploy a NiFi cluster using docker (docker exits with code 0)

user_15411076 0
Viewed 223

I want to deploy a NiFi cluster using docker compose using the following yaml file:

version: "3"
services:
  zookeeper:
    hostname: zookeeper
    container_name: zookeeper
    image: 'bitnami/zookeeper:latest'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  nifi:
    image: apache/nifi:1.12.0-SNAPSHOT-dockermaven
    ports:
      - 8080 # Unsecured HTTP Web Port
    environment:
      - NIFI_WEB_HTTP_PORT=8080
      - NIFI_CLUSTER_IS_NODE=true
      - NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082
      - NIFI_ZK_CONNECT_STRING=zookeeper:2181
      - NIFI_ELECTION_MAX_WAIT=1 min

Then I deploy it using docker-compose like this:

docker-compose up --scale nifi=3

It gets to successfully deploy zookeeper but the three NiFi nodes docker containers stop and docker throws the following information:

nifi_cluster_2-nifi-2  | 2022-03-04 11:26:16,755 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 198 properties from /opt/nifi/nifi-current/./conf/nifi.properties
nifi_cluster_2-nifi-2  | 2022-03-04 11:26:16,801 ERROR [main] o.a.nifi.properties.NiFiPropertiesLoader Clustered Configuration Found: Shared Sensitive Properties Key [nifi.sensitive.props.key] required for cluster nodes
nifi_cluster_2-nifi-2  | 2022-03-04 11:26:16,808 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi
nifi_cluster_2-nifi-2  | java.lang.IllegalArgumentException: There was an issue decrypting protected properties
nifi_cluster_2-nifi-2  |    at org.apache.nifi.NiFi.initializeProperties(NiFi.java:373)
nifi_cluster_2-nifi-2  |    at org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:341)
nifi_cluster_2-nifi-2  |    at org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:337)
nifi_cluster_2-nifi-2  |    at org.apache.nifi.NiFi.main(NiFi.java:329)
nifi_cluster_2-nifi-2  | Caused by: org.apache.nifi.properties.SensitivePropertyProtectionException: Sensitive Properties Key [nifi.sensitive.props.key] not found: See Admin Guide section [Updating the Sensitive Properties Key]
nifi_cluster_2-nifi-2  |    at org.apache.nifi.properties.NiFiPropertiesLoader.getDefaultProperties(NiFiPropertiesLoader.java:225)
nifi_cluster_2-nifi-2  |    at org.apache.nifi.properties.NiFiPropertiesLoader.get(NiFiPropertiesLoader.java:214)
nifi_cluster_2-nifi-2  |    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
nifi_cluster_2-nifi-2  |    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
nifi_cluster_2-nifi-2  |    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
nifi_cluster_2-nifi-2  |    at java.lang.reflect.Method.invoke(Method.java:498)
nifi_cluster_2-nifi-2  |    at org.apache.nifi.NiFi.initializeProperties(NiFi.java:368)
nifi_cluster_2-nifi-2  |    ... 3 common frames omitted
nifi_cluster_2-nifi-1 exited with code 0
nifi_cluster_2-nifi-2 exited with code 0
nifi_cluster_2-nifi-3 exited with code 0

I also tried using other yaml files for deploying a docker-compose for a NiFi cluster but I keep getting the same "exited with code 0" problem.

How can I solve this?

apache-nifi docker-compose docker
1 Answers

With those environment variables you'll only be able to run one NiFi instance. The NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082, for example, would be stored in Zookeeper for each instance, and so when scaling that instance, Zookeeper would be confused which instance is which, and when NiFi nodes try to form a cluster, they would share incorrect data.

I'd recommend using minikube and a Nifi Operator to dynamically set such variables and scale a replicaset.

user_23086830
Related