Connection timeout using local kafka-connect cluster to connect on a remote database

Viewed 1617

I'm trying to run a local kafka-connect cluster using docker-compose. I need to connect on a remote database and i'm also using a remote kafka and schema-registry. I have enabled access to these remotes resources from my machine.

To start the cluster, on my project folder in my Ubuntu WSL2 terminal, i'm running

docker build -t my-connect:1.0.0

docker-compose up

The application runs successfully, but when I try to create a new connector, returns error 500 with timeout.

My Dockerfile

FROM confluentinc/cp-kafka-connect-base:5.5.0

RUN cat /etc/confluent/docker/log4j.properties.template

ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components"
ARG JDBC_DRIVER_DIR=/usr/share/java/kafka/

RUN   confluent-hub install --no-prompt confluentinc/kafka-connect-jdbc:5.5.0 \
   && confluent-hub install --no-prompt confluentinc/connect-transforms:1.3.2

ADD java/kafka-connect-jdbc /usr/share/confluent-hub-components/confluentinc-kafka-connect-jdbc/lib/
COPY java/kafka-connect-jdbc/ojdbc8.jar /usr/share/confluent-hub-components/confluentinc-kafka-connect-jdbc/lib/

ENTRYPOINT ["sh","-c","export CONNECT_REST_ADVERTISED_HOST_NAME=$(hostname -I);/etc/confluent/docker/run"] 

My docker-compose.yaml

services:
  connect:
    image: my-connect:1.0.0
    ports:
     - 8083:8083
    environment:
      - CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL=http=//schema-registry:8081
      - CONNECT_KEY_CONVERTER=io.confluent.connect.avro.AvroConverter
      - CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL=http=//schema-registry:8081
      - CONNECT_BOOTSTRAP_SERVERS=broker1.intranet:9092
      - CONNECT_GROUP_ID=kafka-connect
      - CONNECT_INTERNAL_KEY_CONVERTER=org.apache.kafka.connect.json.JsonConverter
      - CONNECT_VALUE_CONVERTER=io.confluent.connect.avro.AvroConverter
      - CONNECT_INTERNAL_VALUE_CONVERTER=org.apache.kafka.connect.json.JsonConverter
      - CONNECT_OFFSET_STORAGE_TOPIC=kafka-connect.offset
      - CONNECT_CONFIG_STORAGE_TOPIC=kafka-connect.config
      - CONNECT_STATUS_STORAGE_TOPIC=kafka-connect.status
      - CONNECT_CONNECTOR_CLIENT_CONFIG_OVERRIDE_POLICY=All
      - CONNECT_LOG4J_ROOT_LOGLEVEL=INFO
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
      - CONNECT_REST_ADVERTISED_HOST_NAME=localhost

My cluster it's up

~$ curl -X GET http://localhost:8083/
{"version":"5.5.0-ccs","commit":"606822a624024828","kafka_cluster_id":"OcXKHO7eT4m9NBHln6ACKg"}

Connector call

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d
{
    "name": "my-connector",
    "config":  
    { 
    "connector.class" : "io.debezium.connector.oracle.OracleConnector",
    "tasks.max": "1",
    "database.user": "user", 
    "database.password": "pass",    
    "database.dbname":"SID",
    "database.schema":"schema",
    "database.server.name": "dbname",   
    "schema.include.list": "schema",    
    "database.connection.adapter":"logminer",   
    "database.hostname":"databasehost",
    "database.port":"1521"
   }
}

Error

{"error_code": 500,"message": "IO Error trying to forward REST request: java.net.SocketTimeoutException: Connect Timeout"}

## LOG
connect_1  | [2021-07-01 19:08:50,481] INFO Database Version: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
connect_1  | Version 19.4.0.0.0 (io.debezium.connector.oracle.OracleConnection)
connect_1  | [2021-07-01 19:08:50,628] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection)
connect_1  | [2021-07-01 19:08:50,643] INFO AbstractConfig values:
connect_1  |  (org.apache.kafka.common.config.AbstractConfig)
connect_1  | [2021-07-01 19:09:05,722] ERROR IO error forwarding REST request:  (org.apache.kafka.connect.runtime.rest.RestClient)
connect_1  | java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: Connect Timeout

Testing the connection to the database

$ telnet databasehostname 1521 Trying <ip>... Connected to databasehostname

Testing connection to kafka broker

$ telnet broker1.intranet 9092 Trying <ip>... Connected to broker1.intranet

Testing connection to remote schema-registry

$ telnet schema-registry.intranet 8081 Trying <ip>... Connected to schema-registry.intranet

What am I doing wrong? Do I need to configure something else to allow connection to this remote database?

1 Answers
Related