read: connection reset by peer

Viewed 1972

I have deployed rabbitmq service instance in my EKS cluster using rabbitmq operator. I have enabled ssl in the rabbitmq instance defination file. This is my rabbitmq-cluster.yaml

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmq-single-instance-clusterip # name of the service instance.
  labels:
    app: rabbitmq
  annotations:
    purpose: rabbitmq-poc
spec:
  replicas: 1  # 1 replica for single instance.
  image: rabbitmq:3.8.10-management-alpine # this is required if the image has to pull from private registry. In such case imagePullSecrets is also required. Default is from docker.hub/rabbitmq.
  service:
    type: ClusterIP  # Possible values : ClusterIP/NodePort/LoadBalancer. Default is - ClusterIP.  RabbitMQ Cluster Kubernetes Operator currently does not support the ExternalName Service Type
  resources:
    requests:
      cpu: 2
      memory: 8Gi
    limits:
      cpu: 2
      memory: 8Gi
  persistence:
    storageClassName: ebs-sc # The name of the Kubernetes StorageClass to use.
    storage: 10Gi  # The capacity of the persistent volume, expressed as a Kubernetes resource quantity.
  rabbitmq: # this is for rabbitmq cluster configuration. operator by default attaches default parameters.
    additionalConfig: |     # Additional configuration options that will be appended to rabbitmq.conf file.
       channel_max = 1050
       ssl_options.fail_if_no_peer_cert = false
    #envConfig:  # The value of spec.rabbitmq.envConfig will be written to /etc/rabbitmq/rabbitmq-env.conf.
    additionalPlugins:  # Additional plugins to enable in RabbitMQ. By default , rabbitmq_peer_discovery_k8s/rabbitmq_prometheus/rabbitmq_management
      - rabbitmq_top
      - rabbitmq_shovel
  tls:
    secretName: tls-secret
    caSecretName: ca-secret
    disableNonTLSListeners: true

I have set this flag disableNonTLSListeners = true, which means it discards all non-tls connections. The rabbitmq instance is working fine.

k get pods,svc
NAME                                              READY   STATUS             RESTARTS   AGE
pod/rabbitmq-single-instance-clusterip-server-0   1/1     Running            0          3h7m


NAME                                               TYPE           CLUSTER-IP      EXTERNAL-IP                                                                     PORT(S)                          AGE
service/rabbitmq-single-instance-clusterip         ClusterIP      10.100.6.196    <none>                                                                          5671/TCP,15671/TCP               3h7m
service/rabbitmq-single-instance-clusterip-nodes   ClusterIP      None            <none>                                                                          4369/TCP,25672/TCP               3h7m

I am trying to connect to this instance using sample go consumer.

package main

import (
    "crypto/tls"
    "fmt"

    "github.com/streadway/amqp"
)

func main() {
    fmt.Println("Go RabbitMQ Consumer Tutorial")
    fmt.Println("Testing ClusterIP service connection over TLS")
    /*
        cert, err := tls.LoadX509KeyPair("client.crt", "mykey.key")
        if err != nil {
            panic(err)
        }
        //Load CA cert.
        caCert, err := ioutil.ReadFile("ca.crt") // The same you configured in your MQ server
        if err != nil {
            log.Fatal(err)
        }
        caCertPool := x509.NewCertPool()
        caCertPool.AppendCertsFromPEM(caCert)
        tlsConfig := &tls.Config{
            Certificates: []tls.Certificate{cert}, // from tls.LoadX509KeyPair
            RootCAs:      caCertPool,
            // ...other options are just the same as yours
        }*/

    tlsconfig1 := &tls.Config{
        InsecureSkipVerify: true,
    }

    conn, err := amqp.DialTLS("amqps://<username>:<password>@rabbitmq-single-instance-clusterip.demo.svc.cluster.local:5671", tlsconfig1)
    if err != nil {
        fmt.Println("Error is -", err)
        panic(err)
    }

    fmt.Println("Connected to consumer successfully")
    defer conn.Close()

    ch, err := conn.Channel()
    if err != nil {
        fmt.Println(err)
    }
    defer ch.Close()

    if err != nil {
        fmt.Println(err)
    }

    msgs, err := ch.Consume(
        "TestQueue",
        "",
        true,
        false,
        false,
        false,
        nil,
    )

    forever := make(chan bool)
    go func() {
        for d := range msgs {
            fmt.Printf("Recieved Message: %s\n", d.Body)
        }
    }()

    fmt.Println("Successfully Connected to our RabbitMQ Instance")
    fmt.Println(" [*] - Waiting for messages")
    <-forever

}

When I run this code I am getting.

Go RabbitMQ Consumer Tutorial
Testing ClusterIP service connection over TLS
Error is - read tcp 172.20.150.29:58892->10.100.6.196:5671: read: connection reset by peer
panic: read tcp 172.20.150.29:58892->10.100.6.196:5671: read: connection reset by peer

goroutine 1 [running]:
main.main()
    /app/main.go:38 +0x689

Any idea, why this is not working? If I want ti make this working without passing the client side certificates then what should I do?

My Rabbitmq config :

ssl_options.certfile                  = /etc/rabbitmq-tls/tls.crt
ssl_options.keyfile                   = /etc/rabbitmq-tls/tls.key
listeners.ssl.default                 = 5671
management.ssl.certfile               = /etc/rabbitmq-tls/tls.crt
management.ssl.keyfile                = /etc/rabbitmq-tls/tls.key
management.ssl.port                   = 15671
prometheus.ssl.certfile               = /etc/rabbitmq-tls/tls.crt
prometheus.ssl.keyfile                = /etc/rabbitmq-tls/tls.key
prometheus.ssl.port                   = 15691
listeners.tcp                         = none
ssl_options.cacertfile                = /etc/rabbitmq-tls/ca.crt
ssl_options.verify                    = verify_peer
management.ssl.cacertfile             = /etc/rabbitmq-tls/ca.crt
prometheus.ssl.cacertfile             = /etc/rabbitmq-tls/ca.crt
total_memory_available_override_value = 6871947674
channel_max                           = 1050
ssl_options.fail_if_no_peer_cert      = false
0 Answers
Related