Routing traffic back to a specific pod

Question

I have an ingress resource like the following:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: "app-ingress"
  labels:
    app: "app"
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  tls:
    - hosts:
        - api.example.com
      secretName: tls-secret
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /grpc
            pathType: ImplementationSpecific
            backend:
              service:
                name: "app-server"
                port:
                  number: 50051
          - path: /callback
            pathType: ImplementationSpecific
            backend:
              service:
                name: "app-server"
                port:
                  number: 80

app-server runs two services (on the same pod), a grpc web server and a http web server. To start, a client makes a request to api.example.com/grpc. To fulfill the request, the server makes a request to a third party service, waits for a callback on api.example.com/callback, and then completes the client's request (api.example.com/callback is only accessed by the third party service, not the client).

The question is, how can I specifically route api.example.com/callback back to the same pod that made the initial request? Obviously if pod1 were to make a request to the third party service and pod2 were to receive the callback, pod1 would never receive the callback and the client request would have to timeout. I can attach information to the callback I receive from the third party service, so is there a way I could attach some sort of pod id so that kubernetes will route the callback back to the pod that initiated it?

I've looked at maybe using StatefulSets, but I'm not sure exactly how this would work. Or would it be better to have sort of "notification pod" that receives callbacks and routes them to the correct pod?

user_15659347 · Answer

One way would be to user sticky sessons.
If you want to make sure that connections from a particular client are passed to the same Pod each time, you can select the session affinity based on the client's IP addresses by setting service.spec.sessionAffinity to "ClientIP" (the default is "None"). You can also set the maximum session sticky time by setting service.spec.sessionAffinityConfig.clientIP.timeoutSeconds appropriately (the default value is 10800, which works out to be 3 hours).

Another way is to design the Application in away that it doesn't matter which pod receives callback request, the proper pod with the pending client connection will be notified (and provided with data), and send a response back to the client. To communicate requests data you can use some sort of messaging service or distributed in-memory data stores like Redis.

Routing traffic back to a specific pod

1 Answers