Health Checks failing for an ingress created on GKE

I'm creating a load balancer for consul ui on GKE using ingress with the following configs

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: public-apis
spec:
  rules:
    - host: consul.example.com
      http:
        paths:
          - path: /ui/
            pathType: Prefix
            backend:
              serviceName: hashicorp-consul-ui
              servicePort: http

The service hashicorp-consul-ui is a service of type clusterIP.

Also, I'm creating the following backendConfig

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: consul-ui-backendconfig
spec:
  healthCheck:
    type: HTTP
    requestPath: /v1/health/state/failed
    port: 80

I'm adding the following annotation to the clusterIP service hashicorp-consul-ui

'cloud.google.com/backend-config': '{"ports": {"80":"consul-ui-backendconfig"}}'

This backendConfig creates health check for the backend service created by ingress, this endpoint returns 200. But the health check fails and the ingress is in Unhealthy state

I've enabled log on healthCheck and I'm getting the following log on NEG

healthCheckProbeResult: {
   detailedHealthState: "TIMEOUT"    
   healthCheckProtocol: "HTTP"    
   healthState: "UNHEALTHY"    
   ipAddress: "10.24.1.30"    
   previousDetailedHealthState: "UNKNOWN"    
   previousHealthState: "UNHEALTHY"    
   probeCompletionTimestamp: "2021-02-14T03:31:43.337063261Z"    
   probeRequest: "/v1/health/state/failed"    
   probeResultText: "HTTP response: , Error: Connection refused"    
   probeSourceIp: "35.191.9.211"    
   responseLatency: "0.001265s"    
   targetIp: "10.24.1.30"    
   targetPort: 80    
}

Also, not sure why I got this log only once, shouldn't I be getting these logs at regular interval?

Also, can anyone guide me about how can I fix this healthCheck issue?