I am new to Kubernetes, I have a created a tensorflow model and using tensorflow serving to serve the model. I have also created a UI on nginx server. I am running the services on minikube.
services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
abcde-tensorflow-nginx NodePort 10.104.221.127 <none> 8080:30082/TCP 4h2m
abcde-tensorflow-serving NodePort 10.99.45.120 <none> 8501:30081/TCP 4h2m
abcde-ui NodePort 10.106.141.237 <none> 80:30080/TCP 4h2m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4h29m
When I am trying to access the below url, I am getting the Bad Gateway Error:
http://localhost:30082/v1/models/model/metadata
When I did F12 on the UI page:
Request URL: http://localhost:30082/v1/models/model:predict
Request Method: POST
Status Code: 502 Bad Gateway
Remote Address: 127.0.0.1:30082
I am not sure what is the issue here.
The nginx conf is defined like this :
server {
listen 80;
server_name localhost;
client_max_body_size 0;
client_body_timeout 300;
client_header_timeout 300;
keepalive_timeout 300;
send_timeout 300;
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Headers' 'Authorization,Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range';
add_header 'Access-Control-Allow-Methods' 'GET,POST,OPTIONS,PUT,DELETE,PATCH';
location / {
proxy_pass http://abcde-tensorflow-serving:8501/;
proxy_set_header Host $host:30082;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
send_timeout 300;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
proxy_hide_header 'Access-Control-Allow-Origin';
}
}
To access it through the browser I did port forwarding:
kubectl port-forward --address 0.0.0.0 service/abcde-ui 30080:80
kubectl port-forward --address 0.0.0.0 service/abcde-tensorflow-nginx 30082:8080
I am reading the request like this:
this.http.post<any>("http://localhost:30082/v1/models/model:predict",
json).subscribe(data => {
NGINX pod logs:
2022/09/17 20:16:10 [error] 30#30: *146 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: localhost, request: "POST /v1/models/model:predict HTTP/1.1", upstream: "http://10.99.45.120:8501/v1/models/model:predict", host: "localhost:30082", referrer: "http://localhost:30080/"
2022/09/17 20:16:23 [error] 30#30: *148 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: localhost, request: "GET /v1/models/model/metadata HTTP/1.1", upstream: "http://10.99.45.120:8501/v1/models/model/metadata", host: "localhost:30082"
NOTE: And it is working with the docker-compose file.