How to configure nginx as basic load balancer

user_130204 0
Viewed 829

I have a server that I want to setup as a load balancer/reverse proxy.
nginx/1.14.2 running on debian 10

I do not want caching at all, I simply want when people visit the load balancing nginx server it sends the TCP directly to backend servers (based on nginx's ip hash algo) as if they connected to it originally.

I also want to use cloudflare on top of this load balancer for it's CDN and cache.

Here is my current setup:

upstream backend {
    ip_hash;
    server node1.example.com;
    server node2.example.com;
    keepalive 100;
}

server {
    listen 80;
    listen [::]:80;
    access_log off;

    location / {
        proxy_http_version 1.1;

        proxy_set_header Host $http_host;
        real_ip_header X-Forwarded-For;

        proxy_pass http://backend;
        proxy_redirect off;
        proxy_request_buffering off;
        proxy_buffering off;
    }
}

all nodes and the load balancer have this in their conf.d/ (which comes right from cloudflare's recommendation for nginx)

set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 104.16.0.0/12;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 131.0.72.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2c0f:f248::/32;
set_real_ip_from 2a06:98c0::/29;
real_ip_header CF-Connecting-IP;

Which seem to work fine CF-Connecting-IP is set to the client's Ip.

Issue 1

PHP server running on node1.example.com or node2.example.com is currently reporting the the following where (a.b.c.d) is the load balancers IP and (w.x.y.z) is the connecting client's IP

  ["REMOTE_ADDR"]=>  "a.b.c.d"
  ["HTTP_X_FORWARDED_FOR"]=>  "w.x.y.z"

I thought the real_ip_header X-Forwarded-For; would use the HTTP_X_FORWARDED_FOR (which comes from cloudflare) and would store it as the real IP, such that php would say REMOTE_ADDR is the same as the HTTP_X_FORWARDED_FOR

So this is what I want

  ["REMOTE_ADDR"]=> "w.x.y.z"
  ["HTTP_X_FORWARDED_FOR"]=>  "w.x.y.z"

How can I accomplish this?

Issue 2

The load balancer is adding the request HTTP header CACHE_CONTROL: max-age=0
Is this correct? If not, how can I just have the load balancer use whatever CACHE_CONTROL cloudflare sends

Issue 3

The load balancer is making the request HTTP header CONNECTION: closed but if I access the backend I always get CONNECTION: keep-alive is this correct? I set keepalive on the load balancer but it seems to always be closed

nginx-reverse-proxy nginx proxy php
3 Answers

Issue 2

The load balancer is adding the request HTTP header CACHE_CONTROL: max-age=0 Is this correct? If not, how can I just have the load balancer use whatever CACHE_CONTROL cloudflare sends

By default, Nginx's cache does not honour the Cache-Control:no-cache request header, nor the Pragma:no-cache request header. You must explicitly configure Nginx to bypass the cache and pass the request onto the origin server when the user agent sends these request headers.

Probably that will help you:

proxy_cache_bypass $http_pragma;
proxy_cache_bypass $http_cache_control;

In this case, the proxy will honour the cache header, sent by the Cloudflare services.

Issue 3

The load balancer is making the request HTTP header CONNECTION: closed but if I access the backend I always get CONNECTION: keep-alive is this correct? I set keepalive on the load balancer but it seems to always be closed

According that docs for http keepalive you should also set:

proxy_http_version 1.1;
proxy_set_header Connection "";

Please note that the default behavior for proxy_http_version is 1.0

And be sure that 100 connection, which you have set is enough keepalive 100; Because setting too low value could also lead to CONNECTION: closed behavior.

user_95160130

Issue 1 was resolved by using set_real_ip_from a.b.c.d on both node1 and node2 where a.b.c.d is the load balances IP.

user_1302040

Issue 1

...snip...

I thought the real_ip_header X-Forwarded-For; would use the X-Forwarded-For (which comes from CloudFlare) and would store it as the real IP, such that PHP would say $_SERVER["REMOTE_ADDR"] is the same as the X-Forwarded-For

You can, but please don't do that. It's semantics, really. I mean you can use your car's front-left wheel as its steering wheel, for reasons, and for both being wheels, but I'm sure people will look at you funny if you do so.

$_SERVER["REMOTE_ADDR"] in your PHP script or $remote_addr in nginx refers to the direct client it's accepting requests from; your client if they connected directly to your backend, or your load balancer/proxy if your client connected from that.

X-Forwarded-For request header from your load balancer (or proxy) server refers to your real client IP address. Because it's a plain request header, any client can spoof them, either accidentally (let's say client misconfiguration) or on purpose.

This is done for security reasons, so that if your request is a proxied/load-balanced request (having X-Forwarded-For request header); you can accept the connection if their remote address ($remote_addr in nginx or $_SERVER["REMOTE_ADDR"] in PHP) is in your lists of trusted load balancer/proxy, or reject it as a forged request if their remote address is not on the list.

Issue 2

The load balancer is adding the request HTTP header CACHE_CONTROL: max-age=0 Is this correct?

Cache-Control could be a request or response header, so you need confirm who sent this header; either your client, CloudFlare, your nginx load balancer, or your PHP script.

Issue 3

The load balancer is making the request HTTP header CONNECTION: closed but if I access the backend I always get CONNECTION: keep-alive is this correct?

nginx by default adds Host: $proxy_host and Connection: close to every proxy_pass backend requests. Use proxy_set_header directive to prevent that.

user_31038980
Related