How browser knows which headers add to requests

user_11665367 0
Viewed 1412

When I type url of a site to browser's address bar, browser sends a request to get the resource by the url. But when I go to different web sites (google.com, amazon.com, etc.), requests which initialize the page, have different headers for different sites.

Where browser gets the set of request's headers to load the page if browser has only information about URL of this resource at the first initialization?

for example when I go to google.com browser sends such request headers:

:authority: www.google.com
:method: GET
:path: /
:scheme: https
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9,ru-RU;q=0.8,ru;q=0.7
cache-control: max-age=0
sec-fetch-dest: document
sec-fetch-mode: navigate
sec-fetch-site: same-origin
sec-fetch-user: ?1
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36

For amazon.com, the request's headers are different:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,ru-RU;q=0.8,ru;q=0.7
Connection: keep-alive
Host: amazon.com
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36
server request http-headers browser
1 Answers

When you type in a URL into the address the bar this needs to be translated to an HTTP request.

So typing www.google.com means you need to GET the default page (/) from that server. That's basically all covered in the first 4 lines in the first request.

The browser also knows what types of format it can accept. Mostly we deliver HTML back so text/html is certainly in there, but we also accept other formats - including the completely generic */* btw! :-)

Requests are often compressed (with either gzip, deflate or the newer brotli (br) format) so the browser tells the server which of those it supports in the accept-encoding header.

When you installed your browser you also set a default language so we can tell the server that. Some servers will return different content based on this.

Then there are some security headers (I won't go into these as quite complicated).

Finally we have the user-agent header. this is basically where the browser tells the server whether it's Chrome, or Firefox or whatever. But for historical reasons it's much longer than just "Chrome".

So basically the request headers are things the browser sends to the server to give it more information about the browser and it's capabilities. For a request that's just typed into the browser the request headers will basically be the same no matter what the URL is. For additional requests made by the page - e.g. by JavaScript code they may be different if it adds more headers.

As to the differences between the two example requests you gave:

Google uses HTTP/2 (or QUIC if using Chrome but for now that's basically HTTP/2 as far as this question is concerned). you can see this if you add the option Protocol column to developer tools.

HTTP/2 has a couple of changes from HTTP/1, namely:

  • HTTP Header Names are lower cased. Technically in HTTP/1 they are case insensitive, but by convention many tools like browser used title case (capitalising first letter of each word).
  • The request (e.g. GET / HTTP/1.1) is converted to pseudo headers beginning with a colon (:method: GET, :path: /...etc.).
  • Host is basically :authority in HTTP/2.
  • :scheme is basically new in HTTP/2 as previously it wasn't explicitly part of the HTTP request, and handled at a connection level.
  • Connection is defunct in HTTP/2. Even in HTTP/1.1 it defaulted to keep-alive so above header was not necessary but lots of browsers and other clients sent it for historical reasons.

I think that explains all the differences.

So how does the browser know whether to use HTTP/2 or HTTP/1.1? Which already has an answer on Stack Overflow, but basically it's decided when the HTTPS session is established if the server advises it can support HTTP/2 and the browser wants to use it.

user_21445780
Related