Varnish Hash function to serve both SSR and AJAX call from the same cache

Viewed 83

We're using a Varnish Cache Proxy (in a docker container) to serve both SSR as well as AJAX request. The goal is to have a shared cache (for logged out) users independent how they access the data. The options are Nuxt (SSR) or an AJAX call in the browser (through Axios). The browser makes requests to https://api.foobar.tld/levels, while the SSR makes it's requests to the internal docker container at http://api-foobar-cache-proxy/levels. This mostly works, except for when it doesn't (case 4, in the log). The question is why?

Varnish uses the following (custom) hash function:

    sub vcl_hash {
        hash_data(req.url);
        std.log("X-DEBUG-url:" + req.url);
        if (req.http.Origin ~ "foobar") {
            hash_data("HOSTfoobar");
            std.log("X-DEBUG-host:1-HOSTfoobar");
        } elseif (req.http.host ~ "foobar") {
            hash_data("HOSTfoobar");
            std.log("X-DEBUG-host:2-HOSTfoobar");
        }
        if (req.http.Locale) {
            hash_data(req.http.Locale);
            std.log("X-DEBUG-locale:1-" + req.http.Locale);
        } else {
            hash_data("de");
            std.log("X-DEBUG-locale:2-de");
        }
    
        return (lookup);
    }

Debug Log: varnishncsa -F '"%r" %{Varnish:handling}x %{VCL_Log:X-DEBUG-url}x %{VCL_Log:X-DEBUG-host}x %{VCL_Log:X-DEBUG-locale}x' | grep "/levels" output:

    1: "GET http://api-foobar-cache-proxy/levels HTTP/1.1" miss /levels 1-HOSTfoobar 1-de // SSR first call (miss, expected, cold cache)
    2: "GET http://api-foobar-cache-proxy/levels HTTP/1.1" hit /levels 1-HOSTfoobar 1-de // SSR second call (hit, expected)
    3: "OPTIONS https://api.foobar.tld/levels HTTP/1.1" pass /levels 1-HOSTfoobar 2-de // Browser OPTIONS call (miss, expected)
    4: "GET https://api.foobar.tld/levels HTTP/1.1" miss /levels 1-HOSTfoobar 1-de // Browser first call (miss, not expected)
    5: "OPTIONS https://api.foobar.tld/levels HTTP/1.1" pass /levels 1-HOSTfoobar 2-de // Browser OPTIONS call (miss, expected)
    6: "GET https://api.foobar.tld/levels HTTP/1.1" hit /levels 1-HOSTfoobar 1-de // Browser second call (hit, expected)

Miss Expected: 1: Cold Cache

Pass Expected: 3, 5: OPTIONS are not cached, only GET requests

Hit Expected: 2, 6: From cache

Miss Unexpected: 4: Why? for all of them the 3 hash_data params where: /levels, HOSTfoobar and de. Furthermore the return (lookup) should prevent the built in vcl_hash, unless I got that completely wrong.

Some additional Debug Output from varnishlog for the requests: Comparing the varnishlog for the requests. The full log can be found here https://0bin.net/paste/2UYa-YSr#ykSOg11iJxJXA-CsUiK4es1gGekUDN4VL3wqVA9Jqdv, the relevant portions(?):

Request 1:

[...]
-   VCL_call       HASH
-   VCL_Log        X-DEBUG-url:/levels
-   VCL_Log        X-DEBUG-host:1-HOSTfoobar
-   VCL_Log        X-DEBUG-locale:1-de
-   VCL_return     lookup
-   VCL_call       MISS
-   ReqHeader      x-cache: miss
-   VCL_return     fetch
-   Link           bereq 32771 fetch
[...]

Request 2:

[...]
-   VCL_call       HASH
-   VCL_Log        X-DEBUG-url:/levels
-   VCL_Log        X-DEBUG-host:1-HOSTfoobar
-   VCL_Log        X-DEBUG-locale:1-de
-   VCL_return     lookup
-   Hit            32771 3585.968366 3600.000000 0.000000
-   VCL_call       HIT
[...]

Request 4:

[...]
-   VCL_call       HASH
-   VCL_Log        X-DEBUG-url:/levels
-   VCL_Log        X-DEBUG-host:1-HOSTfoobar
-   VCL_Log        X-DEBUG-locale:1-de
-   VCL_return     lookup
-   VCL_call       MISS
-   ReqHeader      x-cache: miss
-   VCL_return     fetch
-   Link           bereq 98318 fetch
[...]

Request 6:

[...]
-   VCL_call       HASH
-   VCL_Log        X-DEBUG-url:/levels
-   VCL_Log        X-DEBUG-host:1-HOSTfoobar
-   VCL_Log        X-DEBUG-locale:1-de
-   VCL_return     lookup
-   Timestamp      Waitinglist: 1636723569.802992 0.183061 0.183061
-   Hit            98318 3600.182590 3600.000000 0.000000
-   VCL_call       HIT
[...]
1 Answers

So, turns out as usual there was nothing wrong with the computer, the user was the problem. On the Backend I use API Platform, which can respond in different content types. Ex. if you request /levels in the browser it will deliver the Swagger UI, however if you request it with Accept: application/json it will deliver the JSON version. Changing the Accept header from */* on the SSR (imitating cURL) requests to application/json, */* did the trick.

Related