SignalR connect blocks while javascript file is pending

I use SignalR on a page that includes the standard google tag manager script. That script is loaded asynchronously (defer=true).

SignalR is used for some long running tasks that can be triggered on that page, so I disable the buttons that trigger those actions until SignalR is ready, i.e. when the call to connect() has returned successfully.

Some customers report that I takes forever for the buttons to be activated. Those customers are on a corporate network where google tag manager is blocked, and for some reasons the request to googletagmanager.com doesn't always return immediately with status 403. Sometimes it only returns the 403 after a timeout (like after 2 minutes).

I don't care if customers block google tag manager, but my SignalR connect call (on document ready) will stall until the call to google tag manager is finished. It shouldn't stall, because they should be logically independent and the script is loaded with defer=true.

I am lost to why the connect call won't finish sooner. Only thing I found was "don't use session state with SignalR" (e.g. SignalR doesn't use Session on server). I use the Session object in my web application, but not from the SignalR hub.

Using .NET Framework 4.5 and SignalR 2.4.1

SignalR startup on server is standard:

using Microsoft.Owin;
using Owin;
using MyNamespace;

[assembly: OwinStartup(typeof(SignalRStartup))]
namespace MyNamespace
{
    public class SignalRStartup
    {
        public void Configuration(IAppBuilder app)
        {
            // Any connection or hub wire up and configuration should go here
            app.MapSignalR();
        }
    }
}

Hub implementation doesn't do anything with the Session

public class CollectiefHub : Hub
{
    private static readonly IHubContext sHubContext = GlobalHost.ConnectionManager.GetHubContext<CollectiefHub>();
    public const string MsgCalcReady = "CalcReady";
    public const string MsgDocGenReady = "DocGenReady";

    public Task JoinGroup(string groupName)
    {
        return sHubContext.Groups.Add(Context.ConnectionId, groupName);
    }

    public Task LeaveGroup(string groupName)
    {
        return sHubContext.Groups.Remove(Context.ConnectionId, groupName);
    }

    private static async Task Send(string group, object msg)
    {
        await sHubContext.Clients.Group(group).receive(msg);
    }

    public static async void SendMessage(string groupName, object message)
    {
        await Send(groupName, message);
    }

    public static async void HandleListDone(string groupName, Dictionary<string, string> list)
    {
        await Send(groupName, new { list });
    }

    public static async void HandleListProgress(string groupName, string guid, int percentage)
    {
        await Send(groupName, new { guid, percentage });
    }
}

Javascript code:

var AsyncCalc = {
    connection: $.hubConnection(),
    collectiefHub: null,
    group: window.location.pathname.split("/")[2], // Contains identifier
    calcDoneEvent: "asyncCalcDone",
    asyncObject: null,
    connected: false,

    setupSignalR: function() {
        AsyncCalc.collectiefHub = AsyncCalc.connection.createHubProxy("collectiefHub");
        AsyncCalc.collectiefHub.on("receive", function (message) {
            // Handle specific messages here
        });

        // Start connection
        AsyncCalc._start();
    },

    _start: function () {
        AsyncCalc.connection.url = "/" + window.location.pathname.split("/")[1] + "/signalr";
        AsyncCalc.connection.start().done(function () {
            AsyncCalc.connected = true;
            AsyncCalc.collectiefHub.invoke("joinGroup", AsyncCalc.group);
            $(".async").removeClass("disabled");
        });
    }
};

$(function () {
    AsyncCalc.setupSignalR();
});

Google tag manager is simply a javascript js file that is included in the <head> section of my page, like this:

<script async src="https://www.googletagmanager.com/gtm.js?id=GTM-XXXXXX"></script>

Any ideas?