Global_asax_BeginRequest - How does this line error?

The described situation does not seem to be possible (I'll explain later) and I would recommend a bit more bullet-proof diagnostics for example

Try
  Dim locReq As HttpRequest = Request
Catch ex As Exception
   'record Exception with the complete stack trace
    Throw
End Try 

and use a HttpRequest variable (locReq) instead of the Request property. The snippet also prevents changing the Request property return value.

Why I think that your assumptions are not possible:

It is not possible to get NullReference in the line

If Request IsNot Nothing ...

but it can come from depth of the Request property. Its (translated) source code is

Public Class HttpApplication
'...
    Public ReadOnly Property Request As HttpRequest
        Get
            Dim request As HttpRequest = Nothing
            If _context IsNot Nothing AndAlso
                 Not _hideRequestResponse Then request = _context.Request

            If request Is Nothing Then Throw New HttpException(SR.GetString(SR.Request_not_available))
            Return request
        End Get
    End Property

Here again is a property Request with (translated) source code

Public NotInheritable Class HttpContext
    '...
    Private _response As HttpResponse
    '...
    Public ReadOnly Property Response As HttpResponse
        Get
            If HideRequestResponse OrElse
               HasWebSocketRequestTransitionCompleted Then Throw New 
                  HttpException(SR.GetString(SR.Response_not_available))
            Return _response
        End Get
    End Property

When I read the code I see no space for a null reference but there can be a HttpException. So my conclusion is that the logging system wrongly records the line or the exception.

Please take a look at IIS7 Integrated mode: Request is not available in this context exception in Application_Start which is connected to the Response_not_available exception.

Edit

I have attempted to interpret the presented stack trace. My conclusion from the deepest procedure (ExecuteStep(IExecutionStep step, ref bool completedSynchronously) is that the exception can occur only in its catch block (translated with original comments).

Catch e As Exception
        [error] = e
        'Since we will leave the context later, we need to remember if we are impersonating
         'before we lose that info - VSWhidbey 494476

        If ImpersonationContext.CurrentThreadTokenExists Then
            e.Data(System.Web.Management.WebThreadInformation.IsImpersonatingKey) = String.Empty
        End If
        'This might force ThreadAbortException to be thrown
        'automatically, because we consumed an exception that was
        'hiding ThreadAbortException behind it

        If TypeOf e Is ThreadAbortException AndAlso ((Thread.CurrentThread.ThreadState And ThreadState.AbortRequested) = 0) Then
            [error] = Nothing
            'Response.End from a COM+ component that re-throws ThreadAbortException
            'It is not a real ThreadAbort
             'VSWhidbey 178556
            _stepManager.CompleteRequest()
        End If
    End Try 

From this it seems that the problem is realy connected with aborting a thread. My speculatiom is that _stepManager is nothing, but I have no idea why.

Just in case this is helpful to someone else.

I've finally figured out how to re-produce the error. It happens if I make a call without the User-Agent header. Like so...

curl -v -H 'User-Agent:' "https://[my_domain]/"

The path for the request doesn't seem to make any difference.

I expect if I can find a way to block all requests without a user-agent, then hopefully this will fix the problem.

What I can't quite understand though, is why Request isn't available in Application_BeginRequest, but is available in Application_Error.