I'm seeing many exception logs in a Kestrel server (ASP.NET Core 3.1) behind HAProxy. I'm trying to understand if this is configuration related that should be improved or if there's an issue that should be looked into:
00:19:40.033 [th-282] [Debug] [Microsoft.AspNetCore.Server.Kestrel] Connection id "0HMH80KISBCOF" started.
00:19:41.002 [th-137] [Debug] [Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets] Connection id "0HMH80KISBCOF" reset.
00:19:41.019 [th-144] [Error] [Microsoft.AspNetCore.Server.Kestrel] Uncaught exception from the OnConnectionAsync method of an IConnectionAdapter. Microsoft.AspNetCore.Connections.ConnectionResetException: An existing connection was forcibly closed by the remote host ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs.<GetResult>g__ThrowSocketException|7_0(SocketError e)
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs.GetResult()
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection.<ProcessReceives>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection.<DoReceive>d__23.MoveNext()
--- End of inner exception stack trace ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
at System.IO.Pipelines.Pipe.GetReadAsyncResult()
at Microsoft.AspNetCore.Server.Kestrel.Core.Adapter.Internal.RawStream.<ReadAsyncInternal>d__22.MoveNext()
00:19:41.035 [th-137] [Debug] [Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets] Connection id "0HMH80KISBCOF" sending FIN because: "The Socket transport's send loop completed gracefully."
00:19:41.050 [th-179] [Debug] [Microsoft.AspNetCore.Server.Kestrel] Connection id "0HMH80KISBCOF" disconnecting.
00:19:41.066 [th-144] [Debug] [Microsoft.AspNetCore.Server.Kestrel] Connection id "0HMH80KISBCOF" stopped.
as well as:
16:07:23.380 [th-191] [Debug] [Microsoft.AspNetCore.Server.Kestrel] Connection id "0HMH80KISBCTS" completed keep alive response.
16:07:24.044 [th-191] [Debug] [Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets] Connection id "0HMH80KISBCTS" reset.
16:07:24.044 [th-505] [Debug] [Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets] Connection id "0HMH80KISBCTS" sending FIN because: "The client closed the connection."
16:07:24.044 [th-505] [Debug] [Microsoft.AspNetCore.Server.Kestrel] Connection id "0HMH80KISBCTS" disconnecting.
16:07:24.044 [th-505] [Info] [Microsoft.AspNetCore.Server.Kestrel] Connection id "0HMH80KISBCTS" request processing ended abnormally. System.IO.IOException: The read operation failed, see inner exception. ---> System.AggregateException: One or more errors occurred. ---> Microsoft.AspNetCore.Connections.ConnectionResetException: An existing connection was forcibly closed by the remote host ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs.<GetResult>g__ThrowSocketException|7_0(SocketError e)
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs.GetResult()
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection.<ProcessReceives>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection.<DoReceive>d__23.MoveNext()
--- End of inner exception stack trace ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
at System.IO.Pipelines.Pipe.GetReadAsyncResult()
at Microsoft.AspNetCore.Server.Kestrel.Core.Adapter.Internal.RawStream.<ReadAsyncInternal>d__22.MoveNext()
--- End of inner exception stack trace ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Server.Kestrel.Core.Adapter.Internal.RawStream.EndRead(IAsyncResult asyncResult)
at System.Net.FixedSizeReader.ReadCallback(IAsyncResult transportResult)
--- End of inner exception stack trace ---
at System.Net.Security._SslStream.EndRead(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncTrimPromise`1.Complete(TInstance thisRef, Func`3 endMethod, IAsyncResult asyncResult, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.AspNetCore.Server.Kestrel.Core.Adapter.Internal.AdaptedPipeline.<ReadInputAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
at System.IO.Pipelines.Pipe.GetReadAsyncResult()
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<ProcessRequests>d__185`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<ProcessRequestsAsync>d__184`1.MoveNext()
16:07:24.044 [th-505] [Debug] [Microsoft.AspNetCore.Server.Kestrel] Connection id "0HMH80KISBCTS" stopped.
The config on the HAProxy side is simply:
listen c-https
bind 1.2.3.4:443
mode tcp
option tcplog
server server1 5.6.7.8:443
server server2 5.6.7.9:443
and Kestrel is using TLS:
{
"Kestrel": {
"Endpoints": {
"HttpsInlineCertStore": {
"Url": "https://+",
"Certificate": {
"Subject": "*.mydomain.com",
"Store": "My",
"Location": "LocalMachine",
"AllowInvalid": "true"
}
}
}
}
}