Connecting to local EventStore is not being maintained when debugging .NET application

user_1696992 0
Viewed 714

Problem: my connection to a locally stored EventStore is not being maintained when I debug my .NET application from Visual Studio.

I get an error:

Connection 'ES-6ab8f860-e44b-40ca-93fa-7370b72547d8' was closed.

Current setup: my Event Store is placed in my solution, application referenced by a PATH variable.

//Path to the local EventStore in solution
private const string Path = @"..\..\EventStore-OSS-Win-v5.0.8\EventStore.ClusterNode.exe";

The following code setups a new connection to the EventStore:

public static void SetupEventStore(StartConflictOption opt = StartConflictOption.Connect) //set default to Connect
{
    // Save the EventStore process in a variable for later use
    var runningEventStores = Process.GetProcessesByName("EventStore.ClusterNode");
    // if a process was found, check the parameter options on what to do with the process
    if (runningEventStores.Length != 0)
    {
        switch (opt)
        {
            case StartConflictOption.Connect:
                _process = runningEventStores[0]; //set the process to the EventStore.ClusterNode
                break;

            case StartConflictOption.Kill:
                foreach (var es in runningEventStores) //makes sure that all running processes are killed
                {
                    es.Kill();
                }
                break;

            case StartConflictOption.Error: 
                throw new Exception("Conflicting EventStore running."); //Will be thrown if there is already a running EventStore process

            default:
                throw new ArgumentOutOfRangeException(nameof(opt), opt, null);
        }
    }

    if (_process == null)
    {
        _process = new Process
                {
                    StartInfo =
                    {
                        UseShellExecute = false, CreateNoWindow = true, FileName = Path, Arguments = Args, Verb = "runas"
                    }
                };
        _process.Start();
    }

    //set default IP endpoint and port (localhost:1113). HTTP uses port 2113, while TCP uses port 1113
    var tcp = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 1113);
    var http = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 2113);

    //Connect to the Event Store
    Connection = EventStoreConnection.Create(new Uri("tcp://admin:changeit@localhost:1113"));
    Connection.ConnectAsync().Wait();
}

When I debug the application, it starts up without issues, but when I attempt to create a new stream to the EventStore, it crashes with the previously mentioned error.

I noticed that the EventStore process is active for a split second, before being terminated when debugging the application.

Question: why is my Event Store process not being maintained while the application is running and what steps can I take to fix this issue?

eventstoredb visual-studio c#
1 Answers

The answer to your question is below, but first, let me express my worry about your setup. I could never imagine an application to execute a database that it uses in a similar fashion. What is the reason to start Event Store from your application, why can't it be started in a different way? Using process.Kill on any database, not only Event Store, is also not the best idea.

If you must carry Event Store as a part of your application, I'd suggest using the embedded version instead (not guaranteed that it will survive in future versions though).

What you observe is normal behaviour. The same happens with any real-time always-on connection, like gRPC streaming.

When you are step-debugging your app and stand still at the breakpoint, the world stops for the whole application. There's no multi-threading or anything else that works when you hit the breakpoint and don't move on.

In practice, you should prepare your application to handle connection failures. One of the fallacies of distributed computing is that "the network is reliable". It is not. The IEventStoreConnection instance has a few events, like 

_connection.Disconnected += (sender, args) =>
{
    log.Warning("Connection closed");
    Task.Run(() => Reconnect()); // Reconnect is some place when you try to connect again
};

I would advise avoiding step-in debugging overall and in applications that use always-on communications in particular. You can get much more value from writing tests that cover your use cases and. Logs are useful too.

If you absolutely must debug with breakpoints, you can change your local environment settings so the connection string will be more tolerant of being disconnected. Check the list of settings in the docs. Some settings that might help are HeartbeatInterval and HeartbeatTimeout. By setting those settings to a minute (for example), you will have enough time for debugging before the connection closes. Do not use such values in production though.

You can also instruct the connection to keep reconnecting (KeepReconnecting(). You need to use the ConnectionSettingsBuilder parameter in the code that creates the connection. There are methods to adjust the heartbeat interval and timeout there as well, which you can use instead of changing the connection string. Using different connection strings though is easier since you can have different configuration files per environment in .NET Core.

An example:

var connectionString = $"ConnectTo=tcp://{user}:{password}@{host}:1113; HeartBeatTimeout=500";

var settingsBuilder = ConnectionSettings
    .Create()
    .KeepReconnecting()
    .LimitReconnectionsTo(10);

_connection = EventStoreConnection.Create(connectionString, settingsBuilder);
user_4840410
Related