Note: according to tests (see Edit below), this occurs only on a Linux machine.
I have an ASP.NET Core Blazor application (using server-side hosting model) running on a Raspberry Pi. Part of the application's functionality is to dim/brighten screen based on when system was last interacted with. To do that, every 1 second or so I spawn a terminal child-process to run xprintidle, parse its output, and act accordingly.
I use DataDog for monitoring, and I am having a memory leak until the system crashes (it takes a few days to use up all memory, but it does occur eventually):

I have pinpointed that the following method is what leaks memory - if I skip calling it and use some constant timespan, the memory does not leak: I have following code to do so:
// note this code has some parts that aren't even needed - I was simply trying anything to solve this problem at this point
public async Task<TerminalResult> ExecuteAndWaitAsync(string command, bool asRoot, CancellationToken cancellationToken = default)
{
using Process prc = CreateNewProcess(command, asRoot);
// we need to redirect stdstreams to read them
prc.StartInfo.RedirectStandardOutput = true;
prc.StartInfo.RedirectStandardError = true;
// start the process
_log.LogTrace("Starting the process");
using Task waitForExitTask = WaitForExitAsync(prc, cancellationToken);
prc.Start();
// read streams
string[] streamResults = await Task.WhenAll(prc.StandardOutput.ReadToEndAsync(), prc.StandardError.ReadToEndAsync()).ConfigureAwait(false);
// wait till it fully exits, but no longer than half a second
// this prevents hanging when process has already finished, but takes long time to fully close
await Task.WhenAny(waitForExitTask, Task.Delay(500, cancellationToken)).ConfigureAwait(false);
// if process still didn't exit, force kill it
if (!prc.HasExited)
prc.Kill(true); // doing it with a try-catch approach instead of HasExited check gives no difference
return new TerminalResult(streamResults[0], streamResults[1]);
}
public Task<int> WaitForExitAsync(Process process, CancellationToken cancellationToken = default)
{
TaskCompletionSource<int> tcs = new TaskCompletionSource<int>();
IDisposable tokenRegistration = null;
EventHandler callback = null;
tokenRegistration = cancellationToken.Register(() =>
{
Unregister();
tcs.TrySetCanceled(cancellationToken);
});
callback = (sender, args) =>
{
Unregister();
tcs.TrySetResult(process.ExitCode);
};
process.Exited += callback;
process.EnableRaisingEvents = true;
void Unregister()
{
lock (tcs)
{
if (tokenRegistration == null)
return;
process.EnableRaisingEvents = false;
process.Exited -= callback;
tokenRegistration?.Dispose();
tokenRegistration = null;
}
}
return tcs.Task;
}
private Process CreateNewProcess(string command, bool asRoot)
{
_log.LogDebug("Creating process: {Command}", command);
Process prc = new Process();
if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
{
string escapedCommand = command.Replace("\"", "\\\"");
// if as root, just sudo it
if (asRoot)
prc.StartInfo = new ProcessStartInfo("/bin/bash", $"-c \"sudo {escapedCommand}\"");
// if not as root, we need to open it as current user
// this may still run as root if the process is running as root
else
prc.StartInfo = new ProcessStartInfo("/bin/bash", $"-c \"{escapedCommand}\"");
}
else if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
prc.StartInfo = new ProcessStartInfo("CMD.exe", $"/C {command}");
if (asRoot)
prc.StartInfo.Verb = "runas";
}
else
throw new PlatformNotSupportedException($"{nameof(ExecuteAndWaitAsync)} is only supported on Windows and Linux platforms.");
prc.StartInfo.UseShellExecute = false;
prc.StartInfo.CreateNoWindow = true;
if (_log.IsEnabled(LogLevel.Trace))
{
_log.LogTrace("exec: {FileName} {Args}", prc.StartInfo.FileName, prc.StartInfo.Arguments);
_log.LogTrace("exec: as root = {AsRoot}", asRoot);
}
return prc;
}
I spent a lot of time (over span of months - literally) trying various changes to solve this issue - WaitForExitAsync was overhauled a lot, tried different ways of disposing. I attempted to call GC.Collect() periodically. Also tried running the application with both Server and Workstation GC mode.
As I mentioned earlier, I am pretty sure it's this code that leaks - if I don't call ExecuteAndWaitAsync, there's no memory leak. The result class is also not stored by the caller - it simply parses a value and uses it right away:
public async Task<TimeSpan> GetSystemIdleTimeAsync(CancellationToken cancellationToken = default)
{
ThrowIfNotLinux();
const string prc = "xprintidle";
TerminalResult result = await _terminal.ExecuteAndWaitAsync(prc, false, cancellationToken).ConfigureAwait(false);
if (result.HasErrors || !int.TryParse(result.Output, out int idleMs))
throw new InvalidOperationException($"{prc} returned invalid data.");
return TimeSpan.FromMilliseconds(idleMs);
}
private static void ThrowIfNotLinux()
{
if (!RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
throw new PlatformNotSupportedException($"{nameof(BacklightControl)} is only functional on Linux systems.");
}
Am I missing something? Is it Process class leaking, or the way I read output?
EDIT: As people in comments asked, I created minimum runnable code, basically fetching all relevant methods in a single class and execute in a loop. The code is available as a gist: https://gist.github.com/TehGM/c953b670ad8019b2b2be6af7b14807c2
I ran it both on my Windows machine and Raspberry Pi. On Windows, memory seemed stable, however on Raspberry Pi it was clearly leaking. I tried both xprintidle and ifconfig to make sure it's not an issue with xprintidle only. Tried both .NET Core 3.0 and .NET Core 3.1, and effect was largely the same.
