Google DriveService C# leaking HttpConnection objects

user_10591853 0
Viewed 328

I have an application that utilizes Google's Drive service using the C# SDK. I am using Service Account credentials, with impersonation to specify the user I am acting on behalf of.

The issue I'm having is that I need to create a new DriveService for each impersonated user, and each of those DriveService isntances creates 2 HttpConnection objects that are NOT cleaned up when I Dispose() the DriveService. I have repro'd this in a unit test. I create the DriveService in this method:

public DriveService GetDriveService(string email)
    {
        var serviceAccountEmail = "service account email";
        var privateKey = "private key";

        var serviceAcctCreds =  new ServiceAccountCredential(new ServiceAccountCredential.Initializer(serviceAccountEmail)
        {
            User = email,
            Scopes = "scopes" }), 
        }.FromPrivateKey(privateKey));

        var driveService = new DriveService(new BaseClientService.Initializer
        { 
            HttpClientInitializer = serviceAcctCreds,
            ApplicationName = "app name",
        });

        return driveService;
    }

Then use it as so:

public async Task DoTheGDriveMemoryTest()
        {
            var userEmails = new List<string>
            {
                 "email1@example.com", "email2@example.com"
            };
            var i = 1;
            while (true)
            {
                foreach (var email in userEmails)
                {
                    //recommend using Fiddler Autoresponders here
                    using (var service = GetDriveService(email))
                    {
                        var request = service.Changes.List("delta id");                        
                        try
                        {
                            var result = await request.ExecuteAsync();
                        }
                        catch (Exception e)
                        {
                            //do nothing
                        }
                    }
                }
            }
        }

This results in an ever-increasing number of HttpConnection objects held in memory, that are not cleaned up with Garbage Collection.

google-drive-api google-api-dotnet-client google-api
2 Answers

If DriveService is using HttpClient under the covers, you may be running into dispose not working on a shared object, or connections being held until a timeout.

But HttpClient is different. Although it implements the IDisposable interface it is actually a shared object.

See https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/

user_92267370

As Mike M has stated, this is a known problem with HttpClient in that its Dispose() method doesn't really dispose of the instance.

Unfortunately because the way the Google.Apis.Auth library ties together the HttpClient and the credentials, I can't see a nice workaround for this.

Here is a very hacky workaround, that allows you to share one HttpClient between all the RPCs. Define a custom IHttpClientFactory:

class SharedHttpClientFactory : IHttpClientFactory
{
    public SharedHttpClientFactory()
    {
        _handler = new ConfigurableMessageHandler(new HttpClientHandler());
        _client = new ConfigurableHttpClient(_handler);
    }

    private readonly ConfigurableMessageHandler _handler;
    private readonly ConfigurableHttpClient _client;
    private Action _clearHandlers = null;

    public ConfigurableHttpClient CreateHttpClient(CreateHttpClientArgs args)
    {
        _clearHandlers?.Invoke();
        var executeInterceptors = args.Initializers.OfType<IHttpExecuteInterceptor>().ToList();
        var unsuccessfulResponseHandlers = args.Initializers.OfType<IHttpUnsuccessfulResponseHandler>().ToList();
        _clearHandlers = () =>
        {
            foreach (var executeInterceptor in executeInterceptors)
            {
                _handler.RemoveExecuteInterceptor(executeInterceptor);
            }
            foreach (var unsuccssfulResponseHandler in unsuccessfulResponseHandlers)
            {
                _handler.RemoveUnsuccessfulResponseHandler(unsuccssfulResponseHandler);
            }
        };
        foreach (var executeInterceptor in executeInterceptors)
        {
            _handler.AddExecuteInterceptor(executeInterceptor);
        }
        foreach (var unsuccssfulResponseHandler in unsuccessfulResponseHandlers)
        {
            _handler.AddUnsuccessfulResponseHandler(unsuccssfulResponseHandler);
        }
        _handler.ApplicationName = args.ApplicationName;
        return _client;
    }
}

Create a single instance of this factory:

var factory = new SharedHttpClientFactory();

Then create every DriveService using this factory:

var client = new StorageService(new Google.Apis.Services.BaseClientService.Initializer
{
    HttpClientFactory = factory,
    HttpClientInitializer = serviceAcctCreds
});

Caveats abound:

  • Do not dispose of the DriveService as this will dispose of the underlying shared HttpClient.
  • All the DriveService instances share the same credentials; these credentials are changed each time a new DriveService is instantiated. This means that all previous instances also have their credentials updated.
  • The credential passed into the initializer HttpClientInitializer property must be a ServiceCredential instance (as it is in your code). This won't work if it's a GoogleCredential instance.
  • This whole thing isn't really advised, is definitely a nasty hack, but I can't see a better/simpler workaround.
Chris_443600
Related