Semaphore slim to handle throttling per time period

user_547239 0
Viewed 755

I have a requirement from a client, to call their API, however, due to the throttling limit, we can only make 100 API calls in a minute. I am using SemaphoreSlim to handle that, Here is my code.

 async Task<List<IRestResponse>> GetAllResponses(List<string> locationApiCalls) 
 {
     var semaphoreSlim = new SemaphoreSlim(initialCount: 100, maxCount: 100);
     var failedResponses = new ConcurrentBag<IReadOnlyCollection<IRestResponse>>();
     var passedResponses = new ConcurrentBag<IReadOnlyCollection<IRestResponse>>();

     var tasks = locationApiCalls.Select(async locationApiCall =>
     {
          await semaphoreSlim.WaitAsync();
          try
          {
              var response = await RestApi.GetResponseAsync(locationApi);
              if (response.IsSuccessful)
              {
                  passedResponses.Add((IReadOnlyCollection<IRestResponse>)response);
              }
              else
              {
                 failedResponses.Add((IReadOnlyCollection<IRestResponse>)response);
              }
           }
           finally
           {
               semaphoreSlim.Release();
           }
     });

     await Task.WhenAll(tasks);
     var passedResponsesList = passedResponses.SelectMany(x => x).ToList();
}

However this line

var passedResponsesList = passedResponses.SelectMany(x => x).ToList();

never gets executed and I see Lots of failedResponses as well, I guess I have to add Task.Delay (for 1 minute) somewhere in the code as well.

throttling semaphore c#
1 Answers

You need to keep track of the time when each of the previous 100 requests was executed. In the sample implementation below, the ConcurrentQueue<TimeSpan> records the relative completion time of each of these previous 100 requests. By dequeuing the first (and hence earliest) time from this queue, you can check how much time has passed since 100 requests ago. If it's been less than a minute, then the next request needs to wait for the remainder of the minute before it can be executed.

async Task<List<IRestResponse>> GetAllResponses(List<string> locationApiCalls)
{
    var semaphoreSlim = new SemaphoreSlim(initialCount: 100, maxCount: 100);
    var total = 0;
    var stopwatch = Stopwatch.StartNew();
    var completionTimes = new ConcurrentQueue<TimeSpan>();

    var failedResponses = new ConcurrentBag<IReadOnlyCollection<IRestResponse>>();
    var passedResponses = new ConcurrentBag<IReadOnlyCollection<IRestResponse>>();

    var tasks = locationApiCalls.Select(async locationApiCall =>
    {
        await semaphoreSlim.WaitAsync();

        if (Interlocked.Increment(ref total) > 100)
        {
            completionTimes.TryDequeue(out var earliest);
            var elapsed = stopwatch.Elapsed - earliest;
            var delay = TimeSpan.FromSeconds(60) - elapsed;
            if (delay > TimeSpan.Zero)
                await Task.Delay(delay);
        }

        try
        {
            var response = await RestApi.GetResponseAsync(locationApi);
            if (response.IsSuccessful)
            {
                passedResponses.Add((IReadOnlyCollection<IRestResponse>)response);
            }
            else
            {
                failedResponses.Add((IReadOnlyCollection<IRestResponse>)response);
            }
        }
        finally
        {
            completionTimes.Enqueue(stopwatch.Elapsed);
            semaphoreSlim.Release();
        }
    });

    await Task.WhenAll(tasks);
    var passedResponsesList = passedResponses.SelectMany(x => x).ToList();
}

If you're calling this method from the UI thread of a WinForms or WPF application, remember to add ConfigureAwait(false) to its await statements.

user_11497730
Related