Cleanest way to write retry logic?

Viewed 319401

Occasionally I have a need to retry an operation several times before giving up. My code is like:

int retries = 3;
while(true) {
  try {
    DoSomething();
    break; // success!
  } catch {
    if(--retries == 0) throw;
    else Thread.Sleep(1000);
  }
}

I would like to rewrite this in a general retry function like:

TryThreeTimes(DoSomething);

Is it possible in C#? What would be the code for the TryThreeTimes() method?

30 Answers

Allowing for functions and retry messages

public static T RetryMethod<T>(Func<T> method, int numRetries, int retryTimeout, Action onFailureAction)
{
 Guard.IsNotNull(method, "method");            
 T retval = default(T);
 do
 {
   try
   {
     retval = method();
     return retval;
   }
   catch
   {
     onFailureAction();
      if (numRetries <= 0) throw; // improved to avoid silent failure
      Thread.Sleep(retryTimeout);
   }
} while (numRetries-- > 0);
  return retval;
}

Building on the previous work, I thought about enhancing the retry logic in three ways:

  1. Specifying what exception type to catch/retry. This is the primary enhacement as retrying for any exception is just plain wrong.
  2. Not nesting the last try in a try/catch, achieving slightly better performance
  3. Making it an Action extension method

    static class ActionExtensions
    {
      public static void InvokeAndRetryOnException<T> (this Action action, int retries, TimeSpan retryDelay) where T : Exception
      {
        if (action == null)
          throw new ArgumentNullException("action");
    
        while( retries-- > 0 )
        {
          try
          {
            action( );
            return;
          }
          catch (T)
          {
            Thread.Sleep( retryDelay );
          }
        }
    
        action( );
      }
    }
    

The method can then be invoked like so (anonymous methods can be used as well, of course):

new Action( AMethodThatMightThrowIntermittentException )
  .InvokeAndRetryOnException<IntermittentException>( 2, TimeSpan.FromSeconds( 1 ) );

I have two implementations of this pattern using Polly. One is async.

My synchronous method is based on this answer by Erik Bergstedt

public static T Retry<T>(Func<T> action, TimeSpan retryWait, int retryCount = 0)
{
    PolicyResult<T> policyResult = Policy
        .Handle<ApiException>(ex => ex.ResponseCode == (int)HttpStatusCode.TooManyRequests)
        .WaitAndRetry(retryCount, retryAttempt => retryWait)
        .ExecuteAndCapture(action);

    if (policyResult.Outcome == OutcomeType.Failure)
    {
        throw policyResult.FinalException;
    }

    return policyResult.Result;
}

Async:

public static async Task<T> RetryAsync<T>(Func<Task<T>> action, TimeSpan retryWait, int retryCount = 0)
{
    PolicyResult<T> policyResult = await Policy
        .Handle<ApiException>(ex => ex.ResponseCode == (int)HttpStatusCode.TooManyRequests)
        .WaitAndRetryAsync(retryCount, retryAttempt => retryWait)
        .ExecuteAndCaptureAsync(action);

    if (policyResult.Outcome == OutcomeType.Failure)
    {
        throw policyResult.FinalException;
    }

    return policyResult.Result;
}

It would also be easy to allow an exception type to be passed in as well as the lambda for the exception type.

This method allows retries on certain exception types (throws others immediately).

public static void DoRetry(
    List<Type> retryOnExceptionTypes,
    Action actionToTry,
    int retryCount = 5,
    int msWaitBeforeEachRety = 300)
{
    for (var i = 0; i < retryCount; ++i)
    {
        try
        {
            actionToTry();
            break;
        }
        catch (Exception ex)
        {
            // Retries exceeded
            // Throws on last iteration of loop
            if (i == retryCount - 1) throw;

            // Is type retryable?
            var exceptionType = ex.GetType();
            if (!retryOnExceptionTypes.Contains(exceptionType))
            {
                throw;
            }

            // Wait before retry
            Thread.Sleep(msWaitBeforeEachRety);
        }
    }
}
public static void DoRetry(
    Type retryOnExceptionType,
    Action actionToTry,
    int retryCount = 5,
    int msWaitBeforeEachRety = 300)
        => DoRetry(new List<Type> {retryOnExceptionType}, actionToTry, retryCount, msWaitBeforeEachRety);

Example usage:

DoRetry(typeof(IOException), () => {
    using (var fs = new FileStream(requestedFilePath, FileMode.Create, FileAccess.Write))
    {
        fs.Write(entryBytes, 0, entryBytes.Length);
    }
});

Here's an async/await version that aggregates exceptions and supports cancellation.

/// <seealso href="https://docs.microsoft.com/en-us/azure/architecture/patterns/retry"/>
protected static async Task<T> DoWithRetry<T>( Func<Task<T>> action, CancellationToken cancelToken, int maxRetries = 3 )
{
    var exceptions = new List<Exception>();

    for ( int retries = 0; !cancelToken.IsCancellationRequested; retries++ )
        try {
            return await action().ConfigureAwait( false );
        } catch ( Exception ex ) {
            exceptions.Add( ex );

            if ( retries < maxRetries )
                await Task.Delay( 500, cancelToken ).ConfigureAwait( false ); //ease up a bit
            else
                throw new AggregateException( "Retry limit reached", exceptions );
        }

    exceptions.Add( new OperationCanceledException( cancelToken ) );
    throw new AggregateException( "Retry loop was canceled", exceptions );
}

Retry helper: a generic java implementation that contains both returnable and void type retries.

import java.util.function.Supplier;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryHelper {
  private static final Logger log = LoggerFactory.getLogger(RetryHelper.class);
  private int retryWaitInMS;
  private int maxRetries;

  public RetryHelper() {
    this.retryWaitInMS = 300;
    this.maxRetries = 3;
  }

  public RetryHelper(int maxRetry) {
    this.maxRetries = maxRetry;
    this.retryWaitInMS = 300;
  }

  public RetryHelper(int retryWaitInSeconds, int maxRetry) {
    this.retryWaitInMS = retryWaitInSeconds;
    this.maxRetries = maxRetry;
  }

  public <T> T retryAndReturn(Supplier<T> supplier) {
    try {
      return supplier.get();
    } catch (Exception var3) {
      return this.retrySupplier(supplier);
    }
  }

  public void retry(Runnable runnable) {
    try {
      runnable.run();
    } catch (Exception var3) {
      this.retrySupplier(() -> {
        runnable.run();
        return null;
      });
    }

  }

  private <T> T retrySupplier(Supplier<T> supplier) {
    log.error("Failed <TASK>, will be retried " + this.maxRetries + " times.");
    int retryCounter = 0;

    while(retryCounter < this.maxRetries) {
      try {
        return supplier.get();
      } catch (Exception var6) {
        ++retryCounter;
        log.error("<TASK> failed on retry: " + retryCounter + " of " + this.maxRetries + " with error: " + var6.getMessage());
        if (retryCounter >= this.maxRetries) {
          log.error("Max retries exceeded.");
          throw var6;
        }

        try {
          Thread.sleep((long)this.retryWaitInMS);
        } catch (InterruptedException var5) {
          var5.printStackTrace();
        }
      }
    }

    return supplier.get();
  }

  public int getRetryWaitInMS() {
    return this.retryWaitInMS;
  }

  public int getMaxRetries() {
    return this.maxRetries;
  }
}

Usage:

    try {
      returnValue = new RetryHelper().retryAndReturn(() -> performSomeTask(args));
      //or no return type:
      new RetryHelper().retry(() -> mytask(args));
    } catch(Exception ex){
      log.error(e.getMessage());
      throw new CustomException();
    }

I've implemented an async version of the accepted answer like so - and it seems to work nicely - any comments?


        public static async Task DoAsync(
            Action action,
            TimeSpan retryInterval,
            int maxAttemptCount = 3)
        {
            DoAsync<object>(() =>
            {
                action();
                return null;
            }, retryInterval, maxAttemptCount);
        }

        public static async Task<T> DoAsync<T>(
            Func<Task<T>> action,
            TimeSpan retryInterval,
            int maxAttemptCount = 3)
        {
            var exceptions = new List<Exception>();

            for (int attempted = 0; attempted < maxAttemptCount; attempted++)
            {
                try
                {
                    if (attempted > 0)
                    {
                        Thread.Sleep(retryInterval);
                    }
                    return await action();
                }
                catch (Exception ex)
                {
                    exceptions.Add(ex);
                }
            }
            throw new AggregateException(exceptions);
        }

And, call it simply like this:

var result = await Retry.DoAsync(() => MyAsyncMethod(), TimeSpan.FromSeconds(5), 4);
Related