Spring Cloud Feign client with Hystrix circuit-breaker timeout defaults in 2 seconds

user_3764965 0
Viewed 5672

Reproducible through a project on GitHub: spring-cloud-feign-hystrix-timeout-problem

I am using Spring Boot 2.3.1.RELEASE with Spring Cloud Hoxton.SR6. Namely Feign client and Hystrix without Zuul and Eureka to fetch REST responses.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-dependencies</artifactId>
   <version>Hoxton.SR6</version>
   <type>pom</type>
   <scope>import</scope>
</dependency>

Then I use use the following dependencies on top of Spring Boot 2.3.1.RELEASE and Spring Cloud Hoxton.SR6:

  • org.springframework.boot:spring-boot-starter
  • org.springframework.boot:spring-boot-starter-web
  • org.springframework.cloud:spring-cloud-starter-openfeign
  • org.springframework.cloud:spring-cloud-starter-netflix-hystrix

I enable @EnableFeignClients and @EnableCircuitBreaker and use a @FeignClient with a simple fallback to log and rethrow an exception:

@FeignClient(name="my-feign", url = "${feign.url}", fallbackFactory = MyFallbackFactory.class) {
public interface MyFeignClient {

    @PostMapping(value = "/api/dto")
    postDto(@RequestBody Dto dto);
}

With the following application.yml the timeout is around 1 second because Hystrix defaults to the very same value:

feign:
  hystrix:
    enabled: true
  client:
    config:
      default:
        connectTimeout: 5000
        readTimeout: 5000

11:52:05.493 INFO 10200 --- [nio-8060-exec-2] com.mycompany.rest.MyController : Calling REST right now!

11:52:06.538 ERROR 24044 --- [nio-8060-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is com.netflix.hystrix.exception.HystrixRuntimeException: MyFeignClient#postDto(Dto) timed-out and fallback failed.] with root cause

What I tried?

As long as I add the following lines to increase the timeouts to 60 seconds, the timeout become effectively around 2 seconds:

hystrix:
  command:
    default:
      execution:
        timeout:
          enabled: true
        isolation:
          thread:
            timeoutInMilliseconds: 60000

11:53:33.590 INFO 16052 --- [nio-8060-exec-2] com.mycompany.rest.MyController : Calling REST right now!

11:53:35.614 ERROR 16052 --- [nio-8060-exec-2] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is com.netflix.hystrix.exception.HystrixRuntimeException: MyFeignClient#postDto(Dto) failed and fallback failed.] with root cause

The call falls into fallback right in 2 seconds as long as the Hystrix read/connect timeouts have been increased. However I expect to achieve 5 seconds as long as I declared it in feign.client.config.default... timeouts. I feel I am missing another configuration.

Q: How to increase the timeout?

Edit:

hystrix spring-cloud-feign spring-cloud spring-boot java
2 Answers

Your configuration is correct and what you are describing is expected behavior. The issue here is that exception with Connection refused isn't thrown after your configured timeout -- 10 seconds. Instead, it is thrown immediately after Java's internal socket implementation finds that server host is not reachable. In simplest case, the server you are calling isn't up and running.

As to why there is a second increase after you set the hystrix timeout, you can debug hystrix's call stack to find out that HystrixRuntimeException isn't being thrown in the same order.

Before your custom hystrix timeout, hystrix had a default timeout of 1 second, which means this runtime exception is always thrown once one second since execution has lapsed, regardless whether the request succeeds or fails. So in your case, Connection refused could very likely happen after HystrixTimeoutException. After you set the timeout be to longer than that of feign client, HystrixTimeoutException only gets created after a feign exception is thrown (due to "Connection refused"), hence the delay.

// 'cause' should be different due to timing
public HystrixRuntimeException(... Exception cause, Throwable fallbackException)

To simulate the timeout, I'd say you could force a timeout on the server, such as Thread.sleep(6000) to halt the execution on server side, or simply do a breakpoint while on debugger.

user_44933520

One thing you can try is to define the configuration property of the Hystrix timeout in a single line (I know that it should be equivalent to the nested definition you proposed in your YAML file, but sometimes I came across properties that only were loaded in this way). As you can see in their tests, this is the way Spring Cloud itself configures Hystrix timeout:

hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds: 5000

Also, although you indicated that you are not using Zuul or Eureka, please, verify if you are using Ribbon in any way, there is some dependencies (see https://github.com/Netflix/Hystrix/issues/1778 and https://github.com/spring-cloud/spring-cloud-netflix/issues/1324) between the Hystrix and Ribbon timeouts.

If it is the case, you must configure the timeouts of both libraries:

hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds: 5000
ribbon:
  ConnectTimeout: 5000
  ReadTimeout: 5000
user_139424480
Related