I'm working on developing both web-client and API server. I've been doing some research regarding default timeout, some are at 800ms, others 1200ms. However, I can't find the reason behind the arbitrary number. Can someone help me regarding this? An explanation behind the arbitrary number would be a great help.
Thanks,
TLDR: Please see paragraph starting with "The arbitrary number" in bold below. The rest is just extra info on the topic.
Although you might know this or have already read this in your research, I can share the following ideas:
Typically the timeout is set depending on the expected complexity of a query, the amount of data to be processed, and the expected load of the system when the query occurs (or any other expected operation that may require attention in terms of modifying a timeout). Also, this can be based on something like the number of requests an API makes to other APIs to handle an incoming request(s) and what those expectations might be.
The arbitrary number ("best guess" of whoever developed the software) would typically be expected when planning for a "most requests should complete in some fraction of this time if there is no issue regardless of what happens" or "this isn't anything to worry about" type of scenario. Hence the default values for timeouts are pretty much based on the assumption that they represent the vast majority of "acceptable" completed requests where no issue is present. It is typically set somewhere between "this should be plenty of time" and "there is most likely something terribly wrong with this request, let's end it" and most successful requests pass this test by "default".
In the case that you have operations that may take several minutes and you expect that this can occur without an actual issue being present, you may want to set the timeout higher than the default so your requests don't timeout when there is no actual problem (for example, most commercial APIs have constraints on the number of requests and time in which they must complete so problematic requests don't clog up the system and other reasons as seen by their developers).
Thus, there really isn't a great answer or standard to this aside from just taking a look at the amount of data/requests to be processed, planning for a reasonable ebb and flow of server load, level of optimization of your code compared to the expected load, and so on... It's almost like error-handling but for things that you don't know might happen yet (such as unexpected bugs) but based on things you already know about your system and its expected usage.
Generally, you won't have many scenarios where the timeout really matters all that much but you always want to have one (at least the default) to prepare for the unexpected.
I found the following article that talks about the topic and some of what I mentioned as well if you haven't seen it already:
https://medium.com/@masnun/always-use-a-timeout-for-http-requests-de4da538b9e3
tl;dr - According to SLA [ Service Level Agreement ] mostly. If not, try to optimize the code as much as possible to bring down the time it takes to give out the response in terms of milliseconds
I'll put the answer in layman's terms since it really depends on various factors.
Let's assume you have an API and it performs some operation and gives the result back. It's quite simple and you'll get the response for that under some milliseconds if they don't perform any complex operations.
And when we move into a more and more complex system where one API talks to another, it adds up the time, and the worst-case scenario, first API which started the request might get the final response after 5 seconds, 30 seconds, or even 60 seconds depending on the number of API calls and how good the system is designed.
And we are only considering the happy flow. What if something goes wrong in one of the APIs that gets called internally?
To avoid this bad experience, the clients will make an SLA that requires the company/developers to design the code in such a way that it gives the response within a certain acceptable range.
I came across this conversation once on Google Groups conversation and it might provide some insight.
So to answer the question about the acceptable range, If you don't have an SLA, try to optimize the code as much as possible to bring down the time it takes to give out the response in terms of milliseconds.
Generally 1 Second is considered acceptable. The reason for this and why the suggested numbers vary so much is most APIs have a lockout if you send requests to fast. However, some APIs will let you send requests faster. In my experience all of the APIs I have seen request a 1s(1000ms) delay between requests to prevent overload/accidental DDOS and have a timeout of 30-60sec.
Edit: It is important to mention to not let another request from the same IP be answered if the first one is still waiting as this would make a DDoS easy
