How to cancel previous request in FastAPI

user_16449265 0
Viewed 1559

I have a FastAPI app with an end point that may have to do a long computation. As a replacement of the actual code let me use this mock example, in which I am trying to keep the features that I think are relevant.

from fastapi import FastAPI
from fastapi.responses import JSONResponse
import subprocess

app = FastAPI()

@app.put("/expensive")
def expensive():
  command = "for x in $(seq 1 10000); do echo ${x}; done"
  subprocess.run(command, shell=True, check=True)
  return JSONResponse(content=True)


if __name__ == "__main__":
  import uvicorn
  uvicorn.run(
    "deleteme:app",
    host='0.0.0.0',
    port=12345,
    reload=True
  )

As in the mock example, the actual code run a third party software that does a computation that could, in some cases, take a long time.

When I send, with Postman, a PUT request, the app will start printing the natural numbers from 1 to 10000. I can cancel the request in Postman. However, the computation in the app will continue.

Question: I would like the user to be able to have a mechanism to cancel a specific request made to http://0.0.0.0:12345/expensive. What are ways to do this?

My lack of experience with Web APIs also causes that I am not finding the right key words to search about the subject. For example, combinations of fastapi request abort, or web api request cancel, don't seem to lead to posts discussing this topic.

A mechanism that I can think about could be:

  1. Send in the initial request an ID request_id.
  2. When the third party software is run, get its PID and store the key:value pair request_id:PID.
  3. Have a second end point to receive cancel requests. The user would submit the request_id of the request that they would like to cancel.
  4. Use the request_id to then kill the subprocess with the corresponding PID.

This seem convoluted.

fastapi python
3 Answers

There's a way to "cancel"(or kill) the request(or subprocess). But it isn't easy.

First, Canceling the actual request is impossible because it is a FastAPI process. I don't know how exactly FastAPI runs internally, but it's a low-level story.

Second, killing the subprocess is possible. You can find the subprocess's PID and the only thing left to do is kill the process. It's not fancy, though.

There are lots of alternative solutions.

  1. Set timeout: subprocess.run() has timeout parameter. It will raise subprocess.TimeoutExpired

  2. Use BackgroundTasks: FastAPI(Starlette, actually) has nice feature, Background Tasks

  3. Separate your service with a 3rd party: It's recommended if your process(or another app) takes a very long time. Celery is for doing this. Many best practices are there, and I think it is a proper way what you think.

user_77770160

In my experience, long running requests have always resulted in one problem or another. A few 'heavy' requests should not be allowed to prevent the workers from serving numerous small requests. I would suggest a polling based mechanism with an ETA feature.

  1. API to initiate jobs. Returns a JOB-ID (a random string that is mapped to PID and other resources internally) and an expected time taken for processing.
  2. API to get the job results. Returns either results or error with details on how long more to wait or if the job failed.
  3. API to kill a job and free resources.
  4. A cleanup daemon process. Since we do not want to keep unfetched job results for ever.

Another option is to switch to websockets. With a pingpong we can detect the connection failure and stop the process when needed.

user_51478340

The very brief version is what you're looking for is probably "multiprocessing". The session between your User/Client and the Server/REST API is like an active bit of python code and will wait for however long it takes "/expensive" to complete. In order to avoid making the User wait for /expensive you would need to push that request to a background process (multiprocessing). How you do that leads into much more complex answers. Fast API has a BackGround feature that I am not familiar with, but implies it has some advice for you about handling processing in the background.

user_33857990
Related