Invoking CloudRun endpoint from within itself

user_13047327 0
Viewed 71

Assuming there is a Flask web server that has two routes, deployed as a CloudRun service over GKE.

@app.route('/cpu_intensive', methods=['POST'], endpoint='cpu_intensive')
def cpu_intensive():
    #TODO: some actions, cpu intensive

@app.route('/batch_request', methods=['POST'], endpoint='batch_request')
def batch_request():
    #TODO: invoke cpu_intensive

A "batch_request" is a batch of many same structured requests - each one is highly CPU intensive and handled by the function "cpu_intensive". No reasonable machine can handle a large batch and thus it needs to be paralleled across multiple replicas. The deployment is configured that every instance can handle only 1 request at a time, so when multiple requests arrive CloudRun will replicate the instance. I would like to have a service with these two endpoints, one to accept "batch_requests" and only break them down to smaller requests and another endpoint to actually handle a single "cpu_intensive" request. What is the best way for "batch_request" break down the batch to smaller requests and invoke "cpu_intensive" so that CloudRun will scale the number of instances?

  • make http request to localhost - doesn't work since the load balancer is not aware of these calls.
  • keep the deployment URL in a conf file and make a network call to it?

Other suggestions?

google-cloud-run kubernetes load-balancing
2 Answers

With more detail, it's now clearer!!

You have 2 responsibilities

  • One to split -> Many request can be handle in parallel, no compute intensive
  • One to process -> Each request must be processed on a dedicated instance because of compute intensive process.

If your split performs internal calls (with localhost for example) you will be only on the same instance, and you will parallelize nothing (just multi thread the same request on the same instance)

So, for this, you need 2 services:

  • one to split, and it can accept several concurrent request
  • The second to process, and this time you need to set the concurrency param to 1 to be sure to accept only one request in the same time.

To improve your design, and if the batch processing can be asynchronous (I mean, the split process don't need to know when the batch process is over), you can add PubSub or Cloud Task in the middle to decouple the 2 parts.

And if the processing requires more than 4 CPUs 4Gb of memory, or takes more than 1 hour, use Cloud Run on GKE and not Cloud Run managed.

Last word: Now, if you don't use PubSub, the best way is to set the Batch Process URL in Env Var of your Split Service to know it.

user_113725930

I believe for this use case it's much better to use GKE rather than Cloud Run. You can create two kubernetes deployements one for the batch_request app and one for the cpu_intensive app. the second one will be used as worker for the batch_request app and will scale on demand when there are more requests to the batch_request app. I believe this is called master-worker architecture in which you separate your app front from intensive work or batch jobs.

user_100532760
Related