Snowflake External Function - Parallel/Concurrent

Question

I have a question regarding parallelism of snowflake external function. Here is my understanding:

external function sends a set of rows to be processed to a remote service (in my case azure function httpTrigger).
remote service handles those rows and sends output - one output for each row.

My question is does external function send multiple httpTrigger requests asynchronously sending a set of rows for each request. Is concurrency pre-built in the snowflake external function request? I know that concurrency in Azure functions to scale. Could you please guide me.

user_132438 · Answer

From the docs:

Furthermore, the concurrency estimate should take into account that Snowflake can parallelize external function calls. A single query from a single user might cause multiple calls to the remote service in parallel. Several factors affect the number of concurrent calls from Snowflake to a proxy service or remote service, including:

https://docs.snowflake.com/en/sql-reference/external-functions-implementation.html#concurrency

Snowflake External Function - Parallel/Concurrent

1 Answers