I wrote a durable function in order to scale data queries to some API internal to our organization. The orchestration function spreads queries across activity functions only two of which will be active at once (in order not to flood the server).
A colleague accidentally created a few instances of the durable function which would create roughly 70 activity functions. Because the output is not needed at this point, I want to terminate both the orchestration and all running and pending activity functions.
In PowerShell, using the HTTP APIs, I queried the id's of all pending and running instances of the durable function on this function app
Invoke-RestMethod -Uri "https://{fapp}.azurewebsites.net/runtime/webhooks/durableTask/instances?taskHub=$taskHub&connection=$connectionName&code=$systemKey&createdTimeFrom=$timestampFrom&createdTimeTo=$timestampTo&runtimeStatus=Pending,Running"
and terminated each one manually
Invoke-RestMethod -Uri "https://{fapp}.azurewebsites.net/runtime/webhooks/durabletask/instances/$instanceId/terminate?reason=$reason&taskHub=$taskhub&connection=$connectionName&code=$systemKey" -Method "Post"
As expected, executing the first query again no longer returns any information as no instances should be pending and running at this point. Also, the returned object when filtering on runtimeStatus=Terminated confirms that my queries were successful.
However, I still see the output generated from the functions (in my case, files are generated in a blob storage container). Hoping that I had deleted all pending activity functions but not currently running ones, I waited for the output worth of one activity function to be generated. However, this amount was exceeded which is why it seems that new activity functions are started.
Questions:
- Why are there still queued and/or running activity functions?
- How can I identify their instance id's and terminate them?
- The easiest option would be to terminate all queued and running instances for this function app. Is that possible? How?
Note: The function app is currently turned off to identify a solution.