Cancel Synapse pipeline from the pipeline itself

user_11042848 0
Viewed 355

I have a pipeline I need to cancel if it runs for too long. It could look something like this: enter image description here

So in case the work takes longer than 10000 seconds, the pipeline will fail and cancel itself. The thing is, I can't get the web activity to work. I've tried something like this: https://docs.microsoft.com/es-es/rest/api/synapse/data-plane/pipeline-run/cancel-pipeline-run

But it doesn't even work using the 'Try it' thing. I get this error:

{"code": "InvalidTokenAuthenticationAudience", "message": "Token Authentication failed with SecurityTokenInvalidAudienceException - IDX10214: Audience validation failed. Audiences: '[PII is hidden]'. Did not match: validationParameters.ValidAudience: '[PII is hidden]' or validationParameters.ValidAudiences: '[PII is hidden]'."}

Using this URL:

POST https://{workspacename}.dev.azuresynapse.net/pipelineruns/729345a-fh67-2344-908b-345dkd725668d/cancel?api-version=2020-12-01

Also, using ADF it seemed quite easy to do this: https://cloudsafari.ca/2020/09/data-engineering/Azure-DataFactory-Cancel-Pipeline-Run

Including authentication using a Managed Identity, which in the case of Synapse I'm not too sure would resource should I use. Any idea on how to achieve what I want or if I'm doing something wrong?

azure-synapse pipeline api
1 Answers

Your URL is correct. Just check the following and then it should work:

  1. Add the MSI of the workspace to the workspace resource itself with Role = Contributor

  2. In the web activity, set the Resource to "https://dev.azuresynapse.net/" (without the quotes, obviously) This was a bit buried in the docs, see last bullet of this section here: https://docs.microsoft.com/en-us/rest/api/synapse/#common-parameters-and-headers

NOTE: the REST API is unable to cancel pipelines run in DEBUG in Synapse (you'll get an error response saying pipeline with that ID is not found). This means for it to work, you have to first publish the pipelines and then trigger them.

user_59387590
Related