How to deal with Negsignal.SIGSEGV in cloud composer

Viewed 54

I recently created a new production cloud composer environment and migrated my existing dags to it and have been getting the error INFO - Task exited with return code Negsignal.SIGSEGV on some tasks that work perfectly fine in my dev environment (also on cloud composer). The production environment is loaded from a snapshot of the dev environment so they should be identical. Is there anything configuration wise that might not have come over in the snapshot that I can look into so I can try and get this resolved?

For added context, I have upgraded the scheduler, web server and worker specs in the prod environment to be more than the dev environment and this is still causing the same issue.

1 Answers

Are the instances the same size? This message means that some tasks run out of resources so you have either to scale up or modify your code so it is more frugal and you don't run out of resources.

Also have you checked how the dags are scheduled? Maybe you have many dags running at the same time and they use up all your resources.

Related