Using celery 4.4.2, I have a group of tasks that connects to remote devices and gathers data, when that is completed the results are collated then emailed with the callback task.
However if gathering the data of one of the remote devices fails, the call back also fails. I've read that using link_error should resolve this but I'm not sure if my implementation, I have executed the below but it still failed
for device in device_ids:
task_count +=1
tasks.append(config_auth.s(device.id, device.hostname, device.get_scripts_ip()))
callback = email_auth_results.s().set(link_error=error_handler.s())
tasks = group(tasks)
r = chord(tasks)(callback)
return '{} tasks sent to queue'.format(task_count)
@app.task
def error_handler(uuid):
result = AsyncResult(uuid)
exc = result.get(propagate=False)
print('Task {0} raised exception: {1!r}\n{2!r}'.format(
uuid, exc, result.traceback))
original error:
celery.exceptions.ChordError: Dependency 5ffc10c9-edc7-4b91-a660-08c372c60ab2 raised NetmikoTimeoutException('Connection to device timed-out')
I still want to log that a task failed so I can see the failures in flower, but I want to ignore the failures or append the results so they just say failure and I can see it in the email results
Thanks