Fail ansible play fast on concurrent async task failure

I've got an ansible play (split across several included playbooks) which looks a bit like this:

- name: Build packer images
  <snip>
  register: packer_run
  async: 2700 # 45 minutes
  poll: 0
- name: Do other stuff
  <snip>
...
- name: Check Packer build finished
  async_status:
    jid: "{{ packer_run.ansible_job_id }}"
  register: packer_result
  until: packer_result.finished
  retries: 30 # allow 15 mins
  delay: 30

Note the async task sets poll=0 to allow subsequent tasks to run concurrently. However even if the async task fails immediately, ansible only fails once it's finished the subsequent tasks and checks on the async task status. Which is really annoying because it can take ~30 minutes to find out that the async task failed 30s in to the run.

Is there a way to preserve the concurrent behaviour but have ansible fail as soon as the async task fails?

I guess the obvious answer is "don't do the concurrency with ansible" but as this is running in a github actions workflow (which doesn't allow concurrent steps) using this approach provides a reasonable workaround apart from this "slow failure" problem.