Why are Vulkan command buffers still "alive" after being submitted to the gpu?

Viewed 177

The Vulkan tutorial that I'm working through (https://vkguide.dev/docs/chapter-1/vulkan_command_flow/) states: "Once a command buffer has been submitted, it’s still “alive”, and being consumed by the GPU, at this point it is NOT safe to reset the command buffer yet."
I had assumed that resetting the command buffer would be fine after submitting it, thinking that the data was now safely over on the gpu (and that the command buffer itself lives on the cpu). Is the issue that the actual transfer of data doesn't occur as soon as vkQueueSubmit is called?

2 Answers

The right term is "pending". When you submit a command buffer it goes to the "pending" state.

This is what the spec says:

Queue submission of a command buffer changes the state of a command buffer from the executable state to the pending state. Whilst in the pending state, applications must not attempt to modify the command buffer in any way - as the device may be processing the commands recorded to it. Once execution of a command buffer completes, the command buffer either reverts back to the executable state, or if it was recorded with VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT, it moves to the invalid state. A synchronization command should be used to detect when this occurs.

You can only reset a command buffer when it's in one of the following states:

  • Recording: yes, it's possible to reset before calling vkEndCommandBuffer.
  • Executable: right after vkEndCommandBuffer or after the execution has completed.
  • Invalid: when a resource used by the command buffer has been destroyed.

You can read about the whole command buffer lifecycle here (there is a handy diagram):

https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/chap6.html#commandbuffers-lifecycle

Thinking that the data was now safely over on the gpu (and that the command buffer itself lives on the cpu).

This is a somewhat "old fashioned" way of thinking.

Most (all?) modern hardware can directly share memory between the CPU and GPU to some extent, although there are some platform-dependent limitations.

On most mobile and integrated graphics platforms there is no such thing thing as dedicated graphics memory - the CPU and GPU are just sharing the same system RAM. Even on discrete graphics cards there is normally a way to access a subset of the system RAM directly from the GPU over PCIe.

Vulkan is designed to exploit this - there is no point wasting memory and CPU cycles creating copies you don't actually need. Resources in Vulkan are assumed to be directly referenced by the GPU, and you can't delete them until the are not longer referenced by a queued command.

Related