Potential missing WAW

[MennoVink]

Describe the situation in which you encountered the missing validation
VK_LAYER_KHRONOS_validation + validate_sync + gpuav_enable + exclusive queue ownership resources
I have a setup in which i'm expecting a WAW error, but i'm not getting it. Now that could be good, but i'm not entirely convinced that my code is actually valid. Here's the steps i'm executing:

On the Transfer QF:

1. vkCmdCopyBuffer
2. Release barrier ALL_TRANSFER / MEMORY_WRITE -> ALL_COMMANDS / MEMORY_READ
3. Signal semaphore

On the Compute QF:

1. Wait semaphore
2. Acquire barrier ALL_TRANSFER / MEMORY_WRITE -> ALL_COMMANDS / MEMORY_READ
3. Signal fence

On Host:

1. WaitForFences

On the Compute QF:

1. vkCmdBindDescriptorSets (containing that buffer)
2. vkCmdDispatch (writing to the buffer)

What i'm worried about is that the QFOT barrier doesn't properly prevent compute memory writes from happening before the resource has been acquired on the Compute QF or the initial data has been copied.

Valid Usage IDs requested
To be honest i have not done the full evidence gathering yet. If you agree this looks suspicious, I'd be willing to do some digging to see if something is listed in the specs about this.

Additional Context
I have a renderdoc capture of a simplified app doing almost the same thing. The difference with the more involved app is that this capture doesn't use writeonly on the output buffer. One could then argue that since the shader is also reading and writing is dependent on reads that it's still valid. Using this example for simplicity of illustration though, the other app also doesn't give the validation error.

https://drive.google.com/file/d/1OnXxckJuXMOtuIClSnfV9g6g7bgCIj_9/view?usp=sharing
EID 107: vkCmdCopyBuffer (to buffer 569)
EID 108: release barrier
EID 112: acquire barrier
EID 114: vkWaitForFences
EID 128: vkUpdateDescriptorSets (buffer 569)
EID 134: vkCmdBindDescriptorSets
EID 136: vkCmdDispatch

[artem-lunarg]

vkCmdBindDescriptorSets (containing that buffer)
vkCmdDispatch (writing to the buffer)

The above commands, are they in a separate QueueSubmit that happens after WaitForFences ?

[MennoVink]

vkCmdBindDescriptorSets (containing that buffer) vkCmdDispatch (writing to the buffer)

The above commands, are they in a separate QueueSubmit that happens after WaitForFences ?

Yes different submit for sure. Now that i think of it, it's even worse. It's getting submitted to a different queue (within the same family) even.
The acquire barrier gets submitted to the last Compute QF's queue and the vkCmdDispatch gets submitted to the first queue in that same family.

[artem-lunarg]

I think the scenario in the issue is not a WAW hazard because of the fence wait which creates a memory dependency for all operations on the queue.

The WAW is possible if Dispatch happens in the same command buffer that runs acquire barrier and that barrier does not create correct memory dependency (as in your use case).

[artem-lunarg]

Yes different submit for sure. Now that i think of it, it's even worse. It's getting submitted to a different queue (within the same family) even.
The acquire barrier gets submitted to the last Compute QF's queue and the vkCmdDispatch gets submitted to the first queue in that same family.

If you think this scenario should report an error (taking into account my comment above) please provide pseudo steps as in the original scenario, would be much easier to analyze. It's quite hard to incrementally update use cases (especially if we need to revisit this later)

[artem-lunarg]

Just to be sure I will ask clarification about the original scenario on the khronos board.

https://gitlab.khronos.org/vulkan/vulkan/-/issues/4648

[MennoVink]

Let's stick to the single-queue setup for now until that one is clear.

I think the scenario in the issue is not a WAW hazard because of the fence wait which creates a memory dependency for all operations on the queue.

The WAW is possible if Dispatch happens in the same command buffer that runs acquire barrier and that barrier does not create correct memory dependency (as in your use case).

I see your point, that makes sense, but i'm not 100% sure about fence wait which creates a memory dependency for all operations on the queue
I'm in no way a spec lawyer so correct me if i'm wrong, but i interpret this part of the spec somewhat differently.
https://registry.khronos.org/vulkan/specs/latest/html/vkspec.html#synchronization-fences specifies:

• When a fence is submitted to a queue as part of a queue submission command, it defines a memory dependency on the batches that were submitted as part of that command, and defines a fence signal operation which sets the fence to the signaled state.
• The first access scope includes all memory access performed by the device.

In my interpretation the dependency is only on the submit, ie the qfot barrier doing layout transition / compression w/e. That is fine, the fence does not get signalled before the qfot barrier is done with everything it needs to do.

• The second synchronization scope only includes the fence signal operation.
• The second access scope is empty.

Since there's no second synchronization scope, the fence itself does not prevent anything from executing. So the driver is allowed to reorder the acquire barrier + fence to happen after the dispatch even though the dispatch is submitted later?
We have Implicit Synchronization Guarantees, but This order does not itself define any execution or memory dependencies;

[artem-lunarg]

Since there's no second synchronization scope, the fence itself does not prevent anything from executing. So the driver is allowed to reorder the acquire barrier + fence to happen after the dispatch even though the dispatch is submitted later?

The above coment does not make much sense unfortunately. Fence wait makes all memory accesses (in the first scope) available to all submissions after fence wait. I recommend Vulkan discord for additional discussions, unfortunately synchronization is a complex topic.

Short explanation: a) fence signal creates memory dependency with all accesses in the first scope, the second scope is only fence signal itself. b) Waiting on the fence creates execution dependency with the fence signal (from second scope of memory dependency). After WaitForFences returns all device accesses are available, e.g. caches are flushed and data is written to memory (result of memory dependency). Subsequent submissions will be able to access that memory.

p.s. lets wait for the board answer and will proceed based on that.