[NPU] Use single command queue #34785
Draft
pereanub wants to merge 2 commits intoopenvinotoolkit:masterfrom
Draft
[NPU] Use single command queue #34785pereanub wants to merge 2 commits intoopenvinotoolkit:masterfrom
pereanub wants to merge 2 commits intoopenvinotoolkit:masterfrom
Conversation
d64cef7 to
48b7071
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors Intel NPU Level Zero command queue handling to support reusing a single (pooled) command queue across graph/pipeline execution paths, moving from per-graph CommandQueue ownership to a descriptor-based approach.
Changes:
- Replaced direct
CommandQueuestorage/exposure withCommandQueueDesc+ versioning onGraph/DynamicGraph, and updated pipelines to reacquire a queue when the descriptor changes. - Introduced a global
CommandQueuePoolkeyed byCommandQueueDescto reuse command queues. - Added
toZeQueueWorkloadType()helper and updated call sites accordingly.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/plugins/intel_npu/src/utils/src/zero/zero_wrappers.cpp | Updates CommandQueue ctor to accept a descriptor and adds CommandQueuePool implementation. |
| src/plugins/intel_npu/src/utils/src/zero/zero_init.cpp | Adds stdout prints about external memory support detection. |
| src/plugins/intel_npu/src/utils/include/intel_npu/utils/zero/zero_wrappers.hpp | Introduces CommandQueueDesc and CommandQueuePool, updates CommandQueue API. |
| src/plugins/intel_npu/src/utils/include/intel_npu/utils/zero/zero_utils.hpp | Adds conversion helper for workload type to Level Zero enum. |
| src/plugins/intel_npu/src/compiler_adapter/src/ze_graph_ext_wrappers.cpp | Switches graph init path to acquire queue via CommandQueuePool. |
| src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp | Switches init-schedule queue creation to pooled queue acquisition. |
| src/plugins/intel_npu/src/compiler_adapter/src/graph.cpp | Replaces stored queue with descriptor + versioning; updates workload type handling and init path. |
| src/plugins/intel_npu/src/compiler_adapter/src/dynamic_graph.cpp | Same as graph.cpp for dynamic graphs; uses descriptor + versioning. |
| src/plugins/intel_npu/src/compiler_adapter/include/graph.hpp | Updates public interface to expose queue descriptor + version instead of queue pointer. |
| src/plugins/intel_npu/src/compiler_adapter/include/dynamic_graph.hpp | Updates public interface to expose queue descriptor + version instead of queue pointer. |
| src/plugins/intel_npu/src/common/src/igraph.cpp | Updates base virtuals to new descriptor-based queue API. |
| src/plugins/intel_npu/src/common/include/intel_npu/common/igraph.hpp | Updates interface: get_command_queue_desc(), get_command_queue_desc_version(), non-const set_workload_type(). |
| src/plugins/intel_npu/src/backend/src/zero_pipeline.cpp | Uses pooled command queue and refreshes it on descriptor version changes; rebinds fences accordingly. |
| src/plugins/intel_npu/src/backend/src/zero_dynamic_pipeline.cpp | Same as zero_pipeline.cpp for dynamic pipeline. |
| src/plugins/intel_npu/src/backend/include/zero_pipeline.hpp | Adds _command_queue and _command_queue_version members to pipeline base. |
src/plugins/intel_npu/src/compiler_adapter/src/dynamic_graph.cpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_npu/src/utils/include/intel_npu/utils/zero/zero_wrappers.hpp
Outdated
Show resolved
Hide resolved
48b7071 to
5966c7c
Compare
src/plugins/intel_npu/src/utils/include/intel_npu/utils/zero/zero_wrappers.hpp
Outdated
Show resolved
Hide resolved
5966c7c to
43d43a2
Compare
src/plugins/intel_npu/src/compiler_adapter/src/dynamic_graph.cpp
Outdated
Show resolved
Hide resolved
43d43a2 to
c06b7b8
Compare
c06b7b8 to
885f246
Compare
src/plugins/intel_npu/src/compiler_adapter/src/dynamic_graph.cpp
Outdated
Show resolved
Hide resolved
885f246 to
86b7777
Compare
src/plugins/intel_npu/src/utils/include/intel_npu/utils/zero/zero_wrappers.hpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_npu/src/utils/include/intel_npu/utils/zero/zero_wrappers.hpp
Outdated
Show resolved
Hide resolved
86b7777 to
4c5c11f
Compare
src/plugins/intel_npu/src/compiler_adapter/src/dynamic_graph.cpp
Outdated
Show resolved
Hide resolved
4c5c11f to
7af021d
Compare
0b24f71 to
82f1667
Compare
Signed-off-by: Bogdan Pereanu <bogdan.pereanu@intel.com>
82f1667 to
d7fd39c
Compare
Signed-off-by: Bogdan Pereanu <bogdan.pereanu@intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details:
Tickets:
AI Assistance: