Skip to content

[NPU] Use single command queue #34785

Draft
pereanub wants to merge 2 commits intoopenvinotoolkit:masterfrom
pereanub:one_command_queue
Draft

[NPU] Use single command queue #34785
pereanub wants to merge 2 commits intoopenvinotoolkit:masterfrom
pereanub:one_command_queue

Conversation

@pereanub
Copy link
Contributor

@pereanub pereanub commented Mar 19, 2026

Details:

  • Use single command queue for all sub-graphs to optimize doorbell usage

Tickets:

AI Assistance:

  • AI assistance used: no / yes
  • If yes, summarize how AI was used and what human validation was performed (build/tests/manual checks).

@github-actions github-actions bot added the category: NPU OpenVINO NPU plugin label Mar 19, 2026
@pereanub pereanub changed the title Add support for CommandQueuePool [NPU] Use single command queue Mar 19, 2026
@pereanub pereanub force-pushed the one_command_queue branch 8 times, most recently from d64cef7 to 48b7071 Compare March 23, 2026 13:21
@pereanub pereanub requested a review from Copilot March 23, 2026 13:23
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors Intel NPU Level Zero command queue handling to support reusing a single (pooled) command queue across graph/pipeline execution paths, moving from per-graph CommandQueue ownership to a descriptor-based approach.

Changes:

  • Replaced direct CommandQueue storage/exposure with CommandQueueDesc + versioning on Graph/DynamicGraph, and updated pipelines to reacquire a queue when the descriptor changes.
  • Introduced a global CommandQueuePool keyed by CommandQueueDesc to reuse command queues.
  • Added toZeQueueWorkloadType() helper and updated call sites accordingly.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/plugins/intel_npu/src/utils/src/zero/zero_wrappers.cpp Updates CommandQueue ctor to accept a descriptor and adds CommandQueuePool implementation.
src/plugins/intel_npu/src/utils/src/zero/zero_init.cpp Adds stdout prints about external memory support detection.
src/plugins/intel_npu/src/utils/include/intel_npu/utils/zero/zero_wrappers.hpp Introduces CommandQueueDesc and CommandQueuePool, updates CommandQueue API.
src/plugins/intel_npu/src/utils/include/intel_npu/utils/zero/zero_utils.hpp Adds conversion helper for workload type to Level Zero enum.
src/plugins/intel_npu/src/compiler_adapter/src/ze_graph_ext_wrappers.cpp Switches graph init path to acquire queue via CommandQueuePool.
src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp Switches init-schedule queue creation to pooled queue acquisition.
src/plugins/intel_npu/src/compiler_adapter/src/graph.cpp Replaces stored queue with descriptor + versioning; updates workload type handling and init path.
src/plugins/intel_npu/src/compiler_adapter/src/dynamic_graph.cpp Same as graph.cpp for dynamic graphs; uses descriptor + versioning.
src/plugins/intel_npu/src/compiler_adapter/include/graph.hpp Updates public interface to expose queue descriptor + version instead of queue pointer.
src/plugins/intel_npu/src/compiler_adapter/include/dynamic_graph.hpp Updates public interface to expose queue descriptor + version instead of queue pointer.
src/plugins/intel_npu/src/common/src/igraph.cpp Updates base virtuals to new descriptor-based queue API.
src/plugins/intel_npu/src/common/include/intel_npu/common/igraph.hpp Updates interface: get_command_queue_desc(), get_command_queue_desc_version(), non-const set_workload_type().
src/plugins/intel_npu/src/backend/src/zero_pipeline.cpp Uses pooled command queue and refreshes it on descriptor version changes; rebinds fences accordingly.
src/plugins/intel_npu/src/backend/src/zero_dynamic_pipeline.cpp Same as zero_pipeline.cpp for dynamic pipeline.
src/plugins/intel_npu/src/backend/include/zero_pipeline.hpp Adds _command_queue and _command_queue_version members to pipeline base.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

@pereanub pereanub force-pushed the one_command_queue branch from 885f246 to 86b7777 Compare March 24, 2026 07:57
@pereanub pereanub requested a review from Copilot March 24, 2026 07:58
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

@pereanub pereanub force-pushed the one_command_queue branch 2 times, most recently from 0b24f71 to 82f1667 Compare March 24, 2026 10:16
Signed-off-by: Bogdan Pereanu <bogdan.pereanu@intel.com>
@pereanub pereanub force-pushed the one_command_queue branch from 82f1667 to d7fd39c Compare March 24, 2026 10:30
Signed-off-by: Bogdan Pereanu <bogdan.pereanu@intel.com>
@pereanub pereanub added this to the 2026.2 milestone Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants