Skip to content

fix(rocprofiler-sdk): emit CPU agents from the WSL topology enumerator - #11423

Open
ChrisLundquist wants to merge 1 commit into
ROCm:developfrom
ChrisLundquist:wsl-cpu-agents
Open

fix(rocprofiler-sdk): emit CPU agents from the WSL topology enumerator#11423
ChrisLundquist wants to merge 1 commit into
ROCm:developfrom
ChrisLundquist:wsl-cpu-agents

Conversation

@ChrisLundquist

Copy link
Copy Markdown

Problem

The wsl-dxcore topology enumerator (platform/wsl/agent.cpp) walks display adapters via D3DKMTEnumAdapters3, so it emits ROCPROFILER_AGENT_TYPE_GPU agents and nothing else.

construct_agent_cache() pairs the HSA runtime's agents with rocprofiler's by logical_node_id, and HSA reports one CPU agent per NUMA node ahead of the GPUs. Emitting GPUs alone breaks that mapping twice over:

  1. the agent counts never match, and
  2. the GPUs are numbered from 0, landing on node ids the runtime assigned to CPUs.

On a single-GPU WSL2 system this is fatal — every rocprofv3 invocation aborts before the target process runs:

F agent.cpp:706] Found 1 rocprofiler agents and 2 HSA agents.
HSA agents contained 1 internal node ids not found by rocprofiler: 1
*** Aborted ***

Shifting the GPU numbering by one does not fix it; it just moves the unmatched node from 1 to 0. The missing agent is the CPU.

Fix

Enumerate the host CPU agents from /sys/devices/system/node first — one per NUMA node, with cpu_cores_count parsed from that node's cpulist and simd_count = 0 — then continue GPU node numbering from there. This is the same ordering the gnulinux enumerator produces from the KFD topology.

Details:

  • logical_node_type_id now uses a separate per-type counter, so GPUs are still numbered from 0 within their type while logical_node_id spans both types.
  • CPU emission happens before the adapter queries, so a topology that fails to find its GPUs (DXCore returning zero adapters, a failed D3DKMTEnumAdapters3) still accounts for the CPU agents HSA reports rather than returning an empty topology into the same fatal.
  • Nodes whose cpulist is unreadable or empty are skipped; if no usable NUMA node is found at all, a warning is logged and behaviour is unchanged from today.

Validation

WSL2, Radeon RX 9070 XT (gfx1201), ROCm 7.2, built from this branch and run via rocprofv3 --rocm-root=<local prefix>:

log exit
develop Found 1 rocprofiler agents and 2 HSA agents ... not found by rocprofiler: 1 134 (SIGABRT)
this PR # agent node maps: 2 0

With the fix, rocprofv3 --kernel-trace --hip-trace completes and the results database contains both agents and the dispatches:

agents: (0, 'CPU', logical 0, type_index 0, 'AMD Ryzen 9 5950X 16-Core Processor')
        (1, 'GPU', logical 1, type_index 0, 'AMD Radeon RX 9070 XT')
kernel dispatches: 2

Also verified the GPU program's own output is still correct under the profiler, and that clang-format v11 (the version rocprofiler_formatting.cmake requires) reports no changes.

This is Linux/WSL-only and touches no code path reachable when platform::gnulinux::is_available() is true, which is every non-WSL Linux system.

🤖 Generated with Claude Code

https://claude.ai/code/session_01951qQBesgNWcb8t9rZf1b3

The wsl-dxcore enumerator only walks display adapters, so it emits GPU
agents and nothing else. construct_agent_cache() pairs the HSA runtime's
agents with ours by logical_node_id, and HSA reports one CPU agent per
NUMA node ahead of the GPUs. Emitting GPUs alone therefore breaks the
mapping twice over: the counts never match, and the GPUs land on node ids
the runtime handed to CPUs.

On a single-GPU WSL2 system that is fatal:

  F agent.cpp:706] Found 1 rocprofiler agents and 2 HSA agents.
  HSA agents contained 1 internal node ids not found by rocprofiler: 1

which aborts every rocprofv3 invocation before the target runs.

Enumerate the host CPU agents from /sys/devices/system/node first, one
per NUMA node with cpu_cores_count taken from the node's cpulist, then
continue GPU node numbering from there — the same ordering the gnulinux
enumerator produces from the KFD topology. logical_node_type_id keeps a
separate per-type counter so GPUs are still numbered from 0 within their
type. CPU emission happens before the adapter queries so that a topology
that fails to find its GPUs still accounts for the agents HSA reports.

Validated on WSL2 with an RX 9070 XT (gfx1201), ROCm 7.2:

  before: Found 1 rocprofiler agents and 2 HSA agents -> SIGABRT (134)
  after:  agent node maps: 2 -> exit 0

rocprofv3 --kernel-trace --hip-trace now completes and records both
agents (CPU absolute_index 0, GPU absolute_index 1) along with the
kernel dispatches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01951qQBesgNWcb8t9rZf1b3
@therock-pr-bot

Copy link
Copy Markdown

❌ PR Check — Action Required

Check Status Details
📝 PR Description ❌ Fail Error: PR description must reference a JIRA ID, ISSUE ID, or a GitHub closing keyword.
Expected: include a JIRA ID / ISSUE ID line (separator : or -, or omitted; value may be a JIRA key, a number with/without #, or a link), OR a closing keyword + issue reference. Accepted examples:
JIRA ID : TESTAUTO-6039
JIRA ID - #330
JIRA ID #330
JIRA ID (on separate line)
ROCM-25757
ISSUE ID : TESTUTO-3334
ISSUE ID #3334
ISSUE ID - TESTAUTO-3433
ISSUE ID (on separate line)
AIRUNTIME-2352
ISSUE ID : https://github.com/<org_name>/<repo_name>/issues/1234
Closes #10
Fixes octo-org/octo-repo#100
Resolves: #123
#123
https://github.com/<org_name>/<repo_name>/issues/123
Current: no valid JIRA/ISSUE/closing-keyword reference found
Forbidden Files ✅ Pass
🧪 Unit Test ⚠️ Warning Error: Source/code files changed without an accompanying unit test.
Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*).
Current: code file(s) changed: projects/rocprofiler-sdk/source/lib/rocprofiler-sdk/platform/wsl/agent.cpp; no test file found
🔎 pre-commit ⏳ Pending ⏳ Still running…
🚫 Draft PR 🔜 To Be Enabled
🚩 Feature Flag 🔜 To Be Enabled
📊 Code Coverage 🔜 To Be Enabled

⚠️ 1 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

  • ❌ PR Description

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ PR Description

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant