scx_p2dq: Add DHQ support and fix migration-disabled task errors

Integrate Double Helix Queue (DHQ) as an alternative to ATQ for LLC-aware
task migration, and fix critical race condition causing migration-disabled
task errors.

DHQ Integration:
- Add --dhq-enabled flag to enable DHQ mode for LLC migration
- Add --dhq-max-imbalance parameter (default: 3) to control strand balance
- Create one DHQ per pair of LLCs in same NUMA node
- Map each LLC to a specific strand (A or B) for cache affinity
- Each CPU inherits strand from its LLC for proper load distribution
- DHQ provides cache-aware migration with controlled cross-LLC movement

Strand-Specific DHQ Operations:
Use scx_dhq_peek_strand() and scx_dhq_pop_strand() instead of generic
operations to ensure CPUs only consume from their designated strand.
This preserves cache locality and prevents load imbalance.

Data Structure Changes:
- Add mig_dhq and dhq_strand to cpu_ctx and llc_ctx
- Add llc_pair_dhqs[] for shared DHQs between LLC pairs
- Add llcs_per_node[] to track LLCs per NUMA node
- Add P2DQ_ENQUEUE_PROMISE_DHQ_VTIME enqueue promise type
- Add enqueue_promise_dhq struct for DHQ-specific metadata

Configuration:
- p2dq_config.dhq_enabled: Enable DHQ mode
- p2dq_config.dhq_max_imbalance: Control strand pairing (0 = unlimited)
- Priority mode: lowest vtime wins across strands

Build System:
- Add lib/dhq.bpf.c to scx_p2dq and scx_chaos builds

scx_chaos Compatibility:
- Update enqueue promise handling to recognize DHQ type
- Error message updated to mention both ATQs and DHQs not supported

Benefits:
- Cache affinity: Tasks stay on origin LLC (strand)
- Controlled migration: max_imbalance prevents migration storms
- Race-free: Atomic affinity handling eliminates migration-disabled errors
- Work conservation: Cross-strand stealing when priority demands
- Scalable: Lock contention distributed across DHQ strands

Signed-off-by: Daniel Hodges <[email protected]>

Author: hodgesds

lib/dhq.bpf.c

@@ -1,4 +1,3 @@
-#include "scxtest/scx_test.h"
 #include <scx/common.bpf.h>
 #include <lib/sdt_task.h>
 

scheds/rust/scx_chaos/build.rs

@@ -10,6 +10,7 @@ fn main() {
         .add_source("src/bpf/lib/arena.bpf.c")
         .add_source("src/bpf/lib/atq.bpf.c")
         .add_source("src/bpf/lib/bitmap.bpf.c")
+        .add_source("src/bpf/lib/dhq.bpf.c")
         .add_source("src/bpf/lib/minheap.bpf.c")
         .add_source("src/bpf/lib/rbtree.bpf.c")
         .add_source("src/bpf/lib/sdt_alloc.bpf.c")

scheds/rust/scx_chaos/src/bpf/main.bpf.c

@@ -505,7 +505,8 @@ complete_p2dq_enqueue_move(struct enqueue_promise  *pro,
 		break;
 	case P2DQ_ENQUEUE_PROMISE_ATQ_FIFO:
 	case P2DQ_ENQUEUE_PROMISE_ATQ_VTIME:
-		scx_bpf_error("chaos: ATQs not supported");
+	case P2DQ_ENQUEUE_PROMISE_DHQ_VTIME:
+		scx_bpf_error("chaos: ATQs/DHQs not supported");
 		break;
 	case P2DQ_ENQUEUE_PROMISE_FAILED:
 		scx_bpf_error("chaos: delayed async_p2dq_enqueue failed");

scheds/rust/scx_p2dq/build.rs

@@ -10,6 +10,7 @@ fn main() {
         .enable_skel("src/bpf/main.bpf.c", "bpf")
         .add_source("src/bpf/lib/arena.bpf.c")
         .add_source("src/bpf/lib/atq.bpf.c")
+        .add_source("src/bpf/lib/dhq.bpf.c")
         .add_source("src/bpf/lib/bitmap.bpf.c")
         .add_source("src/bpf/lib/cpumask.bpf.c")
         .add_source("src/bpf/lib/minheap.bpf.c")