Skip to content

scx_lavd: Lower the verifier pressure. #3033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Nov 13, 2025
Merged

scx_lavd: Lower the verifier pressure. #3033

merged 6 commits into from
Nov 13, 2025
+103 −76

Conversation

@multics69
Copy link
Contributor

@multics69 multics69 commented Nov 12, 2025 •
edited
Loading

scx_lavd has suffered from verifier errors in some kernel/compiler/architecture combinations, failing with -E2BIG (the BPF program is too complex) or -EACCESS (the stack consumption is beyond 512 bytes).

This PR aims to lower the verifier pressure:

  • For the -E2BIG error

    • Replace a loop-based pick_any_bit() to a loop-less version using ctzll().
    • Disable inlining a looo-heavy pick_most_loaded_cpu() function.

  • For -EACCESS error

    • Reduce the size of struct pick_ctx, which is allocated on the stack.
    • Clean up stack variables in lavd_select_cpu().

No functional changes are intended.

arighi
arighi approved these changes Nov 12, 2025
View reviewed changes
Copy link
Contributor

@arighi arighi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, but have you tried setting -Os or -Oz in BPF_BASE_CFLAGS for lavd? Maybe in this way clang can already produce smaller code without adding any extra overhead or requiring code changes (in some cases it might even bring better performance).

@multics69
Copy link
Contributor Author

multics69 commented Nov 13, 2025 •
edited
Loading

This looks good, but have you tried setting -Os or -Oz in BPF_BASE_CFLAGS for lavd? Maybe in this way clang can already produce smaller code without adding any extra overhead or requiring code changes (in some cases it might even bring better performance).

Thanks for the review, @arighi ! I didn't but I found that -E2BIG is more related to the number of branch instructions (especially loop) rather than the actual binary size. That makes sense considering the search space that the verifier should explore is exponentially increases as the number of branch instructions grows.

Changwoo Min added 6 commits November 13, 2025 10:02
scx_lavd: Add a loop-less, verifier-friendly version of pick_any_bit().
0adaef3 
pick_any_bit() often causes a -E2BIG verifier error because it has a
64-iteration loop. Let's replace the naive loop-based logic with loopless
logic using circular rotation of the bitmap and the trailing zeros count
(ctz). This is more verifier-friendly, faster, and even accelerated when
__builtin_ctzll() is supported (on x86).

Signed-off-by: Changwoo Min <[email protected]>
scx_lavd: Prevent inlining pick_most_loaded_cpu().
bdec2da 
pick_most_loaded_cpu() contains nested loops, so it often causes an -E2BIG
verifier error. Let's prevent inlining the function to reduce the verifier
burden on its callers.

Signed-off-by: Changwoo Min <[email protected]>
scx_lavd: Use bitfield to reduce the size of struct pick_ctx.
ed8e4ae 
struct pick_ctx is allocated on the stack, and it is big! It imposes high
stack pressure, causing an EACCESS verifier error in the worst case.
Let's reduce its size by using bitfields for boolean flags.

Signed-off-by: Changwoo Min <[email protected]>
scx_lavd: Separate out struct sticky_ctx from struct pick_ctx.
07da38e 
Separate out variables using only in test_cpu_stickable() from
struct pick_ctx and create struct sticky_ctx. This reduces the size
of struct pick_ctx, lowering the stack pressure.

test_cpu_stickable() also tweaked to workaround a BPF verifier error,
"bitwise operator |= on pointer prohibited", on clang-18 for ARM64.

Signed-off-by: Changwoo Min <[email protected]>
scx_lavd: Arrange fields of struct pick_ctx to reduce holes.
ed71f20 
To reduce the size of struct pick_ctx,
let's rearrange its fields to eliminate holes between them.

Signed-off-by: Changwoo Min <[email protected]>
scx_lavd: Clean up stack variables in lavd_select_cpu().
bfebddb 
Clean up stack variables in lavd_select_cpu() tio reduce stack usage.

Signed-off-by: Changwoo Min <[email protected]>
@multics69
Copy link
Contributor Author

multics69 commented Nov 13, 2025

Currently, CI fails due to the veristat. More specifically, veristat fails at lavd_dispatch. However, I found that there is no actual verification error when loading the scx_lavd on all the tested kernels. After some more digging, the culprit is veristat itself. lavd_dispatch fails only with -o csv option. When giving an extra -v or -d or giving a function name, lavd_dispatch as an argument of veristat, it works fine.

@JakeHillion -- What do you think? Do you think if it is okay to merge the PR and exempt lavd from veristat for a while until the bug is fixed?

@multics69
Copy link
Contributor Author

multics69 commented Nov 13, 2025 •
edited
Loading

Hmm... veristat fails for scx_lavd of PR #3026 . Let me merge this PR first, then fix the veristat problem later.

@multics69 multics69 merged commit 9245943 into sched-ext:main Nov 13, 2025
4 of 6 checks passed
@multics69 multics69 mentioned this pull request Nov 13, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@arighi arighi arighi approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@multics69 @arighi