Skip to content

MeteredExecutor: relax modifyState CAS from seq_cst to acq_rel - #2699

Open
VrihadS wants to merge 1 commit into
facebook:mainfrom
VrihadS:fix-metered-executor-cas-ordering
Open

MeteredExecutor: relax modifyState CAS from seq_cst to acq_rel#2699
VrihadS wants to merge 1 commit into
facebook:mainfrom
VrihadS:fix-metered-executor-cas-ordering

Conversation

@VrihadS

@VrihadS VrihadS commented Sep 12, 2026

Copy link
Copy Markdown

Fixes #2656.

modifyState()'s CAS only needs to (a) linearize state_'s own bit-packed counters — guaranteed by the CAS retry loop's coherence regardless of memory order — and (b) publish/acquire this thread's surrounding effects to whichever thread next observes the new value. Both are satisfied by acq_rel.

Nothing in MeteredExecutor depends on state_ taking part in a seq_cst total order: task transfer through queue_ and worker dispatch through kaInner_->add() already carry their own synchronization independent of state_. pause() already uses a relaxed fetch_or on this same atomic, which is inconsistent with the class ever having relied on a seq_cst-wide ordering guarantee across all of state_'s mutations.

seq_cst costs nothing extra here on x86 (LOCK CMPXCHG is already a full barrier), but acq_rel is measurably cheaper on weakly-ordered architectures (ARM, POWER) where seq_cst RMW requires an additional trailing fence.

Testing: added RealThreadConcurrentStress, which drives add() from real OS threads concurrently with pause()/resume() — the existing PauseResumeStress test only exercises interleavings via DeterministicSchedule's mocked atomics and doesn't exercise actual hardware memory ordering. Verified under ThreadSanitizer (20 passes x 8 producers x 5,000 tasks = 800k tasks) with this change and, as a control, against the original seq_cst code: identical results both ways — full correctness, and the same 4 TSan-flagged races in unrelated folly::hazptr/folly::Function machinery, confirming they're pre-existing and unaffected by this change.

state_'s CAS only needs to linearize its own bit-packed counters (which
the CAS retry loop guarantees regardless of memory order) and to
publish/acquire this thread's surrounding effects to whichever thread
next observes the new value -- both satisfied by acq_rel. Nothing here
depends on state_ taking part in a seq_cst total order: task transfer
through queue_ and worker dispatch through kaInner_->add() already
carry their own synchronization. seq_cst costs nothing extra on x86
(LOCK CMPXCHG is already a full barrier) but requires an additional
trailing fence on acq_rel on weakly-ordered architectures like ARM.

Adds RealThreadConcurrentStress, which hammers add() from real OS
threads concurrently with pause()/resume(), to cover what the existing
DeterministicSchedule-based PauseResumeStress test cannot: actual
hardware memory ordering. Verified under ThreadSanitizer (20 passes x
8 producers x 5000 tasks) with identical results -- same 4 pre-existing,
unrelated races in folly::hazptr/folly::Function -- against both this
change and the original seq_cst code.

Fixes facebook#2656
@meta-cla meta-cla Bot added the CLA Signed label Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MeteredExecutor: is seq_cst on the modifyState CAS required, or would relaxed suffice?

1 participant