Skip to content

Lock-free event distribution for informerProcessor #1226

@radiohead

Description

@radiohead

Problem

Phase 2 removed the RWMutex from the hot path in informerProcessor.distribute(), but the processor still uses mutex-based coordination internally. Further performance gains are possible with lock-free data structures.

Current bottlenecks:

  • Mutex contention during event distribution
  • Allocation overhead in steady state

Proposed Solution

Replace the mutex-based informerProcessor with an xsync MPMC (Multi-Producer Multi-Consumer) lock-free queue.

Key features:

  • Zero allocations in steady state
  • Lock-free event enqueue/dequeue
  • Worker pool with configurable parallelism
  • Blocking enqueue semantics (prevents event loss)

Expected Impact

Based on prototyping:

  • 6.9x throughput improvement: 350K → 2.4M events/sec
  • Zero allocations in steady state: 896B/op → 0B/op
  • CPU contention reduction: 58.63% → <5%
  • Latency: 2847ns → 412ns per operation

Dependencies

  • Go module: github.com/puzpuzpuz/xsync/v3
  • Opt-in via InformerOptions.UseLockFreeProcessor

Related

  • Epic: Improve operator informer performance
  • Design doc: .claude/docs/performance/lock-free-processor-implementation.md
  • Research: .claude/docs/performance/lock-free-queues-research.md

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions