Feat/multipool loadbalancer interface

[vibe-sudo]

Description of new feature

PR: Introduce LoadBalancer interface for extensible MultiPool load balancing

1. Are you opening this pull request for bug-fixes, optimizations or new feature?

New feature — with an internal refactor that is fully backward compatible.

---

2. Please describe how these code changes achieve your intention.

Problem

The current load-balancing strategy in MultiPool, MultiPoolWithFunc, and MultiPoolWithFuncGeneric is implemented as a closed enum (LoadBalancingStrategy) with a switch inside a next() method. This has three limitations:

1. Not extensible — users cannot plug in a custom strategy without waiting for upstream changes. For example, a "least-waiting" strategy (picking the pool with the fewest queued tasks) better reflects real backlog pressure under saturation, but there is no way to add it without forking.
2. Duplicated logic — next() is copy-pasted verbatim across all three MultiPool variants.
3. State coupling — the RoundRobin counter (mp.index) lives on the pool struct rather than the strategy itself.

Solution

Introduce two interfaces in a new lbs.go file:

// PoolMetrics exposes the read-only stats a LoadBalancer needs to make a pick decision.
type PoolMetrics interface {
    Running() int
    Waiting() int
    Free() int
    Cap() int
}

// LoadBalancer picks a pool index from a slice of PoolMetrics.
type LoadBalancer interface {
    Pick(pools []PoolMetrics) int
    // Fallback is called when the pool chosen by Pick is overloaded.
    // Return -1 to indicate no fallback is supported.
    Fallback(pools []PoolMetrics) int
}

All three pool types (*Pool, *PoolWithFunc, *PoolWithFuncGeneric[T]) already satisfy PoolMetrics via the embedded *poolCommon — no extra code needed.

Built-in strategies become small structs:

Constructor	Strategy	Fallback on overload
NewRoundRobinLB()	Atomic round-robin	Falls back to least-running
NewLeastTasksLB()	Fewest running workers	None
NewLeastWaitingLB()	Fewest waiting tasks	None

Three new constructors are added — one per MultiPool variant — that accept any LoadBalancer:

func NewMultiPoolWithLB(size, sizePerPool int, lb LoadBalancer, options ...Option) (*MultiPool, error)
func NewMultiPoolWithFuncAndLB(size, sizePerPool int, fn func(any), lb LoadBalancer, options ...Option) (*MultiPoolWithFunc, error)
func NewMultiPoolWithFuncGenericAndLB[T any](size, sizePerPool int, fn func(T), lb LoadBalancer, options ...Option) (*MultiPoolWithFuncGeneric[T], error)

The existing NewMultiPool / NewMultiPoolWithFunc / NewMultiPoolWithFuncGeneric signatures are unchanged — they now delegate to the new constructors internally after converting the enum to the corresponding built-in LoadBalancer. Zero breaking changes.

Custom strategy example

type myLB struct{}

func (m *myLB) Pick(pools []ants.PoolMetrics) int {
    // pick the pool with the most free capacity
    idx, most := 0, -1
    for i, p := range pools {
        if f := p.Free(); f > most {
            most = f
            idx = i
        }
    }
    return idx
}

func (m *myLB) Fallback(pools []ants.PoolMetrics) int { return -1 }

mp, _ := ants.NewMultiPoolWithLB(10, 100, &myLB{})

Why LeastWaitingLB is worth adding as a built-in

Benchmarks below show that under mixed workloads (uneven task durations — the most common real-world scenario for IO-bound services), LeastTasksLB degrades severely because Running() is a poor proxy for actual backlog pressure. LeastWaitingLB uses Waiting() instead and avoids the problem:

goos: windows / goarch: amd64 / cpu: AMD Ryzen 7 5700X
BenchmarkMultiPool_RoundRobin_CPUThroughput-16      70173303     86.76 ns/op    0 B/op   0 allocs/op
BenchmarkMultiPool_LeastTasks_CPUThroughput-16      18660650    330.9  ns/op    0 B/op   0 allocs/op
BenchmarkMultiPool_LeastWaiting_CPUThroughput-16    18016323    331.9  ns/op    0 B/op   0 allocs/op

BenchmarkMultiPool_RoundRobin_IOThroughput-16       40508790    139.5  ns/op    0 B/op   0 allocs/op
BenchmarkMultiPool_LeastTasks_IOThroughput-16       26041756    302.2  ns/op    1 B/op   0 allocs/op
BenchmarkMultiPool_LeastWaiting_IOThroughput-16     19795819    286.1  ns/op    0 B/op   0 allocs/op

BenchmarkMultiPool_RoundRobin_MixedThroughput-16    22110253    254.1  ns/op    1 B/op   0 allocs/op
BenchmarkMultiPool_LeastTasks_MixedThroughput-16    20750173   1864    ns/op    2 B/op   0 allocs/op  ← degrades ~7x
BenchmarkMultiPool_LeastWaiting_MixedThroughput-16  14748758    360.2  ns/op    1 B/op   0 allocs/op

LeastWaitingLB is also shown as a concrete example of what a custom LoadBalancer implementation looks like, making it a useful reference for users building their own strategies.

---

3. Please link to the relevant issues (if any).

No existing issue. This PR is self-contained and was designed to be fully backward compatible.

---

4. Which documentation changes (if any) need to be made/updated because of this PR?

The README section covering MultiPool should be updated to mention:

• The new NewMultiPoolWithLB / NewMultiPoolWithFuncAndLB / NewMultiPoolWithFuncGenericAndLB constructors
• The LoadBalancer and PoolMetrics interfaces
• A short custom strategy example (similar to the one above)
• The new built-in NewLeastWaitingLB() strategy

I am happy to update the README as part of this PR if the overall direction is accepted.

---

4. Checklist

• I have squashed all insignificant commits.
• I have commented my code for explaining package types, values, functions, and non-obvious lines.
• I have written unit tests and verified that all tests pass.
• I have documented feature info on the README (pending maintainer feedback on the API shape).
• I am willing to help maintain this change if there are issues with it later.

Scenarios for new feature

• Users running IO-bound services with uneven task durations (e.g. mixed fast/slow RPC handlers) who want a smarter pool selection policy
• Users who need application-specific routing — e.g. pinning certain task types to specific pools, or implementing priority-based scheduling across pools
• Anyone who wants to experiment with scheduling strategies without waiting for upstream changes

Breaking changes or not?

No

Code snippets (optional)

Alternatives for new feature

None.

Additional context (optional)

None.

[vibe-sudo]

@panjf2000 Thanks so much for your time reviewing this!

[panjf2000]

Instead of creating this function for each pool, we should consider using functional options for this, check out options.go

[vibe-sudo]

@panjf2000 I have a question about the MultiPool option design.

Current constructor:
func NewMultiPool(size, sizePerPool int, lbs LoadBalancingStrategy, options ...Option) (*MultiPool, error)

The existing options ...Option is for sub-pools.
LoadBalancingStrategy belongs to MultiPool itself, not sub-pools.

Since Go doesn't allow two variadic parameters,
does it mean I need to change the sub-pool options from a variadic to a slice?
Like:
func NewMultiPool(size, sizePerPool int, poolOpts []Option, multiOpts ...MultiPoolOption)

Could you please confirm this?
Thank you!

[panjf2000]

Yes, this is the tricky one. I don't have a feasible solution for that.

[panjf2000]

Besides, please fix the lint issues!

[panjf2000]

ditto

[panjf2000]

ditto

[panjf2000]

ditto

[panjf2000]

ditto

[codecov]

Codecov Report

❌ Patch coverage is 82.22222% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.82%. Comparing base (deeaf50) to head (b6608bf).

Files with missing lines	Patch %	Lines
lbs.go	89.74%	4 Missing ⚠️
multipool.go	76.47%	2 Missing and 2 partials ⚠️
multipool_func.go	76.47%	2 Missing and 2 partials ⚠️
multipool_func_generic.go	76.47%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #400      +/-   ##
==========================================
- Coverage   95.11%   94.82%   -0.29%     
==========================================
  Files          14       15       +1     
  Lines         798      831      +33     
==========================================
+ Hits          759      788      +29     
+ Misses         26       24       -2     
- Partials       13       19       +6     

Flag	Coverage Δ
unittests	94.82% <82.22%> (-0.29%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[panjf2000]

Come to think of it, the LoadBalancer interface seems like overkill. As fancy as it may sound, I doubt many users will implement their own LoadBalancer interfaces. Correct me if I'm wrong, but it seems that all we want is a new load-balancing strategy - LeastWaiting, if so, the simplest way to do that is to add a new LoadBalancingStrategy for it, that way, we don't have to introduce this new LoadBalancer interface and a big chunk of corresponding code.

[vibe-sudo]

@panjf2000 I agree that making it a standalone interface is overkill.
The reason I tried the interface design is to let users customize the load balancing strategy on their end without waiting for a new version release.
But I do agree it makes the code less clean.

Meanwhile, this also exposes an issue: the current options are only for sub-pools.
If MultiPool needs its own options in the future, it will be difficult to add them gracefully.
We may have to change the constructor signature, which could break existing users and force them to adjust their code to use new features.

[vibe-sudo]

So it seems this PR might need to be put on hold for now.
If I open a new PR later, it would be a simple change to add the LeastWaiting strategy as a new enum for compatibility.
But the MultiPool options problem does feel really tricky.

[vibe-sudo]

Would you like me to create a new branch and submit a minimal PR that only adds the LeastWaiting strategy as an enum?
We can keep this PR open to continue discussing the tricky MultiPool options design issue separately.

[panjf2000]

@panjf2000 I agree that making it a standalone interface is overkill. The reason I tried the interface design is to let users customize the load balancing strategy on their end without waiting for a new version release. But I do agree it makes the code less clean.

Meanwhile, this also exposes an issue: the current options are only for sub-pools. If MultiPool needs its own options in the future, it will be difficult to add them gracefully. We may have to change the constructor signature, which could break existing users and force them to adjust their code to use new features.

I expect ants to be a reliable yet simple library of goroutine pool for everyone, which means the fewer the configurations, the better. We can always add new features if needed, nevertheless, we should make decisions adding new features based on real-world demands, not assumptions. This requirement of customizing load-balancing strategies has never been proposed over the years, and you're the first one to bring it up; thus, I have to deem this is a very niche demand. This is why I don't think it's worth so much trouble implementing this interface.

We can revisit this feature when I hear more voices requesting it.

"Premature optimization is the root of all evil."

[panjf2000]

Would you like me to create a new branch and submit a minimal PR that only adds the LeastWaiting strategy as an enum? We can keep this PR open to continue discussing the tricky MultiPool options design issue separately.

just revamp this PR and simply implement the new load-balancing strategy. we can open a new PR for the load-balancing interface in future, provided we really need it.

[vibe-sudo]

Would you like me to create a new branch and submit a minimal PR that only adds the LeastWaiting strategy as an enum? We can keep this PR open to continue discussing the tricky MultiPool options design issue separately.

just revamp this PR and simply implement the new load-balancing strategy. we can open a new PR for the load-balancing interface in future, provided we really need it.

Got it, I’ll revise this PR to implement the LeastWaiting load-balancing strategy directly. We’ll leave the LB interface refactor for a separate PR later if needed. Thanks for your advice!