Rounding up CFS numbers in combination with a common multiprocessing use case will result in the process being throttled

[kujon]

Hi 👋

a very common pattern I've spotted is to spawn cpu_count number of threads with multiprocessing library. That leaves you with cpu_count + 1 threads, but the main thread commonly tends to chill until the others are done, so that's mostly fine if cfs_quota / cfs_period is a whole number. One could guarantee no throttling by spawning cpu_count - 1 threads in such a scenario as well.

You run into a problem if cfs_quota / cfs_period isn't a whole number given ceiling the number has been a chosen strategy. If your threads hammer the cpu, you're guaranteed to be throttled, unless you spawn cpu_count - 2 or fewer threads.

I wonder whether this library could help alleviate this issue, by either providing an option to choose between ceil or floor or by having an option to return float from cpu_count function.

What do you think?

[HeavenVolkoff]

Hello, thanks for taking interest in this lib.

About the issue you raise, those are very good points. I think the matter of choosing between ceil and floor strategies can be better solved with the introduction of a tolerance argument. That would allow users to choose the decimal cut-off they want and also allow avoiding some hypothetical odd edge cases such as a 1000ms quota with a 1999ms period resulting in 1 thread with a hard floor, instead of the more sane 2 threads result.

What I am thinking is something like this:

# Considering a 1000ms quota and 2750ms period on a 4 core machine

# Sane floor
cpu_count(tolerance=0.99) #> 2 Threads

# Sane ceil
cpu_count(tolerance=0.01) #> 3 Threads

# Round
cpu_count(tolerance=0.5) #> 3 Threads

# Skewed Round
cpu_count(tolerance=0.8) #> 2 Threads

# Hard floor
cpu_count(tolerance=1) #> 2 Threads

# Hard ceil
cpu_count(tolerance=0) #> 3 Threads

[kujon]

@rugg2, @cvonsteg any thoughts on the above?