Progress bars when sampling multiple chains

[penelopeysm]

Summary

This PR introduces more informative progress bars with MCMCThreads which can be configured by the user (see options below).

In today's meeting it was generally agreed that having one progress bar per chain was more informative / useful, and that an upper bound could be placed to disable that by default if there were more than MAX_CHAINS_PROGRESS chains.

This PR sets MAX_CHAINS_PROGRESS = 10. The user can override the default on a case-by-case basis using the Symbol versions of progress, or can globally set the threshold using setmaxchainsprogress!(N).

For MCMCSerial no change is made (it already had one progress bar per chain).

For MCMCDistributed, progress = :overall is implemented (progress = true is a synonym for that) but progress = :perchain isn't, not for any good technical reason, but because it was just too much of a faff. I think that in principle it's possible to implement it similarly to how it's done for MCMCThreads (i.e., generate one UUID per chain), but I think each chain has to communicate back to the parent worker using a RemoteChannel, and there needs to be a way of disentangling the inputs from different chains (so you either need N RemoteChannels or the channel needs to take a chain ID). I suppose if somebody really, really wants it I could do it, but given that :overall already works I am hoping nobody will complain.

In a technical sense this is accomplished by passing richer objects in the progress keyword argument (see src/logging.jl). This does mean that people should not override progress in their own implementations (I have added a warning in the docs) but it is IMO better than using callbacks.

The videos below demonstrate the behaviour for MCMCThreads.

Closes #82
Closes TuringLang/Turing.jl#2264

Setup

using AbstractMCMC
struct M <: AbstractMCMC.AbstractModel end
struct S <: AbstractMCMC.AbstractSampler end
function AbstractMCMC.step(rng, ::M, ::S, state=nothing; kwargs...)
    sleep(0.001)
    rand(rng), nothing
end

progress=:perchain

# One progress bar per chain, plus one overall progress bar which tracks number of chains
sample(M(), S(), MCMCThreads(), 500, 6; progress=:perchain)

perchain.mov

progress=:overall

# One overall progress bar which tracks the number of samples
sample(M(), S(), MCMCThreads(), 500, 6; progress=:perchain)

overall.mov

progress=true

# Uses :perchain for 10 or fewer chains, :overall for more than 10 chains
# This is still the default option
sample(M(), S(), MCMCThreads(), 500, 6; progress=true)
sample(M(), S(), MCMCThreads(), 200, 11; progress=true)

true.mov

progress=false or progress=:none

# No progress bars at all
# `progress=:none` also works. We have to preserve the boolean values though so that
# `setprogress!(false)` works globally
sample(M(), S(), MCMCThreads(), 500, 6; progress=false)

false.mov

Bonus: :perchain and :overall on Pluto (it's pretty)

Screen.Recording.2025-06-30.at.23.51.20.mov

Bonus: :perchain in iJulia (in VSCode; it's not pretty)

Screen.Recording.2025-06-30.at.23.59.19.mov

Bonus: :overall in iJulia (works fine)

Screen.Recording.2025-06-30.at.23.59.46.mov

[github-actions]

AbstractMCMC.jl documentation for PR #168 is available at:
https://TuringLang.github.io/AbstractMCMC.jl/previews/PR168/

[yebai]

A gentle reminder @nsiccha

[nsiccha]

Will try to get to this later today :)

[nsiccha]

Will only get to it tomorrow.

[nsiccha]

Should the below functions have a trailing !? They aren't mutating the parameters, but some global state, right?

(Doesn't really matter, we just as well leave the names as is)

[penelopeysm]

Indeed, they are mutating (although the macro hides the worst of it). Will change.

[nsiccha]

:perchain also creates the overall progress bar, doesn't it?

[penelopeysm]

Yup, although in the case of :perchain the extra progress bar tracks chains rather than samples. Thanks, have added a line to clarify.

[nsiccha]

The wording here made me assume that I'd also be able to do setprogress!(:perchain), but AFAICT currently this is not possible? Should this be allowed? Maybe not, as the added benefit would be quite small?

[penelopeysm]

It's kind of tricky. If you do setprogress!(:perchain), the multi-chain methods would understand it, but the single-chain method would have no idea what to do with it. Hence why there are two different settings -- one which takes a boolean, and a second one which controls multi-chain output. You are right that the wording is not that great, I'll make that clearer.

[nsiccha]

Looks good overall to me, but I'll have another look (soonish).

[penelopeysm]

@nsiccha Apologies for pinging again. Did you still want to take more looks at it, or was that done already as part of your review above? 😄

[nsiccha]

@penelopeysm no worries. I had been planning to actually run/test the code at least once on my machine, which I haven't gotten around to yet 🙈

[penelopeysm]

Okay, I'll wait for you :)

[nsiccha]

After me testing the code out locally and us talking on Slack, we've agreed to

• change the default behaviour to :overall (due to Parallel progress logging is slow with the (by default enabled) VSCode progress frontend. julia-vscode/julia-vscode#3853),
• potentially make the overall progress bar for :perchain behave as the progress bar for :overall (updating as samples come in as opposed to when chains finish sampling), and
• warn the user about using :perchain in VSCode.

I think that was mainly it, wasn't it, @penelopeysm?

[nsiccha]

I'll have another look after the changes we've discussed.

[nsiccha]

should be "same as :none"

[nsiccha]

and also "use :perchain for 10 or fewer chains, and :overall" 😅