Impossible to change the Thin parameter in a $run for MCMC with version 1.4.0 of Nimble

[gosselinf]

Dear all,

I am unsure this is the right place to send my message but here it is.

As the maintainer of the runMCMCbtadjust package that relies partly on Nimble, I was surprised that the runMCMC_btadjust function no longer works with version 1.4.0 of Nimble due to the impossibility to change the thin parameter in an update of the MCMC with reset=FALSE. The error message is:
"cannot alter the value of thin, when using reset = FALSE. Called from: CModelMCMC[[i]]$run(niter, thin = thin, progressBar = TRUE, reset = FALSE, resetMV = as.logical(max(control.MCMC$resetMV, resetMV.temporary)))"

My question is whether there is a workaround that I could take into account in runMCMCbtadjust and whether this is a definitive change in Nimble.

Sincerely,

Frédéric

[danielturek]

@gosselinf Thanks for the report. Certainly apologize for breaking your existing code.

@perrydv This warrants some discussion, about desired behaviour.

Referring to changes made in #1602 , which were in response to NCT issue 591 ("MCMC$run with reset=FALSE does not match results from a single run if previous niter was not a multiple of thin"

[perrydv]

@danielturek I am not giving this a thorough look right now, but my quick thought is that the issue of matching results from one longer run vs multiple shorter runs (which we made more robust with #1602 ) should be moot in the case of switching thin, because that can only be done by using multiple runs anyway. I guess one would still want to be able to easily align the results of a longer run with thin=1 (or something small) vs multiple smaller runs with larger thin intervals, for example, but that does seem to get into more remote cases. Maybe instead of a hard stop, we can check if it matters (i.e. if there were "extra" iterations, beyond the last saved iteration, in a previous run) and if so just emit a message instead of a stop? I'll let you suggest any work-arounds that might involve direct manipulation of the MCMC object.

[gosselinf]

Dear Daniel & Perry,

I thank you very much for your replies.

What you have done in #1602 seems interesting and for me should not preclude from changhing thin with reset=FALSE. Actually in runMCMCbtadjust, the preferred method to update thin is to make it a multiple (multiplication by an integer) of the previous thin, which would pose no pb according to me.

By the way, which thinned samples are exactly saved: those with iterationTotal%%Thin==0? or ==1? What is the initial value of iterationTotal: 0 or 1 (to know whether the initial sample is saved in case Thin >1)? Is it the same after a reset=FALSE?

If the change is as I undertsand, it would even get runMCMCbtadjust more rigorous when shifting to a MCMC run to its continuation.

Sincerely,

Frédéric

[danielturek]

Frédéric, thank you for your comments. In regards to your questions, in the v1.4.0 release, the samples are saved when iterationTotal%%Thin==0. That is, when the total number of iterations is a multiple of the thin interval. The initial value of iterationTotal is 0, and it is incremented to 1 at the very beginning of each MCMC iteration (so during the first MCMC iteration, the value of iterationTotal is 1). When thin > 1, the first sample is not saved. Say for example when thin = 5, then the first sample to be saved takes place on the 5th MCMC iteration.

[danielturek]

@perrydv @paciorek I'm making lowest-level changes, trying to address this and make it work in some logical manner, to allow changing the value of thin when MCMC reset = FALSE.

In this update (on branch mcmc_thin2), the value of thin can now be changed when reset = FALSE. If the thinning interval is changed and the previous thinning cycle was incomplete, then a warning is issued. However, this warning can be suppressed by setting the new system option MCMCwarnUnfinishedThinningInterval = FALSE.

The bookkeeping for this is nontrivial. I would very much appreciate any testing that either of you are willing to do using call as:

mcmc$run(niter = X, thin = X, reset = FASLE)

and spot-checking the results, seeing if the warning is issued correctly, and if it makes sense.

[paciorek]

Sure, I can do that.

[danielturek]

@paciorek Thank you. I will not be surprised in the least, if it doesn't operate as you expect it should.