Implement marginal likelihood estimator for ALL samplers using marginal posteriors

[ben18785]

See here. Easy to implement and quite cool!

[sanmitraghosh]

@ben18785 @MichaelClerx Note sure whether this would be helpful, but you can check out this simple example I have created in my ion channel repo for testing thermodynamic integration using pints samplers. I am able to exactly reproduce the Normalizing constant for a linear Gaussian model (where I know the analytical value). This example can be found here. NB; I am using the Friel correction for carrying out the 1-d numerical integration in order to get a more stable estimate.

At the present I am using a slightly modified class in mcmcsampling.py extracted from pints. This is since I want the MCMC sampler to return the likelihood (which pints stores to disk). Also you can find how to extract the loglikelihood in addition to logposterior So, I created two evaluators as:

evaluator = pints.SequentialEvaluator(self._log_pdf) and evaluator_thermo = pints.SequentialEvaluator(self._log_pdf._log_likelihood).

@mirams wants this to be within pints at some point. In principle we can have for samplers such as SMC to call the evaluator_thermo in its respctive __inint__.py and then do the same to get self._log_pdf._log_prior and combine these two to do whatever is needed. that way evaluating normalising constant would be extremely simple without changing much in pints core.