This issue explores implementation of the feature requested in #143 . A WG-wide discussion is to be had
about specific requirements and the final model chosen for the permanent architecture. This issue should be viewed as an exploration informing the architectural choices made, and may exist alongside other diffusion based approaches.

Aim
This issue tracks the exploration of diffusion based forecasting engines, to replace the deterministic transformer currently in place. Diffusion models have demonstrated the capability to accurately capture complex distributions, and through a stochastic generation process have the ability to better reflect inherent uncertainty in weather prediction. A comprehensive list of relevant papers can be found here. Models like GenCast or cBottle have also demonstrated the viability of such methods for climate level studies through long roll-outs or temporal conditioning.

Steps

• Literature review
• Narrow down options suitable for existing architecture
• If necessary, determine architectural changes needed to enable diffusion forecasting engine (e.g. latent space size/structure)
• Draft Design Doc (optional, may be delayed in favour of fast exploration)
• Implement most promising architecture(s)
• Evaluate against one another and the current deterministic transformer
• Document findings and report to the project lead

Updates and findings will be posted to this issue and collected in this hedgedoc.