Issue kamon-io/kanela#116 was reported on Kanela and I ended up tracking it down to this plugin. Long story short is that our plugin somehow messes the class loading for certain Log4J classes, particularly org.apache.logging.log4j.util.PropertiesUtil, throwing the exception that prevent apps from running. I did a horrible hack via 8ca69af and released v2.0.9 with it, but it smells like there might be more behind this issue.
Since the hack works and the could would only be run on Development mode I decided to leave it like that and move on. I'm leaving this note here for the future in case this comes to bite us.