Request for Feedback: Sitemap and other UI component definitions

[mherwege]

I create this issue here, but it is across core and UI.

At this time, UI defined sitemaps are stored as UI components in the UI component registry. All UI components are stored in a single JSON storage.
The object format for sitemaps in the UI therefore matches the object format of other UI components.

Sitemaps so far are the only UI components that can also be defined through a .sitemap file. To make both available in the same way to sitemap UI's, the current code goes through a translation of the UI object format for sitemaps to the sitemap model object format. This translation is one way (from UI component to model).

With #5004 and openhab/openhab-webui#3349, I created a core sitemap registry that decouples sitemap providers from the model. The sitemap model becomes a provider for a core sitemap registry, just like the UISitemapComponentProvider. This new registry uses a sitemap object format that is derived from the sitemap model, with a few slight changes. This format is easy to understand and convert.
The plan is to create another sitemap provider for YAML and create conversions between the formats (DSL, YAML and JSON), like is being done for items and things. This is a lot easier with the central registry.
There is a draft PR converting serializing sitemaps to DSL and YAML, showcasing the target format here: #4945. It does not use the sitemap registry yet. This registry would be needed to also be able to create sitemaps in YAML without depending on the sitemap model code.

I foresee to then enable the UI to visualize all sitemap configurations, not just the managed ones in the UI. The question is what object format I should use for that? There are two options:

1. Use the current UI component object format: This will require translating to that format from the core internal object representation.
2. Change the UI code to use an object format that more closely matches the core sitemap registry object format.

In both cases, changes to and/or additional REST API's will be required.

Option 2 looks attractive because it would allow to use the core format translators to represent sitemaps in JSON, YAML or DSL in the UI quite easily (and get rid of the sitemap parser and generator code in the UI). However, it then is another format for UI components. My understanding is the sitemaps have been shoe-horned into this format in the UI because there originally was an intention to be able to use sitemap components in Main UI pages directly.
With option 2, sitemaps could be extracted from the UI component store and become a separate managed provider. An upgrade script could convert existing managed sitemaps. This effectively decouples sitemaps from UI components.

Still, option 1 has it merits as well. Sticking to this format, which is well understood for Main UI components, would make it easier later on to also create file based YAML definitions of Main UI components. This line of thought requires more torough evalution though. The file YAML definition of sitemaps would more closely map to the UI component object definitions. The proposed YAML format from #4945 would then be invalid and would rather be the UI component object format translated to YAML.

My question: which of the 2 options should I pursue? My preference is option 2.

@lolodomo @florian-h05 @ghys What is your view on this?

[rkoshak]

The only thing I have to mention is to not forget that the phone apps natively render sitemaps, so any changes to the APIs for sitemaps may have impacts on those repos too.

[mherwege]

The only thing I have to mention is to not forget that the phone apps natively render sitemaps, so any changes to the APIs for sitemaps may have impacts on those repos too.

As long as the current REST API for rendering is not changed, there should not be an impact. This is about API’s for defining sitemaps, not rendering them.
BasicUI and CometVisu are impacted by the sitemap registry work because they directly use some Java API’s and servlets for rendering. The addon PR I created should take care of that.

[lolodomo]

The new sitemap registry is a great idea, I am currently reviewing your PRs.
I am in favor of option 2, the idea would be to do something similar to what @jimtng has prepared for items and things in PR:
openhab/openhab-webui#3180
Import from file formats and export to file formats should also be provided features.
A lot of work but you already implemented a significant part

[florian-h05]

I am in favour of option 2 as well.
Wrt to sitemap rendering in Main UI: There was some work to prepare for this in pre-relase versions of Main UI (at least I stumbled across some code for that, not sure it's still there though), but IMO sitemap rendering in Main UI is nothing we need in Main UI, so I see no issue in shifting away from a format well understood by the widget system. Anyway, the benefits of getting rid of the sitemap DSL parser outweigh this in any case.
And decoupling UI components and sitemaps is also a good thing IMO, they are very different concepts to me. Sitemaps are declarative and have a set of widgets, UI components are highly customizable and their definition directly affects rendering. Sitemap widgets don't tell the client how to render them.