Kate/resources

[KatKatKateryna]

Edits of the existing code

• reviewing some "optional" fields, set defaults to None where possible. This will prevent an error message (e.g. 'missing field') on load, and will ignore optional fields with None value on writing to YAML.
• fixed behavior when writing Title and Description: it used to white it as lists instead of strings.
• overwriting dataclass 'asdict' method ('asdict_enum_safe'), to account for Enums. We need to make sure, only string value is used.
• renaming Enums and dataclasses to more specific names (e.g. OnExceed to ServerOnExceedEnum). This is helpful as the classes from different sections are being imported to the same modules.
• reviewed deserialization logic. Former logic only allowed to unpack and assign to dataclasses the values which already had defaults (for example, empty ServerConfig class). This logic did not work for the attributes where default=None, and for the list values where each member can have a slightly different class (even inheriting from the same). Helper functions like ""can_cast_to_dataclass" were introduced to handle lists.

New changes

• introduced subclasses for 3 types of Providers
• separate window for the new Provider (atm, only Postgresql)
• strict typing for all classes/subclasses (e.g. list of possible classes instead of just a parent class). This is needed for proper deserialization behavior from YAML, it needs to know the exact types, e.g. ProviderPostgresql instead of just ProviderTemplate.
• CRS input is split into 2 widgets, so the user can choose from a predefined list of Authorities. Helper Properties introduced to convert between string and the authority+id
• Data validation and saving (to the internal model) has been split into stages: every 'Save' button will update the data model. When adding a new provider, Save btn will save it to the model. When Saving a Resource after editing, the data will be validated and saved into model. Finally, when Saving to YAML, all remaining top-level data will be validated and saved to model (and then saved to file).
• STRING_SEPARATOR variable added as a helper to store multiple values in the same string (e.g. for QListWidgets). This allows us to pack the values to string for display purpose, and then unpack into the values, given the specific order. This is not ideal behavior, might be reviewed later.
• Bbox visual is added (assuming bbox is always in WGS84)

Limitations:

• No specific checks against a full list of CRSs. Theoretically can link a text file and check against it.
• Only able to add Postgresql provider for now, if the setup seems suitable, I will add the rest.
• Provider validator runs after New Provider Window is closed, which clears unsaved changes in case of any error. Might revise this behavior is needed.
• Unpredictable behavior for same-name resources
• Spatial and Temporal extents are strings in the UI for now
• No nice CRS widget in Provider window for now

Screenshots

[KatKatKateryna]

Thank you for the comments @doublebyte1 @PascalLike ! This is very helpful.
Here are the updates:

• revised validation of mandatory fields
• make hreflang and trs a Dropdown
• provider crs as an Array
• move provider logic to the corresponding classes via abstract methods
• split bbox into 4 inputs
• manually validate CRS via requests (not enforced on validation in case there is no internet access). For some reason request is always successful (response 200) so it requires checking for ExceptionReport string in the response content. There might be a better way but I haven't found one.
• add storage_crs
• warnings stay on top, Provider window stays on top
• scrollable warning message on open/save
• support 2 other providers (Map, Tile) and allow editing
• clean up editable fields on resource load

[doublebyte1]

@doublebyte1 the warning when reading the config file is showing because only 3 types of providers have been added as classes, and are able to deserialize properly. I can add more classes for other providers, or allow accepting dictionaries. But then it will be hard to parse the data and represent it in the list of providers. I will see wat can be done. Looking into the rest of the comments now!

Maybe for now, you could just switch off the validation for providers, when loading the file? I think the users will be confused by this.

[doublebyte1]

Thank you for the changes, @KatKatKateryna ! 👍🏽 I think you addressed most of the issues. We only need a few more tweaks.

• The refactoring of the provider class looks 💯
• The provider edition looks much more friendly now. 🥇
• The initial validation dialog is also easier to read, but I still think this error about providers is confusing (I left a comment about it).
• The approach for crs validation seems reasonable; the only issue is when there is no Internet access.
• I tried to activate the GitHub action with no success. Maybe you could tell me if Iḿ doing something wrong? (:

[KatKatKateryna]

• I tried to activate the GitHub action with no success. Maybe you could tell me if Iḿ doing something wrong? (:

Please check these repo settings (can only be accessed by admin) and if adding permissions doesn't help, I'll make a setup for a custom repo-scoped token 🙏

[doublebyte1]

• I tried to activate the GitHub action with no success. Maybe you could tell me if Iḿ doing something wrong? (:

Please check these repo settings (can only be accessed by admin) and if adding permissions doesn't help, I'll make a setup for a custom repo-scoped token 🙏

@KatKatKateryna Thank You! Doing it at organisational level did the trick 👍🏽

[doublebyte1]

Thank you for doing the changes so quickly! 🚀

It all looks good to me now.
We'll figure out how to do a provider validation later.

[doublebyte1]

It is fine to keep it; I just wanted to understand why (👍🏽

[KatKatKateryna]

Summary of the latest changes:

• Handle CRS URL verification (including no internet or timeout)
• fixed zeros in min/max fields
• set Provider names as dropdowns
• switch off the validation for providers and keep them stored as read-only dictionaries in a separate widget. This ensures that unsupported providers are not deleted from Resource if user makes the changes and Saves the Resource

Fixing some details:

• make Provider CRS input optional
• properly delete providers on Delete Selected (not only a list entry from the UI)
• adding new ui folder to zip build
• added CRS validation for Provider.crs list

Some not great behavior I noticed, but it cannot be easily changed without breaking the overall deserialization logic: the incomplete providers will be treated as 'read-only', because they cannot be interpreted as a corresponding Provider class due to missing mandatory fields.

[PascalLike]

LGTM! Thanks @KatKatKateryna!

[PascalLike]

Something to consider for future work #3

[doublebyte1]

Summary of the latest changes:

• Handle CRS URL verification (including no internet or timeout)
• fixed zeros in min/max fields
• set Provider names as dropdowns
• switch off the validation for providers and keep them stored as read-only dictionaries in a separate widget. This ensures that unsupported providers are not deleted from Resource if user makes the changes and Saves the Resource

Fixing some details:

• make Provider CRS input optional
• properly delete providers on Delete Selected (not only a list entry from the UI)
• adding new ui folder to zip build
• added CRS validation for Provider.crs list

Some not great behavior I noticed, but it cannot be easily changed without breaking the overall deserialization logic: the incomplete providers will be treated as 'read-only', because they cannot be interpreted as a corresponding Provider class due to missing mandatory fields.

I think the missing mandatory fields should be flagged by the validator, but neither port or password are mandatory. If a postgres provider does not give these values, it is not incomplete and pygeoapi will process it; on the other hand, if it misses something like table or id_field, it will fail. Would it help if I give you a list of the fields that are mandatory and optional for each provider?