Skip to content

Ingestion + caching fixes - #245

Merged
alycejenni merged 5 commits into
devfrom
ginger/res-update
Apr 23, 2026
Merged

Ingestion + caching fixes#245
alycejenni merged 5 commits into
devfrom
ginger/res-update

Conversation

@alycejenni

Copy link
Copy Markdown
Member

This includes:

  • a fix for a fairly major bug introduced in ba0a3dc that resulted in no data being ingested
  • the beginnings of some CKAN 2.10 compatibility (in order to fix the aforementioned bug)
  • new clear-cache command
  • cache clearing after background tasks
  • new reingest flag for ingest task, which ignores the previous file hash and reads the file even if it's already been read (or at least, it thinks it has, which is what this is for); should probably only be used with replace = True but I haven't enforced that

Fixing the bug from ba0a3dc will also require identifying and ingesting any resources that should have been ingested while it was live.

Added as a post_run() method to the Task class because it applies to all of them currently, but should be able to be overridden if necessary.
Disables the check against the previous file hash, meaning the file will be reingested even if it's the same as the previous version. This shouldn't generally be needed, but is useful for handling ingest errors where the same file may need to be re-parsed.
Fixes an issue introduced in ba0a3dc where identically named methods from IPackageController and IResourceController were both used, meaning the IResourceController methods were being ignored. This renames the methods for 2.10+ compatibility, and makes the old names handle both interfaces (not ideal, but we still need to support 2.9).
@alycejenni alycejenni self-assigned this Apr 23, 2026
@alycejenni
alycejenni merged commit c25774b into dev Apr 23, 2026
4 checks passed
@alycejenni
alycejenni deleted the ginger/res-update branch April 23, 2026 11:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant