feat(datacontroller): add Kubernetes controller for declarative data …

[aburan28]

…and dataset lifecycle management

Add a controller-runtime based component that manages data in the Dragonfly P2P network declaratively through two CRDs in the data.d7y.io group:

• Dataset declares file and image sources, a distribution policy and an optional inline lifecycle. The reconciler distributes the sources with preheat jobs, refreshes them on an interval or cron schedule, verifies them with get_task jobs, expires them after a TTL (purge or retain) and purges them with delete_task jobs when they are removed or the Dataset is deleted, guarded by a finalizer. Task IDs are computed locally the same way the schedulers do, including blob digest based IDs, so verify and purge address exactly the distributed tasks. Failures are retried with backoff.
• DataLifecyclePolicy applies a lifecycle to Datasets selected by labels, with priority based tie breaking and per-Dataset overrides.

The controller talks to the manager open API with a personal access token, reads all credentials from Secrets, records events and metrics, and reports phase, conditions and per-source state in the status.

Also add the datacontroller command, configuration, Dockerfile, deployment manifests with CRDs and RBAC, samples, build and release wiring, a design document and unit tests against a fake API server and manager.

Claude-Session: https://claude.ai/code/session_01GaXA6phNbdRVaiRGeJv1Xc

Description

Related Issue

Motivation and Context

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation Update (if none of the other choices apply)

Checklist

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.