Crypto Database Ingest

Crypto market ingest service for collecting_data.

Quick Start

python3.12 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Run tests:

PYTHONPATH=src python -m unittest discover -s tests -p "test_*.py"

Mandatory Rules

1. All timestamps must use UTC timestamptz.
2. All stg tables must include batch_id and asof_date.
3. core tables must upsert by unique key, with overwrite as default behavior.
4. Merge logic is allowed to read only from stg rows of the current batch_id.
5. Any DQ result with error severity must block merge execution.
6. Every task must write a record to ops.ingest_task_log.