Part of the content-flow redesign — see design doc ardur-pipeline/docs/redesign-content-flow-2026-06.md (ArdurAI/ardur-pipeline#14).
A1 — Remove downstream source ceilings (uncapped per-topic ingestion)
The aggregator already has no upper cap (diversityFloor: 20 in src/sources.ts is a lower bound). The effective ceilings live downstream and must be removed so the full clustered set survives to synthesis:
ardur-top10-engine Top10Entry.references capped at 5 (handled in T1).
ardur-article-synthesizer references.slice(0, 10) (handled in S1).
Scope (this repo)
- Confirm ingestion keeps all deduped members per cluster (no silent truncation in
src/index.ts cluster assembly).
- Carry the full member/source set forward in
AggregationData (works with rev-3 documentsByTopic).
- Practical limits only: per-source fetch budget, dedup, paywall/ToS. Log any drop with a reason (no silent caps).
Acceptance
- A topic with >30 clustered sources emits >30 members; no constant/slice caps the count.
- Coverage report (
coverageByTopic) reflects the true ingested/deduped counts.
Depends on: C1.
Part of the content-flow redesign — see design doc
ardur-pipeline/docs/redesign-content-flow-2026-06.md(ArdurAI/ardur-pipeline#14).A1 — Remove downstream source ceilings (uncapped per-topic ingestion)
The aggregator already has no upper cap (
diversityFloor: 20insrc/sources.tsis a lower bound). The effective ceilings live downstream and must be removed so the full clustered set survives to synthesis:ardur-top10-engineTop10Entry.referencescapped at 5 (handled in T1).ardur-article-synthesizerreferences.slice(0, 10)(handled in S1).Scope (this repo)
src/index.tscluster assembly).AggregationData(works with rev-3documentsByTopic).Acceptance
coverageByTopic) reflects the true ingested/deduped counts.Depends on: C1.