Skip to content

RFC-016 Phase 1.5: OSINT collector executor, KG ingest, resolver wiring, and BGP #154

Description

@rolandpg

Summary

Turn the RFC-016 OSINT scaffold from importable collectors into an end-to-end enrichment path that agents can run and persist into the knowledge graph.

Scope

  • Add a collector executor: input entity -> matching registry collectors -> validated CollectorTuple rows.
  • Persist collector output through KnowledgeGraph.add_node() / add_edge() using the existing kg_nodes / kg_edges storage model.
  • Wire zettelforge.osint.entity_resolver into OSINT ingest for canonical keys, dedupe, and last-write-wins property updates.
  • Promote the BGP collector from registered stub to functional passive enrichment, with mocked tests and fail-closed behavior.
  • Keep active port scanning gated behind explicit operator authorization; decide whether Phase 1.5 should implement it or leave it as documented stub.
  • Add user-facing docs for passive OSINT enrichment, optional dependencies, safety controls, and Python usage.

Acceptance Criteria

  • A Python API can run OSINT collection for a seed DomainName, IPv4Address, IPv6Address, ASNumber, or Netblock and persist resulting KG nodes/edges.
  • Collector results are ontology-validated before persistence.
  • Duplicate OSINT entities collapse through resolver canonicalization.
  • BGP collector is live for passive lookups or intentionally deferred with a narrower follow-up issue.
  • No tests touch the network; all live-source behavior is mocked.
  • Focused OSINT tests and the full regression suite pass.

Notes

Current baseline: RFC-016 Phase 1 is functional, with DNS / WHOIS-RDAP / crt.sh collectors and registered stubs for later phases. tests/test_osint_entities.py and tests/test_osint_collectors.py currently pass (109 passed).

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions