Skip to content

Flow verification module: automated reversed-section detection #198

Description

@jameshgrn

Summary

Built an automated flow verification system that detects and validates reversed flow directions at the section level (junction-to-junction segments). The module is functional but yields are low — parking for now to refocus on higher-impact POM fixes.

What exists

  • src/sword_v17c_pipeline/flow_verification.py — full module with:
    • Multi-signal scoring: DEM WSE slope, SWOT WSE slope (Theil-Sen at node level), facc monotonicity, junction slope validation
    • Confidence tiers: HIGH / MEDIUM / LOW / SKIP
    • Quality filters: ghost junctions (type=5/6), all-lake sections, single-reach sections, all-ghost sections
    • Pre-flip verification: DAG check, connectivity preservation, outlet reachability, facc improvement
    • Batch DuckDB writes with provenance + rollback support
  • scripts/topology/fix_reversed_sections.py — CLI with --dry-run, --apply, --rollback
  • tests/sword_duckdb/test_flow_verification.py — 30 passing tests
  • Figures in figures/flow_verification_NA/ and figures/flow_verification_all/

Results (dry-run, all regions)

Region Approved Rejected Low Skip
NA 3 15 71 731
SA 2 8 204 171
EU 4 2 62 127
AF 1 3 47 108
AS 9 13 331 369
OC 1 1 18 160
Total 20 42 733 1,666

Why parking

  1. Low yield: 20 approved flips out of 2,032 invalid sections (1%)
  2. Small global impact: fixes 46 of 4,814 WSE inversions (~1%), essentially zero facc improvement
  3. DEM signal dominance: MERIT Hydro WSE always has data, SWOT is sparse. DEM controls the vote but is lower quality than SWOT on flat rivers
  4. Diminishing returns: Most remaining invalid sections are ambiguous — flat rivers where noise dominates signal

To pick back up

  1. Drop DEM as a signal — SWOT-only scoring would be higher confidence per flip but even fewer approvals
  2. Tighten slopes_plus_supporting — AS section 18081 was approved despite SWOT actively contradicting DEM (DEM=+0.39, SWOT=-0.40). Require no signal actively contradicts.
  3. Or ship the ~17 clean ones — drop 3 sketchy sections (AS 18081 etc.), apply the rest, move on

Related

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions