Skip to content

topicindex/search: consider wiring DHT-removal feed into search tables (only if searches become long-lived) #89

Description

@srene

Background

The DHT-removal feed (Table.removedFeed / subscribeRemovedNodes, added in #71) currently evicts dead nodes from the ad cache and registration tables only. Search tables are deliberately left out (topicSystem.evictRemovedNodesevictNode fans out to regs + ad cache, not searches).

This issue records the analysis of wiring it into search too, so the "why not search?" question is answered and the trigger condition for revisiting is captured.

Why it's not worth it today (net-negative)

Timing is the crux:

Detection path Latency
Search's own timeout (Search.HandleErrorResponse on errTimeout) 700 ms (V5RespTimeout)
DHT eviction (revalidation) secondsPingInterval 3s × several failed cycles / maxFindnodeFailures 5
  • The search already detects any node it's about to query in ~700 ms, uniformly — whether the node came from the allNodes() seed or from a TOPICQUERY referral.
  • The feed fires seconds later and only for DHT-sourced nodes. Referral nodes (the bulk of dead nodes hit mid-search) aren't in the routing table, so the feed never emits them.
  • Net gain: proactively removes a node only in the narrow intersection (DHT-seeded and not-yet-queried and DHT-evicted within the search's few-second lifetime) → saves at most one 700 ms timeout in that rare case.

Cost if implemented

  • Search.RemoveNode public wrapper (trivial).
  • topicSearch.evictCh + evict() + draining it in both run() and pause() (fresh state per outer iteration is an edge case).
  • A live-search registry in topicSystem — searches are currently created per-iterator and untracked (lifecycle owned by topicSearchIterator.Close()). Fan-out needs a mutex-guarded set with register-on-create / de-register-on-Close and stop-vs-evict race handling. This registry existed in an earlier version of p2p/discover: evict unresponsive nodes in topic RPC response handling #71 and was deliberately removed in the rewrite.
  • evictNode search fan-out.

~40 lines plus a recurring concurrency surface, for a marginal/partial-coverage speedup on top of a 700 ms path that already covers every case.

Trigger to revisit

Revisit if searches become long-lived (e.g. a persistent background search held open for minutes to keep a topic table warm). Then a node can die and sit unqueried long enough that the proactive DHT signal beats waiting for the next query cycle, and the cost/benefit flips.

Context: #71, #21.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions