Skip to content

Conversation

Winter-Soren
Copy link
Contributor

What was wrong?

Gossipsub v1.1 features were incomplete in py-libp2p:

  • PX lacked signed peer records validation and peerstore updates
  • No score-based gates (publish, gossip, PX, graylist)
  • No opportunistic grafting
  • Incomplete scoring event wiring

Refs:

Issue #871

How was it fixed?

  • PX: validate signedPeerRecord envelopes, ensure peerID match, upsert in peerstore, connect
  • Scoring: new PeerScorer with P1–P4 + P5 (weights, caps, decay, thresholds)
  • Gates: enforce publish acceptance (inbound), gossip emission, PX acceptance, and graylisting
  • Mesh: join/leave hooks; opportunistic grafting using median mesh score
  • Heartbeat-driven decay; integrated without breaking existing APIs

Summary of approach

  • Added libp2p/pubsub/score.py with minimal decayed scoring model and thresholds
  • Extended libp2p/pubsub/gossipsub.py:
    • PRUNE emit/handle: carry PX peers; validate/store records; gate PX by score
    • GRAFT/PRUNE/mesh mgmt: join/leave hooks and opportunistic grafting
    • Gossip/publish: score gates applied
  • Updated libp2p/pubsub/pubsub.py:
    • Inbound publish acceptance gate
    • Scoring updates on validation failure and first delivery

To-Do

  • Clean up commit history
  • Add or update documentation related to these changes
  • Add entry to the release notes

Cute Animal Picture

cute animal

@Winter-Soren Winter-Soren changed the title added PX signed peer records, scoring gates, and opportunistic grafting enh/871-add-peerscore-scorefunc-signdpeer-gossipsub-1.1 Aug 26, 2025
@Winter-Soren
Copy link
Contributor Author

Issue Summary

The peer exchange test (test_peer_exchange) started failing after implementing Gossipsub v1.1 signed peer records. This was due to the test environment not having proper signed peer records, which the new v1.1 implementation requires.

Root Cause Analysis

Before Gossipsub v1.1 Changes:

  • emit_prune used peer_info_to_bytes(peer_info) which created simple text format: "peer_id\naddr1\naddr2"
  • _do_px tried to parse this as signed peer records, causing "Error parsing message" errors
  • The test was likely passing due to silent failures or other mechanisms

After Gossipsub v1.1 Changes:

  • emit_prune was updated to use proper signed peer records via envelope.marshal_envelope()
  • Test environment had no signed peer records in peerstore
  • When host_1 sent PRUNE to host_0 with host_2 as PX candidate, no signed records existed for host_2
  • Peer exchange data was empty, so no connection happened
  • Test failed: assert host_2.get_id() in gsub0.mesh[topic]

Error Logs Observed

failed to parse peer info from px peer QmcLkaJx9AumkgYseowvsL9UE9ATE39irYuig8RTZQZ6fo: Error parsing message
failed to parse peer info from px peer QmSg4vX87SYgPwXdLzDyGfDRaL1RyiEnk7UKvnyqimxJTc: Error parsing message

Solution Implemented

  1. Fixed Signed Peer Record Format: Updated emit_prune to use envelope.marshal_envelope() instead of peer_info_to_bytes()

  2. Added Fallback Mechanism: Modified _do_px to handle cases where signed records aren't available:

    # Try to get existing peer info from peerstore
    existing_peer_info = self.pubsub.host.get_peerstore().peer_info(peer_id)
    await self.pubsub.host.connect(existing_peer_info)
  3. Updated Test Setup: Made the test more realistic by:

    • Adding initial connection between host_0 and host_2 so peer info exists in peerstore
    • Disconnecting them to simulate the PX scenario
    • Ensuring proper mesh state for testing

Key Changes

In libp2p/pubsub/gossipsub.py:

  • Fixed emit_prune to use proper signed envelope format
  • Added fallback in _do_px for peers without signed records
  • Removed unused peer_info_to_bytes import

In tests/core/pubsub/test_gossipsub_px_and_backoff.py:

  • Added await connect(host_0, host_2) for initial peer discovery
  • Added disconnection logic to simulate PX scenario
  • Ensured proper mesh state assertions

@seetadev
Copy link
Contributor

seetadev commented Sep 1, 2025

@Winter-Soren : Wonderful progress, Soham. Great work.

Adding @lla-dane and @sumanjeet0012 to keep them in sync with the latest gossipsub developments in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants