Binance Futures order book synchronization not following official documentation

[B0Qi]

Problem Description

The current Binance Futures connector implementation in connector/src/binancefutures/market_data_stream.rs does not follow the official Binance documentation for managing a local order book.

Current Implementation Issues

1. No Event Buffering

The code does not buffer WebSocket events while fetching the REST snapshot. Events received during the ~100-500ms snapshot fetch may be lost.

// Lines 92-96 are commented out:
// pending_depth_messages
//     .entry(data.symbol.clone())
//     .or_insert(Vec::new())
//     .push(data);
// continue;

2. No Update ID Validation

The first processed event validation (U <= lastUpdateId AND u >= lastUpdateId) is commented out:

// Lines 242-247 are commented out:
// if (pending_depth.last_update_id < resp.last_update_id
//     || pending_depth.first_update_id > resp.last_update_id)
//     && new_prev_u.is_none() {
//     continue;
// }

3. No Continuity Check

The pu (previous update ID) validation is disabled:

// Line 70:
if prev_u_val.is_none()
/* fixme: || data.prev_update_id != **prev_u_val.as_ref().unwrap()*/
{

4. Comment Indicates Known Issue

Line 99 explicitly states: // fixme: currently supports natural refresh only.

Expected Behavior (per Binance Documentation)

1. Open WebSocket stream and buffer events
2. Get REST depth snapshot (returns lastUpdateId)
3. Drop any event where u <= lastUpdateId
4. First processed event must satisfy: U <= lastUpdateId AND u >= lastUpdateId
5. Each subsequent event's pu should equal the previous event's u, otherwise re-initialize

Comparison with Other Implementations

I analyzed NautilusTrader's implementation which correctly implements:

• Event buffering via book_buffer dictionary
• Sequence number filtering (deltas.sequence <= snapshot.sequence)
• Automatic snapshot rebuild on WebSocket reconnection

Suggested Fix

struct OrderBookManager {
    pending_events: VecDeque<Depth>,
    snapshot_last_update_id: Option<i64>,
    prev_u: i64,
    initialized: bool,
}

impl OrderBookManager {
    fn on_ws_event(&mut self, event: Depth) {
        if !self.initialized {
            self.pending_events.push_back(event);
            return;
        }
        
        // Continuity check
        if event.prev_update_id != self.prev_u {
            self.reset();  // Re-initialize
            return;
        }
        
        self.prev_u = event.last_update_id;
        self.apply_event(event);
    }
    
    fn on_snapshot(&mut self, snapshot: Depth) {
        let last_update_id = snapshot.last_update_id;
        
        // Drop old events, find first valid event
        while let Some(event) = self.pending_events.pop_front() {
            if event.last_update_id < last_update_id {
                continue;  // Drop
            }
            if event.first_update_id <= last_update_id 
                && event.last_update_id >= last_update_id {
                // First valid event
                self.apply_snapshot(snapshot);
                self.apply_event(event);
                self.prev_u = event.last_update_id;
                self.initialized = true;
                break;
            }
        }
        
        // Process remaining buffered events
        while let Some(event) = self.pending_events.pop_front() {
            self.on_ws_event(event);
        }
    }
}

Impact

• Order book may become inaccurate during snapshot fetches
• Data gaps after WebSocket reconnection go undetected
• Relies on "natural refresh" which may leave stale price levels

Environment

• Connector: binancefutures
• File: connector/src/binancefutures/market_data_stream.rs

[nkaz001]

This is already on the todo list and definitely needs to be implemented correctly. But getting it right is more challenging. In extreme market conditions, sequence gaps occur frequently and repeatedly, which can cause rate-limit issues when fetching snapshots. Also, order book updates are delayed until a full sync is completed. Currently, this relies on a natural refresh, which mitigates these issues, even though the order book may not be fully accurate, but, the BBO remains highly accurate.