Skip to content

Lazy mode silently drops buckaroo_state_change → WS client hangs forever #793

Description

@paddymul

Summary

_handle_buckaroo_state_change (buckaroo/server/websocket_handler.py:73) returns silently when session.mode != "buckaroo". Sessions loaded via POST /load with mode=lazy therefore receive zero response to a buckaroo_state_change WS message — no error frame, no rebroadcast, nothing. The client (AG-Grid, our test driver, any third-party caller) sits waiting on an empty WS read until it times out.

Behaviour matrix (against the Boston restaurant data, full 883K rows):

mode search_pizza (state_change) response
buckaroo (pandas) 1.1 s initial_state rebroadcast with filtered_rows=22366 ✓
lazy (polars) hangs / 20s timeout no frame returned ✗
buckaroo xorq (/load_expr) 11 s initial_state rebroadcast with filtered_rows=22366 ✓

Reproduction

import asyncio, json, tornado.websocket, tornado.httpclient
async def main():
    client = tornado.httpclient.AsyncHTTPClient()
    await client.fetch("http://127.0.0.1:8700/load", method="POST",
        body=json.dumps({"session":"x", "path":"/path/to/file.parquet",
            "mode":"lazy", "no_browser":True}),
        headers={"Content-Type":"application/json"})
    ws = await tornado.websocket.websocket_connect("ws://127.0.0.1:8700/ws/x")
    await ws.read_message()  # initial state arrives fine
    ws.write_message(json.dumps({"type":"buckaroo_state_change",
        "new_state":{"quick_command_args":{"search":["PIZZA"]},
            "post_processing":"","cleaning_method":"","df_display":"main",
            "show_commands":False,"sampled":False,"search_string":"PIZZA"}}))
    # Hangs here. No frame ever arrives.
    frame = await asyncio.wait_for(ws.read_message(), timeout=20.0)
asyncio.run(main())

Why it matters

Either lazy mode supports filter / cleaning / post-processing state changes (in which case the handler needs implementation), or it explicitly doesn't (in which case the WS must return a structured error so the client can render "filter not supported in lazy mode" instead of spinning forever). The current silent-drop is the worst of both: third-party callers can't tell if their request is in flight, lost, or unsupported.

Suggested fix

Two flavors, depending on product intent:

A. Wire state_change into lazy mode. session.ldf is a pl.LazyFrame; quick-command filters can be applied via ldf.filter(...) similar to how handle_infinite_request_lazy already accepts a filtered ldf. Recompute the row count, rebroadcast initial_state. Symmetric with the buckaroo/xorq paths.

B. Reject explicitly. If lazy mode is intentionally read-only:

if session.mode != "buckaroo":
    self.write_message(json.dumps({"type": "error",
        "error_code": "state_change_unsupported_mode",
        "message": f"buckaroo_state_change is not supported in mode={session.mode!r}; "
                   "the lazy reader does not currently apply filters / cleaning / post-processing."}))
    return

A is more useful product-wise; B is a 5-line stop-the-bleed fix. Recommend B first, file A as the follow-up if filter support in lazy mode isn't already on the roadmap.

Test plan

  • Failing-test commit: a Tornado WS test mirroring the repro above, expecting either a non-empty frame within 2s (any frame — error or rebroadcast).
  • Fix commit: A or B above. Existing tests/unit/server/test_load_expr.py and test_rows_first_spike.py continue to pass (they don't touch the lazy path).

Found while stress-testing the smorgasbord branch across pandas/lazy-polars/xorq backends.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions