Skip to content

librx888: short-transfer / PPS-window accounting#24

Draft
ringof wants to merge 8 commits into
mainfrom
claude/detect-dropped-samples-YbnYV
Draft

librx888: short-transfer / PPS-window accounting#24
ringof wants to merge 8 commits into
mainfrom
claude/detect-dropped-samples-YbnYV

Conversation

@ringof
Copy link
Copy Markdown
Owner

@ringof ringof commented May 18, 2026

Why

The current stats (ok_xfers, bad_xfers, bytes_out) can't see the one failure mode that worries people most: PCLK edges silently lost between the ADC and the GPIF. From the FX3's perspective those samples never existed — no overrun fires, no counter increments.

rx888-firmware is moving toward GPIF-side PPS injection: the state machine watches a PPS CTL pin and forces a DMA commit on the rising edge. Sample data stays pristine — the marker lives in the DMA framing, as a USB transfer shorter than the configured buffer size. Bytes between consecutive short transfers = PCLK edges per PPS interval = should be exactly fs. Any deficit = ADC→GPIF silent drop, localised.

This patch lands the host-side detector first, so we can calibrate the instrument and confirm it produces the same answer across repeated captures before the firmware feature ships. Once it does, the same code goes live with no driver change.

What

New rx888_stats_t fields:

  • full_xfers, short_xfers, zero_xfers
  • min_actual_len, max_actual_len, expected_xfer_bytes
  • bytes_in_window, last_window_bytes

Pure classification logic in src/pps_audit.h (no atomics, no I/O). librx888 owns one pps_audit_t per device, mutated only by the writer thread; values mirrored to atomic fields for rx888_get_stats(). With stock firmware that never short-commits, short_xfers stays at 0.

Synthetic debug mode via cfg.debug_synthetic_pps_every (default 0). When >0, every Nth completed transfer is classified as a forced short, exercising the detector end-to-end against firmware that does not (yet) force commits. Real sample data is unaffected.

rx888_stream -v prints full, short, last_window every second.

Tests

  • tests/pps_audit_test.c — unit tests for the pure classification: all-full, natural shorts, two-window math, force_short, zero-length handling, synthetic-pattern coverage
  • tests/librx888_api.c — extended for the new config default and stats fields
  • make check and make check-asan both green

Determinism check (next step)

Before trusting the detector against real drops, we need to confirm the host-side measurement is repeatable. Plan: run rx888_stream for fixed duration N times against healthy hardware, assert bytes_out, bad_xfers, short_xfers (with debug_synthetic_pps_every=K) converge across runs. If they don't, there's noise in the instrument to chase before we can attribute any short-transfer event to a real ADC slip.

What this doesn't do (yet)

  • No firmware dependency. Stock firmware never produces short transfers in steady state, so this is dormant until the GPIF-side PPS commit lands.
  • No GETSTATS / 0xB7 integration. Once firmware exposes a pps_commits counter, the driver should cross-check it against short_xfers to detect any misclassification.
  • No iqrecord integration. The run.json could record short-transfer boundaries so post-hoc verification can assert marker_interval_bytes == fs × 2; deferred.

Draft until determinism testing on real hardware lands.


Generated by Claude Code

claude added 2 commits May 18, 2026 21:32
Adds host-side instrumentation for PPS-aligned DMA commits: rx888-firmware
will force a commit on the rising edge of an external PPS, which the host
sees as a USB bulk transfer shorter than the configured buffer size. The
bytes between consecutive short transfers equal the ADC samples produced
in one PPS interval, so a deficit vs the configured sample rate localises
an ADC->GPIF silent drop (the one class no other counter sees).

New rx888_stats_t fields:
  full_xfers, short_xfers, zero_xfers
  min_actual_len, max_actual_len, expected_xfer_bytes
  bytes_in_window, last_window_bytes

Classification logic lives in src/pps_audit.h (pure, no atomics, no I/O);
librx888 owns one pps_audit_t per device, mutated by the writer thread,
mirrored to atomic fields for rx888_get_stats(). With stock firmware that
never short-commits, short_xfers stays at 0.

cfg.debug_synthetic_pps_every (default 0): when >0, treats every Nth
completed transfer as a forced short for end-to-end exercising of the
detector against firmware that does not (yet) force commits. Real data
is unaffected.

rx888_stream -v prints full/short/last_window per second.

Tests: new pps_audit_test exercises the pure logic with synthetic
transfer sequences; librx888_api gains coverage for the new config
default and stats fields.
@ringof ringof force-pushed the claude/detect-dropped-samples-YbnYV branch from f6752bf to 79b6bfd Compare May 18, 2026 21:32
claude added 6 commits May 18, 2026 21:39
Surfaces librx888's cfg.debug_synthetic_pps_every through the CLI so the
short-transfer / PPS-window detector can be exercised end-to-end against
stock firmware (which doesn't yet force commits). Every Nth completed
transfer is classified as a forced short; real sample data is unaffected.

Useful sanity check at 32 MS/s with default 1 MiB transfers:

  ./rx888_stream -v -s 32000000 --debug-synth-pps 64 > /dev/null

should print short_xfers incrementing by ~1/sec and last_window stable
across the run.

cli_smoke.sh now asserts the new flag appears in --help output.
Adds rx888_set_pps_callback() and rx888_pps_event_t. The callback
fires once per closed PPS window (real or synthetic), on the writer
thread, after the closing transfer's samples have been delivered to
the sample callback. sample_index is bytes_out/2 at that boundary —
the offset a GNURadio source block needs when emitting an rx_time
stream tag for the corresponding UTC second.

This unblocks gr-rx888 development end-to-end: with
debug_synthetic_pps_every set, the callback fires at the configured
cadence without any PPS hardware or firmware GPIF mod, so the OOT
module can be built and tested against stock firmware before the
real PPS path lands.

https://claude.ai/code/session_01HXg8aqeEaAGF9BadH5SyuZ
The Verify librx888 ABI step diffs nm output against a hardcoded
list; the new public symbol added by the previous commit needs to
be enumerated there too.

https://claude.ai/code/session_01HXg8aqeEaAGF9BadH5SyuZ
make install requires the firmware blob, which gr-rx888 and other
library consumers don't need. install-dev installs only librx888.so,
librx888.a, librx888.h, and librx888.pc — enough to build against
the library from a CMakeLists.txt using pkg_check_modules(LIBRX888).

https://claude.ai/code/session_01HXg8aqeEaAGF9BadH5SyuZ
debug_no_device=1 in rx888_config_t makes rx888_open() succeed without
any USB device present.  rx888_start() spawns a synthetic_writer_main
thread that generates zero-valued samples at samplerate, pacing the
sample callback in real time.  Combine with debug_synthetic_pps_every=N
to drive the full PPS callback path (including rx_time tag emission in
gr-rx888) in CI or Docker without hardware attached.

debug_synthetic_pps_every already covered the hardware-present-but-no-
PPS-firmware case; debug_no_device covers the no-hardware case (Docker
CI, GitHub Actions).

Also adds:
- Dockerfile.dev + docker-compose.dev.yml: bind-mount both rx888-tools
  and gr-rx888, pass through /dev/bus/usb for a real RX888, bootstrap
  librx888 install-dev + gr-rx888 CMake build in one step.
- scripts/dev-container-init.sh: install-dev + gr-rx888 cmake/build,
  drop into interactive shell for the dev loop.
- test_synthetic_no_device(): opens with debug_no_device=1, fires 3
  synthetic PPS events via callback, verifies clean exit.  Zero leaks
  under valgrind; clean under ASan + UBSan.

https://claude.ai/code/session_01HXg8aqeEaAGF9BadH5SyuZ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants