Skip to content

Add python bindings#101

Merged
jessicard merged 24 commits intomainfrom
python-bindings
Jan 28, 2026
Merged

Add python bindings#101
jessicard merged 24 commits intomainfrom
python-bindings

Conversation

@jessicard
Copy link
Collaborator

@jessicard jessicard commented Jan 23, 2026

Add Python bindings for compute_report(). (closes #100). Uses PyO3.

Dependencies:

Note: These all link to line numbers. GitHub seems to be having an issue where they're stripped if you just click on the link. However, if you cmd+click or shift+click to open in a new tab, they work correctly.

TODO:

Questions

  • PpaRelevantEventSelector.filter_data is simplified from the Rust API. The Python bindings take
    int | None, where int matches events with that exact filter_data value, and None matches all events. A Python callable would require acquiring the GIL for every event checked, and I wasn't sure if that kind of latency is OK. Do we want to support the Rust API closure in a future PR, or is the simplified version sufficient for these bindings?
  • Experimental features from the Rust library (unfiltered_bin_values, oob_filters) are not currently exposed in
    these bindings. Do we want to add these in a future PR?
  • This PR exposes both PpaHistogramConfig and DirectPpaHistogramConfig for parity with the Rust API. Do we want to keep both of these in the bindings, or did we want to simplify?

@jessicard jessicard self-assigned this Jan 23, 2026
@jessicard jessicard changed the title [WIP] Add python bindings Add python bindings Jan 27, 2026
@jessicard jessicard marked this pull request as ready for review January 27, 2026 18:17
@jessicard jessicard requested a review from giorgi-o January 27, 2026 18:19
Copy link
Member

@giorgi-o giorgi-o left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this is great, thank you!

To answer your questions:

PpaRelevantEventSelector.filter_data is simplified from the Rust API. The Python bindings take int | None, where int matches events with that exact filter_data value, and None matches all events. A Python callable would require acquiring the GIL for every event checked, and I wasn't sure if that kind of latency is OK. Do we want to support the Rust API closure in a future PR, or is the simplified version sufficient for these bindings?

AFAIK grabbing the GIL is just a mutex lock, which is super duper fast in the uncontested case. So the case where this might cause a problem is if:

  1. the python code calls pdslib
  2. pdslib releases the GIL
  3. another python thread grabs the GIL and doesn't let go
  4. pdslib can't grab the GIL again

But I don't think we will encounter that scenario in our Python tests. Plus in this case, it would also cause a latency spike when the Rust code re-acquires the GIL when it eventually returns from the function back into python code.

So this version is fine for now, and if we want to extend it we can :)

Experimental features from the Rust library (unfiltered_bin_values, oob_filters) are not currently exposed in these bindings. Do we want to add these in a future PR?

Maybe, but really depends what we end up using these bindings for (other than AttributionML).

This PR exposes both PpaHistogramConfig and DirectPpaHistogramConfig for parity with the Rust API. Do we want to keep both of these in the bindings, or did we want to simplify?

To be honest I forgot that the Direct version existed! Now that they're implemented we can keep both, but I don't think the Direct variant has much use anymore (not sure what it's doing there in the first place tbh).

Thanks for the PR!

@giorgi-o
Copy link
Member

Out of curiosity, does this code even release the GIL? Don't you have to call allow_threads() for that? (renamed to Python::detatch() in the latest PyO3 version)

@jessicard
Copy link
Collaborator Author

jessicard commented Jan 28, 2026

Out of curiosity, does this code even release the GIL? Don't you have to call [allow_threads()](https://docs.rs/pyo3/0.23.5/pyo3/marker/struct.Python.html#method.allow_threads) for that? (renamed to Python::detatch() in the latest PyO3 version)

@giorgi-o Oh yeah you're right, we don't release the GIL anywhere, so that wouldn't be the concern with using python callables in our case. The overhead would be from going into python for every event instead of staying in rust land I suppose?

@jessicard
Copy link
Collaborator Author

@giorgi-o:

Maybe, but really depends what we end up using these bindings for (other than AttributionML).

Makes sense. We can keep these in the icebox for now, and add them if we need them!

To be honest I forgot that the Direct version existed! Now that they're implemented we can keep both, but I don't think the Direct variant has much use anymore (not sure what it's doing there in the first place tbh).

I'd be happy to remove direct if you think it's just noise in the bindings, up to you :)

Thanks for the review!!

@giorgi-o
Copy link
Member

No reason to remove Direct now, but if it causes any maintenance burden then off to the chopping block it goes.

@jessicard jessicard merged commit 9b780b9 into main Jan 28, 2026
2 checks passed
@jessicard jessicard deleted the python-bindings branch January 28, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python bindings

2 participants