Add helper function and class methods to create filter instance from hdf5 "cd_values" #366
Conversation
|
As it is, reading hdf5 filter option works for the currently embedded version of the compression filters. Same for endianness checks, let's do it in a separate PR. |
payno
left a comment
payno
left a comment
There was a problem hiding this comment.
I though I catch some byte operation error at one point... but no 😇
Thanks for the unit test :)
| :raises ValueError: Unsupported filter_options | ||
| :raises NotImplementedError: Support of filter_options version is not implemented | ||
| """ | ||
| # ZFP header parsing reference: |
There was a problem hiding this comment.
I hope those won't be changed to much other time. Should it be part of the documentation ?
| if mode == 2: | ||
| return cls( | ||
| peak_signal_to_noise_ratio=quality, | ||
| swap=swap, | ||
| missing_value_mode=missing_value_mode, | ||
| ) | ||
| if mode == 3: | ||
| return cls( | ||
| absolute=quality, swap=swap, missing_value_mode=missing_value_mode | ||
| ) | ||
| if mode == 1: | ||
| return cls(rate=quality, swap=swap, missing_value_mode=missing_value_mode) |
There was a problem hiding this comment.
This is a bit unexpected to have mode condition sorted as 2, 3, 1 instead of 1, 2, 3...
Except if there is a reason that I don't see at the moment...
| if mode == 2: | |
| return cls( | |
| peak_signal_to_noise_ratio=quality, | |
| swap=swap, | |
| missing_value_mode=missing_value_mode, | |
| ) | |
| if mode == 3: | |
| return cls( | |
| absolute=quality, swap=swap, missing_value_mode=missing_value_mode | |
| ) | |
| if mode == 1: | |
| return cls(rate=quality, swap=swap, missing_value_mode=missing_value_mode) | |
| if mode == 1: | |
| return cls(rate=quality, swap=swap, missing_value_mode=missing_value_mode) | |
| if mode == 2: | |
| return cls( | |
| peak_signal_to_noise_ratio=quality, | |
| swap=swap, | |
| missing_value_mode=missing_value_mode, | |
| ) | |
| if mode == 3: | |
| return cls( | |
| absolute=quality, swap=swap, missing_value_mode=missing_value_mode | |
| ) | |
| Get dataset compression | ||
| +++++++++++++++++++++++ | ||
|
|
||
| For compression filters provided by HDF5 and `h5py`_ (i.e., GZIP, LZF, SZIP), |
There was a problem hiding this comment.
Maybe:
| For compression filters provided by HDF5 and `h5py`_ (i.e., GZIP, LZF, SZIP), | |
| For **built-in** compression filters (i.e., GZIP, LZF, SZIP), |
| `compression <https://docs.h5py.org/en/stable/high/dataset.html#h5py.Dataset.compression>`_ and | ||
| `compression_opts <https://docs.h5py.org/en/stable/high/dataset.html#h5py.Dataset.compression_opts>`_ properties. | ||
|
|
||
| For third-party compression filters such as the one supported by `hdf5plugin`, |
There was a problem hiding this comment.
| For third-party compression filters such as the one supported by `hdf5plugin`, | |
| For **third-party** compression filters such as the one supported by `hdf5plugin`, |
| self.filter_options = (0, 0, 0, 0, clevel, shuffle, compression) | ||
|
|
||
| @classmethod | ||
| def _from_filter_options(cls, filter_options: tuple[int, ...]) -> Blosc: |
There was a problem hiding this comment.
Maybe those could be a use case for match...
But I think no much people are using them and this could be more confusing than something else
2 participants
This PR adds helpers to instantiate filter class from
cd_valuesfor each filter (only tested for Bitshuffle and Blosc so far)- [ ] Check endianness issues with filter packing double or bits: Sperr, SZ, SZ3, ZFP- [ ] Check which versions of the filter are supported and raise an exception for unsupported version if the version is stored in thecd_valuesUsage:
Related to #365