ipa: rpi: pisp: Add decompand support using PiSP hardware block #284

asofam · 2025-07-15T07:26:25Z

Summary

This pull request adds support for decompanding sensor data encoded with companding, using the PiSP hardware block
available on Raspberry Pi 5. The decompand operation is applied prior to ISP processing, enabling linearization of pixel values early in the pipeline.

Changes

1. Decompanding Algorithm

Introduced a new decompanding algorithm:
- decompand_algorithm.h (interface)
- decompand.cpp/h (implementation)
Integrated in the controller framework under ipa/rpi/controller/.

2. PiSP Hardware Support

The algorithm connects to the PiSP's hardware decompanding block for efficient processing.
pisp.cpp was modified to enable and configure this block based on the provided tuning parameters.

3. Sensor Tuning Configuration

Included a tuning file: data/imx585.json (based on external source)
- Contains "rpi.decompand" parameters, including lut and pad
- The imx585.json tuning file is derived from a version provided by soho-enterprise.co, with modifications limited to the "rpi.decompand" section.
The purpose of this file is to validate and integrate the new decompanding algorithm.
It may be updated or replaced in the future as upstream sensor support becomes available.

4. Architecture Consideration

Maintains modular separation between controller logic and hardware interface
Designed to be extensible for additional sensors or alternative LUTs

Rationale

Decompanding is a critical early-stage operation in many modern image pipelines,
especially for sensors that output companded pixel values. By enabling this in the PiSP,
we offload the processing to dedicated hardware, improve performance, and lay the foundation
for accurate linear-space ISP tuning.

Notes

This patch does not affect existing sensors or pipelines unless the "rpi.decompand" key is defined.
The IMX585 sensor was tested using a kernel driver provided by soho-enterprise.co,
which is not part of the upstream Linux kernel or libcamera.

Testing

Verified functional decompanding behavior with test frames and PiSP configuration
Confirmed that no impact occurs for sensors lacking "rpi.decompand" configuration

Signed-off-by

Signed-off-by: Sena Asotani [email protected]

This patch integrates a new decompand algorithm that utilizes the PiSP hardware block available on Raspberry Pi 5. The implementation enables conversion of companded sensor data into linear format prior to ISP processing. Changes include: - Implementation of decompand logic for controller and pipe interfaces - Enabling decompand block by "rpi.decompand" in tuning.json Signed-off-by: Sena Asotani <[email protected]>

naushir · 2025-07-15T10:31:06Z

Adding @njhollinghurst and @davidplowman for visilbity.

@asofam thank you for this work! We would be happy to include this functionality into the IPA after review.

njhollinghurst · 2025-07-15T11:19:49Z

src/ipa/rpi/pisp/pisp.cpp

+		return;
+
+	pisp_fe_decompand_config config = {};
+	config.pad = decompandStatus->pad;


I would remove the pad field everywhere (except the existing config structure).

Our convention is to set padding/reserved fields to zero. The {} initializer will zero it here, which is all that's needed. fe->SetDecompand() will also replace it with zero.

I don't think we need any such field to exist in the tuning file or the metadata.

P.S. if you use it for some other purpose, e.g. as a validity flag, please give it a different name!

Thank you for the review! I've removed the pad field from the tuning file, status structure, and controller code as suggested. The PiSP config structure still retains the field, but it's now zero-initialized using {}.

This patch removes it from the tuning file, DecompandStatus, and related code. As per reviewer feedback, the 'pad' field is unnecessary outside the PiSP hardware config structure. Signed-off-by: Sena Asotani <[email protected]>

naushir · 2025-07-16T09:09:44Z

src/ipa/rpi/controller/rpi/decompand.cpp

+void Decompand::initialValues(uint16_t LUT[])
+{
+  for (size_t i = 0; i < sizeof(decompandLUT_) / sizeof(decompandLUT_[0]); ++i)
+	{


There are a few formatting issues like this that will need to be fixed up before we can merge. There is a handy git commit hook that can be setup to run a check and give you the formatting errors. You can find more details here.

Thank you for pointing that out. I wasn't aware of such a helpful tool!
I've run checkstyle.py and corrected the formatting issues accordingly.

naushir · 2025-07-16T09:14:41Z

src/ipa/rpi/pisp/data/imx585.json

+                    29696, 31744, 33792, 35840, 37888, 39936, 41984, 44032,
+                    46080, 48128, 50176, 52224, 54272, 56320, 58368, 60416,
+                    62464
+                ]


With the "gamma" tuning block, we specify a PWL instead of the hardware LUT. This is easier for users to visualize, manipulate, and tune for. The IPA (pisp.cpp) translates the PWL into the HW LUT needed to be programmed into the registers.

It would be nice if we can do the same with the decompand block as well - and it provides all the advantages of using PWLs instead of hardware specified LUTs. Would it be possible for you to make this change? Sorry, this is more work for you, but I think it will be better in the long run. Feel free to ask questions if things are unclear!

Thank you for the suggestion!
I will give it a try to switch the decompand block to use a PWL instead of a hardware LUT, following the approach used in the gamma block. It may take a bit of time, but I will look into the gamma implementation and work from there.

If I run into any unclear points along the way, I would appreciate your guidance. Thank you again!

Adjusted indentation and formatting to comply with project style guidelines,as reported by checkstyle.py. Signed-off-by: Sena Asotani <[email protected]>

njhollinghurst · 2025-07-22T16:28:27Z

Hi, could you explain a little more how this is used? As I don't know the details about IMX585...

The code as it stands will apply the curve unconditionally, without regard to mode or pixel format. In general, a sensor might support multiple modes or pixel formats, only some of which need decompanding. Is that the case for IMX585?

The example curve is flat up to 3072 -- is this as specified by Sony? For IMX585, what is the nominal black level after expansion to 16 bits and after decompanding?

asofam · 2025-07-23T07:52:33Z

Thank you for the feedback.

As with other sensors, the IMX585 requires decompanding only when Gradation Compression is enabled. So ideally, the decompanding should be applied conditionally based on the sensor mode. Would it make sense to make this controllable via a driver flag or through a libcamera command-line option?

As for the flat portion of the curve up to 3072, my understanding is that the sensor clamps any values below the configured black level to that level internally. This clamping would occur before decompanding, so I think mapping the full range [0–3072] to a constant value (e.g., 0 → 3072, 1024 → 3072) better approximates the sensor's actual behavior than a linear mapping like 0 → 0, 1024 → 1024.

For the nominal black level, the value after decompanding and expansion to 16-bit is 3072:

The sensor’s black level register is set to 192, which applies to the 12-bit compressed data.
When shifted to 16-bit (by 4 bits), it becomes 192 × 16 = 3072.

Note that the default black level of the IMX585 is 200, but I set it to 192 to align with values 1024 × n, which tends to work better with ISP behavior.

njhollinghurst · 2025-07-23T09:39:37Z

As with other sensors, the IMX585 requires decompanding only when Gradation Compression is enabled. So ideally, the decompanding should be applied conditionally based on the sensor mode. Would it make sense to make this controllable via a driver flag or through a libcamera command-line option?

I'm not sure. This could be opening up a can of worms, as API support for it might be lacking both in V4L2 and libcamera.

Are you using a special driver to enable Graduation Compression, or is it a normal / default mode for IMX585? If it's the normal mode of operation, then I guess it's OK to do it unconditionally. I just wanted to understand if that was the case.

EDIT: One possible generalization might be for the tuning file to contain a mapping: { bitdepth → LUT }. The algorithm could then look up the appropriate curve, based on the current mode's bit-depth. I'm assuming this might distinguish between decompanded and non-decompanded modes... Do you think that would be a helpful addition? (Note that in JSON, all keys have to be strings, not integers.)

As for the flat portion of the curve up to 3072, my understanding is that the sensor clamps any values below the configured black level to that level internally

For some sensors, there is variation both above and below the black level, due to various kinds of noise. Clamping at exactly the black level could introduce bias, so that noisier (higher gain) images become brighter on average. If lower values could occur with IMX585, then I think we'd want to extend the linear slope below the black level (and even if they can't, it seems harmless).

naushir · 2025-07-24T07:26:04Z

As with other sensors, the IMX585 requires decompanding only when Gradation Compression is enabled. So ideally, the decompanding should be applied conditionally based on the sensor mode. Would it make sense to make this controllable via a driver flag or through a libcamera command-line option?

I'm not sure. This could be opening up a can of worms, as API support for it might be lacking both in V4L2 and libcamera.

I suggest for now we simply use the mode_ to decide if the decompanding should be enabled or not. Any additional plumbing to v4l2/libcamera, while nice, would add too much scope to this change.

asofam · 2025-07-24T08:14:37Z

Thank you for the feedback.

Are you using a special driver to enable Graduation Compression, or is it a normal / default mode for IMX585?

I using a special driver.

For some sensors, there is variation both above and below the black level, due to various kinds of noise. Clamping at exactly the black level could introduce bias, so that noisier (higher gain) images become brighter on average. If lower values could occur with IMX585, then I think we'd want to extend the linear slope below the black level (and even if they can't, it seems harmless).

Regarding the black level handling, I have rechecked the IMX585 data and confirmed the presence of pixel values below the black level. Based on this, I agree with the concern about introducing bias through clamping, and will modify the curve to maintain a linear slope below the black level rather than flattening it.

As with other sensors, the IMX585 requires decompanding only when Gradation Compression is enabled. So ideally, the decompanding should be applied conditionally based on the sensor mode. Would it make sense to make this controllable via a driver flag or through a libcamera command-line option?

I'm not sure. This could be opening up a can of worms, as API support for it might be lacking both in V4L2 and libcamera.

I suggest for now we simply use the mode_ to decide if the decompanding should be enabled or not. Any additional plumbing to v4l2/libcamera, while nice, would add too much scope to this change.

I agree that using the mode_ information to determine whether decompanding should be applied is a practical approach for now. I will update the patch to make decompanding conditional based on the selected mode.

njhollinghurst reviewed Jul 15, 2025

View reviewed changes

naushir reviewed Jul 16, 2025

View reviewed changes

ipa: rpi: pisp: Fix formatting issues in decompand

7545d16

Adjusted indentation and formatting to comply with project style guidelines,as reported by checkstyle.py. Signed-off-by: Sena Asotani <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ipa: rpi: pisp: Add decompand support using PiSP hardware block #284

ipa: rpi: pisp: Add decompand support using PiSP hardware block #284

Uh oh!

asofam commented Jul 15, 2025

Uh oh!

naushir commented Jul 15, 2025

Uh oh!

njhollinghurst Jul 15, 2025

Uh oh!

njhollinghurst Jul 15, 2025 •

edited

Loading

Uh oh!

asofam Jul 16, 2025

Uh oh!

naushir Jul 16, 2025

Uh oh!

asofam Jul 17, 2025

Uh oh!

naushir Jul 16, 2025

Uh oh!

asofam Jul 17, 2025

Uh oh!

njhollinghurst commented Jul 22, 2025

Uh oh!

asofam commented Jul 23, 2025

Uh oh!

njhollinghurst commented Jul 23, 2025 •

edited

Loading

Uh oh!

naushir commented Jul 24, 2025

Uh oh!

asofam commented Jul 24, 2025

Uh oh!

Uh oh!

ipa: rpi: pisp: Add decompand support using PiSP hardware block #284

Are you sure you want to change the base?

ipa: rpi: pisp: Add decompand support using PiSP hardware block #284

Uh oh!

Conversation

asofam commented Jul 15, 2025

Summary

Changes

1. Decompanding Algorithm

2. PiSP Hardware Support

3. Sensor Tuning Configuration

4. Architecture Consideration

Rationale

Notes

Testing

Signed-off-by

Uh oh!

naushir commented Jul 15, 2025

Uh oh!

njhollinghurst Jul 15, 2025

Choose a reason for hiding this comment

Uh oh!

njhollinghurst Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asofam Jul 16, 2025

Choose a reason for hiding this comment

Uh oh!

naushir Jul 16, 2025

Choose a reason for hiding this comment

Uh oh!

asofam Jul 17, 2025

Choose a reason for hiding this comment

Uh oh!

naushir Jul 16, 2025

Choose a reason for hiding this comment

Uh oh!

asofam Jul 17, 2025

Choose a reason for hiding this comment

Uh oh!

njhollinghurst commented Jul 22, 2025

Uh oh!

asofam commented Jul 23, 2025

Uh oh!

njhollinghurst commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

naushir commented Jul 24, 2025

Uh oh!

asofam commented Jul 24, 2025

Uh oh!

Uh oh!

njhollinghurst Jul 15, 2025 •

edited

Loading

njhollinghurst commented Jul 23, 2025 •

edited

Loading