The color is brighter on Quest3?

[bingothreed]

It seems like that on Quest3, the app shows brighter color?

In Openxr Spec, SRGB is recommended:
https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#rendering-swapchain-image-management

And also, in Quest3 official sample code, SRGB color format is also used for rendering: https://github.com/meta-quest/Meta-OpenXR-SDK/blob/main/Samples/XrSamples/XrCompositor_NativeActivity/Src/XrCompositor_NativeActivity.c#L1842

I have also done some experiments, if I follow the instructions that Oculus sample shown to us, I can get desired colors on the screen. That is to say: if I program the glclear value as (0.5,0.5,0.5), the color on Quest3's screen is (127,127,127). It is equal to middle gray 255/2=127.5

But with OpenXR-SDK-Source, the color one the screen is (187,187,187), it is more like gamma correction is applied to (127,127,127).

【Note】While using sRGB in the code, we need to use GL(glDisable(GL_FRAMEBUFFER_SRGB_EXT)) as Quest has done:
https://github.com/meta-quest/Meta-OpenXR-SDK/blob/main/Samples/XrSamples/XrSceneModel/Src/SceneModelGl.cpp#L1089

From my point of view, the color should on the screen should be the same as what I wrote in the code, not gamma corrected.

[bingothreed]

After doing some experiments, I can manage the color on the screen. Here is an example.

Use grey color to debug

Change the color of the boxes to grey (0.5,0.5,0.5) in https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/main/src/tests/hello_xr/geometry.h#L33

Default results

The default program on Quest3 will show (188,188,188) which is much brighter than (127,127,127).

What if we want to get (127,127,127) on Quest3 screen

GLES

Similar work like Quest3 official sample code
Add glDisable(GL_FRAMEBUFFER_SRGB_EXT) in this function https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/main/src/tests/hello_xr/graphicsplugin_opengles.cpp#L140

Then you will get (127,127,127) on Quest3 for Hello_xr-gles

Vulkan

Pay attention to vk format and vk color space. I have noticed that on my PC, only one combination can show (127,127,127) on my computer screen, that is RGB vk_format and SRGB vk_colorspace.

Following this, the only thing we need to do is to change the vkFormat to VK_FORMAT_R8G8B8A8_UNORM at: https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/main/src/tests/hello_xr/graphicsplugin_vulkan.cpp#L1007

As for SRGB color space, the swapchain color format already help us achieve this at https://github.com/KhronosGroup/OpenXR-SDK-Source/blob/main/src/tests/hello_xr/graphicsplugin_vulkan.cpp#L1555

I don't know if it is correct, but I think if I write (0.5,0.5,0.5) in my program, I really hope the color on the screen shows me (127,127,127), not (188,188,188)

[rpavlik-bot]

An issue (number 2433) has been filed to correspond to this issue in the internal Khronos GitLab (Khronos members only: KHR:openxr/openxr#2433 ), to facilitate working group processes.

This GitHub issue will continue to be the main site of discussion.

[SafariMonkey]

Above the proposed fix in the Meta sample is the comment:

        // This app was originally written with the presumption that
        // its swapchains and compositor front buffer were RGB.
        // In order to have the colors the same now that its compositing
        // to an sRGB front buffer, we have to write to an sRGB swapchain
        // but with the linear->sRGB conversion disabled on write.

This does not sound like the application is a very good example of "correct" sRGB handling. It sounds like it might be, for example, rendering 0.5 as 127 to a texture that is later interpreted as sRGB, thus resulting in a linear color of 0.212 when interpreted by the compositor. This may be consistent with the sample's previous behaviour, but mesh color and shader output is usually linear color, not sRGB.

In the explanation on how to debug this, cubes of color "grey (0.5,0.5,0.5)" are used. The value written to the texture for these should correspond to a linear color of 0.5, which when using an sRGB texture should be around 187-188. This should later be interpreted, based on the non-linear encoding of the texture, as a light intensity of approximately 0.5. (This does look closer to white than black, but that's because our eyes don't perceive brightness linearly - that's exactly why we use non-linear sRGB!) If the compositor is not handling texture color spaces correctly, and the output for the same mesh color differs between sRGB and linear textures, it should be caught by the GradientFormatsLinearVsNonLinear test.

[bingothreed]

Hi, @SafariMonkey

Thanks for explaining the details. Do you mean considering the human eye is non-linear sRGB, the pixel value 188 is more reasonable on the screen when we use 0.5 in program.

There is always a user case: we want to render a specific color on the screen like (127,127,127), how should we implement this? Using the method like Quest2? Or find the value which is equal to 127 in non-linear sRGB?

[SafariMonkey]

Do you mean considering the human eye is non-linear sRGB, the pixel value 188 is more reasonable on the screen when we use 0.5 in program.

No. What I am saying is that 188 in non-linear sRGB encoding is used to represent a light level of 0.5, i.e. the same average light level as a black and white checkerboard. The reason for this is that we want to minimise how noticeable banding artifacts are. Because the difference between e.g. 5% absolute brightness and 6% absolute brightness is much more noticeable than 95% brightness and 96% brightness, we use an encoding that has the steps closer together for darker shades and farther apart for brighter shades. This is what we mean when we say a texture is sRGB encoded: it uses more values to represent darker shades than lighter shades, so to represent a brightness of 0.5, it uses a number around 187-188. If you wrote 127 to that texture instead, it would not output 50% brightness, it would output 21% brightness, because 127 is not how that texture represents 50% brightness.

You already imply that 127 is the "correct" representation of 0.5, but this is just an encoding. What you mean is that 127/255 is close to 0.5. But sRGB textures don't represent their values using the function x/255, but a more complicated function we'll call from_sRGB(x).

Imagine we encoded some data with the function √x, to be decoded with the function x². If I want to calculate half of an encoded value of 10, that represents 10², so I cannot just divide the 10 by 2 and get 5². I must expand the function to get 100, halve it to get 50, and then re-encode it by wrapping it with decode(encode(x)) to get (√50)²=(5√2)²≈(7.07)². In the same way, an sRGB texture is encoded with the function to_sRGB(x), to be decoded with from_sRGB(x). (This decoding function is automatically applied for every texture lookup on an sRGB texture when linear brightness is required.) A pixel of maximum brightness is represented by 255, and its linear brightness can be represented by the expression from_sRGB(255). If we want the expression in that form that produces approximately half that linear brightness, we must follow the same procedure as above. First we resolve the from_sRGB function to get 1.0, then we halve the linear value to get 0.5, then we express that as from_sRGB(to_sRGB(0.5)), and finally we resolve the inner to_sRGB, giving ~from_sRGB(187.5). 187.5 is thus the value that we must write to the encoded texture in order to see a brightness of 0.5 on the screen.

In summary, no, 187-188 in an sRGB texture is not incorrect, and will result in approximately the same brightness on the screen as 127 in a linear texture. It's a different encoding of the same data.

[bingothreed]

Hi @SafariMonkey

Thanks for your detailed explanation. Do you mean that all the color values while designing an application should be processed with to_sRGB() to get the expected color on the screen since the from_sRGB() is automatically applied in decoding procedures.

I'm curious why it's rare to see code manually applying encoding functions including the sample code that Quest provided.

And also, there is a standard for web_colors: https://en.wikipedia.org/wiki/Web_colors; it says that mid-grey is #808080 which is (128,128,128) in the sample picture. If I want to design a web browser with mid grey via sRGB texture, the correct way is to use to_sRGB(0.5), then automatically decoding procedures make 127 for me on the screen?

[SafariMonkey]

I'm curious why it's rare to see code manually applying encoding functions including the sample code that Quest provided.

The functions I used, from_sRGB and to_sRGB, are not functions you would see in code a lot of the time when dealing with sRGB textures because they are implicit. By default, all writes implicitly convert to sRGB and all reads implicitly convert from sRGB. (The Quest code here is an anomaly - most code using sRGB textures both read and written with shaders will use both the implicit encoding and the implicit decoding. This is why it has a comment explaining why it's doing a weird thing.)

Do you mean that all the color values while designing an application should be processed with to_sRGB() to get the expected color on the screen since the from_sRGB() is automatically applied in decoding procedures.

In the CTS, for example, the only time we manually convert sRGB is that we convert any non-sRGB-encoded images to sRGB in CreateStaticSwapchainImage, which writes raw colour data to an sRGB texture. Converting from 8-bit non-sRGB to 8-bit sRGB is not recommended but I think we are trying to only rely on 8-bit sRGB being supported). All other sRGB textures are either loaded from data that's already been sRGB encoded, or they use the implicit sRGB encoding on write. But yes, if you are writing the raw contents of an sRGB texture, you do need to make sure your data is sRGB-encoded as we do there. (Ideally, this should be done before it's converted down to 8 bits, but in that case we don't have a choice.)

And also, there is a standard for web_colors: https://en.wikipedia.org/wiki/Web_colors; it says that mid-grey is #808080 which is (128,128,128) in the sample picture. If I want to design a web browser with mid grey via sRGB texture, the correct way is to use to_sRGB(0.5), then automatically decoding procedures make 127 for me on the screen?

I am no expert on web standards, but that appears to be the sRGB encoding as a hexadecimal value, as reinforced by this quote:

A linearized variant of sRGB is also defined for color mixing.

implying that the default is non-linear sRGB. (sRGB is mostly referenced on that page as a gamut, not an encoding, which makes it harder to interpret.) I don't feel like reading the standard right now, so I'll assume it's correct for purposes of this discussion.

In that case, if you wanted to implement a web browser that wrote these colours from a shader, that is indeed a case where you would need to either first convert the colours to a linear form (to_sRGB, as I called it), or you would need to disable sRGB conversion on write as you have. If you also wanted to do operations like blending, they require linear colour to do correctly, so I would probably choose the former option and convert your colours to float32 (or even float16!) since there are so many more float values that I'm pretty sure the operation would be lossless. If you instead turned off sRGB-on-write, you would have to do some additional calculations to implement blending correctly.

This does actually seem to mean that many of the colour definitions we use in the CTS, like DarkSlateGray, do not actually match the WebColor definition, since the latter is supposed to already be sRGB encoded, while the former is not, and is interpreted as linear colour. (For example, if you take the first number in our definition of DarkSlateGrey, you see that 0.184313729f*255 = 47.000... which shows that it has not been sRGB decoded. And, indeed, it looks more like a cyan than the dark colour shown on the Wikipedia page.)

Like with the Oculus sample, we will probably not want to change how our colours appear, since we are not trying to implement a standard by writing DarkSlateGrey, but just defining a colour to use, and we don't necessarily want to change the background colour just because that name should really refer to a different colour. However, we can at least add a comment above our colour definitions explaining that some colours are named per their WebColor names but do not match the intended appearance of those colours due to this discrepancy. Maybe we can even rename the colour to BackgroundCyan or something to avoid the confusion in future.

I hope this has helped to clarify.