sRGB and linear color spaces

[nlguillemot]

Colors in linear spaces currently aren't acting linearly. For example, if you create an ImGui::ColorButton with (0.5,0.5,0.5), you get a pixel with exactly (128,128,128) as the RGB components showing up on your screen. This value of (128,128,128) is actually darker than mid-grey because display devices have a non-linear response to color. (see: http:https://docs.cryengine.com/pages/viewpage.action?pageId=1605651)
If you render to an sRGB framebuffer, then the ColorButton will appear correctly as mid-grey. However, the rest of ImGui is too bright. This problem appears many times in the screenshot thread (#123), where different aspects of ImGui are either too dark or too bright depending on whether the renderer is writing to an sRGB framebuffer or not.

Compare:

In both screenshots there's an example green-blue-ish "clear color" slider. Despite having values generally over 128, it appears darker than a proper sRGB mid-grey (see below). This is because the sliders represent an already encoded sRGB color, meaning that settings the sliders to (128,128,128) will produce a color much darker than mid-grey.

For example, this also affects the color palette the in custom rendering example in the README (below). The exact middle of the palette is (128,128,0), which means that the exact middle of the palette is not actually a yellow shade of mid-grey, instead it appears much darker. The line between the red and green half should actually be yellow, rather than brown.

When manipulating RGBA8 colors, it makes sense for ImGui to treat them as already encoded in sRGB space, since allocating more bits to darker colors means allocating more bits to differences in color more noticeable to humans. However, when manipulating RGBA32F colors, there's no reason to use this compression. Instead, RGBA32F colors should be in linear space, since this is required for blending to be correct. Another confusing aspect is that the font textures are linear RGBA8 (I think?), so the multiplication of non-linear RGB (from eg ColorButton) with the fonts' linear RGB in the pixel shader is just plain wrong.

There's many ways to fix these problems. First off, we should be certain about which color space should be used where in ImGui. Secondly, I suggest that ImGui outputs linear colors in the end, with an optional post-processing step to convert everything to sRGB for renderers that expect everything to arrive already encoded in sRGB. Naturally, we should try to handle this in a way that doesn't break existing ImGui applications.

Good background info for this topic: https://www.youtube.com/watch?v=LKnqECcg6Gw

[ocornut]

Sorry for not answering this earlier, this is partly overwhelming partly not a priority. If you wish to elaborate with a list of concrete things to change I'd be interested.

ImGui doesn't do any computation on colors, those 4 components sliders are completely abstract values from the point of view of the library.

Isn't it just a problem that you can work out by:

• Adjusting the color theme (perhaps we could provide a helper function to convert all those colors)
• Adjusting your renderer/shader? (e.g. that game shot could apply a powf(1/2.2) in the vertex shader?

EDIT I am clueless about sRGB / linear space vs gamma space so will need some assistance here.

[nlguillemot]

Adjusting colors by pow(1/2.2) in the shader sounds like it would work for use with an sRGB rendertarget. My main confusion right now is stuff like ImGui::ColorButton taking float colors but interpreting it as sRGB. I have to think more about this to come up with a thorough analysis. Thanks for your patience Omar!

[darenw]

As a color expert, I can understand the desire for proper color handling. However, one of the key features of ImGui, as I see it, is speed, to draw live data plots quickly. We use ImGui for developers' tools which analyze real-time performance. I'll take speed over correct color, if there's any tradeoff. We can always adjust our RGB values to look good.

[nlguillemot]

fair point.

[nem0]

@ocornut Would you accept PR with new color mode ImGuiColorEditMode_Float?

[meshula]

Themes can cover most of the issues like a correct shade of blue on a button. To Nicholas' original post, things like color pickers should for sure allow a color space to be set, or at least a gamma. Proper color space conversion is a big job that consumes a lot of cycles.

[ocornut]

@nem0 yes probably, but probably start from the (idle) ColorPicker branch.
Feel free to clarify what you mean precisely, in case its work you haven't done yet and you are worried for some reason it wouldn't fit into core.

[sherief]

@nlguillemot I'm assuming the images you posted weren't taken on a High DPI display? correct?

I'm seeing text become blurrier when switching to sRGB, came here wondering what color space is the text --> texture --> screen pipeline working under.

Pictures attached.

[gjaegy]

Hi,

just adding my 2 cents here. We also use a sRGB back buffer, and of course I came across the same issue mentionned by Nicolas :)

So, I wanted to check the status of that issue...

What is the best way of handling this right now, simply by modifying the pixel shader to cancel the linear->sRGB applied on the GPU, i.e.:

	// take into account the fact that our back-buffer is sRGB
	OUT.cCOLOR.rgb = pow(OUT.cCOLOR.rgb, 2.2);

For me that option is working correctly and I can afford the cost for that additional instruction...

[matt77hias]

@gjaegy The vertex should already have a color attribute in linear color space instead of using a color attribute in sRGB color space + applying an approximate sRGB-to-linear conversion in the Pixel/Fragment Shader. The barycentric interpolation of the color attribute for the fragment generation will be wrong, since the color isn't linear.

To be correct, one should change the following

struct ImDrawVert 
{
    ImVec2  pos;
    ImVec2  uv;
    ImU32   col; // currently in sRGB color space
};

to

struct ImDrawVert 
{
    ImVec2  pos;
    ImVec2  uv;
    ImVec4  col; // should be in linear color space
};

and apply a color conversion from sRGB-to-linear space when writing the color value.

This also means that the shaders will stay the same. Only the vertex generation on the CPU will be affected.

Or the vertex shader needs to perform the conversion. This will probably result in the smallest change. I didn't thought of this shader, though it is probably the most appropriate and convenient converter. :)

[gjaegy]

Hi Matthias, good point, I didn't think about the vertex color actually.
Not even sure we use it actually, will have to check!

[matt77hias]

A short explanation/summary of the relevant color spaces:

The actual color of a pixel, outputted on a monitor, does not linearly depend on the applied voltage signal for that pixel of the monitor. For CRT monitors, the actual color is approximately proportional to the applied voltage raised to the power of a so-called gamma value, which depends on the monitor. This gamma value typically lies between 2.2 or 2.5 for CRT monitors. To bypass this gamma value, you need to gamma correct the computed colors of each pixel before presenting. By raising the computed color to a power of the reciprocal of that same gamma value, the computed color becomes proportional to the actual color. Formally:

L_actual ~ V^gamma
V ~ L_computed^(1/gamma)
=> L_actual ~ L_computed

So in general gamma correction is a technique that adapts the computed colors to the transfer function of the monitor used for outputting. Non-CRT monitors each have their own transfer function. This means that to obtain correct actual colors, the final rendering pass should adapt the computed color depending on the used monitor.

These linear-to-gamma color space conversions seem like unnecessary overhead from a monitor construction point of view. It is physically perfectly possible to construct a CRT monitor with a gamma value of exactly one, ensuring that the computed color is already proportional to the actual color, and eliminating the need for gamma correction.

From a perceptual point of view, removing the need for gamma correction and using a monitor where computed colors are proportional to actual colors is actually a bad idea. Typically, actual colors are represented with 8 (or 10) bits for each of the red, green or blue channel. This quantization only supports 256 (or 1024) different colors. Here, a 0 value represents completely black and a 255 (or a 1023) value represents completely white. But what about the intermediate values? If a linear encoding is used (i.e. a gamma value of 1), the majority of values would be perceptually very close to white and a very small minority would be perceptually close to black. By using a gamma encoding (e.g. a gamma value of 2.2), the distribution is perceptually more linear (i.e. equidistant intervals between black and white).

The sRGB color space has similar goals, assigning a perceptual linear range of color values to the available 256 different color values. A rough approximation is transforming linear color values to sRGB color values by raising to a power of 2.2. A more accurate approximation would distinguish between a more linear correspondence near black and a gamma (2.4) encoding near white. And finally, you can just use the exact transformation between linear and sRGB color spaces. The more accurate, the more expensive the calculation will be. The sRGB color space is obviously used for sRGB monitors which are primarily used for content creation (e.g. textures, etc.). Images with a R8G8B8A8 format are in most cases represented in sRGB color space and definitely not in linear color space (as a user, you need of course to know which color space is used for the encoding). RGB color pickers typically operate in sRGB color space as well.

Colors represented in linear color space can be mutually added and multiplied.
Colors represented in a gamma color space cannot be mutually added, but can be mutually multiplied.
Colors represented in sRGB color space support cannot be mutually added and multiplied.
(I explicitly differentiate between a gamma color space and sRGB color space, since not everyone will use the most cheapest linear-to-sRGB approximation).

This above implies that colors represented in a gamma color space or in sRGB color space cannot be used as vertex attributes during the barycentric interpolation for fragment generation. The color attribute of a vertex needs to be expressed in linear space before passing to the rasterizer stage. The pixel/fragment shader will thus always obtain a fragment color expressed in linear space.

The above applies to vertex attributes, so single color coefficients. But what about textures? For textures there isn't a choice. Most textures will contain colors expressed in sRGB color space, and thus these textures should be explicitly encoded as sRGB colors (e.g. DXGI_FORMAT...SRGB). By using an explicit sRGB texture format, the hardware will perform the sRGB-to-linear conversion when appropriate. So with regard to texture sampling/filtering:

hardware sRGB-to-linear conversion -> filtering != filtering -> user defined sRGB-to-linear conversion

Similarly for blending. If you use blending (i.e. all blending except opaque blending), the hardware will perform the linear-to-sRGB conversion when appropriate if the render target has an explicit sRGB texture format.

blending -> hardware linear-to-sRGB conversion != user defined linear-to-sRGB conversion -> blending

Alternatively, a halffloat render target can and should be used, if linear color values need to be stored instead. This, however, is not the responsibility of ImGui. ImGui should output linear colors, and the user of ImGui should provide an appropriate render target (i.e. R8G8B8A8_SRGB or R16G16B16A16).
(Most games let the user use a custom gamma encoding to adjust brightness instead of the default sRGB encoding. So in these uses cases, ImGui will output to a halffloat render target in a first pass. The final pass will apply the custom gamma encoding and output without blending to a R8G8B8A8 render target.)

So to summarize the changes required to ImGui:

• ImGui uses one font texture by default. This texture should be formatted correctly: sRGB if appropriate. Though, I guess it just contain completely black and white color values which should map to the same values independent of the used color space.
• If ImGui performs (sRGB) color additions and/or multiplications on the CPU, the involved colors should be transformed first to linear color space, then the additions and/or multiplications are applied, finally the resulting colors are transformed back to sRGB color space.
• The vertex shader should transform the vertex color attribute from sRGB to linear color space.

Without knowing all the details of ImGui, I assume only the last aspect should be dealt with.

Application to the D3D11 demo:

main.cpp

sd.BufferDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
-> sd.BufferDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM_SRGB; // perceptually a better choice

imgui_impl_dx11.cpp

static const char* vertexShader =
            "cbuffer vertexBuffer : register(b0) \
            {\
            float4x4 ProjectionMatrix; \
            };\
            struct VS_INPUT\
            {\
            float2 pos : POSITION;\
            float4 col : COLOR0;\
            float2 uv  : TEXCOORD0;\
            };\
            \
            struct PS_INPUT\
            {\
            float4 pos : SV_POSITION;\
            float4 col : COLOR0;\
            float2 uv  : TEXCOORD0;\
            };\
            \
            PS_INPUT main(VS_INPUT input)\
            {\
            PS_INPUT output;\
            output.pos = mul( ProjectionMatrix, float4(input.pos.xy, 0.f, 1.f));\
            output.col = input.col;\
            -> output.col = SRGBtoLinear(input.col); // Using sRGB conversion of choice? Or just gamma of 2.2?
            output.uv  = input.uv;\
            return output;\
            }";

[parasyte]

The colors for the classic style were incorrectly transferred into linear-space without a gamma transform when they were authored. I imagine a mock style was created in PhotoShop or Gimp, and the sRGB values were sampled from there and divided by 255. That leads to really invalid colors, because sRGB (128,128,128) is not mid-gray, but (0.5,0.5,0.5) in linear-space is mid-gray.

The rule of thumb for color space handling is to keep the sRGB color storage format in an array of 4 bytes (values 0-255), and linear-space color storage format in an array of 4 floats (values 0.0-1.0). Any conversion between these formats must go through gamma correction (e.g. the pow(n / 255.0, 2.2) and pow(n, 1.0 / 2.2) * 255.0 approximations)

As @matt77hias mentioned, color pickers usually operate in sRGB space, so it makes sense for it to output (128,128,128) when the user wants a nice dark gray; When this color is transferred to the vertex buffer, though, it needs to go through the gamma-to-linear transformation, which results in approximately (0.2176,0.2176,0.2176). If the framebuffer is sRGB, this color will be rasterized as (128,128,128), as expected. But it's up to the application developer to configure the environment for proper rasterization. And while on the subject, you probably don't want to implement a color picker in linear-space because it will be perceptually incorrect.

This is actually a very easy problem to solve, but it is a convoluted challenge and practically everybody gets it wrong. As long as you stick to the rule of thumb, though, it's hard to screw it up.

[matt77hias]

The rule of thumb for color space handling is to keep the sRGB color storage format in an array of 4 bytes (values 0-255), and linear-space color storage format in an array of 4 floats (values 0.0-1.0).

I see two problems related to this approach:

• ImGui should change the signature of its color pickers to use a uint32_t instead of a float array which will break backwards compatibility.
• Both linear and sRGB color spaces can be represented as 4x floats. Because images and render targets perform a quantization step (i.e. 256 possible values), both linear and sRGB colors eventually end up as uint32_t. In the mean time, however, you can use whatever extra precision, you prefer in my opinion. On the one hand, there are various color spaces (e.g. linear, sRGB, etc.) and on the other hand, there are various quantizations (e.g. 8bit/channel, 10bit/channel, etc.). By chance, the sRGB color space happens to be more perceptually efficient in its 8bit quantization.

[parasyte]

The first point is a good one. But that's why SemVer exists. Breaking changes will have to be made at some point, and it can be done safely.

For the second point, you're also not wrong! The storage format can technically be anything you want. But if you allow room for confusion, you will end up with users doing things they should not. The only practical choice is to make the concrete types incompatible without an explicit transformation.

both linear and sRGB colors eventually end up as uint32_t.

This is the part I disagree with. That may be an implementation detail of the existing code, but it is just that; an implementation detail. Other implementation details are that the colors also eventually end up as four floats on their way to the vertex shader (which may be 16-bit, 24-bit, etc.), and then eventually end up as any number of formats as they are rasterized to a framebuffer (RGBA8, BGRA8, R5G6B5, etc.), and eventually they get emitted as linear voltages. These are implementation details of various storage formats along the fixed and programmable pipelines, but they say nothing about the color space of these storage formats.

The only thing I am arguing here (and I hope I shouldn't have to argue it) is that the storage representation is only important at the API level for end users. If theoretically we had types called ImSRGBColor and ImLinearColor, the only real requirement is that I (as a user of the software) cannot use an ImLinearColor in an API that expects an ImSRGBColor, and vice versa. Wisely choosing different concrete types is one way to accomplish this. Enforcing the requirement means that doing the wrong thing becomes extremely difficult. And in some languages, like the safe subset of Rust, it's impossible. ❤️

[matt77hias]

In many cases, you'll end up with a R8G8B8. But you're right that you can use less or more precision. If you know this in advance, however, you can dedicate less or more precision to the format used for representing these colors on the CPU to save memory.

My color space spectra are implemented as classes inheriting my base "containers" (e.g. F32x3, F32x4, etc.). These child classes do not add extra functionality, but merely provide explicit/non-explicit constructors to support different kind of conversions internally. The notation is very explicit and cannot be missed when browsing code statements. Though, I like ImGui's use of primitive types (float, int, etc.).

[hanatos]

my 2ct: i'm rendering everything in linear rec2020 internally and want to display colour managed for my screen. i like how currently imgui is oblique to colour spaces, so everybody can do as they like. the only change i need is at the very end i need to transform the output like so:

hanatos@608c97e

which i suppose could be a no-op (gamma=1, matrix = identity) for folks who'd rather composite in sRGB/display colour space. to get right ui colours, i un-gamma all theme colours (convert from sRGB to linear) when initing imgui.

[gdianaty]

We developed a quick solution to this problem for DX9:

// SRGB corrects according to https://github.com/ocornut/imgui/issues/578
D3DCOLOR D3DCOLOR_sRGB_CORRECTION(D3DCOLOR color)
{
#define GAMMAVAL 2.2

    // unpack D3DCOLOR into something we can manipulate
    D3DCOLORVALUE color2;
    color2.a = ((color & 0xff000000) >> 24) / 255.0f;
    color2.r = ((color & 0x00ff0000) >> 16) / 255.0f;
    color2.g = ((color & 0x0000ff00) >> 8) / 255.0f;
    color2.b = (color & 0x000000ff) / 255.0f;

    // gamma correct colors...
    color2.r = pow(color2.r, GAMMAVAL);
    color2.g = pow(color2.g, GAMMAVAL);
    color2.b = pow(color2.b, GAMMAVAL);

    // repack the aforementioned data into the colorvalue for returning
    return D3DCOLOR_COLORVALUE(color2.r, color2.g, color2.b, color2.a);
}

A practical example can be found at https://gist.github.com/gdianaty/7b81dacbb28b38afc3ba7be09edc7904

[TheMostDiligent]

Another problem that was not thoroughly discussed here is the implications of alpha-blending performed by the GPU. All ImGui examples perform blending in sRGB space, so a final color output by the display can be expressed as

sRgbToLinear(Src * Alpha + Dst * (1-Alpha))

If colors are manually converted to linear space in the shader, then blending will be equivalent to the following:

sRgbToLinear(Src) * Alpha + sRgbToLinear(Dst) * (1-Alpha)

which clearly is quite a different equation. This manifests itself as color shift which is especially bad on color pickers.

Compare what correct color picker looks like:

with the one that is rendered with sRGB frame buffer and manual conversion of colors to linear space:

If you want to get results that are visually close to ImGui examples when using sRGB framebuffer, the following changes are needed:

1. Use alpha-premultplied blending mode:

SrcBlend = D3D11_BLEND_SRC_ONE;
DestBlend = D3D11_BLEND_INV_SRC_ALPHA;
BlendOp = D3D11_BLEND_OP_ADD;
SrcBlendAlpha = D3D11_BLEND_ONE;

2. Premultiply color with alpha and convert to linear:

float4 out_col = input.col * texture0.Sample(sampler0, input.uv);
out_col.rgb *= out_col.a;
out_col.r = GammaToLinear(out_col.r);
out_col.g = GammaToLinear(out_col.g);
out_col.b = GammaToLinear(out_col.b);

3. Finally, tweak the alpha value as follows:

out_col.a = 1.0 - GammaToLinear(1.0 - out_col.a);

With these changes to the shader, the color picker (and other UI elements) will look properly when rendered to sRGB framebuffer:

[matt77hias]

@TheMostDiligent
Do note that the Rasterizer Stage performs barycentric interpolation of the vertex attributes upon fragment generation. This implies that the sRGB-to-linear conversion and pre-multiplication must be done prior to the Rasterizer Stage.

Fwiiw a quick fix is adding something like

// Convert from sRGB to linear color space before the RS.
output.spectrum = SRgbaToLinear(input.m_spectrum);
// Premultiply with alpha before the RS.
output.spectrum.rgb *= output.spectrum.a;

in your vertex shader.

[TheMostDiligent]

@matt77hias

Do note that the Rasterizer Stage performs barycentric interpolation of the vertex attributes upon fragment generation.

I do remember. However the question is what you are trying to achieve and what problem to solve. Imgui does all math in sRGB space which is mathematically incorrect at each step (barycentric interpolation, texture filtering, alpha blending), but which is also a given now.
My goal was to achieve the same visual results using the sRGB framebuffer (which virtually all rendering applications use) as in imgui samples that use linear framebuffer. So I had to left the barycentric interpolation in sRGB space is this is what imgui samples do and fix alpha blending.