vo_gpu_next: linearize sRGB as a pure 2.2 power function

[kasper93]

The sRGB EOTF is a pure gamma 2.2 function. There is some disagreement regarding the sRGB specification and whether it should be treated as a piecewise function. Many displays are actually gamma 2.2, and content mastered for PC is typically affected by that. Therefore, linearize it as such to avoid raised blacks.

See:
IEC 61966-2-1-1999
https://community.acescentral.com/t/srgb-piece-wise-eotf-vs-pure-gamma/4024
KhronosGroup/DataFormat#19
https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/12
https://github.com/dylanraga/win11hdr-srgb-to-gamma2.2-icm

[github-actions]

Download the artifacts for this pull request:

Windows

• mpv-aarch64-pc-windows-msvc
• mpv-i686-w64-mingw32
• mpv-x86_64-pc-windows-msvc
• mpv-x86_64-w64-mingw32

macOS

• mpv-macos-13-intel
• mpv-macos-14-arm
• mpv-macos-15-arm

[Headcrabed]

@kasper93 What about enable this on ACM-disabled device by default?

Another read material that could be added into documentation: https://projects.blender.org/blender/blender/issues/145022

[na-na-hi]

The sRGB EOTF is a pure gamma 2.2 function. There is some disagreement regarding the sRGB specification and whether it should be treated as a piecewise function.

I am not seeing any disagreement here. The sRGB specification is very clear about this:

• The sRGB reference display has an EOTF of a pure gamma 2.2 function. This is for the reference display ONLY - it is intended to be used by the sRGB reference display to map the sRGB encoded values to light intensity.
• The piecewise function is used for everything else: converting between sRGB and other color representations like XYZ, which requires the function to be invertible.

The above indicate that the "gamma 2.2" treatment should only be explicitly done during the final output phase when displaying sRGB content on an HDR display (to keep the light intensity the same as on sRGB display). For sRGB display which accepts sRGB-encoded values, no conversion should be done. The source should also not be treated as encoded in gamma 2.2 in any situation.

[Headcrabed]

The above indicate that the "gamma 2.2" treatment should only be explicitly done during the final output phase when displaying sRGB content on an HDR display (to keep the light intensity the same as on sRGB display). For sRGB display which accepts sRGB-encoded values, no conversion should be done. The source should also not be treated as encoded in gamma 2.2 in any situation.

@na-na-hi Similar discussions happened on every link provided above.... Please read some of them.

[kasper93]

@na-na-hi: I provided 4 links with discussions that should explain to you why this option exists and why it is useful when linearizing sRGB to encode in PQ. (also this is a draft for a reason...)

[na-na-hi]

@na-na-hi Similar discussions happened on every link provided above.... Please read some of them.

I read through these links, and everyone there is confused by the ambiguous use of terms "EOTF" and "OETF". This is the main reason why they cannot comprehend that both of my points can be correct at the same time, and have to declare one of them wrong to resolve the apparent conflict.

Even Jack Holm, who is mostly correct on this issue, is confused: In his email he said "The sRGB standard does not specify an OETF", while at the same time saying that sRGB->XYZ is specified using a two part EOTF. However, both sRGB->XYZ and XYZ->sRGB conversion formulas are defined, which means that if the conversion specifies EOTF, it must also specify OETF, which conflicts with his other statement.

By applying the strict standard regarding these terms to only refer conversions between real, physical light and digital encodings, it can be concluded that the sRGB standard indeed does not specify an OETF. It only specifies a gamma 2.2 EOTF. The sRGB->XYZ and XYZ->sRGB conversion is only between digital values and has no bearing on OETF/EOTF in a strict sense.

He is also wrong on the gamma 2.2 being an "approximation": nowhere is the standard says it is an approximation, and the gamma 2.2 formula is presented in an unambiguous way. In fact, it refers to Annex A, which directly rejects the usage of ambiguous "gamma" term, suggesting the intention of an accurate power law representation. sobotka also argued why it is the case (although also with confusion about EOTF/OETF terminology). This is easier to understand considering that the sRGB->XYZ conversion is NOT an EOTF. The purpose is to convert sRGB into other colorspace specifications, and that colorspace is converted to display using the appropriate display EOTF.

@na-na-hi: I provided 4 links with discussions that should explain to you why this option exists and why it is useful when linearizing sRGB to encode in PQ

But I did acknowledge the case for mapping sRGB for a target PQ display (and I think this is the correct thing to do, since mpv takes the role of sRGB reference display (exact power 2.2 EOTF) here to map sRGB to light intensity). What is the problem here?

[kasper93]

But I did acknowledge the case for mapping sRGB for a target PQ display (and I think this is the correct thing to do, since mpv takes the role of sRGB reference display (exact power 2.2 EOTF) here to map sRGB to light intensity). What is the problem here?

Sorry, it's probably me misunderstanding.

The source should also not be treated as encoded in gamma 2.2 in any situation.

I this this was what prompted my reply. There is still a bit of engineering issue on our color management side. Because depending on input and output transfer it need to decide different things. But similarly as with bt.1886 where we use inverse of it to linearize source, I think we can use gamma 2.2 to linearize sRGB. This way we are working in display space linear light. Now the engineering part I mention is to infer what transfer function use to delinearize. For example if target-trc=srgb we would delinearize with gamma2.2 to preserve gamma. For target-trc=pq we would use directly display refered linear light to convert to luminance values. Targeting linear should also be fine, because it should be encoded for display response by compositor, so our "environment encoding" can stay in it.

Does it make sense, if something is not clear let me know.

EDIT: Of course we could also do sRGB decoding and convert to display light later, but I think it effectively would be the same, for our processing sake that we do in linear light. (and hopefully we only linearize / delinearize once)
EDIT2: I will also remove this option, because it's not needed in fact.

[na-na-hi]

But similarly as with bt.1886 where we use inverse of it to linearize source, I think we can use gamma 2.2 to linearize sRGB. This way we are working in display space linear light.

This is the incorrect thing to do according to sRGB standard, because you are linearizing to another colorspace for further processing on the linear light intensity values (scaling, shader, etc) and not to a reference display, so it needs to use the piecewise function, not the gamma 2.2 function.

Of course we could also do sRGB decoding and convert to display light later, but I think it effectively would be the same, for our processing sake that we do in linear light.

It is not the same. In general f(g(x)) is different from g(f(x)). Processing should be done in absolute intensity, not display intensity.

[llyyr]

This is the incorrect thing to do according to sRGB standard, because you are linearizing to another colorspace for further processing on the linear light intensity values (scaling, shader, etc) and not to a reference display, so it needs to use the piecewise function, not the gamma 2.2 function.

Could you point out what section of the spec supports this interpretation? If I understand you correctly, mpv linearizing with power2.2 function as the EOTF would be correct if it were to output sRGB/gamma2.2*, but it would not be correct to do so if mpv were to output some other colorspace, say bt.1886?

*I say sRGB/gamma2.2 but they have the same EOTF function so they should be the same.

[kasper93]

But similarly as with bt.1886 where we use inverse of it to linearize source, I think we can use gamma 2.2 to linearize sRGB. This way we are working in display space linear light.

This is the incorrect thing to do according to sRGB standard, because you are linearizing to another colorspace for further processing on the linear light intensity values (scaling, shader, etc) and not to a reference display, so it needs to use the piecewise function, not the gamma 2.2 function.

Of course we could also do sRGB decoding and convert to display light later, but I think it effectively would be the same, for our processing sake that we do in linear light.

It is not the same. In general f(g(x)) is different from g(f(x)). Processing should be done in absolute intensity, not display intensity.

Well of course, the whole standard is about this.

And yet, this is what is required to preserve perceptual look of the image as it is intended to be viewed. Similarly we don't linearize bt.709 with inverse of camera oetf and instead use bt.1886.

Our rendering pipeline internally is in display intensities (not encoding ones), HDR like. Also with BPC already applied.

This is the incorrect thing to do according to sRGB standard, because you are linearizing to another colorspace for further processing on the linear light intensity values (scaling, shader, etc) and not to a reference display, so it needs to use the piecewise function, not the gamma 2.2 function.

Could you point out what section of the spec supports this interpretation? If I understand you correctly, mpv linearizing with power2.2 function as the EOTF would be correct if it were to output sRGB/gamma2.2*, but it would not be correct to do so if mpv were to output some other colorspace, say bt.1886?

*I say sRGB/gamma2.2 but they have the same EOTF function so they should be the same.

This is what is defined by sRGB standard and the "mismatch" is discussed in introduction

[llyyr]

This is what is defined by sRGB standard and the "mismatch" is discussed in introduction

This doesn't explain why it would be incorrect to use power2.2 EOTF if mpv were to output bt.1886 or PQ. (I'm still waiting for nanahi to say that this is what they meant)

From my understanding, such an operation would be as follows:

srgb video -> mpv linearizes using power2.2 -> work in linear light -> mpv delinearizes to pq or bt.1886 or whatever.

From my understanding, nanahi is saying that mpv linearizes using power2.2 step is wrong if mpv would in the end delinearize to any colorspace except sRGB.

[kasper93]

(I'm still waiting for nanahi to say that this is what they meant)

Ok, sorry. I won't interrupt you anymore.

[sobotka]

It is not the same. In general f(g(x)) is different from g(f(x)). Processing should be done in absolute intensity, not display intensity.

As folks are grasping the nuance here, it becomes a question as to what the “correct” composition operation would be.

We can see that if we consider the more obvious case of BT.709 to BT.1886, it is the operator, aka the display transfer characteristic and colourimetry, that determines the relative wattages of the presented stimuli. It would appear be unsound advice to suggest that compositing the presented stimuli of window A over window B should be decoded to relative wattages using the inverse of the BT.709 encoding characteristic; if the goal were to be to emulate a composition as though the planes of the windows were physicalist / materialist panels, it is strictly the presented stimuli relative wattages that matter here.

Given that the operator drives the meaning of the data encoding, we can see some sense for considering the PQ or HLG EOTF as driving the compositing of the presentation mechanisms. If we remove the composition out of the equation, the normative presentation of an sRGB two part encoding is a vanilla 2.2 EOTF, and to draw an equivalent presentation, the relative wattages should align. Therefore the presenting stimuli state would logically be the appropriate encoding for the output data state operation; placing the sRGB encoding into the PQ / HLG encoded state as though it were being emitted from a 2.2 EOTF presenting medium.

It follows that if we agree the presentation state is the driving function, then as with the BT.1886 example, the normative 2.2 operator as decoding for presented relative wattages should be employed for composition.

[na-na-hi]

This doesn't explain why it would be incorrect to use power2.2 EOTF if mpv were to output bt.1886 or PQ. (I'm still waiting for nanahi to say that this is what they meant)

From my understanding, such an operation would be as follows:

srgb video -> mpv linearizes using power2.2 -> work in linear light -> mpv delinearizes to pq or bt.1886 or whatever.

From my understanding, nanahi is saying that mpv linearizes using power2.2 step is wrong if mpv would in the end delinearize to any colorspace except sRGB.

PQ specifies the displayed luminance. It is different from the absolute luminance level converted from sRGB using the piecewise function.

If mpv is outputing to PQ the correct pipeline should look like this:

sRGB video -> mpv linearizes using sRGB piecewise function -> work in absolute linear light -> processed absolute linear light -> mpv delinearizes using sRGB piecewise function -> processed sRGB video -> mpv linearizes to display light using power2.2 -> mpv delinearizes to pq

Note that the "processed absolute linear light -> mpv delinearizes using sRGB piecewise function -> processed sRGB video -> mpv linearizes to display light using power2.2" step can be a single "absolute linear light -> display linear light" step if we precompute the relationship.

So what this PR should do is the following: when source is sRGB and target display does not accept sRGB encoded values, add the "absolute linear light -> display linear light" step before converting from linear colorspace to output colorspace.

[Headcrabed]

@kasper93 By setting to "Both" by default, you are breaking ACM support....

[kasper93]

@kasper93 By setting to "Both" by default, you are breaking ACM support....

By not setting both by default, we are breaking Wayland support. I'm aware of that, and frankly you should be using PQ output from mpv if your display is in HDR mode.

[sobotka]

sRGB video -> mpv linearizes using sRGB piecewise function -> work in absolute linear light -> processed absolute linear light -> mpv delinearizes using sRGB piecewise function -> processed sRGB video -> mpv linearizes to display light using power2.2 -> mpv delinearizes to pq

It is not logical to suggest that the direct to PQ relative stimuli wattages should be power 2.2, and that the compositing of two “layers” be calculated differently, using the two part decoding.

If we consider two buffers A and B, which are “unoccluded” and presented under the 2.2 power function simultaneously on the display medium, the relative “linear” wattages follow from the 2.2 power function for both regions.

If we attempt to emulate moving A over B, and occluding B by some degree of A for an “overlapping window” emulation, it is logical that the as presented relative wattages derived from the 2.2 power function should be employed. The measured energy at the faceplate would indicate the power 2.2 output, not the two part input.

[Headcrabed]

@kasper93 By setting to "Both" by default, you are breaking ACM support....

By not setting both by default, we are breaking Wayland support. I'm aware of that, and frankly you should be using PQ output from mpv if your display is in HDR mode.

No it won't break. haasn/libplacebo@4d4938d would make it choose linear color space on Linux and macOS. Only windows ACM uses VK_COLOR_SPACE_SRGB_NONLINEAR_KHR.

[Headcrabed]

Also seems on some devices, SDR 10bit out would only work when ACM is enabled... And PQ won't help here.

[llyyr]

haasn/libplacebo@4d4938d would make it choose linear color space on Linux and macOS

FWIW linear output is wrong unless you set --target-contrast=inf... except the option is also buggy on mpv and doesn't have the same result as in plplay (if you set min_luma to the 1e-6 minimum). I will look into this sometime soon when I have the time

[Headcrabed]

haasn/libplacebo@4d4938d would make it choose linear color space on Linux and macOS

FWIW linear output is wrong unless you set --target-contrast=inf... except the option is also buggy on mpv and doesn't have the same result as in plplay (if you set min_luma to the 1e-6 minimum). I will look into this sometime soon when I have the time

But we don't have any other better choices... Linear, sRGB and linear scRGB are the only choices on some Wayland compositors, and some of them just implement sRGB piece-wise as 2.2.

[mahkoh]

RE the rest of the discussion here:

One of the goals of the wayland color management protocol (possibly the goal) is to have the same content look the same regardless of the color space conversions performed, as long as the final and all intermediate color spaces can correctly represent the content. For example, consider the following two pipelines:

Pipeline 1:

1. mpv plays sRGB video
2. mpv sends the video to the compositor using sRGB color description
3. compositor converts to PQ
4. compositor displays on HDR10 display

Pipeline 2:

1. mpv plays sRGB video
2. mpv converts to PQ
3. mpv sends the video to the compositor using PQ color description
4. compositor displays on HDR10 display

To achieve the goal, mpv and the compositor must agree how to convert sRGB to PQ. If compositors always linearize sRGB content using gamma22, mpv has to do the same if it wants to achieve the same goal. Whether or not that is defined in any standard other than wayland is irrelevant.

[Headcrabed]

RE the rest of the discussion here:

One of the goals of the wayland color management protocol (possibly the goal) is to have the same content look the same regardless of the color space conversions performed, as long as the final and all intermediate color spaces can correctly represent the content. For example, consider the following two pipelines:

Pipeline 1:

1. mpv plays sRGB video

2. mpv sends the video to the compositor using sRGB color description

3. compositor converts to PQ

4. compositor displays on HDR10 display

Pipeline 2:

1. mpv plays sRGB video

2. mpv converts to PQ

3. mpv sends the video to the compositor using PQ color description

4. compositor displays on HDR10 display

To achieve the goal, mpv and the compositor must agree how to convert sRGB to PQ. If compositors always linearize sRGB content using gamma22, mpv has to do the same if it wants to achieve the same goal. Whether or not that is defined in any standard other than wayland is irrelevant.

And that's why I made this commit. haasn/libplacebo@4d4938d

[kasper93]

To achieve the goal, mpv and the compositor must agree how to convert sRGB to PQ. If compositors always linearize sRGB content using gamma22, mpv has to do the same if it wants to achieve the same goal. Whether or not that is defined in any standard other than wayland is irrelevant.

Yes, that's clear. Currently mpv agrees with what Windows does. This PR will allow to match Wayland behavior. I like the platform as a default, which will work on both platforms and frankly avoids the headache of deciding what should be the default. Smarter people already decided.

And that's why I made this commit. haasn/libplacebo@4d4938d

Linear output is currently not supported on Windows. We don't implement scRGB which would be possible to use on Windows. Also PQ output from mpv should be compatible with whatever compositors are doing, unless compositors are wrong, but that's why this option can be adjusted by user to decide for themselves.

[Headcrabed]

haasn/libplacebo@4d4938d would make it choose linear color space on Linux and macOS

FWIW linear output is wrong unless you set --target-contrast=inf... except the option is also buggy on mpv and doesn't have the same result as in plplay (if you set min_luma to the 1e-6 minimum). I will look into this sometime soon when I have the time

@llyyr Sorry to bother but can we get this fix before 0.41 comes out?

[llyyr]

Sorry to bother but can we get this fix before 0.41 comes out?

Sorry, I must've been testing wrong because target-contrast does work. However we need to set target-contrast=10000 for linear trc to look similar to how bt.1886 does for bt.709 content, which I'm not sure as to why. But this isn't tied to mpv, but instead a libplacebo issue

[Headcrabed]

Sorry to bother but can we get this fix before 0.41 comes out?

Sorry, I must've been testing wrong because target-contrast does work. However we need to set target-contrast=10000 for linear trc to look similar to how bt.1886 does for bt.709 content, which I'm not sure as to why. But this isn't tied to mpv, but instead a libplacebo issue

Maybe we can change auto for target-contrast to behave like inf when linear TRC is selected?

[kasper93]

Sorry to bother but can we get this fix before 0.41 comes out?

Sorry, I must've been testing wrong because target-contrast does work. However we need to set target-contrast=10000 for linear trc to look similar to how bt.1886 does for bt.709 content, which I'm not sure as to why. But this isn't tied to mpv, but instead a libplacebo issue

Maybe we can change auto for target-contrast to behave like inf when linear TRC is selected?

This needs change in libplacebo to add ability to scale linear on output. Now delinearize is noop if already linear and this is where we would need to scale according to contrast.

[Headcrabed]

Sorry to bother but can we get this fix before 0.41 comes out?

Sorry, I must've been testing wrong because target-contrast does work. However we need to set target-contrast=10000 for linear trc to look similar to how bt.1886 does for bt.709 content, which I'm not sure as to why. But this isn't tied to mpv, but instead a libplacebo issue

@llyyr Btw I get better visual result on Wayland KDE with target-contrast=inf than target-contrast=10000

[rijnhard]

Might be worth adding this to the list of sources, since there was some serious debate over on mesa as well.

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36444#note_3104620

And recent wayland side as a result
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/456

[llyyr]

Might be worth adding this to the list of sources, since there was some serious debate over on mesa as well.

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36444#note_3104620

And recent wayland side as a result https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/456

Both of them stem from the same sources already linked in the commit description, in fact it's even the same people discussing

[kasper93]

I've updated with a workaroud for libplacebo linear issue, fixed by https://code.videolan.org/videolan/libplacebo/-/merge_requests/759 but before it's merged we need to support older libplacebo versions too.

I will merge this PR soon, it's not perfect, because we depend on moving target currently, but it makes current status a bit better.

[kasper93]

I changed libplacebo workaround to switch to sRGB, which itself can be adjusted with other options. We can iterate on default values for that and fix linear output too.

[Headcrabed]

I changed libplacebo workaround to switch to sRGB, which itself can be adjusted with other options. We can iterate on default values for that and fix linear output too.

I would advice to just don't add this commit... Since some Wayland compositors are still using piecewise function for sRGB. Will it take long time before that PR lands?

[kasper93]

I changed libplacebo workaround to switch to sRGB, which itself can be adjusted with other options. We can iterate on default values for that and fix linear output too.

I would advice to just don't add this commit... Since some Wayland compositors are still using piecewise function for sRGB. Will it take long time before that PR lands?

What would you suggest? At least this PR adds an option, so you can control things. Wayland and Mesa is currently revising this sRGB thing and I'm not sure what will be the end result of it.

We could use VK_COLOR_SPACE_PASS_THROUGH_EXT and set G2.2 directly, but frankly I feel like using Vulkan shouldn't require use of Wayland protocols for color descriptors.

Either way, there is no good solution, neither current master is working well.

Even fixing linear output in libplacebo, is not that great because by using 16bit surface we tradeoff some memory bandwidth.

[llyyr]

This might be a different bug, but bt709 content have hint.transfer GAMMA22. Not BT1886. So with this PR now, we'll decode bt709 with gamma22 instead of bt1886.

[kasper93]

Excepted.

So with this PR now, we'll decode bt709 with gamma22 instead of bt1886.

Will we? Show stats.lua screenshot with default config.

[llyyr]

Never mind, bt.1886 is still picked but the score difference is much smaller now because bt.1886 doesn't get if (space.transfer == hint->transfer) score += 20000; anymore. Though bt.1886 still gets ranked higher.

[kasper93]

Strange, it's should output to VK_COLOR_SPACE_SRGB_NONLINEAR_KHR

[kasper93]

Changed to --treat-srgb-as-power22=input by default, there is currently nothing that say that VK_COLOR_SPACE_SRGB_NONLINEAR_KHR should not be encoded with sRGB piecewise function. And if you know that your compositor know better and scans it out or read as G2.2, you can change the option.

[kasper93]

Also note that I will probably revert all those hacks at some point in the future.

[Headcrabed]

@kasper93 I found something weird. With current git master code+KDE6.5, when playing SDR video, shift+i shows display is BT.1886, however mpv log shows it picked VK_COLOR_SPACE_SRGB_NONLINEAR_KHR, not VK_COLOR_SPACE_BT709_NONLINEAR_EXT.

[kasper93]

@kasper93 I found something weird. With current git master code+KDE6.5, when playing SDR video, shift+i shows display is BT.1886, however mpv log shows it picked VK_COLOR_SPACE_SRGB_NONLINEAR_KHR, not VK_COLOR_SPACE_BT709_NONLINEAR_EXT.

Yes, this is expected. hint-mode=target is using preferred colorspace reported by compositor. sdr-adjust-gamma=no is preventing gamma conversions. So you end up outputting bt.1886 to sRGB surface. Just like was requested in this thread.

[Headcrabed]

@kasper93 I found something weird. With current git master code+KDE6.5, when playing SDR video, shift+i shows display is BT.1886, however mpv log shows it picked VK_COLOR_SPACE_SRGB_NONLINEAR_KHR, not VK_COLOR_SPACE_BT709_NONLINEAR_EXT.

Yes, this is expected. hint-mode=target is using preferred colorspace reported by compositor. sdr-adjust-gamma=no is preventing gamma conversions. So you end up outputting bt.1886 to sRGB surface. Just like was requested in this thread.

Ahhh I understand now. Thanks for clarify. Besides, shouldn't auto behave like yes when running on macOS? It's always color managed.

Also, I'm thinking about implementing ACM detection in MPV, do you know which file should I start with?