Bug 94921

Allow me to further explain a bit about what I think.

First of all, any PC (or any source device that does not only/mainly serve as a video playback device) should not conform the cea(/hdmi/dp) guidelines in the current "automagical" way. The driver should only conform them in the way of giving capabilities for the users to switch the color range and color space ACCORDING TO THEIR OWN NEEDS/USE CASES and the capability of their monitor. The mechanism might be rational for products like DVD/Blu-ray players since the source they deal with is mostly encoded in YCbCr/limited range, but it's totally not the case for PCs.

As what I have mentioned in the last post script, we can assume that monitors which support YCbCr color space would support limited range since full range is not allowed for YCbCr in both HDMI/DP specs (See Refs.), or inheritly. Hence what color space a monitor supports could imply its capabilitiy on color range.

So if you insists that the driver should conform the cea guideline in the current way, it should only conform when the EDID indicates that the monitor support YCbCr. In other case, the driver COULD NOT have determined its capabilities on color range, and hence SHOULD NOT conform the guidelines NO MATTER WHAT.

References (Freely available after filling out a form, kind of outdated though):
http://www.hdmi.org/manufacturer/specification.aspx Table 6-3
http://www.vesa.org/vesa-standards/free-standards/ Table 5-1

P.S. I withdraw the idea that limited range can be used as default for HDMI. I mentioned the idea only because it's more likely for a monitor with HDMI to support YCbCr and hence limited range than a monitor with only other ports. Yet according to the specs above, it is not a must unless it has another port with existing capabilities of supporting YCbCr or similar. See Section 6.2.3 in the HDMI spec and Table 5-1 in the DP spec.

Created attachment 171651 [details]
Patch to make Automatic equivalent to Full

Created attachment 173511 [details]
Patch completely remove Automatic mode

Automatic is our best-guess to what we should be doing, and yes there are piles of screens out there which need this. In short this is about which set of users we're going to piss of by forcing them to manually correct the kernel's choice. If we'd apply your patches we just get piles more bugs from all the people who have a washed-out grey soup because their screen really needs automatic mode.

And my experience is that there's a lot more HDMI screens out there which really need the automatic mode than which break with that mode, so this will be default.

Thanks anyway for your report and my apologies that this is a lose-lose situation unfortunately.

I don't actually expect you guys to fix this because you have been lying to yourselves.

If you think majority needs limited range you can set limited as default for hdmi.

It's not a best guess. But rather self deceiving.

Also if a HDMI monitor can only take limited range, it would be black crushed rathered than washed.

You don't even know about color range. I just hope nouveau and radeon guys won't do stupid thing like this.

Yeah why should I bother. If you guys see the logic here the patch wouldn't have been signed off. That's why intel drivers are always the worst.

Also I even mentioned that if you really want a best guess, whether edid indicates the support of ycbcr is what you have. Mode (resolution and fresh rate!?) has nothing to do with color range.

Don't insult my (our) intelligence repeated. I say it as a nobody and a intel CUSTOMER.

Please keep the conversation civilized. Thank you.

Defaulting to either full or limited will fail for some sinks. The only sane choice is to follow the specs that actually tell us what to do. That's what auto means here: go by the specs. It is also a much more rational and objective choice than doing what works with the sink that the bug reporter of the day happens to have.

There is no rationality at all when this  mechanism is applied here. If there is, you can easily say something how modes are related to color range. None of you ever talk about a single thing about it here or else. You guys didn't even think about any possible reason that spec exists.

You adopt it just because you can tell your user and customer "shut up, I got my holy bible here", because you don't even bother to think and compromise with a more sensible way. Not even when someone give you multiple possible solutions. Not even a second. That's your implementation of objectivity.

And someone show exactly what I said "civilized way" with the "piles of bug" theory.

The reason I'm pissed is not because I'm annoyed by the fact that I am the "unlucky half". It's because I'm determined to be the unlucky half by a rule which is totally irrelevant to the issue.

Never mind. I shouldn't bother to care anything about this driver since day one I saw that patch. I'm so ashamed because that makes me the most illogical thing here. Reason with unreasonable? How irrational I am.

Hello,

I think Tom Yan is perfectly right on this.

Even if adhering to the spec might sound like a good idea, it totally makes no sense, since basically no display device is adhering to the spec.

Furthermore, determining the range based on the display mode makes no sense at all either, since it doesn't work like that in the real world.

One question though:

Is it possible to change the value from auto to full or limited manually?

If yes: How?

If not: Why not?

I came across the following Arch Wiki section:

https://wiki.archlinux.org/index.php/intel_graphics#Weathered_colors_.28color_range_problem.29

It says:

> Unfortunately, the Intel driver does not support setting the color range
> through an xorg.conf.d configuration file.

Is that true?

Regards

Related to:

https://bugs.freedesktop.org/show_bug.cgi?id=94248

I can't change the RGB range when using wayland, when will intel fix the RGB issue??

I'm using Dell Precision 7520 laptop + Dell e-Port replicator dock with two exactly same monitors and got different color ranges on them. One of dock output is connected to integrated Intel card and another one is connected to nvidia graphcis  (using nouveau). 

I spent a few weeks playing with different cables, different monitors, trying to adjust gamma, trying to switch from nouveau to nvidia blob. 

And finally found that followed command fixes it for me:

xrandr --output HDMI-1 --set "Broadcast RGB" "Full"

It's really not funny and looks horrible from user point of view.

HDMI I can understand why the default output range is 16:235 RGB, as that conforms directly to the old HDMI spec. However, the fact that full-range RGB is not correctly used over DisplayPort is absolutely nonsensical. No monitors that use DisplayPort expect to receive a limited 16:235 RGB picture. A more apt example is that 16:235 is used exclusively for televisions which do not and never will use DisplayPort. After questioning why everything looked so washed-out on my Linux installation, I discovered the xrandr --output DP-1 --set "Broadcast RGB" "Full" command.

However, with the introduction of Wayland and further unwillingness of developers to add color range settings to individual compositors (which is appropriate given such a demand is frankly a massive mistake on the Wayland developers' end), it eventually comes down to literally needing to fish through the i915 module and recompile the kernel completely using a variant of Tom Yan's patch to receive a normal picture. Aside from somewhat caustic remarks by Tom, the issue still stands and his point remains valid over three years later.

For some arcane reason, HDMI output from my i7-7700 HD 630 graphics actually does default to sending full-range color to my monitor. There doesn't really appear to be much rhyme or reason for "following the Intel spec" that clearly doesn't follow decade-long established DisplayPort standards. Dmitry is correct here, this really is horrible from a user point of view.

Actually Intel may have been "correctly" following the DP spec. VESA probably wanted to replace HDMI in the "AV world" so it made sure that it behaves the same as HDMI on CEA modes.

It's just in reality it never made and will never make it (the bosses in the "AV world" will never let DP get on the AMPs or so, because of the patents in HDMI they hold or stuff like that, money/benefits you know), and sadly some vendors ignore that silly part of the spec (which deserves to be ignored though).

As you have mentioned, the really awful part is that Intel implements the attribute in kernel which can only be changed via X. But honestly why would I be surprised? It's i915.

TL;DR, Never buy monitors with both DP and CEA mode as optimal but without color range setting. Try your best to avoid Intel iGPU. (It's not like Intel consider it not okay to sell you broken CPU without recalling it.)

P.S. Does it hurt (for me) to reopen it?

Tom, I am quite glad you could reopen this. I have tried several monitors that take DisplayPort inputs and the result is the same: limited RGB. As consumers, we simply *can't* know whether a certain monitor will automagically be set to full range RGB until we purchase it, and other times we don't have much choice (like in an educational or work environment). The fact that a variety of HP, Dell, Samsung, monitors I've tried default to limited RGB over Displayport using Intel graphics *only on Linux* makes no sense whatsoever. Even if the monitors do/don't conform to the Displayport spec, we also cannot expect manufacturers to include hardware settings for color that very well might get ignored by software anyway. Personally, I have never seen a single monitor with limited/full color settings builtin the hardware. 

The hands-down strangest thing I've seen yet regarding this illogical color-range-by-mode thing Intel is doing occured with a monitor including AMD Freesync settings that increased the hardware reported refresh rate, somehow tricking the kernel to send a full RGB signal only when the display was at an overclocked refresh rate. Monitor modes should simply not be directly connected to color range output at all. 

The best solution by far would be to implement and allow passing kernel parameters regarding things like color range for the i915 module as you (and many others) have suggested. Even with the proptest hack proposed elsewhere for Wayland, KMS things like Plymouth are still stuck in awful limited RGB. Ultimately, this demands a kernel-level fix going forward. Why Intel deliberately chose to regress their Linux drivers by defaulting to limited RGB makes no sense at all, especially when it's clear what they are currently doing (basing color range on display mode) is unhelpful and often incorrect by default for many less well-informed consumers. No monitors I am aware of are even currently manufactured with only AV limited RGB available. Defaulting to full RGB over DP regardless of manufacturing idiosyncrasies just makes sense here, even the Intel patch you referenced from 2013 said as much. 

See, I would love more than anything to have an AMD GPU at my disposal, but as it stands, I have no choice other than Intel integrated graphics on my Elitedesk 800 G3 Desktop Mini enterprise machine. Unfortunately, many other consumers and enterprise users have little choice either. Intel has an incredibly massive share of the market. Something even more upsetting to me about Intel's kernel changes is that the LSPCON HDMI connector on my system board is no longer working properly as of kernels 4.16 and above (there is an active Redhat Fedora bug report open), so I'm generally stuck with using Displayport only. 

I'm finding it somewhat difficult to even repatch the 4.17 kernel to default outputting full RGB, as all sorts of things have radically changed in three years. Could you perhaps update the patch?

Created attachment 277661 [details]
True Color 24bpp patch for Intel Graphics over DisplayPort

Oh my God...a single line was the root of my entire problem (and likely quite a few others' problems) in the intel_dp.c file. Basically, 24-bpp True Color is not accounted for over DisplayPort. The fix is detailed in full with a patch for the current mainline kernel 4.17.11 attached in a separate bug report here: https://bugzilla.kernel.org/show_bug.cgi?id=200711

My display is blissfully usable on Linux again, without any nasty workarounds and without undoing the (now reasonable) "automagical" color range detection. Thank you so much for pointing me in the right direction, Tom! I hope this fix can be merged upstream shortly.

Seriously, Intel, per-port sysfs.

 (In reply to Tom Yan)
> Essentially your patch means to avoid using limited range as default for
> "all DP in reality", which is one of the approaches I proposed, so good luck
> having Intel accpeting it. (No offense but the extra check makes less sense
> than just equate auto to full.)

Oh...wait. My logic was incorrect indeed. Tom is basically right here, as what is CURRENTLY happening in the conditional statement means ANY DP 18bpp display (and with my small patch ANY DP 24bpp display) will result in an immediate short-circuit boolean 'false' value, setting pipe_config->limited_color_range = false. See the unpatched condition below: 

pipe_config->limited_color_range =
  bpp != 18 &&
  drm_default_rgb_quant_range(adjusted_mode) ==
  HDMI_QUANTIZATION_RANGE_LIMITED;

On further inspection, Intel currently isn't even testing whether displays with 18bpp adhere to the CEA/VESA spec, as the function drm_default_rgb_quant_range(...) is never actually executed. Which just makes me confused. Why should those of us with 24bpp True Color or 30+bpp HDR displays *not* be getting full-range color? It doesn't even make sense. So now I suppose we are right back where we started at bug #94921.

I checked the history here, and patches and spec and we (me and Paulo) spend few times in some experiments here. So a few clarification is needed:

- i915 code follows spec. We are not changing that. The spec is the only thing that is common here and we just have hope that all sinks and sources are respecting the spec as the driver is. otherwise it gets messy so easily.
So yes, this should be religiously followed by everyone involved.
- DP spec is not "years old". I checked old and even new released DP-1.4 spec and the recommendation is to respect "CTA (former CEA) range" when the mode is a mode described by CTA-861-F (former CEA-861-F).
- Paulo, who review the initial "automatic" code is also suffering from the automatic option. But on his case the best for him is the limited!!! So user preference is needed here.
- i915 already provide the connector property that could be set per user need in a atomic way. There is nothing special support for X. Wayland could implement the support to toggle that option. We are not providing a connector property throught sysfs. That is out of question. the broadcast range option is a connector property and will stay as is. Option is Wayland to implement the toggle support.

But no, I'm not closing the bug here as "wontfix" this time.

I believe that we have room for improvements on the color
space as documented on ilk_load_csc_matrix():

1. /* FIXME if there's a gamma LUT after the CSC, we should                                                                                                                                                                                               
    * do the range compression using the gamma LUT instead. */

and

2. /* TODO: Check what kind of values actually come out of the                                                                                                                                                                                    
    * pipe with these coeff/postoff values and adjust to get the                                                                                                                                                                                  
    * best accuracy. Perhaps we even need to take the bpc value                                                                                                                                                                                   
    * into consideration.*/

Does the DP spec specifically say that 18bpp color displays should unconditionally receive full range RGB? Because that's what your code is doing.  The function in that conditional statement responsible for checking the CTA range is never called in that case, which means that color depth in bpp alone is being used to determine color range for the connector. Look at the code, that's what has been happening literally since automatic mode was implemented some time ago. Does that follow the DP spec? If so...why don't 24-bit color displays (which are now significantly more common) get the same treatment? Intel's Windows drivers default to full range RGB over Displayport for 24bpp correctly. Honestly I feel kind of insulted by the dismissive reply. This is not a matter of "personal preference", it's something objectively missing and unaccounted for that affects desktop usability on either side badly that there has to be a better solution here. There is no reason all the 24bpp VESA compliant HP business monitors I have tried should not pass the driver's compliance check.

I don't mean to say that the automatic mode selection is an incorrect approach to solving this problem, in fact it seems to be the only thing that makes sense given the reality. But it does appear to be fairly unreliable for users who require either side of the RGB color range (limited/full) and there's quite a bit of unpleasantness when it fails to select the appropriate range because of something in the driver code or a quirk of the monitor itself. 

Exposing color range as an i915 kernel parameter while keeping automatic mode in place would solve both problems. Just like other features of the i915 driver module, this could be added as a sort of manual override to ensure kernel mode setting is correct before the initialization of the display server. As a kernel parameter, this would eliminate the need of compositor workarounds in X and Wayland. Furthermore, it would be ideal for consumers with either limited or full color displays experiencing this issue with the automatic driver mode selecting the incorrect color range.

I apologize for my less than civil reply in the previous comment. This particular issue is very troubling when I'm stuck on the wrong end. Still...I think the point merits some thought. I look forward to developments regarding the improvements you suggested for color space, particularly the "need to take the bpc value into consideration."

Yes, Table 5.1.1 exclude 18bpp from the CTA range.

As I told i915 already provides the interface for manual override. That's on the connector level. We are not adding another parameter or a sysfs interface.
The standard interface already exist.

If we don't keep the discussion at civilized levels I'm going to just close it again.

I am unsure why the function call "drm_default_rgb_quant_range(adjusted_mode)" seems to always return the value corresponding to "HDMI_QUANTIZATION_RANGE_LIMITED" on every DisplayPort connector of each monitor I try. That is perhaps the real question, and the core of the bug I rather missed. Over native HDMI, interestingly, the i915 driver always selects the correct color range for me, but only limited range color is ever selected over DisplayPort. 

I understand now that the code conforms to spec, and I don't wish to argue about that. What I don't get is why I always end up with washed-out limited RGB when I should be getting full RGB. I seem to be experiencing the same problem as others in this active bug report https://bugs.freedesktop.org/show_bug.cgi?id=100023 , from which the following comment of mine seems appropriate:

(In reply to Michał Kopeć from comment # 21)
> DisplayPort output defaults to 16-235 color range, while any other outputs
> on the laptop will correctly output 0-255 full RGB. There is currently no
> way to set the driver to default to Full RGB regardless of what the display
> EDID is.

Michal's comment sums up exactly what is currently happening. 

(In reply to N. W. from comment # 8)
> That being said: Please just fix
> https://bugzilla.kernel.org/show_bug.cgi?id=94921 instead of closing it
> without listening to your customers and the other issue will solve itself.

I wholeheartedly agree with N.W. that something more needs to be done about https://bugzilla.kernel.org/show_bug.cgi?id=94921

However, my experience with Intel has been similar to Tom Yan's on https://bugzilla.kernel.org/show_bug.cgi?id=94921 and to N.W.'s on this bug thread. I don't think our concerns are really being taken into account here, and personally I feel like my failed attempts to figure out what precisely is going wrong were unkindly and summarily dismissed. I would greatly appreciate if Intel could take this issue more seriously and propose some kind of fix instead of threatening to close the issue entirely again. 

This is not a personal matter or an attack on Intel graphics developers, nor should it be treated as such. There is objectively something wrong when every DisplayPort output connector powered by Intel integrated graphics defaults to washed-out, limited 16-235 color on modern full range color displays. This issue is mainly about general usability of Intel graphics over DisplayPort on Linux, which likely affects far more users than the few of us who dug deep enough to find this bug report here.

I can confirm this bug, I've resorted to a systemd unit to workaround this for now at every boot.

Regarding the threat of close this issue again: Our community follows a code of conduct as defined at: https://www.freedesktop.org/wiki/CodeOfConduct/. So please keep the discussion at civilized ways. This is my only request as i915 maintainer. Or I will just close this bug for disrespecting our code of conduct.

But I really appreciate that last comments were civilized. Thanks for that.

Back to the technical aspect of things comment 24 brings an important information. With HDMI monitor on the same machine the automatic range selection that you see is different from the one you see when using DP connector on the very same machine with the very same monitor. Is this correct? Could you please attach logs with both cases booting with drm.debug=0x1e.

If you want a clean start on the issue with bug reported with incorrect range detection for DP when compared with HDMI, please close this one here and start the new one at bugzilla.freedesktop.org providing all logs as requested at https://01.org/linuxgraphics/documentation/how-report-bugs

Thanks,
Rodrigo.

And I just noticed that you already created the clean bug:
https://bugs.freedesktop.org/show_bug.cgi?id=107476

I think we should close this one and move the discussion there.

i915 already provide the connector property that could be set per user need in a atomic way. There is nothing special support for X. Wayland could implement the support to toggle that option. We are not providing a connector property throught sysfs. That is out of question. the broadcast range option is a connector property and will stay as is. Option is Wayland to implement the toggle support.

Even if Wayland could implement the support, why even should it be done this way? What's wrong in doing it with sysfs, which would be available no matter you are you Intel DDX / modesetting DDX (hey) / Wayland?

As if the laziness of Intel hasn't polluted the ecosystem enough.

http://patchwork.freedesktop.org/patch/msgid/20180814060001.18224-1-jani.nikula@intel.com

commit dc5977da99ea28094b8fa4e9bacbd29bedc41de5
Author: Jani Nikula <jani.nikula@intel.com>
Date:   Tue Aug 14 09:00:01 2018 +0300

    drm/i915: set DP Main Stream Attribute for color range on DDI platforms