bug#8402: 24.0.50; Hex colors are not rendered correctly on OS X (Cocoa)

bug#8402: 24.0.50; Hex colors are not rendered correctly on OS X (Cocoa)

[Steve Purcell]

Hi,

If I set face colors using hex values, those colors are not displayed
correctly.

For example, 'M-x set-face-background RET default RET #fdf6e3 RET'
results in a frame background color which, when sampled using Apple's
"Digital Color Meter" utility, has hex value #fff8e8.

And, in fact, all colors are apparently skewed similarly, while other
Cocoa apps (e.g. iterm2) render them correctly.

To give an example from out in the field, here is a screenshot of a
color theme as rendered by Emacs: http://dropup.net/lz64u0ctpcge.png.html

and here is how those same colors would look if they were rendered
correctly:

http://ethanschoonover.com/img/solarized/solarized-screen-pandoc-dark.png

Note the significant difference in the background color, for instance.

Could it be that Emacs needs to hook into system-provided color
management routines on this platform in order for its colors to be
correctly calibrated?

-Steve



In GNU Emacs 24.0.50.1 (x86_64-apple-darwin, NS apple-appkit-1038.35)
of 2011-03-20 on black.porkrind.org
Windowing system distributor `Apple', version 10.3.1038
configured using `configure  '--host=x86_64-apple-darwin' '--build=i686-apple-darwin' '--with-ns' 'build_alias=i686-apple-darwin' 'host_alias=x86_64-apple-darwin' 'CC=gcc -mmacosx-version-min=10.5''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8
  default enable-multibyte-characters: t

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

I should add that I don't see any mention of 'screen-gamma in the frame parameters, and setting different values, e.g. with the following code, has no effect.

 (set-frame-parameter (selected-frame) 'screen-gamma 1.5)

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

The behavior I am seeing (and had taken to be incorrect) is possibly a result of misunderstanding Emacs' color handling, and if that's the case then please feel free to close this bug.

Specifically, it seems like the colors I *wanted* to see (and had applied as hex colors to Emacs' faces) were sRGB colors, but if I convert those sRGB colors to "Generic RGB" values in Apple's "Colorsync Utility" and then plug them into Emacs, I get the desired sRGB screen colors.

So are Emacs' RGB colors explicitly Generic RGB rather than sRGB? I'm pretty sure the answer must be "yes", and that the bug should be closed.


On 1 Apr 2011, at 12:03, Steve Purcell wrote:

> I should add that I don't see any mention of 'screen-gamma in the frame parameters, and setting different values, e.g. with the following code, has no effect.
> 
> (set-frame-parameter (selected-frame) 'screen-gamma 1.5)

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Erik Andrejko]

I was able to fix this problem on my OS X 10.6 machine by modifying nsterm.m:

diff --git a/src/nsterm.m b/src/nsterm.m
index af1f21a..696dbdc 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -1389,7 +1389,7 @@ ns_get_color (const char *name, NSColor **col)
 
   if (r >= 0.0)
     {
-      *col = [NSColor colorWithCalibratedRed: r green: g blue: b alpha: 1.0];
+      *col = [NSColor colorWithDeviceRed: r green: g blue: b alpha: 1.0];
       UNBLOCK_INPUT;
       return 0;
     }


After the change, the RGB values of the colors displayed matched the expected RGB values according to the DigitalColor Meter utility.

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

Erik - what's the color behavior on other window-systems? I haven't had the opportunity to compare. It does seem like your fix would be more likely to give the results a user would want.

-Steve

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Chong Yidong]

Steve Purcell <steve@sanityinc.com> writes:

> what's the color behavior on other window-systems? I haven't had the
> opportunity to compare. It does seem like your fix would be more
> likely to give the results a user would want.

On X, we just pass color names to Xlib functions like XParseColor as
simple RGB, i.e. not specifying any particular color space, letting that
be chosen by X.  I don't know how this choice is made; the Xlib docs
simply say that the color space is device-dependent.  In general, I
think we use whatever color space happens to be the system default.

I am not familiar with the code in question, but from what I can tell
the NS code always uses NSCalibratedRGBColorSpace, which indicates that
using colorWithCalibratedRed to specify RGB components ought to do the
right thing.  I don't know why it doesn't.

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

On 10 Apr 2011, at 23:38, Chong Yidong wrote:

> Steve Purcell <steve@sanityinc.com> writes:
> 
>> what's the color behavior on other window-systems? I haven't had the
>> opportunity to compare. It does seem like your fix would be more
>> likely to give the results a user would want.
> 
> On X, we just pass color names to Xlib functions like XParseColor as
> simple RGB, i.e. not specifying any particular color space, letting that
> be chosen by X.  I don't know how this choice is made; the Xlib docs
> simply say that the color space is device-dependent.  In general, I
> think we use whatever color space happens to be the system default.
> 
> I am not familiar with the code in question, but from what I can tell
> the NS code always uses NSCalibratedRGBColorSpace, which indicates that
> using colorWithCalibratedRed to specify RGB components ought to do the
> right thing.  I don't know why it doesn't.


I'm told by a user with access to an Emacs on X11 that hex colors on that window system display in what appears to be the same way; that is, the hex colors applied by Emacs do not correspond to the resultant sRGB display colors, but rather to a generic RGB color space which is translated by the host system into that sRGB color space.

I guess it'd be ideal if the hex colors entered in Emacs corresponded with the host window system's notion of hex colors, but it could also be argued that the existing behavior is at least consistent across platforms.

The downside of the current system is that it's not possible to directly apply to Emacs colors that have been found on the web or via a color-picker tool on the host machine.

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[David De La Harpe Golden]

On 10/04/11 23:38, Chong Yidong wrote:
> Steve Purcell<steve@sanityinc.com>  writes:
>
>> what's the color behavior on other window-systems? I haven't had the
>> opportunity to compare. It does seem like your fix would be more
>> likely to give the results a user would want.
>
> On X, we just pass color names to Xlib functions like XParseColor as
> simple RGB,

To nitpick:  AFAIK we pass the /color string names themselves/ to 
XParseColor() to resolve them, don't we? -  color name -> color mapping 
is not internally handled by emacs on X11.

You can even define your own named colors on x11 over and above the 
familiar historic rgb.txt set if you fiddle about e.g.

$ cat ~/.xcms.txt
XCMS_COLORDB_START 0.1
Blobby	rgb:7fe2/ffe1/331a
XCMS_COLORDB_END
$ export XCMSDB=/home/david/.xcms.txt
$ emacs

- Now "Blobby" is an additional named color like "green".

> i.e. not specifying any particular color space, letting that
> be chosen by X.

We-ell. See the XParseColor() manpage, section COLOR NAMES [1] - you
can explicitly specify various device-independent colorspaces, and they 
do in principle work if you put them into a color field in emacs on x11, 
because emacs just passes the string straight on through as mentioned 
above. You just need to use the right syntax as per the manpage (or 
define them as named colors similar to above).  But the 
"rgb:<red>/<green>/<blue>" (or "#RRGGBB") syntax in particular is 
defined to use the device space.

None of the device-independent colorspaces documented as standard by 
XParseColor() are sRGB though, that came later, which is a tad 
inconvenient in the modern era of wide use of sRGB. OTOH, CIEXYZ is 
available and there are forward and reverse transforms that can be 
applied. [2]

HOWEVER... the Xcms [3] scheme predates ICC and isn't really the color 
calibration system people tend to use nowadays AFAIK, people use xicc 
[4] which uses the same ICC profiles as found on windows etc., so the 
chances of a "CIEXYZ:X/Y/Z" color passed to XParseColor() actually 
winding up mapped to the right CIE XYZ point, well, AFAIUI, that would 
depend heavily on a user having actually bothered to set up a reasonable 
Xcms calibration via xcmsdb [5], not just having setup xicc calibration. 
  I suppose some sufficiently advanced tool might exist to extrapolate 
the Xcms calibration from a now-standard ICC calibration, but that 
sounds tricky to me (then again, I'm not an expert).

**  So if you want NS emacs to treat "#RRGGBB" values like x11 emacs 
currently does, it does sound to me like the thing to do is 
colorWithDeviceRed. colorWithCalibratedRed uses the 
NSCalibratedRGBColorSpace, which AFAICS isn't actually sRGB but isn't 
the device space either [6][7].

** It would be also presumably be possible to extend ns emacs to 
internally support some explicitly spelled out device-independent 
colorspace specification in the color spring like XParseColor() and 
therefore x11 emacs supports. If doing that, I do suggest using the same 
syntaxes as x11 for them...

** OTOH it's possible to imagine the emacs project deciding - rather 
backward incompatibly for x11 users mind you - that, say, "#RRGGBB" 
syntax color strings in emacs on all platforms are actually to be 
considered to be in the sRGB colorspace henceforth, like they are 
officially in html/css.

But #RRGGBB couldn't then be passed straight through to XParseColor() 
anymore on x11, because #RRGGBB there is defined to be a backward 
compatibility way of writing colors in the device space also available 
via "rgb:<red>/<green>/<blue>". They would need to be processed into 
CIEXYZ and passed to XParseColor() (in the probably vain hope the user 
has set up Xcms calibration), or (likely yielding better results in the 
modern era in practice AFAIUI) converted via application of an ICC 
profile found via xicc into the device space with a library like 
littlecms [8], which is what GIMP and the like now do.

So a more backward-compat friendly thing to do than using #RRGGBB for 
srgb, might be to add support within emacs for an 
"sRGB:<red>/<green>/<blue>"  syntax by analogy with the standard 
XParseColor() ones, and handle it internally (I guess it's unlikely at 
this stage the xlib maintainers will add such a thing as standard to 
XParseColor(), but I suppose they might...)

Yeesh,that got long. Hope I wasn't talking nonsense.


[1] http://linux.die.net/man/3/xparsecolor

[2] http://en.wikipedia.org/wiki/SRGB#Specification_of_the_transformation

[3] http://insar.stanford.edu/~lharcke/programming/Xcms/

[4] http://burtonini.com/blog/computers/xicc

[5] http://linux.die.net/man/1/xcmsdb

[6] 
http://developer.apple.com/library/mac/#documentation/cocoa/Conceptual/DrawColor/Concepts/AboutColorSpaces.html#//apple_ref/doc/uid/20000758-BBCHACHA

[7] 
http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/ApplicationKit/Classes/NSColorSpace_Class/Reference/Reference.html#//apple_ref/occ/cl/NSColorSpace


[8] http://www.littlecms.com/

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

On 4 May 2011, at 17:58, David De La Harpe Golden wrote:

> ** OTOH it's possible to imagine the emacs project deciding - rather backward incompatibly for x11 users mind you - that, say, "#RRGGBB" syntax color strings in emacs on all platforms are actually to be considered to be in the sRGB colorspace henceforth, like they are officially in html/css.


This would be a great step forward, IMO: sRGB is the standard on the web and on most displays, and a clear definition of Emacs' colorspace would be a Good Thing.


> So a more backward-compat friendly thing to do than using #RRGGBB for srgb, might be to add support within emacs for an "sRGB:<red>/<green>/<blue>"  syntax by analogy with the standard XParseColor() ones, and handle it internally


Adding a new "sRGB:r/g/b" syntax seems less clean -- if you'd have to handle it explicitly before passing it to XParseColor, why not just make it the default?

-Steve

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[David De La Harpe Golden]

On 04/05/11 20:31, Steve Purcell wrote:

> Adding a new "sRGB:r/g/b" syntax seems less clean
> if you'd have to handle it explicitly before passing it
 > to XParseColor, why not just make it the default?
>

Remember, the XParseColor() system remains in wide use by other
x11 apps.  If emacs used "#RRGGBB" in an incompatible manner, we'd then
be getting bugs about the colors not matching anymore between emacs
and  other apps on x11 when people use the "same" values, e.g.
launching both
xterm -fg '#fedcba' -bg '#123456'
emacs -fg '#fedcba' -bg '#123456'
one would expect them to look the same.

I think keeping emacs unsurprising to x11 folk is good, given x11's use
as the main free gui...

Another option in the design space could be to have an "emacs:" super-
prefix that does colors in a standard emacs space* e.g.
"emacs:#334455". Emacs could also bundle its named-colors table (it
already does for non-x11 use) declared in the space, so you could do
"emacs:Tomato", while anything not prefixed that way is passed through
to XParseColor().  Then e.g. "emacs:#RRGGBB" would be potentially
different to an "#RRGGBB", "emacs:Tomato" potentially different to
"Tomato".

Or you could flip that, and have  an "x11:" super-prefix that means
"pass the remainder through to XParseColor()", while unprefixed is
handled in the emacs space, but people will, as above, expect command
line args and X resources on x11 to be interpreted in the x11 manner,
so such a prefix might then have to be implicitly added for those cases.

Either way, that's a bit different to just having srgb: - it handles
named colors in a cross-platform manner, and leaves it possible for
emacs to support more syntaxes internally - Hey, I suppose supporting
the CSS3-color syntax [1] in general would be neat, either prefixed or
unprefixed.  Have to remember that the x11 (and therefore historic 
emacs) color names and html/css color names clash somewhat, sigh [2]

...Or we could stop using string specs and have an actual color type,
maybe with strings in color slots interpreted backward compatibly...

* presumably srgb in practice, while it's sometimes criticised, it's a
reasonable choice, it is a text editor we're talking about.

[1] http://www.w3.org/TR/css3-color/

[2] 
http://en.wikipedia.org/wiki/X11_color_names#Color_names_that_clash_between_X11_and_HTML.2FCSS

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

On 5 May 2011, at 09:14, David De La Harpe Golden wrote:

> On 04/05/11 20:31, Steve Purcell wrote:
> 
>> Adding a new "sRGB:r/g/b" syntax seems less clean
>> if you'd have to handle it explicitly before passing it
> > to XParseColor, why not just make it the default?
>> 
> 
> Remember, the XParseColor() system remains in wide use by other
> x11 apps.  If emacs used "#RRGGBB" in an incompatible manner, we'd then
> be getting bugs about the colors not matching anymore between emacs
> and  other apps on x11 when people use the "same" values, e.g.
> launching both
> xterm -fg '#fedcba' -bg '#123456'
> emacs -fg '#fedcba' -bg '#123456'
> one would expect them to look the same.
> 
> I think keeping emacs unsurprising to x11 folk is good, given x11's use
> as the main free gui...


Makes sense.

Would there be a problem if X11 was left as-is, and NS started to interpret #rrggbb as sRGB?

Seems like the current situation on X11 is dependent on system configuration anyway, and using sRGB on NS would keep emacs unsurprising to OS X folk (to paraphrase you).

-Steve

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[David De La Harpe Golden]

On 05/05/11 10:17, Steve Purcell wrote:
>
> Would there be a problem if X11 was left as-is, and NS started to interpret #rrggbb as sRGB?
>

Shrug. Insofar as with the current situation one can't expect the
colors to match across the platforms anyway, I guess it wouldn't matter
much. And I do suspect (but n.b. haven't yet checked) that on w32, they
already end up sRGB rather than device-specific, just as a result of
most w32 apis being defined to be sRGB IIRC.

** However, another option, likely quite easily implemented, though,
could  be to just offer a customization as to which of the suitable
predefined spaces to interpret values in on ns, e.g.

(defcustom ns-color-space 'srgb
    "Interpret color specifications in the given color space."
    :type '(choice (const :tag "sRGB" srgb)
                   (const :tag "Compatible with Adobe RGB(1998)" adobe)
                   (const :tag "Calibrated/Generic RGB" generic)
                   (const :tag "Device RGB" device)))

and then use the relevant NSColorSpace classmethods [1] and NSColor
colorWithColorSpace. [2] in the relevant spots, instead of just
colorWithCalibratedRed or colorWithDeviceRed.

That seems to me to be a fairly good option for now?

(A decision would also need to be made as to whether the named color
values bundled with non-x11 emacs are interpreted relative to that
configured space or considered to be sRGB (or one of the others))

 > using sRGB on NS would keep emacs unsurprising to OS X folk (to
 > paraphrase you).

Thing is, the Apple docs do seem to be encouraging use of their
NSCalibratedRGBColorSpace, and not sRGB, so I'm not actually convinced
sRGB would be unsurprising to OS X folk, at least if other OS X apps
are following those guidelines. The people it primarily wouldn't
surprise might be web and windows folk...

Another little issue is that there _is no_ predefined sRGBColorSpace
available until macosx 10.5+ ! [1]. Do we still support < 10.5 ?


[1] 
http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/ApplicationKit/Classes/NSColorSpace_Class/Reference/Reference.html#//apple_ref/occ/cl/NSColorSpace

[2] 
http://developer.apple.com/library/mac/documentation/Cocoa/Reference/ApplicationKit/Classes/NSColor_Class/Reference/Reference.html#//apple_ref/doc/uid/20000353-BBCJCBJA

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[David De La Harpe Golden]

On 05/05/11 12:13, David De La Harpe Golden wrote:
> On 05/05/11 10:17, Steve Purcell wrote:
>>
>> Would there be a problem if X11 was left as-is, and NS started to
>> interpret #rrggbb as sRGB?
>>
>
> Shrug. Insofar as with the current situation one can't expect the
> colors to match across the platforms anyway, I guess it wouldn't matter
> much.

though (just to spell out more for the archives than respondents who 
likely understood this) - ns emacs is currently using 
NSCalibratedRGBColorSpace as Chong Yidong already mentioned and afaics, 
so it would be a change on ns itself for ns switch to sRGB, I just meant 
it wouldn't be making interplatform compat worse particularly.

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

On 5 May 2011, at 14:07, David De La Harpe Golden wrote:

> On 05/05/11 12:13, David De La Harpe Golden wrote:
>> On 05/05/11 10:17, Steve Purcell wrote:
>>> 
>>> Would there be a problem if X11 was left as-is, and NS started to
>>> interpret #rrggbb as sRGB?
>>> 
>> 
>> Shrug. Insofar as with the current situation one can't expect the
>> colors to match across the platforms anyway, I guess it wouldn't matter
>> much.
> 
> though (just to spell out more for the archives than respondents who likely understood this) - ns emacs is currently using NSCalibratedRGBColorSpace as Chong Yidong already mentioned and afaics, so it would be a change on ns itself for ns switch to sRGB, I just meant it wouldn't be making interplatform compat worse particularly.


I think I've been conflating sRGB and device colors. What I'd be in favor of right now would be somebody changing NS to use device RGB colors by default, and seeing if the world falls apart. The indications are that it would improve matters for NS emacs users without upsetting anyone else.

Having a customization for the NS colorspace would be a nice-to-have, and the possibility of having colors displayed identically across all window-systems would be a laudable longer-term goal.

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[David De La Harpe Golden]

[-- Attachment #1: Type: text/plain, Size: 2672 bytes --]

On 05/05/11 19:43, Steve Purcell wrote:

> I think I've been conflating sRGB and device colors.
> What I'd be in favor of right now would be somebody changing NS
 > to use device RGB colors by default, and seeing if the world
 > falls apart
 >
 > The indications are that it would improve matters for NS emacs
 > users without upsetting anyone else.

Apple docs do seem quite insistent that the right thing to do is to
stick to NSCalibratedRGBColorSpace.

Uh. Then I saw that iterm2 that you mentioned is GPL licensed, so, hey,
I can see what it's doing. And, it, like emacs, seems to be using the
NSCalibratedRGBColorSpace throughout, at least at first glance. [1]

So why then would the colors be different? Well, one possibility would 
be emaccs somehow not using the api right, but that reminded me of
something quite important:

If iterm2 acts as a 256 color terminal emulator, it likely has a fixed
256 color palette, just like an x11 256 color terminal. [2]. Some
24-bit colors can be displayed perfectly with such a  palette, but the
bulk get approximated to the nearest available from the palette.

To show you the quantization effect I mean, see the attached
emacs_tt_vs_gui_cols.png, it's a screenshot from my x11 system of emacs
using the same face color settings in x11 emacs and a 256-color text
terminal. On a text terminal, emacs can only pick similar but not quite
identical colors.

So, uh, what other apps did you compare emacs to besides iterm2?
Colors shown via iterm2 are suspect!  (I realise you  mention applying
an srgb to apply-generic-rgb transform "manually" and  it giving
acceptable results, but I'm now wondering was that all some sort of
unlikely coincidence)

I'd also be worried that "DigitalColor Meter" utility might lead astray
a bit. Say it shows the about-to-go-to-device values - that would
meaning that sure, as Erik reported with his patch, when emacs uses the
device space, effectively a passthru, values you put into emacs exactly
match the utility's feedback... but... does that patched emacs then
match other common apps, and do other common apps which allow rgb entry
match the utility's feedback? (not clear to me from the thread)

Unfortunately I don't have a mac, so right now I can't personally
compare to other apps or test anything...

> Having a customization for the NS colorspace would be a nice-to-have,

Yup.

> and the possibility of having colors displayed identically across
 > all window-systems would be a laudable longer-term goal.

Indeed.


[1] 
http://www.google.com/codesearch?hl=en&lr=&q=Calibrated+package%3Ahttp%3A%2F%2Fiterm2\.googlecode\.com

[2] http://www.pixelbeat.org/docs/terminal_colours/#256

[-- Attachment #2: emacs_tt_vs_gui_cols.png --]
[-- Type: image/png, Size: 119134 bytes --]

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

On 5 May 2011, at 23:53, David De La Harpe Golden wrote:

> On 05/05/11 19:43, Steve Purcell wrote:
> 
>> I think I've been conflating sRGB and device colors.
>> What I'd be in favor of right now would be somebody changing NS
> > to use device RGB colors by default, and seeing if the world
> > falls apart
> >
> > The indications are that it would improve matters for NS emacs
> > users without upsetting anyone else.
> 
> Apple docs do seem quite insistent that the right thing to do is to
> stick to NSCalibratedRGBColorSpace.
> 
> Uh. Then I saw that iterm2 that you mentioned is GPL licensed, so, hey,
> I can see what it's doing. And, it, like emacs, seems to be using the
> NSCalibratedRGBColorSpace throughout, at least at first glance. [1]


Yes, you're right -- I double-checked, and it looks like the authors of precision color themes for iTerm2 have had to back-calculate color values that would look right at display time:

https://github.com/altercation/solarized/tree/master/iterm2-colors-solarized



> So why then would the colors be different? Well, one possibility would be emaccs somehow not using the api right, but that reminded me of
> something quite important:
> 
> If iterm2 acts as a 256 color terminal emulator, it likely has a fixed
> 256 color palette, just like an x11 256 color terminal. [2]. Some
> 24-bit colors can be displayed perfectly with such a  palette, but the
> bulk get approximated to the nearest available from the palette.


iTerm2 does indeed act like that, but don't worry - I'm not running emacs inside iTerm.


> So, uh, what other apps did you compare emacs to besides iterm2?
> Colors shown via iterm2 are suspect!  (I realise you  mention applying
> an srgb to apply-generic-rgb transform "manually" and  it giving
> acceptable results, but I'm now wondering was that all some sort of
> unlikely coincidence)


The poster child app for comparisons is any web browser, in which a hex value of #nnnnnn gives the exact same color, as perceived visually and by checking with Apple's Digital Color Meter tool. In other words, the system's color picker yields hex colors which, when put in a web page, show up as the desired color. I'd naïvely expect every app to work like this.

In my case, to get the right colors for Emacs I had to fire up the arcane color calculator inside Apple's ColorSync Utility and then laboriously pick desired colors from a web page, and translate them individually to Generic RGB values which I could then plug into Emacs. And the results in Emacs still don't exactly match the original colors.


> I'd also be worried that "DigitalColor Meter" utility might lead astray
> a bit. Say it shows the about-to-go-to-device values - that would
> meaning that sure, as Erik reported with his patch, when emacs uses the
> device space, effectively a passthru, values you put into emacs exactly
> match the utility's feedback... but... does that patched emacs then
> match other common apps, and do other common apps which allow rgb entry
> match the utility's feedback? (not clear to me from the thread)


Perhaps Erik can comment on whether his device-color-patched Emacs produces colors that are perceptually identical?

I'd hope that Digital Color Meter displays about-to-go-to-device values; I presume that those color values are the important ones, since all correctly calibrated devices will display those colors identically.

In the interests of fairness, I'll also note that TextMate (an extremely popular non-free editor on OS X) interprets color values in its themes as being in the Apple Generic RGB color space, so I guess internally it's using colorFromCalibratedRed, just like Emacs.

Nonetheless, I think the browsers are getting it right by doing what users expect, and the other apps are confusing.

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[David De La Harpe Golden]

On 06/05/11 08:00, Steve Purcell wrote:
 > The poster child app for comparisons is any web browser,

No, that's the one kind of app that I consider invalid for fair
comparison, as standards-compliant browsers are simply required to use
sRGB by default, regardless of any platform-specific conventions.

 > I'd hope that Digital Color Meter displays about-to-go-to-device
 > values; I presume that those color values are the important ones,
 > since all correctly calibrated devices will display those colors
 > identically.

Uh, no, devices can have all sorts of gamut, color values about to go
to the device would be in a post-any-software-transform
pre-any-hardware-LUT-transform form, that may or may not coincide with
sRGB.  Think of how much extra you can pay for a "wide gamut" display
capable of colors well outside sRGB [1] - and how annoyed you'd be if
your OS wasn't sending them.

 > In other words, the system's color picker yields hex colors which,
 > when put in a web page, show up as the desired color. I'd naïvely
 > expect every app to work like this.

I still don't know exactly what DigitalColor Meter is doing.  But maybe
it actually takes steps to always report in sRGB so that it reliably
gives the same value [s]RGB value for the same color shown via
different devices. That  would kinda  make sense, though it would mean
you'd presumably get an  error or clipped answers (or maybe it would
switch to scRGB [2] or something) on wide-gamut displays.

Though it is does seem strange they'd make the system color tool report
in sRGB if they're also telling apps to use Apple Generic RGB, or at
least not offer values in both in it (AFAICS, having seen a screnshot,
it's got a dropdown list of a few spaces to report in) and make it
clear what space its "RGB" really is (by the sounds of it, sRGB...),
but OTOH, maybe it's geared mostly to web dev needs in the first place
- I expect graphics pros are more likely to use photoshop [3].

i.e. the DigitalColor Meter utility might just be doing what browser
users, particularly the web developer types likely to use it, naively
expect.

 > In the interests of fairness,
 > I'll also note that TextMate (an extremely popular non-free editor on
 > OS X) interprets color values in its themes as being in the Apple
 > Generic RGB color space, so I guess internally it's using
 > colorFromCalibratedRed, just like Emacs.

It does sound to me like TextMate is just following the Word Of Apple
like iterm2, then. And emacs is after all an app in the exact same
class as TextMate [3]

 > Nonetheless, I think the browsers are getting it right by doing what
 > users expect,

Er. I think that's sorta the wrong way round, what's happening is that
browsers are starting to generate user expectations for other apps -
like yours. Browsers, again, are flat-out required to use sRGB [5]. Web
folk of course are a major group of emacs users, so it's IMO worth
catering to their needs.  But I'm not sure it's sufficient reason to
switch the default on its own, more of an argument for adding the
switching capability previously discussed.

Though if emacs were to wholeheartedly adopt a standard interpretation
of intraemacs color strings on all color-management-capable platforms,
then Apple Generic RGB would obviously be a hopeless basis (though could
still be provided for with a prefix, say, "AppleGenericRGB:r/g/b").

Perhaps it could be semiseriously argued that emacs, as it is capable
of acting as a quite usable web browser via w3 or emacs-w3m, should use
sRGB and the web named-colors values.

[1] http://www.eizo.com/global/products/coloredge/cg221/index.html

[2] http://en.wikipedia.org/wiki/ScRGB

[3] 
http://blogs.adobe.com/jnack/2008/02/make_photoshop_sample_colors_outside_the_a.html

[4] TextMate is even the first text editor in quite a while to do 
anything particularly interesting from the emacs community perspective - 
right now, you have to add the sorta-kludgey "mumamo" found in the 
"nxhtml" suite for emacs to catch up with it in some areas.

[5] http://www.w3.org/TR/css3-color/#rgb-color

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

On 6 May 2011, at 20:59, David De La Harpe Golden wrote:

> I still don't know exactly what DigitalColor Meter is doing.


Same here.


> > Nonetheless, I think the browsers are getting it right by doing what
> > users expect,
> 
> Er. I think that's sorta the wrong way round, what's happening is that
> browsers are starting to generate user expectations for other apps -
> like yours. Browsers, again, are flat-out required to use sRGB [5]. Web
> folk of course are a major group of emacs users, so it's IMO worth
> catering to their needs.  But I'm not sure it's sufficient reason to
> switch the default on its own, more of an argument for adding the
> switching capability previously discussed.


+1.


> Though if emacs were to wholeheartedly adopt a standard interpretation
> of intraemacs color strings on all color-management-capable platforms,
> then Apple Generic RGB would obviously be a hopeless basis (though could
> still be provided for with a prefix, say, "AppleGenericRGB:r/g/b").
> 
> Perhaps it could be semiseriously argued that emacs, as it is capable
> of acting as a quite usable web browser via w3 or emacs-w3m, should use
> sRGB and the web named-colors values.


<chuckle>

I certainly think there's an opportunity to review the status quo on all supported window systems, and to figure out if there's a desire/basis for standardization on something like sRGB.

I'm going to bow out, because I don't think I have any further insights to offer.



> [4] TextMate is even the first text editor in quite a while to do anything particularly interesting from the emacs community perspective - right now, you have to add the sorta-kludgey "mumamo" found in the "nxhtml" suite for emacs to catch up with it in some areas.


Agreed - I use the somewhat-antiquated MMM mode myself, which has less baggage than mumamo, but is decidedly creaky.

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[David De La Harpe Golden]

On 04/05/11 17:58, David De La Harpe Golden wrote:

> HOWEVER... the Xcms [3] scheme predates ICC and isn't really the color
> calibration system people tend to use nowadays AFAIK, people use xicc
> [4]

Aside, I'm out-of-date here: xicc (make ICC profile available) in 
conjuction with xcalib (LUT load), but apparently more-nowadays, people 
favour dispwin from argyllcms (debian package argyll) that manages both, 
and also makes it easily autoapplied from a known location [1][2]

[1] http://www.argyllcms.com/doc/dispprofloc.html#Linux
[2] http://en.wikipedia.org/wiki/Linux_color_management

(yes, [2] is misnamed, it's mentioned on the talk page as an issue with 
the article)

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[David De La Harpe Golden]

On 06/05/11 00:56, David De La Harpe Golden wrote:
> On 04/05/11 17:58, David De La Harpe Golden wrote:
>
> Aside, I'm out-of-date here:

Uh. And just to come screaming further toward the present day, it seems
in 2010/2011 work was/is/will-be ongoing to allow x11 clients to opt to
offload color correction via a "net-color" fd.o spec to a "color
server" (part of the compositing manager), for hardware-accelerated
corrections on a per-subregion-of-window basis, with easy libs for
clients to use. [1] Seems to be in an early/unstable but usable state
right now, so by the  time anyone gets to this on the emacs side...

So, while implementing for x11 emacs the xicc all-client-side way would
presumably work right now, it may get easier and faster shortly.  They
seem to be maintaining backward compat so the
(presently-most-widely-used) xicc client-side way keeps on working (but
won't benefit from acceleration).

[1] http://www.freedesktop.org/wiki/OpenIcc

bug#8402: bug 8402

[Travis Vachon]

fwiw, Erik's patch worked like a charm for me too: digital color meter
matches the solarized palette perfectly post-patch.

I've pushed a homebrew formula for installing emacs with this patch:

https://github.com/travis/homebrew

bug#8402: Acknowledgement (24.0.50; Hex colors are not rendered correctly on OS X (Cocoa))

[Steve Purcell]

Chong Yidong wrote:
> On X, we just pass color names to Xlib functions like XParseColor as
> simple RGB, i.e. not specifying any particular color space, letting that
> be chosen by X.  I don't know how this choice is made; the Xlib docs
> simply say that the color space is device-dependent.  In general, I
> think we use whatever color space happens to be the system default.
> 
> I am not familiar with the code in question, but from what I can tell
> the NS code always uses NSCalibratedRGBColorSpace, which indicates that
> using colorWithCalibratedRed to specify RGB components ought to do the
> right thing.  I don't know why it doesn't.



Thinking about this, if XParseColor interprets the RGB value as a device-dependent color, then it seems Erik is right that the correct NS function to use would be colorWithDeviceRed, not colorWithCalibratedRed.

It feels right to me that when the user sees a color on his display and enters its hex code in his Emacs config, the same color should get displayed. Switching to colorWithDeviceRed would seem to be the way to achieve this on NS, and under X11 the current behavior appears undefined.

-Steve

bug#8402: 24.0.50; Hex colors are not rendered correctly on OS X (Cocoa)

[Lars Ingebrigtsen]

Steve Purcell <steve@sanityinc.com> writes:

> If I set face colors using hex values, those colors are not displayed
> correctly.
>
> For example, 'M-x set-face-background RET default RET #fdf6e3 RET'
> results in a frame background color which, when sampled using Apple's
> "Digital Color Meter" utility, has hex value #fff8e8.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

There was a patch suggested to alter ns_get_color, but I see that the
function has changed greatly since this was reported ten years ago.

Is this still an issue with recent Emacs versions?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#8402: 24.0.50; Hex colors are not rendered correctly on OS X (Cocoa)

[Howard Melman]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Steve Purcell <steve@sanityinc.com> writes:
>
>> If I set face colors using hex values, those colors are not displayed
>> correctly.
>>
>> For example, 'M-x set-face-background RET default RET #fdf6e3 RET'
>> results in a frame background color which, when sampled using Apple's
>> "Digital Color Meter" utility, has hex value #fff8e8.
>
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
>
> There was a patch suggested to alter ns_get_color, but I see that the
> function has changed greatly since this was reported ten years ago.
>
> Is this still an issue with recent Emacs versions?

When I try it with emacs 28 on macOS 11.6.5 I get the
background color 0xFCF6E5.  So it seems quite close.

-- 

Howard

bug#8402: 24.0.50; Hex colors are not rendered correctly on OS X (Cocoa)

[Alan Third]

On Sat, Apr 23, 2022 at 05:33:35PM -0400, Howard Melman wrote:
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
> > Steve Purcell <steve@sanityinc.com> writes:
> >
> >> If I set face colors using hex values, those colors are not displayed
> >> correctly.
> >>
> >> For example, 'M-x set-face-background RET default RET #fdf6e3 RET'
> >> results in a frame background color which, when sampled using Apple's
> >> "Digital Color Meter" utility, has hex value #fff8e8.
> >
> > (I'm going through old bug reports that unfortunately weren't resolved
> > at the time.)
> >
> > There was a patch suggested to alter ns_get_color, but I see that the
> > function has changed greatly since this was reported ten years ago.
> >
> > Is this still an issue with recent Emacs versions?
> 
> When I try it with emacs 28 on macOS 11.6.5 I get the
> background color 0xFCF6E5.  So it seems quite close.

I think all this is a result of macOS's heavy use of colourspaces,
specifically conversion between "generic" and "SRGB" colourspaces.

But to be perfectly frank, colour on macOS confuses me so much so I
could be talking nonsense. If anyone does want to mess with this bear
in mind that if you mess it up it results in HUGE performance
penalties as every drawing action then has to run through colour
conversion routines.
-- 
Alan Third

bug#8402: 24.0.50; Hex colors are not rendered correctly on OS X (Cocoa)

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

>> >> For example, 'M-x set-face-background RET default RET #fdf6e3 RET'
>> >> results in a frame background color which, when sampled using Apple's
>> >> "Digital Color Meter" utility, has hex value #fff8e8.

[...]

>> When I try it with emacs 28 on macOS 11.6.5 I get the
>> background color 0xFCF6E5.  So it seems quite close.

Thanks for checking.

> I think all this is a result of macOS's heavy use of colourspaces,
> specifically conversion between "generic" and "SRGB" colourspaces.
>
> But to be perfectly frank, colour on macOS confuses me so much so I
> could be talking nonsense. If anyone does want to mess with this bear
> in mind that if you mess it up it results in HUGE performance
> penalties as every drawing action then has to run through colour
> conversion routines.

As Howard says, the colours are a lot closer now than when the bug
report was filed, so perhaps it's good enough now and we don't have to
change anything?  (For other systems, the colours we end up to are
pretty much defined by the system.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#8402: 24.0.50; Hex colors are not rendered correctly on OS X (Cocoa)

[steve]

[-- Attachment #1: Type: text/html, Size: 1297 bytes --]

bug#8402: 24.0.50; Hex colors are not rendered correctly on OS X (Cocoa)

[Lars Ingebrigtsen]

steve@sanityinc.com writes:

> No, I believe this issue is long since fixed, thanks!

OK; closing this bug report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no