bug#4033: 23.1; list-colors-display is misleading

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

emacs -Q
M-x list-colors-display
 
The RGB values listed at the right side are misleading.
 
I have had at least one user state that he thought that, since the
form shown is #RRGGBB, his colors had only that granularity.  IOW,
even if one's system allows colors of the form #RRRRGGGGBBBB (more
colors), the color names are translated to hex strings of the form
#RRGGBB (fewer colors).
 
The displayed RGB hex string ideally should reflect the user's actual
color possibilities.  If there is no way for Emacs to know that, then
it's better to err on the side of providing more information:
#RRRRGGGGBBBB, rather than less: #RRGGBB.  E.g., it's better to
translate LightBlue as #befded5effff than as #beedff.
 
And if the user's real color space cannot be known, then a note/legend
should be added (at least in the `C-h m' help for the
`list-colors-display' buffer, but preferably at the top or bottom of
the buffer itself), saying that the RGB code shown is not necessarily
representative of the color granularity for your system.
 

In GNU Emacs 23.1.1 (i386-mingw-nt5.1.2600)
 of 2009-07-29 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (4.4)'
 

bug#4033: 23.1; list-colors-display is misleading

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Tue, 4 Aug 2009 09:01:21 -0700
> Cc: 
> 
> emacs -Q
> M-x list-colors-display
>  
> The RGB values listed at the right side are misleading.

Only if you interpret them to mean not what they were supposed to
mean.

> The displayed RGB hex string ideally should reflect the user's actual
> color possibilities.  If there is no way for Emacs to know that, then
> it's better to err on the side of providing more information:
> #RRRRGGGGBBBB, rather than less: #RRGGBB.  E.g., it's better to
> translate LightBlue as #befded5effff than as #beedff.

16-bit RGB components is what Emacs uses internally, IIRC.  That is
the reason we show each one as two letters.

I think this bug should be closed.

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> > The RGB values listed at the right side are misleading.
> 
> Only if you interpret them to mean not what they were supposed to
> mean.
>
> > The displayed RGB hex string ideally should reflect the 
> > user's actual color possibilities.  If there is no way for
> > Emacs to know that, then it's better to err on the side of
> > providing more information: #RRRRGGGGBBBB, rather than less:
> > #RRGGBB.  E.g., it's better to translate LightBlue as
> > #befded5effff than as #beedff.

(I meant #ADADD8D8E6E6 and #ADD8E6 for LightBlue - got my numbers wrong; sorry.)

> 16-bit RGB components is what Emacs uses internally, IIRC.  That is
> the reason we show each one as two letters.

Is that right? Could you please check about this?

I was under the (perhaps mistaken) impression that Emacs treats colors
differently, depending on your display's color support. In particular, I thought
that `display-color-cells' would show how many colors are supported, and
therefore how many RGB hex digits would be appropriate (available) for a given
display. IOW, I thought that on some displays you might be able to use only
#RRGGBB, while on others you could use #RRRGGGBBB (where there would be a
perceived difference when using fewer or more digits).

> I think this bug should be closed.

If you're sure about what you say, then yes. But please check to be sure. Thx.

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> > > The RGB values listed at the right side are misleading.
> > 
> > Only if you interpret them to mean not what they were supposed to
> > mean.
> >
> > > The displayed RGB hex string ideally should reflect the 
> > > user's actual color possibilities.  If there is no way for
> > > Emacs to know that, then it's better to err on the side of
> > > providing more information: #RRRRGGGGBBBB, rather than less:
> > > #RRGGBB.  E.g., it's better to translate LightBlue as
> > > #befded5effff than as #beedff.
> 
> (I meant #ADADD8D8E6E6 and #ADD8E6 for LightBlue - got my 
> numbers wrong; sorry.)
> 
> > 16-bit RGB components is what Emacs uses internally, IIRC.  That is
> > the reason we show each one as two letters.
> 
> Is that right? Could you please check about this?
> 
> I was under the (perhaps mistaken) impression that Emacs treats colors
> differently, depending on your display's color support. In 
> particular, I thought
> that `display-color-cells' would show how many colors are 
> supported, and
> therefore how many RGB hex digits would be appropriate 
> (available) for a given
> display. IOW, I thought that on some displays you might be 
> able to use only
> #RRGGBB, while on others you could use #RRRGGGBBB (where 
> there would be a
> perceived difference when using fewer or more digits).
> 
> > I think this bug should be closed.
> 
> If you're sure about what you say, then yes. But please check 
> to be sure. Thx.

BTW, if what you say is the case, then it is all the more unfortunate, since
`color-values' returns values up to 65535 (or 65280, for some platforms). That's
16 ** 4, which means that each color component can be represented by up to 4 hex
digits: #RRRRGGGGBBBB. That's one reason I've always assumed that up to 4 hex
digits were handled by Emacs.

If what you say is true, then what is the reason for such a limitation? Is there
some inherent limitation, or is this just a design or implementation bug, which
could be fixed?

bug#4033: 23.1; list-colors-display is misleading

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Tue, 4 Aug 2009 12:40:32 -0700
> 
> BTW, if what you say is the case, then it is all the more unfortunate, since
> `color-values' returns values up to 65535 (or 65280, for some platforms). That's
> 16 ** 4, which means that each color component can be represented by up to 4 hex
> digits: #RRRRGGGGBBBB. That's one reason I've always assumed that up to 4 hex
> digits were handled by Emacs.

Maybe it's too late and my brain is already asleep, but isn't 65535
2^16-1, i.e. the largest 16-bit number?

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> > BTW, if what you say is the case, then it is all the more 
> > unfortunate, since `color-values' returns values up to
> > 65535 (or 65280, for some platforms). That's
> > 16 ** 4, which means that each color component can be 
> > represented by up to 4 hex digits: #RRRRGGGGBBBB.
> > That's one reason I've always assumed that up to 4 hex
> > digits were handled by Emacs.
> 
> Maybe it's too late and my brain is already asleep, but isn't 65535
> 2^16-1, i.e. the largest 16-bit number?

I might be asleep too, though it's not late here. ;-)

Yes, 65535 is 2^16 - 1. And it is also 16^4 - 1.

16^4 means four hex digits. FFFF is 65535.

So if we have color values from 0 to 65535 inclusive, then we ought to be able
to treat hex codes from 0000 to FFFF also. Four hex digits for each color
component: R, G, and B.

bug#4033: 23.1; list-colors-display is misleading

[Juri Linkov]

> I have had at least one user state that he thought that, since the
> form shown is #RRGGBB, his colors had only that granularity.  IOW,
> even if one's system allows colors of the form #RRRRGGGGBBBB (more
> colors), the color names are translated to hex strings of the form
> #RRGGBB (fewer colors).

Could you show a formula that calculates the color granularity?

We could use it to print colors in the short format #RRGGBB
for the smaller color space and #RRRRGGGGBBBB otherwise.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#4033: 23.1; list-colors-display is misleading

[Jason Rumney]

Drew Adams wrote:
>> 16-bit RGB components is what Emacs uses internally, IIRC.  That is
>> the reason we show each one as two letters.
>>     
>
> Is that right? Could you please check about this?
>   
Emacs uses 16 bit components internally, because that is what X uses in 
its APIs. I'm not aware of any graphics driver that allows more than 8 
bit granularity though, and on Windows you are restricted to 8 bit by 
the API anyway.

bug#4033: 23.1; list-colors-display is misleading

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> 16-bit RGB components is what Emacs uses internally, IIRC.  That is
> the reason we show each one as two letters.

16-bit components would require 4 letters.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> > I have had at least one user state that he thought that, since the
> > form shown is #RRGGBB, his colors had only that granularity.  IOW,
> > even if one's system allows colors of the form #RRRRGGGGBBBB (more
> > colors), the color names are translated to hex strings of the form
> > #RRGGBB (fewer colors).
> 
> Could you show a formula that calculates the color granularity?
> 
> We could use it to print colors in the short format #RRGGBB
> for the smaller color space and #RRRRGGGGBBBB otherwise.

Dunno. I'm really no expert on this.

But IIUC `display-color-cells' gives info about the color depth / number of
colors supported by the display. I was assuming that whatever the display
supports Emacs would support too, but maybe that's a mistaken assumption.

So I was assuming that `display-color-cells' would indicate the number of colors
available (supported). Then, based on that I figured we should be able to know
how many hex digits to use for each component.

Maybe I'm wrong (it's beginning to sound like it), but that's how I was
thinking. Again, I know nothing about this stuff. It's quite likely I'm
confusing apples and oranges. Thanks to whoever clarifies this for me.

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> >> 16-bit RGB components is what Emacs uses internally, IIRC.  That is
> >> the reason we show each one as two letters.
> >
> > Is that right? Could you please check about this?
>   
> Emacs uses 16 bit components internally, because that is what 
> X uses in its APIs. I'm not aware of any graphics driver that allows 
> more than 8 bit granularity though, and on Windows you are restricted
> to 8 bit by the API anyway.

I'm ignorant about these things. Does that imply that Emacs does not support
#RRRRGGGGBBBB (or #RRRGGGBBB) on any systems? What is the relation between these
two 

Doesn't 16-bits per component mean 4 hex digits per component? IOW,
#RRRRGGGGBBBB (internally).

See my reply to Juri. I'd be grateful for clarification. What's the point of
having `display-color-cells' tell me that I have, say 16777216 colors available,
if that's not really the case as far as Emacs is concerned?

Consider me mixed up about this, and please set me straight. Thx.

bug#4033: 23.1; list-colors-display is misleading

[Jason Rumney]

Drew Adams wrote:
> See my reply to Juri. I'd be grateful for clarification. What's the point of
> having `display-color-cells' tell me that I have, say 16777216 colors available,
> if that's not really the case as far as Emacs is concerned?
>   
16777216 = 2^24 = 2^(3*8): #RRGGBB

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> > See my reply to Juri. I'd be grateful for clarification. 
> > What's the point of having `display-color-cells' tell me
> > that I have, say 16777216 colors available,
> > if that's not really the case as far as Emacs is concerned?
>  
> 16777216 = 2^24 = 2^(3*8): #RRGGBB

= 16^6: 6 hex digits, yes.

OK, so that says that my display only handles 8 bits per component.

But what about this?

> > Doesn't 16-bits per component mean 4 hex digits per component?
> > IOW, #RRRRGGGGBBBB (internally).

IOW, even if my particular display only supports 2 hex digits per component, if
Emacs supports 16 bits per component internally, doesn't that mean that it
supports up to #RRRRGGGGBBBB?

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> > I have had at least one user state that he thought that, since the
> > form shown is #RRGGBB, his colors had only that granularity.  IOW,
> > even if one's system allows colors of the form #RRRRGGGGBBBB (more
> > colors), the color names are translated to hex strings of the form
> > #RRGGBB (fewer colors).
> 
> Could you show a formula that calculates the color granularity?
> 
> We could use it to print colors in the short format #RRGGBB
> for the smaller color space and #RRRRGGGGBBBB otherwise.

What about something like this:

(defun rgb-color-format-for-display ()
  (let ((ncolors  (display-color-cells (selected-frame)))
        (exp      0))
    (while (> (lsh ncolors (- exp)) 1) (setq exp  (1+ exp)))
    (setq exp  (/ exp 12))
    (format "#%%0%dx%%0%dx%%0%dx" exp exp exp)))

For 16777216 colors, that gives #%02x%02x%02x, which seems right.

In `list-colors-print', we would then do this:

(insert (apply 'format (rgb-color-format-for-display)
               (mapcar (lambda (c) (lsh c -8))
                         (color-values (car color)))))

IOW, replace the hard-coded "#%02x%02x%02x" with (rgb-color-format-for-display).

But I guess Jason and Eli are saying that that wouldn't work or wouldn't be
appropriate. It's still not clear to me.

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> What about something like this:
> 
> (defun rgb-color-format-for-display ()
>   (let ((ncolors  (display-color-cells (selected-frame)))
>         (exp      0))
>     (while (> (lsh ncolors (- exp)) 1) (setq exp  (1+ exp)))
>     (setq exp  (/ exp 12))
>     (format "#%%0%dx%%0%dx%%0%dx" exp exp exp)))
> 
> For 16777216 colors, that gives #%02x%02x%02x, which seems right.
> 
> In `list-colors-print', we would then do this:
> 
> (insert (apply 'format (rgb-color-format-for-display)
>                (mapcar (lambda (c) (lsh c -8))
>                          (color-values (car color)))))
> 
> IOW, replace the hard-coded "#%02x%02x%02x" with 
> (rgb-color-format-for-display).

We would also want to first do that for one of the colors, to get the field
width, then use that here:

(indent-to (max (- (window-width) RGB-WIDTH) 44))

instead of hard-coding a width of 8, as now:

(indent-to (max (- (window-width) 8) 44))

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> > > I have had at least one user state that he thought that, since the
> > > form shown is #RRGGBB, his colors had only that granularity.  IOW,
> > > even if one's system allows colors of the form #RRRRGGGGBBBB (more
> > > colors), the color names are translated to hex strings of the form
> > > #RRGGBB (fewer colors).
> > 
> > Could you show a formula that calculates the color granularity?
> > 
> > We could use it to print colors in the short format #RRGGBB
> > for the smaller color space and #RRRRGGGGBBBB otherwise.

This seems to DTRT, for printing with an RGB format that reflects the color
support of the display:

(defun list-colors-print (list)
  (let ((rgb-format  (facemenu-rgb-format-for-display))
        (col         (car list))
        rgb-width)
    (when (consp col) (setq col  (car col)))
    (setq rgb-width  (1+ (length (format rgb-format 1 1 1))))
    (dolist (color list)
      (if (consp color)
          (when (cdr color)
            (setq color (sort color (lambda (a b)
                                      (string< (downcase a) (downcase b))))))
        (setq color (list color)))
      (put-text-property (prog1 (point)
                           (insert (car color)) (indent-to 22))
                         (point) 'face (list ':background (car color)))
      (put-text-property (prog1 (point)
                           (insert " " (if (cdr color)
                                           (mapconcat 'identity (cdr color) ",
")
                                         (car color))))
                         (point) 'face (list ':foreground (car color)))
      (indent-to (max (- (window-width) rgb-width) 44))
      (insert (apply 'format rgb-format (mapcar (lambda (c) (lsh c -8))
                                                (color-values (car color)))))
      (insert "\n")))
  (goto-char (point-min)))

The question remains whether this is a useful/reasonable thing to do.

bug#4033: 23.1; list-colors-display is misleading

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <4033@emacsbugs.donarmstrong.com>, "'Eli Zaretskii'" <eliz@gnu.org>
> Date: Tue, 4 Aug 2009 16:55:44 -0700
> 
> Emacs supports 16 bits per component internally, doesn't that mean that it
> supports up to #RRRRGGGGBBBB?

It supports that, but it cannot actually display that if the display
driver has only 8 bits per component.  I.e., the extra bits get lost
when the color is actually displayed.

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> > IOW, even if my particular display only supports 2 hex digits 
> > per component, if Emacs supports 16 bits per component
> > internally, doesn't that mean that it supports up to
> > #RRRRGGGGBBBB?
> 
> It supports that, but it cannot actually display that if the display
> driver has only 8 bits per component.  I.e., the extra bits get lost
> when the color is actually displayed.

I think that's what I just said, so it sounds like I'm on the same page:
internally 16 bits, but a given display might be more limited.

The idea then is to have Emacs use an RGB format that is appropriate for the
user's display. If the display supports 8 bits per component, it would show
#RRGGBB. If the display supports 16 bits per component, it would show
#RRRRGGGGBBBB. If the display supports 12 bits per component, it would show
#RRRGGGBBB. 

Whatever the display's support, the user would see the most appropriate RGB
format. See the code I sent, which I think does that.

bug#4033: 23.1; list-colors-display is misleading

[Jason Rumney]

Drew Adams wrote:

> The idea then is to have Emacs use an RGB format that is appropriate for the
> user's display. If the display supports 8 bits per component, it would show
> #RRGGBB. If the display supports 16 bits per component, it would show
> #RRRRGGGGBBBB. If the display supports 12 bits per component, it would show
> #RRRGGGBBB. 
>   

The number of displays supporting more than 8 bits per component is so 
miniscule that I don't think it is really worth worrying about.  I'm 
also not sure what display-color-cells reports on Windows 7 with 30, 36 
or 48 bit color output enabled. If it reports anything > 2^24, then it 
does not reflect reality, which is that the standard Windows API's do 
not support more than 8 bits per component.

bug#4033: 23.1; list-colors-display is misleading

[Eli Zaretskii]

> Date: Wed, 05 Aug 2009 14:57:10 +0800
> From: Jason Rumney <jasonr@gnu.org>
> CC: 'Eli Zaretskii' <eliz@gnu.org>, 4033@emacsbugs.donarmstrong.com
> 
> The number of displays supporting more than 8 bits per component is so 
> miniscule that I don't think it is really worth worrying about.

Right.  The file rgb.txt we distribute with Emacs uses only 8 bits per
component, as do many (maybe most or even all) X distributions -- all
those I could look up did.  And that is another evidence to what Jason
says above.

bug#4033: 23.1; list-colors-display is misleading

[Juri Linkov]

> What about something like this:
>
> (defun rgb-color-format-for-display ()
>   (let ((ncolors  (display-color-cells (selected-frame)))
>         (exp      0))
>     (while (> (lsh ncolors (- exp)) 1) (setq exp  (1+ exp)))
>     (setq exp  (/ exp 12))
>     (format "#%%0%dx%%0%dx%%0%dx" exp exp exp)))
>
> For 16777216 colors, that gives #%02x%02x%02x, which seems right.
>
> In `list-colors-print', we would then do this:
>
> (insert (apply 'format (rgb-color-format-for-display)
>                (mapcar (lambda (c) (lsh c -8))
>                          (color-values (car color)))))
>
> IOW, replace the hard-coded "#%02x%02x%02x" with (rgb-color-format-for-display).
>
> But I guess Jason and Eli are saying that that wouldn't work or wouldn't be
> appropriate. It's still not clear to me.

I don't understand what is the reason to unnecessarily complicate this.
In your original report you said that one user had some confusion with
hex formats.  Is this user using a system supporting more than 8 bits
per component?

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#4033: 23.1; list-colors-display is misleading

[Drew Adams]

> I don't understand what is the reason to unnecessarily 
> complicate this. In your original report you said that one
> user had some confusion with hex formats.  Is this user using
> a system supporting more than 8 bits per component?

If no one sees the utility, then feel free to ignore and close the bug. 

To answer your question: Regardless of what that user's system supports, from
Jason I've learned that only 8 bits can be effectively supported by Emacs (for
nearly all systems) because of the system APIs that are available to Emacs.

As to how Emacs should display the RGB that corresponds to a named color: I
still think that TRT is to base that feedback on what `display-color-cells'
tells about the user's display - its color support. If tomorrow the API
limitations are alleviated, then that user feedback will automatically reflect
Emacs's improved color support.

It's easy enough to do that, as the code I sent shows. The difference is only a
few lines of uncomplicated code.

It's OK if you disagree about whether that should be done.