Colors on TTY (v26)

Colors on TTY (v26)

[Aleksey Midenkov]

Trial and error shown that emacs holds special knowledge on terminal names:
f.ex. "xterm-something" works as 256 colors and "xterm2" works as 16
colors. This is in fact very strange because there are terminals like
"screen.xterm-256color".
So it's not good to hold special magic about term names for that. Why it
doesn't check terminfo db for that (though it checks for 24bit as doc
states)? Looks like a bug to me.

To test it:
1. infocmp your current term into file;
2. rename file and rename term inside it;
3. tic renamed file;
4. set TERM to renamed term.

And voila: it's either 256 or 16 colors depending on term name!

Another bug:
(list-color-display)
always shows 16 colors from .emacs file (or from -f option). It detects 256
colors somehow later and then list-colors-display executed interactively
shows different result.


-- 
All the best,

Aleksey Midenkov
@midenok

Re: Colors on TTY (v26)

[Eli Zaretskii]

> From: Aleksey Midenkov <midenok@gmail.com>
> Date: Sun, 31 Mar 2019 11:22:59 +0300
> 
> Trial and error shown that emacs holds special knowledge on terminal names:
> f.ex. "xterm-something" works as 256 colors and "xterm2" works as 16
> colors. This is in fact very strange because there are terminals like
> "screen.xterm-256color".
> So it's not good to hold special magic about term names for that. Why it
> doesn't check terminfo db for that (though it checks for 24bit as doc
> states)? Looks like a bug to me.
> 
> To test it:
> 1. infocmp your current term into file;
> 2. rename file and rename term inside it;
> 3. tic renamed file;
> 4. set TERM to renamed term.
> 
> And voila: it's either 256 or 16 colors depending on term name!

Emacs doesn't hard-code terminal names, but it has Lisp libraries
under lisp/term/ which are loaded by the terminal name.  If you have a
terminal that behaves like one of the ones known to Emacs, you need
only copy the relevant Lisp library under lisp/term/ to the name of
your terminal (with the .el extension), and things should work from
there.

> Another bug:
> (list-color-display)
> always shows 16 colors from .emacs file (or from -f option). It detects 256
> colors somehow later and then list-colors-display executed interactively
> shows different result.

The full color list is indeed set during startup, so if you invoke
list-colors-display too early during startup, you will not see that.
This is not a bug, as Emacs can not know the color capabilities of the
terminal until it probes them.

Re: Colors on TTY (v26)

[Aleksey Midenkov]

On Sun, Mar 31, 2019 at 6:15 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Aleksey Midenkov <midenok@gmail.com>
> > Date: Sun, 31 Mar 2019 11:22:59 +0300
> >
> > Trial and error shown that emacs holds special knowledge on terminal
> names:
> > f.ex. "xterm-something" works as 256 colors and "xterm2" works as 16
> > colors. This is in fact very strange because there are terminals like
> > "screen.xterm-256color".
> > So it's not good to hold special magic about term names for that. Why it
> > doesn't check terminfo db for that (though it checks for 24bit as doc
> > states)? Looks like a bug to me.
> >
> > To test it:
> > 1. infocmp your current term into file;
> > 2. rename file and rename term inside it;
> > 3. tic renamed file;
> > 4. set TERM to renamed term.
> >
> > And voila: it's either 256 or 16 colors depending on term name!
>
> Emacs doesn't hard-code terminal names, but it has Lisp libraries
> under lisp/term/ which are loaded by the terminal name.  If you have a
> terminal that behaves like one of the ones known to Emacs, you need
> only copy the relevant Lisp library under lisp/term/ to the name of
> your terminal (with the .el extension), and things should work from
> there.
>

I tried different terminal names. It seems, that it understands "xterm-"
prefix. The remaining part can be arbitrary: I literally tried
"xterm-something" and it worked. I guess, there is no such file
"xterm-something" in the library. In any case, it's much better to rename
terminal to "xterm-*", than deal with distributed files.


>
> > Another bug:
> > (list-color-display)
> > always shows 16 colors from .emacs file (or from -f option). It detects
> 256
> > colors somehow later and then list-colors-display executed interactively
> > shows different result.
>
> The full color list is indeed set during startup, so if you invoke
> list-colors-display too early during startup, you will not see that.
> This is not a bug, as Emacs can not know the color capabilities of the
> terminal until it probes them.
>
>
This is indeed a bug: different result returned by same routine. Either
list-colors-display should not exist at an early stage or color capability
probe should be done earlier.


-- 
All the best,

Aleksey Midenkov
@midenok

Re: Colors on TTY (v26)

[Eli Zaretskii]

> From: Aleksey Midenkov <midenok@gmail.com>
> Date: Sun, 31 Mar 2019 20:20:56 +0300
> Cc: help-gnu-emacs@gnu.org
> 
>  Emacs doesn't hard-code terminal names, but it has Lisp libraries
>  under lisp/term/ which are loaded by the terminal name.  If you have a
>  terminal that behaves like one of the ones known to Emacs, you need
>  only copy the relevant Lisp library under lisp/term/ to the name of
>  your terminal (with the .el extension), and things should work from
>  there.
> 
> I tried different terminal names. It seems, that it understands "xterm-" prefix. The remaining part can be
> arbitrary: I literally tried "xterm-something" and it worked. I guess, there is no such file "xterm-something" in the
> library. In any case, it's much better to rename terminal to "xterm-*", than deal with distributed files.

Just do "ls lisp/term/*.el" and see what you've got there.

>  The full color list is indeed set during startup, so if you invoke
>  list-colors-display too early during startup, you will not see that.
>  This is not a bug, as Emacs can not know the color capabilities of the
>  terminal until it probes them.
> 
> This is indeed a bug: different result returned by same routine. Either list-colors-display should not exist at an
> early stage or color capability probe should be done earlier.

What do you mean "should not exists"?  The code is there, it uses
whatever knowledge it has when you call it.  Where you did call it, it
didn't yet query the terminal to see how many colors it supports, and
didn't set up their names.  IOW, you are shooting yourself in the foot
by calling the function too early.  There are various hooks provided
by startup.el which allow you to call this function when colors are
set up; do that, and Bob's your uncle.

Re: Colors on TTY (v26)

[Aleksey Midenkov]

On Sun, Mar 31, 2019 at 8:41 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Aleksey Midenkov <midenok@gmail.com>
> > Date: Sun, 31 Mar 2019 20:20:56 +0300
> > Cc: help-gnu-emacs@gnu.org
> >
> >  Emacs doesn't hard-code terminal names, but it has Lisp libraries
> >  under lisp/term/ which are loaded by the terminal name.  If you have a
> >  terminal that behaves like one of the ones known to Emacs, you need
> >  only copy the relevant Lisp library under lisp/term/ to the name of
> >  your terminal (with the .el extension), and things should work from
> >  there.
> >
> > I tried different terminal names. It seems, that it understands "xterm-"
> prefix. The remaining part can be
> > arbitrary: I literally tried "xterm-something" and it worked. I guess,
> there is no such file "xterm-something" in the
> > library. In any case, it's much better to rename terminal to "xterm-*",
> than deal with distributed files.
>
> Just do "ls lisp/term/*.el" and see what you've got there.
>

And?


>
> >  The full color list is indeed set during startup, so if you invoke
> >  list-colors-display too early during startup, you will not see that.
> >  This is not a bug, as Emacs can not know the color capabilities of the
> >  terminal until it probes them.
> >
> > This is indeed a bug: different result returned by same routine. Either
> list-colors-display should not exist at an
> > early stage or color capability probe should be done earlier.
>
> What do you mean "should not exists"?  The code is there, it uses
> whatever knowledge it has when you call it.  Where you did call it, it
> didn't yet query the terminal to see how many colors it supports, and
> didn't set up their names.  IOW, you are shooting yourself in the foot
> by calling the function too early.  There are various hooks provided
> by startup.el which allow you to call this function when colors are
> set up; do that, and Bob's your uncle.
>
>
You may call it anything you like, but this misleading behavior causes
trouble.


-- 
All the best,

Aleksey Midenkov
@midenok

Re: Colors on TTY (v26)

[Eli Zaretskii]

> From: Aleksey Midenkov <midenok@gmail.com>
> Date: Sun, 31 Mar 2019 21:09:56 +0300
> Cc: help-gnu-emacs@gnu.org
> 
>  > I tried different terminal names. It seems, that it understands "xterm-" prefix. The remaining part can be
>  > arbitrary: I literally tried "xterm-something" and it worked. I guess, there is no such file
>  "xterm-something" in the
>  > library. In any case, it's much better to rename terminal to "xterm-*", than deal with distributed files.
> 
>  Just do "ls lisp/term/*.el" and see what you've got there.
> 
> And?

And you will see what names are supported without any need to guess.
And if you want more details about how this works, I can suggest
reading the section "Terminal-Specific Initialization" in the ELisp
manual.

>  What do you mean "should not exists"?  The code is there, it uses
>  whatever knowledge it has when you call it.  Where you did call it, it
>  didn't yet query the terminal to see how many colors it supports, and
>  didn't set up their names.  IOW, you are shooting yourself in the foot
>  by calling the function too early.  There are various hooks provided
>  by startup.el which allow you to call this function when colors are
>  set up; do that, and Bob's your uncle.
> 
> You may call it anything you like, but this misleading behavior causes trouble. 

Yes, invoking code in .emacs which depends on stuff that gets set up
during startup can get some using to.  It does make sense, eventually,
though.

Re: Colors on TTY (v26)

[Aleksey Midenkov]

On Sun, Mar 31, 2019 at 9:26 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Aleksey Midenkov <midenok@gmail.com>
> > Date: Sun, 31 Mar 2019 21:09:56 +0300
> > Cc: help-gnu-emacs@gnu.org
> >
> >  > I tried different terminal names. It seems, that it understands
> "xterm-" prefix. The remaining part can be
> >  > arbitrary: I literally tried "xterm-something" and it worked. I
> guess, there is no such file
> >  "xterm-something" in the
> >  > library. In any case, it's much better to rename terminal to
> "xterm-*", than deal with distributed files.
> >
> >  Just do "ls lisp/term/*.el" and see what you've got there.
> >
> > And?
>
> And you will see what names are supported without any need to guess.
> And if you want more details about how this works, I can suggest
> reading the section "Terminal-Specific Initialization" in the ELisp
> manual.


You could describe it briefly in the first place: same amount or less in
characters, but more benefit for readers.


>
>
>  What do you mean "should not exists"?  The code is there, it uses
> >  whatever knowledge it has when you call it.  Where you did call it, it
> >  didn't yet query the terminal to see how many colors it supports, and
> >  didn't set up their names.  IOW, you are shooting yourself in the foot
> >  by calling the function too early.  There are various hooks provided
> >  by startup.el which allow you to call this function when colors are
> >  set up; do that, and Bob's your uncle.
> >
> > You may call it anything you like, but this misleading behavior causes
> trouble.
>
> Yes, invoking code in .emacs which depends on stuff that gets set up
> during startup can get some using to.  It does make sense, eventually,
> though.
>
>

-- 
All the best,

Aleksey Midenkov
@midenok

Re: Colors on TTY (v26)

[Stefan Monnier]

> This is indeed a bug: different result returned by same routine.  Either
> list-colors-display should not exist at an early stage or color capability
> probe should be done earlier.

The probe can't be done earlier, because we want the user to be able to
affect this probe via settings in ~/.emacs.

We could try and make it better, e.g. by making list-colors-display
aware of the fact that in ttys the color info is not immediately
available, so when called early it could signal an error, or it could
eagerly trigger the probe.

Feel free to report a bug about it.

But my guess is that it will not be very high on the priority list of
other people (tho, it's always hard to predict those things), so you're
more likely to see it move forward if you can include some
(proof-of-concept) patch with your bug-report (or if you make the report
into a "could you help me fix it" request more than a "please fix it for
me").


        Stefan