bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

[Jim Porter]

[-- Attachment #1: Type: text/plain, Size: 286 bytes --]

In bug#50179, I added support for bright ANSI colors, but neglected to 
update the termcap/terminfo settings to indicate this. Attached is a 
patch to do so. Note that it doesn't need to be merged into Emacs 29, 
since bug#50806 already fixed this when adding 256/24-bit color support.

[-- Attachment #2: 0001-Update-termcap-terminfo-to-indicate-16-color-support.patch --]
[-- Type: text/plain, Size: 1893 bytes --]

From cff4300e43cea788450c0f399677c3ff06942de8 Mon Sep 17 00:00:00 2001
From: Jim Porter <jporterbugs@gmail.com>
Date: Tue, 19 Oct 2021 16:42:59 -0700
Subject: [PATCH] Update termcap/terminfo to indicate 16-color support.

* etc/e/eterm-color.ti:
* lisp/term.el (term-termcap-format): Indicate 16-color support.
Do not merge to master.
---
 etc/e/eterm-color    | Bin 1179 -> 1179 bytes
 etc/e/eterm-color.ti |   2 +-
 lisp/term.el         |   2 +-
 3 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/etc/e/eterm-color b/etc/e/eterm-color
index bd3f5003ae620db49b89a2c1387b0ba1c836f4f1..441316517d77a1421b2d2118e47e7d112515b8d3 100644
GIT binary patch
delta 12
TcmbQuIh%7r4Wq!uS~(T~8pi{+

delta 12
TcmbQuIh%7r4I{_KS~(T~8lMBM

diff --git a/etc/e/eterm-color.ti b/etc/e/eterm-color.ti
index a6ef814990..4cc4a3c95e 100644
--- a/etc/e/eterm-color.ti
+++ b/etc/e/eterm-color.ti
@@ -9,7 +9,7 @@ eterm-color|Emacs term.el terminal emulator term-protocol-version 0.96,
 # Any change to this file should be done at the same time with a
 # corresponding change to the TERMCAP environment variable in term.el.
 # Comments in term.el specify where each of these capabilities is implemented.
-	colors#8,
+	colors#16,
 	cols#80,
 	lines#24,
 	pairs#64,
diff --git a/lisp/term.el b/lisp/term.el
index e76eb77647..30f26b7ee3 100644
--- a/lisp/term.el
+++ b/lisp/term.el
@@ -1584,7 +1584,7 @@ term-termcap-format
 :so=\\E[7m:se=\\E[m:us=\\E[4m:ue=\\E[m:md=\\E[1m:mr=\\E[7m:me=\\E[m\
 :UP=\\E[%%dA:DO=\\E[%%dB:LE=\\E[%%dD:RI=\\E[%%dC\
 :kl=\\EOD:kd=\\EOB:kr=\\EOC:ku=\\EOA:kN=\\E[6~:kP=\\E[5~:@7=\\E[4~:kh=\\E[1~\
-:mk=\\E[8m:cb=\\E[1K:op=\\E[39;49m:Co#8:pa#64:AB=\\E[4%%dm:AF=\\E[3%%dm:cr=^M\
+:mk=\\E[8m:cb=\\E[1K:op=\\E[39;49m:Co#16:pa#64:AB=\\E[4%%dm:AF=\\E[3%%dm:cr=^M\
 :bl=^G:do=^J:le=^H:ta=^I:se=\\E[27m:ue=\\E[24m\
 :kb=^?:kD=^[[3~:sc=\\E7:rc=\\E8:r1=\\Ec:"
   ;; : -undefine ic
-- 
2.25.1

bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

[miha--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1.1: Type: text/plain, Size: 1697 bytes --]

Jim Porter <jporterbugs@gmail.com> writes:

> In bug#50179, I added support for bright ANSI colors, but neglected to 
> update the termcap/terminfo settings to indicate this. Attached is a 
> patch to do so. Note that it doesn't need to be merged into Emacs 29, 
> since bug#50806 already fixed this when adding 256/24-bit color support.

> -	colors#8,
> +	colors#16,

True, we now do support 16 colors, but the setab and setaf capnames have
to be adjusted as well.  For setaf, for example, the first 8 colors are
specified with sequences \e[30m - \e[37m, and the next 8 colors are
specified with sequences \e[90m - \e[97m.  There is a "gap".  See the
attached patch, which adjusts setab and setaf.

> @@ -1584,7 +1584,7 @@ term-termcap-format
>  :so=\\E[7m:se=\\E[m:us=\\E[4m:ue=\\E[m:md=\\E[1m:mr=\\E[7m:me=\\E[m\
>  :UP=\\E[%%dA:DO=\\E[%%dB:LE=\\E[%%dD:RI=\\E[%%dC\
>  :kl=\\EOD:kd=\\EOB:kr=\\EOC:ku=\\EOA:kN=\\E[6~:kP=\\E[5~:@7=\\E[4~:kh=\\E[1~\
> -:mk=\\E[8m:cb=\\E[1K:op=\\E[39;49m:Co#8:pa#64:AB=\\E[4%%dm:AF=\\E[3%%dm:cr=^M\
> +:mk=\\E[8m:cb=\\E[1K:op=\\E[39;49m:Co#16:pa#64:AB=\\E[4%%dm:AF=\\E[3%%dm:cr=^M\
>  :bl=^G:do=^J:le=^H:ta=^I:se=\\E[27m:ue=\\E[24m\
>  :kb=^?:kD=^[[3~:sc=\\E7:rc=\\E8:r1=\\Ec:"
>    ;; : -undefine ic

As for termcap, I don't think there is a way to specify this gap.  I
think it's best to just leave this unchanged.  In bug#50806, I did
adjust the number of colors in termcap, but I could do it because I
added a way to specify colors without the gap
(with \e[38;2;0m - \e[38;2;15m for setaf).

In conclusion, I think we should update the terminfo, changing colors,
setab and setaf, and leave term.el unchanged.  Patch attached.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Update-terminfo-to-indicate-16-color-support.patch --]
[-- Type: text/x-patch, Size: 1717 bytes --]

From 158e84e403017d2bfc73d9dabeb44bb3ba48f4dc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miha=20Rihtar=C5=A1i=C4=8D?= <miha@kamnitnik.top>
Date: Wed, 20 Oct 2021 10:24:19 +0200
Subject: [PATCH] Update terminfo to indicate 16-color support.

* etc/e/eterm-color.ti: Indicate 16-color support.
Do not merge to master.
---
 etc/e/eterm-color    | Bin 1179 -> 1227 bytes
 etc/e/eterm-color.ti |   6 +++---
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/etc/e/eterm-color b/etc/e/eterm-color
index bd3f5003ae620db49b89a2c1387b0ba1c836f4f1..06e8dc01ed6c136c3b8a32ae42eed81745efd1fb 100644
GIT binary patch
delta 109
zcmbQud76`3iqV~c9|$uUD<^UnG74<0z0AZIxH*tnkI~0owZKrd+M-s~MzzEk#7<F7
awS<Z30(sWC4ARl6$jVHh$_x$A)Bym^7aLdr

delta 61
zcmX@jIh&JPiqV~c9|$uUJtuM(GIDIJz0AaDwK<Skk5Q(;P_^3FpjK5|H6@opI$9OT
JSJzO7Z~?9&4dDO)

diff --git a/etc/e/eterm-color.ti b/etc/e/eterm-color.ti
index a6ef814990..a6ed6d28ab 100644
--- a/etc/e/eterm-color.ti
+++ b/etc/e/eterm-color.ti
@@ -9,7 +9,7 @@ eterm-color|Emacs term.el terminal emulator term-protocol-version 0.96,
 # Any change to this file should be done at the same time with a
 # corresponding change to the TERMCAP environment variable in term.el.
 # Comments in term.el specify where each of these capabilities is implemented.
-	colors#8,
+	colors#16,
 	cols#80,
 	lines#24,
 	pairs#64,
@@ -65,8 +65,8 @@ eterm-color|Emacs term.el terminal emulator term-protocol-version 0.96,
 	rmul=\E[24m,
 	rs1=\Ec,
 	sc=\E7,
-	setab=\E[%p1%{40}%+%dm,
-	setaf=\E[%p1%{30}%+%dm,
+	setab=\E[%?%p1%{8}%<%t4%p1%d%e10%p1%{8}%-%d%;m,
+	setaf=\E[%?%p1%{8}%<%t3%p1%d%e9%p1%{8}%-%d%;m,
 	sgr0=\E[m,
 	smir=\E[4h,
 	smul=\E[4m,
-- 
2.33.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

[Jim Porter]

On 10/20/2021 1:29 AM, miha--- via Bug reports for GNU Emacs, the Swiss 
army knife of text editors wrote:
> In conclusion, I think we should update the terminfo, changing colors,
> setab and setaf, and leave term.el unchanged.  Patch attached.

Thanks, this seems reasonable to me. I don't know much about the details 
of terminfo, so I wasn't aware of setab/setaf.

More generally, I do wonder if we should be concerned with making it 
possible to distinguish between different versions of Emacs, since 27, 
28, and 29 all have different capabilities. At least as I understand it, 
If I SSH into a system, having `TERM=eterm-color' isn't enough to know 
whether 8, 16, or 256 colors are supported.

bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

[miha--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2215 bytes --]

Jim Porter <jporterbugs@gmail.com> writes:

> On 10/20/2021 1:29 AM, miha--- via Bug reports for GNU Emacs, 
> the Swiss 
> army knife of text editors wrote:
>> In conclusion, I think we should update the terminfo, changing 
>> colors, setab and setaf, and leave term.el unchanged.  Patch 
>> attached.
>
> Thanks, this seems reasonable to me. I don't know much about the 
> details  of terminfo, so I wasn't aware of setab/setaf.
>
> More generally, I do wonder if we should be concerned with 
> making it  possible to distinguish between different versions of 
> Emacs, since 27,  28, and 29 all have different capabilities. At 
> least as I understand it,  If I SSH into a system, having 
> `TERM=eterm-color' isn't enough to know  whether 8, 16, or 256 
> colors are supported.

Yeah, that's a somewhat complicated problem.  I thought about the 
option of having something like 'TERM=eterm-16color' on Emacs-28 
in order to make Emacs-28 and Emacs-27 term.el distinguishable. 
The major downside of this option is that many terminal programs 
simply refuse to start if eterm-16color is not registered in the 
terminfo database (try running
'env TERM=eterm-16color top').  ssh-ing from term.el onto a system 
that rarely updates its terminfo database wouldn't be pleasant at 
all.

That's why I think its best to keep using 'TERM=eterm-color' and 
simply update the terminfo file.  Since all our updates to this 
file are backwards-compatible, there shouldn't be any major 
problems even if the terminfo database on the ssh host isn't up to 
date.  A problem does arise in the opposite case when the terminfo 
database on the host is up to date but the ssh client is term.el 
from Emacs version 27 or older.  TUI programs will think they can 
output sequences that the older term.el doesn't understand, but I 
think that term.el handles unknown sequences quite gracefully by 
simply ignoring them.

After Emacs-28 is released, we should probably talk to the ncurses 
maintainer about updating the eterm-color terminfo file 
distributed with the ncurses package.  From my understanding, the 
option I described in the previous paragraph is also the proffered 
way to update terminfo files in the ncurses package.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

[Stefan Kangas]

<miha@kamnitnik.top> writes:

> That's why I think its best to keep using 'TERM=eterm-color' and simply update
> the terminfo file.  Since all our updates to this file are backwards-compatible,
> there shouldn't be any major problems even if the terminfo database on the ssh
> host isn't up to date.  A problem does arise in the opposite case when the
> terminfo database on the host is up to date but the ssh client is term.el from
> Emacs version 27 or older.  TUI programs will think they can output sequences
> that the older term.el doesn't understand, but I think that term.el handles
> unknown sequences quite gracefully by simply ignoring them.

If they are ignored, I see no problem here.  Someone should perhaps double-check
that this is indeed the case?

> After Emacs-28 is released, we should probably talk to the ncurses maintainer
> about updating the eterm-color terminfo file distributed with the ncurses
> package.  From my understanding, the option I described in the previous
> paragraph is also the proffered way to update terminfo files in the ncurses
> package.

Are there any other terminfo files elsewhere that are worth updating, or
do they generally just base themselves on ncurses?

bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

[Lars Ingebrigtsen]

miha@kamnitnik.top writes:

> In conclusion, I think we should update the terminfo, changing colors,
> setab and setaf, and leave term.el unchanged.  Patch attached.
>
> From 158e84e403017d2bfc73d9dabeb44bb3ba48f4dc Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Miha=20Rihtar=C5=A1i=C4=8D?= <miha@kamnitnik.top>
> Date: Wed, 20 Oct 2021 10:24:19 +0200
> Subject: [PATCH] Update terminfo to indicate 16-color support.
>
> * etc/e/eterm-color.ti: Indicate 16-color support.
> Do not merge to master.

I think it's too late to put something like this in emacs-28.  Do I
understand correctly that it's not needed in Emacs 29?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

[Jim Porter]

On 11/4/2021 4:20 PM, Lars Ingebrigtsen wrote:
> miha@kamnitnik.top writes:
> 
>> In conclusion, I think we should update the terminfo, changing colors,
>> setab and setaf, and leave term.el unchanged.  Patch attached.
>>
>>  From 158e84e403017d2bfc73d9dabeb44bb3ba48f4dc Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Miha=20Rihtar=C5=A1i=C4=8D?= <miha@kamnitnik.top>
>> Date: Wed, 20 Oct 2021 10:24:19 +0200
>> Subject: [PATCH] Update terminfo to indicate 16-color support.
>>
>> * etc/e/eterm-color.ti: Indicate 16-color support.
>> Do not merge to master.
> 
> I think it's too late to put something like this in emacs-28.  Do I
> understand correctly that it's not needed in Emacs 29?

Correct, there's already a fix for this in Emacs 29 from (I think) 
bug#50806. If it's too late to add this for 28, then I think we can 
close this.

For Emacs 28, the only negative to not merging Miha's patch is that 
users will have to manually update eterm-color (and install it to their 
systems) if they want to be sure their terminfo database correctly 
represents the capabilities in Emacs 28's `term-mode'. That's probably 
not a big deal, and worst-case scenario, some commands might only use 8 
colors instead of 16.

bug#51297: 28.0.60; [PATCH] Update termcap/terminfo to indicate 16-color support

[Lars Ingebrigtsen]

Jim Porter <jporterbugs@gmail.com> writes:

> Correct, there's already a fix for this in Emacs 29 from (I think)
> bug#50806. If it's too late to add this for 28, then I think we can
> close this.

OK; closing.

> For Emacs 28, the only negative to not merging Miha's patch is that
> users will have to manually update eterm-color (and install it to
> their systems) if they want to be sure their terminfo database
> correctly represents the capabilities in Emacs 28's
> `term-mode'. That's probably not a big deal, and worst-case scenario,
> some commands might only use 8 colors instead of 16.

Yeah, the repercussions to not applying this are fairly low, but the
unknown effects this may have (on obscure systems) is fairly high, so
it's just too late in the development cycle for emacs-28, in my opinion.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no