display-supports-face-attributes-p bug?

display-supports-face-attributes-p bug?

[Richard Stallman]

According to display-supports-face-attributes-p, the console
implemented by Linux supports underline.  But underlining is not
visible.

Is display-supports-face-attributes-p wrong, or is there a bug
in displaying underlines?

Re: display-supports-face-attributes-p bug?

[James Cloos]

>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:

Richard> According to display-supports-face-attributes-p, the console
Richard> implemented by Linux supports underline.  But underlining is
Richard> not visible.

Richard> Is display-supports-face-attributes-p wrong, or is there a
Richard> bug in displaying underlines?

The console supports the escape codes for half-brite and underline,
but uses a different colour rather than different glyphs.

I suspect that -- at least on VGA hardware -- the BSDs also do so.

-JimC
-- 
James H. Cloos, Jr. <cloos@jhcloos.com>

Re: display-supports-face-attributes-p bug?

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Wed, 08 Jun 2005 12:00:10 -0400
> 
> According to display-supports-face-attributes-p, the console
> implemented by Linux supports underline.  But underlining is not
> visible.

How did you test that?  If you type "M-x list-faces-display RET",
don't you see the face names underlined?

Re: display-supports-face-attributes-p bug?

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> According to display-supports-face-attributes-p, the console
> implemented by Linux supports underline.  But underlining is not
> visible.

Underlining is implemented with a specific color, which can be set with
"setterm -ulcolor" (default is bright white, IIRC).  The framebuffer
console does not support real underlining (except on a 1-bit display).

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: display-supports-face-attributes-p bug?

[James Cloos]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

Eli> How did you test that?  If you type "M-x list-faces-display RET",
Eli> don't you see the face names underlined?

I just gave that a test in gnome-terminal and at a virtual console:

Using:

   env TERM=linux emacs -nw

the underlines do not occur -- neither real underlines in gnome-term
nor a colour change on the console.

Using:

   env TERM=xterm emacs -nw

they both work as expected.

Strace(8)ing setterm(1) shows that it also uses terminfo and get's the
same escape sequence for underline for xterm as it does for linux.

So emacs should know that \033[4m turns on underline in both cases,
but somehow doesn't output it if %ENV{TERM} ~= 'linux' (to use Perl
syntax -- I forget the elisp construct).  Is something special-casing
consoles?

-JimC
-- 
James H. Cloos, Jr. <cloos@jhcloos.com>

Re: display-supports-face-attributes-p bug?

[Miles Bader]

On 6/9/05, Richard Stallman <rms@gnu.org> wrote:
> According to display-supports-face-attributes-p, the console
> implemented by Linux supports underline.  But underlining is not
> visible.
> 
> Is display-supports-face-attributes-p wrong, or is there a bug
> in displaying underlines?

At least on my system (a vaguely up-to-date Debian "sarge"), the
terminfo entry for TERM=linux _claims_ an underlining capability:

   (miles) TERM=linux infocmp -L
   #       Reconstructed via infocmp from file: /etc/terminfo/l/linux
   linux|linux console,
   ...
	   enter_underline_mode=\E[4m, erase_chars=\E[%p1%dX,
   ...
	   exit_underline_mode=\E[24m,
   ...

However my actual linux (VGA) console seems to display this as bold.

Perhaps this is a bug in the "linux" terminfo entry.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: display-supports-face-attributes-p bug?

[Andreas Schwab]

Miles Bader <snogglethorpe@gmail.com> writes:

> However my actual linux (VGA) console seems to display this as bold.

That's the default underline color (bright white).

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: display-supports-face-attributes-p bug?

[Miles Bader]

On 6/9/05, Andreas Schwab <schwab@suse.de> wrote:
> Miles Bader <snogglethorpe@gmail.com> writes:
> 
> > However my actual linux (VGA) console seems to display this as bold.
> 
> That's the default underline color (bright white).

Er, what difference does it make?  It's still not an underline.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: display-supports-face-attributes-p bug?

[Eli Zaretskii]

> Cc: rms@gnu.org, emacs-devel@gnu.org
> From: James Cloos <cloos@jhcloos.com>
> Date: Wed, 08 Jun 2005 17:00:01 -0400
> 
> Strace(8)ing setterm(1) shows that it also uses terminfo and get's the
> same escape sequence for underline for xterm as it does for linux.
> 
> So emacs should know that \033[4m turns on underline in both cases,
> but somehow doesn't output it if %ENV{TERM} ~= 'linux' (to use Perl
> syntax -- I forget the elisp construct).  Is something special-casing
> consoles?

I don't think Emacs does anything special for the Linux console, if
that's what you meant.

I think the way to debug this is to step through the code in term.c
that produces underlining.  First, make sure that
TS_enter_underline_mode and TS_exit_underline_mode hold the right
strings, then, in turn_on_face, make sure that the check

  if (face->tty_underline_p && MAY_USE_WITH_COLORS_P (NC_UNDERLINE))

does TRT (where TRT is to underline if the Linux console can mix
underlining with colors).

If it turns out that Emacs thinks the Linux console cannot mix colors
with underlining, you can try "emacs --color=no" to see if that causes
underlining to work.  If it does, it means that mixing colors with
underlining is the issue; if the Linux terminal can actually do that,
perhaps the way Emacs checks for this needs some work.

Re: display-supports-face-attributes-p bug?

[Miles Bader]

On 6/9/05, Eli Zaretskii <eliz@gnu.org> wrote:
> I think the way to debug this is to step through the code in term.c
> that produces underlining.  First, make sure that
> TS_enter_underline_mode and TS_exit_underline_mode hold the right
> strings, then, in turn_on_face, make sure that the check

I don't understand this message at all -- there seems to be no problem
with Emacs, the problem is that terminfo claims that the linux console
can do underlining, but in fact the linux console can't do
underlining.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: display-supports-face-attributes-p bug?

[James Cloos]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

Eli> you can try "emacs --color=no" to see if that causes underlining
Eli> to work.

Yes, that does change emacs' output.  With TERM=linux underlining
works iff --color=no.

I presume infocmp -d linux xterm should show the relevant differences
between the linux and xterm terminfo files.

-JimC
-- 
James H. Cloos, Jr. <cloos@jhcloos.com>

Re: display-supports-face-attributes-p bug?

[Andreas Schwab]

Miles Bader <snogglethorpe@gmail.com> writes:

> On 6/9/05, Andreas Schwab <schwab@suse.de> wrote:
>> Miles Bader <snogglethorpe@gmail.com> writes:
>> 
>> > However my actual linux (VGA) console seems to display this as bold.
>> 
>> That's the default underline color (bright white).
>
> Er, what difference does it make?  It's still not an underline.

In the context of the Linux console, this is how underline is displayed.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: display-supports-face-attributes-p bug?

[Richard Stallman]

    The console supports the escape codes for half-brite and underline,
    but uses a different colour rather than different glyphs.

When I do M-x list-faces-display, the output for `underline' seems to
be identical to the default.

    Using:

       env TERM=xterm emacs -nw

    they both work as expected.

When I try that, the underline face comes out as bold.

I would guess that the difference in behavior is due to a difference
in the terminfo specs.  The terminfo specs for type `linux' may
be erroneous.  Could someone check that?

However, given that Emacs can ask for the same result as "bold", it
would probably be better for display-supports-face-attributes-p to
honestly report that this terminal does not really support
underlining.  It should return nil for both `linux' and `xterm'.

If we want to preserve the current behavior of the face `underline'
for type `xterm' (and maybe extend it to type `linux'), it could
specify bold for terminals that support bold but not underline.

Re: display-supports-face-attributes-p bug?

[Eli Zaretskii]

> Cc: rms@gnu.org, emacs-devel@gnu.org
> From: James Cloos <cloos@jhcloos.com>
> Date: Thu, 09 Jun 2005 00:46:27 -0400
> 
> Eli> you can try "emacs --color=no" to see if that causes underlining
> Eli> to work.
> 
> Yes, that does change emacs' output.  With TERM=linux underlining
> works iff --color=no.

So that probably means Emacs thinks colors and underlining cannot be
used together on that terminal.

> I presume infocmp -d linux xterm should show the relevant differences
> between the linux and xterm terminfo files.

That is not an efficient method, as the terminfo entries have lots of
differences.  It is much better to step with a debugger through the
code in term.c in the places I mentioned earlier in this thread.

Re: display-supports-face-attributes-p bug?

[Eli Zaretskii]

> Date: Thu, 9 Jun 2005 13:44:55 +0900
> From: Miles Bader <snogglethorpe@gmail.com>
> Cc: James Cloos <cloos@jhcloos.com>, rms@gnu.org, emacs-devel@gnu.org
> 
> On 6/9/05, Eli Zaretskii <eliz@gnu.org> wrote:
> > I think the way to debug this is to step through the code in term.c
> > that produces underlining.  First, make sure that
> > TS_enter_underline_mode and TS_exit_underline_mode hold the right
> > strings, then, in turn_on_face, make sure that the check
> 
> I don't understand this message at all -- there seems to be no problem
> with Emacs, the problem is that terminfo claims that the linux console
> can do underlining, but in fact the linux console can't do
> underlining.

But James just wrote that turning off colors causes underlining to
work correctly.  That means the Linux console is in fact capable of
underlining.

Re: display-supports-face-attributes-p bug?

[James Cloos]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

Eli> But James just wrote that turning off colors causes underlining
Eli> to work correctly.  That means the Linux console is in fact
Eli> capable of underlining.

What I meant is that is displays what a user of the linux consoles
expects for underlining, which is a different colour.  Ie, it supports
the standard vt100 escape sequence, but the output is not physically
an underline.

When using TERM=linux in gnome-terminal you do get real underlines.

One issue then is whether emacs should ignore faux underlining
support.  Console users will have certain expectations on what
underlining, bold, el al mean that are obviously different than
what users of terminal emulators may expect.  I'd certainly
expect emacs to do what all of the other apps do and ignore that
the console emulates underlining via a colour change.

The other issue is how emacs determines that a given term's
underlining support is via real underlines or some alternative
representation and, if the latter, whether colouring text might
collide with the application of the underline attribute.

-JimC

Re: display-supports-face-attributes-p bug?

[Richard Stallman]

    If it turns out that Emacs thinks the Linux console cannot mix colors
    with underlining, you can try "emacs --color=no" to see if that causes
    underlining to work.

--color=no does cause underlining to "work"--namely, to display as
bold.

This doesn't alter the conclusions I sent in the previous message: on
this terminal, and on xterm, Emacs should say that underlining is NOT
supported.

Re: display-supports-face-attributes-p bug?

[Miles Bader]

On 6/10/05, Richard Stallman <rms@gnu.org> wrote:
> This doesn't alter the conclusions I sent in the previous message: on
> this terminal, and on xterm, Emacs should say that underlining is NOT
> supported.

That's correct for the linux console, but underlining _does_ work
properly on xterm (and on every other X terminal emulator  I've
tried).

As it seems unlikely for terminfo to be changed anytime soon (and
broken installations would remain for a long time anyway), it seems
that Emacs just needs to special-case the terminal name "linux" to
reflect that underlining isn't supported there.

Currently there doesn't seem to be a mechanism for overriding the
low-level C terminal code from lisp though.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: display-supports-face-attributes-p bug?

[James Cloos]

RMS> --color=no does cause underlining to "work"--namely, to display
RMS> as bold.

RMS> This doesn't alter the conclusions I sent in the previous
RMS> message: on this terminal, and on xterm, Emacs should say that
RMS> underlining is NOT supported.

In that case, at least when --color=yes I suppose Emacs is already
doing what you want it to:  ignoring that the console has an escape
sequence for ul when all that sequence does in a colour change.

But note that it is possible that real underlining will be added in
the future for those that use a framebuffer console.  It may even
already work on non-x86, non-VGA hardware -- such as sun or ppc
hardware -- but I do not have such a box to check.

The colour change thing comes from the limits of VGA, and has been
duplicated by -- at least -- the current framebuffers on x86.  This
is the same thing emulators like kermit did on PCs before linux --
which is of course why linux does it.

-JimC  (who, btw, is wearing more of a kernel-hacker hat than
        an emacs-hacker hat in this thread)
-- 
James H. Cloos, Jr. <cloos@jhcloos.com>

Re: display-supports-face-attributes-p bug?

[Miles Bader]

On 6/10/05, James Cloos <cloos@jhcloos.com> wrote:
> But note that it is possible that real underlining will be added in
> the future for those that use a framebuffer console.  It may even
> already work on non-x86, non-VGA hardware -- such as sun or ppc
> hardware -- but I do not have such a box to check.

Unless there's some way Emacs can find out the true state of things,
there's not much that can be done.  If it's impossible to tell whether
there's correct underlining support, it seems safer to assume there's
not -- or at the least don't _advertise_ that there is.  In other
words, probably the right thing to do is still emit
enter/exit-underlining escape codes for underlined faces, but have
`display-supports-face-attribute-p' return nil for the :underline
attribute.

It also seems reasonable to base the default Emacs behavior on the
current state of the most popular platform, in cases where there's no
way of distinguishing.  [Of course if we add a hack to override
terminfo, it would also be good to have a way for the user to override
that.]

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: display-supports-face-attributes-p bug?

[James Cloos]

>>>>> "Miles" == Miles Bader <snogglethorpe@gmail.com> writes:

Miles> If it's impossible to tell whether there's correct underlining
Miles> support, it seems safer to assume there's not -- or at the
Miles> least don't _advertise_ that there is.  In other words,
Miles> probably the right thing to do is still emit
Miles> enter/exit-underlining escape codes for underlined faces, but
Miles> have `display-supports-face-attribute-p' return nil for the
Miles> :underline attribute.

Sounds like a most reasonable compromise.

Miles> It also seems reasonable to base the default Emacs behavior on
Miles> the current state of the most popular platform, in cases where
Miles> there's no way of distinguishing.

Certainly.  I just pointed out that things might change because there
is a desire in lkml-land to move the vt100-ish part of the console
from kernelspace to userspace, and that will likely result in
emulation that is much more like xterm than anything else.  And since
xterm supports real underlining....

Miles> [Of course if we add a hack to override terminfo, it would also
Miles> be good to have a way for the user to override that.]

That would be cool.

-JimC

(I'll probably bow out of this thread now.)
-- 
James H. Cloos, Jr. <cloos@jhcloos.com>

Re: display-supports-face-attributes-p bug?

[Eli Zaretskii]

> Cc: miles@gnu.org, emacs-devel@gnu.org
> From: James Cloos <cloos@jhcloos.com>
> Date: Thu, 09 Jun 2005 17:38:54 -0400
> 
> What I meant is that is displays what a user of the linux consoles
> expects for underlining, which is a different colour.  Ie, it supports
> the standard vt100 escape sequence, but the output is not physically
> an underline.

Ah, sorry, I misunderstood you.

> One issue then is whether emacs should ignore faux underlining
> support.  Console users will have certain expectations on what
> underlining, bold, el al mean that are obviously different than
> what users of terminal emulators may expect.  I'd certainly
> expect emacs to do what all of the other apps do and ignore that
> the console emulates underlining via a colour change.

If you describe how those other apps do this, perhaps we could do the
same in Emacs.

> The other issue is how emacs determines that a given term's
> underlining support is via real underlines or some alternative
> representation and, if the latter, whether colouring text might
> collide with the application of the underline attribute.

See term.c: Emacs looks at the bits in the value of the "NC"
capability of the terminal.

Re: display-supports-face-attributes-p bug?

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: cloos@jhcloos.com, emacs-devel@gnu.org
> Date: Thu, 09 Jun 2005 20:14:11 -0400
> 
> This doesn't alter the conclusions I sent in the previous message: on
> this terminal, and on xterm, Emacs should say that underlining is NOT
> supported.

How can Emacs do this if the terminfo data tells us that underlining
_is_ supported?

Re: display-supports-face-attributes-p bug?

[Eli Zaretskii]

> Date: Fri, 10 Jun 2005 09:59:04 +0900
> From: Miles Bader <snogglethorpe@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, cloos@jhcloos.com, emacs-devel@gnu.org
> 
> That's correct for the linux console, but underlining _does_ work
> properly on xterm (and on every other X terminal emulator  I've
> tried).

Right.  And that includes many xterm emulators running on Windows as
well, such as PuTTY.

> As it seems unlikely for terminfo to be changed anytime soon (and
> broken installations would remain for a long time anyway), it seems
> that Emacs just needs to special-case the terminal name "linux" to
> reflect that underlining isn't supported there.

Yuck!  I'd prefer we invest our energy in talking the Linux
maintainers into changing their terminfo entries.  Hardcoding terminal
names is ugly and would be a maintenance mess if that terminal expands
its capabilities in the future.

We always insisted in behaving as the terminfo entry says.  The only
places where we currently look at the terminal's name is when there's
no known way if figuring out something, like with the (unknown)
default background color of the terminal.

> Currently there doesn't seem to be a mechanism for overriding the
> low-level C terminal code from lisp though.

Right.  The only existing mechanism is through the tty-color-mode
frame parameter, and it only supports overriding the number of
available colors.

Re: display-supports-face-attributes-p bug?

[Miles Bader]

On 6/10/05, Eli Zaretskii <eliz@gnu.org> wrote:
> > As it seems unlikely for terminfo to be changed anytime soon (and
> > broken installations would remain for a long time anyway), it seems
> > that Emacs just needs to special-case the terminal name "linux" to
> > reflect that underlining isn't supported there.
> 
> Yuck!  I'd prefer we invest our energy in talking the Linux
> maintainers into changing their terminfo entries.  Hardcoding terminal
> names is ugly and would be a maintenance mess if that terminal expands
> its capabilities in the future.

Hey I don't like it either, but relying terminfo to be correct seems
overly optimistic -- even if we can quickly get the terminfo
maintainers to make a change, it will take a long time to propagate to
all installations (people often update major apps like Emacs while
background stuff like terminfo languishes).

Probably we should do both, regarding the name check as a short-term workaround.

> We always insisted in behaving as the terminfo entry says.  The only
> places where we currently look at the terminal's name is when there's
> no known way if figuring out something, like with the (unknown)
> default background color of the terminal.

If the terminfo entry is most likely wrong, and we know it, then it
doesn't make sense to follow it.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: display-supports-face-attributes-p bug?

[Eli Zaretskii]

> Date: Fri, 10 Jun 2005 15:41:53 +0900
> From: Miles Bader <snogglethorpe@gmail.com>
> Cc: miles@gnu.org, cloos@jhcloos.com, emacs-devel@gnu.org
> 
> If the terminfo entry is most likely wrong, and we know it, then it
> doesn't make sense to follow it.

Nevertheless, until now, we always did.

Re: display-supports-face-attributes-p bug?

[Miles Bader]

On 6/10/05, Eli Zaretskii <eliz@gnu.org> wrote:
> > If the terminfo entry is most likely wrong, and we know it, then it
> > doesn't make sense to follow it.
> 
> Nevertheless, until now, we always did.

So.... should we not fix old bugs?

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: display-supports-face-attributes-p bug?

[David Kastrup]

Miles Bader <snogglethorpe@gmail.com> writes:

> On 6/10/05, Eli Zaretskii <eliz@gnu.org> wrote:
>> > If the terminfo entry is most likely wrong, and we know it, then it
>> > doesn't make sense to follow it.
>> 
>> Nevertheless, until now, we always did.
>
> So.... should we not fix old bugs?

Why fix an old bug if you can write three new ones in the same time?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: display-supports-face-attributes-p bug?

[Richard Stallman]

    But James just wrote that turning off colors causes underlining to
    work correctly.

It causes "underlined" text to appear as bold.  That's not correct
underlining.  As a substitute, it may be better than nothing, for
other apps.

    That means the Linux console is in fact capable of
    underlining.

It means the console is capable of pretending to underline.

Re: display-supports-face-attributes-p bug?

[Richard Stallman]

    When using TERM=linux in gnome-terminal you do get real underlines.

What value of TERM does gnome-terminal normally use?  I hope Emacs can
distinguish gnome-terminal (which really handles underlining) from
the console and xterm (which do not).

Re: display-supports-face-attributes-p bug?

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> What value of TERM does gnome-terminal normally use?  I hope Emacs can
> distinguish gnome-terminal (which really handles underlining) from
> the console and xterm (which do not).

xterm can display underline.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: display-supports-face-attributes-p bug?

[Miles Bader]

On 6/10/05, Richard Stallman <rms@gnu.org> wrote:
>     When using TERM=linux in gnome-terminal you do get real underlines.
> 
> What value of TERM does gnome-terminal normally use?  I hope Emacs can
> distinguish gnome-terminal (which really handles underlining) from
> the console and xterm (which do not).

My gnome-terminals generally seem to have TERM=xterm -- which is OK,
because xterm _does_ support (real) underlining, and its terminfo
entry reflects this.

It's only the linux console which seems problematic.

-Miles
-- 
Do not taunt Happy Fun Ball.

Re: display-supports-face-attributes-p bug?

[James Cloos]

>>>>> "Miles" == Miles Bader <snogglethorpe@gmail.com> writes:

Miles> If the terminfo entry is most likely wrong, and we know it,
Miles> then it doesn't make sense to follow it.

One issue with this, btw, that I forgot at first is logging in to
another box from the shell.  There is no guarantee that the remote
box has a 'linux' entry in its terminfo/termcap db, so users will
likely specify vt100, vt220 or xterm on the remote box.  

In general there is no way to be certain that the ul escape sequence
in the terminfo does real underlining, no matter what the entry is
named.

-JimC

Re: display-supports-face-attributes-p bug?

[Richard Stallman]

I installed a feature to make it possible for term/*.el to say
that the terminal does not really support underlining.
It is not elegant but it should do the job.

I also changed the underline face as follows:

(defface underline '((((supports :underline t))
		      :underline t)
		     (((supports :weight bold))
		      :weight bold)
		     (t :underline t))
  "Basic underlined face."
  :group 'basic-faces)

I think this is the right way for such faces to work.
It seems to work, but I have not installed it yet.