bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii <eliz@gnu.org>]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Mon, 6 Jun 2016 07:42:37 -0400
> Cc: John Mastro <john.b.mastro@gmail.com>, 23574@debbugs.gnu.org, cwoodbury@azavea.com
> 
> > When the newline does not have the underline attribute, the underline
> > is not contiguous, so you are radically changing the use case.
> 
> Yes. However, I believe that this is what the original ensime code
> intended to do; it only underlines the newlines themselves because
> it's easier to make 1 overlay for all the lines at once and the
> programmer didn't notice it was wrong because it happens to give the
> desired effect in GUI mode.

I don't see how the effect on GUI frames could be considered
"desired".  What about the fact that the underline extends one space
beyond the line's text?  So on GUI frames we see the same problem,
just of smaller dimensions.  Or am I missing something?

> Regardless, by experimentation I find that the space at the edge of
> the screen takes the face from the final newline, not the last
> displayed glyph character in the line. Is this documented anywhere?

That space at the edge of the line is not a space at all, although it
looks like one.  It is a special glyph produced by the display engine,
primarily so that we could display the cursor at the end of the line.
Its attributes are invented by the display engine out of thin air for
its own purposes; for example, the buffer position recorded in that
glyph is zero, not the position of the newline.

As for your conclusion, I believe there's a misunderstanding here.
You are talking about the face of a buffer position, while I was
talking about the glyphs on the screen.  Other that this minor
disconnect, I don't see any contradiction between what we say.

Also note that the display engine doesn't examine each character's
face when it produces glyphs for display, it only examines the faces
where they change.  Which in this case means that the face of the
newline is immaterial; what matters is whether it is identical to that
of preceding characters.  To be precise, the face used for extension
is the one loaded into the iterator object when it hits the end of the
line.  As there are too many ways to specify faces in Emacs, I won't
risk confirming that your conclusion is 100% correct ;-)  My
description is correct, but perhaps less useful to a Lisp programmer.

As for documentation: these all are fine details of the display engine
that are not documented anywhere except in the code comments.  Even
the face extension itself isn't mentioned anywhere, I believe simply
because the effect is natural and expected, whereas its accurate
documentation is not simple at all.  Does it really make sense to
document just this specific subtlety?

IOW, if you are interested in these details, you should be hacking the
display engine code long ago ;-)

Going back to the bug report, there's still one issue to consider:
should we add underline (and then also overline and strikethrough) to
the list of face attributes that cause face extension on GUI frames.
The logic behind the current code seems to be to extend attributes
that are related to background of the text.  The above 3 seem to be a
kind-of background, so maybe we should add them.