bug#21835: 25.0.50; cursor height wrong when line-spacing is used

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 3235 bytes --]

Start Emacs 25 (master branch)  (with -Q if you like)

(setq line-spacing 1.0)


(Note how the display doesn’t update, so enter C-l to force it)


Result: cursor is displayed with wrong (large, two-line) height when at the end of a line, and correctly when not at the end of a line.

Works similarly with different cursor types.

This wasn’t the case in Emacs 24.

Also note bug#17977, although I disagree about the reporter’s preferences.  In any case, the cursor’s height should not change.  If it has to be this high, the actual text line should be vertically centered.







In GNU Emacs 25.0.50.1 (x86_64-apple-darwin15.0.0, NS appkit-1404.11 Version 10.11 (Build 15A284))
 of 2015-11-04
Repository revision: 5401bb8645f955b6fde8c0a5e5c8cf5ec1cab626
Windowing system distributor 'Apple', version 10.3.1404
Configured using:
 'configure 'CFLAGS=-O3 -g''

Configured features:
RSVG DBUS ACL LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  which-function-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
.emacs: (lambda (x) ...) quoted with ' rather than with #' [2 times]
For information about GNU Emacs and the GNU system, type C-h C-a.
1.0
next-line: End of buffer [2 times]

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message dired format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns help-mode easymenu cl-loaddefs pcase cl-lib mail-prsvr
mail-utils warnings ghostedit ghostedit-stopwords-en easy-mmode
which-func imenu time-date mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel ns-win term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese charscript
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind cocoa ns
multi-tty make-network-process emacs)

Memory information:
((conses 16 87331 6412)
 (symbols 48 19011 0)
 (miscs 40 49 232)
 (strings 32 16448 5934)
 (string-bytes 1 461013)
 (vectors 16 10826)
 (vector-slots 8 452328 5852)
 (floats 8 156 200)
 (intervals 56 221 6)
 (buffers 976 12))


[-- Attachment #2.1: Type: text/html, Size: 5337 bytes --]

[-- Attachment #2.2: PastedGraphic-6.png --]
[-- Type: image/png, Size: 7270 bytes --]

[-- Attachment #2.3: PastedGraphic-5.png --]
[-- Type: image/png, Size: 8828 bytes --]

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[Eli Zaretskii]

> From: David Reitter <david.reitter@gmail.com>
> Date: Wed, 4 Nov 2015 23:21:13 -0500
> 
> Start Emacs 25 (master branch)  (with -Q if you like)
> 
> (setq line-spacing 1.0)
> 
> (Note how the display doesn’t update, so enter C-l to force it)

It does update for me if I evaluate that with C-j at the right
parenthesis.  How did you evaluate it?

> Result: cursor is displayed with wrong (large, two-line) height when at the end of a line, and correctly when not at the end of a line.

Strange as it might sound, this is not a bug, but intended behavior.
It is a consequence of improvements in display code that avoid
extremely large lines with some fonts (in particular, math fonts) that
declare preposterously large font sizes.  Previously, the line height,
and the height of the cursor at EOL, was determined by the font-global
ascent/descent values.  We now determine the height of the line by
looking at the glyphs we actually display on that line.  See bug#20628
for more details.

The problem with displaying the cursor at the end of the line is that
there's no displayable glyph there, whose dimensions in the font we
could use to determine the cursor height.  So we set the cursor height
to the line height.

> This wasn’t the case in Emacs 24.

Yes, the changes to use the glyph metrics are new in Emacs 25.

> Also note bug#17977

That's unrelated to this issue, AFAICS.  That bug is NS specific,
whereas what you describe happens on all supported GUI frames.

> In any case, the cursor’s height should not change.

When you say "should not change", what do you have in mind?  Should
not change from what value?  If you think that the value of the cursor
height is fixed when it is displayed on other characters of the line,
then this is not so, unfortunately.

Please keep in mind that characters on a line can have different
dimensions, even if they come from the same font, let alone if they
come from different fonts (which happens quite a lot in Emacs, given
today's abundance of rare characters in plain text).  When the cursor
is displayed on a character, it gets its dimensions from that
character (you can see the effect of that if you display a line where
some characters come from a larger font).  But that doesn't work when
the cursor is at the end of a line, where there's no character to
begin with.  So we use the line's height instead.  I agree that the
result looks somewhat ugly, especially when the value of line-spacing
is large.  Suggestions for better defaults are welcome; at the time, I
tried several other ideas, but they all had worse problems.

> If it has to be this high the actual text line should be vertically
> centered.

We cannot do that without changing how line-spacing is applied.  Since
Emacs 21.1, when this feature was added, we always added the spacing
_below_ each line.  The ELisp manual documents this:

     You can specify the line spacing for all lines in a buffer via the
  buffer-local ‘line-spacing’ variable.  An integer specifies the number
  of pixels put below lines.  A floating-point number specifies the
  spacing relative to the default frame line height.

and elsewhere:

  ‘line-spacing’
       Additional space to leave below each text line, in pixels (a
       positive integer).

(Note the "below" parts in both excerpts.)

I personally never use these features, so I don't object to changing
the effect of line-spacing to have it distributes between the ascent
and descent parts somehow.  Not sure what others will say.  In any
case, doing this will require non-trivial changes to display code.

Thanks.

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[David Reitter]

Hi Eli,

On Nov 5, 2015, at 11:12 AM, Eli Zaretskii <eliz@gnu.org> wrote:

>> (setq line-spacing 1.0)
>> 
>> (Note how the display doesn’t update, so enter C-l to force it)
> 
> It does update for me if I evaluate that with C-j at the right
> parenthesis.  How did you evaluate it?

C-x C-e.  
C-j, indeed, updates the display.


> Strange as it might sound, this is not a bug, but intended behavior.
> It is a consequence of improvements in display code that avoid
> extremely large lines with some fonts (in particular, math fonts) that
> declare preposterously large font sizes.  Previously, the line height,
> and the height of the cursor at EOL, was determined by the font-global
> ascent/descent values.  We now determine the height of the line by
> looking at the glyphs we actually display on that line.  See bug#20628
> for more details.

That, in principle, should not be a problem - if you indeed determine a line height rather than a glyph height (at the cursor) and use that for the cursor anywhere in that line.  But...

> The problem with displaying the cursor at the end of the line is that
> there's no displayable glyph there, whose dimensions in the font we
> could use to determine the cursor height.  So we set the cursor height
> to the line height.

That creates an inconsistency, and it looks bad.  

So, you’re asking, what did I have in mind?

One could default to the height of the last glyph of the line. Normally, we would expect the size of the cursor to indicate the general height of the text that would be inserted there, and the last glyph is the best guess.

Alternatively, use the line height for all cursor positions in that line.  In that case, things would still need to be adjusted for `line-spacing’, and based on what you wrote, I think that’s less preferable.

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[Eli Zaretskii]

> From: David Reitter <david.reitter@gmail.com>
> Date: Thu, 5 Nov 2015 22:12:03 -0500
> Cc: 21835@debbugs.gnu.org
> 
> One could default to the height of the last glyph of the line. Normally, we would expect the size of the cursor to indicate the general height of the text that would be inserted there, and the last glyph is the best guess.

I tried that, and it was worse.  The problem is, the last glyph could
have very small height, at least with some fonts.  Then you have a
very small cursor at the end.  What's more, as you type at EOL, the
cursor height will constantly change.

> Alternatively, use the line height for all cursor positions in that line.  In that case, things would still need to be adjusted for `line-spacing’, and based on what you wrote, I think that’s less preferable.

Indeed.  Our cursor, when it is drawn on a character, shows the
dimensions of that character, both horizontal and vertical.  Losing
that, even only for the vertical dimension, would be a step backward,
I think.

There are also differences in how we draw the hollow cursor (in
non-selected windows) vs the cursor in selected windows.  They should
have the same dimensions, or else it will look like a bug.  The
solution I coded solves this problem as well (some other ideas
didn't).

So once again, I don't know of any better way of solving this.

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[Eli Zaretskii]

> From: David Reitter <david.reitter@gmail.com>
> Date: Thu, 5 Nov 2015 22:12:03 -0500
> Cc: 21835@debbugs.gnu.org
> 
> >> (setq line-spacing 1.0)
> >> 
> >> (Note how the display doesn’t update, so enter C-l to force it)
> > 
> > It does update for me if I evaluate that with C-j at the right
> > parenthesis.  How did you evaluate it?
> 
> C-x C-e.  
> C-j, indeed, updates the display.

Thanks.  This was a regression since Emacs 24.4, now fixed.  (C-j
modifies the current buffer, so it caused redisplay of the selected
window, but "C-x C-e" only caused the echo area to be redisplayed.)

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[Stefan Monnier]

>> Result: cursor is displayed with wrong (large, two-line) height when at
>> the end of a line, and correctly when not at the end of a line.
> Strange as it might sound, this is not a bug, but intended behavior.

I think including the line-spacing in the cursor's height is not
"intended".  At least, it seems to be a clear bug (can't think of
a case where it would be a good idea).

Maybe it's not easy to fix it (and/or not super important), of course.


        Stefan

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: David Reitter <david.reitter@gmail.com>,  21835@debbugs.gnu.org
> Date: Fri, 06 Nov 2015 09:10:32 -0500
> 
> >> Result: cursor is displayed with wrong (large, two-line) height when at
> >> the end of a line, and correctly when not at the end of a line.
> > Strange as it might sound, this is not a bug, but intended behavior.
> 
> I think including the line-spacing in the cursor's height is not
> "intended".

Intended by me, I meant.  IOW, that's what the code I wrote was
supposed to do.  Sorry for being unclear.

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[Nick Helm]

I also have set line-spacing in my init.el. I like how the size of the cursor now
reflects the size of the glyph under point - it avoids many of the problems of
badly designed typefaces. However, I also find the changing EOL cursor-height
size between line-body and line-end quite jarring while writing and editing.

Could the EOL cursor-height be set to the height of the tallest glyph in the line?
In Emacs 24, my understanding is that cursor-height = line-height, where
line-height was determined by adding the height of the tallest glyph in the line
to the top and bottom vertical spacing. This is the same, just without the
spacing.

Alternatively, how about setting the EOL cursor-height to the same height as the
last *displayable* glyph in the line, but with a lower size boundary? If there are
no displayable glyphs in the line or the line is empty, could the height of the
closest previous displayable glyph be used, even if it's on an earlier line? If
the buffer is empty or contains no displayable glyphs, use (default line-height -
line-spacing).

If none of that's possible or causes more grief than it's worth, how about also
letting the user arbitrarily set the cursor height? There'd need to be some
boundaries, perhaps a percentage of line-height, but this might also work well for
some people and fonts.

bug#21835: 25.0.50; cursor height wrong when line-spacing is used

[Eli Zaretskii]

> From: Nick Helm <nick@tenpoint.co.nz>
> Date: Mon, 21 Mar 2016 17:03:42 +1300
> 
> Could the EOL cursor-height be set to the height of the tallest glyph in the 
> line?

I don't think your problem is the height of the cursor.  I think your
problem is the vertical position of the lower edge of the cursor
block.  You should keep in mind that a glyph can (and normally does)
rise above the base line of the display line, but it can also descent
below that base line.  So the height alone does not determine the
cursor appearance, unless you also specify the vertical position of
its lower edge.

But to answer your question: we already do that, in a way.  The
display engine records the maximum vertical dimensions of the glyphs
in a display line as it lays out each line.  The EOL cursor simply
uses these maximum values, both the max ascent value and the max
descent value; the latter includes the line spacing.

> In Emacs 24, my understanding is that cursor-height = line-height, where
> line-height was determined by adding the height of the tallest glyph in the line
> to the top and bottom vertical spacing.

No, Emacs 24 didn't use the height of the tallest glyph.  It didn't
even track the height of individual glyphs.  Instead, it used a single
global value of height specified by the font.  Some fonts specify very
large value there, so this was changed to provide a better display.  I
explained this up-thread, see

  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21835#8

> Alternatively, how about setting the EOL cursor-height to the same height as the
> last *displayable* glyph in the line, but with a lower size boundary?

Tried that, it gave much worse results when the last glyph is very
small -- a surprisingly frequent situation, because some fonts have
their SPC glyph specify a small height.  Limitation from below doesn't
work because there are display features, such as the 'line-height'
property, that become broken by such limitations.

And then there are empty lines, of course.

> If there are
> no displayable glyphs in the line or the line is empty, could the height of the
> closest previous displayable glyph be used, even if it's on an earlier line? If
> the buffer is empty or contains no displayable glyphs, use (default line-height 
> -
> line-spacing).

This would slow down redisplay in many important cases, because
redisplay optimizations frequently cause Emacs to lay out a single
display line.  What you suggest would require searching for the last
non-empty line followed by costly layout of that line (to determine
its height) even though that line doesn't need to be redrawn at all.

The case of the empty line is actually one of the main reasons for the
current behavior: we want the cursor on an empty line look the same as
the EOL cursor on a non-empty line, and we want to do that without
examining the glyphs of any other display lines.

The other gotcha is to make sure the hollow cursor, drawn by default
in non-selected windows, will have the same dimensions as the block
cursor on the same character, otherwise switching to a different
window will have unpleasant effect.  These two cursors are drawn using
2 very different methods, so there be dragons.

Anyway, you (or anyone else) are welcome to try different ideas for
making the display of EOL cursor better, the relevant function is
append_space_for_newline in xdisp.c.  Just make sure you cover all the
use cases uncovered by the various bugs referenced in the commits that
changed the code of this function in the past ("git log -L" will show
them).

> If none of that's possible or causes more grief than it's worth, how about also
> letting the user arbitrarily set the cursor height?

I think you can do that already by customizing cursor-type to
'(hbar . N)' where N is some suitable number that produces the cursor
you like.