bug#18195: 24.3.92; window-screen-lines is not accurate

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry]

If I have line-spacing set, the value the above function returns is
smaller than the actuall number of screen lines the window can contain.

For example, when I have the current Emacs frame maximized and
line-spacing set to 0.2, (window-screen-lines) returns 46.66(6), whereas
there are 48 fully-visible lines in the given window and one
partially-visible one.

Here's my default font:

  (set-face-attribute 'default nil :height 105 :family "Fira Mono")

Related:
http://lists.gnu.org/archive/html/help-gnu-emacs/2014-08/msg00026.html

In GNU Emacs 24.3.92.3 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8)
 of 2014-07-24 on axl
Repository revision: 117398 stephen.berman@gmx.net-20140722213204-51v7bp0chfei6wbx
Windowing system distributor `The X.Org Foundation', version 11.0.11501000
System Description:	Ubuntu 14.04.1 LTS

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 > If I have line-spacing set, the value the above function returns is
 > smaller than the actuall number of screen lines the window can contain.
 >
 > For example, when I have the current Emacs frame maximized and
 > line-spacing set to 0.2, (window-screen-lines) returns 46.66(6), whereas
 > there are 48 fully-visible lines in the given window and one
 > partially-visible one.
 >
 > Here's my default font:
 >
 >    (set-face-attribute 'default nil :height 105 :family "Fira Mono")
 >
 > Related:
 > http://lists.gnu.org/archive/html/help-gnu-emacs/2014-08/msg00026.html

IIUC

https://github.com/company-mode/company-mode/issues/160#issuecomment-51022508

mentioned in that thread, all you want is to display all lines of a
completion buffer.  Is that correct?  If so, then why can't/don't you
use `fit-window-to-buffer'?

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/05/2014 12:38 PM, martin rudalics wrote:
> IIUC
>
> https://github.com/company-mode/company-mode/issues/160#issuecomment-51022508
>
> mentioned in that thread, all you want is to display all lines of a
> completion buffer.  Is that correct?

Not at all. Have you tried Company?

This issue is about completion popup, which displays completions in the 
current buffer and window, in a rectangle rendered using an overlay. 
There are screenshots in that issue thread.

Before the popup is rendered, we need to determine how much of it would 
fit below the current line, and if there's not enough space, we render 
it above the current line.

Even if the buffer is shorter than the window, we can temporarily pad 
it, so when calculating the height of the window, we need to take the 
whole of it into account. That's why `count-screen-line' doesn't seem 
like a good approach either.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 > Have you tried Company?

No.

 > This issue is about completion popup, which displays completions in the current buffer and window, in a rectangle rendered using an overlay. There are screenshots in that issue thread.
 >
 > Before the popup is rendered, we need to determine how much of it would fit below the current line, and if there's not enough space, we render it above the current line.

Aha... Could you try whether `window-text-pixel-size' could be used for
that?  IIUC you'd have to call it twice: Once for calculating how much
space you need and once for how much space is left below the current
line.  `window-text-height' with PIXELWISE t should give you the overall
height of the window.

 > Even if the buffer is shorter than the window, we can temporarily pad it, so when calculating the height of the window, we need to take the whole of it into account. That's why `count-screen-line' doesn't seem like a good approach either.

If things like line spacing are involved you really should calculate
pixelwise.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/05/2014 02:16 PM, martin rudalics wrote:

> Aha... Could you try whether `window-text-pixel-size' could be used for
> that?

It won't help to determine how much space is left below the current line 
if the current line is the last in the buffer but not in the window, right?

 > `window-text-height' with PIXELWISE t should give you the overall
 > height of the window.

`count-screen-lines' already helps with "space above the current line", 
and we can find out the total height of the window in pixels. So if I'm 
going to do the arithmetics myself, the one thing missing is the 
accurate value of the height of a line in the buffer.

How do I obtain it (let's consider it mostly-constant)?

> If things like line spacing are involved you really should calculate
> pixelwise.

The docstring of `window-screen-lines' seems to claim that it's 
suitable. The implementation looks kinda right, too, but the result it 
returns is not accurate.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 > It won't help to determine how much space is left below the current line if the current line is the last in the buffer but not in the window, right?

You have to calculate the height of the text from window start till the
end of the buffer and subtract the result from the window's text height.

 >  > `window-text-height' with PIXELWISE t should give you the overall
 >  > height of the window.
 >
 > `count-screen-lines' already helps with "space above the current line", and we can find out the total height of the window in pixels. So if I'm going to do the arithmetics myself, the one thing missing is the accurate value of the height of a line in the buffer.

`count-screen-lines' doesn't return a pixel value.  How do you get that
with lines of different heights and line spacing?

 > How do I obtain it (let's consider it mostly-constant)?
 >
 >> If things like line spacing are involved you really should calculate
 >> pixelwise.
 >
 > The docstring of `window-screen-lines' seems to claim that it's suitable. The implementation looks kinda right, too, but the result it returns is not accurate.

It again just returns a line value.  IIRC someone has set the font size
to 1 pixel to make this return something reasonable.  But why do all
these acrobatics?  `window-text-pixel-size' should give you everything
you need in a uniform way.  And if something's missing we can easily add
it.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/05/2014 04:20 PM, martin rudalics wrote:
>  > It won't help to determine how much space is left below the current
> line if the current line is the last in the buffer but not in the
> window, right?
>
> You have to calculate the height of the text from window start till the
> end of the buffer and subtract the result from the window's text height.

Suppose I do that. When do I convert any of the resulting values to 
number of lines?

Please remember, we're using an overlay for popup rendering. Overlays 
work with lines of text.

> `count-screen-lines' doesn't return a pixel value.  How do you get that
> with lines of different heights and line spacing?

As above, we really don't need a pixel value in the end result. If the 
overlay is made to span over lines with different heights and lines 
spacing, we can either make it follow all those dimensions, or override 
them to the same values.

But while we can do that, ignoring the global value of line-spacing 
wouldn't be comfortable for users that have it set.

> It again just returns a line value.  IIRC someone has set the font size
> to 1 pixel to make this return something reasonable.  But why do all
> these acrobatics?  `window-text-pixel-size' should give you everything
> you need in a uniform way.  And if something's missing we can easily add
> it.

See above.

By the way, `window-screen-lines' not being accurate doesn't seem to be 
directly caused by pixel-perfect positioning. If I copy the necessary 
definitions (the 3 new function) to Emacs 24.3, it exhibits the same 
problem.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 >>  > It won't help to determine how much space is left below the current
 >> line if the current line is the last in the buffer but not in the
 >> window, right?
 >>
 >> You have to calculate the height of the text from window start till the
 >> end of the buffer and subtract the result from the window's text height.
 >
 > Suppose I do that. When do I convert any of the resulting values to number of lines?

IIUC you want to check whether some text of pixel height O would fit
into the rest of a window with pixel height W when the upper part of
that window is occupied by text with pixel height T.  So if O <= W - T
holds, you can put the overlay below the current line, and do something
else otherwise.  Why would you ever want to convert a resulting value to
a number of lines?

 > Please remember, we're using an overlay for popup rendering. Overlays work with lines of text.

Overlays don't know about newlines.  I suppose the overlay starts at the
beginning of some line after text T and has as many characters as there
are in the text with height O.  If you want to cover any text with the
overlay do that.

 >> `count-screen-lines' doesn't return a pixel value.  How do you get that
 >> with lines of different heights and line spacing?
 >
 > As above, we really don't need a pixel value in the end result. If the overlay is made to span over lines with different heights and lines spacing, we can either make it follow all those dimensions, or override them to the same values.
 >
 > But while we can do that, ignoring the global value of line-spacing wouldn't be comfortable for users that have it set.

IIUC you neither need pixels nor lines in the end result.  You just want
to know whether a given overlay fits into a given window at a given
position.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 15:50:34 +0200
> From: martin rudalics <rudalics@gmx.at>
> 
> IIUC you want to check whether some text of pixel height O would fit
> into the rest of a window with pixel height W when the upper part of
> that window is occupied by text with pixel height T.

There's currently no way to determine the pixel height of arbitrary
text string from Lisp, without actually displaying it in some window.
So there's no way to know the value of O, except in some simple
scenarios.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/05/2014 05:50 PM, martin rudalics wrote:

> Overlays don't know about newlines.

We're using a multiline overlay: it both spans several lines, and its 
`before-string' property contains a multiline string when it's "on".

> I suppose the overlay starts at the
> beginning of some line after text T and has as many characters as there
> are in the text with height O.  If you want to cover any text with the
> overlay do that.

Sorry, I don't understand.

> IIUC you neither need pixels nor lines in the end result.  You just want
> to know whether a given overlay fits into a given window at a given
> position.

We don't have the overlay at that point in time yet: when it's created, 
it's constructed using the underlying buffer text, too.

Being able to just show a rectangle would require some dedicated API, 
like Eli mentioned.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 >> Overlays don't know about newlines.
 >
 > We're using a multiline overlay: it both spans several lines, and its `before-string' property contains a multiline string when it's "on".

Still, an Emacs overlay is a display object that replaces a possibly
empty buffer string with a possibly empty string.  Any of these strings
can contain newlines but the overlay is specified only by the buffer
positions and the replacement string.

 >> IIUC you neither need pixels nor lines in the end result.  You just want
 >> to know whether a given overlay fits into a given window at a given
 >> position.
 >
 > We don't have the overlay at that point in time yet: when it's created, it's constructed using the underlying buffer text, too.

But you have the underlying buffer text at the time you create the
overlay I suppose.  So what hinders you to get the size of the overlay
by using a temporary buffer, inserting the underlying buffer text, put
the overlay there, and calculate the resulting height?

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 17:05:20 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> If the overlay is made to span over lines with different heights and
> lines spacing, we can either make it follow all those dimensions, or
> override them to the same values.

No, you can't do that.  Unless an overlay entirely obscures a screen
line, it can never make the line height smaller, only larger.

Besides, even if you could do what you hint here, that would cause
unpleasant effects on display, whereby the text would jump up and down
when the overlay is popped up/down.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 14:20:58 +0200
> From: martin rudalics <rudalics@gmx.at>
> 
>  > The docstring of `window-screen-lines' seems to claim that it's suitable. The implementation looks kinda right, too, but the result it returns is not accurate.
> 
> It again just returns a line value.

But that value is a floating-point number, so accuracy is not lost.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

> But that value is a floating-point number, so accuracy is not lost.

Text displayed may be of completely different height.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 16:08:56 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
> > But that value is a floating-point number, so accuracy is not lost.
> 
> Text displayed may be of completely different height.

Of course.  But that's of no importance for window-screen-lines,
because its contract is to count lines in units of the height of the
font used by the default face.

The fact that the metrics of the default face is used is the reason
why I wrote that this is a slippery slope, see my other message.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 14:24:59 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> `count-screen-lines' already helps with "space above the current line", 
> and we can find out the total height of the window in pixels.

How do you convert the line count returned by count-screen-lines into
pixels, if the lines are of non-constant height.

Also note the caveats in the doc string of that function: the value is
not 100% reliable.

> So if I'm going to do the arithmetics myself, the one thing missing
> is the accurate value of the height of a line in the buffer.

Line of what font?

> The docstring of `window-screen-lines' seems to claim that it's 
> suitable. The implementation looks kinda right, too, but the result it 
> returns is not accurate.

It is accurate now.  But, as I wrote elsewhere in this thread, that
will only solve some of your problems.  In general, we don't support
layout of arbitrary text from Lisp.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 12:16:40 +0200
> From: martin rudalics <rudalics@gmx.at>
> 
> If things like line spacing are involved you really should calculate
> pixelwise.

He can't: we currently don't expose enough information for Lisp
programs to perform layout.  E.g., the metrics of glyphs and lines
that use arbitrary fonts and display images are not easily available
on the Lisp level, and neither are dimensions and line counts of
arbitrary regions of the screen (although you could write some clunky
Lisp to do the latter).  Likewise with dimensions of text in overlays
and display properties, and with images.

This was never supposed to be needed, and thus was never designed and
implemented.  If we want to allow such jobs from Lisp, we need to
close these gaps.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 > He can't: we currently don't expose enough information for Lisp
 > programs to perform layout.  E.g., the metrics of glyphs and lines
 > that use arbitrary fonts and display images are not easily available
 > on the Lisp level, and neither are dimensions and line counts of
 > arbitrary regions of the screen (although you could write some clunky
 > Lisp to do the latter).  Likewise with dimensions of text in overlays
 > and display properties, and with images.
 >
 > This was never supposed to be needed, and thus was never designed and
 > implemented.  If we want to allow such jobs from Lisp, we need to
 > close these gaps.

Why would `window-text-pixel-size' fail to provide that?  It might be a
bit clumsy to work with because you have to temporarily show another
buffer in its window but apart from that I see no problems ...

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 15:57:57 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  > He can't: we currently don't expose enough information for Lisp
>  > programs to perform layout.  E.g., the metrics of glyphs and lines
>  > that use arbitrary fonts and display images are not easily available
>  > on the Lisp level, and neither are dimensions and line counts of
>  > arbitrary regions of the screen (although you could write some clunky
>  > Lisp to do the latter).  Likewise with dimensions of text in overlays
>  > and display properties, and with images.
>  >
>  > This was never supposed to be needed, and thus was never designed and
>  > implemented.  If we want to allow such jobs from Lisp, we need to
>  > close these gaps.
> 
> Why would `window-text-pixel-size' fail to provide that?

Because there's no way of calculating the pixel size of arbitrary text
(in this case: the list of completion candidates to be popped up
overlaid on the buffer text), except if that text is displayed in some
window.

> It might be a bit clumsy to work with because you have to
> temporarily show another buffer in its window

That _is_ the problem.  Momentarily flashing some text in some window
is not my idea of a good application.  It simply looks like a bug.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 >> It might be a bit clumsy to work with because you have to
 >> temporarily show another buffer in its window
 >
 > That _is_ the problem.  Momentarily flashing some text in some window
 > is not my idea of a good application.  It simply looks like a bug.

Well, nobody asked for such a feature so far.  Can you fix it so it
simply works on the current buffer instead of the window's buffer?

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 16:12:53 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  >> It might be a bit clumsy to work with because you have to
>  >> temporarily show another buffer in its window
>  >
>  > That _is_ the problem.  Momentarily flashing some text in some window
>  > is not my idea of a good application.  It simply looks like a bug.
> 
> Well, nobody asked for such a feature so far.  Can you fix it so it
> simply works on the current buffer instead of the window's buffer?

Fix what exactly?  What is "it" in this context?

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 >>   >> It might be a bit clumsy to work with because you have to
 >>   >> temporarily show another buffer in its window
 >>   >
 >>   > That _is_ the problem.  Momentarily flashing some text in some window
 >>   > is not my idea of a good application.  It simply looks like a bug.
 >>
 >> Well, nobody asked for such a feature so far.  Can you fix it so it
 >> simply works on the current buffer instead of the window's buffer?
 >
 > Fix what exactly?  What is "it" in this context?

`window-text-pixel-size'.  Never mind.  I'll find a fix.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 16:38:36 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  >> Can you fix it so it simply works on the current buffer instead
>  >> of the window's buffer?
>  >
>  > Fix what exactly?  What is "it" in this context?
> 
> `window-text-pixel-size'.  Never mind.  I'll find a fix.

You'll need to designate a window anyway, since layout is impossible
without knowing in which window it will be done.

There's also the issue of the buffer text on which the text will be
overlaid: the dimensions of the screen lines to be (partially)
overlaid affect the pixel dimensions of the overlaying text.  So if
the buffer text uses different fonts, the text overlaid on it will
have different dimensions than the same text when displayed in its own
buffer.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 4267 bytes --]

 > You'll need to designate a window anyway, since layout is impossible
 > without knowing in which window it will be done.

`window-text-pixel-size' works on any life window, by default the
selected.

 > There's also the issue of the buffer text on which the text will be
 > overlaid: the dimensions of the screen lines to be (partially)
 > overlaid affect the pixel dimensions of the overlaying text.  So if
 > the buffer text uses different fonts, the text overlaid on it will
 > have different dimensions than the same text when displayed in its own
 > buffer.

This would have to be done on the Lisp level by selecting some font
before calculating the size of the overlay.  But I have problems with
the attached patch.  If I invoke `window-text-pixel-size' with a
different buffer than that displayed in the window I get an arguments
out of range error.  Any ideas?

martin

A backtrace goes as follows:

#0  args_out_of_range (a1=..., a2=...) at data.c:217
#1  0x011eadda in validate_interval_range (object=..., begin=0x82b310, end=0x82b310, force=false) at textprop.c:182
#2  0x011ec8a4 in Ftext_properties_at (position=..., object=...) at textprop.c:601
#3  0x011ec931 in Fget_text_property (position=..., prop=..., object=...) at textprop.c:621
#4  0x010f5806 in face_at_buffer_position (w=0x17b8bf8, pos=82, endptr=0x82b438, limit=182, mouse=0, base_face_id=0) at xfaces.c:5976
#5  0x0102eb20 in handle_face_prop (it=0x82e180) at xdisp.c:3923
#6  0x0102db70 in handle_stop (it=0x82e180) at xdisp.c:3427
#7  0x0103a40c in next_element_from_buffer (it=0x82e180) at xdisp.c:8242
#8  0x01036f16 in get_next_display_element (it=0x82e180) at xdisp.c:6874
#9  0x0103aec6 in move_it_in_display_line_to (it=0x82e180, to_charpos=311, to_x=-1, op=MOVE_TO_POS) at xdisp.c:8582
#10 0x0103d080 in move_it_to (it=0x82e180, to_charpos=311, to_x=-1, to_y=-1, to_vpos=-1, op=10) at xdisp.c:9149
#11 0x0103ec0d in Fwindow_text_pixel_size (window=..., from=..., to=..., x_limit=..., y_limit=..., mode_and_header_line=..., buffer=...) at xdisp.c:9833
#12 0x0118d7f5 in eval_sub (form=...) at eval.c:2202
#13 0x0118cf36 in Feval (form=..., lexical=...) at eval.c:1996
#14 0x0118e9e6 in Ffuncall (nargs=3, args=0x82ebdc) at eval.c:2811
#15 0x011d01d6 in exec_byte_code (bytestr=..., vector=..., maxdepth=..., args_template=..., nargs=1, args=0x82ef10) at bytecode.c:918
#16 0x0118f18c in funcall_lambda (fun=..., nargs=1, arg_vector=0x82ef0c) at eval.c:2976
#17 0x0118ebde in Ffuncall (nargs=2, args=0x82ef08) at eval.c:2857
#18 0x011d01d6 in exec_byte_code (bytestr=..., vector=..., maxdepth=..., args_template=..., nargs=1, args=0x82f33c) at bytecode.c:918
#19 0x0118f18c in funcall_lambda (fun=..., nargs=1, arg_vector=0x82f338) at eval.c:2976
#20 0x0118ebde in Ffuncall (nargs=2, args=0x82f334) at eval.c:2857
#21 0x01186548 in Ffuncall_interactively (nargs=2, args=0x82f334) at callint.c:270
#22 0x0118e8ac in Ffuncall (nargs=3, args=0x82f330) at eval.c:2789
#23 0x01188170 in Fcall_interactively (function=..., record_flag=..., keys=...) at callint.c:872
#24 0x0118ea15 in Ffuncall (nargs=4, args=0x82f54c) at eval.c:2815
#25 0x011d01d6 in exec_byte_code (bytestr=..., vector=..., maxdepth=..., args_template=..., nargs=1, args=0x82f894) at bytecode.c:918
#26 0x0118f18c in funcall_lambda (fun=..., nargs=1, arg_vector=0x82f890) at eval.c:2976
#27 0x0118ebde in Ffuncall (nargs=2, args=0x82f88c) at eval.c:2857
#28 0x0118e4a5 in call1 (fn=..., arg1=...) at eval.c:2607
#29 0x010ffcee in command_loop_1 () at keyboard.c:1545
#30 0x0118b92a in internal_condition_case (bfun=0x10ff67c <command_loop_1>, handlers=..., hfun=0x10feee7 <cmd_error>) at eval.c:1347
#31 0x010ff332 in command_loop_2 (ignore=...) at keyboard.c:1169
#32 0x0118aede in internal_catch (tag=..., func=0x10ff30e <command_loop_2>, arg=...) at eval.c:1111
#33 0x010ff2ec in command_loop () at keyboard.c:1148
#34 0x010fea83 in recursive_edit_1 () at keyboard.c:769
#35 0x010fec40 in Frecursive_edit () at keyboard.c:840
#36 0x010fcdaf in main (argc=1, argv=0xa327e8) at emacs.c:1650

Lisp Backtrace:
"window-text-pixel-size" (0x82ea80)
"eval" (0x82ebe0)
"eval-last-sexp-1" (0x82ef0c)
"eval-last-sexp" (0x82f338)
"funcall-interactively" (0x82f334)
"call-interactively" (0x82f550)
"command-execute" (0x82f890)

[-- Attachment #2: window-text-pixel-size.diff --]
[-- Type: text/plain, Size: 2497 bytes --]

=== modified file 'src/xdisp.c'
--- src/xdisp.c	2014-08-01 13:10:07 +0000
+++ src/xdisp.c	2014-08-05 15:28:23 +0000
@@ -9713,7 +9713,7 @@
 	  && it->dpvec + it->current.dpvec_index != it->dpend);
 }

-DEFUN ("window-text-pixel-size", Fwindow_text_pixel_size, Swindow_text_pixel_size, 0, 6, 0,
+DEFUN ("window-text-pixel-size", Fwindow_text_pixel_size, Swindow_text_pixel_size, 0, 7, 0,
        doc: /* Return the size of the text of WINDOW's buffer in pixels.
 WINDOW must be a live window and defaults to the selected one.  The
 return value is a cons of the maximum pixel-width of any text line and
@@ -9746,9 +9746,13 @@
 include the height of the mode- or header-line of WINDOW in the return
 value.  If it is either the symbol `mode-line' or `header-line', include
 only the height of that line, if present, in the return value.  If t,
-include the height of both, if present, in the return value.  */)
+include the height of both, if present, in the return value.
+
+Optional argument BUFFER, if non-nil, means to work on another buffer
+than that displayed in WINDOW.  If BUFFER denotes a live buffer, use
+that buffer.  If BUFFER equals t, use the current buffer.  */)
   (Lisp_Object window, Lisp_Object from, Lisp_Object to, Lisp_Object x_limit, Lisp_Object y_limit,
-   Lisp_Object mode_and_header_line)
+   Lisp_Object mode_and_header_line, Lisp_Object buffer)
 {
   struct window *w = decode_live_window (window);
   Lisp_Object buf;
@@ -9760,14 +9764,25 @@
   void *itdata = NULL;
   int c, max_y = -1, x = 0, y = 0;

-  buf = w->contents;
-  CHECK_BUFFER (buf);
-  b = XBUFFER (buf);
-
-  if (b != current_buffer)
+  if (!EQ (buffer, Qt))
     {
-      old_buffer = current_buffer;
-      set_buffer_internal (b);
+      if (NILP (buffer))
+	{
+	  buf = w->contents;
+	  CHECK_BUFFER (buf);
+	  b = XBUFFER (buf);
+	}
+      else
+	{
+	  CHECK_BUFFER (buffer);
+	  b = XBUFFER (buffer);
+	}
+
+      if (b != current_buffer)
+	{
+	  old_buffer = current_buffer;
+	  set_buffer_internal (b);
+	}
     }

   if (NILP (from))
@@ -15814,6 +15829,17 @@
 	 left.  */
       whole = max (whole, w->hscroll + box_width);
       whole = max (whole, end - start);
+      if (it.bidi_p)
+	{
+	  Lisp_Object pdir;
+
+	  pdir = Fcurrent_bidi_paragraph_direction (Qnil);
+	  if (EQ (pdir, Qright_to_left))
+	    {
+	      end = whole - w->hscroll * FRAME_COLUMN_WIDTH (WINDOW_XFRAME (w));
+	      start = end - box_width;
+	    }
+	}

       if (old_buffer)
 	set_buffer_internal (old_buffer);

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 17:45:25 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  > There's also the issue of the buffer text on which the text will be
>  > overlaid: the dimensions of the screen lines to be (partially)
>  > overlaid affect the pixel dimensions of the overlaying text.  So if
>  > the buffer text uses different fonts, the text overlaid on it will
>  > have different dimensions than the same text when displayed in its own
>  > buffer.
> 
> This would have to be done on the Lisp level by selecting some font
> before calculating the size of the overlay.

Which font would you select?  I'm talking about a situation where the
text over which the candidate list will be popped uses several
different fonts, and so each line of the buffer text has a different
height.

> But I have problems with the attached patch.  If I invoke
> `window-text-pixel-size' with a different buffer than that displayed
> in the window I get an arguments out of range error.  Any ideas?

You switched the current buffer, but not the buffer pointed to by
w->contents.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 >> This would have to be done on the Lisp level by selecting some font
 >> before calculating the size of the overlay.
 >
 > Which font would you select?  I'm talking about a situation where the
 > text over which the candidate list will be popped uses several
 > different fonts, and so each line of the buffer text has a different
 > height.

I would pop up that list in a temporary buffer and run
`window-text-pixel-size' over it.

 >> But I have problems with the attached patch.  If I invoke
 >> `window-text-pixel-size' with a different buffer than that displayed
 >> in the window I get an arguments out of range error.  Any ideas?
 >
 > You switched the current buffer, but not the buffer pointed to by
 > w->contents.

That was the problem, indeed.

Thanks, martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 11:41:26 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
> >> This would have to be done on the Lisp level by selecting some font
> >> before calculating the size of the overlay.
> >
> > Which font would you select?

You didn't answer this question.  The only trivial answer is the font
of the default face, a choice which has the problems described below.

> > I'm talking about a situation where the text over which the
> > candidate list will be popped uses several different fonts, and so
> > each line of the buffer text has a different height.
> 
> I would pop up that list in a temporary buffer and run
> `window-text-pixel-size' over it.

And therein lies the problem.  You will get the size of the text under
the assumption that the text will be displayed using a single font,
the one used by the frame's 'default' face.  But if the text in the
region over which you will pop the candidate list uses a different
font, or shows images on some of the lines, the pixel height needed to
display the popped list will be different.  That's because the
candidates are displayed by Company by putting overlays on consecutive
lines of text, each overlay showing one candidate.  However,
displaying such overlays in the middle of a line of buffer text cannot
decrease the height of the line, it can only increase it.  Therefore,
if the line of buffer text was taller than what is required for the
default face's font, the displayed overlay string will take up more
pixels than what you measured in the temporary buffer.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/06/2014 06:39 PM, Eli Zaretskii wrote:

> That's because the
> candidates are displayed by Company by putting overlays on consecutive
> lines of text, each overlay showing one candidate.

Not really. We have a pull request that changes the code to do what 
you're describing, but for the moment we only use one overlay at a time.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 18:41:21 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: 18195@debbugs.gnu.org
> 
> On 08/06/2014 06:39 PM, Eli Zaretskii wrote:
> 
> > That's because the
> > candidates are displayed by Company by putting overlays on consecutive
> > lines of text, each overlay showing one candidate.
> 
> Not really. We have a pull request that changes the code to do what 
> you're describing, but for the moment we only use one overlay at a time.

I'm talking about a kind of menu of candidates, which definitely takes
more than one line.  That cannot be done with a single overlay, unless
you copy the buffer text into the overlay string, and completely cover
the buffer text itself.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/06/2014 08:23 PM, Eli Zaretskii wrote:

>> Not really. We have a pull request that changes the code to do what
>> you're describing, but for the moment we only use one overlay at a time.
>
> I'm talking about a kind of menu of candidates, which definitely takes
> more than one line.  That cannot be done with a single overlay, unless
> you copy the buffer text into the overlay string, and completely cover
> the buffer text itself.

Indeed, that's what we do currently.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 20:26:44 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 18195@debbugs.gnu.org
> 
> On 08/06/2014 08:23 PM, Eli Zaretskii wrote:
> 
> >> Not really. We have a pull request that changes the code to do what
> >> you're describing, but for the moment we only use one overlay at a time.
> >
> > I'm talking about a kind of menu of candidates, which definitely takes
> > more than one line.  That cannot be done with a single overlay, unless
> > you copy the buffer text into the overlay string, and completely cover
> > the buffer text itself.
> 
> Indeed, that's what we do currently.

But that will break if the buffer text has other display properties,
because Emacs doesn't support display strings on display strings.
IOW, this solution only works for relatively simple text in the
buffer.

And what do you do with line-prefix and wrap-prefix?

That's really crazy design (no offense)!

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/06/2014 09:24 PM, Eli Zaretskii wrote:

> But that will break if the buffer text has other display properties,
> because Emacs doesn't support display strings on display strings.
> IOW, this solution only works for relatively simple text in the
> buffer.

Multiple overlays aren't much easier to work with: you can't position 
them in the middle of a piece of text that's on `display' property, 
can't position them after the end of the buffer or overlapping the 
`line-prefix' value. And if the desired column is in the middle of a 
multiple-width character like \t, you're forced to do unnatural things 
with it anyway.

> And what do you do with line-prefix and wrap-prefix?

The former is incorporated into the "replaced" text and set to "" on the 
overlay; the latter, so far, has been ignored.

> That's really crazy design (no offense)!

I didn't design it personally, just fixed some corner cases. But it has 
its advantages, see above. For example, otherwise I'm having hard time 
imagining what to do if the target column is inside the line-prefix.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 22:02:51 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 18195@debbugs.gnu.org
> 
> On 08/06/2014 09:24 PM, Eli Zaretskii wrote:
> 
> > But that will break if the buffer text has other display properties,
> > because Emacs doesn't support display strings on display strings.
> > IOW, this solution only works for relatively simple text in the
> > buffer.
> 
> Multiple overlays aren't much easier to work with: you can't position 
> them in the middle of a piece of text that's on `display' property, 

But you could copy the display string into the overlay.

> can't position them after the end of the buffer

Yes, you can: you can include newlines in the overlay string.

> or overlapping the `line-prefix' value.

Why do you need that?  No buffer text will ever appear in the prefix,
so your drop-down list doesn't need to invade the prefix area, because
it is aligned with some buffer text.

> And if the desired column is in the middle of a multiple-width
> character like \t, you're forced to do unnatural things with it
> anyway.

Yes, which is why I asked why not use tooltips and text-mode menus
instead.  AFAICT, they solve all those problems seamlessly.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/06/2014 10:36 PM, Eli Zaretskii wrote:

>> Multiple overlays aren't much easier to work with: you can't position
>> them in the middle of a piece of text that's on `display' property,
>
> But you could copy the display string into the overlay.

I don't see how this could work with overlays that don't spread over 
multiple visual lines if the aforementioned text on `display' property 
is multiline.

Going back to one-overlay approach, this could work. But then, oh joy, 
instead of "go down 10 lines, collect them and add required pieces at 
between columns 5 and 20", I'll have to see if there's maybe a display 
property on line 3 that spans 15 lines, stop after it, include the 
display property in the text-to-be-replaced. Or, if the text with the 
`display' property starts before the supposed beginning of the overlay, 
I'll have to measure the place where to start replacing the lines 
from... somewhere. If there are multiple pieces of text with `display', 
all the merrier.

All of the above really sounds like a job for a display engine.

>> can't position them after the end of the buffer
>
> Yes, you can: you can include newlines in the overlay string.

Guess so. If another package tries the same with its own overlays, it'll 
be a mess, but here I'm going over into fantasy territory.

>> or overlapping the `line-prefix' value.
>
> Why do you need that?  No buffer text will ever appear in the prefix,
> so your drop-down list doesn't need to invade the prefix area, because
> it is aligned with some buffer text.

The prefixes can be different between lines. They also can be of 
different lengths. `org-indent-mode' is fairly popular, and it indents 
headings of different levels and their contents using `line-prefix'. 
This way, if the completion is initiated on the first column before the 
first header, the popup has to cover line-prefixes below. See 
https://github.com/company-mode/company-mode/issues/24.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 >>>> This would have to be done on the Lisp level by selecting some font
 >>>> before calculating the size of the overlay.
 >>>
 >>> Which font would you select?
 >
 > You didn't answer this question.  The only trivial answer is the font
 > of the default face, a choice which has the problems described below.

I probably did no understand it.  I do not want to select any font.  The
application has to know which font the overlay should appear in.  If its
the font of the buffer where the overlay finally appears, it should take
this.  If it wants to use any other font, it should take that.

 >>> I'm talking about a situation where the text over which the
 >>> candidate list will be popped uses several different fonts, and so
 >>> each line of the buffer text has a different height.
 >>
 >> I would pop up that list in a temporary buffer and run
 >> `window-text-pixel-size' over it.
 >
 > And therein lies the problem.  You will get the size of the text under
 > the assumption that the text will be displayed using a single font,
 > the one used by the frame's 'default' face.  But if the text in the
 > region over which you will pop the candidate list uses a different
 > font, or shows images on some of the lines, the pixel height needed to
 > display the popped list will be different.  That's because the
 > candidates are displayed by Company by putting overlays on consecutive
 > lines of text, each overlay showing one candidate.  However,
 > displaying such overlays in the middle of a line of buffer text cannot
 > decrease the height of the line, it can only increase it.  Therefore,
 > if the line of buffer text was taller than what is required for the
 > default face's font, the displayed overlay string will take up more
 > pixels than what you measured in the temporary buffer.

But as I explained in my mail to Dmitry this is the problem of the
application.  Whatever it wants to do in the target window it has to
emulate in the temporary buffer: Put the original text there, apply any
font or property settings from the original buffer to that buffer, put
any overlays there, apply any font or property settings for the overlays
there, and have move_it_to get the overall height.  What am I missing?

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 18:15:00 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  >>>> This would have to be done on the Lisp level by selecting some font
>  >>>> before calculating the size of the overlay.
>  >>>
>  >>> Which font would you select?
>  >
>  > You didn't answer this question.  The only trivial answer is the font
>  > of the default face, a choice which has the problems described below.
> 
> I probably did no understand it.  I do not want to select any font.  The
> application has to know which font the overlay should appear in.

If the popped up text overlays existing buffer display, it should
ideally use the same fonts as the overlaid text uses.  But if that
overlaid text uses several different fonts, shadowing that in the
overlay is all but trivial.

> If its the font of the buffer where the overlay finally appears, it
> should take this.  If it wants to use any other font, it should take
> that.

I'm talking about "fonts", plural.  When the buffer text uses several
different fonts, selection of font for the overlay is complicated at
best.

>  >> I would pop up that list in a temporary buffer and run
>  >> `window-text-pixel-size' over it.
>  >
>  > And therein lies the problem.  You will get the size of the text under
>  > the assumption that the text will be displayed using a single font,
>  > the one used by the frame's 'default' face.  But if the text in the
>  > region over which you will pop the candidate list uses a different
>  > font, or shows images on some of the lines, the pixel height needed to
>  > display the popped list will be different.  That's because the
>  > candidates are displayed by Company by putting overlays on consecutive
>  > lines of text, each overlay showing one candidate.  However,
>  > displaying such overlays in the middle of a line of buffer text cannot
>  > decrease the height of the line, it can only increase it.  Therefore,
>  > if the line of buffer text was taller than what is required for the
>  > default face's font, the displayed overlay string will take up more
>  > pixels than what you measured in the temporary buffer.
> 
> But as I explained in my mail to Dmitry this is the problem of the
> application.  Whatever it wants to do in the target window it has to
> emulate in the temporary buffer: Put the original text there, apply any
> font or property settings from the original buffer to that buffer, put
> any overlays there, apply any font or property settings for the overlays
> there, and have move_it_to get the overall height.  What am I missing?

You are talking about things that are possible only on the C level,
not from Lisp.  My point was that we don't expose enough APIs to Lisp
to allow such applications, and you now agreed with me.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 > You are talking about things that are possible only on the C level,
 > not from Lisp.  My point was that we don't expose enough APIs to Lisp
 > to allow such applications, and you now agreed with me.

I was over-complicating things.  The solution is much simpler and all
tools are already on board.  Dmitry just has to insert his overlays
where he wants them and run `window-text-pixel-size' over that window.
If the text including the overlays fits he's done.  Otherwise, he has to
remove the overlays and proceed with whatever alternate solution he has.
No hands needed.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 19:46:17 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  > You are talking about things that are possible only on the C level,
>  > not from Lisp.  My point was that we don't expose enough APIs to Lisp
>  > to allow such applications, and you now agreed with me.
> 
> I was over-complicating things.  The solution is much simpler and all
> tools are already on board.  Dmitry just has to insert his overlays
> where he wants them and run `window-text-pixel-size' over that window.

I obviously fail to explain to you why this will not work in general.
Never mind.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 > I obviously fail to explain to you why this will not work in general.
 > Never mind.

If by "in general" you mean that he won't be able to align his overlays
properly with varying font sizes, prefixes, line warping and the like if
he continues to use the original text, I obviously agree.

My point is that he can have a large overlay replace the original text
with the same text minus any unwanted side-effects introduced by text
properties, fonts, or other overlays.  Within that overlay he can then
additioanlly show whatever his original overlays were supposed to show.

Whether this is nice is a different issue.  But since he wants to draw
some text arbitrarily over some other text anyway, this sounds like a
neglible inconvenience to me.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Thu, 07 Aug 2014 17:08:36 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  > I obviously fail to explain to you why this will not work in general.
>  > Never mind.
> 
> If by "in general" you mean that he won't be able to align his overlays
> properly with varying font sizes, prefixes, line warping and the like if
> he continues to use the original text, I obviously agree.

No, I mean that it is generally unreliable to measure the pixel
dimensions of some text in one buffer, then assume that the dimensions
will be the same when displayed as overlay in another buffer.  This is
only true when both buffers use the same single font and the same line
spacing; otherwise, it's simply false.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 > No, I mean that it is generally unreliable to measure the pixel
 > dimensions of some text in one buffer, then assume that the dimensions
 > will be the same when displayed as overlay in another buffer.  This is
 > only true when both buffers use the same single font and the same line
 > spacing; otherwise, it's simply false.

There is no other buffer.  Everything is done in the buffer where the
overlay is supposed to appear.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Thu, 07 Aug 2014 17:39:18 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  > No, I mean that it is generally unreliable to measure the pixel
>  > dimensions of some text in one buffer, then assume that the dimensions
>  > will be the same when displayed as overlay in another buffer.  This is
>  > only true when both buffers use the same single font and the same line
>  > spacing; otherwise, it's simply false.
> 
> There is no other buffer.  Everything is done in the buffer where the
> overlay is supposed to appear.

How do you prevent the overlay from momentarily flashing in the
window?

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 >> There is no other buffer.  Everything is done in the buffer where the
 >> overlay is supposed to appear.
 >
 > How do you prevent the overlay from momentarily flashing in the
 > window?

I'd instantaneously file a bug report.  Seriously, why should

(let ((overlay (make-overlay (point-min) (point-max))))
   (overlay-put overlay 'display "???????")
   (prog1
       (window-text-pixel-size)
     (delete-overlay overlay)))

cause a "momentarily flashing in the window"?

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Fri, 08 Aug 2014 10:41:13 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
> Seriously, why should
> 
> (let ((overlay (make-overlay (point-min) (point-max))))
>    (overlay-put overlay 'display "???????")
>    (prog1
>        (window-text-pixel-size)
>      (delete-overlay overlay)))
> 
> cause a "momentarily flashing in the window"?

It shouldn't.  But AFAIU, this is not what is wanted here, because the
problem is where in the buffer to put the overlay.  In the above
example, you put it at point-min, but that position needs to be
_computed_ based on the pixel-size of the text to be put in the
overlay.

IOW, your suggested solution is incomplete, unless you intend the
application to do some iterative process, each time changing the
position where it puts the overlay and adjusting it in the next
iteration based on the results of window-text-pixel-size.

bug#18195: 24.3.92; window-screen-lines is not accurate

[martin rudalics]

 > It shouldn't.  But AFAIU, this is not what is wanted here, because the
 > problem is where in the buffer to put the overlay.  In the above
 > example, you put it at point-min, but that position needs to be
 > _computed_ based on the pixel-size of the text to be put in the
 > overlay.

I'd put the overlay at the desired position right away and check whether
it fits (usually it should).  If it doesn't, I'd have to remove it and
put it somewhere else.  I can't imagine a solution where multiple
attempts could be avoided.  After all, one prior aim seems to be that a
certain line of the original text must not be covered by the overlay.
Hence, neither aligning the overlay at the bottom or top line of a
window seem to work always.

 > IOW, your suggested solution is incomplete, unless you intend the
 > application to do some iterative process, each time changing the
 > position where it puts the overlay and adjusting it in the next
 > iteration based on the results of window-text-pixel-size.

This precisley formulates what the application should do and cannot be
avoided IMHO.  After all, tooltips and menus seem to face exactly the
same problem when the frame is at the bottom of the display.  Otherwise,
these are better off since they can are not constrained by the edges of
the window.

martin

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Fri, 08 Aug 2014 12:10:44 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 18195@debbugs.gnu.org
> 
>  > IOW, your suggested solution is incomplete, unless you intend the
>  > application to do some iterative process, each time changing the
>  > position where it puts the overlay and adjusting it in the next
>  > iteration based on the results of window-text-pixel-size.
> 
> This precisley formulates what the application should do and cannot be
> avoided IMHO.

We should provide an API that allows applications to avoid that.

> After all, tooltips and menus seem to face exactly the same problem
> when the frame is at the bottom of the display.

No, we never do any iterations or multiple trials when displaying
menus and tooltips.  We compute the required space and then compare it
with what is available to make the decision.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Tue, 05 Aug 2014 14:03:37 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 08/05/2014 12:38 PM, martin rudalics wrote:
> > IIUC
> >
> > https://github.com/company-mode/company-mode/issues/160#issuecomment-51022508
> >
> > mentioned in that thread, all you want is to display all lines of a
> > completion buffer.  Is that correct?
> 
> Not at all. Have you tried Company?
> 
> This issue is about completion popup, which displays completions in the 
> current buffer and window, in a rectangle rendered using an overlay. 
> There are screenshots in that issue thread.
> 
> Before the popup is rendered, we need to determine how much of it would 
> fit below the current line, and if there's not enough space, we render 
> it above the current line.

I fixed window-screen-lines.

However, please note that you are on a slippery slope doing this.
Emacs currently doesn't provide Lisp interfaces that would allow to
perform such layout tasks from Lisp with reliable results.  Overlays
and text properties were never supposed to be used for layout
purposes.

For example, AFAIU, your code currently assumes that the font used for
popup is the same as the one used for the underlying buffer text.  But
since you provide faces for the popup, a user could legitimately
customize those faces to use a different font, and thus invalidate
your calculations, because window-screen-lines uses the metrics of the
default face's font.

And that is just an example of what could go wrong when you try to do
what in Emacs can only be done reliably by the display engine.

IMO, instead of overloading existing display features with jobs they
were never designed to do, and then live forever with the situation
where development breaks the resulting applications (like the
pixel-wise resizing of windows did with this one), it would be a much
better investment of energy to come up with requirements for Lisp APIs
that could be used by applications for laying out text.  These
requirements, when implemented, would then benefit everybody, and
allow solid dependable applications that don't break that often.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/05/2014 05:46 PM, Eli Zaretskii wrote:

> I fixed window-screen-lines.

Thanks!

> For example, AFAIU, your code currently assumes that the font used for
> popup is the same as the one used for the underlying buffer text.  But
> since you provide faces for the popup, a user could legitimately
> customize those faces to use a different font, and thus invalidate
> your calculations, because window-screen-lines uses the metrics of the
> default face's font.

Sure. This doesn't add any constraints over what we've been living with 
for years.

> IMO, instead of overloading existing display features with jobs they
> were never designed to do, and then live forever with the situation
> where development breaks the resulting applications (like the
> pixel-wise resizing of windows did with this one)

pixel-wise resizing wasn't actually the culprit: I've just never tested 
the code with non-zero line-spacing. Even with it, the bug wasn't too 
easy to notice.

 > it would be a much
> better investment of energy to come up with requirements for Lisp APIs
> that could be used by applications for laying out text.

That sounds great, but I don't even know where to start. What we're 
trying to emulate, though, is multi-layered display. That sounds 
somewhat different from a "text layout API" to me.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 01:21:56 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 18195@debbugs.gnu.org
> 
> > For example, AFAIU, your code currently assumes that the font used for
> > popup is the same as the one used for the underlying buffer text.  But
> > since you provide faces for the popup, a user could legitimately
> > customize those faces to use a different font, and thus invalidate
> > your calculations, because window-screen-lines uses the metrics of the
> > default face's font.
> 
> Sure. This doesn't add any constraints over what we've been living with 
> for years.

Until some user would ask to lift them.

But if you never want to lift them, why not define colors instead of
full-fledged faces?  Then those restrictions are really carved in
stone and clear for all.

> > IMO, instead of overloading existing display features with jobs they
> > were never designed to do, and then live forever with the situation
> > where development breaks the resulting applications (like the
> > pixel-wise resizing of windows did with this one)
> 
> pixel-wise resizing wasn't actually the culprit: I've just never tested 
> the code with non-zero line-spacing.

The bug had two parts; one of them _was_ due to pixel-wise resizing.

>  > it would be a much
> > better investment of energy to come up with requirements for Lisp APIs
> > that could be used by applications for laying out text.
> 
> That sounds great, but I don't even know where to start. What we're 
> trying to emulate, though, is multi-layered display. That sounds 
> somewhat different from a "text layout API" to me.

I don't mind how it is called and what is the correct abstraction.  I
do know that we need some abstraction and APIs to support it.  If you
already know what the correct abstraction is, then you are in a good
position to formulate the requirements.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/06/2014 06:52 AM, Eli Zaretskii wrote:

> Until some user would ask to lift them.

Apparently, every user that tried to change the font understood that any 
resulting problems stemmed from their actions, and so we've never had 
any bug reports about that.

> But if you never want to lift them, why not define colors instead of
> full-fledged faces?  Then those restrictions are really carved in
> stone and clear for all.

Well, there are also text decorations, and users should be able to add 
(most of) them without running into problems.

Faces seem to be a natural fit for themes, and they're supported by 
quite of a few of existing themes out there. Making the 
backward-incompatible change you're suggesting will break that.

Finally, who's to say the current limitations will stay with us forever? 
If Emacs defines a suitable API, the current faces should be able to 
transition seamlessly, while users will also gain the ability to change 
the fonts.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 16:09:41 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 18195@debbugs.gnu.org
> 
> On 08/06/2014 06:52 AM, Eli Zaretskii wrote:
> 
> > Until some user would ask to lift them.
> 
> Apparently, every user that tried to change the font understood that any 
> resulting problems stemmed from their actions, and so we've never had 
> any bug reports about that.

AFAIU, you are developing infrastructure.  It is generally expected
from infrastructure, at least that in Emacs core, to support
everything that wasn't explicitly documented as forbidden or
unsupported.

OTOH, perhaps next time someone complains about some weird combination
of display features that happen not to work, I should respond
"restriction" and see what happens.

> > But if you never want to lift them, why not define colors instead of
> > full-fledged faces?  Then those restrictions are really carved in
> > stone and clear for all.
> 
> Well, there are also text decorations, and users should be able to add 
> (most of) them without running into problems.

Which decorations are those?

> Finally, who's to say the current limitations will stay with us forever? 
> If Emacs defines a suitable API, the current faces should be able to 
> transition seamlessly, while users will also gain the ability to change 
> the fonts.

Nothing like that will happen in Emacs without motivated individuals
driving the development.  IMO, we should either push development in
the direction I suggested (or similar), or explicitly forbid using
different fonts in Company completion, at least in the documentation,
if not by error messages.  Anything else is unwise, IMO.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/06/2014 06:45 PM, Eli Zaretskii wrote:

> AFAIU, you are developing infrastructure.  It is generally expected
> from infrastructure, at least that in Emacs core, to support
> everything that wasn't explicitly documented as forbidden or
> unsupported.

I'm inclined to consider this a missing feature, of which Emacs display 
engine has plenty. But documenting using different fonts in faces as 
prohibited sounds fine to me as well.

Anyway, I've written to emacs-devel about features which would make my 
life easier.

>> Well, there are also text decorations, and users should be able to add
>> (most of) them without running into problems.
>
> Which decorations are those?

Underline, bold, italic..?

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 19:07:32 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 18195@debbugs.gnu.org
> 
> >> Well, there are also text decorations, and users should be able to add
> >> (most of) them without running into problems.
> >
> > Which decorations are those?
> 
> Underline, bold, italic..?

They all use fonts that are different from the default one, and thus
change the height of the screen lines, if only by a few pixels.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Dmitry Gutov]

On 08/06/2014 08:25 PM, Eli Zaretskii wrote:

>> Underline, bold, italic..?
>
> They all use fonts that are different from the default one, and thus
> change the height of the screen lines, if only by a few pixels.

Not necessarily. At least, I can change my default face's :bold and 
:underline properties to t, and the line height doesn't change.

This probably depends on whether you have bold, underline, etc, variants 
of the same font installed in the system.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> Date: Wed, 06 Aug 2014 20:37:02 +0400
> From: Dmitry Gutov <dgutov@yandex.ru>
> CC: rudalics@gmx.at, 18195@debbugs.gnu.org
> 
> On 08/06/2014 08:25 PM, Eli Zaretskii wrote:
> 
> >> Underline, bold, italic..?
> >
> > They all use fonts that are different from the default one, and thus
> > change the height of the screen lines, if only by a few pixels.
> 
> Not necessarily. At least, I can change my default face's :bold and 
> :underline properties to t, and the line height doesn't change.

Sheer luck.

> This probably depends on whether you have bold, underline, etc, variants 
> of the same font installed in the system.

No, it depends on the font design AFAIK.  In general, you cannot
expect the bold, underline, and italics variants to have the same
metrics, IME.

bug#18195: 24.3.92; window-screen-lines is not accurate

[Eli Zaretskii]

> From: Dmitry <dgutov@yandex.ru>
> Date: Tue, 05 Aug 2014 06:00:24 +0400
> 
> If I have line-spacing set, the value the above function returns is
> smaller than the actuall number of screen lines the window can contain.
> 
> For example, when I have the current Emacs frame maximized and
> line-spacing set to 0.2, (window-screen-lines) returns 46.66(6), whereas
> there are 48 fully-visible lines in the given window and one
> partially-visible one.

Fixed in revision 117425 on the emacs-24 branch.