bug#29279: Sharing the margins

bug#29279: Sharing the margins

[Dmitry Gutov]

This is a rough proposal on how separate bodies of code (minor modes, 
etc) can use the same margin without conflict.

Previous discussions:

http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01171.html
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27427#53

Primary goals:

- Allow multiple kinds of margin content to coexist (examples: 
http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01505.html).
- Continue to use the text properties to set the contents of the margins 
(as opposed to having a line number based API, which seems to come with 
its own difficulties).

Outline:

We introduce two new variables, named like left-margin-columns-alist.

Each element is a cons which specifies some properties of the element, 
for instance:

- Priority (so the columns are in the same order on each line)
- Minimum width (if we want that, as opposed to just adding strings with 
that length the contents of this margin column)
- Minimum "total" width, which would allow to use this column as padding 
so that the combined width of the margin reaches a given number 
(hopefully solving the writeroom-mode problem).
- Padding (whether to pad this column with a space on one side if there 
is a next column)
- Text alignment within the column (left or right)

The keys in left-margin-columns-alist will be used as alternatives to 
`left-margin' in margin display specs.

The display engine would scan the contents of the current window, 
process said specs, calculate which lines fit the window and which do 
not, set the total margin width appropriately, and display all columns 
in it. Some reflowing might be required.

If the latter is considered too difficult, we can add "width" as a 
necessary parameter to the column properties. I think that would rule 
out the possibility of efficiently using the margins for the line 
numbers feature, though, which seems unfortunate. But the other uses of 
the margin that I'm aware of are not as dynamic.

Similar feature can be added for the fringes, too (for them, dynamic 
sizing isn't needed at all, probably).

Comments?

bug#29279: Sharing the margins

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 13 Nov 2017 01:51:53 +0200
> 
> This is a rough proposal on how separate bodies of code (minor modes, etc) can use the same margin without conflict.

Thanks for reviving the issue.

> Previous discussions:
> 
> http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01171.html
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27427#53

The second discussion doesn't seem to be relevant, although it touches
this tangentially.  It has nothing useful to add to this discussion,
AFAICT.

> Primary goals:
> 
> - Allow multiple kinds of margin content to coexist (examples: http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01505.html).

Yes, that's the goal.

> - Continue to use the text properties to set the contents of the margins (as opposed to having a line number based API, which seems to come with its own difficulties).

Not sure I understand this.  AFAIK no one proposed to get rid of the
mechanism of specifying margin display, be it via text properties or
overlays.  The line number display of Emacs 26 avoids using the
margins, but that wasn't proposed as some more general API for
anything else.  So I think this is a non-issue, and will just leads us
astray.

> Outline:
> 
> We introduce two new variables, named like left-margin-columns-alist.
> 
> Each element is a cons which specifies some properties of the element, for instance:

I think we need to come up with a specific list of properties, because
discussing an example will always be too vague and uncommitted.  So is
this the actual list you propose, or are there other properties you
envision?  If the latter, please add them to the list.

> - Priority (so the columns are in the same order on each line)

You mean "order", right?  That is, which part will be rendered first,
which after it, etc., right?  If not, what is the practical meaning of
"priority" here?

> - Minimum width (if we want that, as opposed to just adding strings with that length the contents of this margin column)
> - Minimum "total" width, which would allow to use this column as padding so that the combined width of the margin reaches a given number (hopefully solving the writeroom-mode problem).

I don't understand the difference between these two.  Can you
elaborate?

And what about the maximum width?  I think this is much more important
than the minimum.

> - Padding (whether to pad this column with a space on one side if there is a next column)

Doesn't the minimum width already cover this?  If the actual width is
less than the advertised minimum, text should be padded, otherwise it
shouldn't.  Right?

> - Text alignment within the column (left or right)

Why does this need to be a separate property?  We don't have anything
like that for any other kind of text.

> The keys in left-margin-columns-alist will be used as alternatives to `left-margin' in margin display specs.

I think we need to agree on the model of the display in the margins.
The fact that you use "columns" in your proposal hints that each
"user" of the margin (a Lisp program which displays there) will have a
separate "column" in the margin, and that column will be of the same
pixel width for each screen line.  If this is the model, then it makes
little sense to have different display specs regarding the "column"
width for each buffer position where display in the margin is
requested, because the resulting column width will be the same for all
such displays.  If we specify these in each display spec, we are in
effect wasting Lisp storage, and potentially also working against the
fundamental design of the display engine (more about that below).

> The display engine would scan the contents of the current window, process said specs, calculate which lines fit the window and which do not, set the total margin width appropriately, and display all columns in it. Some reflowing might be required.

This will not work with the current design of the display engine, at
least not without significant pains and at least twofold performance
degradation.  To realize why, you need to remember that when the
display engine is redisplaying a window, it initially has no idea
where the window will end, it normally (but not always) knows only
where it will begin.  This is because the stuff to be displayed in the
window could have changed significantly since the last redisplay
cycle, so any "memory" of where the window ended back then could
easily be invalid (which is why the display engine keeps almost no
memory about the results of the last redisplay).

The actual place where the window display ends is the consequence of
the display engine trying to lay out the window starting at
window-start, and going on until the window bottom is reached, at
which point it checks that point is visible in the window, and if not,
chooses a different window-start to bring point into the view, and
retries.

(The above description is a simplification: it omits various redisplay
optimizations which refrain from displaying the entire window in the
frequent use cases.  But for the purposes of this discussion, let's
forget about those optimizations, because their semantics is, and must
be, identical to redisplaying the whole window anew each time.)

For this reason, "scanning the contents of the current window" is not
something the display engine can do at the start of a window
redisplay.  It can only do so after one full cycle of window layout.
In addition, changing the dimensions of the margins requires to start
the window display anew, after reallocating the glyph matrices.

So what you suggest can only be implemented by displaying each window
twice.  On top of that, it will disable important redisplay
optimizations, which refrain from examining all of the screen lines
and the corresponding buffer text -- since you require to scan all of
the display specs in the window to dynamically compute the margin
dimensions.

The way the current display engine is designed, it makes the layout
decisions either at the start of a window's redisplay, or as it
traverses buffer text one character at a time.  The decisions related
to the dimensions of the canvas are best made at the beginning,
otherwise they will cause the redisplay to be abandoned and restarted
anew, which slows it down.  We have a few cases where we do that, but
they should be rare to provide good user experience.  (These cases
could also complicate the move_it_* functions, which simulate display.
As of now, they don't handle such cases, but won't be able to ignore
the consequences of your proposal to calculate margins dynamically as
part of redisplay.)

Therefore, features that require dynamic recalculation of the window
dimensions as part of redisplay should IMO be avoided at all costs.
We should try to find a way of making these calculations in Lisp, as
part of the program(s) that require display in the margins, so that by
the time redisplay kicks in, the calculation of the margin widths,
including the column width for each "user" of the margin, was already
done and stored in some form that the display engine could use when it
initializes redisplay of a window.

> If the latter is considered too difficult

I think "unworkable" is a more proper word, unfortunately, because I
don't see anyone who'd step forward to perform a major redesign of the
display engine required for your proposal.  (And if we are redesigning
the display engine, I have a few more important requirements for it ;-)

I will post an alternative proposal in a separate email.

> we can add "width" as a necessary parameter to the column properties. I think that would rule out the possibility of efficiently using the margins for the line numbers feature, though, which seems unfortunate. But the other uses of the margin that I'm aware of are not as dynamic.

Sorry, I think you lost me here.  What is that "width" parameter,
which "column properties" are being alluded to, and why would it
disallow dynamic resizing of the margins?  The only requirement for a
feature that will allow relatively simple and efficient implementation
is that the necessary total width of each window margin is known, or
can be calculated by accessing some buffer- or window-local variable,
at the beginning of a redisplay cycle.  Is that hard to accomplish?

> Similar feature can be added for the fringes, too (for them, dynamic sizing isn't needed at all, probably).

I think fringes are a separate issue.  AFAIK, we current cannot
display more than one bitmap on the fringe at any given screen line.

bug#29279: Sharing the margins

[Eli Zaretskii]

I tend to like better the proposal originally made by Joost Kremers in
http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01540.html.
Here it is with slight variations of my own:

We add 3 new functions:

 (defun window-margin-add (window side symbol cols &optional ordinal)
   "Request allocation of COLS columns of WINDOW's marginal area.
 WINDOW must be a live window and defaults to the selected one.
 SIDE is the window side, either `left' or `right', in whose margin
   the allocation is requested.
 SYMBOL identifies the request, in case the allocation needs to be
   changed or deleted.
 ORDINAL is the optional ordinal number of the requested area, counted
   from left to right.  Negative ordinal numbers count from right to
   left.  Zero means the value of COLS is the maximum width of the
   marginal area, and no separate allocation is requested.

 Value is the actual positive ordinal number of the allocated area."

 (defun window-margin-remove (window side symbol)
   "Remove a previously allocated part of WINDOW's marginal area.
 WINDOW must be a live window and defaults to the selected one.
 SIDE is the window side, either `left' or `right', in whose margin
   the removal is requested.
 SYMBOL identifies the request to be deleted."

 (defun window-margin-modify (window side symbol cols)
   "Modify a previously allocated part of WINDOW's marginal area.
 WINDOW must be a live window and defaults to the selected one.
 SIDE is the window side, either `left' or `right', in whose margin
   the modification is requested.
 SYMBOL identifies the request to be modified."

The details of the allocated portions of the marginal area are
maintained in window parameters `left-margin-params' and
`right-margin-params', which are modified by those 2 functions.  The
display engine uses these parameters to calculate the number of the
portions and the width of each one of them, and pre-allocates them in
the window's glyph matrices whenever a redisplay follows a change in
margin allocation.  The display spec for display in the margins will
include the symbol of the allocated portion pertaining to the display
spec.  Like this:

  '(display ((margin left-margin 'toto) STRING))

If the stuff (text, image, etc.) to be displayed in a specified
portion of the marginal area is too wide and doesn't fit the width of
the portion, it will be truncated; if it is narrower, it will be
padded.

This keeps the display engine design intact, and I think will allow
relatively straightforward implementation (although some non-trivial
changes in management of the glyph matrices will be necessary).  It
can also be implemented in a way that keeps backward compatibility
with the current APIs.

WDYT?

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/13/17 5:43 PM, Eli Zaretskii wrote:

> The second discussion doesn't seem to be relevant, although it touches
> this tangentially.  It has nothing useful to add to this discussion,
> AFAICT.

Maybe not. Except as a use case. The first one is also not hugely 
relevant, because it's mostly about workroom-mode, and that one is a 
rare kind of margin usage conflict.

>> - Continue to use the text properties to set the contents of the margins (as opposed to having a line number based API, which seems to come with its own difficulties).
> 
> Not sure I understand this.  AFAIK no one proposed to get rid of the
> mechanism of specifying margin display, be it via text properties or
> overlays.

I meant that the alternative would be a new line-based programmatic API 
with functions like (add-margin vsual-line-number, width, contents). 
Which I've briefly considered and discarded, for now.

 > The line number display of Emacs 26 avoids using the
 > margins, but that wasn't proposed as some more general API for
 > anything else.  So I think this is a non-issue, and will just leads us
 > astray.

No reference to "native line numbers" intended.

>> Each element is a cons which specifies some properties of the element, for instance:
> 
> I think we need to come up with a specific list of properties, because
> discussing an example will always be too vague and uncommitted.  So is
> this the actual list you propose, or are there other properties you
> envision?  If the latter, please add them to the list.

I think that was it, but additions welcome. These ones should be 
sufficient for the uses I'm thinking of now.

>> - Priority (so the columns are in the same order on each line)
> 
> You mean "order", right?  That is, which part will be rendered first,
> which after it, etc., right?  If not, what is the practical meaning of
> "priority" here?

Order priority, yes. It's just when I think of an "order" argument, the 
value is usually something like "user_name ASC", and not a number.

>> - Minimum width (if we want that, as opposed to just adding strings with that length the contents of this margin column)
>> - Minimum "total" width, which would allow to use this column as padding so that the combined width of the margin reaches a given number (hopefully solving the writeroom-mode problem).
> 
> I don't understand the difference between these two.  Can you
> elaborate?

The first one sizes a given column (all the margins contents in this 
column get padded to at least this width).

The second one helps to size the whole margin, using this column. A way 
to set the minimum total margin width, IOW.

> And what about the maximum width?  I think this is much more important
> than the minimum.

I've decided to leave it out, because choosing the places to "cut" is 
not trivial in advance, until we know of specific situations.

>> - Padding (whether to pad this column with a space on one side if there is a next column)
> 
> Doesn't the minimum width already cover this?  If the actual width is
> less than the advertised minimum, text should be padded, otherwise it
> shouldn't.  Right?

Yes, and padding should be omitted if there is padding on every line of 
the given column.

But if there is a line which didn't need to be padded, it will look 
"smushed" together with the next column. For some uses, it might be 
okay, for others, it will be not as readable.

>> - Text alignment within the column (left or right)
> 
> Why does this need to be a separate property?  We don't have anything
> like that for any other kind of text.

IDK, why not? Will we always align to the right? I don't really mind, 
but if we anticipate the users to want different alignments, the 
property needs to be here. Otherwise, the callers will have to take care 
of "minimum width" themselves. And that basically requires them to 
refresh the margin display specs after every scrolling (but before 
redisplay, apparently).

>> The keys in left-margin-columns-alist will be used as alternatives to `left-margin' in margin display specs.
> 
> I think we need to agree on the model of the display in the margins.
> The fact that you use "columns" in your proposal hints that each
> "user" of the margin (a Lisp program which displays there) will have a
> separate "column" in the margin, and that column will be of the same
> pixel width for each screen line.

Yes.

> If this is the model, then it makes
> little sense to have different display specs regarding the "column"
> width for each buffer position where display in the margin is
> requested, because the resulting column width will be the same for all
> such displays.

The width of the string inside the margin display specs can very, can't it?

> If we specify these in each display spec, we are in
> effect wasting Lisp storage, and potentially also working against the
> fundamental design of the display engine (more about that below).

Sorry, specify what? The margin display spec will continue to be

   ((margin column-name) spec)

where SPEC is a text string, most of the time.

>> The display engine would scan the contents of the current window, process said specs, calculate which lines fit the window and which do not, set the total margin width appropriately, and display all columns in it. Some reflowing might be required.
> 
> This will not work with the current design of the display engine, at
> least not without significant pains and at least twofold performance
> degradation.  To realize why, you need to remember that when the
> display engine is redisplaying a window, it initially has no idea
> where the window will end, it normally (but not always) knows only
> where it will begin.  This is because the stuff to be displayed in the
> window could have changed significantly since the last redisplay
> cycle, so any "memory" of where the window ended back then could
> easily be invalid (which is why the display engine keeps almost no
> memory about the results of the last redisplay).

I think I'm getting the rough idea.

> The actual place where the window display ends is the consequence of
> the display engine trying to lay out the window starting at
> window-start, and going on until the window bottom is reached, at
> which point it checks that point is visible in the window, and if not,
> chooses a different window-start to bring point into the view, and
> retries.

Yes, and choosing a different margin width will obviously affect that, 
as we as the set of margin specs we encounter in the visible part of the 
buffer. Which might make a naive implementation restart the process 
multiple times in certain cases, and maybe even infloop.

> (The above description is a simplification: it omits various redisplay
> optimizations which refrain from displaying the entire window in the
> frequent use cases.  But for the purposes of this discussion, let's
> forget about those optimizations, because their semantics is, and must
> be, identical to redisplaying the whole window anew each time.)
> 
> For this reason, "scanning the contents of the current window" is not
> something the display engine can do at the start of a window
> redisplay.  It can only do so after one full cycle of window layout.
> In addition, changing the dimensions of the margins requires to start
> the window display anew, after reallocating the glyph matrices.
> 
> So what you suggest can only be implemented by displaying each window
> twice.  On top of that, it will disable important redisplay
> optimizations, which refrain from examining all of the screen lines
> and the corresponding buffer text -- since you require to scan all of
> the display specs in the window to dynamically compute the margin
> dimensions.

I was hoping that we might consider some parts of redisplay to be "fast 
enough" by now (posn-at-point is fast enough for ~500 FPS on my machine, 
for instance), but indeed it should require some smart programming.

> The way the current display engine is designed, it makes the layout
> decisions either at the start of a window's redisplay, or as it
> traverses buffer text one character at a time.  The decisions related
> to the dimensions of the canvas are best made at the beginning,
> otherwise they will cause the redisplay to be abandoned and restarted
> anew, which slows it down.  We have a few cases where we do that, but
> they should be rare to provide good user experience.  (These cases
> could also complicate the move_it_* functions, which simulate display.
> As of now, they don't handle such cases, but won't be able to ignore
> the consequences of your proposal to calculate margins dynamically as
> part of redisplay.)
>
> Therefore, features that require dynamic recalculation of the window
> dimensions as part of redisplay should IMO be avoided at all costs.
> We should try to find a way of making these calculations in Lisp, as
> part of the program(s) that require display in the margins, so that by
> the time redisplay kicks in, the calculation of the margin widths,
> including the column width for each "user" of the margin, was already
> done and stored in some form that the display engine could use when it
> initializes redisplay of a window.

What your description tells me, foremost, is that it should be 
impossible to "perfectly" make these decisions in Lisp, too.

Like, what can linum-mode do? After scrolling, it would look at the 
window-start, its height, and calculate the _approximate_ physical line 
at the end of the window. It can't get the precise number because the 
layout still hasn't happened, right? Furthermore, after it increases the 
width of its margin column (suppose we were scrolling up so line numbers 
increased), it can create a different layout where the last line is not 
visible anymore. And if its number was 100, that makes the specified 
margin width inaccurate by 1. Luckily, line numbers' width changes 
slowly, so this won't be apparent often.

Is this the idea?

>> If the latter is considered too difficult
> 
> I think "unworkable" is a more proper word, unfortunately, because I
> don't see anyone who'd step forward to perform a major redesign of the
> display engine required for your proposal.  (And if we are redesigning
> the display engine, I have a few more important requirements for it ;-)

OK :)

> I will post an alternative proposal in a separate email.
> 
>> we can add "width" as a necessary parameter to the column properties. I think that would rule out the possibility of efficiently using the margins for the line numbers feature, though, which seems unfortunate. But the other uses of the margin that I'm aware of are not as dynamic.
> 
> Sorry, I think you lost me here.  What is that "width" parameter,
> which "column properties" are being alluded to,

The properties inside left-margin-columns-alist.

So we'll have a WIDTH there (global for each column), instead of 
MINIMUM-WIDTH, and PADDING and TEXT-ALIGNMENT might become unnecessary.

> and why would it
> disallow dynamic resizing of the margins?

I guess more accurately, it would disallow dynamic *sizing* of the 
margins by the display engine.

Column widths will be accessible for manipulation from e.g. 
pre-redisplay-functions, but like described above, it doesn't seem like 
choosing margin width precisely is feasible in Lisp, even in the use 
case of showing line numbers.

> The only requirement for a
> feature that will allow relatively simple and efficient implementation
> is that the necessary total width of each window margin is known, or
> can be calculated by accessing some buffer- or window-local variable,
> at the beginning of a redisplay cycle.  Is that hard to accomplish?

With a column-global WIDTH property, I don't think it's hard.

>> Similar feature can be added for the fringes, too (for them, dynamic sizing isn't needed at all, probably).
> 
> I think fringes are a separate issue.  AFAIK, we current cannot
> display more than one bitmap on the fringe at any given screen line.

Technical limitations of some GUI toolkits, no doubt. Hopefully, more 
easily overcome than display engine complexity.

Fringes have other problems anyway, like inability to use more than 2 
colors, vector graphics, etc.

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/13/17 5:46 PM, Eli Zaretskii wrote:
> I tend to like better the proposal originally made by Joost Kremers in
> http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01540.html.
> Here it is with slight variations of my own:

Overall, it seems like a friendlier API over fairly similar structures 
as what I suggested (just replace left-margin-columns-alist with 
left-margin-params). Except SYMBOL is not given a meaningful name (they 
are "columns" in my description).

> We add 3 new functions:
> 
>   (defun window-margin-add (window side symbol cols &optional ordinal)
>     "Request allocation of COLS columns of WINDOW's marginal area.
>   WINDOW must be a live window and defaults to the selected one.
>   SIDE is the window side, either `left' or `right', in whose margin
>     the allocation is requested.
>   SYMBOL identifies the request, in case the allocation needs to be
>     changed or deleted.
>   ORDINAL is the optional ordinal number of the requested area, counted
>     from left to right.  Negative ordinal numbers count from right to
>     left.  Zero means the value of COLS is the maximum width of the
>     marginal area, and no separate allocation is requested.

ORDINAL meaning the same as "order priority"? And the lower the value 
is, the more "left" the column should be positioned? Sounds OK to me.

I'm not sure I understand the "Zero means ..." passage, though.

In addition to signifying a neutral position, does it supposed to switch 
the meaning of this function into something that 
set-right-margin/set-left-margin can call, for backward compatibility?

Seems like a wart, using ORDINAL this way. And what's going to happen 
when somebody else calls window-margin-add with non-zero ORDINAL? Will 
the end result depend on which call happens first?

>   Value is the actual positive ordinal number of the allocated area."

Return value?

> If the stuff (text, image, etc.) to be displayed in a specified
> portion of the marginal area is too wide and doesn't fit the width of
> the portion, it will be truncated; if it is narrower, it will be
> padded.

Padded from the left or from the right? :)

> This keeps the display engine design intact, and I think will allow
> relatively straightforward implementation (although some non-trivial
> changes in management of the glyph matrices will be necessary).  It
> can also be implemented in a way that keeps backward compatibility
> with the current APIs.
> 
> WDYT?

Aside from the ORDINAL edge case, it sounds good to me.

Here's an interesting question: after such an API is added, will it be 
feasible to re-implement display-line-numbers-mode using a margin 
column, instead of the special separate area?

And if not, how can we make this API better that you could?

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 13 Nov 2017 19:24:21 +0200
> 
> >> - Text alignment within the column (left or right)
> > 
> > Why does this need to be a separate property?  We don't have anything
> > like that for any other kind of text.
> 
> IDK, why not? Will we always align to the right?

No, to the left.  If a Lisp program wants to align to the right, it
should insert white space before the actual text.

> > I think we need to agree on the model of the display in the margins.
> > The fact that you use "columns" in your proposal hints that each
> > "user" of the margin (a Lisp program which displays there) will have a
> > separate "column" in the margin, and that column will be of the same
> > pixel width for each screen line.
> 
> Yes.
> 
> > If this is the model, then it makes
> > little sense to have different display specs regarding the "column"
> > width for each buffer position where display in the margin is
> > requested, because the resulting column width will be the same for all
> > such displays.
> 
> The width of the string inside the margin display specs can very, can't it?

The width can change, but the column width will not change.  That
means the column width should be specified only once, not multiple
times.

> > If we specify these in each display spec, we are in
> > effect wasting Lisp storage, and potentially also working against the
> > fundamental design of the display engine (more about that below).
> 
> Sorry, specify what? The margin display spec will continue to be
> 
>    ((margin column-name) spec)
> 
> where SPEC is a text string, most of the time.

I thought you wanted all the other parameters in the spec as well.

But if you didn't mean that, then what kind of scanning and
calculations are you talking about here:

> >> The display engine would scan the contents of the current window, process said specs, calculate which lines fit the window and which do not, set the total margin width appropriately, and display all columns in it. Some reflowing might be required.

If everything is already spelled out in a variable
left-margin-columns-alist, then why do we need to scan the contents of
the window?

> > So what you suggest can only be implemented by displaying each window
> > twice.  On top of that, it will disable important redisplay
> > optimizations, which refrain from examining all of the screen lines
> > and the corresponding buffer text -- since you require to scan all of
> > the display specs in the window to dynamically compute the margin
> > dimensions.
> 
> I was hoping that we might consider some parts of redisplay to be "fast 
> enough" by now (posn-at-point is fast enough for ~500 FPS on my machine, 
> for instance), but indeed it should require some smart programming.

posn-at-point goes _once_ from window-start to the specified position,
so on average it traverses half a window, once.  By contrast, we are
now talking about redisplaying the window twice, and one of these 2
times must traverse the entire window.  So we are talking about
threefold slow-down on the average.

> What your description tells me, foremost, is that it should be 
> impossible to "perfectly" make these decisions in Lisp, too.
> 
> Like, what can linum-mode do? After scrolling, it would look at the 
> window-start, its height, and calculate the _approximate_ physical line 
> at the end of the window.

linum uses window-end (which could be nil in some cases).  window-end
is updated by redisplay when it ends successfully.  linum then updates
the width of the window margins if needed, and updates all the
overlays with line numbers, which triggers an immediate additional
redisplay.  That's why it is slow.

So yes, if the margin display depends heavily on what is in the
window, especially on how many lines/characters are in the window, it
will have hard time being Speedy Gonzales; such features are better
implemented in C as an integral part of redisplay.  But not all uses
of the margins are like that.  I will even risk saying that most of
them aren't.  For that majority, calculating the maximum width they
need from the margin at the end of a command is not a big deal, and
could probably change relatively rarely.

> I guess more accurately, it would disallow dynamic *sizing* of the 
> margins by the display engine.

With the current design of the display engine, dynamic resizing is not
a good idea.  Any such resizing is painful: it requires throwing away
the window's glyph matrices, reallocating them anew with the new
dimensions, and then completely redrawing the entire window.  It's
expensive and should be avoided.

> Column widths will be accessible for manipulation from e.g. 
> pre-redisplay-functions, but like described above, it doesn't seem like 
> choosing margin width precisely is feasible in Lisp, even in the use 
> case of showing line numbers.

I think it's feasible in many cases.  Linum and its ilk are a rare use
case, although the resulting feature is very popular (which is why we
now have the native line numbers).

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 13 Nov 2017 19:54:45 +0200
> 
> >   ORDINAL is the optional ordinal number of the requested area, counted
> >     from left to right.  Negative ordinal numbers count from right to
> >     left.  Zero means the value of COLS is the maximum width of the
> >     marginal area, and no separate allocation is requested.
> 
> ORDINAL meaning the same as "order priority"?

Yes.

> And the lower the value is, the more "left" the column should be
> positioned? Sounds OK to me.

Yes.

> I'm not sure I understand the "Zero means ..." passage, though.

That's your "total width" thing, for margin users that just want to
set the overall width of the margins without displaying anything
there.  Like Joost Kramer's visual-fill-column and similar packages.

> In addition to signifying a neutral position, does it supposed to switch 
> the meaning of this function into something that 
> set-right-margin/set-left-margin can call, for backward compatibility?

Yes, set-window-margins will most probably be reimplemented by calling
the above.

> Seems like a wart, using ORDINAL this way. And what's going to happen 
> when somebody else calls window-margin-add with non-zero ORDINAL? Will 
> the end result depend on which call happens first?

Yes.  And the result is returned, so the caller knows that.  If you
have better ideas for requesting a particular position in the margin,
let's hear them.

> >   Value is the actual positive ordinal number of the allocated area."
> 
> Return value?

Yes.

> > If the stuff (text, image, etc.) to be displayed in a specified
> > portion of the marginal area is too wide and doesn't fit the width of
> > the portion, it will be truncated; if it is narrower, it will be
> > padded.
> 
> Padded from the left or from the right? :)

On the right.

> Here's an interesting question: after such an API is added, will it be 
> feasible to re-implement display-line-numbers-mode using a margin 
> column, instead of the special separate area?

Yes.  But using margins from Emacs internals means that the
window-parameters which hold the column specs will change behind the
back of the Lisp applications, which I'm not sure is a Good Thing.  It
will also be somewhat slower.

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/13/17 8:15 PM, Eli Zaretskii wrote:

> No, to the left.  If a Lisp program wants to align to the right, it
> should insert white space before the actual text.

That's only possible if it knows the resulting width. Which it will, in 
the current state of the discussion.

Still, it seems unnecessary if the somewhat faster C code could 
implement that for every user.

>>>> The display engine would scan the contents of the current window, process said specs, calculate which lines fit the window and which do not, set the total margin width appropriately, and display all columns in it. Some reflowing might be required.
> 
> If everything is already spelled out in a variable
> left-margin-columns-alist, then why do we need to scan the contents of
> the window?

In the first proposal, the actual width of the column would be 
auto-computed by the display engine based in the width of all relevant 
SPECs.

But we seem to have discarded this idea now.

>> I was hoping that we might consider some parts of redisplay to be "fast
>> enough" by now (posn-at-point is fast enough for ~500 FPS on my machine,
>> for instance), but indeed it should require some smart programming.
> 
> posn-at-point goes _once_ from window-start to the specified position,
> so on average it traverses half a window, once.  By contrast, we are
> now talking about redisplaying the window twice, and one of these 2
> times must traverse the entire window.  So we are talking about
> threefold slow-down on the average.

3-fold slowdown from 500 FPS seems acceptable to me.

> So yes, if the margin display depends heavily on what is in the
> window, especially on how many lines/characters are in the window, it
> will have hard time being Speedy Gonzales; such features are better
> implemented in C as an integral part of redisplay.  But not all uses
> of the margins are like that.  I will even risk saying that most of
> them aren't.  For that majority, calculating the maximum width they
> need from the margin at the end of a command is not a big deal, and
> could probably change relatively rarely.

Let's hope so.

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/13/17 8:29 PM, Eli Zaretskii wrote:

>> I'm not sure I understand the "Zero means ..." passage, though.
> 
> That's your "total width" thing, for margin users that just want to
> set the overall width of the margins without displaying anything
> there.  Like Joost Kramer's visual-fill-column and similar packages.

OK, but why "maximum width"? workroom-mode wanted to set the total 
width, but if we want to describe what will happen with the column in 
question, the value sounds more like "minimum total width".

>> In addition to signifying a neutral position, does it supposed to switch
>> the meaning of this function into something that
>> set-right-margin/set-left-margin can call, for backward compatibility?
> 
> Yes, set-window-margins will most probably be reimplemented by calling
> the above.

Which area will the left-margin specs be drawn on, then? Ones without 
any particular symbol specified.

>> Seems like a wart, using ORDINAL this way. And what's going to happen
>> when somebody else calls window-margin-add with non-zero ORDINAL? Will
>> the end result depend on which call happens first?
> 
> Yes.  And the result is returned, so the caller knows that.  If you
> have better ideas for requesting a particular position in the margin,
> let's hear them.

Having ORDINAL to be a number is totally fine to me.

Having ORDINAL = 0 mean something else, not so great. Especially if the 
result is to have the padding in this column, necessary to reach the 
specified total width.

I imagine workroom-mode might have a idea where they want the padding to 
end up (to the left or to the right of all columns). So instead of 
co-opting the ORDINAL argument to mean "cols will  total cols"

>> Here's an interesting question: after such an API is added, will it be
>> feasible to re-implement display-line-numbers-mode using a margin
>> column, instead of the special separate area?
> 
> Yes.  But using margins from Emacs internals means that the
> window-parameters which hold the column specs will change behind the
> back of the Lisp applications, which I'm not sure is a Good Thing.

I was thinking more along the lines of being able to specify a 
spec-returning function (that uses the current buffer position), instead 
of only using the text properties. But changing the margins directly 
sounds faster.

Maybe not an entirely good thing, but certainly an improvement for the 
authors of third-party code.

> It will also be somewhat slower.

We should probably measure before discarding this idea.

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 13 Nov 2017 21:02:23 +0200
> 
> On 11/13/17 8:15 PM, Eli Zaretskii wrote:
> 
> > No, to the left.  If a Lisp program wants to align to the right, it
> > should insert white space before the actual text.
> 
> That's only possible if it knows the resulting width. Which it will, in 
> the current state of the discussion.
> 
> Still, it seems unnecessary if the somewhat faster C code could 
> implement that for every user.

We could add that later if needed.  It's not rocket science, it's just
more complexity.

> > posn-at-point goes _once_ from window-start to the specified position,
> > so on average it traverses half a window, once.  By contrast, we are
> > now talking about redisplaying the window twice, and one of these 2
> > times must traverse the entire window.  So we are talking about
> > threefold slow-down on the average.
> 
> 3-fold slowdown from 500 FPS seems acceptable to me.

For each redisplay cycle?  On top of disabling most redisplay
optimizations?  I doubt that.

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 13 Nov 2017 21:16:30 +0200
> 
> On 11/13/17 8:29 PM, Eli Zaretskii wrote:
> 
> >> I'm not sure I understand the "Zero means ..." passage, though.
> > 
> > That's your "total width" thing, for margin users that just want to
> > set the overall width of the margins without displaying anything
> > there.  Like Joost Kramer's visual-fill-column and similar packages.
> 
> OK, but why "maximum width"? workroom-mode wanted to set the total 
> width, but if we want to describe what will happen with the column in 
> question, the value sounds more like "minimum total width".

Indeed, I meant to write "total", not "maximum".

> > Yes, set-window-margins will most probably be reimplemented by calling
> > the above.
> 
> Which area will the left-margin specs be drawn on, then? Ones without 
> any particular symbol specified.

Either without any symbol, or with nil, or with some invented symbol.
Something ti figure out as part of the implementation.

> Having ORDINAL = 0 mean something else, not so great. Especially if the 
> result is to have the padding in this column, necessary to reach the 
> specified total width.

My idea was not to create a column, just make sure the total width is
no less than the requested value.  Which means some of the requested
columns will be wider than requested, I guess.

> I imagine workroom-mode might have a idea where they want the padding to 
> end up (to the left or to the right of all columns). So instead of 
> co-opting the ORDINAL argument to mean "cols will  total cols"

We need to study the needs of potential users, no doubt, before
finalizing the API.

> > It will also be somewhat slower.
> 
> We should probably measure before discarding this idea.

The slowdown will be caused by resizing of the margins (and all the
window-configuration-change-hooks that triggers).

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/13/17 9:22 PM, Eli Zaretskii wrote:

>> Still, it seems unnecessary if the somewhat faster C code could
>> implement that for every user.
> 
> We could add that later if needed.  It's not rocket science, it's just
> more complexity.

Yup.

>>> posn-at-point goes _once_ from window-start to the specified position,
>>> so on average it traverses half a window, once.  By contrast, we are
>>> now talking about redisplaying the window twice, and one of these 2
>>> times must traverse the entire window.  So we are talking about
>>> threefold slow-down on the average.
>>
>> 3-fold slowdown from 500 FPS seems acceptable to me.
> 
> For each redisplay cycle?  On top of disabling most redisplay
> optimizations?  I doubt that.

Hard for me to tell. Like, in the recent discussion of a performance 
problem related to double-buffering, apparently Emacs itself wasn't the 
rendering bottleneck.

You probably know better, though.

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/13/17 9:32 PM, Eli Zaretskii wrote:

>> OK, but why "maximum width"? workroom-mode wanted to set the total
>> width, but if we want to describe what will happen with the column in
>> question, the value sounds more like "minimum total width".
> 
> Indeed, I meant to write "total", not "maximum".

I see.

>>> Yes, set-window-margins will most probably be reimplemented by calling
>>> the above.
>>
>> Which area will the left-margin specs be drawn on, then? Ones without
>> any particular symbol specified.
> 
> Either without any symbol, or with nil, or with some invented symbol.
> Something ti figure out as part of the implementation.

I think set-window-margins, and the nil/unknown symbols should work with 
the 'default' symbol. And it will have the ordinal = 0.

Then, older packages that are not updated to use the new API can fight 
between themselves for the use of the default column, but the winner can 
peacefully coexist with the packages using the new API.

>> Having ORDINAL = 0 mean something else, not so great. Especially if the
>> result is to have the padding in this column, necessary to reach the
>> specified total width.
> 
> My idea was not to create a column, just make sure the total width is
> no less than the requested value.  Which means some of the requested
> columns will be wider than requested, I guess.

It would probably look not too great. Just like 'text-align: justify' 
often works bad on web pages.

So I'd personally prefer to have all padding on one side.

Then, unless people disagree, setting the total width could be made into 
a separate call. With three arguments: side, width and the side from 
which to pad (inside/outside, for instance).

>> I imagine workroom-mode might have a idea where they want the padding to
>> end up (to the left or to the right of all columns). So instead of
>> co-opting the ORDINAL argument to mean "cols will  total cols"
> 
> We need to study the needs of potential users, no doubt, before
> finalizing the API.

Inviting Joost into the discussion.

>>> It will also be somewhat slower.
>>
>> We should probably measure before discarding this idea.
> 
> The slowdown will be caused by resizing of the margins (and all the
> window-configuration-change-hooks that triggers).

Doesn't the use of the special area trigger the window configuration 
changes as well, in similar situations? After all, it also changes the 
accessible window body width, right?

bug#29279: Sharing the margins

[martin rudalics]

 >   ORDINAL is the optional ordinal number of the requested area, counted
 >     from left to right.  Negative ordinal numbers count from right to
 >     left.

I'd prefer ORDINAL represent an integer such that 0 means to put it in
the middle of that margin, a negative value farther to the left and a
positive value farther to the right with clashes decided in some
unspecified way.  I tried a similar thing with side windows.  The idea
is that a "don't care where to put it" application has a neutral choice
against others which explicitly want to display their elements left- or
rightmost.

 >     Zero means the value of COLS is the maximum width of the
 >     marginal area, and no separate allocation is requested.

I'd use a separate symbol (say 'pad-to-cols') for that.

BTW is ‘window-margin-modify’ just for the sake of the line numbers
codes or which other users would it have?

martin

bug#29279: Sharing the margins

[martin rudalics]

 > Yes.  But using margins from Emacs internals means that the
 > window-parameters which hold the column specs will change behind the
 > back of the Lisp applications, which I'm not sure is a Good Thing.

I can see no harm in that.

martin

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org, Joost Kremers <joostkremers@fastmail.fm>
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 13 Nov 2017 23:16:15 +0200
> 
> I think set-window-margins, and the nil/unknown symbols should work with 
> the 'default' symbol. And it will have the ordinal = 0.
> 
> Then, older packages that are not updated to use the new API can fight 
> between themselves for the use of the default column, but the winner can 
> peacefully coexist with the packages using the new API.

Could be.

> > My idea was not to create a column, just make sure the total width is
> > no less than the requested value.  Which means some of the requested
> > columns will be wider than requested, I guess.
> 
> It would probably look not too great. Just like 'text-align: justify' 
> often works bad on web pages.
> 
> So I'd personally prefer to have all padding on one side.

But then requests for the rightmost (or leftmost) column will go
unsatisfied, for apparently no good reason.

> Then, unless people disagree, setting the total width could be made into 
> a separate call. With three arguments: side, width and the side from 
> which to pad (inside/outside, for instance).

That can be done, but the main issue is not the API in this case, I
think, it's the effect of the call.

> > The slowdown will be caused by resizing of the margins (and all the
> > window-configuration-change-hooks that triggers).
> 
> Doesn't the use of the special area trigger the window configuration 
> changes as well, in similar situations?

No, and I still hope we can avoid the need to do that.
tabulated-list-mode came close, but I eventually succeeded in making
it happy with the other hooks.

> After all, it also changes the accessible window body width, right?

It depends on your POV.  My POV is that it doesn't, since the window
dimensions and the dimensions of the text area are unaltered.  At
least one other person disagreed (vociferously).

bug#29279: Sharing the margins

[Eli Zaretskii]

> Date: Tue, 14 Nov 2017 10:54:36 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 29279@debbugs.gnu.org
> 
>  >   ORDINAL is the optional ordinal number of the requested area, counted
>  >     from left to right.  Negative ordinal numbers count from right to
>  >     left.
> 
> I'd prefer ORDINAL represent an integer such that 0 means to put it in
> the middle of that margin, a negative value farther to the left and a
> positive value farther to the right with clashes decided in some
> unspecified way.

This would make it hard to request to be the leftmost or rightmost
column, I think.

> I tried a similar thing with side windows.  The idea is that a
> "don't care where to put it" application has a neutral choice
> against others which explicitly want to display their elements left-
> or rightmost.

I think this should be possible by a simple transformation of the
original argument.

> BTW is ‘window-margin-modify’ just for the sake of the line numbers
> codes or which other users would it have?

Any package whose requirement of the margin width is not constant, but
has to change due to some event.  Even visual-fill-column should need
it, I think.

bug#29279: Sharing the margins

[Eli Zaretskii]

> Date: Tue, 14 Nov 2017 10:54:45 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 29279@debbugs.gnu.org
> 
>  > Yes.  But using margins from Emacs internals means that the
>  > window-parameters which hold the column specs will change behind the
>  > back of the Lisp applications, which I'm not sure is a Good Thing.
> 
> I can see no harm in that.

It's...unexpected.

bug#29279: Sharing the margins

[martin rudalics]

 > This would make it hard to request to be the leftmost or rightmost
 > column, I think.

most-negative/positive-fixnum maybe?

 >> BTW is ‘window-margin-modify’ just for the sake of the line numbers
 >> codes or which other users would it have?
 >
 > Any package whose requirement of the margin width is not constant, but
 > has to change due to some event.  Even visual-fill-column should need
 > it, I think.

What is visual-fill-column?

martin

bug#29279: Sharing the margins

[Eli Zaretskii]

> Date: Tue, 14 Nov 2017 19:30:54 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: dgutov@yandex.ru, 29279@debbugs.gnu.org
> 
>  > This would make it hard to request to be the leftmost or rightmost
>  > column, I think.
> 
> most-negative/positive-fixnum maybe?

That's a possibility, but it seems less natural to me.

Anyway, these are small tidbits that are better figured out during
implementation, I think.

>  >> BTW is ‘window-margin-modify’ just for the sake of the line numbers
>  >> codes or which other users would it have?
>  >
>  > Any package whose requirement of the margin width is not constant, but
>  > has to change due to some event.  Even visual-fill-column should need
>  > it, I think.
> 
> What is visual-fill-column?

See 

   http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01171.html

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/14/17 5:30 PM, Eli Zaretskii wrote:

>> So I'd personally prefer to have all padding on one side.
> 
> But then requests for the rightmost (or leftmost) column will go
> unsatisfied, for apparently no good reason.

It might be just a matter of interpretation: even if all the padding is 
on the right, the "rightmost" column will remain such, among all the 
visible columns.

Alternatively, the new function will be able to set the padding to 
"justify" as well, but I doubt a lot of users will like it.

>> Doesn't the use of the special area trigger the window configuration
>> changes as well, in similar situations?
> 
> No, and I still hope we can avoid the need to do that.
> tabulated-list-mode came close, but I eventually succeeded in making
> it happy with the other hooks.
>
>> After all, it also changes the accessible window body width, right?
> 
> It depends on your POV.  My POV is that it doesn't, since the window
> dimensions and the dimensions of the text area are unaltered.  At
> least one other person disagreed (vociferously).

But the effects are almost entirely the same, aren't they? Both the 
change of margin width and the change of line numbers column width force 
the reflowing of buffer contents display.

If the latter doesn't have to run window-configuration-change-hook, 
maybe the margin changes don't have to either, and "other hooks" will 
suffice as well?

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org, joostkremers@fastmail.fm
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 15 Nov 2017 00:39:34 +0200
> 
> On 11/14/17 5:30 PM, Eli Zaretskii wrote:
> 
> >> So I'd personally prefer to have all padding on one side.
> > 
> > But then requests for the rightmost (or leftmost) column will go
> > unsatisfied, for apparently no good reason.
> 
> It might be just a matter of interpretation: even if all the padding is 
> on the right, the "rightmost" column will remain such, among all the 
> visible columns.

Yes, but "being the rightmost" might mean "being right next to the
text", for whatever purposes.  A stretch of white space between the
text and the margin display might not be what the package wants.

> > It depends on your POV.  My POV is that it doesn't, since the window
> > dimensions and the dimensions of the text area are unaltered.  At
> > least one other person disagreed (vociferously).
> 
> But the effects are almost entirely the same, aren't they? Both the 
> change of margin width and the change of line numbers column width force 
> the reflowing of buffer contents display.

Again, as long as the text-area dimensions didn't change, it could be
argued that the hook shouldn't run.

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/15/17 5:42 AM, Eli Zaretskii wrote:

>> It might be just a matter of interpretation: even if all the padding is
>> on the right, the "rightmost" column will remain such, among all the
>> visible columns.
> 
> Yes, but "being the rightmost" might mean "being right next to the
> text", for whatever purposes.

So far, I doubt it's going to come up as a real, hard requirement. But 
we'll probably see.

> A stretch of white space between the
> text and the margin display might not be what the package wants.

OK, can we think of a case where a package might want its margin content 
to be _as far away_ from the text as possible? And become troubled if 
some padding stands between it and the window edge?

If not, let's put all padding on the outside and be done with that concern.

>> But the effects are almost entirely the same, aren't they? Both the
>> change of margin width and the change of line numbers column width force
>> the reflowing of buffer contents display.
> 
> Again, as long as the text-area dimensions didn't change, it could be
> argued that the hook shouldn't run.

That's some creative nomenclature lawyering. :-)

OK then, what's the practical problem with saying that margins are also 
part of "text-area dimensions"?

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org, joostkremers@fastmail.fm
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 15 Nov 2017 16:23:01 +0200
> 
> On 11/15/17 5:42 AM, Eli Zaretskii wrote:
> 
> >> It might be just a matter of interpretation: even if all the padding is
> >> on the right, the "rightmost" column will remain such, among all the
> >> visible columns.
> > 
> > Yes, but "being the rightmost" might mean "being right next to the
> > text", for whatever purposes.
> 
> So far, I doubt it's going to come up as a real, hard requirement. But 
> we'll probably see.

I could think of something similar to overlay-arrow, only using the
margin.  Such a feature will want the arrow to be as close to buffer
text as possible.

> > A stretch of white space between the
> > text and the margin display might not be what the package wants.
> 
> OK, can we think of a case where a package might want its margin content 
> to be _as far away_ from the text as possible? And become troubled if 
> some padding stands between it and the window edge?
> 
> If not, let's put all padding on the outside and be done with that concern.

This is doable, but the implementation will be more complex.
Remember: the display engine lays out stuff left to right.  So padding
what's left after we are done with all of the "columns" is easy;
padding _before_ requires that you either compute all the widths in
advance, or that you come back after laying out the columns and insert
the stretch before it, moving all the glyphs to the right.

> >> But the effects are almost entirely the same, aren't they? Both the
> >> change of margin width and the change of line numbers column width force
> >> the reflowing of buffer contents display.
> > 
> > Again, as long as the text-area dimensions didn't change, it could be
> > argued that the hook shouldn't run.
> 
> That's some creative nomenclature lawyering. :-)

It isn't.  It's just how the implementation works.

> OK then, what's the practical problem with saying that margins are also 
> part of "text-area dimensions"?

It's hard to implement.  window-configuration-change-hook currently
runs from functions that change window dimensions or the buffer
displayed in it; those never run inside redisplay.  What you (and some
others) propose will have to run in the middle of redisplaying a
window, and who knows what running arbitrary Lisp at that point will
or could do?

In addition, it will require us to record somewhere (probably, in the
window object) the last used width for number display, so we could
compare that with the new one.  This is not as simple as it sounds,
because most redisplay cycles avoid redrawing the entire window, so
determining just where in the code to perform this comparison is a
non-trivial decision.

So I'd like to avoid this if possible.

bug#29279: Sharing the margins

[martin rudalics]

 >>   > Yes.  But using margins from Emacs internals means that the
 >>   > window-parameters which hold the column specs will change behind the
 >>   > back of the Lisp applications, which I'm not sure is a Good Thing.
 >>
 >> I can see no harm in that.
 >
 > It's...unexpected.

Unless we are content with showing only the left part of the left margin
when a window gets too small as we do currently, we might also provide a
solution using "nominal" margin widths which apply when a window is wide
enough, and "realized" margin widths which apply when a window is too
narrow.  For line numbers, the nominal width could be somewhat
stretchable (as for the whiteroom, visual-fill-column cases) while most
others may have it fixed.  Probably, in addition an application may want
to specify that its realized margin width should be reset to zero
whenever the nominal width does not fit (line numbers probably want to
do that) or truncated to what is available.

Note that we earlier did shrink the margins when a window got too small
but did not re-enlarge them when the window was made wide again because
the nominal size was lost.  I eventually dropped that behavior and was
quite surprised that no-one protested.  But if we were to revive such
dynamical adjustment as sketched above, we'd have an unexpected "change
behind the back of the Lisp applications" as well.

martin

bug#29279: Sharing the margins

[martin rudalics]

 > If the latter doesn't have to run window-configuration-change-hook,
 > maybe the margin changes don't have to either, and "other hooks" will
 > suffice as well?

I intend to drop most calls of 'window-configuration-change-hook' and
replace them by 'window-size-change-functions' if all they do is to
change the sizes of windows only.  Maybe we should then change the
semantics of the latter: Call it when just the window body size changes
too.  Provide a 'window-body-pixel-width-before-size-change' function
which could be used in connection with 'window-body-height'.

martin

bug#29279: Sharing the margins

[Eli Zaretskii]

> Date: Wed, 15 Nov 2017 19:51:50 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: 29279@debbugs.gnu.org, joostkremers@fastmail.fm
> 
> I intend to drop most calls of 'window-configuration-change-hook' and
> replace them by 'window-size-change-functions' if all they do is to
> change the sizes of windows only.

Regardless of the reasons for this change, it doesn't affect the issue
at hand: calling window-size-change-functions from inside redisplay
has the same issues as calling window-configuration-change-hook.

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/15/17 10:03 PM, Eli Zaretskii wrote:

>> I intend to drop most calls of 'window-configuration-change-hook' and
>> replace them by 'window-size-change-functions' if all they do is to
>> change the sizes of windows only.
> 
> Regardless of the reasons for this change, it doesn't affect the issue
> at hand: calling window-size-change-functions from inside redisplay
> has the same issues as calling window-configuration-change-hook.

But margin size changes won't have to call window-size-change-functions, 
right? Then it might be possible to move all the heavy consumers to this 
hook, leaving window-configuration-change-hook relatively lightweight 
(right)?

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/15/17 8:00 PM, Eli Zaretskii wrote:

>>> Yes, but "being the rightmost" might mean "being right next to the
>>> text", for whatever purposes.
>>
>> So far, I doubt it's going to come up as a real, hard requirement. But
>> we'll probably see.
> 
> I could think of something similar to overlay-arrow, only using the
> margin.  Such a feature will want the arrow to be as close to buffer
> text as possible.

Yeah, OK.

>> If not, let's put all padding on the outside and be done with that concern.
> 
> This is doable, but the implementation will be more complex.
> Remember: the display engine lays out stuff left to right.  So padding
> what's left after we are done with all of the "columns" is easy;
> padding _before_ requires that you either compute all the widths in
> advance, or that you come back after laying out the columns and insert
> the stretch before it, moving all the glyphs to the right.

Sounds straightforward to me. Since we know the sizes of all the columns 
in advance, we can just substract them from the target total width, and 
pad with the resulting number of spaces.

Having extra padding between columns (your original idea) would be more 
complex to implement, IIUC.

>>> Again, as long as the text-area dimensions didn't change, it could be
>>> argued that the hook shouldn't run.
>>
>> That's some creative nomenclature lawyering. :-)
> 
> It isn't.  It's just how the implementation works.

Still. To the user, the line numbers column is more like an extra margin 
than anything else.

Further, even though we have a separate accessor for its width 
(line-number-display-width), if a package depends on it and needs to 
draw something based on its value, it should want to be notified when 
there is a change (*). window-configuration-change-hook seems natural. 
Unless we have a separate hook for that?

>> OK then, what's the practical problem with saying that margins are also
>> part of "text-area dimensions"?
> 
> It's hard to implement.  window-configuration-change-hook currently
> runs from functions that change window dimensions or the buffer
> displayed in it; those never run inside redisplay.  What you (and some
> others) propose will have to run in the middle of redisplaying a
> window, and who knows what running arbitrary Lisp at that point will
> or could do?

Not necessarily. Can't you save the necessary data to a variable, finish 
redisplay, and then run the hook (if the data says so)?

> In addition, it will require us to record somewhere (probably, in the
> window object) the last used width for number display, so we could
> compare that with the new one.  This is not as simple as it sounds,
> because most redisplay cycles avoid redrawing the entire window, so
> determining just where in the code to perform this comparison is a
> non-trivial decision.

*resigned shrug*

> So I'd like to avoid this if possible.

It's somewhat hypothetical, but I'd like to refer to (*) above. That is, 
somebody will probably ask for that anyway, sooner or later.

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org, joostkremers@fastmail.fm
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 15 Nov 2017 23:09:21 +0200
> 
> On 11/15/17 10:03 PM, Eli Zaretskii wrote:
> 
> >> I intend to drop most calls of 'window-configuration-change-hook' and
> >> replace them by 'window-size-change-functions' if all they do is to
> >> change the sizes of windows only.
> > 
> > Regardless of the reasons for this change, it doesn't affect the issue
> > at hand: calling window-size-change-functions from inside redisplay
> > has the same issues as calling window-configuration-change-hook.
> 
> But margin size changes won't have to call window-size-change-functions, 
> right? Then it might be possible to move all the heavy consumers to this 
> hook, leaving window-configuration-change-hook relatively lightweight 
> (right)?

For some value of "lightweight".  The ones that I think about would
want to change window dimensions, e.g. by enlarging or shrinking the
margins, and that is not "lightweight", as I explained up-thread.

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org, joostkremers@fastmail.fm
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 15 Nov 2017 23:49:05 +0200
> 
> >> If not, let's put all padding on the outside and be done with that concern.
> > 
> > This is doable, but the implementation will be more complex.
> > Remember: the display engine lays out stuff left to right.  So padding
> > what's left after we are done with all of the "columns" is easy;
> > padding _before_ requires that you either compute all the widths in
> > advance, or that you come back after laying out the columns and insert
> > the stretch before it, moving all the glyphs to the right.
> 
> Sounds straightforward to me. Since we know the sizes of all the columns 
> in advance, we can just substract them from the target total width, and 
> pad with the resulting number of spaces.

Maybe, but IME things are rarely so easy.

> Further, even though we have a separate accessor for its width 
> (line-number-display-width), if a package depends on it and needs to 
> draw something based on its value, it should want to be notified when 
> there is a change (*). window-configuration-change-hook seems natural. 
> Unless we have a separate hook for that?

The way this feature is designed and implemented, it doesn't lend
itself easily to hooks, primarily because it works in the inner-most
level of redisplay.

> Can't you save the necessary data to a variable, finish redisplay,
> and then run the hook (if the data says so)?

That would be pointless, because there are already hooks which work
before redisplay or after it finishes.  All such a hook needs to do is
compare the value returned by line-number-display-width with the last
value it saw.  That's what I did in tabulated-list-mode, which has
some unique requirements in this area.  Avoiding the comparison
doesn't justify a new hook.

And anyway, what do you envision that a hook function will want to do?
Most probably, it will want to change the window dimensions, or affect
what's on display in some other way, which means an immediate second
redisplay cycle.  So we gain nothing by making the display engine call
the hook.

> It's somewhat hypothetical, but I'd like to refer to (*) above. That is, 
> somebody will probably ask for that anyway, sooner or later.

Somebody already did, and I declined for now, because I think the same
effect can be achieved via existing hooks.

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/16/17 5:39 PM, Eli Zaretskii wrote:

>>> Regardless of the reasons for this change, it doesn't affect the issue
>>> at hand: calling window-size-change-functions from inside redisplay
>>> has the same issues as calling window-configuration-change-hook.

Could you clarify what is the exact issue, BTW?

>> But margin size changes won't have to call window-size-change-functions,
>> right? Then it might be possible to move all the heavy consumers to this
>> hook, leaving window-configuration-change-hook relatively lightweight
>> (right)?
> 
> For some value of "lightweight".  The ones that I think about would
> want to change window dimensions, e.g. by enlarging or shrinking the
> margins, and that is not "lightweight", as I explained up-thread.

Are window-size-change-functions really supposed to be called when only 
the margin size has changed, but not the window's outer dimensions?

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/16/17 5:50 PM, Eli Zaretskii wrote:

>> Sounds straightforward to me. Since we know the sizes of all the columns
>> in advance, we can just substract them from the target total width, and
>> pad with the resulting number of spaces.
> 
> Maybe, but IME things are rarely so easy.

In that case, I think we've made sure they are easy (by requiring all 
columns' widths to be known in advance).

>> Further, even though we have a separate accessor for its width
>> (line-number-display-width), if a package depends on it and needs to
>> draw something based on its value, it should want to be notified when
>> there is a change (*). window-configuration-change-hook seems natural.
>> Unless we have a separate hook for that?
> 
> The way this feature is designed and implemented, it doesn't lend
> itself easily to hooks, primarily because it works in the inner-most
> level of redisplay.

Then maybe using the margins for it will be a necessary price, with the 
corresponding performance hit (though hopefully not), just to enable the 
extensibility our users are accustomed to.

>> Can't you save the necessary data to a variable, finish redisplay,
>> and then run the hook (if the data says so)?
> 
> That would be pointless, because there are already hooks which work
> before redisplay or after it finishes.  All such a hook needs to do is
> compare the value returned by line-number-display-width with the last
> value it saw.  That's what I did in tabulated-list-mode, which has
> some unique requirements in this area.  Avoiding the comparison
> doesn't justify a new hook.

Hmm, I'm not sure it would be as pointless as you say: normally, it's 
most important to be notified of some change _eventually_, and not 
necessarily during some process such as redisplay. It would at least 
save the user the problem of puzzling out how to do this, and what to 
compare.

On the other hand, it could be the argument for margin changes not to 
run the usual hooks, because any sane called could compare margin widths 
before and after.

> And anyway, what do you envision that a hook function will want to do?
> Most probably, it will want to change the window dimensions, or affect
> what's on display in some other way, which means an immediate second
> redisplay cycle.

Affect what's on display, yes, most likely.

> So we gain nothing by making the display engine call
> the hook.

Yeah. I was suggesting to call the hook later, though.

>> It's somewhat hypothetical, but I'd like to refer to (*) above. That is,
>> somebody will probably ask for that anyway, sooner or later.
> 
> Somebody already did, and I declined for now, because I think the same
> effect can be achieved via existing hooks.

Do you have margin-using examples that likewise couldn't be "achieved 
via existing hooks" if margin changes didn't call 
window-configuration-change-hook?

bug#29279: Sharing the margins

[Joost Kremers]

On Mon, Nov 13 2017, Dmitry Gutov wrote:
> On 11/13/17 9:32 PM, Eli Zaretskii wrote:
>>> I imagine workroom-mode might have a idea where they want the 
>>> padding to
>>> end up (to the left or to the right of all columns). So 
>>> instead of
>>> co-opting the ORDINAL argument to mean "cols will  total cols"
>>
>> We need to study the needs of potential users, no doubt, before
>> finalizing the API.
>
> Inviting Joost into the discussion.

Apologies for replying so late, I've been quite busy. I must also 
admit that even though I've been going over the thread, I'm not 
entirely sure what it is you were hoping to find out by inviting 
me into the discussion.

My personal itch prompting me to write writeroom-mode was wanting 
to get rid of any and all on-screen distractions while writing. So 
I want a full-screen Emacs frame (so none of the window manager's 
notification areas, panels, task bars etc. are visible) with only 
a single window displaying the buffer I'm working on, with the 
text centred on the screen, because modern monitors are too wide 
to use the entire width when writing texts.

So if I understand correctly what you're talking about, I don't 
really care whether the margins are padded on the right or the 
left, because I have nothing in the margins, so I wouldn't be able 
to tell.

I know from user feedback that people do use writeroom-mode with 
e.g., (n)linum-mode, or other packages that display stuff in the 
margins. Among them, some want whatever is being displayed in the 
margins to be close to the text, while other want it at the edge 
of the screen.

So, *if* I'm understanding the issue correctly, it would be best 
if the padding side were user-configurable. If that's not an 
option, either side will do, because there doesn't seem to be any 
clear preference either way.

HTH



-- 
Joost Kremers
Life has its moments

bug#29279: Sharing the margins

[Dmitry Gutov]

Hi Joost!

On 11/19/17 2:47 AM, Joost Kremers wrote:

> So if I understand correctly what you're talking about, I don't really 
> care whether the margins are padded on the right or the left, because I 
> have nothing in the margins, so I wouldn't be able to tell.
> 
> I know from user feedback that people do use writeroom-mode with e.g., 
> (n)linum-mode, or other packages that display stuff in the margins. 
> Among them, some want whatever is being displayed in the margins to be 
> close to the text, while other want it at the edge of the screen.
> 
> So, *if* I'm understanding the issue correctly, it would be best if the 
> padding side were user-configurable. If that's not an option, either 
> side will do, because there doesn't seem to be any clear preference 
> either way.

That's what we've been wondering about. Thanks!

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org, joostkremers@fastmail.fm
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 19 Nov 2017 01:46:29 +0200
> 
> On 11/16/17 5:39 PM, Eli Zaretskii wrote:
> 
> >>> Regardless of the reasons for this change, it doesn't affect the issue
> >>> at hand: calling window-size-change-functions from inside redisplay
> >>> has the same issues as calling window-configuration-change-hook.
> 
> Could you clarify what is the exact issue, BTW?

The same ones described up-thread: the adverse effects of calling
these due to change in line-number width.

> >> But margin size changes won't have to call window-size-change-functions,
> >> right? Then it might be possible to move all the heavy consumers to this
> >> hook, leaving window-configuration-change-hook relatively lightweight
> >> (right)?
> > 
> > For some value of "lightweight".  The ones that I think about would
> > want to change window dimensions, e.g. by enlarging or shrinking the
> > margins, and that is not "lightweight", as I explained up-thread.
> 
> Are window-size-change-functions really supposed to be called when only 
> the margin size has changed, but not the window's outer dimensions?

It's the other way around: in the cases discussed here the hooks will
most probably _want_ to change the margins or some other window
dimensions.

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org, joostkremers@fastmail.fm
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 19 Nov 2017 01:55:23 +0200
> 
> > The way this feature is designed and implemented, it doesn't lend
> > itself easily to hooks, primarily because it works in the inner-most
> > level of redisplay.
> 
> Then maybe using the margins for it will be a necessary price, with the 
> corresponding performance hit (though hopefully not), just to enable the 
> extensibility our users are accustomed to.

Not from my POV.  The extensibility is already there, you can look at
the bundled modes which were adapted to this feature.

> >> Can't you save the necessary data to a variable, finish redisplay,
> >> and then run the hook (if the data says so)?
> > 
> > That would be pointless, because there are already hooks which work
> > before redisplay or after it finishes.  All such a hook needs to do is
> > compare the value returned by line-number-display-width with the last
> > value it saw.  That's what I did in tabulated-list-mode, which has
> > some unique requirements in this area.  Avoiding the comparison
> > doesn't justify a new hook.
> 
> Hmm, I'm not sure it would be as pointless as you say: normally, it's 
> most important to be notified of some change _eventually_, and not 
> necessarily during some process such as redisplay. It would at least 
> save the user the problem of puzzling out how to do this, and what to 
> compare.

What is the difference between being notified when the width changed,
and figuring out when it was changed by comparing two numbers?

> On the other hand, it could be the argument for margin changes not to 
> run the usual hooks, because any sane called could compare margin widths 
> before and after.

It's the other way around: we are talking about hooks that would like
to change the margins.

> > And anyway, what do you envision that a hook function will want to do?
> > Most probably, it will want to change the window dimensions, or affect
> > what's on display in some other way, which means an immediate second
> > redisplay cycle.
> 
> Affect what's on display, yes, most likely.
> 
> > So we gain nothing by making the display engine call
> > the hook.
> 
> Yeah. I was suggesting to call the hook later, though.

Same difference.

> >> It's somewhat hypothetical, but I'd like to refer to (*) above. That is,
> >> somebody will probably ask for that anyway, sooner or later.
> > 
> > Somebody already did, and I declined for now, because I think the same
> > effect can be achieved via existing hooks.
> 
> Do you have margin-using examples that likewise couldn't be "achieved 
> via existing hooks" if margin changes didn't call 
> window-configuration-change-hook?

I didn't try to look.

Anyway, could we please stop mixing these two issues?  This discussion
is about margin sharing, not about a (missing) hook for changes in
line-number width.

bug#29279: Sharing the margins

[Dmitry Gutov]

On 11/19/17 5:34 PM, Eli Zaretskii wrote:

>> Hmm, I'm not sure it would be as pointless as you say: normally, it's
>> most important to be notified of some change _eventually_, and not
>> necessarily during some process such as redisplay. It would at least
>> save the user the problem of puzzling out how to do this, and what to
>> compare.
> 
> What is the difference between being notified when the width changed,
> and figuring out when it was changed by comparing two numbers?

The latter approach requires more user code.

I'd say it's a bit like the difference between having a 
post-self-insert-hook and not having it, asking users to rely on 
post-command-hook.

>> On the other hand, it could be the argument for margin changes not to
>> run the usual hooks, because any sane called could compare margin widths
>> before and after.
> 
> It's the other way around: we are talking about hooks that would like
> to change the margins.

I don't know these uses, so can't really comment, I guess.

> Anyway, could we please stop mixing these two issues?

Sure. I think we've collected the requirements by now, though. Time for 
an implementation?

 > This discussion
 > is about margin sharing, not about a (missing) hook for changes in
 > line-number width.

IMHO the question of removing the irregularity introduced by the native 
line numbers using the new shared margins support is a fairly important 
one. But it's not urgent, of course.

bug#29279: Sharing the margins

[Eli Zaretskii]

> Cc: 29279@debbugs.gnu.org, joostkremers@fastmail.fm
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 21 Nov 2017 00:23:26 +0200
> 
> > What is the difference between being notified when the width changed,
> > and figuring out when it was changed by comparing two numbers?
> 
> The latter approach requires more user code.

Yes, but not a lot more, and the code is fairly straightforward.

> I'd say it's a bit like the difference between having a 
> post-self-insert-hook and not having it, asking users to rely on 
> post-command-hook.

You know me well enough: if the hook you are asking for were easy to
implement safely, it would have been written long ago.

> I think we've collected the requirements by now, though. Time for an
> implementation?

Yes.