Drawing UI elements behind text

Drawing UI elements behind text

[Cecilio Pardo]

Hello,

I am working on a mechanism to draw things on the background, behind
the text of a buffer.

This would allow to draw UI elements (such as indent lines or a fill
column indicator) and eye candy like background images or colored
windows.

This two first features (indent lines and fill indicator) are controlled
by declaring a list of vertical segments linked to buffer
positions.

Is there any other feature that could be implemented like this? Not just
by segments, but by any graphics drawn behind the text, vectors or
bitmap.

Thanks.

Re: Drawing UI elements behind text

[Jim Porter]

On 11/10/2024 8:39 AM, Cecilio Pardo wrote:
> Is there any other feature that could be implemented like this? Not just
> by segments, but by any graphics drawn behind the text, vectors or
> bitmap.

It's not "behind" text, but with some work on setting per-line fill 
widths, you could perhaps use this in EWW to display images alongside 
text (like with the "float: right" CSS rule).

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Sun, 10 Nov 2024 10:09:04 -0800
> From: Jim Porter <jporterbugs@gmail.com>
> 
> On 11/10/2024 8:39 AM, Cecilio Pardo wrote:
> > Is there any other feature that could be implemented like this? Not just
> > by segments, but by any graphics drawn behind the text, vectors or
> > bitmap.
> 
> It's not "behind" text, but with some work on setting per-line fill 
> widths, you could perhaps use this in EWW to display images alongside 
> text (like with the "float: right" CSS rule).

Given the right Lisp API (which is not easy to come up with, AFAIR),
this can be done today, because Emacs already knows how to show only a
part of an image on a screen line.  So we need a way to tell Emacs
"split this image's display between these N lines".

Re: Drawing UI elements behind text

[Jim Porter]

On 11/10/2024 10:44 AM, Eli Zaretskii wrote:
> Given the right Lisp API (which is not easy to come up with, AFAIR),
> this can be done today, because Emacs already knows how to show only a
> part of an image on a screen line.  So we need a way to tell Emacs
> "split this image's display between these N lines".

I've thought about this before. Doing it via sliced images scares me a 
bit since it would require extreme care to make sure everything lines up 
correctly. For example, what if the text has mixed font sizes resulting 
in different heights for some lines? Or what about applying text scaling?

I suppose if we deemed this feature important, we could probably address 
all of those, but I'm not sure it's worth the effort. In fact, I'm not 
sure it would be worth using this new feature to help EWW out here 
either (though that's not to say the feature isn't worthwhile 
elsewhere). I just though I'd mention it as another case that's worth at 
least a cursory look.

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Sun, 10 Nov 2024 11:06:13 -0800
> Cc: cpardo@imayhem.com, emacs-devel@gnu.org
> From: Jim Porter <jporterbugs@gmail.com>
> 
> On 11/10/2024 10:44 AM, Eli Zaretskii wrote:
> > Given the right Lisp API (which is not easy to come up with, AFAIR),
> > this can be done today, because Emacs already knows how to show only a
> > part of an image on a screen line.  So we need a way to tell Emacs
> > "split this image's display between these N lines".
> 
> I've thought about this before. Doing it via sliced images scares me a 
> bit since it would require extreme care to make sure everything lines up 
> correctly. For example, what if the text has mixed font sizes resulting 
> in different heights for some lines?

Once the display engine knows at what X coordinate the image slice
should start, it can stop laying out other display elements and switch
to the slice when that X coordinate is reached.

> Or what about applying text scaling?

How is this different from what we do today when text-scaling happens
in a buffer where images are displayed together with text?

> I suppose if we deemed this feature important, we could probably address 
> all of those, but I'm not sure it's worth the effort.

What alternative do we have?  Showing the images alongside text
without letting the images scroll with the text (vertically and
horizontally) will not fly.  How do you scroll the images with the
text except by displaying them as part of the "normal" display layout?

Re: Drawing UI elements behind text

[Jim Porter]

On 11/10/2024 11:22 AM, Eli Zaretskii wrote:
> Once the display engine knows at what X coordinate the image slice
> should start, it can stop laying out other display elements and switch
> to the slice when that X coordinate is reached.

I was thinking more about the Y coordinates and having gaps between 
consecutive slices if we made a mistake. That's probably fixable, but it 
seems like it would require a lot of care to avoid bugs.

>> Or what about applying text scaling?
> 
> How is this different from what we do today when text-scaling happens
> in a buffer where images are displayed together with text?

I'm just worried about this due to seeing some similar bugs come up in 
the visual-wrap-prefix-mode code.

>> I suppose if we deemed this feature important, we could probably address
>> all of those, but I'm not sure it's worth the effort.
> 
> What alternative do we have?  Showing the images alongside text
> without letting the images scroll with the text (vertically and
> horizontally) will not fly.  How do you scroll the images with the
> text except by displaying them as part of the "normal" display layout?

Personally, I'm ok with saying we just don't support laying out images 
like this in EWW. For it to have a practical effect, we'd probably need 
to have more-complete CSS support anyway. That's not easy either (though 
I have some ideas about how we could do it if we thought it would be 
worthwhile).

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Sun, 10 Nov 2024 11:29:27 -0800
> Cc: cpardo@imayhem.com, emacs-devel@gnu.org
> From: Jim Porter <jporterbugs@gmail.com>
> 
> On 11/10/2024 11:22 AM, Eli Zaretskii wrote:
> > Once the display engine knows at what X coordinate the image slice
> > should start, it can stop laying out other display elements and switch
> > to the slice when that X coordinate is reached.
> 
> I was thinking more about the Y coordinates and having gaps between 
> consecutive slices if we made a mistake. That's probably fixable, but it 
> seems like it would require a lot of care to avoid bugs.

Yes, it requires care.

> >> Or what about applying text scaling?
> > 
> > How is this different from what we do today when text-scaling happens
> > in a buffer where images are displayed together with text?
> 
> I'm just worried about this due to seeing some similar bugs come up in 
> the visual-wrap-prefix-mode code.

Bugs will always be with us, no way around that.

> >> I suppose if we deemed this feature important, we could probably address
> >> all of those, but I'm not sure it's worth the effort.
> > 
> > What alternative do we have?  Showing the images alongside text
> > without letting the images scroll with the text (vertically and
> > horizontally) will not fly.  How do you scroll the images with the
> > text except by displaying them as part of the "normal" display layout?
> 
> Personally, I'm ok with saying we just don't support laying out images 
> like this in EWW. For it to have a practical effect, we'd probably need 
> to have more-complete CSS support anyway. That's not easy either (though 
> I have some ideas about how we could do it if we thought it would be 
> worthwhile).

This entire sub-thread is about how to provide such a feature;
leaving it unsupported is easy.  If we do want to provide it, I think
going the image-slice way is a better alternative.

Re: Drawing UI elements behind text

[JD Smith]

> On Nov 10, 2024, at 11:39 AM, Cecilio Pardo <cpardo@imayhem.com> wrote:
> 
> Hello,
> 
> I am working on a mechanism to draw things on the background, behind
> the text of a buffer.
> 
> This would allow to draw UI elements (such as indent lines or a fill
> column indicator) and eye candy like background images or colored
> windows.

I support this idea.  For indent-bars, I use display :stipple as a "separate band" of graphical information that can lie atop text in a buffer, but remain independent of it.  Stipples are flexible and very performant, but have a few issues:

- There is inconsistent stipple support across emacs versions.
- Stipple patterns begin at the frame origin, so up-to-date window-specific remapping is required to prevent unwanted pattern aliasing.  
- Aligning stipples vertically requires fixed space content.

More commonly, modes also make use of :box styling, e.g. for producing boxed "labels", as in org-modern.

An API for drawing simple styled lines, rounded corners, boxes etc. — either in absolute window coordinates, or associated with, but "behind" text in the buffer — would represent a real step forward. 

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 25/11/2024 0:37, JD Smith wrote:
> I support this idea.  For indent-bars, I use display :stipple as a "separate band" of graphical information that can lie atop text in a buffer, but remain independent of it.  Stipples are flexible and very performant, but have a few issues:
> 
> - There is inconsistent stipple support across emacs versions.
> - Stipple patterns begin at the frame origin, so up-to-date window-specific remapping is required to prevent unwanted pattern aliasing.
> - Aligning stipples vertically requires fixed space content.

Stipples also disable the scrolling of buffer pixels, affecting 
performance of redisplay.

> 
> More commonly, modes also make use of :box styling, e.g. for producing boxed "labels", as in org-modern.
> 
> An API for drawing simple styled lines, rounded corners, boxes etc. — either in absolute window coordinates, or associated with, but "behind" text in the buffer — would represent a real step forward.

I haven't found a way to draw behind the text that is not too 
complicated or affects redisplay too much. So now the idea is drawing 
over the text. For the applications I have in mind this works equally well.

The starting drawing elements are going to be vertical and horizontal 
segments, with color (maybe with alpha), width and different patterns. 
They can be placed on pixel positions or character positions when using 
fixed size fonts, using floating point numbers so you can put, for 
example, indent lines in the middle of characters.

They are associated with a buffer, and have a 'source' marker so the 
facility can be shared by several programs.

Any ideas for more elements or applications are welcome.

--
Cecilio Pardo.

Re: Drawing UI elements behind text

[JD Smith]

--

> On Nov 26, 2024, at 6:19 PM, Cecilio Pardo <cpardo@imayhem.com> wrote:
> 
> On 25/11/2024 0:37, JD Smith wrote:
>> I support this idea.  For indent-bars, I use display :stipple as a "separate band" of graphical information that can lie atop text in a buffer, but remain independent of it.  Stipples are flexible and very performant, but have a few issues:
>> - There is inconsistent stipple support across emacs versions.
>> - Stipple patterns begin at the frame origin, so up-to-date window-specific remapping is required to prevent unwanted pattern aliasing.
>> - Aligning stipples vertically requires fixed space content.
> 
> Stipples also disable the scrolling of buffer pixels, affecting performance of redisplay.

On all ports?  I didn’t know this, though haven’t had any related issues. 

>> More commonly, modes also make use of :box styling, e.g. for producing boxed "labels", as in org-modern.
>> An API for drawing simple styled lines, rounded corners, boxes etc. — either in absolute window coordinates, or associated with, but "behind" text in the buffer — would represent a real step forward.
> 
> I haven't found a way to draw behind the text that is not too complicated or affects redisplay too much. So now the idea is drawing over the text. For the applications I have in mind this works equally well.

For mine as well that would work, but features like this are added so rarely it makes sense to make it robust for various uses, if possible.  

> The starting drawing elements are going to be vertical and horizontal segments, with color (maybe with alpha), width and different patterns.

Nice.  Is there a patch to try?  Bit patterns or similar are the most flexible.  

> They can be placed on pixel positions or character positions when using fixed size fonts, using floating point numbers so you can put, for example, indent lines in the middle of characters.

Would they be associated with buffer text and move with it (aka :box), or fixed to the buffer window?  What API did you have in mind?

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Wed, 27 Nov 2024 00:19:43 +0100
> Cc: emacs-devel <emacs-devel@gnu.org>
> From: Cecilio Pardo <cpardo@imayhem.com>
> 
> I haven't found a way to draw behind the text that is not too 
> complicated or affects redisplay too much. So now the idea is drawing 
> over the text. For the applications I have in mind this works equally well.
> 
> The starting drawing elements are going to be vertical and horizontal 
> segments, with color (maybe with alpha), width and different patterns. 
> They can be placed on pixel positions or character positions when using 
> fixed size fonts, using floating point numbers so you can put, for 
> example, indent lines in the middle of characters.
> 
> They are associated with a buffer, and have a 'source' marker so the 
> facility can be shared by several programs.

How/whether are these drawing elements connected to the glyph
matrices?

If they are completely disconnected, how does this work when the
window is scrolled several lines?

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 27/11/2024 15:36, Eli Zaretskii wrote:
>> Date: Wed, 27 Nov 2024 00:19:43 +0100
>> Cc: emacs-devel <emacs-devel@gnu.org>
>> From: Cecilio Pardo <cpardo@imayhem.com>
>>
>> I haven't found a way to draw behind the text that is not too
>> complicated or affects redisplay too much. So now the idea is drawing
>> over the text. For the applications I have in mind this works equally well.
>>
>> The starting drawing elements are going to be vertical and horizontal
>> segments, with color (maybe with alpha), width and different patterns.
>> They can be placed on pixel positions or character positions when using
>> fixed size fonts, using floating point numbers so you can put, for
>> example, indent lines in the middle of characters.
>>
>> They are associated with a buffer, and have a 'source' marker so the
>> facility can be shared by several programs.
> 
> How/whether are these drawing elements connected to the glyph
> matrices?
> 
> If they are completely disconnected, how does this work when the
> window is scrolled several lines?

They are anchored at locations on the buffer, specified by line/column. 
This is transformed into pixels using the size of the default char cell 
for the buffer.  The buffer scroll position on each window is used to 
compute the final position on the frame/screen.

For buffers that use different font sizes or images this is probably not 
very useful. Direct pixel positions can be used too.

They are drawn after switching the back paint buffer, so that the 
redisplay result is left alone.

At first I tried intercepting the background clearing to draw behind the 
text. But to allow for redisplay to reuse the output I had to redraw the 
glyphs affected with a clear background, and this was too costly in the 
worst cases.

I will have a patch in a few days.

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 27/11/2024 3:31, JD Smith wrote:

>> Stipples also disable the scrolling of buffer pixels, affecting performance of redisplay.
> 
> On all ports?  I didn’t know this, though haven’t had any related issues.

Yes, on all ports. I would just degrade performance on certain cases.

>> The starting drawing elements are going to be vertical and horizontal segments, with color (maybe with alpha), width and different patterns.
> 
> Nice.  Is there a patch to try?  Bit patterns or similar are the most flexible.

In a few days.

> Would they be associated with buffer text and move with it (aka :box), or fixed to the buffer window?  What API did you have in mind?

They are associated with buffer positions, no need for text. Probably 
will add the other way too, anchoring to the window.

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Wed, 27 Nov 2024 19:28:58 +0100
> Cc: jdtsmith@gmail.com, emacs-devel@gnu.org
> From: Cecilio Pardo <cpardo@imayhem.com>
> 
> > If they are completely disconnected, how does this work when the
> > window is scrolled several lines?
> 
> They are anchored at locations on the buffer, specified by line/column. 
> This is transformed into pixels using the size of the default char cell 
> for the buffer.  The buffer scroll position on each window is used to 
> compute the final position on the frame/screen.
> 
> For buffers that use different font sizes or images this is probably not 
> very useful. Direct pixel positions can be used too.
> 
> They are drawn after switching the back paint buffer, so that the 
> redisplay result is left alone.

So you clear them up when redisplay starts and then redraw them in
their entirety when it ends, something like that?  Doesn't that slow
down redisplay, especially if there are many pixels and a large
window?

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 27/11/2024 19:58, Eli Zaretskii wrote:

>> They are drawn after switching the back paint buffer, so that the
>> redisplay result is left alone.
> 
> So you clear them up when redisplay starts and then redraw them in
> their entirety when it ends, something like that?  Doesn't that slow
> down redisplay, especially if there are many pixels and a large
> window?

Yes. We do a full frame copy on each double buffered redisplay, I expect 
we can add this with reasonable impact.

The very naive implementation I have now adds something like 5ms to each 
redisplay on a 1900px frame, and I hope to bring it down a lot from there.

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Wed, 27 Nov 2024 21:01:42 +0100
> From: Cecilio Pardo <cpardo@imayhem.com>
> 
> On 27/11/2024 19:58, Eli Zaretskii wrote:
> 
> >> They are drawn after switching the back paint buffer, so that the
> >> redisplay result is left alone.
> > 
> > So you clear them up when redisplay starts and then redraw them in
> > their entirety when it ends, something like that?  Doesn't that slow
> > down redisplay, especially if there are many pixels and a large
> > window?
> 
> Yes. We do a full frame copy on each double buffered redisplay, I expect 
> we can add this with reasonable impact.
> 
> The very naive implementation I have now adds something like 5ms to each 
> redisplay on a 1900px frame, and I hope to bring it down a lot from there.

What takes 5ms, exactly? what kind of redisplay did you trigger, and
how did you trigger it?

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 27/11/2024 21:12, Eli Zaretskii wrote:
>> Date: Wed, 27 Nov 2024 21:01:42 +0100
>> From: Cecilio Pardo <cpardo@imayhem.com>
>>
>> On 27/11/2024 19:58, Eli Zaretskii wrote:
>>
>>>> They are drawn after switching the back paint buffer, so that the
>>>> redisplay result is left alone.
>>>
>>> So you clear them up when redisplay starts and then redraw them in
>>> their entirety when it ends, something like that?  Doesn't that slow
>>> down redisplay, especially if there are many pixels and a large
>>> window?
>>
>> Yes. We do a full frame copy on each double buffered redisplay, I expect
>> we can add this with reasonable impact.
>>
>> The very naive implementation I have now adds something like 5ms to each
>> redisplay on a 1900px frame, and I hope to bring it down a lot from there.
> 
> What takes 5ms, exactly? what kind of redisplay did you trigger, and
> how did you trigger it?

A minimal redisplay, just moving the cursor around. This takes less than 
1ms, including the time to copy the back buffer to the window.

When the segment drawing is activated (with tens of thousands of them, 
though only tens of them visible in some window) the process takes 
around 5ms.

This process includes:

- Making a new bitmap, and copy the result of redisplay to it.
- Iterate trough lisp objects to prepare the segments, clip them and 
draw them to this bitmap.

There is a lot of room here to make it faster.

Re: Drawing UI elements behind text

[JD Smith]

> On Nov 27, 2024, at 1:58 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> They are anchored at locations on the buffer, specified by line/column.

Would it make sense to attach the drawn elements to text rather than row/column positions?  Otherwise you may need to recompute and redraw the entire buffer’s line content even after inserting a single character. For concreteness, suppose you are drawing a fancy box around a paragraph. 

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 27/11/2024 22:15, JD Smith wrote:
> 
>> On Nov 27, 2024, at 1:58 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>>
>> They are anchored at locations on the buffer, specified by line/column.
> 
> Would it make sense to attach the drawn elements to text rather than row/column positions?  Otherwise you may need to recompute and redraw the entire buffer’s line content even after inserting a single character. For concreteness, suppose you are drawing a fancy box around a paragraph.

Yes, that makes sense as an additional option. Maybe attach it to an 
overlay position + (x,y).

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Wed, 27 Nov 2024 21:30:54 +0100
> Cc: emacs-devel@gnu.org
> From: Cecilio Pardo <cpardo@imayhem.com>
> 
> >> The very naive implementation I have now adds something like 5ms to each
> >> redisplay on a 1900px frame, and I hope to bring it down a lot from there.
> > 
> > What takes 5ms, exactly? what kind of redisplay did you trigger, and
> > how did you trigger it?
> 
> A minimal redisplay, just moving the cursor around. This takes less than 
> 1ms, including the time to copy the back buffer to the window.
> 
> When the segment drawing is activated (with tens of thousands of them, 
> though only tens of them visible in some window) the process takes 
> around 5ms.
> 
> This process includes:
> 
> - Making a new bitmap, and copy the result of redisplay to it.
> - Iterate trough lisp objects to prepare the segments, clip them and 
> draw them to this bitmap.
> 
> There is a lot of room here to make it faster.

OK, but I was more interested in more complex redisplay-related
scenarios.  For example, what happens if you add a few lines of text
to the bottom of the window (by yanking several lines at once)?  This
should cause Emacs to scroll the window, so that the text previously
shown at the top of the window will scroll out of the viewport, and
the yanked lines will be scrolled in.  How long does this take, with
and without these segments?

Similarly, what are the timings (again, with and without these
segments) when you just move the cursor, but in a way that it goes out
of the window, or ends up in a display margin at the top of the
window, or in a partially-visible line at the bottom of the window?

Those (and other similar ones) are the scenarios where the display
engine activates its non-trivial optimization methods, and the
questions of interest related to that are:

  . do we need to disable some of the optimizations when these
    segments are on display (because their assumptions are
    incompatible with how these segments are drawn)?
  . how do these segments affect redisplay times in each such case,
    after we disable the optimizations, if any, that are incompatible
    with them?

These questions are prerequisites for deciding whether we need to make
this particular case faster by some special measures.

Thanks.

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 28/11/2024 7:53, Eli Zaretskii wrote:
> Those (and other similar ones) are the scenarios where the display
> engine activates its non-trivial optimization methods, and the
> questions of interest related to that are:
> 
>    . do we need to disable some of the optimizations when these
>      segments are on display (because their assumptions are
>      incompatible with how these segments are drawn)?
>    . how do these segments affect redisplay times in each such case,
>      after we disable the optimizations, if any, that are incompatible
>      with them?
> 
> These questions are prerequisites for deciding whether we need to make
> this particular case faster by some special measures.

The segments are drawn independently of redisplay, on top of its result, 
without affecting it (so that the next redisplay finds it exactly as it 
was). So no optimizations are disabled, and the time added by them is 
independent of what redisplay had to do.

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Thu, 28 Nov 2024 09:43:21 +0100
> Cc: emacs-devel@gnu.org
> From: Cecilio Pardo <cpardo@imayhem.com>
> 
> On 28/11/2024 7:53, Eli Zaretskii wrote:
> > Those (and other similar ones) are the scenarios where the display
> > engine activates its non-trivial optimization methods, and the
> > questions of interest related to that are:
> > 
> >    . do we need to disable some of the optimizations when these
> >      segments are on display (because their assumptions are
> >      incompatible with how these segments are drawn)?
> >    . how do these segments affect redisplay times in each such case,
> >      after we disable the optimizations, if any, that are incompatible
> >      with them?
> > 
> > These questions are prerequisites for deciding whether we need to make
> > this particular case faster by some special measures.
> 
> The segments are drawn independently of redisplay, on top of its result, 
> without affecting it (so that the next redisplay finds it exactly as it 
> was). So no optimizations are disabled, and the time added by them is 
> independent of what redisplay had to do.

I'm probably missing something: if Emacs uses the scroll_run_hook,
which bitblts a portion of the screen's pixels, doesn't that move also
the pixels of the segments?  If not, does it mean those segments'
pixels are not drawn on the screen, but somehow overlaid on it using
some video driver trick?

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 28/11/2024 10:57, Eli Zaretskii wrote:
>> The segments are drawn independently of redisplay, on top of its result,
>> without affecting it (so that the next redisplay finds it exactly as it
>> was). So no optimizations are disabled, and the time added by them is
>> independent of what redisplay had to do.
> 
> I'm probably missing something: if Emacs uses the scroll_run_hook,
> which bitblts a portion of the screen's pixels, doesn't that move also
> the pixels of the segments?  If not, does it mean those segments'
> pixels are not drawn on the screen, but somehow overlaid on it using
> some video driver trick?

This is working only if emacs is using a double buffered configuration. 
Redisplay works on the back buffer, where we don't draw anything. When 
the back buffer goes to the screen, we add the segments to the screen. 
scroll_run_hook works on the back buffer.

Re: Drawing UI elements behind text

[Eli Zaretskii]

> Date: Thu, 28 Nov 2024 11:41:25 +0100
> Cc: emacs-devel@gnu.org
> From: Cecilio Pardo <cpardo@imayhem.com>
> 
> On 28/11/2024 10:57, Eli Zaretskii wrote:
> >> The segments are drawn independently of redisplay, on top of its result,
> >> without affecting it (so that the next redisplay finds it exactly as it
> >> was). So no optimizations are disabled, and the time added by them is
> >> independent of what redisplay had to do.
> > 
> > I'm probably missing something: if Emacs uses the scroll_run_hook,
> > which bitblts a portion of the screen's pixels, doesn't that move also
> > the pixels of the segments?  If not, does it mean those segments'
> > pixels are not drawn on the screen, but somehow overlaid on it using
> > some video driver trick?
> 
> This is working only if emacs is using a double buffered configuration. 
> Redisplay works on the back buffer, where we don't draw anything. When 
> the back buffer goes to the screen, we add the segments to the screen. 
> scroll_run_hook works on the back buffer.

I see.  So the display code basically knows nothing about these
segments?  And you need to redraw them in their entirety each
redisplay cycle?

Re: Drawing UI elements behind text

[Cecilio Pardo]

On 28/11/2024 12:56, Eli Zaretskii wrote:

>> This is working only if emacs is using a double buffered configuration.
>> Redisplay works on the back buffer, where we don't draw anything. When
>> the back buffer goes to the screen, we add the segments to the screen.
>> scroll_run_hook works on the back buffer.
> 
> I see.  So the display code basically knows nothing about these
> segments?  And you need to redraw them in their entirety each
> redisplay cycle?

Yes, it knows nothing.

I still don't have the final form of the drawing, but probably yes, 
redraw on each redisplay.