redisplay and expose_frame

redisplay and expose_frame

[Alan Third]

Hi all,

I’m looking for a little bit of guidance as to how exactly Emacs draws
to a GUI frame.

The background here is that I’m working on splitting the NS port’s GUI
code into its own thread, and while working on that I noticed that
Apple have deprecated the method we use to draw, so I thought that
since I’m working with the code anyway, I may as well try to
‘modernise’ it.

The ‘new’ way of drawing to the screen works thus:

 1. When you want to update a portion of the screen you mark it as
    dirty.

 2. At some point you have to call [emacs-frame display], which does
    some set‐up and then calls drawRect with an argument that is a
    rectangle showing which parts of the screen need to be updated.

 3. *All* the drawing *must* occur in drawRect.

 4. done.

At the moment NS Emacs calls expose_frame from within drawRect which
works well, however it also does some drawing outside of drawRect, and
I strongly suspect that redisplay_internal must, somehow, call
expose_frame (or an equivalent), but I can’t see where.

Does Emacs explicitly draw to the screen as part of redisplay? If so,
where is it doing it?

Thanks!
-- 
Alan Third

Re: redisplay and expose_frame

[Eli Zaretskii]

> Date: Sun, 22 Jul 2018 13:44:18 +0100
> From: Alan Third <alan@idiocy.org>
> 
> At the moment NS Emacs calls expose_frame from within drawRect which
> works well, however it also does some drawing outside of drawRect

Where is the code that draws outside of drawRect?  Can you point me to
it?

> and I strongly suspect that redisplay_internal must, somehow, call
> expose_frame (or an equivalent)

redisplay_internal does NOT call expose_frame.  See the large
commentary at the beginning of xdisp.c, which explains that
expose_frame is a separate entry point into redisplay.

> Does Emacs explicitly draw to the screen as part of redisplay?

Yes, for some value of "draw".

> If so, where is it doing it?

When redisplay_internal is almost done, you will see it call
update_frame or update_window.  These two functions are implemented in
dispnew.c, and what they do is this:

  . compare the "current" glyph matrix with the "desired" one, and
    determine which part(s) of each window need to be redrawn
  . for each screen line that needs to be redrawn, call the
    terminal-specific hook function to actually redraw (see
    update_text_area)

However, what exactly do the hooks such as write_glyphs and
clear_end_of_line do is up to the terminal code (in your case,
nsterm.m); it could do something very different from actually drawing
characters.

By contrast, expose_frame uses the "current" glyph matrices, assuming
(after it verifies) that the glyph matrix accurately describes what
should be on the glass, and then calls those terminal-specific hook
functions directly, see draw_glyphs.  It bypasses redisplay_internal,
whose job is to generate the "desired" glyph matrix, because
expose_frame doesn't need that step.

Let me know if you need more help with untangling this web.

Re: redisplay and expose_frame

[YAMAMOTO Mitsuharu]

On Sun, 22 Jul 2018 21:44:18 +0900,
Alan Third wrote:

> The background here is that I’m working on splitting the NS port’s GUI
> code into its own thread, and while working on that I noticed that
> Apple have deprecated the method we use to draw, so I thought that
> since I’m working with the code anyway, I may as well try to
> ‘modernise’ it.

A related thread can be found around
http://lists.gnu.org/archive/html/emacs-devel/2010-07/msg00821.html .

Mojave, the next version of macOS, already prohibits drawing outside
"drawRect:" by default if the executable was linked on that version.
An attempt to draw outside "drawRect:" triggers invalidation of all
the contents instead, and it is completely redrawn (even for a single
cursor movement) via "drawRect:" at the next cycle of the event loop.
This is kinda usable for small frames on recent machines, but not
efficient of course, especially for full screen frames.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: redisplay and expose_frame

[Alan Third]

On Mon, Jul 23, 2018 at 06:58:22PM +0900, YAMAMOTO Mitsuharu wrote:
> On Sun, 22 Jul 2018 21:44:18 +0900,
> Alan Third wrote:
> 
> > The background here is that I’m working on splitting the NS port’s GUI
> > code into its own thread, and while working on that I noticed that
> > Apple have deprecated the method we use to draw, so I thought that
> > since I’m working with the code anyway, I may as well try to
> > ‘modernise’ it.
> 
> A related thread can be found around
> http://lists.gnu.org/archive/html/emacs-devel/2010-07/msg00821.html .

That’s interesting, and confirms some of what I’d thought. I take it
nothing ever came of it, though?

> Mojave, the next version of macOS, already prohibits drawing outside
> "drawRect:" by default if the executable was linked on that version.
> An attempt to draw outside "drawRect:" triggers invalidation of all
> the contents instead, and it is completely redrawn (even for a single
> cursor movement) via "drawRect:" at the next cycle of the event loop.
> This is kinda usable for small frames on recent machines, but not
> efficient of course, especially for full screen frames.

This doesn’t sound good, and I wonder if bug#31904 is related,
although it doesn’t sound like it should be.

Is the Mac port affected by this, or does it avoid this Cocoa stuff?

I suppose, to avoid changes in redisplay code, I could make the NS
drawing functions detect whether they are running in the main thread
(or perhaps they can just ask whether they’re able to draw or not),
and if they’re not then invalidate the relevant rectangle, and if they
are then do the actual drawing task.

Then call display for each frame to force an expose ‘event’ at the end
of redisplay.

-- 
Alan Third

Re: redisplay and expose_frame

[YAMAMOTO Mitsuharu]

On Wed, 25 Jul 2018 05:42:20 +0900,
Alan Third wrote:

> > > The background here is that I’m working on splitting the NS port’s GUI
> > > code into its own thread, and while working on that I noticed that
> > > Apple have deprecated the method we use to draw, so I thought that
> > > since I’m working with the code anyway, I may as well try to
> > > ‘modernise’ it.
> > 
> > A related thread can be found around
> > http://lists.gnu.org/archive/html/emacs-devel/2010-07/msg00821.html .
> 
> That’s interesting, and confirms some of what I’d thought. I take it
> nothing ever came of it, though?

AFAIK, nothing public.  It's been difficult for me to tell how
important the "expose" handler is to developers on other platforms
(e.g., the thread starting from
http://lists.gnu.org/archive/html/emacs-devel/2013-04/msg00392.html).

> > Mojave, the next version of macOS, already prohibits drawing outside
> > "drawRect:" by default if the executable was linked on that version.
> > An attempt to draw outside "drawRect:" triggers invalidation of all
> > the contents instead, and it is completely redrawn (even for a single
> > cursor movement) via "drawRect:" at the next cycle of the event loop.
> > This is kinda usable for small frames on recent machines, but not
> > efficient of course, especially for full screen frames.
> 
> This doesn’t sound good, and I wonder if bug#31904 is related,
> although it doesn’t sound like it should be.
> 
> Is the Mac port affected by this, or does it avoid this Cocoa stuff?

There were several problems, but non-performance issues are mostly
solved in the "work" branch of the Mac port repository
(https://bitbucket.org/mituharu/emacs-mac).  It now works as in a
"kinda usable" way with the whole invalidation mentioned above.

> I suppose, to avoid changes in redisplay code, I could make the NS
> drawing functions detect whether they are running in the main thread
> (or perhaps they can just ask whether they’re able to draw or not),
> and if they’re not then invalidate the relevant rectangle, and if they
> are then do the actual drawing task.
> 
> Then call display for each frame to force an expose ‘event’ at the end
> of redisplay.

Back in early 2007, I privately experimented something like that
(without threading) with "modern Carbon" (HIWindow), where drawing
outside the "expose" callback was deprecated like the current
situation for Cocoa.  I could actually find a platform-independent bug
that was otherwise difficult to find/reproduce
(http://lists.gnu.org/archive/html/emacs-devel/2007-04/msg01279.html)
by this experiment, but the overall performance was not quite
satisfactory then.  So, I was surprised to see that (the "work" branch
of) the Mac port with the whole invalidation was "kinda usable" on
recent machines running Mojave.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: redisplay and expose_frame

[Alan Third]

On Wed, Jul 25, 2018 at 06:33:35PM +0900, YAMAMOTO Mitsuharu wrote:
> > I suppose, to avoid changes in redisplay code, I could make the NS
> > drawing functions detect whether they are running in the main thread
> > (or perhaps they can just ask whether they’re able to draw or not),
> > and if they’re not then invalidate the relevant rectangle, and if they
> > are then do the actual drawing task.
> > 
> > Then call display for each frame to force an expose ‘event’ at the end
> > of redisplay.
> 
> Back in early 2007, I privately experimented something like that
> (without threading) with "modern Carbon" (HIWindow), where drawing
> outside the "expose" callback was deprecated like the current
> situation for Cocoa.  I could actually find a platform-independent bug
> that was otherwise difficult to find/reproduce
> (http://lists.gnu.org/archive/html/emacs-devel/2007-04/msg01279.html)
> by this experiment, but the overall performance was not quite
> satisfactory then.  So, I was surprised to see that (the "work" branch
> of) the Mac port with the whole invalidation was "kinda usable" on
> recent machines running Mojave.

I’ve implemented this (without threading) and, like you say, the
performance is pretty good. I can’t actually see any difference on my
Mac.

Unfortunately GNUstep is not as happy. It runs MUCH slower, to the
extent that scrolling really doesn’t work at all. Turning off the
menubar improves things, but not enough.

Mind you, looking at the scrolling code again now, I think it’s
forcing essentially the whole window to be redrawn anyway, so that
won’t help.
-- 
Alan Third