Re: [NS] Emacs 28 (IOSurface) renders much slower than Emacs 27 on macOS Big Sur

[Alan Third <alan@idiocy.org>]

On Sun, May 02, 2021 at 09:40:03PM +0800, Yiming Chen wrote:
> Hi there!
> I know that we've switched to a IOSurface based rendering for nsterm in
> master branch.
> Now master renders more precisely than emacs-27 (i.e. less flickers or
> blank screens.)
> But I feel that its rendering performance is much worse due to the new
> buffering mechanism backed by IOSurface.

Yup. That's the trade-off.

> I think this poor rendering performance obscures the improvements
> brought by native-compile.
> So I want to fix this performance issue and make Emacs runs on macOS as
> fast as on Linux.

I've spent over a year on-and-off on this. Good luck.

> 1. Try to restore the nsterm behavior from Emacs27 (which is much
>    simpler than master IMHO), and start again from there
> 
>    I've reverted the code and made it compile (see
>    <https://github.com/emacs-mirror/emacs/compare/master...dsdshcym:revert-to-emacs-27-nsterm>)
>    But Emacs failed to start and raised an error `Font ‘Menlo’ is not
>    defined'

That error must be something unrelated, as the drawing buffer changes
didn't touch the font code... I've got no suggestions on where to
start, though. My knowledge of font handling is non-existent.

Actually, a quick look shows that you've enabled the nsfont backend
which just doesn't work in macOS, so I'd bet that's something to
investigate.

I notice quite a few other problems... If I'm being honest, master has
diverged so much from emacs-27 that if I were to do this I'd probably
do it manually rather than try reverting commits.

And I disagree that master is more complex than emacs-27. The endless
little work-arounds to try and get the emacs-27 rendering working made
it a much more brittle setup.

> 2. Use layer-based view for rendering
> 
>    I found that macvim also faced the similar issue when macOS 10.14
>    came out (<https://github.com/macvim-dev/macvim/issues/751>).
>    They used NSImage to implement a double rendering solution
>    (<https://github.com/macvim-dev/macvim/pull/757/files>), which was
>    also slower than before.
>    Then they switched from NSImage to layer-based view in
>    <https://github.com/macvim-dev/macvim/commit/dba6293677e0633917e3054cfddec1293e5ab3fb>.
>    So I wonder if we can do the same.
>
> 3. Switch to CALayer (Core Animation) to use GPU (Metal) rendering for
>    even better performance than before
> 
>    If 2 (layer-based view) is possible, then I hope we can leverage
>    CALayer to make nsterm renders even faster.

So... yes. The IOSurface system is already using layer-based drawing.
Have a look at updateLayer, for example.

I'm not sure that CALayer really gives you anything in terms of
performance for this sort of thing. CoreAnimation is all about moving
different layers about efficiently, afaict, which isn't something we
need to do in Emacs.

And bear in mind that in order to send an image to the Metal renderer
you still need to transfer it to the GPU, and Apple recommend using
IOSurface as it's the most efficient way to do that.

AND Yamamoto Mitsuharu disabled Metal rendering in the Mac port as he
found the simple IOSurface based layer backing was faster.

AND the Chromium developers decided to go with the simple IOSurface
based layer backing for performance reasons too.

One place that a big improvement could be found is in disabling double
buffering and only drawing to one IOSurface, as you don't need to copy
the contents of the different IOSurfaces to each other. But I found
that produced unacceptable tearing effects.

But all that said, it could be that copying viewWillDraw from master
to emacs-27 could solve some of the flickering. In my experimentation
I don't believe it does, but I've never been very good at inducing the
"flickering".

-- 
Alan Third