Re: [NS] Emacs 28 (IOSurface) renders much slower than Emacs 27 on macOS Big Sur

[Yiming Chen <me@yiming.dev>]

Hi Alan,

Thanks for your reply and sorry for my late response.

I've been using 
https://github.com/emacs-mirror/emacs/commit/6f1e1ba633 for a few 
days,
and switched to 
https://github.com/emacs-mirror/emacs/commit/784ec4b752 today.

I feel the drawing performance improves a lot in 
scratch/ns/surface-stuff branch,
and I can finally see the improvements brought by native-compile.
(since drawing is no longer a bottle-neck, maybe?)

Thanks a lot for your work!

Best,
Yiming


Alan Third <alan@idiocy.org> writes:

> On Sun, May 02, 2021 at 09:40:03PM +0800, Yiming Chen wrote:
>> Hi there!
>> I know that we've switched to a IOSurface based rendering for 
>> nsterm in
>> master branch.
>> Now master renders more precisely than emacs-27 (i.e. less 
>> flickers or
>> blank screens.)
>> But I feel that its rendering performance is much worse due to 
>> the new
>> buffering mechanism backed by IOSurface.
>
> Yup. That's the trade-off.
>
>> I think this poor rendering performance obscures the 
>> improvements
>> brought by native-compile.
>> So I want to fix this performance issue and make Emacs runs on 
>> macOS as
>> fast as on Linux.
>
> I've spent over a year on-and-off on this. Good luck.
>
>> 1. Try to restore the nsterm behavior from Emacs27 (which is 
>> much
>>    simpler than master IMHO), and start again from there
>>
>>    I've reverted the code and made it compile (see
>>    <https://github.com/emacs-mirror/emacs/compare/master...dsdshcym:revert-to-emacs-27-nsterm>)
>>    But Emacs failed to start and raised an error `Font ‘Menlo’ 
>>    is not
>>    defined'
>
> That error must be something unrelated, as the drawing buffer 
> changes
> didn't touch the font code... I've got no suggestions on where 
> to
> start, though. My knowledge of font handling is non-existent.
>
> Actually, a quick look shows that you've enabled the nsfont 
> backend
> which just doesn't work in macOS, so I'd bet that's something to
> investigate.
>
> I notice quite a few other problems... If I'm being honest, 
> master has
> diverged so much from emacs-27 that if I were to do this I'd 
> probably
> do it manually rather than try reverting commits.
>
> And I disagree that master is more complex than emacs-27. The 
> endless
> little work-arounds to try and get the emacs-27 rendering 
> working made
> it a much more brittle setup.
>
>> 2. Use layer-based view for rendering
>>
>>    I found that macvim also faced the similar issue when macOS 
>>    10.14
>>    came out 
>>    (<https://github.com/macvim-dev/macvim/issues/751>).
>>    They used NSImage to implement a double rendering solution
>>    (<https://github.com/macvim-dev/macvim/pull/757/files>), 
>>    which was
>>    also slower than before.
>>    Then they switched from NSImage to layer-based view in
>>    <https://github.com/macvim-dev/macvim/commit/dba6293677e0633917e3054cfddec1293e5ab3fb>.
>>    So I wonder if we can do the same.
>>
>> 3. Switch to CALayer (Core Animation) to use GPU (Metal) 
>> rendering for
>>    even better performance than before
>>
>>    If 2 (layer-based view) is possible, then I hope we can 
>>    leverage
>>    CALayer to make nsterm renders even faster.
>
> So... yes. The IOSurface system is already using layer-based 
> drawing.
> Have a look at updateLayer, for example.
>
> I'm not sure that CALayer really gives you anything in terms of
> performance for this sort of thing. CoreAnimation is all about 
> moving
> different layers about efficiently, afaict, which isn't 
> something we
> need to do in Emacs.
>
> And bear in mind that in order to send an image to the Metal 
> renderer
> you still need to transfer it to the GPU, and Apple recommend 
> using
> IOSurface as it's the most efficient way to do that.
>
> AND Yamamoto Mitsuharu disabled Metal rendering in the Mac port 
> as he
> found the simple IOSurface based layer backing was faster.
>
> AND the Chromium developers decided to go with the simple 
> IOSurface
> based layer backing for performance reasons too.
>
> One place that a big improvement could be found is in disabling 
> double
> buffering and only drawing to one IOSurface, as you don't need 
> to copy
> the contents of the different IOSurfaces to each other. But I 
> found
> that produced unacceptable tearing effects.
>
> But all that said, it could be that copying viewWillDraw from 
> master
> to emacs-27 could solve some of the flickering. In my 
> experimentation
> I don't believe it does, but I've never been very good at 
> inducing the
> "flickering".