Emacs graphical frontends / responsiveness

Emacs graphical frontends / responsiveness

[Jordan Ellis Coppard]

Hello friends,


I've been using Emacs as properly as I can for the past year-ish (read: 
all-in instead of bailing out constantly) after circling around the edge 
of the pool for a few years.

Overall I think I like it, although I am still less productive than I 
would be in say an IDE from JetBrains or Helix (a TUI editor). I'm sure 
I can close that gap as I tinker with Emacs more and more.

One thing that bothers me without fail is how slow, visually, Emacs is. 
I can often times type faster than the GUI can keep up, with noticeable 
hitching and delays from when I have typed a character to the point 
where I see it in a buffer and so forth. This is probably one of the 
biggest things ruining my experience because no matter how focused I am 
on the task at hand seeing this behaviour always rips me out of focus. 
That's also why I feel less productive with Emacs.

I run a fairly lean Emacs configuration, however even basic behaviour 
under `emacs -Q` is slow. Take `show-paren-mode` with `show-paren-delay` 
set to 0; Helix, VS Code, Neovim are able to instantly highlight matched 
brackets as I move my cursor around. With Emacs there is a noticeable 
delay (yes even with a delay of 0 and in `emacs -Q`). Often times I can 
type faster than the minibuffer input area can handle when filtering 
completion candidates e.g. `C-h o`.

I don't want my editor (or whatever specific title you'd like to apply) 
to EVER not immediately respond to input unless it's crashed. I don't 
want my editor running locally only accessing local resources to feel 
like an ssh connection with 200 ms of latency when nothing else, nor any 
other editor I've tried, does. I understand if I provide some input that 
I may be asking it to do heavy lifting and that eventually it will get 
back to me with the result (that's fine), but blocking everything else 
while that happens is frustrating as I have described above.

I recently saw in this mailing list Yuan benchmark character insertion 
into a buffer (with syntax highlighting) at around ~30 ms (if I recall 
correctly). That's an update rate of 33 times a second, with presumably 
(for benchmarking's sake) nothing else going on; thus one would assume 
even lower update rates if you use a few features here and there 
built-in or otherwise. Emacs struggling to update display in under 16 ms 
was... shocking to see.

I also recently read this post: 
https://gist.github.com/ghosty141/c93f21d6cd476417d4a9814eb71dd46e  -- 
which left me feeling quite sad (if it is accurate) regarding the 
internals surrounding the GUI and so how apparently needlessly difficult 
in today's age it would/will be to improve (over how it currently is). I 
understand the historical legacy.

So I ask: is it possible that Emacs' GUI be decoupled from the lisp core 
such that updates can happen as fast as possible (less perceived delay), 
and that the (what I gather from the above post) rube goldberg display 
internals be remade from scratch, or more likely, a new frontend (from 
scratch) as an alternative?

I don't ask as a "please do this for me".

I'll get involved but if the answer ahead of time is a "no it's never 
going to happen even with a completely working patch in future" then I 
think it'd best for me to abandon Emacs. I have ADHD and the visual 
hitching and lack of responsiveness to input drives me insane. I fear 
it's something I'll never be able to get over and in a world of less 
configurable vs. visually hitching and slow I would rather take the former.

I realise this email has been fairly critical so far. I have said all of 
this without malice, I just feel it prudent to describe how important 
this is (at least to me) by not mincing words.

Besides the criticisms: Emacs, so far, is amazing. I can make it do 
whatever I want, however I want, and mould it to be uniquely my own 
which is something I cannot do with anything else. The effort from 
scores of people over ~40 years as well as the continued effort shows. I 
can't imagine so much effort or such an enduring project.

But as I've said this behaviour (of Emacs) ruins it for me and I don't 
think I'll ever be able to get over it so at this juncture I'd rather 
help contribute and improve Emacs for myself and others and I hope 
that's something this email can be the genesis of.


Regards,
Jordan

Re: Emacs graphical frontends / responsiveness

[Corwin Brust]

Hi Jordan, all,

On Sun, Oct 20, 2024 at 1:46 PM Jordan Ellis Coppard <jc+o.emacs@wz.ht> wrote:
>
> I've been using Emacs as properly as I can for the past year-ish (read:
> all-in instead of bailing out constantly) after circling around the edge
> of the pool for a few years.
>

Woo!

> One thing that bothers me without fail is how slow, visually, Emacs is.

Would you be comfortable sharing the system confirmation and other
information generated at the bottom when you run M-x report-emacs-bug
?  I think this might help folks (even me, maybe) make suggestions for
improving the performance of your setup or pin-point possible
improvements to Emacs, itself.

Re: Emacs graphical frontends / responsiveness

[Eli Zaretskii]

> Date: Mon, 21 Oct 2024 03:46:05 +0900
> From: Jordan Ellis Coppard <jc+o.emacs@wz.ht>
> 
> One thing that bothers me without fail is how slow, visually, Emacs is. 

This is partly true (but for good reasons, see below), and partly an
illusion based on wrong expectations (also see below).

> I can often times type faster than the GUI can keep up, with noticeable 
> hitching and delays from when I have typed a character to the point 
> where I see it in a buffer and so forth. This is probably one of the 
> biggest things ruining my experience because no matter how focused I am 
> on the task at hand seeing this behaviour always rips me out of focus. 
> That's also why I feel less productive with Emacs.

This is too general.  Please describe specific situations where this
happens, and please accompany that by the list of optional features
you have turned ON (or try the experiments in "emacs -Q" and see if
the GUI still cannot keep up).  As you might know, some optional
features work by using a post-command-hook, which basically calls some
Lisp when Emacs finishes to process a command.  And since each
keyboard input event is a command in Emacs, the responsiveness is
being slowed by these post-command activities.  Turn on too many of
such features, and you have a slowly-responding Emacs.

> I run a fairly lean Emacs configuration, however even basic behaviour 
> under `emacs -Q` is slow. Take `show-paren-mode` with `show-paren-delay` 
> set to 0; Helix, VS Code, Neovim are able to instantly highlight matched 
> brackets as I move my cursor around. With Emacs there is a noticeable 
> delay (yes even with a delay of 0 and in `emacs -Q`).

show-paren-delay runs off the idle timer, which means it only kicks in
when Emacs detects that it's "idle": it processed the last command and
has no pending input.  Even if you set the delay to zero, the function
which looks for and highlights the matching paren will only be called
when Emacs is idle.  If you type fast enough, it will _never_ be
called.  This is by design: most of such features (at least those of
them that are well-designed) are coded to stay away and not to
interfere with the fast typists among us.  I guess similar features in
Helix, Neovim, etc. are based on different principles.  If you want an
immediate highlighting of the matching parens in Emacs, you will need
a different mode, not show-paren-mode.

> Often times I can type faster than the minibuffer input area can
> handle when filtering completion candidates e.g. `C-h o`.

Once again, tell the details.  Depending on the back-end of the
completion (i.e. the way completion candidates are generated), this
could be slow, indeed.  But the basic completion is quite fast.  I
just typed "C-h o f TAB" and got an immediate popup with the list of
candidates -- and that was in an unoptimized debugging build.  If you
see something else, please tell the details.

> I don't want my editor (or whatever specific title you'd like to apply) 
> to EVER not immediately respond to input unless it's crashed.

See above: some interactive features of Emacs are designed to only
kick in when Emacs is idle, so as not to slow down fast typists.  So
what you say above is either an over-generalization, or maybe you
don't like this design principle, in which case you will need to look
for alternative implementations of the relevant features.

> I don't want my editor running locally only accessing local
> resources to feel like an ssh connection with 200 ms of latency when
> nothing else, nor any other editor I've tried, does.

Again: details, please.  When taken at face values, the above is
simply false, based on 30 years of using Emacs.

> I recently saw in this mailing list Yuan benchmark character insertion 
> into a buffer (with syntax highlighting) at around ~30 ms (if I recall 
> correctly). That's an update rate of 33 times a second, with presumably 
> (for benchmarking's sake) nothing else going on; thus one would assume 
> even lower update rates if you use a few features here and there 
> built-in or otherwise. Emacs struggling to update display in under 16 ms 
> was... shocking to see.

This part is a price we pay for having such a complete and flexible
control on what Emacs does and how it processes and shows stuff.  The
fact that each input event is bound to a command, the fact that these
bindings go through keymaps (which is necessary to allow
customizations of the bindings by major and minor modes), the fact
that many commands have hooks which can be and are used to customize
what the command does and what is done to the editor's state as result
of performing a command, the fact that a Lisp extension can create a
simple data structure which affects how Emacs displays a certain
portion of buffer text (i.e., the display code must consult Lisp data
structures and sometimes run Lisp code) -- these and many other
features don't come for free.  If you want to understand why character
insertion with syntax highlighting is so "slow" in Emacs, you need to
study the processing triggered by inserting a single character, and
realize and appreciate the level of customizability, flexibility, and
control a user-defined Lisp program can have on this processing.  This
is what makes Emacs what it is!  This is why people who like Emacs and
use it for their production tasks will never agree to have an editor
that is less powerful, less flexible, less able to control everything
it does.

> I also recently read this post: 
> https://gist.github.com/ghosty141/c93f21d6cd476417d4a9814eb71dd46e  -- 
> which left me feeling quite sad (if it is accurate) regarding the 
> internals surrounding the GUI and so how apparently needlessly difficult 
> in today's age it would/will be to improve (over how it currently is). I 
> understand the historical legacy.

That blog wants to make a point by exaggerating some aspects of Emacs
and presenting others as hopelessly outdated.  I invite you to read
the code and make up your own mind.

> So I ask: is it possible that Emacs' GUI be decoupled from the lisp core 
> such that updates can happen as fast as possible (less perceived delay), 
> and that the (what I gather from the above post) rube goldberg display 
> internals be remade from scratch, or more likely, a new frontend (from 
> scratch) as an alternative?

I don't know if it's possible.  I do know that it is very hard,
because Emacs wants to control the GUI completely -- and it must do so
to let our users do the crazy, sometimes insane, stuff by writing
relatively simple Lisp.  If someone knows how to decouple while
keeping what makes Emacs Emacs -- such ideas, let alone code, will be
very welcome.

> Besides the criticisms: Emacs, so far, is amazing. I can make it do 
> whatever I want, however I want, and mould it to be uniquely my own 
> which is something I cannot do with anything else. The effort from 
> scores of people over ~40 years as well as the continued effort shows. I 
> can't imagine so much effort or such an enduring project.

You need to understand that this comes at a price.  As the saying
goes, "the impossible we do immediately; miracles take a little
longer".  Eventually, the insane amount of processing Emacs needs to
do in order to provide the expected flexibility and control hits the
bare machine performance wall.  When that happens, my best advice is
to buy a faster machine and/or optimize the optional features you turn
ON.

Happy hacking!

Re: Emacs graphical frontends / responsiveness

[Suhail Singh]

Jordan Ellis Coppard <jc+o.emacs@wz.ht> writes:

> One thing that bothers me without fail is how slow, visually, Emacs
> is. I can often times type faster than the GUI can keep up, with
> noticeable hitching and delays from when I have typed a character to
> the point where I see it in a buffer and so forth.

This is odd.  Starting from emacs -Q, when does this issue manifest
itself?  Specifically, is this the case when typing in a new text buffer
(pointing to a local file, and not over the network) in
fundamental-mode?  If not, what's the minimal reproducible example
starting from emacs -Q?  If you have already submitted a bug-report with
those details, could you please post the reference?

Also, what version of Emacs is this based on?  Which platform?  And how
was the binary built?  Etc.

> Often times I can type faster than the minibuffer input area can
> handle when filtering completion candidates e.g. `C-h o`.

Is this with emacs -Q?  For me, with emacs -Q, in *scratch* buffer the
above keybinding invokes `describe-symbol'.  On Emacs 29.4 on GNU/Linux,
I experience no noticeable delays when typing in minibuffer.  Similar to
above could you please share a minimal reproducible example?

-- 
Suhail

Re: Emacs graphical frontends / responsiveness

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 6978 bytes --]

If you're on a Mac, which I'm guessing, and, if by chance you are using
this build https://emacsformacosx.com/, I highly suggest that you try this
one instead https://github.com/jimeh/emacs-builds/releases/tag/Emacs-29.4.
It is much, much faster and exhibits correct behavior with code that
previously misbehaved and has a few patches applied that correct some GUI
features (internal-border colors, for example). I've been using the jimeh
build successfully for a while now after much frustration of my own this
summer. Pretty much all resolved and I have a lot of customizations. On
gotcha is that the jimeh build directly packages the gcc support needed for
native packages which is great BUT it is missing one component during
trampoline bootstrap (jimeh is aware) so start it once from the command
line and let it "warm up," and assuming you have gcc somewhere on your path
(via homebrew or somewhere else), thereafter it's fine launched from
Spotlight or wherever.

I noticed the following in what might be your init.el, though not directly
performance related, just FYI:

;; Make native compilation silent and prune its cache.
(when (native-comp-available-p)
  (setq native-comp-async-report-warnings-errors 'silent)
  (setq native-compile-prune-cache t)) ; <-- this is a function not a
variable

Take some care with your mode-line entries such as display-battery-mode as
I believe it will get queried during every mode-line refresh.

Also take some care with post-command-hook. Check to see what's on the hook
is what you need and nothing more. These get called after every character
you type (as Eli explained, they're all implemented via commands, though
the main "hot path" commands are in C, as you'd expect). Run M-x
list-timers and see if there are any timers running very frequently on
short intervals that you didn't expect.

I took a quick peek to see if you'd experience issues with some of the
caching challenges using project.el, tramp, etc. and nothing jumped out at
me. If there were calls to project-current / project-name on your mode-line
or tab-bar customizations, those could get in the way.

We're all in the same boat and happy to help. I speak for everyone as I've
observed the sheer amount of consideration, work, and thoughtful feedback
that goes into Emacs as a platform on a continual basis.

On Sun, Oct 20, 2024 at 2:48 PM Jordan Ellis Coppard <jc+o.emacs@wz.ht>
wrote:

> Hello friends,
>
>
> I've been using Emacs as properly as I can for the past year-ish (read:
> all-in instead of bailing out constantly) after circling around the edge
> of the pool for a few years.
>
> Overall I think I like it, although I am still less productive than I
> would be in say an IDE from JetBrains or Helix (a TUI editor). I'm sure
> I can close that gap as I tinker with Emacs more and more.
>
> One thing that bothers me without fail is how slow, visually, Emacs is.
> I can often times type faster than the GUI can keep up, with noticeable
> hitching and delays from when I have typed a character to the point
> where I see it in a buffer and so forth. This is probably one of the
> biggest things ruining my experience because no matter how focused I am
> on the task at hand seeing this behaviour always rips me out of focus.
> That's also why I feel less productive with Emacs.
>
> I run a fairly lean Emacs configuration, however even basic behaviour
> under `emacs -Q` is slow. Take `show-paren-mode` with `show-paren-delay`
> set to 0; Helix, VS Code, Neovim are able to instantly highlight matched
> brackets as I move my cursor around. With Emacs there is a noticeable
> delay (yes even with a delay of 0 and in `emacs -Q`). Often times I can
> type faster than the minibuffer input area can handle when filtering
> completion candidates e.g. `C-h o`.
>
> I don't want my editor (or whatever specific title you'd like to apply)
> to EVER not immediately respond to input unless it's crashed. I don't
> want my editor running locally only accessing local resources to feel
> like an ssh connection with 200 ms of latency when nothing else, nor any
> other editor I've tried, does. I understand if I provide some input that
> I may be asking it to do heavy lifting and that eventually it will get
> back to me with the result (that's fine), but blocking everything else
> while that happens is frustrating as I have described above.
>
> I recently saw in this mailing list Yuan benchmark character insertion
> into a buffer (with syntax highlighting) at around ~30 ms (if I recall
> correctly). That's an update rate of 33 times a second, with presumably
> (for benchmarking's sake) nothing else going on; thus one would assume
> even lower update rates if you use a few features here and there
> built-in or otherwise. Emacs struggling to update display in under 16 ms
> was... shocking to see.
>
> I also recently read this post:
> https://gist.github.com/ghosty141/c93f21d6cd476417d4a9814eb71dd46e  --
> which left me feeling quite sad (if it is accurate) regarding the
> internals surrounding the GUI and so how apparently needlessly difficult
> in today's age it would/will be to improve (over how it currently is). I
> understand the historical legacy.
>
> So I ask: is it possible that Emacs' GUI be decoupled from the lisp core
> such that updates can happen as fast as possible (less perceived delay),
> and that the (what I gather from the above post) rube goldberg display
> internals be remade from scratch, or more likely, a new frontend (from
> scratch) as an alternative?
>
> I don't ask as a "please do this for me".
>
> I'll get involved but if the answer ahead of time is a "no it's never
> going to happen even with a completely working patch in future" then I
> think it'd best for me to abandon Emacs. I have ADHD and the visual
> hitching and lack of responsiveness to input drives me insane. I fear
> it's something I'll never be able to get over and in a world of less
> configurable vs. visually hitching and slow I would rather take the former.
>
> I realise this email has been fairly critical so far. I have said all of
> this without malice, I just feel it prudent to describe how important
> this is (at least to me) by not mincing words.
>
> Besides the criticisms: Emacs, so far, is amazing. I can make it do
> whatever I want, however I want, and mould it to be uniquely my own
> which is something I cannot do with anything else. The effort from
> scores of people over ~40 years as well as the continued effort shows. I
> can't imagine so much effort or such an enduring project.
>
> But as I've said this behaviour (of Emacs) ruins it for me and I don't
> think I'll ever be able to get over it so at this juncture I'd rather
> help contribute and improve Emacs for myself and others and I hope
> that's something this email can be the genesis of.
>
>
> Regards,
> Jordan
>
>

[-- Attachment #2: Type: text/html, Size: 8718 bytes --]

Re: Emacs graphical frontends / responsiveness

[Sean Whitton]

Hello,

Your post made me remember this article which rates Emacs's display
latency rather highly: <https://lwn.net/Articles/751763/>.

I'm not sure what conclusions to draw; just sharing a resource.

Thanks for writing.

-- 
Sean Whitton