continuation passing in Emacs vs. JUST-THIS-ONE

[Thomas Koch <thomas@koch.ro>]

TL;DR: (Why) is there no standard way for continuation passing style[1] ("event driven") programming in Emacs?

During the investigation of an Emacs freeze[1] while starting eglot over tramp, we made a couple of observations. It was proposed to me to share these observations with you. I don't know elips or Emacs internals though, so apologies for any errors.

[1] https://debbugs.gnu.org/61350

Bug #61350 happens because Tramp calls accept-process-output with JUST-THIS-ONE set to t. Tramp does this since Bug #12145[2]. However it seems, that this bug should have been rather fixed in function `find-dired` instead. (See following separate email on this.)

[2] https://debbugs.gnu.org/12145

The JUST-THIS-ONE argument was introduced with this entry in etc/NEWS.22, emphasis from me:

"""
*** Function 'accept-process-output' has a new optional fourth arg
JUST-THIS-ONE.  If non-nil, only output from the specified process
is handled, suspending output from other processes.  If value is an
integer, also inhibit running timers.  THIS FEATURE IS GENERALLY NOT
RECOMMENDED, but may be necessary for specific applications, such as
speech synthesis.
"""

The argument was discussed here:
https://lists.gnu.org/archive/html/emacs-devel/2004-08/msg00141.html

and introduced in this commit:
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=107ed38d4bdec03002b2a23619e205722cd5b8d1

I don't even think that the original motivation for introducing JUST-THIS-ONE was valid. Unfortunately there was not much discussion about it. It was argued, that it would be hard to make a process filter function reentrant. And I think that this was an invalid root cause analysis to start with.

First, the emacs manual says[3]: "Note that if any of those functions are called by the filter, the filter may be called recursively." - So one should make the filter reentrant, If I understand correctly.

[3] https://www.gnu.org/software/emacs/manual/html_node/elisp/Filter-Functions.html

Second, the manual further says: "Quitting is normally inhibited within a filter function". This indicates to me, that a filter function should be (mostly) "side effect free" besides putting its input somewhere (e.g. in a buffer or message queue) and trigger an event if there is enough input for further processing. This also reduces the risk, that the function could be called recursively in a damaging way.

It seems to me, that there is not yet a standard way in Emacs for continuations (or event driven programming) although the Emacs Wiki refers to the emacs-deferred library: https://www.emacswiki.org/emacs/ConcurrentEmacs

Because there is no such library in Emacs, people either write their own code for continuations (eglot?) or do too much work in a process filter function (speechd-el in 2004 which led to JUST-THIS-ONE).

While I don't know elisp, I unfortunately had to do JavaScript. Like Emacs, JS is single-threaded. While I share the sentiment about JS, there are still things to learn from it, e.g. event driven programming.

See also:

- 2011 emacs-dev discussion: https://lists.gnu.org/archive/html/emacs-devel/2011-05/msg00575.html
- 2016 Blogpost https://jyp.github.io/posts/elisp-cps.html
- https://stable.melpa.org/#/deferred
- https://www.gnu.org/software/emacs/manual///html_node/elisp/Transaction-Queues.html
- Maybe: https://elpa.gnu.org/packages/fsm.html