bug#70221: [PATCH] New function `funcall-later`

bug#70221: [PATCH] New function `funcall-later`

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

Tags: patch

In the patch(es) below I suggest the addition of a new function
`funcall-later` which exposes the already existing `pending_funcalls`
mechanism to ELisp and fixes it so those are processed in the correct
order and so they're not just thrown away when used in batch mode.

`run-with-timer` does largely the same thing when passed a 0 timeout
and no repetition, but it is a bit more costly and relies on the timers,
which don't work as well in batch mode.


        Stefan


 In GNU Emacs 30.0.50 (build 1, i686-pc-linux-gnu, GTK+ Version 3.24.41,
 cairo version 1.18.0) of 2024-03-27 built on lechazo
Repository revision: a13cfe9bb17448e104dc86b7a33761ca60297871
Repository branch: work
Windowing system distributor 'The X.Org Foundation', version 11.0.12101011
System Description: Debian GNU/Linux trixie/sid

Configured using:
 'configure -C --enable-checking --enable-check-lisp-object-type --with-modules --with-cairo --with-tiff=ifavailable
 'CFLAGS=-Wall -g3 -Og -Wno-pointer-sign' --without-native-compilation
 PKG_CONFIG_PATH=/home/monnier/lib/pkgconfig'


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Ffuncall_later-New-function-to-expose-pending_funcal.patch --]
[-- Type: text/patch, Size: 8135 bytes --]

From 0041d6272d91baceada67aec04a0bca66a64919b Mon Sep 17 00:00:00 2001
From: Stefan Monnier <monnier@iro.umontreal.ca>
Date: Fri, 5 Apr 2024 14:58:53 -0400
Subject: [PATCH 1/2] (Ffuncall_later): New function to expose
 'pending_funcalls' to ELisp

* src/keyboard.c (pending_funcalls): Make static.
(pending_funcalls_r): New var.
(Ffuncall_later): New Lisp function.
(Frun_pending_funcalls): New function, extracted from 'timer_check_2'.
Improve the code to preserve the order in which functions were added.
(timer_check_2): Use it.
(syms_of_keyboard): Initialize and staticpro 'pending_funcalls_r',
and defsubr 'Sfuncall_later' and 'Srun_pending_funcalls'.
(syms_of_keyboard_for_pdumper): Reset 'pending_funcalls_r'.

* lisp/startup.el (command-line): Run pending funcalls before exiting,
in batch mode.

* src/lisp.h (pending_funcalls): Remove declaration.

* src/frame.c (delete_frame):
* src/terminal.c (Fdelete_terminal):
* src/comp.c (maybe_defer_native_compilation): Use 'Ffuncall_later'.
---
 etc/NEWS        |  4 ++++
 lisp/startup.el |  2 +-
 src/comp.c      |  4 +---
 src/frame.c     | 10 ++++------
 src/keyboard.c  | 47 +++++++++++++++++++++++++++++++++++++----------
 src/lisp.h      |  1 -
 src/terminal.c  |  6 ++----
 7 files changed, 49 insertions(+), 25 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 32cec82f970..8e8cbcdeb5d 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1761,6 +1761,10 @@ correctly UTF-8 encoded.
 \f
 * Lisp Changes in Emacs 30.1
 
+** New function 'funcall-later', to run code at the next convenient time.
+This is very similar to 'run-with-timer' with a 0 time argument, but cheaper
+and it also works in batch mode.
+
 ** New function 'help-fns-function-name'.
 For named functions, it just returns the name and otherwise
 it returns a short "unique" string that identifies the function.
diff --git a/lisp/startup.el b/lisp/startup.el
index 357a4154e4c..c105886908c 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -1630,7 +1630,7 @@ command-line
                             :warning))))
 
   ;; If -batch, terminate after processing the command options.
-  (if noninteractive (kill-emacs t))
+  (when noninteractive (internal--run-pending-funcalls) (kill-emacs t))
 
   ;; In daemon mode, start the server to allow clients to connect.
   ;; This is done after loading the user's init file and after
diff --git a/src/comp.c b/src/comp.c
index 99f51e07048..f59881a1b84 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -5217,9 +5217,7 @@ maybe_defer_native_compilation (Lisp_Object function_name,
 
   Fputhash (function_name, definition, Vcomp_deferred_pending_h);
 
-  pending_funcalls
-    = Fcons (list (Qnative__compile_async, src, Qnil, Qlate),
-             pending_funcalls);
+  CALLN (Ffuncall_later, Qnative__compile_async, src, Qnil, Qlate);
 }
 
 \f
diff --git a/src/frame.c b/src/frame.c
index abd6ef00901..f2ed55046d6 100644
--- a/src/frame.c
+++ b/src/frame.c
@@ -2138,9 +2138,8 @@ delete_frame (Lisp_Object frame, Lisp_Object force)
   if (NILP (Vrun_hooks) || is_tooltip_frame)
     ;
   else if (EQ (force, Qnoelisp))
-    pending_funcalls
-      = Fcons (list3 (Qrun_hook_with_args, Qdelete_frame_functions, frame),
-	       pending_funcalls);
+    CALLN (Ffuncall_later, Qrun_hook_with_args,
+	   Qdelete_frame_functions, frame);
   else
     {
 #ifdef HAVE_X_WINDOWS
@@ -2456,9 +2455,8 @@ delete_frame (Lisp_Object frame, Lisp_Object force)
   if (NILP (Vrun_hooks) || is_tooltip_frame)
     ;
   else if (EQ (force, Qnoelisp))
-    pending_funcalls
-      = Fcons (list3 (Qrun_hook_with_args, Qafter_delete_frame_functions, frame),
-	       pending_funcalls);
+    CALLN (Ffuncall_later, Qrun_hook_with_args,
+           Qafter_delete_frame_functions, frame);
   else
     safe_calln (Qrun_hook_with_args, Qafter_delete_frame_functions, frame);
 
diff --git a/src/keyboard.c b/src/keyboard.c
index 91faf4582fa..45f7295d5d1 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -4624,10 +4624,38 @@ timer_resume_idle (void)
   timer_idleness_start_time = timer_last_idleness_start_time;
 }
 
-/* List of elisp functions to call, delayed because they were generated in
-   a context where Elisp could not be safely run (e.g. redisplay, signal,
+/* List of elisp functions to call, delayed e.g. because they were generated in
+   a context where that code could not be safely run (e.g. redisplay, signal,
    ...).  Each element has the form (FUN . ARGS).  */
-Lisp_Object pending_funcalls;
+static Lisp_Object pending_funcalls, pending_funcalls_r;
+
+DEFUN ("funcall-later", Ffuncall_later, Sfuncall_later, 1, MANY, 0,
+       doc: /* Schedule FUNCTION for later execution.
+It will be called a soon as possible with ARGS, but not now.  */)
+  (ptrdiff_t nargs, Lisp_Object *args)
+{
+  pending_funcalls_r = Fcons (Flist (nargs, args), pending_funcalls_r);
+  return Qnil;
+}
+
+DEFUN ("internal--run-pending-funcalls", Frun_pending_funcalls, Srun_pending_funcalls, 0, 0, 0,
+       doc: /* Run the still pending `funcall-later'.  */)
+ (void)
+{
+  while (CONSP (pending_funcalls) || CONSP (pending_funcalls_r))
+    if (CONSP (pending_funcalls))
+      {
+        Lisp_Object funcall = XCAR (pending_funcalls);
+        pending_funcalls = XCDR (pending_funcalls);
+        CALLN (Fapply, funcall);
+      }
+    else
+      {
+        pending_funcalls = Fnreverse (pending_funcalls_r);
+        pending_funcalls_r = Qnil;
+      }
+  return Qnil;
+}
 
 /* Return true if TIMER is a valid timer, placing its value into *RESULT.  */
 static bool
@@ -4671,12 +4699,7 @@ timer_check_2 (Lisp_Object timers, Lisp_Object idle_timers)
   chosen_timer = Qnil;
 
   /* First run the code that was delayed.  */
-  while (CONSP (pending_funcalls))
-    {
-      Lisp_Object funcall = XCAR (pending_funcalls);
-      pending_funcalls = XCDR (pending_funcalls);
-      safe_calln (Qapply, XCAR (funcall), XCDR (funcall));
-    }
+  Frun_pending_funcalls ();
 
   if (CONSP (timers) || CONSP (idle_timers))
     {
@@ -12783,8 +12806,9 @@ is_ignored_event (union buffered_input_event *event)
 void
 syms_of_keyboard (void)
 {
-  pending_funcalls = Qnil;
+  pending_funcalls = pending_funcalls_r = Qnil;
   staticpro (&pending_funcalls);
+  staticpro (&pending_funcalls_r);
 
   Vlispy_mouse_stem = build_pure_c_string ("mouse");
   staticpro (&Vlispy_mouse_stem);
@@ -13084,6 +13108,8 @@ syms_of_keyboard (void)
   defsubr (&Scurrent_idle_time);
   defsubr (&Sevent_symbol_parse_modifiers);
   defsubr (&Sevent_convert_list);
+  defsubr (&Sfuncall_later);
+  defsubr (&Srun_pending_funcalls);
   defsubr (&Sinternal_handle_focus_in);
   defsubr (&Sread_key_sequence);
   defsubr (&Sread_key_sequence_vector);
@@ -13855,6 +13881,7 @@ syms_of_keyboard_for_pdumper (void)
      early init functions see the environment they expect.  */
 
   PDUMPER_RESET_LV (pending_funcalls, Qnil);
+  PDUMPER_RESET_LV (pending_funcalls_r, Qnil);
   PDUMPER_RESET_LV (unread_switch_frame, Qnil);
   PDUMPER_RESET_LV (internal_last_event_frame, Qnil);
   PDUMPER_RESET_LV (last_command_event, Qnil);
diff --git a/src/lisp.h b/src/lisp.h
index 3cb4361e75e..d9308cbc6dd 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -5106,7 +5106,6 @@ fast_c_string_match_ignore_case (Lisp_Object regexp,
 #if defined (USABLE_SIGIO) || defined (USABLE_SIGPOLL)
 void handle_input_available_signal (int);
 #endif
-extern Lisp_Object pending_funcalls;
 extern bool detect_input_pending (void);
 extern bool detect_input_pending_ignore_squeezables (void);
 extern bool detect_input_pending_run_timers (bool);
diff --git a/src/terminal.c b/src/terminal.c
index 23a5582d4d9..fd13ba90792 100644
--- a/src/terminal.c
+++ b/src/terminal.c
@@ -384,10 +384,8 @@ DEFUN ("delete-terminal", Fdelete_terminal, Sdelete_terminal, 0, 2, 0,
   if (NILP (Vrun_hooks))
     ;
   else if (EQ (force, Qnoelisp))
-    pending_funcalls
-      = Fcons (list3 (Qrun_hook_with_args,
-		      Qdelete_terminal_functions, terminal),
-	       pending_funcalls);
+    CALLN (Ffuncall_later, Qrun_hook_with_args,
+           Qdelete_terminal_functions, terminal);
   else
     safe_calln (Qrun_hook_with_args, Qdelete_terminal_functions, terminal);
 
-- 
2.43.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-lisp-Prefer-funcall-later-over-run-with-timer.patch --]
[-- Type: text/patch, Size: 5089 bytes --]

From 24d444a272bd4b42739a1883c4d5f0545dfce58a Mon Sep 17 00:00:00 2001
From: Stefan Monnier <monnier@iro.umontreal.ca>
Date: Fri, 5 Apr 2024 15:49:34 -0400
Subject: [PATCH 2/2] lisp: Prefer 'funcall-later' over 'run-with-timer'

* lisp/subr.el (do-after-load-evaluation):
* lisp/server.el (server-goto-toplevel):
* lisp/progmodes/compile.el (compilation-error-properties):
* lisp/menu-bar.el (menu-bar-mode):
* lisp/jit-lock.el (jit-lock-fontify-now):
* lisp/emacs-lisp/edebug.el (edebug-kill-buffer)
(edebug-unload-function): Use 'funcall-later' i.s.o 'run-with-timer'.
---
 lisp/emacs-lisp/edebug.el | 4 ++--
 lisp/jit-lock.el          | 6 +++---
 lisp/menu-bar.el          | 6 +++---
 lisp/progmodes/compile.el | 4 ++--
 lisp/server.el            | 2 +-
 lisp/subr.el              | 3 +--
 6 files changed, 12 insertions(+), 13 deletions(-)

diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index b27ffbca908..fc76552312f 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -4023,7 +4023,7 @@ edebug-mode
 
 (defun edebug-kill-buffer ()
   "Used on `kill-buffer-hook' when Edebug is operating in a buffer of Lisp code."
-  (run-with-timer 0 nil #'top-level))
+  (funcall-later #'top-level))
 
 ;;; edebug eval list mode
 
@@ -4597,7 +4597,7 @@ edebug-unload-function
     (unwind-protect
         (abort-recursive-edit)
       ;; We still want to run unload-feature to completion
-      (run-with-idle-timer 0 nil #'(lambda () (unload-feature 'edebug)))))
+      (funcall-later #'unload-feature 'edebug)))
   (remove-hook 'called-interactively-p-functions
                #'edebug--called-interactively-skip)
   (edebug-uninstall-read-eval-functions)
diff --git a/lisp/jit-lock.el b/lisp/jit-lock.el
index 05c0bd847b3..14c172c1e6a 100644
--- a/lisp/jit-lock.el
+++ b/lisp/jit-lock.el
@@ -478,9 +478,9 @@ jit-lock-fontify-now
                ;; eagerly extend the refontified region with
                ;; jit-lock-after-change-extend-region-functions.
                (when (< loose-beg orig-start)
-                 (run-with-timer 0 nil #'jit-lock-force-redisplay
-                                 (copy-marker loose-beg)
-                                 (copy-marker orig-start)))
+                 (funcall-later #'jit-lock-force-redisplay
+                                (copy-marker loose-beg)
+                                (copy-marker orig-start)))
 
                ;; Skip to the end of the fully refontified part.
                (setq start tight-end)))
diff --git a/lisp/menu-bar.el b/lisp/menu-bar.el
index 320fabb54cf..7a4de133b74 100644
--- a/lisp/menu-bar.el
+++ b/lisp/menu-bar.el
@@ -2632,9 +2632,9 @@ menu-bar-mode
   ;; directly.  The minor-mode message "Menu Bar mode disabled" comes
   ;; after this function returns, overwriting any message we do here.
   (when (and (called-interactively-p 'interactive) (not menu-bar-mode))
-    (run-with-idle-timer 0 nil 'message
-                         (substitute-command-keys
-                          "Menu Bar mode disabled.  \
+    (funcall-later #'message
+                   (substitute-command-keys
+                    "Menu Bar mode disabled.  \
 Use \\[menu-bar-mode] to make the menu bar appear."))))
 
 ;;;###autoload
diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index 11d400e145a..952f03d3155 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1402,8 +1402,8 @@ compilation-error-properties
       (when (and compilation-auto-jump-to-next
                  (>= type compilation-skip-threshold))
         (kill-local-variable 'compilation-auto-jump-to-next)
-        (run-with-timer 0 nil 'compilation-auto-jump
-                        (current-buffer) (match-beginning 0)))
+        (funcall-later #'compilation-auto-jump
+                       (current-buffer) (match-beginning 0)))
 
       (compilation-internal-error-properties
        file line end-line col end-col type fmt rule))))
diff --git a/lisp/server.el b/lisp/server.el
index b65053267a6..8e3c1d0a467 100644
--- a/lisp/server.el
+++ b/lisp/server.el
@@ -1031,7 +1031,7 @@ server-goto-toplevel
     ;; to open a frame on a new display, we might end up with an unusable
     ;; frame because input from that display will be blocked (until exiting
     ;; the minibuffer).  Better exit this minibuffer right away.
-    (run-with-timer 0 nil (lambda () (server-execute-continuation proc)))
+    (funcall-later #'server-execute-continuation proc)
     (top-level)))
 
 ;; We use various special properties on process objects:
diff --git a/lisp/subr.el b/lisp/subr.el
index fba70342154..20fdf8deded 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -5981,8 +5981,7 @@ do-after-load-evaluation
 					byte-compile-current-file
 					byte-compile-root-dir)))
 	    (byte-compile-warn "%s" msg)))
-         (noninteractive (funcall fun msg)) ;; No timer will be run!
-	 (t (run-with-idle-timer 0 nil fun msg))))))
+         (t (funcall-later fun msg))))))
 
   ;; Finally, run any other hook.
   (run-hook-with-args 'after-load-functions abs-file))
-- 
2.43.0

bug#70221: [PATCH] New function `funcall-later`

[Felician Nemeth]

> `run-with-timer` does largely the same thing when passed a 0 timeout
> and no repetition, but it is a bit more costly and relies on the timers,
> which don't work as well in batch mode.

Do this mean (run-with-timer 0 nil ...) can always be replaced with a
more efficient (funcall-later ...)?  If so, would it make sense to
modify run-with-time as well to call funcall-later when if its first two
argument is (0 nil)?

Thanks.

bug#70221: [PATCH] New function `funcall-later`

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Do this mean (run-with-timer 0 nil ...) can always be replaced with a
> more efficient (funcall-later ...)?  If so, would it make sense to
> modify run-with-time as well to call funcall-later when if its first two
> argument is (0 nil)?

No: beside the fact that they are run at slightly different times
(which might be OK), `funcall-later` doesn't return a handle you can
then use to cancel the call.


        Stefan

bug#70221: [PATCH] New function `funcall-later`

[Eli Zaretskii]

> Date: Fri, 05 Apr 2024 15:56:40 -0400
> From:  Stefan Monnier via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> In the patch(es) below I suggest the addition of a new function
> `funcall-later` which exposes the already existing `pending_funcalls`
> mechanism to ELisp and fixes it so those are processed in the correct
> order and so they're not just thrown away when used in batch mode.
> 
> `run-with-timer` does largely the same thing when passed a 0 timeout
> and no repetition, but it is a bit more costly and relies on the timers,
> which don't work as well in batch mode.

Thanks.  We should document this in the ELisp manual, and should
explain there the meaning of "at the next convenient time".  (I think
"convenient" here is very much misleading and thus inappropriate.)

bug#70221: [PATCH] New function `funcall-later`

[Eli Zaretskii]

> Date: Fri, 05 Apr 2024 15:56:40 -0400
> From:  Stefan Monnier via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> +DEFUN ("internal--run-pending-funcalls", Frun_pending_funcalls, Srun_pending_funcalls, 0, 0, 0,
> +       doc: /* Run the still pending `funcall-later'.  */)
> + (void)
> +{
> +  while (CONSP (pending_funcalls) || CONSP (pending_funcalls_r))
> +    if (CONSP (pending_funcalls))
> +      {
> +        Lisp_Object funcall = XCAR (pending_funcalls);
> +        pending_funcalls = XCDR (pending_funcalls);
> +        CALLN (Fapply, funcall);
> +      }
> +    else

You are using CALLN here, whereas the previous implementation used
safe_calln.  Is that intended?  Calling Lisp in unsafe ways in that
place might not be a good idea.  You didn't even inhibit QUIT.

As another difference between run-with-time and this mechanism, the
former took care of preserving deactivate-mark around the call, wheres
funcall-later doesn't -- this is at least one difference that we
should document (assuming we want it).

bug#70221: [PATCH] New function `funcall-later`

[Eli Zaretskii]

> Cc: 70221@debbugs.gnu.org
> Date: Fri, 05 Apr 2024 18:44:05 -0400
> From:  Stefan Monnier via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> > Do this mean (run-with-timer 0 nil ...) can always be replaced with a
> > more efficient (funcall-later ...)?  If so, would it make sense to
> > modify run-with-time as well to call funcall-later when if its first two
> > argument is (0 nil)?
> 
> No: beside the fact that they are run at slightly different times
> (which might be OK), `funcall-later` doesn't return a handle you can
> then use to cancel the call.

This should be mentioned in the documentation.

bug#70221: [PATCH] New function `funcall-later`

[Felician Nemeth]

>> Cc: 70221@debbugs.gnu.org
>> Date: Fri, 05 Apr 2024 18:44:05 -0400
>> From:  Stefan Monnier via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> > Do this mean (run-with-timer 0 nil ...) can always be replaced with a
>> > more efficient (funcall-later ...)?  If so, would it make sense to
>> > modify run-with-time as well to call funcall-later when if its first two
>> > argument is (0 nil)?
>> 
>> No: beside the fact that they are run at slightly different times
>> (which might be OK), `funcall-later` doesn't return a handle you can
>> then use to cancel the call.

Does "running at slightly different times" mean that the compat package
cannot reimplement funcall-later with run-with-time?

Thanks again.

bug#70221: [PATCH] New function `funcall-later`

[Sean Whitton]

Hello,

On Fri 05 Apr 2024 at 03:56pm -04, Stefan Monnier wrote:

> Tags: patch
>
> In the patch(es) below I suggest the addition of a new function
> `funcall-later` which exposes the already existing `pending_funcalls`
> mechanism to ELisp and fixes it so those are processed in the correct
> order and so they're not just thrown away when used in batch mode.
>
> `run-with-timer` does largely the same thing when passed a 0 timeout
> and no repetition, but it is a bit more costly and relies on the timers,
> which don't work as well in batch mode.

Very nice.  You could also update xref--ensure-default-directory.

-- 
Sean Whitton

bug#70221: [PATCH] New function `funcall-later`

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> +DEFUN ("internal--run-pending-funcalls", Frun_pending_funcalls, Srun_pending_funcalls, 0, 0, 0,
>> +       doc: /* Run the still pending `funcall-later'.  */)
>> + (void)
>> +{
>> +  while (CONSP (pending_funcalls) || CONSP (pending_funcalls_r))
>> +    if (CONSP (pending_funcalls))
>> +      {
>> +        Lisp_Object funcall = XCAR (pending_funcalls);
>> +        pending_funcalls = XCDR (pending_funcalls);
>> +        CALLN (Fapply, funcall);
>> +      }
>> +    else
>
> You are using CALLN here, whereas the previous implementation used
> safe_calln.  Is that intended?  Calling Lisp in unsafe ways in that
> place might not be a good idea.

I found it annoying that the debugger isn't brought up when a bug
occurs in such delayed evaluations.  Note also the subsequent call to
`timer-event-handler` uses `calln` rather than `safe_calln`, so I think
my original use of `safe_call2` in commit 58555d8187f3425 was a mistake.
I can't see any good reason why we'd need to protect the
C code from non-local exits in `timer_check_2`.

> You didn't even inhibit QUIT.

I'm torn on this one:

OT1H while the time at which the code is run is loosely specified, it's
not really asynchronous: it's run the next time Emacs "waits", which is
usually about the same time as `post-command-hook` (which does use
`inhibit-quit`), so there doesn't seem to be a good justification for
`inhibit-quit`.

OTOH when the `funcall-later` is performed from asynchronous code
(e.g. from a timer), then running the delayed function call without
`inhibit-quit` ends up "hoisting" that code outside of its original
`inhibit-quit` context.

For users such as `track-changes.el`, `inhibit-quit` is not
needed/desired.  But indeed when I use it in `futur.el` it's common to
call `funcall-later` from process filters and timers.

Usually part of the purpose of `funcall-later` is to run the code in
a different (dynamic) context, but maybe `funcall-later` should
preserve `inhibit-quit`?

> As another difference between run-with-time and this mechanism, the
> former took care of preserving deactivate-mark around the call, wheres
> funcall-later doesn't

This is again because `run-with-time` is designed for the general case
where the delay is not 0s, so the code will be run asynchronously,
wheres `funcall-later` is designed for "different time but not async".

> -- this is at least one difference that we
> should document (assuming we want it).

AFAICT we don't document anywhere that `deactive-mark` is bound
around timers, so I guess fixing that would be the best way to document
the difference.

>> No: beside the fact that they are run at slightly different times
>> (which might be OK), `funcall-later` doesn't return a handle you can
>> then use to cancel the call.
> This should be mentioned in the documentation.

How/where?  Maybe the doc can just say that it returns nil?
Other than that, it would seem odd to write something like
"There is no `cancel-funcall-later`".

> Thanks.  We should document this in the ELisp manual, and should
> explain there the meaning of "at the next convenient time".  (I think
> "convenient" here is very much misleading and thus inappropriate.)

That's the big question, really.  E.g. I currently use `funcall-later`
in `track-changes.el` and in `futur.el`.  I use it because it's handy.
But fundamentally those two need different things:

- `track-changes.el` uses `funcall-later` because it wants to delay the
  execution in order to combine the current buffer modification with
  potential soon-to-come buffer modifications.
- `futur.el` uses `funcall-later` to "wake up" a piece of code that was
  waiting for the current event.  In this case it uses `funcall-later`
  not because it wants to delay execution but because it wants that
  execution to be unaffected by the current dynamic scoping and it
  doesn't want that execution to block the current execution.

So, if we were to change `funcall-later` so that the code gets run more
promptly (e.g. via true concurrency [ let's pretend that this was
actually possible and that the race-condition problems were magically
solved somehow ]), it would be great for `futur.el` but would defeat the
purpose for `track-changes.el`.

Current users of `funcall-later` (well: of its underlying mechanism):
- We use it in `comp.c` to launch the "jit" compilation of a package.
- We use it in `frame.c` and `terminal.c` to run
  `(after-)delete-frame-functions` and `delete-terminal-functions` when
  the current context does not allow running ELisp.

These are more like `futur.el` in the sense that the main purpose is
to run the code in a different context, without blocking the
current execution.


        Stefan

bug#70221: [PATCH] New function `funcall-later`

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 70221@debbugs.gnu.org
> Date: Sat, 06 Apr 2024 10:33:05 -0400
> 
> >> +DEFUN ("internal--run-pending-funcalls", Frun_pending_funcalls, Srun_pending_funcalls, 0, 0, 0,
> >> +       doc: /* Run the still pending `funcall-later'.  */)
> >> + (void)
> >> +{
> >> +  while (CONSP (pending_funcalls) || CONSP (pending_funcalls_r))
> >> +    if (CONSP (pending_funcalls))
> >> +      {
> >> +        Lisp_Object funcall = XCAR (pending_funcalls);
> >> +        pending_funcalls = XCDR (pending_funcalls);
> >> +        CALLN (Fapply, funcall);
> >> +      }
> >> +    else
> >
> > You are using CALLN here, whereas the previous implementation used
> > safe_calln.  Is that intended?  Calling Lisp in unsafe ways in that
> > place might not be a good idea.
> 
> I found it annoying that the debugger isn't brought up when a bug
> occurs in such delayed evaluations.  Note also the subsequent call to
> `timer-event-handler` uses `calln` rather than `safe_calln`, so I think
> my original use of `safe_call2` in commit 58555d8187f3425 was a mistake.

timer-event-handler uses condition-case-unless-debug, so I don't see a
problem with timers, and I think your use of safe_call2 was not a
mistake.

> I can't see any good reason why we'd need to protect the
> C code from non-local exits in `timer_check_2`.

Because it will prevent timers from being called?

> > You didn't even inhibit QUIT.
> 
> I'm torn on this one:
> 
> OT1H while the time at which the code is run is loosely specified, it's
> not really asynchronous: it's run the next time Emacs "waits", which is
> usually about the same time as `post-command-hook` (which does use
> `inhibit-quit`), so there doesn't seem to be a good justification for
> `inhibit-quit`.
> 
> OTOH when the `funcall-later` is performed from asynchronous code
> (e.g. from a timer), then running the delayed function call without
> `inhibit-quit` ends up "hoisting" that code outside of its original
> `inhibit-quit` context.
> 
> For users such as `track-changes.el`, `inhibit-quit` is not
> needed/desired.  But indeed when I use it in `futur.el` it's common to
> call `funcall-later` from process filters and timers.
> 
> Usually part of the purpose of `funcall-later` is to run the code in
> a different (dynamic) context, but maybe `funcall-later` should
> preserve `inhibit-quit`?

From my POV, any code that runs from some background facility must
inhibit QUIT, because the user can type C-g at any moment.

> > As another difference between run-with-time and this mechanism, the
> > former took care of preserving deactivate-mark around the call, wheres
> > funcall-later doesn't
> 
> This is again because `run-with-time` is designed for the general case
> where the delay is not 0s, so the code will be run asynchronously,
> wheres `funcall-later` is designed for "different time but not async".

I don't think it's a wise approach.  You are converting several places
to using this new facility, so the prudent thing, in a program as
complex and devious as Emacs, is to keep compatibility even if our
hubris tempts us to think we are smarter.  Bitter experience has
taught me that we are not smarter, so I now prefer humility instead.

> > -- this is at least one difference that we
> > should document (assuming we want it).
> 
> AFAICT we don't document anywhere that `deactive-mark` is bound
> around timers, so I guess fixing that would be the best way to document
> the difference.

I mean where we explain the differences between zero-delay timers and
funcall-later.  We _will_ explain those, right?

> >> No: beside the fact that they are run at slightly different times
> >> (which might be OK), `funcall-later` doesn't return a handle you can
> >> then use to cancel the call.
> > This should be mentioned in the documentation.
> 
> How/where?

In the same place, see above, where we compare those two features.

> Maybe the doc can just say that it returns nil?
> Other than that, it would seem odd to write something like
> "There is no `cancel-funcall-later`".

Not "out of the blue", but in the context of explaining how it is
different (which is important for Lisp programmers to decide which one
to use in what situations), it is reasonable.

> > Thanks.  We should document this in the ELisp manual, and should
> > explain there the meaning of "at the next convenient time".  (I think
> > "convenient" here is very much misleading and thus inappropriate.)
> 
> That's the big question, really.  E.g. I currently use `funcall-later`
> in `track-changes.el` and in `futur.el`.  I use it because it's handy.
> But fundamentally those two need different things:
> 
> - `track-changes.el` uses `funcall-later` because it wants to delay the
>   execution in order to combine the current buffer modification with
>   potential soon-to-come buffer modifications.
> - `futur.el` uses `funcall-later` to "wake up" a piece of code that was
>   waiting for the current event.  In this case it uses `funcall-later`
>   not because it wants to delay execution but because it wants that
>   execution to be unaffected by the current dynamic scoping and it
>   doesn't want that execution to block the current execution.
> 
> So, if we were to change `funcall-later` so that the code gets run more
> promptly (e.g. via true concurrency [ let's pretend that this was
> actually possible and that the race-condition problems were magically
> solved somehow ]), it would be great for `futur.el` but would defeat the
> purpose for `track-changes.el`.
> 
> Current users of `funcall-later` (well: of its underlying mechanism):
> - We use it in `comp.c` to launch the "jit" compilation of a package.
> - We use it in `frame.c` and `terminal.c` to run
>   `(after-)delete-frame-functions` and `delete-terminal-functions` when
>   the current context does not allow running ELisp.
> 
> These are more like `futur.el` in the sense that the main purpose is
> to run the code in a different context, without blocking the
> current execution.

I don't think I disagree with what you write, but I fail to see how it
is relevant to the need to explain better what is that "next
convenient time" when the function will run.  In particular, I don't
see how the different uses of funcall-later affect that "next
convenient time".

bug#70221: [PATCH] New function `funcall-later`

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> timer-event-handler uses condition-case-unless-debug, so I don't see a
> problem with timers, and I think your use of safe_call2 was not a
> mistake.

`condition-case-unless-debug` is very different from `safe_calln`.
It doesn't prevent non-local exits nor prevent showing a debugger.

>> I can't see any good reason why we'd need to protect the
>> C code from non-local exits in `timer_check_2`.
> Because it will prevent timers from being called?

Why would it?  after the non-exit is caught somewhere up the stack, we'd
eventually come back to `timer_check_2` and run the timer then.

> From my POV, any code that runs from some background facility must
> inhibit QUIT, because the user can type C-g at any moment.

Agreed, and `funcall-later` doesn't run it "in the background", it runs
it at the end of the current code.

> You are converting several places to using this new facility, so the
> prudent thing, in a program as complex and devious as Emacs, is to
> keep compatibility even if our hubris tempts us to think we are
> smarter.  Bitter experience has taught me that we are not smarter, so
> I now prefer humility instead.

Ah, you're talking about the second patch.  Yes, I'm not really sure we
want that one.  I included it more to show that there is interest in
such a facility, but I don't think the risk of breakage justifies
installing it.

>> AFAICT we don't document anywhere that `deactive-mark` is bound
>> around timers, so I guess fixing that would be the best way to document
>> the difference.
> I mean where we explain the differences between zero-delay timers and
> funcall-later.  We _will_ explain those, right?

As soon as we've figured the main "when is it run" question 🙂

>> That's the big question, really.  E.g. I currently use `funcall-later`
>> in `track-changes.el` and in `futur.el`.  I use it because it's handy.
>> But fundamentally those two need different things:
[...]
> I don't think I disagree with what you write, but I fail to see how it
> is relevant to the need to explain better what is that "next
> convenient time" when the function will run.  In particular, I don't
> see how the different uses of funcall-later affect that "next
> convenient time".

It's relevant in two ways:

- It determines which part of the time-behavior we should consider as
  something we want to document and guarantee, as opposed to the part
  which is incidental and which we may prefer to document as not to be
  relied on.
- It might mean that `funcall-later` is ill-defined and should be
  rejected, or split into two, or should take extra arguments, or ...


        Stefan

bug#70221: [PATCH] New function `funcall-later`

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 70221@debbugs.gnu.org
> Date: Sat, 06 Apr 2024 11:46:28 -0400
> 
> > timer-event-handler uses condition-case-unless-debug, so I don't see a
> > problem with timers, and I think your use of safe_call2 was not a
> > mistake.
> 
> `condition-case-unless-debug` is very different from `safe_calln`.
> It doesn't prevent non-local exits nor prevent showing a debugger.

It catches errors, doesn't it?  That's what bothers me with CALLN.

> >> I can't see any good reason why we'd need to protect the
> >> C code from non-local exits in `timer_check_2`.
> > Because it will prevent timers from being called?
> 
> Why would it?  after the non-exit is caught somewhere up the stack, we'd
> eventually come back to `timer_check_2` and run the timer then.

Unless the same buggy funcall-later is again in the list, right?  Or
do we have a machinery to disable them, like we disable faulty timers?

> > From my POV, any code that runs from some background facility must
> > inhibit QUIT, because the user can type C-g at any moment.
> 
> Agreed, and `funcall-later` doesn't run it "in the background", it runs
> it at the end of the current code.

How is this different from running timers?

> > I don't think I disagree with what you write, but I fail to see how it
> > is relevant to the need to explain better what is that "next
> > convenient time" when the function will run.  In particular, I don't
> > see how the different uses of funcall-later affect that "next
> > convenient time".
> 
> It's relevant in two ways:
> 
> - It determines which part of the time-behavior we should consider as
>   something we want to document and guarantee, as opposed to the part
>   which is incidental and which we may prefer to document as not to be
>   relied on.

I'm not sure I understand where you are going with this.  It seems
very easy to tell when the delayed functions will be called, so why
are we arguing?

> - It might mean that `funcall-later` is ill-defined and should be
>   rejected, or split into two, or should take extra arguments, or ...

If the implementation changes, we will change the documentation to go
along.  But Lisp programmers who want to use this facility must have
a pretty good idea of when the delayed code will be called, or else
they are in for a bumpy ride.

bug#70221: [PATCH] New function `funcall-later`

[Dmitry Gutov]

On 06/04/2024 11:30, Sean Whitton wrote:
> Very nice.  You could also update xref--ensure-default-directory.

With a FIXME comment, maybe. xref is in "core elpa", so not yet.

bug#70221: [PATCH] New function `funcall-later`

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> `condition-case-unless-debug` is very different from `safe_calln`.
>> It doesn't prevent non-local exits nor prevent showing a debugger.
> It catches errors, doesn't it?

What's good about it?

> That's what bothers me with CALLN.

With CALLN the errors don't get caught by `safe_calln` so they get
caught further up the stack, which seems fine by me.
What problem/scenario are you thinking about?

>> >> I can't see any good reason why we'd need to protect the
>> >> C code from non-local exits in `timer_check_2`.
>> > Because it will prevent timers from being called?
>> 
>> Why would it?  after the non-exit is caught somewhere up the stack, we'd
>> eventually come back to `timer_check_2` and run the timer then.
>
> Unless the same buggy funcall-later is again in the list, right?

No: we bump it off the list before the CALLN, specifically to
avoid this problem.  They don't auto-repeat like some timers.

>> > From my POV, any code that runs from some background facility must
>> > inhibit QUIT, because the user can type C-g at any moment.
>> Agreed, and `funcall-later` doesn't run it "in the background", it runs
>> it at the end of the current code.
> How is this different from running timers?

Non-0s timers are run in the context of some future command (or in the
middle of "idle" time).  `funcall-later` are run before we get to idle
time or to the next user input, so if the user hits C-g during them,
it's no different from hitting C-g during the main part of the command.
It's realy more like `post-command-hook` (incidentally, I've been
thinking that maybe we should call `internal--run-pending-funcalls`
when we run `post-command-hook`).

>> - It determines which part of the time-behavior we should consider as
>>   something we want to document and guarantee, as opposed to the part
>>   which is incidental and which we may prefer to document as not to be
>>   relied on.
> I'm not sure I understand where you are going with this.  It seems
> very easy to tell when the delayed functions will be called, so why
> are we arguing?

I'm not sure the current implementation provides the behavior we want.
So I think it's worthwhile thinking about what it is that we want.
E.g. another implementation could be to have a separate thread running
those functions.  Or as mentioned above we could run them from
`post-command-hook`. ...


        Stefan

bug#70221: [PATCH] New function `funcall-later`

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 70221@debbugs.gnu.org
> Date: Sat, 06 Apr 2024 16:00:53 -0400
> 
> >> `condition-case-unless-debug` is very different from `safe_calln`.
> >> It doesn't prevent non-local exits nor prevent showing a debugger.
> > It catches errors, doesn't it?
> 
> What's good about it?
> 
> > That's what bothers me with CALLN.
> 
> With CALLN the errors don't get caught by `safe_calln` so they get
> caught further up the stack, which seems fine by me.
> What problem/scenario are you thinking about?

That we throw to top-level from a normal processing loop.  Why should
we not prevent that if we can?  Why are you so objected to doing what
we have always done there?

> >> >> I can't see any good reason why we'd need to protect the
> >> >> C code from non-local exits in `timer_check_2`.
> >> > Because it will prevent timers from being called?
> >> 
> >> Why would it?  after the non-exit is caught somewhere up the stack, we'd
> >> eventually come back to `timer_check_2` and run the timer then.
> >
> > Unless the same buggy funcall-later is again in the list, right?
> 
> No: we bump it off the list before the CALLN, specifically to
> avoid this problem.  They don't auto-repeat like some timers.

The Lisp program that initiated it could initiate another one soon
enough.  Why risk that when the solution is so easy?  I really don't
understand your objections here.  Is there any harm that can possibly
be done by catching errors, like timers do?  If not, why not do it?

> >> > From my POV, any code that runs from some background facility must
> >> > inhibit QUIT, because the user can type C-g at any moment.
> >> Agreed, and `funcall-later` doesn't run it "in the background", it runs
> >> it at the end of the current code.
> > How is this different from running timers?
> 
> Non-0s timers are run in the context of some future command (or in the
> middle of "idle" time).  `funcall-later` are run before we get to idle
> time or to the next user input

The funcall-later functions are run in exactly the same context.  Look
at the code: they are called inside the same loop in timer_check_2.
So the context is exactly the same.

> so if the user hits C-g during them, it's no different from hitting
> C-g during the main part of the command.  It's realy more like
> `post-command-hook` (incidentally, I've been thinking that maybe we
> should call `internal--run-pending-funcalls` when we run
> `post-command-hook`).

Whatever happens with zero timers happens also with funcall-later
functions.  So the same considerations apply.  From the user POV,
Emacs is idle, so the user can press C-g at any moment.  E.g., it is
customary to press C-g when Emacs sits at the prompt in the
minibuffer: both timers and funcall-later functions can be run at that
time.

> >> - It determines which part of the time-behavior we should consider as
> >>   something we want to document and guarantee, as opposed to the part
> >>   which is incidental and which we may prefer to document as not to be
> >>   relied on.
> > I'm not sure I understand where you are going with this.  It seems
> > very easy to tell when the delayed functions will be called, so why
> > are we arguing?
> 
> I'm not sure the current implementation provides the behavior we want.
> So I think it's worthwhile thinking about what it is that we want.
> E.g. another implementation could be to have a separate thread running
> those functions.  Or as mentioned above we could run them from
> `post-command-hook`. ...

As long as the code is on the branch, I don't really care.  But the
moment it lands on master, it should have the proper documentation,
and if by that time the implementation doesn't change, I insist on
saying something about when these functions are called, because I
think it's important.  If the implementation does change, I will, of
course, revisit my opinion (hopefully, during the review of the
modified code that is to follow).