bug#13400: 23.4; overlapping process filter calls

[Noam Postavsky <npostavs@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]

Hendrik Tews <hendrik@askra.de> writes:

> because Stefan Monnier asked for it
> (http://lists.inf.ed.ac.uk/pipermail/proofgeneral-devel/2013/000296.html)

[Note: meanwhile the section number has changed to 38 instead of 37.]

> - Section "37.9 Receiving Output from Processes" does not list
>   process-send-string. How about other blocking I/O functions?

In the attached patch, I've added a mention/xref for functions which send
data to processes.

> - Same in "37.9.2. Process Filter Functions"

This section is repeated twice (I addressed the second instance below).

> - Same in "37.4 Creating an Asynchronous Process" ,
>   process-send-string is neither waiting for input not time
>   delay.

I don't see any mention of process-send-string in that section, nor how
it's relevant to the rest of this report.

> - "37.7 Sending Input to Processes" says that filters can run
>   inside process-send-string, but it could be clearer about the
>   point that this can also happen inside the same filter for the
>   same process.

I'm not really convinced that is necessary.

> - "37.9.2 Process Filter Functions" ignores the problem
>   completely. There should be a paragraph clearly stating this
>   problem. Further, it would be nice, if the filter function
>   example could be extended to correctly deal with this problem.

I added a mention of the possibility of recursion.  I'm not sure about
making an example (specifically, what is the best way to deal with this
problem?).


[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 2100 bytes --]

From 022fb3f287d4fd351078f2b134d187ff584b380c Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Fri, 26 Jul 2019 23:20:37 -0400
Subject: [PATCH] Note that process filter can be called recursively
 (Bug#13400)

* doc/lispref/processes.texi (Output from Processes): Note that
functions which send data may also trigger reading from processes.
(Filter Functions): Note that filter functions may be called
recursively.
---
 doc/lispref/processes.texi | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/doc/lispref/processes.texi b/doc/lispref/processes.texi
index a93f4db428..208005772e 100644
--- a/doc/lispref/processes.texi
+++ b/doc/lispref/processes.texi
@@ -1416,9 +1416,10 @@ Output from Processes
 
   Output from a subprocess can arrive only while Emacs is waiting: when
 reading terminal input (see the function @code{waiting-for-user-input-p}),
-in @code{sit-for} and @code{sleep-for} (@pxref{Waiting}), and in
-@code{accept-process-output} (@pxref{Accepting Output}).  This
-minimizes the problem of timing errors that usually plague parallel
+in @code{sit-for} and @code{sleep-for} (@pxref{Waiting}), in
+@code{accept-process-output} (@pxref{Accepting Output}), and in
+functions which send data to processes (pxref{Input to Processes}).
+This minimizes the problem of timing errors that usually plague parallel
 programming.  For example, you can safely create a process and only
 then specify its buffer or filter function; no output can arrive
 before you finish, if the code in between does not call any primitive
@@ -1683,6 +1684,10 @@ Filter Functions
 or more batches of output; one way to do this is to insert the
 received text into a temporary buffer, which can then be searched.
 
+Note that if the filter calls a function which can wait for process
+output (pxref{Output from Processes}), the filter may be called
+recursively.
+
 @defun set-process-filter process filter
 This function gives @var{process} the filter function @var{filter}.  If
 @var{filter} is @code{nil}, it gives the process the default filter,
-- 
2.11.0


[-- Attachment #3: Type: text/plain, Size: 1366 bytes --]


> To make it easier in the future to deal with this problem, I
> suggest to add a process specific flag
> ``process-no-concurrent-filters''. When this flag is t for a
> process, Emacs accepts output from this process inside a filter
> but buffers it without calling the filter. The call to the filter
> is delayed until a point where no filter for this process is
> running. An error is signaled, if the buffered output exceeds a
> certain size.

I thought of just making a wrapper in Lisp instead, this saves the need
to complicate the process C code with yet another flag; it's already
tricky enough as it is.

;;; -*- lexical-binding: t -*-

(defun make-buffered-filter (filter)
  (let ((filtering nil)
        (buffered nil)
        (process nil))
    (lambda (proc str)
      (if process
          (unless (eq process proc)
            (error "Buffered filter used in different processes: %S, %S"
                   proc process))
        (setq process proc))
      (push str buffered)
      (unless filtering
        (setq filtering t)
        (unwind-protect
            (while buffered
              (setq str (apply #'concat (nreverse buffered)))
              (setq buffered nil)
              (funcall filter proc str))
          (setq filtering nil))))))

;; Can be used like
(set-process-filter my-process (make-buffered-filter #'my-filter-function))