bug#33747: 26.1; process-send-string exceeds max-specpdl-size

bug#33747: 26.1; process-send-string exceeds max-specpdl-size

[Markus Triska]

To reproduce this issue, please install the PostScript viewer
Ghostscript ("gs"), and put the following forms in "pipe.el":

    (setq target (start-process "gs" nil "gs" "-g100x100")
          source (start-process "bash" nil "bash"
                                "-c" "while true; do echo \
      rand 2 31 exp div setgray 0 0 100 100 rectfill; done"))

    (set-process-filter source 'send-to-target)

    (defun send-to-target (proc str)
      (process-send-string target str))

When you then invoke Emacs via:

    $ emacs -Q pipe.el --eval '(eval-buffer)'

you get almost instantly:

    error in process filter: Variable binding depth exceeds max-specpdl-size

The reason for this seems to be that process-send-string also builds up
frames for unwind-protect when it waits for input if the queue is full.

However, in this concrete case, I would prefer to simply send (or queue)
the string and *not* wait for any input within the process filter.

Is there a way to accomplish this? Alternatively, would you please
consider to provide such a mechanism?

For example, when sending output to a process and the output queue of
that process is full, would it be admissible or even preferable to wait
only for input of that specific process, instead of also waiting for and
handling input from other processes (as seems to be the case now)?

Thank you and all the best!
Markus


In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars)
 of 2018-11-18 built on debian
Repository revision: 07f8f9bc5a51f5aa94eb099f3e15fbe0c20ea1ea
Windowing system distributor 'The X.Org Foundation', version 11.0.11902000
System Description:	Debian GNU/Linux 9.6 (stretch)

bug#33747: 26.1; process-send-string exceeds max-specpdl-size

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Date: Fri, 14 Dec 2018 18:49:09 +0100
> 
> The reason for this seems to be that process-send-string also builds up
> frames for unwind-protect when it waits for input if the queue is full.
> 
> However, in this concrete case, I would prefer to simply send (or queue)
> the string and *not* wait for any input within the process filter.
> 
> Is there a way to accomplish this?

AFAIK, process-send-string is a blocking function: it cannot return
before the entire string was sent.

bug#33747: 26.1; process-send-string exceeds max-specpdl-size

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> AFAIK, process-send-string is a blocking function: it cannot return
> before the entire string was sent.

Yes indeed. However, the C function send_process internally calls
wait_reading_process_output, and this may again invoke the filter!  What
I find surprising is that this call of wait_reading_process_output is
not limited to the process whose output queue is full, i.e., the one for
which process-send-string was actually invoked. Would it work to limit
it to that process, or could there be an alternative means to prevent
reading from other processes in that case? It would be very useful in
such situations where I simply want to send, and not read anything.

bug#33747: 26.1; process-send-string exceeds max-specpdl-size

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Cc: 33747@debbugs.gnu.org
> Date: Fri, 14 Dec 2018 19:50:39 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > AFAIK, process-send-string is a blocking function: it cannot return
> > before the entire string was sent.
> 
> Yes indeed. However, the C function send_process internally calls
> wait_reading_process_output

That's so as not to wedge the entire session.  Communications with
async subprocesses aren't supposed to stop everything in its tracks.

> and this may again invoke the filter!

In general, calling APIs that could take a long time from a process
filter is not a very good idea, btw.  Remember that process filters
are run when Emacs is idle, so when you call an API that could take a
long time, you make Emacs less responsive.  And then there's the
infinite recursion case that you hit.

> What I find surprising is that this call of
> wait_reading_process_output is not limited to the process whose
> output queue is full, i.e., the one for which process-send-string
> was actually invoked.

I don't see any reason why it should only wait for output from the
same process.  Once again, this is Emacs's way to wait without
stopping everything else, so waiting for input from any source makes
perfect sense to me.

bug#33747: 26.1; process-send-string exceeds max-specpdl-size

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> stopping everything else, so waiting for input from any source makes
> perfect sense to me.

In this concrete case, I am looking for a way to make Emacs not wait
for, or at least not process, input from a specific process while
sending output to a different process.

For the case I posted, I can alleviate the issue a bit if I set:

    (setq max-specpdl-size 5000)

However, if I then use for example the following definition of
send-to-target instead of the one I posted:

   (defun send-to-target (proc str)
     (if (process-live-p target)
         (process-send-string target str)
       (kill-process source)))

then I get, for the receipe I posted:

    Lisp nesting exceeds ‘max-lisp-eval-depth’

This is unfortunate, because increasing this limit will eventually crash
Emacs. The root cause is that process-send-string triggers the filter of
the same process that caused the filter to be invoked. I find this
unexpected, because the target process is different, and if possible, I
would like a way do prevent this. Could you please consider adding a
feature that prevents triggering the filter in such cases?

Thank you and all the best!
Markus