accept-process-output throws

accept-process-output throws

[JD Smith]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]

I find that, in a loop like:

(unwind-protect
    (while waiting-for-output
      (accept-process-output process))
  (if waiting-for-output
      (warn "accept-process-output exited non-locally!")))

i.e. waiting for an async process (iPython), accept-process-output (emacs 27.2) sometimes exits non-locally. I checked for any signals that might cause this (none), and then I suspected re-entering a-p-e could throw, so guarded against that, but still no luck.  

Does anyone know any other reasons that accept-process-output would (occasionally) exit non-locally?  Is this expected?


[-- Attachment #2: Type: text/html, Size: 1779 bytes --]

Re: accept-process-output throws

[Eli Zaretskii]

> From: JD Smith <jdtsmith@gmail.com>
> Date: Tue, 23 Nov 2021 15:36:38 -0500
> 
> 
> [1:text/plain Show]
> 
> 
> [2:text/html Hide Save:noname (1kB)]
> 
> I find that, in a loop like:
> 
>  (unwind-protect
>      (while waiting-for-output
>        (accept-process-output process))
>    (if waiting-for-output
>        (warn "accept-process-output exited non-locally!")))
> 
> i.e. waiting for an async process (iPython), accept-process-output (emacs 27.2) sometimes exits
> non-locally. I checked for any signals that might cause this (none), and then I suspected re-entering a-p-e
> could throw, so guarded against that, but still no luck.  
> 
> Does anyone know any other reasons that accept-process-output would (occasionally) exit non-locally?  Is
> this expected?

Is this in "emacs -Q"?  If so, how did you start the process, and what
do you have in the process-filter and process-sentinel functions for
this process?

IOW, please tell more about the situation, since it is not yet clear
which code could be involved in this.

Re: accept-process-output throws

[JD Smith]

> On Nov 23, 2021, at 10:25 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> Is this in "emacs -Q"?  If so, how did you start the process, and what
> do you have in the process-filter and process-sentinel functions for
> this process?

This is in a customized inferior python mode (a comint-derived mode), running iPython as the process.   I discovered the issue with `python-shell-send-string-no-output', which adds its own pre-output-filter to accumulate and discard process output.  The no-output commands are being sent to the process from a post-command hook which kicks off a timer (to gather completion and syntax data).  The command results arrive promptly, but when the non-local exit occurs, they “escape" the pre-output filtering and show up in the buffer.  When this occurs, it is always as a result of the non-local exit of `accept-process-output’.  I’ve found the problem gets more severe when the PCH-launched timer's delay is very small (<0.1s), i.e. when commands are sent in quick succession while typing.

I do not have an emacs -Q demo of this but can look at creating one with the stock python-shell-mode.  In the meantime I’ve found that if I bind (inhibit-quit t), remove `with-local-quit' and specify a timeout in `accept-process-output' (to silence the warning about the blocking call), this entirely prevents the non-local exit.  But this I feel is more of a workaround than a solution, and I still do not understand the original problem.

Thanks.

Re: accept-process-output throws

[Eli Zaretskii]

> From: JD Smith <jdtsmith@gmail.com>
> Date: Wed, 24 Nov 2021 00:14:06 -0500
> Cc: emacs-devel@gnu.org
> 
> 
> 
> > On Nov 23, 2021, at 10:25 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> > Is this in "emacs -Q"?  If so, how did you start the process, and what
> > do you have in the process-filter and process-sentinel functions for
> > this process?
> 
> This is in a customized inferior python mode (a comint-derived mode), running iPython as the process.   I discovered the issue with `python-shell-send-string-no-output', which adds its own pre-output-filter to accumulate and discard process output.  The no-output commands are being sent to the process from a post-command hook which kicks off a timer (to gather completion and syntax data).  The command results arrive promptly, but when the non-local exit occurs, they “escape" the pre-output filtering and show up in the buffer.  When this occurs, it is always as a result of the non-local exit of `accept-process-output’.  I’ve found the problem gets more severe when the PCH-launched timer's delay is very small (<0.1s), i.e. when commands are sent in quick succession while typing.
> 
> I do not have an emacs -Q demo of this but can look at creating one with the stock python-shell-mode.  In the meantime I’ve found that if I bind (inhibit-quit t), remove `with-local-quit' and specify a timeout in `accept-process-output' (to silence the warning about the blocking call), this entirely prevents the non-local exit.  But this I feel is more of a workaround than a solution, and I still do not understand the original problem.

In this kind of complicated situations, I usually find it much simpler
and more efficient to run under GDB with a breakpoint on Fsignal and
Fthrow.  Then, when one of these breaks, examine the backtrace to find
out which code exited non-locally.

Re: accept-process-output throws

[JD Smith]

> In this kind of complicated situations, I usually find it much simpler
> and more efficient to run under GDB with a breakpoint on Fsignal and
> Fthrow.  Then, when one of these breaks, examine the backtrace to find
> out which code exited non-locally.

Thanks Eli.  I believe I have figured it out (without GDB’s help) — the UI I was using with the small timer delay was effectively wrapping my function in while-no-input.  The fix could have been as easy as (let (throw-on-input) ….).  I mention this in the hopes it saves someone else a day of debugging!  

What this does make clear is that async process output and while-no-input are like oil and water.  It also points to the value of cancellable futures as a kinder way to tell a process-awaiting function “stop what you are doing, I’ve got more input here”.  I suppose this would require accept-process-output to have such a matching cancellation function, and to then signal a specific “I was cancelled" error.