Single process output reading

Single process output reading

[Milan Zamazal]

I think a way to read an output of just a single asynchronous process in
Emacs is needed.  I.e. a way to invoke accept-process-output without
possible invocation of another process filter function or a timer.

Why is it needed?  Currently, there's no way to do it AFAIK, which
basically means that all process filter functions must be reentrant.
Making a process filter function reentrant is not always a trivial task
and it's useless to do so except that the scenario like the following
one can happen:

A process filter function `foo' is invoked.  It reports about the
process reading progress via the `message' function.  `message' can be
advised, e.g. by some speech output function.  The speech output
function communicates with a speech synthesizer through a network
process and it calls accept-process-output to read the synthesizer's
answer.  But within the accept-process-output, `foo' can be invoked
again.  If it is not reentrant, errors can happen.

This is what actually happens in the speechd-el speech output system
which, among others, advises the `message' function.  So any filter
function calling `message' (or doing some other actions handled by the
speech output system) is fragile to this problem.

There's a workaround for this: To write an external program resending
the data/answer to/from the desired network connection and to call this
program synchronously from Emacs (anytime a piece of data is to be sent
to the network connection) with call-process.  This is what I actually
did, but I don't think it's a good way of solving the problem, both from
the point of view of the programmer and the user.

IMO a better way would be to allow restricting accept-process-output to
just a single process output reading.  Then the speech output system,
aware of possible consequences of advising 'message' etc., could invoke
accept-process-output in the restricted form, without the danger of
unwanted side effects.

I need this feature, so I'm willing to implement it unless someone more
competent would like to do it.

My questions are:

- Do you agree with my analysis or do you have a better suggestion how
  to solve the presented problem?

- Do you have some suggestions how the restricted invocation of
  accept-process-output might look?  New optional argument?

- Do you have some implementation advice?

Thanks for any help.

Regards,

Milan Zamazal

-- 
http://www.zamazal.org

Re: Single process output reading

[Kim F. Storm]

Milan Zamazal <pdm@brailcom.org> writes:

> I think a way to read an output of just a single asynchronous process in
> Emacs is needed.  I.e. a way to invoke accept-process-output without
> possible invocation of another process filter function or a timer.

Your reasons sound valid to me, but blocking everything including timers
isn't very good, but may be acceptable for very short periods of time.

> IMO a better way would be to allow restricting accept-process-output to
> just a single process output reading.  Then the speech output system,
> aware of possible consequences of advising 'message' etc., could invoke
> accept-process-output in the restricted form, without the danger of
> unwanted side effects.

Here's an (untested) patch that adds a "just-this-one" arg to
accept-process-output

To avoid running timers, specify an integer as fourth arg, e.g.
  (accept-process-output your-process nil nil 1)


*** process.c	29 Jul 2004 16:46:31 +0200	1.434
--- process.c	16 Aug 2004 16:25:16 +0200	
***************
*** 3718,3734 ****
  }
  \f
  DEFUN ("accept-process-output", Faccept_process_output, Saccept_process_output,
!        0, 3, 0,
         doc: /* Allow any pending output from subprocesses to be read by Emacs.
  It is read into the process' buffers or given to their filter functions.
  Non-nil arg PROCESS means do not return until some output has been received
  from PROCESS.
  Non-nil second arg TIMEOUT and third arg TIMEOUT-MSECS are number of
  seconds and microseconds to wait; return after that much time whether
  or not there is input.
  Return non-nil iff we received any output before the timeout expired.  */)
!      (process, timeout, timeout_msecs)
!      register Lisp_Object process, timeout, timeout_msecs;
  {
    int seconds;
    int useconds;
--- 3718,3737 ----
  }
  \f
  DEFUN ("accept-process-output", Faccept_process_output, Saccept_process_output,
!        0, 4, 0,
         doc: /* Allow any pending output from subprocesses to be read by Emacs.
  It is read into the process' buffers or given to their filter functions.
  Non-nil arg PROCESS means do not return until some output has been received
  from PROCESS.
+ If optional fourth arg JUST-THIS-ONE is non-nil, only accept output
+ from PROCESS, suspending reading output from other processes.
+ If JUST-THIS-ONE is an integer, don't run any timers either.
  Non-nil second arg TIMEOUT and third arg TIMEOUT-MSECS are number of
  seconds and microseconds to wait; return after that much time whether
  or not there is input.
  Return non-nil iff we received any output before the timeout expired.  */)
!      (process, timeout, timeout_msecs, just_this_one)
!      register Lisp_Object process, timeout, timeout_msecs, just_this_one;
  {
    int seconds;
    int useconds;
***************
*** 3776,3782 ****
      XSETFASTINT (process, 0);
  
    return
!     (wait_reading_process_input (seconds, useconds, process, 0)
       ? Qt : Qnil);
  }
  
--- 3779,3787 ----
      XSETFASTINT (process, 0);
  
    return
!     (wait_reading_process_input (seconds, useconds, process,
! 				 NILP (just_this_one) ? 0 :
! 				 !INTEGERP (just_this_one) ? -1 : -2)
       ? Qt : Qnil);
  }
  
***************
*** 4032,4037 ****
--- 4037,4043 ----
    EMACS_TIME timeout, end_time;
    int wait_channel = -1;
    struct Lisp_Process *wait_proc = 0;
+   int just_the_wait_proc = 0;
    int got_some_input = 0;
    /* Either nil or a cons cell, the car of which is of interest and
       may be changed outside of this routine.  */
***************
*** 4048,4053 ****
--- 4054,4064 ----
        wait_proc = XPROCESS (read_kbd);
        wait_channel = XINT (wait_proc->infd);
        XSETFASTINT (read_kbd, 0);
+       if (do_display < 0)
+ 	{
+ 	  just_the_wait_proc = do_display;
+ 	  do_display = 0;
+ 	}
      }
  
    /* If waiting for non-nil in a cell, record where.  */
***************
*** 4122,4128 ****
  	 But not if wait_for_cell; in those cases,
  	 the wait is supposed to be short,
  	 and those callers cannot handle running arbitrary Lisp code here.  */
!       if (NILP (wait_for_cell))
  	{
  	  EMACS_TIME timer_delay;
  
--- 4133,4140 ----
  	 But not if wait_for_cell; in those cases,
  	 the wait is supposed to be short,
  	 and those callers cannot handle running arbitrary Lisp code here.  */
!       if (NILP (wait_for_cell)
! 	  && just_the_wait_proc != -2)
  	{
  	  EMACS_TIME timer_delay;
  
***************
*** 4258,4264 ****
  
        /* Wait till there is something to do */
  
!       if (!NILP (wait_for_cell))
  	{
  	  Available = non_process_wait_mask;
  	  check_connect = check_delay = 0;
--- 4270,4281 ----
  
        /* Wait till there is something to do */
  
!       if (just_the_wait_proc)
! 	{
! 	  FD_SET (wait_channel, &Available);
! 	  check_connect = check_delay = 0;
! 	}
!       else if (!NILP (wait_for_cell))
  	{
  	  Available = non_process_wait_mask;
  	  check_connect = check_delay = 0;


--
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Single process output reading

[Milan Zamazal]

>>>>> "KFS" == Kim F Storm <storm@cua.dk> writes:

    KFS> blocking everything including timers isn't very good, but may
    KFS> be acceptable for very short periods of time.

Yes, that's what I need.

    KFS> Here's an (untested) patch that adds a "just-this-one" arg to
    KFS> accept-process-output

Thanks!

Unfortunately, when called as follows:

  (accept-process-output your-process nil nil 1)

Emacs segfaults.  FWIW, the traceback is 

  #0  0x08186fa7 in wait_reading_process_input (time_limit=-1, microsecs=0,
      read_kbd=0, do_display=0) at process.c:4275
  #1  0x08186574 in Faccept_process_output (process=148636188, timeout=0,
      timeout_msecs=138317841, just_this_one=8) at process.c:3781
  #2  0x08154eb8 in Feval (form=138037816) at eval.c:2101
  ...

The problem is that wait_channel is -1 on the segfaulting line:

  FD_SET (wait_channel, &Available);

I don't understand it, since wait_proc->infd is 72 (`pr' says 9) and the
same call without the JUST-THIS-ONE argument

  (accept-process-output your-process nil nil nil)

works fine.

Regards,

Milan Zamazal

-- 
It is the reformer, who is anxious for the reform, and not society, from which
he should expect nothing better than opposition, abhorrence and even mortal
persecution.                                                   -- M. K. Gandhi

Re: Single process output reading

[Kim F. Storm]

Milan Zamazal <pdm@brailcom.org> writes:

> The problem is that wait_channel is -1 on the segfaulting line:

Ah, yes, that can happen.

> 
>   FD_SET (wait_channel, &Available);

Try this line instead:

	  FD_SET (XINT (wait_proc->infd), &Available);

Re: Single process output reading

[Milan Zamazal]

>>>>> "KFS" == Kim F Storm <storm@cua.dk> writes:

    >> FD_SET (wait_channel, &Available);

    KFS> Try this line instead:

    KFS> 	  FD_SET (XINT (wait_proc->infd), &Available);

It works well now, thanks!

Regards,

Milan Zamazal

-- 
It is the reformer, who is anxious for the reform, and not society, from which
he should expect nothing better than opposition, abhorrence and even mortal
persecution.                                                   -- M. K. Gandhi

Re: Single process output reading

[Richard Stallman]

    IMO a better way would be to allow restricting accept-process-output to
    just a single process output reading.  Then the speech output system,
    aware of possible consequences of advising 'message' etc., could invoke
    accept-process-output in the restricted form, without the danger of
    unwanted side effects.

I think it would be reasonable to provide a way to tell
wait_reading_process_output to ignore certain processes and read from
certain other processes.  I worry about the idea of not running timers,
though, and that seems unnecessary for what you need.  I think it
would be better to implement the feature so that timers do run.

I think it would be better to clean up the calling conventions
of wait_reading_process_output rather than try to squeeze everything
into the existing argumemts.

Re: Single process output reading

[Milan Zamazal]

>>>>> "RS" == Richard Stallman <rms@gnu.org> writes:

    RS> I worry about the idea of not running timers, though, 

Why?  There are already many ways how to prevent running timers for a
while in Emacs, like running a piece of Elisp code or calling a process
synchronously.  So what's the problem with the special-case
accept-process-output call?

    RS> and that seems unnecessary for what you need.  I think it would
    RS> be better to implement the feature so that timers do run.

I think that would mean that timers should be generally reentrant, as
well as some kinds of speech output functions.

Regards,

Milan Zamazal

-- 
SomeProgrammersLikeWritingLikeThis.IDontThinkThisFormOfCommunicationIs\
AGoodIdea.IApologizeToAllWhoCantReadMyTextsWrittenInATraditionalStyle.

Re: Single process output reading

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

>     IMO a better way would be to allow restricting accept-process-output to
>     just a single process output reading.  Then the speech output system,
>     aware of possible consequences of advising 'message' etc., could invoke
>     accept-process-output in the restricted form, without the danger of
>     unwanted side effects.
> 
> I think it would be reasonable to provide a way to tell
> wait_reading_process_output to ignore certain processes and read from
> certain other processes.  I worry about the idea of not running timers,
> though, and that seems unnecessary for what you need.  I think it
> would be better to implement the feature so that timers do run.
> 
> I think it would be better to clean up the calling conventions
> of wait_reading_process_output rather than try to squeeze everything
> into the existing argumemts.

I can do that.

Should I rename wait_reading_process_input to wait_reading_process_output ?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Single process output reading

[Richard Stallman]

    Why?  There are already many ways how to prevent running timers for a
    while in Emacs, like running a piece of Elisp code or calling a process
    synchronously.  So what's the problem with the special-case
    accept-process-output call?

I guess you are right.

Re: Single process output reading

[Richard Stallman]

    Should I rename wait_reading_process_input to wait_reading_process_output ?

Indeed, it would be better to make the code consistent in regard
to whether we call this "input" or "output" ;-).