no buffer to flush the error output in ?

no buffer to flush the error output in ?

[Phil Estival]

[[info:elisp#Synchronous Processes]] says

   You can’t directly specify a buffer to put the
   error output in; that is too difficult to
   implement.

Can someone explain why please ?

   But you can achieve this result by
   sending the error output to a temporary file and
   then inserting the file into a buffer when the
   subprocess finishes.

Only when it finishes? But there must certainly exist a way
to have stdout/err be separated buffered streams... right ?

Phil

Re: no buffer to flush the error output in ?

[Eli Zaretskii]

> Date: Tue, 17 Sep 2024 12:06:54 +0200
> From: Phil Estival <pe@7d.nz>
> 
> [[info:elisp#Synchronous Processes]] says
> 
>    You can’t directly specify a buffer to put the
>    error output in; that is too difficult to
>    implement.
> 
> Can someone explain why please ?

Because you'd need to read process output from two independent file
descriptors at the same time, I guess.  E.g., the rate of writing to
each one of the descriptors could be (and very frequently is) very
different, and you don't want to slow down reading from stdout because
you want to check stderr.

>    But you can achieve this result by
>    sending the error output to a temporary file and
>    then inserting the file into a buffer when the
>    subprocess finishes.
> 
> Only when it finishes? But there must certainly exist a way
> to have stdout/err be separated buffered streams... right ?

You can separate them, yes, but the problem is how to read from both,
see above.  And since Emacs waits for the process to exit anyway, why
does it matter that you get the stderr stuff only after it exits?

Re: no buffer to flush the error output in ?

[Phil Estival]

* [2024-09-17 15:06] Eli Zaretskii:
>> Date: Tue, 17 Sep 2024 12:06:54 +0200
>> From: Phil Estival <pe@7d.nz>
>>
>> [[info:elisp#Synchronous Processes]] says
>>
>>     You can’t directly specify a buffer to put the
>>     error output in; that is too difficult to
>>     implement.
>>
>> Can someone explain why please ?
> 
> Because you'd need to read process output from two independent file
> descriptors at the same time, I guess.  E.g., the rate of readning to
> each one of the descriptors could be (and very frequently is) very
> different, 
 > and you don't want to slow down reading from stdout because
 > you want to check stderr.

I see. This is occuring in wait_reading_process_output.
There is a process prioritization of lower file descriptors,
but this would impede the overall output reading.

I had missed [[info:elisp#Asynchronous Processes][elisp#Asynchronous 
Processes]].  make-process can have a buffer or a pipe process attached
to the standard error of subprocess.  (If STDERR is nil, standard error 
is mixed with standard output).


My questionning comes from an interactive command shell based on comint,
the performance are not significant. In the end the output has to be
separated somehow (prompt, result output and errors). When not
separating the channels, additional escapde codes are needed
plus various semantics that has to be filtered with regex acrobatics,
e.g.
- 165 matches in 147 lines for "regex" in buffer: comint.el.gz<28.2>
- 
https://gitlab.freedesktop.org/Per_Bothner/specifications/blob/master/proposals/semantic-prompts.md
- http://per.bothner.com/blog/2019/shell-integration-proposal/


 >>     But you can achieve this result by
 >>     sending the error output to a temporary file and
 >>     then inserting the file into a buffer when the
 >>     subprocess finishes.
 >>
 >> Only when it finishes? But there must certainly exist a way
 >> to have stdout/err be separated buffered streams... right ?
 >
 > You can separate them, yes, but the problem is how to read from both,
 > see above.  And since Emacs waits for the process to exit anyway, why
 > does it matter that you get the stderr stuff only after it exits?
 >

I'll to get back to comint and try to understading if starting the
subprocess by providing the stderr channel is meaningfull.

Cheers,

Phil

Re: no buffer to flush the error output in ?

[Eli Zaretskii]

> Date: Wed, 18 Sep 2024 01:04:25 +0200
> Cc: emacs-devel@gnu.org
> From: Phil Estival <pe@7d.nz>
> 
> I had missed [[info:elisp#Asynchronous Processes][elisp#Asynchronous 
> Processes]].  make-process can have a buffer or a pipe process attached
> to the standard error of subprocess.  (If STDERR is nil, standard error 
> is mixed with standard output).

We do have a solution for async subprocesses, but you specifically
asked about synchronous subprocesses, and the passage from the manual
which you quoted was talking about synchronous subprocesses, and only
about them.

> My questionning comes from an interactive command shell based on comint,
> the performance are not significant. In the end the output has to be
> separated somehow (prompt, result output and errors). When not
> separating the channels, additional escapde codes are needed
> plus various semantics that has to be filtered with regex acrobatics,
> e.g.
> - 165 matches in 147 lines for "regex" in buffer: comint.el.gz<28.2>

If you use comint, then you are using async subprocesses, don't you?
If so using :stderr attribute (and/or pipe processes) is what you
want, I think.