bug#56745: Cannot forward process output to both stdout and stderr (in batch mode)

bug#56745: Cannot forward process output to both stdout and stderr (in batch mode)

[Paul Pogonyshev]

[-- Attachment #1.1: Type: text/plain, Size: 2261 bytes --]

Especially in batch mode it is often useful to run an external process,
letting it write to stdout and stderr as if it was the main executed
process. As far as I see (and I have spent quite a lot of time
investigating this), with Emacs it is not possible.

I found two options for stdout+stderr, but with neither allowing me to
achieve quite what I want.

1. You can run a process with stdout and stderr combined, then use e.g.
process filter function to e.g. `princ' received output. The disadvantage
is that stdout and stderr get bunched together. For example, if someone
runs your script with `> /dev/null', he will also miss stderr output of the
child process. Perhaps even more importantly, robust automated processing
of output becomes impossible, as it can contain "garbage", i.e. output
meant for stderr.

2. You can split stdout and stderr e.g. by using `make-pipe-process' for
stderr and then e.g. use `princ' in the filter for the "real" process (only
stdout output) and `message' for stderr process. The problem is that here
output comes completely unsynchronized and (at least usually) in larger
chunks. If the child process intermixes stdout and stderr output, it comes
out not in the original order. You also lose interactivity if the child
process e.g. runs for minutes and prints progress messages regularly during
that time.

In comparison, with Python (or pretty much any other "real" language) this
is trivial, see attached `output-forwarder.py'. I don't see a way to
replicate these three lines in Elisp.

It would be nice if e.g. `make-process' grew an argument `:forward'
(possible values `t', `stderr' and `stdout'; the latter to forward only one
stream). It would be fine if this was incompatible with `:buffer' and
`:stderr', i.e. if this couldn't be specified together with those options.
I wouldn't mind any other interface either, as long as it worked.

Attachments:
* `mixed-output.py': a simple Python scripts that writes to both stdout and
stderr with small delays;
* `output-forwarder.py': a simple Python scripts that runs whatever command
is specified on the command line and forwards it output cleanly; usage,
e.g.: `$ ./output-forwarder.py ./mixed-output.py' or `$
./output-forwarder.py ./mixed-output.py >/dev/null'.

Paul

[-- Attachment #1.2: Type: text/html, Size: 2587 bytes --]

[-- Attachment #2: mixed-output.py --]
[-- Type: text/x-python, Size: 181 bytes --]

#! /usr/bin/env python

import sys
import time
for k in range (3):
    sys.stdout.write ("[STDOUT]\n")
    time.sleep (0.2)
    sys.stderr.write ("[stderr]\n")
    time.sleep (0.2)

[-- Attachment #3: output-forwarder.py --]
[-- Type: text/x-python, Size: 84 bytes --]

#! /usr/bin/env python

import subprocess
import sys

subprocess.run (sys.argv[1:])