Re: Forwarding stdout/stderr of a child process in batch mode

[Paul Pogonyshev <pogonyshev@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3848 bytes --]

> You may want to pass the "-u" option to Python, so that it uses
> unbuffered outputs.
>
> You're example is working for me, when using the "-u" option; i.e. I'm
> getting interleaved chunks of stderr/stdout.

It is only an example. In real usecase I will not call a Python script, in
fact, it can be any executable whatsoever, I don't know in advance. Also,
shell itself doesn't need `-u'. For example:

    sh -c 'sh -c "python 10.py"'
    sh -c 'sh -c "python 10.py"' >/dev/null

The second command shows that it doesn't write everything to stdout,
it still maintains stdout/stderr separation. The nested `-c' shows that
it works through any (I guess) level of indirection, and also if I use
different shells (`bash', `dash'), so it doesn't look like shell is special-
casing, it really can handle any commands.

Finally, even with `python -u' I get non-ideal behavior from Emacs:
stderr chunks come before the stdout one, even though the script writes
to stdout before stderr.

Paul


On Tue, 8 Dec 2020 at 22:43, Bruno Barbier <brubar.cs@gmail.com> wrote:

>
> Hi Paul,
>
> Paul Pogonyshev <pogonyshev@gmail.com> writes:
>
> > So, I got to separating stdout and stderr, but I get them in a wrong
> > order now...  Here is an example Python program that I use to generate
> > output (store as `10.py'):
> ...
>
> > When I run the Emacs script, I receive all stderr output line-by-line
> > (which is fine), but all stdout output comes in one chunk at the very
> > end, which is not...
> >
>
> You may want to pass the "-u" option to Python, so that it uses
> unbuffered outputs.
>
> You're example is working for me, when using the "-u" option; i.e. I'm
> getting interleaved chunks of stderr/stdout.
>
> Bruno
>
> > I cannot really use `pty' connection type, because I want (in real
> > usecase, not in the script) to additionally keep stdout and stderr in
> > separate buffers, for possible later processing.  Also, with pty I
> > wouldn't be able to use `message' or `princ' depending on the stream
> > used by the child process.
> >
> > Is there any way to fix this?
> >
> > Paul
> >
> >
> > On Sun, 6 Dec 2020 at 22:40, Paul Pogonyshev <pogonyshev@gmail.com>
> wrote:
> >
> >> Thank you, this seems to work (haven't tried separating stderr from
> >> stdout yet). Is there a normal way to wait for asynchronous process to
> >> terminate? Currently I have managed to make it work by throwing from
> >> sentinel, but this feels a bit hackish, maybe there is something
> >> better?
> >>
> >> Currently I have something like this:
> >>
> >>     (let ((process (make-process ... :sentinel (lambda (_process
> >> _event) (throw 'done nil)))))
> >>       (catch 'done
> >>         (while (process-live-p process)
> >>           (accept-process-output nil 60)))
> >>       ...
> >>
> >> Paul
> >>
> >> On Sun, 6 Dec 2020 at 14:37, Zhu Zihao <all_but_last@163.com> wrote:
> >> >
> >> >
> >> > IMO, in batch mode, `message` writes to stderr, `princ` writes to
> >> > stdout. You can install a filter for childprocess, and run functions I
> >> > mentioned above to forward these outputs.
> >> >
> >> >
> >> > Paul Pogonyshev writes:
> >> >
> >> > > Hi,
> >> > >
> >> > > I'm using Emacs in batch mode. I need to invoke a child process that
> >> > > is a longish operation (a few minutes). During this time, it writes
> to
> >> > > its stdout, so user will see that it is working and what exactly is
> >> > > being done. However, if I invoke it from Emacs (e.g. using
> >> > > `call-process') I see no way of forwarding this output to the "real"
> >> > > stdout. So, for a user this looks like the process (or batch Emacs
> on
> >> > > top of it) is hung.
> >> > >
> >> > > Am I missing a way to forward output?
> >> > >
> >> > > Paul
> >> >
> >> >
> >> > --
> >> > Retrieve my PGP public key: https://meta.sr.ht/~citreu.pgp
> >> >
> >> > Zihao
> >>
>

[-- Attachment #2: Type: text/html, Size: 5823 bytes --]