From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ioannis Kappas Newsgroups: gmane.emacs.bugs Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Date: Thu, 11 Feb 2021 08:10:34 +0000 Message-ID: References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="22703"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 46388@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Feb 11 09:11:15 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lA74E-0005oi-R5 for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 11 Feb 2021 09:11:14 +0100 Original-Received: from localhost ([::1]:34688 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lA74D-0002Aa-KD for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 11 Feb 2021 03:11:13 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59186) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lA743-0002AL-Du for bug-gnu-emacs@gnu.org; Thu, 11 Feb 2021 03:11:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:46104) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lA743-0003pc-6P for bug-gnu-emacs@gnu.org; Thu, 11 Feb 2021 03:11:03 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lA743-00024e-2S for bug-gnu-emacs@gnu.org; Thu, 11 Feb 2021 03:11:03 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 11 Feb 2021 08:11:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs Original-Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.16130310527942 (code B ref 46388); Thu, 11 Feb 2021 08:11:03 +0000 Original-Received: (at 46388) by debbugs.gnu.org; 11 Feb 2021 08:10:52 +0000 Original-Received: from localhost ([127.0.0.1]:57648 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lA73s-000242-5s for submit@debbugs.gnu.org; Thu, 11 Feb 2021 03:10:52 -0500 Original-Received: from mail-ot1-f41.google.com ([209.85.210.41]:34448) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lA73q-00023p-Ku for 46388@debbugs.gnu.org; Thu, 11 Feb 2021 03:10:51 -0500 Original-Received: by mail-ot1-f41.google.com with SMTP id y11so4434249otq.1 for <46388@debbugs.gnu.org>; Thu, 11 Feb 2021 00:10:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ykpcj/2Mr1v/Y/rDQ3NeRhAv8rELUE5N/GaeWcd3udo=; b=pL4fR8TJjZXqCRk85z9sIT2+tfWK6Z2E8mO0BOeKL12RnHD+ZOZlFfBwuHFXZnK5wz QlScD3F5vka3577Xkspk5CgM+aCkY91tLMr0PfdZXkWYrfVSgwzRw45vzfUzDrjrAHtr hUbeL2mLD1Hqc4EFw0QLCNbnxZU+Np7bbf6D1HzqkN6dOy+WBZegM8geM9zlkKojmGRM VqISrO/C5hQpqWfMzSNflwRonbeziCmRrzQObHwRA+Y5yy8bu+/zlG+43qrLQLSHN0Wy UkVCoo1AujcctGFoSPnE1zSM5FrkAGXjeThpS53bpf00lVhl2mmGGrjMtbVkGDAc7w18 +n7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ykpcj/2Mr1v/Y/rDQ3NeRhAv8rELUE5N/GaeWcd3udo=; b=GaWnF51Hv4iONdqqMccGvPK6GHISqog0iozkanAbBWRhbhZW17lIjvb3ojzxoe2lZH XVIO5Cv6/mOsZ+vBJAAJqwSbn3EPOKO42fPf67fgX+mUYbi9QJaKKyKW8OtQ/hUcvNHt Z0AsqexrAzug3+rrwCuxQ+xVn4b0rSyyyZN/Z3JFJNzqq8c4r661fH2ZLqc+2q8WbPGO zWU9n6TUZxNtVorXTJ8EhKCIpkMxwG+vhVkg2so1vqks7CMIa8wT6+HCUf96TRW5BeAs IX2oXETyJ0pOF38a908/5N8ZNxyZQgLibu76iUaCJMmDLeRmiUzzjJnBJT023FtozAU4 UoyA== X-Gm-Message-State: AOAM531toiszV9s3ZJWGQIp8P+A9S/JVN03miy0/6PDNhyH3POijS55r ttLWEfwt3PGOVukQjHh7t+nssmNwaq1AtX+JbQP57fpdIuc5TQ== X-Google-Smtp-Source: ABdhPJzhXL5hgZEH6t0OxDNWEtusJij10unciUuExl5N8qe2IloPyRmSLrlmqBimPY+2F/CatIETsct2dQjZe70KI70= X-Received: by 2002:a9d:506:: with SMTP id 6mr5161059otw.5.1613031044953; Thu, 11 Feb 2021 00:10:44 -0800 (PST) In-Reply-To: <83pn17j4pw.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:199800 Archived-At: On Wed, Feb 10, 2021 at 3:57 PM Eli Zaretskii wrote: > > > > | # | System | emacs -batch invoked from | MESSAGE behavior > > > > | > > |---+------------+---------------------------+----------------------------------------------------------------------------------------------------| > > | 1 | Linux | bash | any MESSAGE is > > immediately printed, i.e. stderr is unbuffered > > | > > | 2 | Linux | emacs eshell/shell etc | >> > > Did you try this 2nd item when the connection type for the subprocess > is 'pipe'? Because otherwise we are comparing apples to oranges. I haven't, how do I do this? I was merely describing the plain user experience of running emacs -batch from a shell. for case #2 for example: 1. On Linux, open emacs 2. M-x eshell 3. at the eshell prompt type the following command: 3.1 emacs -Q --batch --eval="(progn (message \"hi\") (sit-for 5))" 4. observe the result 4.1 "hi" is printed immediately, emacs -batch exits after five seconds. In my tests, the second command was ran in all five cases, without applying any other configuration to the parent command prompt/shell/eshell process. Apologies if this was not clear. > > I think I've just realized what you were saying from the > > beginning. That the difference in behavior is expected, since > > it is the parent process which decides the buffering regime to be > > used for the subprocess. Thus in #5, it is emacs on windows that > > decided to invoke emacs -batch as a subprocess using pipes, which > > has resulted in emacs -batch's stderr to be buffered. > > That is correct. As we don't support PTYs on Windows, we can only use > pipes for communicating with subprocesses there. > > Btw, did you try to play with the value of w32-pipe-buffer-size, > e.g. setting it to a small value? No, I haven't experimented with pipes yet, my next plan is to study how subprocess are invoked from within emacs and mintty (since both demonstrate the same behavior). > > If the current behavior is indeed the correct expected behavior, how do I > > flush text message to stderr (or even stdout) from an emacs > > -batch script/eval? > > My reading of the code is that we already fflush stderr after emitting > a message, so this should already happen. See message_to_stderr. If > that still doesn't help, then there's some buffering in the OS (for > example, in the pipe machinery itself), which we cannot control. the xdisp.c:message_to_stderr() is the first function i studied with gdb when I started the investigation. Unless I've missed something, it does not seem to lead to calling fflush (under windows at least): 1. it will sysdep.c:errwrite() the message 1.1 errwrite() will call sysdep.c:errstream() to get stderr 1.1.1 errstream() *may* call fflush on stderr in some system (via buferr static variable), but this does not happen under windows-nt at least. 1.2 will call fwrite to write the message 2. it may call sysdep.c:errputc() to to append a newline, following exactly the same logic as for errwrite(), but using fputc() instead /* Log the message M to stderr. Log an empty line if M is not a string. */ static void message_to_stderr (Lisp_Object m) { if (noninteractive_need_newline) { noninteractive_need_newline = false; errputc ('\n'); } if (STRINGP (m)) { Lisp_Object coding_system = Vlocale_coding_system; Lisp_Object s; if (!NILP (Vcoding_system_for_write)) coding_system = Vcoding_system_for_write; if (!NILP (coding_system)) s = code_convert_string_norecord (m, coding_system, true); else s = m; errwrite (SDATA (s), SBYTES (s)); } if (STRINGP (m) || !cursor_in_echo_area) errputc ('\n'); } void errputc (int c) { fputc_unlocked (c, errstream ()); } void errwrite (void const *buf, ptrdiff_t nbuf) { fwrite_unlocked (buf, 1, nbuf, errstream ()); } /* Return the error output stream. */ static FILE * errstream (void) { FILE *err = buferr; if (!err) return stderr; fflush_unlocked (stderr); return err; } in sysdep.c:init_standard_fds() ... /* Set buferr if possible on platforms defining _PC_PIPE_BUF, as they support the notion of atomic writes to pipes. */ #ifdef _PC_PIPE_BUF buferr = fdopen (STDERR_FILENO, "w"); if (buferr) setvbuf (buferr, NULL, _IOLBF, 0); #endif ... > There were some changes in this area lately (that's the discussion > from 2019 I mentioned before): we now try to make a buffered variant > of stderr, and use that for error messages. The reason, in a > nutshell, is that when you build Emacs with "make -jN", several copies > of the Emacs process can work in parallel, so it was deemed better to > have their messages be emitted atomically, instead of being > interspersed with one another, which produces an illegible mess. > However, that change makes a line-buffered variant of stderr, and on > Windows line buffering is the same as full buffering. So maybe we > would need more changes in that area, for example some variable to > control this behavior instead of making it unconditional. Noted, thanks again!