bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt

[Ioannis Kappas <ioannis.kappas@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3381 bytes --]

Some analysis follows

    Looking at the code, it appears that /message/s in -batch mode are
    delivered to the process' standard error by the
    [editfnc.c:message()]
      ->[xdisp.c:message3_nolog()]
        ->[xdispl.c:message_to_stderr()]
    function, i.e. messages are written to stderr. The latter function
    uses the standard fwrite and fputc calls to deliver the message to
    stderr.

    The buffering regime used for stderror on windows appears to be different
    depending whether a program was started from the command prompt or
    not. When started from the command prompt stderr is
    unbuffered, while there is a buffer employed otherwise and output
    is only delivered when that buffer is full.

    Below is a sample C program that can demonstrate the behaviour,
    which writes a single character to the standard output and then
exits after 2 seconds:

      #include <stdio.h>
      #include <unistd.h>
      #include <windows.h>

      int main()
      {

fwrite("t", 1, sizeof("t"), stderr);

sleep(2);

return 0;
      }


    | case | invoked from           | result
         |
    |------+------------------------+-------------------------------------------|
    |  1   | command prompt         | prints "t", then after 2 seconds exits |
    |  2   | outside command prompt | after about 2 seconds prints
"t", then exits |

    It appears that the buffer size used in #2 is 2048 bytes (i.e. the
    message is only immediately displayed when is more than 2048 bytes long).

    A solution thus to correct this behavior in emacs -batch on
    windows-nt would be to check if it is connected to the console,
    and when not, set stderr mode to unbuffered.

    A) Likely the win32 api provide GetConsoleMode() that can be used to
    check if a standard HANDLE is attached to the console. Given the
    STD_ERROR_HANDLE as an argument, it will  return non-zero if is
    attached to the console or a 0 otherwise, setting GetLastError()
to "The handle is
    invalid" message.

    B) To set stderr to an unbuffered mode, we can use standard C's
    setvbuf() with _IONBF (no buffering).

    C) The location where the descriptors are prepared for use in emacs
    appear to be at w32.c:init_ntproc().

    If the above analysis is correct, here is an example patch that
puts A/B/C together:

--- a/src/w32.c
+++ b/src/w32.c
@@ -10216,6 +10216,19 @@ init_ntproc (int dumping)
     else
       _open ("nul", O_TEXT | O_NOINHERIT | O_WRONLY);
     _fdopen (2, "w");
+
+    /* When in -batch mode, windows appears to buffer stderr when it
+       is invoked outside of the command prompt. This has the
+       undesired side effect that /message/s are not output unless the
+       buffer is full or emacs exits */
+    DWORD console_mode;
+    if (noninteractive &&
+ // stderr is not attached to the console
+ !GetConsoleMode (GetStdHandle(STD_ERROR_HANDLE), &console_mode))
+      {
+ // set stderr to unbuffered
+ setvbuf (stderr, NULL, _IONBF, 0);
+      }

  I have also attached an ert tests to capture the correct
  behavior. The test invokes an emacs -batch process which prints a
  /message/ and exits after five seconds. The test check immediately
  after two seconds if the batch process has output the message as
  expected. It can be invoked with:

: emacs.exe -batch -l ert -l batch-message-test.el -f
ert-run-tests-batch-and-exit

[-- Attachment #2: batch-message-test.el --]
[-- Type: application/octet-stream, Size: 1141 bytes --]

;;; -*- lexical-binding: t; -*-
(require 'ert)

(ert-deftest batch-message ()
  "Test that when invoking emacs in batch mode as a
subprocess (i.e. not directly from a terminal), /message/s are
flushed to output immediately. "
  (message ":system %s :version %s" system-configuration emacs-version)

  (with-temp-buffer 
    (let* ((proc-buf (current-buffer))

	   ;; start a new emacs process that will print a message,
	   ;; wait for five second and then exit.
	   (cmd (format "emacs -Q --batch --eval=\"%s\""
			'(progn (message \\\"hi\\\") 
				(sit-for 5))))
	   (proc (start-file-process-shell-command
                  "test/batch-message" proc-buf cmd))

	   (outputs '()))
      
      ;; capture emacs output
      (set-process-filter proc (lambda (proc output)
				 (push output outputs)))
      
      ;; wait for the process to start
      (sleep-for 2)
      (should (equal 'run (process-status proc)))
      ;; program should have printed out "hi\n"
      (should (equal '("hi\n") outputs))

      ;; kill process and wait for it to die
      (delete-process proc)
      (sleep-for 1))))