bug#48118: 27.1; 28; Only first process receives output with multiple running processes

[miha--- via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

[-- Attachment #1.1: Type: text/plain, Size: 853 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: miha@kamnitnik.top
>> Cc: mail@daniel-mendler.de, eliz@gnu.org, 48118@debbugs.gnu.org
>> Date: Mon, 24 May 2021 23:05:47 +0200
>> 
>> I propose the following simple patch. It makes output from multiple
>> /bin/yes programs arrive at the same rate and multiple grep processes
>> can run without them seemingly blocking each other.
>
> Thanks, but I don't think we can make such changes unconditionally.
> I'm okay with trying this by default, but we should have a Lisp
> variable that would allow to get back to the old behavior.  That's
> because if some user complains about some problems, and we think the
> problems are caused by this change, we could tell that user to flip
> the variable and see if the problems go away.
>
> That variable should also be in NEWS.

Revised patch.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Try-to-not-prioritise-reading-from-lower-fds.patch --]
[-- Type: text/x-patch, Size: 3816 bytes --]

From 77fb8097f2ed20f034230a5c61dee00880bbcd24 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miha=20Rihtar=C5=A1i=C4=8D?= <miha@kamnitnik.top>
Date: Mon, 24 May 2021 22:46:47 +0200
Subject: [PATCH] Try to not prioritise reading from lower fds

* src/process.c (wait_reading_process_output): When looping through
fds, continue from where we left off.
(syms_of_process): Vprocess_prioritize_lower_fds: New variable
---
 etc/NEWS      |  7 +++++++
 src/process.c | 27 +++++++++++++++++++++++++--
 2 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 1541b74a3b..cc767c8dc2 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -2706,6 +2706,13 @@ the Emacs Lisp reference manual for background.
 * Lisp Changes in Emacs 28.1
 
 +++
+** New variable 'process-prioritize-lower-fds'
+When looping through file descriptors to handle subprocess output, try
+to continue from where the previous loop left off instead of always
+beginning from file descriptor zero.  Set this variable to t to get
+the old behaviour.
+
+---
 ** New function 'sxhash-equal-including-properties'.
 This is identical to 'sxhash-equal' but accounting also for string
 properties.
diff --git a/src/process.c b/src/process.c
index 47a2a6f1a3..7bf55c203e 100644
--- a/src/process.c
+++ b/src/process.c
@@ -5134,6 +5134,7 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
 			     Lisp_Object wait_for_cell,
 			     struct Lisp_Process *wait_proc, int just_wait_proc)
 {
+  static int last_read_channel = -1;
   int channel, nfds;
   fd_set Available;
   fd_set Writeok;
@@ -5188,6 +5189,8 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
   while (1)
     {
       bool process_skipped = false;
+      bool wrapped;
+      int channel_start;
 
       /* If calling from keyboard input, do not quit
 	 since we want to return C-g as an input character.
@@ -5722,8 +5725,20 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
             d->func (channel, d->data);
 	}
 
-      for (channel = 0; channel <= max_desc; channel++)
-	{
+      channel_start
+	= !NILP (Vprocess_prioritize_lower_fds) ? 0 : last_read_channel + 1;
+
+      for (channel = channel_start, wrapped = false;
+	   !wrapped || (channel < channel_start && channel <= max_desc);
+	   channel++)
+        {
+	  if (channel > max_desc)
+	    {
+	      wrapped = true;
+	      channel = -1;
+	      continue;
+	    }
+
 	  if (FD_ISSET (channel, &Available)
 	      && ((fd_callback_info[channel].flags & (KEYBOARD_FD | PROCESS_FD))
 		  == PROCESS_FD))
@@ -5761,6 +5776,7 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
 		     don't try to read from any other processes
 		     before doing the select again.  */
 		  FD_ZERO (&Available);
+		  last_read_channel = channel;
 
 		  if (do_display)
 		    redisplay_preserve_echo_area (12);
@@ -8477,6 +8493,13 @@ syms_of_process (void)
 The variable takes effect when `start-process' is called.  */);
   Vprocess_adaptive_read_buffering = Qt;
 
+  DEFVAR_LISP ("process-prioritize-lower-fds", Vprocess_prioritize_lower_fds,
+	       doc: /* If nil, try to not prioritize reading from any process.
+Emacs loops through file descriptors to receive data from subprocesses.  After
+accepting output from the first file descriptor with available data, restart the
+loop from the file descriptor 0 if this option is non-nil.  */);
+  Vprocess_prioritize_lower_fds = Qnil;
+
   DEFVAR_LISP ("interrupt-process-functions", Vinterrupt_process_functions,
 	       doc: /* List of functions to be called for `interrupt-process'.
 The arguments of the functions are the same as for `interrupt-process'.
-- 
2.31.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]