From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#48118: 27.1; 28; Only first process receives output with multiple running processes Date: Fri, 30 Apr 2021 21:06:22 +0300 Message-ID: <83y2czabq9.fsf@gnu.org> References: <64c194f9-b984-adaa-d5fd-86aa3ed3833a@daniel-mendler.de> <83wnsjc0vd.fsf@gnu.org> <83tunnc0hz.fsf@gnu.org> <83pmybc03l.fsf@gnu.org> <83o8dvbyyz.fsf@gnu.org> <83bl9vbw8h.fsf@gnu.org> <70ea83e2-fc9e-6feb-240c-ed41abac5254@daniel-mendler.de> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="19301"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 48118@debbugs.gnu.org To: Daniel Mendler Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Apr 30 20:24:11 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lcXoB-0004vA-DR for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 30 Apr 2021 20:24:11 +0200 Original-Received: from localhost ([::1]:49694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcXoA-00044p-Fx for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 30 Apr 2021 14:24:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51028) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcXXa-0002Ac-M0 for bug-gnu-emacs@gnu.org; Fri, 30 Apr 2021 14:07:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:48267) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcXXa-0003ny-CR for bug-gnu-emacs@gnu.org; Fri, 30 Apr 2021 14:07:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lcXXa-0006Ty-8w for bug-gnu-emacs@gnu.org; Fri, 30 Apr 2021 14:07:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 30 Apr 2021 18:07:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48118 X-GNU-PR-Package: emacs Original-Received: via spool by 48118-submit@debbugs.gnu.org id=B48118.161980600124890 (code B ref 48118); Fri, 30 Apr 2021 18:07:02 +0000 Original-Received: (at 48118) by debbugs.gnu.org; 30 Apr 2021 18:06:41 +0000 Original-Received: from localhost ([127.0.0.1]:59813 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcXXF-0006TO-E3 for submit@debbugs.gnu.org; Fri, 30 Apr 2021 14:06:41 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:42620) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcXXE-0006TD-F6 for 48118@debbugs.gnu.org; Fri, 30 Apr 2021 14:06:40 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:53643) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcXX8-0003dr-BC; Fri, 30 Apr 2021 14:06:34 -0400 Original-Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2678 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lcXX4-000491-Rx; Fri, 30 Apr 2021 14:06:32 -0400 In-Reply-To: <70ea83e2-fc9e-6feb-240c-ed41abac5254@daniel-mendler.de> (message from Daniel Mendler on Fri, 30 Apr 2021 18:17:49 +0200) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:205294 Archived-At: > Cc: 48118@debbugs.gnu.org > From: Daniel Mendler > Date: Fri, 30 Apr 2021 18:17:49 +0200 > > > Bytes read has a problem when processes produce output a very > > different rates. Time spent to handle may (and usually does) mean the > > filter function does something expensive, it doesn't necessarily tell > > anything about the output from the subprocess. > > Of course it is not possible to find a perfect scheduling algorithm. But > how does the OS handle it if you have multiple processes which produce > output with vastly different rates? I am not claiming this problem has > been solved, but there are certainly some heuristics. Emacs is also > dependent on the OS scheduling, depending on how Emacs schedules its > reads/writes from the processes, the OS scheduler adjusts accordingly. > This furthermore complicates the picture. I'm sure patches to tune the scheduling to specific use cases will be welcome. My gut feeling is that we will need some variables to allow Lisp programs to tell Emacs how to handle the various kinds of processes and combinations thereof, but if you can come up with patches that automatically adapt to the process's behavior, that would be even better. You could also try playing with the value of read-process-output-max, perhaps enlarging it will make the problem in your case less severe. > > If you read the code, you will see this isn't what happens. What > > happens is that Emacs reads a chunk of output from the first process > > it sees ready, then it goes back and re-checks which processes are > > ready -- and in your scenario I think it again sees that the first > > process is ready. > > This is what we assumed. Emacs could check the second process the next > time. This way one may get a slightly more fair behavior. It would > certainly not be perfect and you could throw scenarios at it which would > make it behave unexpectedly. It may behave a bit more expectedly in the > common case? We could try that (conditioned on some new variable) and see if this has downsides. > > I suggest to read the code of wait_reading_process_output, it has some > > non-trivial logic in this department. > > I will do that. Has this problem discussed before? I don't think so, but my memory is not to be trusted.