From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Daniel Mendler Newsgroups: gmane.emacs.bugs Subject: bug#48118: 27.1; 28; Only first process receives output with multiple running processes Date: Fri, 30 Apr 2021 16:45:25 +0200 Message-ID: References: <64c194f9-b984-adaa-d5fd-86aa3ed3833a@daniel-mendler.de> <83wnsjc0vd.fsf@gnu.org> <83tunnc0hz.fsf@gnu.org> <83pmybc03l.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="26396"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 48118@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Apr 30 16:48:08 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lcUR6-0006mH-47 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 30 Apr 2021 16:48:08 +0200 Original-Received: from localhost ([::1]:46736 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcUR5-00073t-4j for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 30 Apr 2021 10:48:07 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34314) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcUP6-0005No-K3 for bug-gnu-emacs@gnu.org; Fri, 30 Apr 2021 10:46:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:47964) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcUP4-0006ga-K2 for bug-gnu-emacs@gnu.org; Fri, 30 Apr 2021 10:46:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lcUP4-0003B2-HK for bug-gnu-emacs@gnu.org; Fri, 30 Apr 2021 10:46:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Daniel Mendler Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 30 Apr 2021 14:46:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48118 X-GNU-PR-Package: emacs Original-Received: via spool by 48118-submit@debbugs.gnu.org id=B48118.161979394112175 (code B ref 48118); Fri, 30 Apr 2021 14:46:02 +0000 Original-Received: (at 48118) by debbugs.gnu.org; 30 Apr 2021 14:45:41 +0000 Original-Received: from localhost ([127.0.0.1]:59510 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcUOd-0003AD-5Z for submit@debbugs.gnu.org; Fri, 30 Apr 2021 10:45:41 -0400 Original-Received: from server.qxqx.de ([178.63.65.180]:56285 helo=mail.qxqx.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcUOb-0003A0-5l for 48118@debbugs.gnu.org; Fri, 30 Apr 2021 10:45:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=cT+XHcynz2B8BbxLZsA6qcxV6cPuLkkwHKdjDp3KpzY=; b=JmxbkwzxHNg1FCSeCUDJaruoW5 hPz+UCPgojxgA5YE6LWPTMUWCuQ+ClOevu4CzGGUGM5qVxZ3FLLNlMRBn1INFOyQCoDH4XybsYEoT ulAm1CHyUnpgoqTH3dmpqxC3RSwtmTEu2sdBZ+YDjWBPjkHzTz6vUmaAn8NaNUodN6E8=; In-Reply-To: <83pmybc03l.fsf@gnu.org> Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:205254 Archived-At: On 4/30/21 4:34 PM, Eli Zaretskii wrote: >> So you say I should repeatedly stop the current process in the filter >> function in order to allow the other process to take precedence > > Yes. This is not a good solution. What if I have multiple packages which read from asynchronous processes? Maybe I cannot control all of the processes and their scheduling. >> since the underlying Emacs handling of asynchronous processes is >> unable to read from two processes at once? > > No. The problem is not the _ability_ to read from more than one > subprocess -- the ability does exist. The problem is that doing so > would run afoul of other scenarios. Which scenarios break? >> me. What is preventing Emacs from treating multiple processes >> fairly? > > I asked elsewhere what you mean by "fairly" in this context. > > But the general answer to your question is that Emacs knows nothing > about the processes, their importance, their output rates, and the > respective filter functions. Okay good. How can I configure it such that two processes both populate their buffers in a round-robin fashion? > First, what does "fairness" mean in this context? Given that there > are multiple simultaneous asynchronous subprocesses that produce > output at different rates and consume CPU at different levels, what > would it mean for Emacs to be "fair"? > > Second, suppose we have multiple ripgrep subprocesses running, and > Emacs will somehow read from each one of them in a round-robin > fashion: what and how do you expect the user to do to handle the > results of all those subprocesses simultaneously and "fairly"? I agree with you that fairness is a difficult problem. But the problem is omnipresent at the os level. There you have scheduling problems in the io layer, in the process scheduling layer, in the memory management layer and so on. There is certainly some heuristic that one can apply. For example by comparing the amount of data produced by multiple processes one could decide which process is read next. Or one can use a deadline criterion. I am not happy with the argument that Emacs cannot do any better than stopping the second process and only handle the first process. If you don't want to hardcode the scheduling behavior there could be some pluggable scheduler. This would be better than having to write my own scheduling by hand for each `make-process` call.