bug#48118: 27.1; 28; Only first process receives output with multiple running processes

[Daniel Mendler <mail@daniel-mendler.de>]

On 4/30/21 4:34 PM, Eli Zaretskii wrote:
>> So you say I should repeatedly stop the current process in the filter
>> function in order to allow the other process to take precedence
> 
> Yes.

This is not a good solution. What if I have multiple packages which read
from asynchronous processes? Maybe I cannot control all of the processes
and their scheduling.

>> since the underlying Emacs handling of asynchronous processes is
>> unable to read from two processes at once?
> 
> No.  The problem is not the _ability_ to read from more than one
> subprocess -- the ability does exist.  The problem is that doing so
> would run afoul of other scenarios.

Which scenarios break?

>> me. What is preventing Emacs from treating multiple processes
>> fairly?
> 
> I asked elsewhere what you mean by "fairly" in this context.
> 
> But the general answer to your question is that Emacs knows nothing
> about the processes, their importance, their output rates, and the
> respective filter functions.

Okay good. How can I configure it such that two processes both populate
their buffers in a round-robin fashion?

> First, what does "fairness" mean in this context?  Given that there
> are multiple simultaneous asynchronous subprocesses that produce
> output at different rates and consume CPU at different levels, what
> would it mean for Emacs to be "fair"?
>
> Second, suppose we have multiple ripgrep subprocesses running, and
> Emacs will somehow read from each one of them in a round-robin
> fashion: what and how do you expect the user to do to handle the
> results of all those subprocesses simultaneously and "fairly"?

I agree with you that fairness is a difficult problem. But the problem
is omnipresent at the os level. There you have scheduling problems in
the io layer, in the process scheduling layer, in the memory management
layer and so on. There is certainly some heuristic that one can apply.
For example by comparing the amount of data produced by multiple
processes one could decide which process is read next. Or one can use a
deadline criterion.

I am not happy with the argument that Emacs cannot do any better than
stopping the second process and only handle the first process.

If you don't want to hardcode the scheduling behavior there could be
some pluggable scheduler. This would be better than having to write my
own scheduling by hand for each `make-process` call.