bug#48118: 27.1; 28; Only first process receives output with multiple running processes

[Daniel Mendler <mail@daniel-mendler.de>]

On 4/30/21 4:59 PM, Eli Zaretskii wrote:
> That's not what I meant.  I meant that if your Lisp program launches a
> subprocess that is known to spill huge amounts of output at high rate,
> and you don't want to starve other subprocesses, your filter function
> can stop the process from time to time to give others an opportunity
> to have their outputs read.

This is true. However I expect Emacs to do that for me. I see the
situation like this - I have multiple packages creating processes and
competing for runtime. Then I expect Emacs to take some kind of
preemptive role and ensure that none of the processes misbehaves.
However one can also take the standpoint that each process should try to
behave well by itself and should be scheduled explicitly; Emacs should
stay out of this business.

>>>> since the underlying Emacs handling of asynchronous processes is
>>>> unable to read from two processes at once?
>>>
>>> No.  The problem is not the _ability_ to read from more than one
>>> subprocess -- the ability does exist.  The problem is that doing so
>>> would run afoul of other scenarios.
>>
>> Which scenarios break?
> 
> For example, if the filter function call accept-process-output.  Or
> does anything else that changes output from which processes is or
> isn't available.

Does this necessarily prevent scheduling? I interpret
`accept-process-output` as a function which prioritizes a process, but I
am unsure if this makes it impossible to implement additional scheduling.

> What does this mean, exactly?  Which quantity should be doled in a
> round-robin fashion? bytes read from the processes? something else?
> 
> If the bytes read, then how do you suggest to handle two processes
> which produce output at very different rates?

For example bytes read or time spent to handle a process (time spent in
the filter function?). If a process has eaten up its time it has to wait
until it gets scheduled next. If you have two process with very
different rates, the slow process may not use up its allotted time slot
and the faster process is still allowed to run.

>> I am not happy with the argument that Emacs cannot do any better than
>> stopping the second process and only handle the first process.
> 
> I'm not saying that Emacs cannot do that, I'm trying to understand
> what that would mean in practice.

Actually I would also like to understand what the best process handling
looks like. When I stumbled over this issue, it astonished me that Emacs
does not seem to do any scheduling at all and handles only a single
process. As far as I know other language runtimes handle this problem
differently, attempting some kind of scheduling.

What is the reason for the current behavior? Is it predictability? If I
understand correctly, Emacs always reads from the first process. If data
arrives, Emacs does not read from the second processes at all. Only if
no data is available from the first process, the second process is
handled. Is it like this?