bug#48118: 27.1; 28; Only first process receives output with multiple running processes

[Daniel Mendler <mail@daniel-mendler.de>]

On 4/30/21 5:58 PM, Eli Zaretskii wrote:
> A call to accept-process-output prioritizes a process only if it
> explicitly requests output from that single process.  Which is not
> necessarily true in all cases.

Yes, I have seen that in the documentation.

>>> What does this mean, exactly?  Which quantity should be doled in a
>>> round-robin fashion? bytes read from the processes? something else?
>>>
>>> If the bytes read, then how do you suggest to handle two processes
>>> which produce output at very different rates?
>>
>> For example bytes read or time spent to handle a process (time spent in
>> the filter function?).
> 
> Bytes read has a problem when processes produce output a very
> different rates.  Time spent to handle may (and usually does) mean the
> filter function does something expensive, it doesn't necessarily tell
> anything about the output from the subprocess.

Of course it is not possible to find a perfect scheduling algorithm. But
how does the OS handle it if you have multiple processes which produce
output with vastly different rates? I am not claiming this problem has
been solved, but there are certainly some heuristics. Emacs is also
dependent on the OS scheduling, depending on how Emacs schedules its
reads/writes from the processes, the OS scheduler adjusts accordingly.
This furthermore complicates the picture.

>> When I stumbled over this issue, it astonished me that Emacs
>> does not seem to do any scheduling at all and handles only a single
>> process.
> 
> If you read the code, you will see this isn't what happens.  What
> happens is that Emacs reads a chunk of output from the first process
> it sees ready, then it goes back and re-checks which processes are
> ready -- and in your scenario I think it again sees that the first
> process is ready.

This is what we assumed. Emacs could check the second process the next
time. This way one may get a slightly more fair behavior. It would
certainly not be perfect and you could throw scenarios at it which would
make it behave unexpectedly. It may behave a bit more expectedly in the
common case?

> I suggest to read the code of wait_reading_process_output, it has some
> non-trivial logic in this department.

I will do that. Has this problem discussed before?