bug#46978: 27.1; Shell mode performs extremely poorly in Linux-libre 5.10.20

[Eli Zaretskii <eliz@gnu.org>]

> From: Mark H Weaver <mhw@netris.org>
> Cc: 46978@debbugs.gnu.org
> Date: Sun, 07 Mar 2021 03:50:15 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Mark H Weaver <mhw@netris.org>
> >> Date: Sat, 06 Mar 2021 21:39:54 -0500
> >> 
> >> Mark H Weaver <mhw@netris.org> writes:
> >> 
> >> > I've skimmed the list of changes between Linux 5.10.19 and 5.10.20, and
> >> > found these two commits that might be relevant:
> >> >
> >> > tty: implement read_iter
> >> > <https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.10.20&id=41c6f6b926d0e712d0321f8a8f6511fea748e814>
> >> >
> >> > tty: convert tty_ldisc_ops 'read()' function to take a kernel pointer
> >> > <https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.10.20&id=279e54536ddbb4dbd337fca74926b68651160043>
> >> >
> >> > These changes are also included in Linux-libre 5.11.3 (the newest
> >> > release), but I haven't yet tried that version.
> >> >
> >> > I'm currently building a variant of Linux-libre 5.10.20 with these two
> >> > commits reverted, to see if that makes the problem go away, and I will
> >> > report back when I have those results.
> >> 
> >> Reverting the two commits cited above fixes the problem.
> >
> > So does this mean this isn't an Emacs issue, but the issue with that
> > kernel?
> 
> That's not clear to me.  What I know is that these changes to Linux's
> TTY subsystem, authored by Linus Torvalds himself and recently included
> in upstream Linux 5.11.3 and 5.10.20, have lead to this regression in
> Emacs.  This might simply be a kernel bug, or it could be that Emacs is
> making an improper assumption about how the kernel behaves.

Emacs doesn't assume anything about the kernel, it simply reads from
the PTY which serves as stdout for the subprocess.  Your description
seems to indicate that the reads stall, because that's the only
explanation I can come up with to the fact that we display 'cat's
output one character at a time.

Do these changes affect the way characters are written to the PTY?
For example, do they affect the frequency with which stuff is written,
or the batching of the bytes (i.e. how many bytes are written in one
go)?  Or maybe they affect the read side of the PTY?

Does it help to set process-connection-type to nil?

Does it help to play with process-adaptive-read-buffering?

> I would suggest investigating to find out what's going wrong here, and
> then:

If someone here knows enough about this part of the Linux kernel to
understand its effect on Emacs communications with subprocess, I hope
they speak up soon.  I looked at the two commits you mentioned and
didn't see anything that immediately caught my eye, but then I have no
idea how TTYs work on Linux at that level.

Failing that, my suggestion is to inform the kernel developers of the
problem and see if they have any suggestions for where in Emacs to
look for the cause, if at all.