bug#71355: 30.0.50; [PATCH] Improve performance of buffered output in Eshell

[Jim Porter <jporterbugs@gmail.com>]

On 6/6/2024 4:14 PM, Stefan Kangas wrote:
> FWIW, I've experimented a bit on my machine, and I see the following
> using the command "time cat config.log" from an initially empty eshell
> buffer:
> 
> | eshell-buffered-print-size | secs       |       |
> |----------------------------+------------+-------|
> |                        256 | 1.922 secs |     1 |
> |                        512 | 1.413 secs |  0.74 |
> |                       1024 | 1.065 secs |  0.55 |
> |                       2048 | 0.996 secs |  0.52 |
> |                       4096 | 0.860 secs |  0.45 |
> |                       8192 | 0.835 secs |  0.43 |
> |                      16384 | 0.829 secs |  0.43 |
> 
> To me, these numbers seem to suggest that, at least on this system,
> there is a sweet spot around 4096, but 2048 admittedly does already get
> us most of the way there.  However, going above 8192 doesn't lead to any
> appreciable speedup.  This is on a fast M2 machine; it would be
> interesting to see some experiments on slower machines as well.
> 
> I'm assuming that we don't want to set it to some arbitrarily large
> number, but do we expect any adverse affects from choosing a slightly
> higher value?  If not, is there a case to be made for choosing 4096 as
> the default?

This is consistent with what I saw, but I thought it best to err on the 
side of smallness for the buffer size, just to ensure that we flush the 
buffer faster than the redisplay rate (defaulting to 40Hz), even on 
slower machines. Since the difference between 2048 and 4096 is small, I 
figured it wasn't too high a price to pay.

In your tests, just like mine, the difference between 1024 and 2048 is 
also small, but I opted for the larger 2048 because it produced a 
more-noticeable improvement when running `time ls -Al /usr/bin`.

That said, if we wanted to use a buffer size of 4096, I don't think that 
would cause any major issues. (At least on my system, when running an 
external command in Eshell, it writes 4095 bytes at a time, since that's 
how much data we get at once from the process filter. As far as I'm 
aware, it's been that way forever.)

On some level, I think any value between about 1024 and 4096 is probably 
somewhat arbitrary. They'd likely all be fine in most cases, and have 
much better performance than what we have today.