bug#71355: 30.0.50; [PATCH] Improve performance of buffered output in Eshell

[Jim Porter <jporterbugs@gmail.com>]

On 6/5/2024 9:51 AM, Eli Zaretskii wrote:
>> Date: Wed, 5 Jun 2024 09:42:43 -0700
>> Cc: 71355@debbugs.gnu.org, stefankangas@gmail.com
>> From: Jim Porter <jporterbugs@gmail.com>
>>
>> On 6/5/2024 5:06 AM, Eli Zaretskii wrote:
>>> Is 2K indeed the optimal size?  It is about 25 80-column lines, which
>>> is quite a lot.  "Normal" shells send output to the screen in smaller
>>> chunks.  How about 128 instead? or maybe some small multiple of the
>>> line length, like 1 or 2?
>>
>> Yes, I believe 2k is the optimal size, or close to it. Trying a value of
>> 128 results in basically no change in performance from baseline. That
>> makes sense to me, since 128 is actually fairly close to the old value
>> for this buffering (which was five *lines*[1]; the old code measured
>> this differently).
> 
> That's strange, because I see no output at all until all of it is
> available and showsn, and I thought you said the same in your OP?

Yes, without my patch that's expected. When I talk about changes in 
performance, I mean the total time to complete the command, as measured 
by, e.g. "time cat config.log".

Here's what's happening: all of the output in 'eshell/cat' occurs in a 
loop, periodically calling 'eshell-interactive-print' (how often it 
calls this depends on the buffering settings). That runs the functions 
in 'eshell-output-filter-functions', which can be fairly expensive. So 
one way to make output faster would be to optimize those functions, 
which I did in my second patch. However, a larger buffer size is still 
faster even when there are no output filter functions, due to other 
overheads in the code. So I think even if we could make 
'eshell-output-filter-functions' all very cheap, it's worth increasing 
the buffer size.

In addition to this, the performance improvements I made allowed me to 
add in the extra work of redisplaying periodically when using this 
buffered output scheme. That's all new in my patch, and previously you'd 
have to wait until the command was finished to see any output. From 
Emacs's perspective, everything in 'eshell/cat' is synchronous, so I 
needed to manually trigger the redisplay (or do some other sorcery to 
hand control back to the command loop).