bug#71355: 30.0.50; [PATCH] Improve performance of buffered output in Eshell

[Eli Zaretskii <eliz@gnu.org>]

> Date: Wed, 5 Jun 2024 10:35:08 -0700
> Cc: 71355@debbugs.gnu.org, stefankangas@gmail.com
> From: Jim Porter <jporterbugs@gmail.com>
> 
> On 6/5/2024 9:51 AM, Eli Zaretskii wrote:
> >> Date: Wed, 5 Jun 2024 09:42:43 -0700
> >> Cc: 71355@debbugs.gnu.org, stefankangas@gmail.com
> >> From: Jim Porter <jporterbugs@gmail.com>
> >>
> >> On 6/5/2024 5:06 AM, Eli Zaretskii wrote:
> >>> Is 2K indeed the optimal size?  It is about 25 80-column lines, which
> >>> is quite a lot.  "Normal" shells send output to the screen in smaller
> >>> chunks.  How about 128 instead? or maybe some small multiple of the
> >>> line length, like 1 or 2?
> >>
> >> Yes, I believe 2k is the optimal size, or close to it. Trying a value of
> >> 128 results in basically no change in performance from baseline. That
> >> makes sense to me, since 128 is actually fairly close to the old value
> >> for this buffering (which was five *lines*[1]; the old code measured
> >> this differently).
> > 
> > That's strange, because I see no output at all until all of it is
> > available and showsn, and I thought you said the same in your OP?
> 
> Yes, without my patch that's expected. When I talk about changes in 
> performance, I mean the total time to complete the command, as measured 
> by, e.g. "time cat config.log".
> 
> Here's what's happening: all of the output in 'eshell/cat' occurs in a 
> loop, periodically calling 'eshell-interactive-print' (how often it 
> calls this depends on the buffering settings). That runs the functions 
> in 'eshell-output-filter-functions', which can be fairly expensive. So 
> one way to make output faster would be to optimize those functions, 
> which I did in my second patch. However, a larger buffer size is still 
> faster even when there are no output filter functions, due to other 
> overheads in the code. So I think even if we could make 
> 'eshell-output-filter-functions' all very cheap, it's worth increasing 
> the buffer size.
> 
> In addition to this, the performance improvements I made allowed me to 
> add in the extra work of redisplaying periodically when using this 
> buffered output scheme. That's all new in my patch, and previously you'd 
> have to wait until the command was finished to see any output. From 
> Emacs's perspective, everything in 'eshell/cat' is synchronous, so I 
> needed to manually trigger the redisplay (or do some other sorcery to 
> hand control back to the command loop).

I think we are miscommunicating.  I wasn't talking about performance,
I was talking about the fact that I don't see text delivered to the
screen in chunks.  You said that the current code sends text to the
screen in chunks of 5 lines, and that therefore using the value 128 is
almost the same.  But at least part of your patch calls redisplay
after each chunk (AFAIU), something that is not done with the current
code.  So I expect the effect to be a difference in behavior, whereby
test appears on the screen in chunks, and the user does not need to
wait till all of it is sent before he/she sees anything at all
displayed.

I hope I've succeeded to explain myself now.