bug#32728: bug#32729: Xemacs 23 times as fast as GNU Emacs

[Eli Zaretskii <eliz@gnu.org>]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: benjamin.benninghofen@airbus.com,  layer@franz.com,
>   32729@debbugs.gnu.org,  32728@debbugs.gnu.org
> Date: Sat, 12 Oct 2019 19:55:54 +0200
> 
> >> (Note: Don't visit the " *zeroes*" buffer after this, because that will
> >> hang Emacs totally.  I guess the long-line display problem hasn't been
> >> fixed after all?)
> >
> > It's unrelated.  Did you try to insert that into a unibyte buffer
> > instead?
> 
> Nope.  Does Emacs need to do a lot of recomputing when going to
> multibyte buffers?

Of course: we try to display the multibyte text as characters, search
for fonts, invoke bidi reordering, etc.

> >> And it's a real world problem: When reading data from any network
> >> source, you have to use filters because the protocol is usually based on
> >> parsing the output to find out when it's over, so you can't use
> >> sentinels.
> >
> > Why can't you use the default filter, and in the sentinel work on the
> > buffer with the complete response?
> 
> No sentinel is called, because the process status doesn't change,
> typically.  All the relevant network protocols do not close after having
> "done something" (IMAP, HTTP, etc), but instead use in-protocol markers
> to say that an operation is done.  So a filter has to be used.

OK, so not in sentinel, but in a timer or somesuch.

> > Please also note that, GC aside, inserting stuff of size that is
> > unknown in advance is not free, either: we need to enlarge the buffer
> > text each time more stuff arrives.
> 
> Yes, I was wondering about that -- how slow is resizing buffers, really?
> Does Emacs rely on OS-level realloc that doesn't necessitate actually
> copying all that data all the time?

Not necessarily realloc, sometimes mmap or similar.  And yes, this
requires copying the data when the OS cannot grow the previous
allocation, but instead gives us a memory block at another address.
In most systems this copying is under the hood, but it does happen.

> Also -- some of the networking operations know in advance how much data
> is going to be received.  For instance, when using non-chunked HTTP
> (quite common when fetching "data", i.e., images, PDFs etc), we get the
> content size in a header.  Would it make sense to have a way to tell
> Emacs about this in advance so that it could pre-size the buffer?

We could provide a Lisp interface for such cases.  The C
implementation is in enlarge_buffer_text (but we need to remember the
decoding of human-readable text, when applicable).

However, I would begin by measuring the effect of this resizing on the
time it takes to receive large amounts of data.  Maybe other factors
make this part negligible.