On Mon, Feb 27, 2023 at 10:44:43PM +0200, Petteri Hintsanen wrote:

[...]

> Also, if I interpreted profiler's hieroglyphs correctly, it told me that
> this setq
> 
>   (setq stream (vconcat stream (plist-get page :stream)))
> 
> is a pig -- well, of course it is.  I'm accumulating byte vector by
> copying its parts.  Similarly bindat consumes a lot of memory.
> 
> I think I can replace vectors with strings, which should, according to
> the elisp manual, "occupy one-fourth the space of a vector of the same
> elements."  And I guess that accumulation would be best done with a
> buffer, not with strings or vectors.

I must admit I didn't look too closely into your code, but this one
stuck out too. Not only the copying, but the throwing away of so many
vectors.

I don't know whether it applies in your case, but one "classical"
Lispy pattern when you have to concatenate many things in order is
just consing them (at the front of the list) and nreversing the
list at the end, like so:

  (let ((result '()))
    (while (more)
      (setq result (cons (next) result)))
    (nreverse result))

(nreverse does things "in place", so it reuses the cons pairs:
never do that when someone else is looking ;-)

Another, more functional, of course is to arrange things so you
can use map or similar.

Then, at the end you can concatenate the whole list, if need be.

Basically it pays off when the "spine" of the whole thing (i.e.
all those cons pairs you are using) is significantly smaller than
whatever hangs off it

Cheers
-- 
t