bug#67536: 29.1; Calc mode's math-read-preprocess-string conses unnecessarily

[Eli Zaretskii <eliz@gnu.org>]

Mattias, is this okay with you?  Should I install these patches?

> From: Raffael Stocker <r.stocker@mnet-mail.de>
> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>,
>  67536@debbugs.gnu.org
> Date: Tue, 05 Dec 2023 19:14:36 +0100
> 
> Mattias Engdegård <mattias.engdegard@gmail.com> writes:
> 
> > Here's a new patch.
> > A lot less pretty this time.
> >
> > In any case, make sure to include unit tests in your final patch.
> 
> Ok, here it comes.
> 
> I have constructed two org tables for the test (see test-input.org) that
> are reasonably long and contain a few not too unreasonable formulas.
> The nature of the formulas is not too important for the tests, I just
> took them from my original "offending" table.
> 
> I also added unit tests for the function, as requested.
> 
> One of the tables has no special characters, so
> ‘math-read-preprocess-string’ just has to walk through without doing any
> real work (arguably the most common case).  The other table is full of
> replaceable stuff.  I used these two tables to compare the original with
> the new version of ‘math-read-preprocess-string’.  The results are in
> test-results.org.
> 
> As before, I compared ‘gcs-done’ and ‘gc-elapsed’ (with standard
> settings for ‘gc-cons-threshold’ and ‘gc-cons-percentage’) with both
> versions and tables.  I also instrumented ‘org-table-recalculate’ and
> ‘math-read-preprocess-string’ using elp.  Then I set ‘gc-cons-threshold’
> to a large value and ran the same tests again.  For the latter test, I
> only report ‘elp-results’ (as no GC was triggered).
> 
> I recalculated every table three times per reported elp result, as it
> takes three evaluations for the calculation to converge.
> 
> Some interesting results are:
> 
> - time spent in GC reduces from 2.11 s to  0.78 s for the second table
>   (with replacements)
> - elp report of average time decreases from 4.6316224e-3 s to
>   3.0294569e-4 s for the second table
> - with GC prevented, elp reports avg. time 3.4286452e-4 s vs. 4.9444e-5 s
> - elp report of elapsed time for ‘org-table-recalculate’ for three evaluations
>   (what the user sees) decreases from 8.36 s to 4.11 s (with standard GC
>   settings again)
> 
> This is a real win especially for large tables.  Needless to say, I am
> very excited about this optimization.
> 
> (Just for completeness, I quickly compared my version to Mattias'
> version, the latter is about a factor of two faster according to elp.)
> 
> Regards,
> Raffael