Mattias Engdegård <mattias.engdegard@gmail.com> writes:

> Here's a new patch.
> A lot less pretty this time.
>
> In any case, make sure to include unit tests in your final patch.

Ok, here it comes.

I have constructed two org tables for the test (see test-input.org) that
are reasonably long and contain a few not too unreasonable formulas.
The nature of the formulas is not too important for the tests, I just
took them from my original "offending" table.

I also added unit tests for the function, as requested.

One of the tables has no special characters, so
‘math-read-preprocess-string’ just has to walk through without doing any
real work (arguably the most common case).  The other table is full of
replaceable stuff.  I used these two tables to compare the original with
the new version of ‘math-read-preprocess-string’.  The results are in
test-results.org.

As before, I compared ‘gcs-done’ and ‘gc-elapsed’ (with standard
settings for ‘gc-cons-threshold’ and ‘gc-cons-percentage’) with both
versions and tables.  I also instrumented ‘org-table-recalculate’ and
‘math-read-preprocess-string’ using elp.  Then I set ‘gc-cons-threshold’
to a large value and ran the same tests again.  For the latter test, I
only report ‘elp-results’ (as no GC was triggered).

I recalculated every table three times per reported elp result, as it
takes three evaluations for the calculation to converge.

Some interesting results are:

- time spent in GC reduces from 2.11 s to  0.78 s for the second table
  (with replacements)
- elp report of average time decreases from 4.6316224e-3 s to
  3.0294569e-4 s for the second table
- with GC prevented, elp reports avg. time 3.4286452e-4 s vs. 4.9444e-5 s
- elp report of elapsed time for ‘org-table-recalculate’ for three evaluations
  (what the user sees) decreases from 8.36 s to 4.11 s (with standard GC
  settings again)

This is a real win especially for large tables.  Needless to say, I am
very excited about this optimization.

(Just for completeness, I quickly compared my version to Mattias'
version, the latter is about a factor of two faster according to elp.)

Regards,
Raffael