bug#75359: [scratch/elisp-benchmarks] test dependencies

[Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

>> There are dependencies between emacs-lisp-benchmarks tests, meaning we
>> can't always modify one test without affecting others.  The ones I'm
>> aware of are:
>>
>> - elb-bytecomp.el uses elb-smie.el as compilation input (in this
>>   context, elb-smie.el is a "resource", not source code).  If we
>>   modify elb-smie.el, elb-bytecomp.el results will change and become
>>   uncomparable.
>
> Shouldn't we just copy of elb-smie.el to resources?

If so, we could just as well use some other file.  I used `elb-smie.el`
because it was a large enough file and it was already part of
`elisp-benchmarks`.

>> - elb-scroll.el uses elb-smie.el as a prerequisite.  If we modify
>>   elb-smie.el, elb-scroll.el results will change and become
>>   uncomparable.  Furthermore, elb-scroll.el relies on elb-smie.el
>>   being compiled in the appropriate fashion by some external
>>   mechanism.  (I think the best solution here is to merge elb-smie.el
>>   and elb-scroll.el into a single file declaring several tests, using
>>   the established ERT test framework)
>
> I'm not sure these are real/serious bugs, different versions of
> elisp-benchmarks don't guarantee to give comparable results (like every
> benchmark).

Agreed.
This said, merging `elb-smie.el` and `elb-scroll.el` would be fine.
Can't remember why I didn't do that from the start.

[ Not sure what ERT has to do with it, BTW.  ]

>> - bubble.el and bubble-no-cons.el both "provide" the same feature.
>>   I'm not sure whether this causes any acute problems, but it
>>   demonstrates that the inconsistent use of (provide) in the
>>   benchmarks is problematic.  If we ever load tests using Frequire, we
>>   will load the wrong files.  If we don't, provide shouldn't be
>>   necessary.
>
> AFAIU the provide is not used by elisp-benchmarks, anyway the typo is
> now fixed.

FWIW, these two files date back to the time where the benchmarks were
defined by their file (i.e. one file per benchmark), whereas nowadays
the benchmarks are defined by the `elb-*-entry` functions, so we should
probably merge those two files into one.

>> - xmenu.c is used as a resource; this is fine.  But we must make sure
>>   we never copy the "current" xmenu.c to the benchmarks directory
>>   without clearly indicating that the benchmark results will become
>>   uncomparable.
> Again, changes to the benchmark pertubating results should be tagged as
> a new version, so will be fine to update xmenu.c if necessary as long as
> we update elisp-benchmarks version.

Also I can't think of any reason why we'd want to "update" that file.
At least not until its use of the C language is considered too
antiquated/deprecated.


        Stefan