Re: master 2e090da: Revert commit 1f44a776729adf9c6468a76f8310616fde62eeaa for Flymake.

[haj@posteo.de (Harald Jörg)]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>     Revert commit 1f44a776729adf9c6468a76f8310616fde62eeaa for Flymake.
>>     Flymake supports all Emacs versions back to 26.1, so it can’t use
>>     ‘ert-resource-file’.
>>     * test/lisp/progmodes/flymake-tests.el (flymake-tests-data-directory):
>>     Recreate.
>>     (flymake-tests--call-with-fixture): Stop using ‘ert-resource-file’.
>
> It's common for a package to say "require Emacs version NN.MM" but to
> provide extra features on more recent Emacsen.  And the tests can
> definitely fall into the category of extra features.

I have also been bitten by this and am pondering what's the best way out
of that dilemma.  I'm working on cperl-mode and am trying to keep it
compatible with Emacs 26.1 (the version which comes with Debian stable).
cperl-mode isn't even on ELPA yet, but might end up there.

I *really* like 'ert-resource-file' because it reduces a lot of
boilerplate from the tests.  Therefore I don't consider reverting that
commit.  However, for the moment, I lost the chance to run the tests in
the environment which comes with Emacs "as shipped by the
distributions".

In my opinion, running the tests in older versions _is_ important,
otherwise the claim "supports Emacs 26.1" isn't honest.  I want to
support older Emacsen so that I can get feedback from users who don't
want to compile their own Emacs (which includes myself on production
systems), and I also want to add new features for which new tests should
be written.

So, what I've come up with is a file ert-backport.el, with the
description "Run ERT tests from Emacs master under older Emacsen".

To activate this file, I add its directory to the environment variable
EMACSLOADPATH - and voila, tests from Emacs' master branch pass with
Debian buster's Emacs 26, using the ERT provided by this Debian package.

Whenever the tests are upgraded to use new ERT features (which I
consider to be rare events anyway), ert-backport.el needs to be updated.

Could such an ert-backport.el be a candidate for ELPA?  It should be
useful for all dual-life (Emacs + ELPA) packages.
--
Cheers,
haj