bug#72004: 30.0.50, master: 'erc--check-prompt-input-for-multiline-blanks' test fail

["J.P." <jp@neverwas.me>]

Hi Andrea,

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <acorallo@gnu.org>
>> Date: Tue, 09 Jul 2024 04:08:06 -0400
>> 
>> Since few days I see 'erc--check-prompt-input-for-multiline-blanks'
>> failing.  I think the fail is intermittent and because of that I could
>> not determine the commit that introduced it.
>> 
>> I observe this both on emacs-30 both on master, the first commit in
>> emacs-30 where I observed it is 2fb6a98ecfa1579273a640e923f2e52f75e1f7ad
>> which seems unrelated (but I mention it so we have a point in time).

I've not yet witnessed the test in question fail, but I can definitely
imagine it doing so because it's rather flimsy, which is my bad. I've
therefore attempted a superficial fix on the release branch:

  ef3f26ec02d ; Tag ERC multiline blanks test as :expensive

Here's what (I think) is going on. That test relies on the macro
`ert-with-message-capture'. Because that macro advises a few primitive
functions, its first appearance in any make-check run exhibits a
trampoline penalty in terms of execution time (as I'm sure you, more
than anyone, are acutely aware). For example, if you put this at the
bottom of test/lisp/emacs-lisp/ert-x-tests.el

  (ert-deftest ert-with-message-capture/1 ()
    (ert-with-message-capture string (ignore string)))
  
  (ert-deftest ert-with-message-capture/2 ()
    (ert-with-message-capture string (ignore string)))

you'll notice the first takes upwards of a few seconds (when built with
debugging symbols), while subsequent occurrences are comparatively free:

  passed  21/30  ert-with-message-capture/1 (3.067526 sec)
  passed  22/30  ert-with-message-capture/2 (0.000396 sec)

I think it's possible that this phenomenon, combined with added CPU
pressure from unbounded "make -j" runs, may account for the intermittent
failures you've been seeing. And correct me if I'm wrong, but I'm
guessing that during parallel make-check runs, the first occurrence of
`ert-with-message-capture' in any one file always incurs such a penalty
because `ert-run-tests-batch-and-exit' deletes its .eln cache on exit
and "make -j" runs every test file in a separate process.

(Incidentally, I count eleven files in the suite currently using this
macro. Granted, none likely depends on a brittle hard timeout, like the
"sleep 10" in my test, so there's surely no risk of similar failures,
but if the mere appearance of that macro or ones like it translates to
noticeable overhead, perhaps it's worth looking into eventually.)

Anyway, thanks for bringing this to ERC's attention.

J.P.