Re: emacs-30 6bca138d60e: Fix latest test for dabbrev-expand

[Andrea Corallo <acorallo@gnu.org>]

Stephen Berman <stephen.berman@gmx.net> writes:

> On Mon, 02 Dec 2024 15:45:23 -0500 Andrea Corallo <acorallo@gnu.org> wrote:
>
>> Stephen Berman via Mailing list for Emacs changes <emacs-diffs@gnu.org>
>> writes:
>>
>>> branch: emacs-30
>>> commit 6bca138d60e47d0fbab4faf8e83c5b9b319f260d
>>> Author: Stephen Berman <stephen.berman@gmx.net>
>>> Commit: Stephen Berman <stephen.berman@gmx.net>
>>>
>>>     Fix latest test for dabbrev-expand
>>>
>>>     * test/lisp/dabbrev-tests.el (dabbrev-expand-after-killing-buffer):
>>>     Fix typo in a 'should' test, use part of return value of 'should-error'
>>>     test, and remove mistaken and unfounded FIXME comment.
>>
>> Hello Stephen,
>>
>> this commit is introducing in my test-bed the following three failures
>> in emacs-30: dabbrev-expand-test-other-buffer-4,
>> dabbrev-expand-test-other-buffer-3, dabbrev-expand-after-killing-buffer.
>> Could you please have a look?
>>
>>   Andrea
>
> How does your testbed run the tests?  When I invoke `emacs-30 -batch -l
> ~/src/emacs/emacs-30/test/lisp/dabbrev-tests.el -f
> ert-run-tests-batch-and-exit' from the shell, all tests pass.  Likewise,
> when I start emacs-30 -Q, then `M-x load-file
> ~/src/emacs/emacs-30/test/lisp/dabbrev-tests.el', then kill all
> non-hidden buffers except *scratch*, and then run `M-x ert', all tests
> pass.  Since the batch run is also used by `make check', I've assumed
> that, since the tests succeed (at least for me) in this mode, they are
> valid regression tests.
>
> In contrast, when I start emacs -Q, visit
> ~/src/emacs/emacs-30/test/lisp/dabbrev-tests.el, evaluate the buffer and
> run `M-x ert', then I get five test failures:
> dabbrev-expand-test-other-buffer-{1,2,3,4} and
> dabbrev-expand-after-killing-buffer.  The latter fails because
> dabbrev-expand finds an expansion in the test file buffer (but succeeds
> when there is no such buffer, as in the previous run starting with -Q).
> The four other tests fail possibly for the same reason; however, these
> four tests fail in this way since being introduced in commit 8113b5c562
> (see
> https://lists.gnu.org/archive/html/bug-gnu-emacs/2024-10/msg01752.html),
> so it puzzles me that you only see just two of those failures and only
> after my latest commit.  (However, I did get the same three failures you
> reported when I ran ert on the test file from commit bd8a6f70fb,
> i.e. the one prior to the above commit, but only after uncommenting the
> code lines that are commented out in that commit.  Could it be that you
> accidentally tested using that previous commit with those code lines
> uncommented?)
>
> Steve Berman

Hi Steve,

my testbench is running "make check -j24" on a fresh checkout on
x86_64-pc-linux-gnu for few different configurations.  For instance I
can see the failures on the latest emacs-30
e618d5ae941d005e9da6d5d5cba68cc0ecade0e5 configured with
"--with-native-compilation=no --with-x-toolkit=no".

Test dabbrev-expand-test-other-buffer-3 backtrace:
  signal(ert-test-failed (((should (string= (buffer-string) "Indic and
  ert-fail(((should (string= (buffer-string) "Indic and")) :form (stri
  #f(compiled-function () #<bytecode 0x15984b7a4b7dc916>)()
  #f(compiled-function () #<bytecode 0x1ea04fa01345ba2a>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x1ea04fa01345ba2a
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name dabbrev-expand-test-other-buffer-3 :d
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l
  command-line()
  normal-top-level()
Test dabbrev-expand-test-other-buffer-3 condition:
    (ert-test-failed
     ((should (string= (buffer-string) "Indic and")) :form
      (string= "Indic\")) (execute-kbd-macro" "Indic and") :value nil
      :explanation
      (arrays-of-different-length 27 9 "Indic\")) (execute-kbd-macro"
                                  "Indic and" first-mismatch-at 5)))
   FAILED  11/16  dabbrev-expand-test-other-buffer-3 (0.042668 sec) at lisp/dabbrev-tests.el:175


Test dabbrev-expand-test-other-buffer-4 backtrace:
  user-error("No%s dynamic expansion for `%s' found" "" "execute-kbd-m
  dabbrev-expand(nil)
  funcall-interactively(dabbrev-expand nil)
  command-execute(dabbrev-expand)
  execute-kbd-macro([73 110 100 134217775])
  #f(compiled-function () #<bytecode -0x15ea54be0ab851ac>)()
  #f(compiled-function () #<bytecode 0x1ea04fa01345ba2a>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x1ea04fa01345ba2a
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name dabbrev-expand-test-other-buffer-4 :d
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l
  command-line()
  normal-top-level()
Test dabbrev-expand-test-other-buffer-4 condition:
    (user-error "No dynamic expansion for `execute-kbd-macroInd' found")
   FAILED  12/16  dabbrev-expand-test-other-buffer-4 (0.021285 sec) at lisp/dabbrev-tests.el:194


Test dabbrev-expand-after-killing-buffer backtrace:
  signal(ert-test-failed (((should (string= msg "No further dynamic ex
  ert-fail(((should (string= msg "No further dynamic expansion for ‘ab
  #f(compiled-function () #<bytecode 0x18257815d6a0b741>)()
  #f(compiled-function () #<bytecode 0x1ea04fa01345ba2a>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x1ea04fa01345ba2a
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name dabbrev-expand-after-killing-buffer :
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l
  command-line()
  normal-top-level()
Test dabbrev-expand-after-killing-buffer condition:
    (ert-test-failed
     ((should (string= msg "No further dynamic expansion for ‘ab’ found"))
      :form
      (string= "No further dynamic expansion for `ab' found"
               "No further dynamic expansion for ‘ab’ found")
      :value nil :explanation
      (array-elt 33
                 (different-atoms (96 "#x60" "?`") (8216 "#x2018" "?‘")))))
   FAILED   3/16  dabbrev-expand-after-killing-buffer (0.001730 sec) at lisp/dabbrev-tests.el:278

Thanks

  Andrea