* test failures
@ 2020-08-21 20:23 Stephen Berman
2020-08-21 20:36 ` Lars Ingebrigtsen
2020-08-22 6:31 ` Eli Zaretskii
0 siblings, 2 replies; 9+ messages in thread
From: Stephen Berman @ 2020-08-21 20:23 UTC (permalink / raw)
To: emacs-devel
I just built current master (at commit 348686b3f3) and then ran make
check (for the first time in a long time), and this is the result:
SUMMARY OF TEST RESULTS
-----------------------
Files examined: 298
Ran 4216 tests, 4148 results as expected, 20 unexpected, 48 skipped
3 files contained unexpected results:
lisp/progmodes/elisp-mode-tests.log
lisp/help-fns-tests.log
lisp/emacs-lisp/cl-generic-tests.log
Here are the tests that unexpectedly failed:
Ran 45 tests, 28 results as expected, 14 unexpected, 3 skipped (2020-08-21 21:40:33+0200, 2.127913 sec)
14 unexpected results:
FAILED xref-elisp-test-find-defs-defgeneric-co-located-default
FAILED xref-elisp-test-find-defs-defgeneric-el
FAILED xref-elisp-test-find-defs-defgeneric-implicit-generic
FAILED xref-elisp-test-find-defs-defgeneric-no-default
FAILED xref-elisp-test-find-defs-defgeneric-no-methods
FAILED xref-elisp-test-find-defs-defgeneric-separate-default
FAILED xref-elisp-test-find-defs-define-overload-co-located-default
FAILED xref-elisp-test-find-defs-define-overload-no-default
FAILED xref-elisp-test-find-defs-define-overload-no-methods
FAILED xref-elisp-test-find-defs-define-overload-separate-default
FAILED xref-elisp-test-find-defs-defun-defvar-el
FAILED xref-elisp-test-find-defs-defun-el
FAILED xref-elisp-test-find-defs-defvar-el
FAILED xref-elisp-test-find-defs-feature-el
Ran 19 tests, 14 results as expected, 5 unexpected (2020-08-21 21:39:58+0200, 0.689917 sec)
5 unexpected results:
FAILED help-fns-test-alias-to-defun
FAILED help-fns-test-bug23887
FAILED help-fns-test-lisp-defsubst
FAILED help-fns-test-lisp-defun
FAILED help-fns-test-lisp-macro
Ran 17 tests, 16 results as expected, 1 unexpected (2020-08-21 21:39:05+0200, 0.073067 sec)
1 unexpected results:
FAILED cl-generic-tests--method-files--finds-methods
Are these known failures, or did something go wrong with my build, or
would perhaps make bootstrap eliminate the failures?
Steve Berman
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: test failures
2020-08-21 20:23 test failures Stephen Berman
@ 2020-08-21 20:36 ` Lars Ingebrigtsen
2020-08-21 20:45 ` Stephen Berman
2020-08-22 11:32 ` Stephen Berman
2020-08-22 6:31 ` Eli Zaretskii
1 sibling, 2 replies; 9+ messages in thread
From: Lars Ingebrigtsen @ 2020-08-21 20:36 UTC (permalink / raw)
To: Stephen Berman; +Cc: emacs-devel
Stephen Berman <stephen.berman@gmx.net> writes:
> Are these known failures, or did something go wrong with my build, or
> would perhaps make bootstrap eliminate the failures?
I'm not getting any errors with the current trunk on Debian (with make
check).
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: test failures
2020-08-21 20:36 ` Lars Ingebrigtsen
@ 2020-08-21 20:45 ` Stephen Berman
2020-08-22 11:32 ` Stephen Berman
1 sibling, 0 replies; 9+ messages in thread
From: Stephen Berman @ 2020-08-21 20:45 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: emacs-devel
On Fri, 21 Aug 2020 22:36:40 +0200 Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> Are these known failures, or did something go wrong with my build, or
>> would perhaps make bootstrap eliminate the failures?
>
> I'm not getting any errors with the current trunk on Debian (with make
> check).
Hm, maybe I should try bootstrapping...
Steve Berman
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: test failures
2020-08-21 20:23 test failures Stephen Berman
2020-08-21 20:36 ` Lars Ingebrigtsen
@ 2020-08-22 6:31 ` Eli Zaretskii
1 sibling, 0 replies; 9+ messages in thread
From: Eli Zaretskii @ 2020-08-22 6:31 UTC (permalink / raw)
To: Stephen Berman; +Cc: emacs-devel
> From: Stephen Berman <stephen.berman@gmx.net>
> Date: Fri, 21 Aug 2020 22:23:05 +0200
>
> 3 files contained unexpected results:
> lisp/progmodes/elisp-mode-tests.log
> lisp/help-fns-tests.log
> lisp/emacs-lisp/cl-generic-tests.log
These all pass for me with the current master.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: test failures
2020-08-21 20:36 ` Lars Ingebrigtsen
2020-08-21 20:45 ` Stephen Berman
@ 2020-08-22 11:32 ` Stephen Berman
1 sibling, 0 replies; 9+ messages in thread
From: Stephen Berman @ 2020-08-22 11:32 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, emacs-devel
On Fri, 21 Aug 2020 22:36:40 +0200 Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> Are these known failures, or did something go wrong with my build, or
>> would perhaps make bootstrap eliminate the failures?
>
> I'm not getting any errors with the current trunk on Debian (with make
> check).
On Sat, 22 Aug 2020 09:31:29 +0300 Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Stephen Berman <stephen.berman@gmx.net>
>> Date: Fri, 21 Aug 2020 22:23:05 +0200
>>
>> 3 files contained unexpected results:
>> lisp/progmodes/elisp-mode-tests.log
>> lisp/help-fns-tests.log
>> lisp/emacs-lisp/cl-generic-tests.log
>
> These all pass for me with the current master.
It seems that the failures arise because my Emacs sources are in a
directory on a different partition from my home directory, and the
source directory is symlinked from §HOME, and my Emacs build directory
is under $HOME. When I do batch runs of the test three files in
question starting from my home directory (i.e. dereferencing the symlink
to the sources), I get the the failures I reported; but when I start the
batch runs from the real directory containing the test sources, the one
test in cl-generic-tests.el that failed now passes, and of the fourteen
xref tests in elisp-mode-tests.el that failed, now only five fail; yet
all five of the failing tests in help-fns-tests.el still fail.
However, concerning the latter, it seems that the tests expect
help-fns-function-description-header to return the quoted basename of
the file, while in my environment it returns the absolute filename.
Adjusting the regexp used in the tests fixes this issue (see the first
patch below) and then all tests pass regardless of where I start the
batch run.
Concerning cl-generic-tests.el, using file-truename as in the second
patch below prevents the one failure, also regardless of where I start
the batch run.
As for the failing xref tests in elisp-mode-tests.el, simply wrapping
the uses of expand-file-name in file-truename did not prevent the
errors. I don't know cl-generic well enough to debug further without
assistence.
Steve Berman
diff --git a/test/lisp/help-fns-tests.el b/test/lisp/help-fns-tests.el
index da2b49e6b8..7782a41b9f 100644
--- a/test/lisp/help-fns-tests.el
+++ b/test/lisp/help-fns-tests.el
@@ -56,28 +56,28 @@ help-fns-test-interactive-built-in
(should (string-match regexp result))))
(ert-deftest help-fns-test-lisp-macro ()
- (let ((regexp "a Lisp macro in .subr\\.el")
+ (let ((regexp "a Lisp macro in .+subr\\.el")
(result (help-fns-tests--describe-function 'when)))
(should (string-match regexp result))))
(ert-deftest help-fns-test-lisp-defun ()
- (let ((regexp "a compiled Lisp function in .subr\\.el")
+ (let ((regexp "a compiled Lisp function in .+subr\\.el")
(result (help-fns-tests--describe-function 'last)))
(should (string-match regexp result))))
(ert-deftest help-fns-test-lisp-defsubst ()
- (let ((regexp "a compiled Lisp function in .subr\\.el")
+ (let ((regexp "a compiled Lisp function in .+subr\\.el")
(result (help-fns-tests--describe-function 'posn-window)))
(should (string-match regexp result))))
(ert-deftest help-fns-test-alias-to-defun ()
- (let ((regexp "an alias for .set-file-modes. in .subr\\.el")
+ (let ((regexp "an alias for .set-file-modes. in .+subr\\.el")
(result (help-fns-tests--describe-function 'chmod)))
(should (string-match regexp result))))
(ert-deftest help-fns-test-bug23887 ()
"Test for https://debbugs.gnu.org/23887 ."
- (let ((regexp "an alias for .re-search-forward. in .subr\\.el")
+ (let ((regexp "an alias for .re-search-forward. in .+subr\\.el")
(result (help-fns-tests--describe-function 'search-forward-regexp)))
(should (string-match regexp result))))
diff --git a/test/lisp/emacs-lisp/cl-generic-tests.el b/test/lisp/emacs-lisp/cl-generic-tests.el
index 5aa58782f3..9582907e51 100644
--- a/test/lisp/emacs-lisp/cl-generic-tests.el
+++ b/test/lisp/emacs-lisp/cl-generic-tests.el
@@ -240,7 +240,7 @@ cl-generic-tests--method-files--finds-methods
(let ((retval (cl--generic-method-files 'cl-generic-tests--generic)))
(should (equal (length retval) 2))
(mapc (lambda (x)
- (should (equal (car x) cl-generic-tests--this-file))
+ (should (equal (file-truename (car x)) cl-generic-tests--this-file))
(should (equal (cadr x) 'cl-generic-tests--generic)))
retval)
(should-not (equal (nth 0 retval) (nth 1 retval)))))
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Test Failures
@ 2020-10-28 3:00 Vin Shelton
2020-10-28 8:20 ` Lars Ingebrigtsen
0 siblings, 1 reply; 9+ messages in thread
From: Vin Shelton @ 2020-10-28 3:00 UTC (permalink / raw)
To: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 592 bytes --]
Using the most recent sources and building in the source tree:
(
today=$(date +'%Y-%m-%d')
git clone --quiet git://git.savannah.gnu.org/emacs.git emacs-$today
cd ./emacs-$today
./autogen.sh >& /dev/null
./configure >& /dev/null
make bootstrap >& mk.out
make -k check |& tee check.out
)
I'm seeing three test failures:
3 unexpected results:
FAILED python-shell-buffer-substring-10
FAILED python-shell-buffer-substring-11
FAILED python-shell-buffer-substring-12
Please let me know if you need more details.
Regards,
Vin Shelton
--
*If Cooperstown is calling, it's no fluke*
[-- Attachment #2: Type: text/html, Size: 1874 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Test Failures
2020-10-28 3:00 Test Failures Vin Shelton
@ 2020-10-28 8:20 ` Lars Ingebrigtsen
2020-10-28 13:59 ` Vin Shelton
0 siblings, 1 reply; 9+ messages in thread
From: Lars Ingebrigtsen @ 2020-10-28 8:20 UTC (permalink / raw)
To: Vin Shelton; +Cc: emacs-devel
Vin Shelton <acs@alumni.princeton.edu> writes:
> 3 unexpected results:
> FAILED python-shell-buffer-substring-10
> FAILED python-shell-buffer-substring-11
> FAILED python-shell-buffer-substring-12
Should be fixed now.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Test Failures
2020-10-28 8:20 ` Lars Ingebrigtsen
@ 2020-10-28 13:59 ` Vin Shelton
2020-10-29 2:30 ` Dmitry Gutov
0 siblings, 1 reply; 9+ messages in thread
From: Vin Shelton @ 2020-10-28 13:59 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 839 bytes --]
I can confirm that the python-shell failures are gone. Thanks, Larsi.
FYI, on an unrelated note, I'm seeing five failures when BLDDIR != SRCDIR:
FAILED xref-elisp-test-find-defs-defgeneric-el
FAILED xref-elisp-test-find-defs-defun-defvar-el
FAILED xref-elisp-test-find-defs-defun-el
FAILED xref-elisp-test-find-defs-defvar-el
FAILED xref-elisp-test-find-defs-feature-el
HTH,
Vin
On Wed, Oct 28, 2020 at 4:20 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Vin Shelton <acs@alumni.princeton.edu> writes:
>
> > 3 unexpected results:
> > FAILED python-shell-buffer-substring-10
> > FAILED python-shell-buffer-substring-11
> > FAILED python-shell-buffer-substring-12
>
> Should be fixed now.
>
> --
> (domestic pets only, the antidote for overdose, milk.)
> bloggy blog: http://lars.ingebrigtsen.no
>
[-- Attachment #2: Type: text/html, Size: 2091 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Test Failures
2020-10-28 13:59 ` Vin Shelton
@ 2020-10-29 2:30 ` Dmitry Gutov
0 siblings, 0 replies; 9+ messages in thread
From: Dmitry Gutov @ 2020-10-29 2:30 UTC (permalink / raw)
To: Vin Shelton, Lars Ingebrigtsen; +Cc: emacs-devel
On 28.10.2020 15:59, Vin Shelton wrote:
> FYI, on an unrelated note, I'm seeing five failures when BLDDIR != SRCDIR:
>
> FAILED xref-elisp-test-find-defs-defgeneric-el
> FAILED xref-elisp-test-find-defs-defun-defvar-el
> FAILED xref-elisp-test-find-defs-defun-el
> FAILED xref-elisp-test-find-defs-defvar-el
> FAILED xref-elisp-test-find-defs-feature-el
This looks like bug#43004.
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-10-29 2:30 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-10-28 3:00 Test Failures Vin Shelton
2020-10-28 8:20 ` Lars Ingebrigtsen
2020-10-28 13:59 ` Vin Shelton
2020-10-29 2:30 ` Dmitry Gutov
-- strict thread matches above, loose matches on Subject: below --
2020-08-21 20:23 test failures Stephen Berman
2020-08-21 20:36 ` Lars Ingebrigtsen
2020-08-21 20:45 ` Stephen Berman
2020-08-22 11:32 ` Stephen Berman
2020-08-22 6:31 ` Eli Zaretskii
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).