bug#36940: tests slowness and failure after recent Tramp changes

[Michael Albinus <michael.albinus@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 4129 bytes --]

Stefan Kangas <stefan@marxist.se> writes:

> Hi Michael,

Hi Stefan,

> I'm still seeing failures on my MacOS machine on current master.  I
> have attached a log with two failing tests from when I run "make
> check" in the top level emacs folder as the file tramp-tests.log.
>
> Weirdly enough, when I say "cd test ; make tramp-tests", I get six
> errors instead -- but no log file is saved.  Not sure if I'm doing
> something wrong here, but I've included the console output from this
> test as tramp-tests2.console.log.  I tried running the tests in both
> ways a couple of times, and this odd result is reproducible every
> time.

If you say "make check", the default ert selector is applied, which
excludes expensive tests. if you say "make FOO", all test from package
foo are applied, also the expensive tests. You see it by the different
numbers of tests, 47 vs 67 tests.

If you want to get a log file when running tests for package FOO, say
"make FOO.log". This seems to be broken now; will check.

All of this is described in test/README.

> (Not sure if you want to reopen the bug report for this.)

For the time being, I'll continue to reply here. If needed (more complex
failure, not reported initially here) we might need to create another
bug report.

> (t 3 501 20 (0 0 0 1000) (0 0 0 1000) (0 0 0 1000) 0 "drwxr-xr-x" nil 41369843 (-1 . 18))
> (t 5 501 20 (0 0 0 1000) (0 0 0 1000) (0 0 0 1000) 0 "drwxr-xr-x" nil (631 . 16627) (-1 . 18))
> Test tramp-test19-directory-files-and-attributes condition:
>     (ert-test-failed
>      ((should
>        (tramp--test-file-attributes-equal-p
> 	(file-attributes ...)
> 	(cdr elt)))
>       :form
>       (tramp--test-file-attributes-equal-p
>        (t 3 501 20
> 	  (23904 37780)
> 	  (23904 37780)
> 	  (23904 37780)
> 	  102 "drwxr-xr-x" nil ...)
>        (t 5 501 20
> 	  (23904 37780)
> 	  (23904 37780)
> 	  (23904 37780)
> 	  170 "drwxr-xr-x" nil ...))
>       :value nil :explanation
>       (list-elt 1
> 		(different-atoms
> 		 (3 "#x3" "?")
> 		 (5 "#x5" "?")))))
>    FAILED  30/67  tramp-test19-directory-files-and-attributes (0.996916 sec)

Well, this time it is because of a different inode number, 41369843 vs
(631 . 16627). Will check what's up.

> Test tramp-test22-file-times condition:
>     (ert-test-failed
>      ((should
>        (equal
> 	(tramp-compat-file-attribute-modification-time ...)
> 	(seconds-to-time 1)))
>       :form
>       (equal
>        (0 1)
>        (0 1 0 0))
>       :value nil :explanation
>       (proper-lists-of-different-length 2 4
> 					(0 1)
> 					(0 1 0 0)
> 					first-mismatch-at 2)))
>    FAILED  33/67  tramp-test22-file-times (0.567633 sec)

Grrr, the time stamps (0 1) and (0 1 0 0) are the same, but handled
differently internally. Fixed in master.

> Test tramp-test30-make-process condition:
>     (ert-test-failed
>      ((should
>        (string-match "killed
> \\'"
> 		     (buffer-string)))
>       :form
>       (string-match "killed
> \\'" "killed: 9
> ")
>       :value nil))
>    FAILED  41/67  tramp-test30-make-process (1.024851 sec)

Ahh, the message on macOS is slightly different. Fixed in master.

> Test tramp-test41-utf8 condition:
>     (search-failed "^VAR_971C287AFA5BBEDD54ACB58B1CE718B4=Γυρίστε το Γαλαξία με Ώτο Στοπ$")
>    FAILED  58/67  tramp-test41-utf8 (35.641851 sec)

> Test tramp-test41-utf8-with-ls condition:
>     (search-failed "^VAR_971C287AFA5BBEDD54ACB58B1CE718B4=Γυρίστε το Γαλαξία με Ώτο Στοπ$")
>    FAILED  59/67  tramp-test41-utf8-with-ls (29.942527 sec)

> Test tramp-test41-utf8-with-perl condition:
>     (search-failed "^VAR_971C287AFA5BBEDD54ACB58B1CE718B4=Γυρίστε το Γαλαξία με Ώτο Στοπ$")
>    FAILED  60/67  tramp-test41-utf8-with-perl (36.502001 sec)

These three errors are basically the same. Could you please apply the
following patch to tramp-tests.el, and show me the output? You might run

make -C test tramp-tests >tramp-tests.log 2>&1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-diff, Size: 546 bytes --]

diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index 180f746c64..42d44f780b 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -5459,7 +5459,8 @@ tramp--test-utf8
   (skip-unless (not (tramp--test-windows-nt-and-batch)))
   (skip-unless (not (tramp--test-windows-nt-and-pscp-psftp-p)))

-  (tramp--test-utf8))
+  (tramp--test-instrument-test-case 10
+  (tramp--test-utf8)))

 (ert-deftest tramp-test41-utf8-with-stat ()
   "Check UTF8 encoding in file names and file contents.

[-- Attachment #3: Type: text/plain, Size: 51 bytes --]


> Thanks,
> Stefan Kangas

Best regards, Michael.