bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 3415 bytes --]

Even with yesterday's patch to test/lisp/gnus/mml-sec-tests.el, Emacs master 
'make check' is currently failing the mml-first-secure-en-decrypt-sign-1 test on 
Fedora 31 x86-64. I'm attaching a failed test log.

Here are details about the configuration.

In GNU Emacs 28.0.50 (build 6, x86_64-pc-linux-gnu, GTK+ Version 3.24.13, cairo 
version 1.16.0)
  of 2020-08-05 built on Penguin.CS.UCLA.EDU
Repository revision: b22b1f935d47c71b59d59da1b6d125635470beca
Repository branch: master
System Description: Fedora 31 (Workstation Edition)

Recent messages:

Configured using:
  'configure --enable-gcc-warnings'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD
JSON PDUMPER LCMS2

Important settings:
   value of $LC_ALL: en_US.utf8
   value of $LC_COLLATE: en_US.UTF-8
   value of $LC_CTYPE: en_US.UTF-8
   value of $LC_MESSAGES: en_US.UTF-8
   value of $LANG: C
   locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
   global-whitespace-mode: t
   tooltip-mode: t
   global-eldoc-mode: t
   eldoc-mode: t
   electric-quote-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils term/xterm
xterm whitespace package easymenu browse-url url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json subr-x map url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit
x multi-tty make-network-process emacs)

Memory information:
((conses 16 648701 9307)
  (symbols 48 7636 2)
  (strings 32 20659 1692)
  (string-bytes 1 689813)
  (vectors 16 10653)
  (vector-slots 8 109771 9795)
  (floats 8 25 269)
  (intervals 56 205 0)
  (buffers 992 10))

[-- Attachment #2: mml-sec-tests.log --]
[-- Type: text/x-log, Size: 2724 bytes --]

Running 20 tests (2020-08-05 08:05:47-0700, selector `(not (or (tag :expensive-test) (tag :unstable)))')
Test mml-first-secure-en-decrypt-sign-1 backtrace:
  signal(epg-error ("Encrypt failed" "Exit"))
  mml-secure-test-fixture(#f(compiled-function () #<bytecode 0x6ccefb2
  mml-secure-test-key-fixture(#f(compiled-function () #<bytecode -0x77
  #f(compiled-function () #<bytecode -0xc26ac755dcd02aa>)()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name mml-first-secure-en-decrypt-sign-1 :d
  ert-run-or-rerun-test(#s(ert--stats :selector (not ...) :tests [... 
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/gnus/mml-sec-tests" 
  command-line()
  normal-top-level()
Test mml-first-secure-en-decrypt-sign-1 condition:
    (epg-error "Encrypt failed" "Exit")
   FAILED   1/20  mml-first-secure-en-decrypt-sign-1 (0.673274 sec)
   passed   2/20  mml-secure-en-decrypt-1 (0.454633 sec)
   passed   3/20  mml-secure-en-decrypt-2 (0.472967 sec)
   passed   4/20  mml-secure-en-decrypt-3 (0.511085 sec)
   passed   5/20  mml-secure-en-decrypt-4 (0.496392 sec)
  skipped   6/20  mml-secure-en-decrypt-passphrase-cache (0.000148 sec)
  skipped   7/20  mml-secure-en-decrypt-passphrase-no-cache-openpgp-todo (0.003517 sec)
   failed   8/20  mml-secure-en-decrypt-passphrase-no-cache-smime-todo (0.000149 sec)
   passed   9/20  mml-secure-en-decrypt-sign-2 (1.770544 sec)
   passed  10/20  mml-secure-en-decrypt-sign-3 (0.599651 sec)
   passed  11/20  mml-secure-find-usable-keys-1 (0.088486 sec)
   passed  12/20  mml-secure-find-usable-keys-2 (0.099421 sec)
   passed  13/20  mml-secure-key-checks (0.143075 sec)
   passed  14/20  mml-secure-select-preferred-keys-1 (0.006005 sec)
   passed  15/20  mml-secure-select-preferred-keys-2 (0.036085 sec)
   passed  16/20  mml-secure-select-preferred-keys-3 (0.016259 sec)
   passed  17/20  mml-secure-select-preferred-keys-4 (0.042690 sec)
   passed  18/20  mml-secure-sign-verify-1 (1.206041 sec)
   passed  19/20  mml-secure-sign-verify-2 (1.485268 sec)
   passed  20/20  mml-secure-sign-verify-3 (0.020467 sec)

Ran 20 tests, 17 results as expected, 1 unexpected, 2 skipped (2020-08-05 08:05:56-0700, 8.330873 sec)
1 expected failures

1 unexpected results:
   FAILED  mml-first-secure-en-decrypt-sign-1

2 skipped results:
  SKIPPED  mml-secure-en-decrypt-passphrase-cache
  SKIPPED  mml-secure-en-decrypt-passphrase-no-cache-openpgp-todo

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Lars Ingebrigtsen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Even with yesterday's patch to test/lisp/gnus/mml-sec-tests.el, Emacs
> master 'make check' is currently failing the
> mml-first-secure-en-decrypt-sign-1 test on Fedora 31 x86-64. I'm
> attaching a failed test log.

I was able to reproduce the failure every other run yesterday on one of
my machines (when it wasn't the first test), but today I'm failing to
make it fail even once.

Since that test was rather long (and tested several different things),
I've now split it into three, so that it's easier to see just what it is
that fails.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

On 8/5/20 12:20 PM, Lars Ingebrigtsen wrote:

> Since that test was rather long (and tested several different things),
> I've now split it into three, so that it's easier to see just what it is
> that fails.

I'm still getting a test failure with current master (commit 
99f609c6f3361e101cd665fbaad4a98f4631cab8) with 'make -j5 check' in an in-tree 
build. A failed mml-sec-tests.log attached.

(cd test && make -j5 lisp/gnus/mml-sec-tests) works. So perhaps it's a 
combination of a missing dependency and 'make -j'.

[-- Attachment #2: mml-sec-tests.log --]
[-- Type: text/x-log, Size: 2879 bytes --]

Running 22 tests (2020-08-06 15:27:36-0700, selector `(not (or (tag :expensive-test) (tag :unstable)))')
   passed   1/22  mml-secure-en-decrypt-1 (0.496084 sec)
   passed   2/22  mml-secure-en-decrypt-2 (0.497113 sec)
   passed   3/22  mml-secure-en-decrypt-3 (0.471411 sec)
   passed   4/22  mml-secure-en-decrypt-4 (0.493818 sec)
  skipped   5/22  mml-secure-en-decrypt-passphrase-cache (0.004017 sec)
  skipped   6/22  mml-secure-en-decrypt-passphrase-no-cache-openpgp-todo (0.006509 sec)
   failed   7/22  mml-secure-en-decrypt-passphrase-no-cache-smime-todo (0.002728 sec)
Test mml-secure-en-decrypt-sign-1-1-single backtrace:
  signal(epg-error ("Encrypt failed" "Exit"))
  mml-secure-test-fixture(#f(compiled-function () #<bytecode 0x6fb23f8
  mml-secure-test-key-fixture(#f(compiled-function () #<bytecode -0xa4
  #f(compiled-function () #<bytecode -0x113103c4c2be173d>)()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name mml-secure-en-decrypt-sign-1-1-single
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/gnus/mml-sec-tests" 
  command-line()
  normal-top-level()
Test mml-secure-en-decrypt-sign-1-1-single condition:
    (epg-error "Encrypt failed" "Exit")
   FAILED   8/22  mml-secure-en-decrypt-sign-1-1-single (0.655476 sec)
   passed   9/22  mml-secure-en-decrypt-sign-1-2-double (0.648772 sec)
   passed  10/22  mml-secure-en-decrypt-sign-1-3-double (0.597429 sec)
   passed  11/22  mml-secure-en-decrypt-sign-2 (1.853518 sec)
   passed  12/22  mml-secure-en-decrypt-sign-3 (0.585654 sec)
   passed  13/22  mml-secure-find-usable-keys-1 (0.070525 sec)
   passed  14/22  mml-secure-find-usable-keys-2 (0.133455 sec)
   passed  15/22  mml-secure-key-checks (0.114117 sec)
   passed  16/22  mml-secure-select-preferred-keys-1 (0.009963 sec)
   passed  17/22  mml-secure-select-preferred-keys-2 (0.088404 sec)
   passed  18/22  mml-secure-select-preferred-keys-3 (0.025307 sec)
   passed  19/22  mml-secure-select-preferred-keys-4 (0.049515 sec)
   passed  20/22  mml-secure-sign-verify-1 (1.157003 sec)
   passed  21/22  mml-secure-sign-verify-2 (1.462676 sec)
   passed  22/22  mml-secure-sign-verify-3 (0.024052 sec)

Ran 22 tests, 19 results as expected, 1 unexpected, 2 skipped (2020-08-06 15:27:46-0700, 9.695887 sec)
1 expected failures

1 unexpected results:
   FAILED  mml-secure-en-decrypt-sign-1-1-single

2 skipped results:
  SKIPPED  mml-secure-en-decrypt-passphrase-cache
  SKIPPED  mml-secure-en-decrypt-passphrase-no-cache-openpgp-todo

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Lars Ingebrigtsen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> I'm still getting a test failure with current master (commit
> 99f609c6f3361e101cd665fbaad4a98f4631cab8) with 'make -j5 check' in an
> in-tree build. A failed mml-sec-tests.log attached.
>
> (cd test && make -j5 lisp/gnus/mml-sec-tests) works. So perhaps it's a
> combination of a missing dependency and 'make -j'.

I've tried running the test with "make -j5 check" here on a Debian
machine, but I'm not able to reproduce it.  It's clearly timing-related
in one way or another, which makes it difficult to debug.

Should we just mark the test as unstable?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Paul Eggert]

On 8/6/20 11:59 PM, Lars Ingebrigtsen wrote:
> Should we just mark the test as unstable?

Would that cause 'make check' to work? If so, I guess it's good enough. Another 
option would be to disable the test.

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Stefan Kangas]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Would that cause 'make check' to work? If so, I guess it's good enough. Another
> option would be to disable the test.

Yes, then it would only be run on `make check-all'.

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Lars Ingebrigtsen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 8/6/20 11:59 PM, Lars Ingebrigtsen wrote:
>> Should we just mark the test as unstable?
>
> Would that cause 'make check' to work? If so, I guess it's good
> enough. Another option would be to disable the test.

I've now marked it as unstable -- does that get rid of the "make check"
failure for you?

And I've forgotten again where the Emacs build bot is located...
Googling for "emacs buildbot" got me nowhere.  :-/

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Robert Pluim]

>>>>> On Sat, 08 Aug 2020 12:01:57 +0200, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Paul Eggert <eggert@cs.ucla.edu> writes:
    >> On 8/6/20 11:59 PM, Lars Ingebrigtsen wrote:
    >>> Should we just mark the test as unstable?
    >> 
    >> Would that cause 'make check' to work? If so, I guess it's good
    >> enough. Another option would be to disable the test.

    Lars> I've now marked it as unstable -- does that get rid of the "make check"
    Lars> failure for you?

    Lars> And I've forgotten again where the Emacs build bot is located...
    Lars> Googling for "emacs buildbot" got me nowhere.  :-/

Which one? Thereʼs at least two, Hydra and EMBA, and EMBA is at:

<https://emba.gnu.org/emacs/emacs>

Robert

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Paul Eggert]

On 8/8/20 3:01 AM, Lars Ingebrigtsen wrote:
> I've now marked it as unstable -- does that get rid of the "make check"
> failure for you?

Yes, seems to work for me now; thanks.

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Which one? Thereʼs at least two, Hydra and EMBA, and EMBA is at:
>
> <https://emba.gnu.org/emacs/emacs>

Thanks!

All the builds on emba seems to fail:

From https://emba.gnu.org/emacs/emacs
13 * [new ref]               refs/pipelines/6293 -> refs/pipelines/6293
14error: cannot lock ref 'refs/remotes/origin/master': Unable to create '/builds/emacs/emacs/.git/refs/remotes/origin/master.lock': File exists.
15Another git process seems to be running in this repository, e.g.
16an editor opened by 'git commit'. Please make sure all processes
17are terminated then try again. If it still fails, a git process
18may have crashed in this repository earlier:
19remove the file manually to continue.
20 ! acd2cb28ef..67ffffa666  master              -> origin/master  (unable to update local ref)
21Auto packing the repository in background for optimum performance.
22See "git help gc" for manual housekeeping.


But Hydra seems to work:

https://hydra.nixos.org/jobset/gnu/emacs-trunk

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Lars Ingebrigtsen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 8/8/20 3:01 AM, Lars Ingebrigtsen wrote:
>> I've now marked it as unstable -- does that get rid of the "make check"
>> failure for you?
>
> Yes, seems to work for me now; thanks.

Thanks for checking; closing this report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42720: 28.0.50; mml-first-secure-en-decrypt-sign-1 test failing on Fedora 31

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 213 bytes --]

"make -j5 check" just failed for me on Ubuntu 18.04.5 with similar symptoms in 
mml-secure-en-decrypt-sign-2 so I took the liberty of marking that unstable too. 
Failing log attached, not that the log helps much.

[-- Attachment #2: mml-sec-tests.log --]
[-- Type: text/x-log, Size: 2648 bytes --]

Running 19 tests (2020-08-12 18:31:44-0700, selector `(not (or (tag :expensive-test) (tag :unstable)))')
   passed   1/19  mml-secure-en-decrypt-1 (0.459199 sec)
   passed   2/19  mml-secure-en-decrypt-2 (0.470843 sec)
   passed   3/19  mml-secure-en-decrypt-3 (0.498642 sec)
   passed   4/19  mml-secure-en-decrypt-4 (0.456075 sec)
  skipped   5/19  mml-secure-en-decrypt-passphrase-cache (0.001965 sec)
  skipped   6/19  mml-secure-en-decrypt-passphrase-no-cache-openpgp-todo (0.003543 sec)
   failed   7/19  mml-secure-en-decrypt-passphrase-no-cache-smime-todo (0.001812 sec)
   passed   8/19  mml-secure-en-decrypt-sign-1-2-double (0.572457 sec)
Test mml-secure-en-decrypt-sign-2 backtrace:
  signal(epg-error ("Encrypt failed" "Exit"))
  mml-secure-test-fixture(#f(compiled-function () #<bytecode 0x6cd2142
  mml-secure-test-key-fixture(#f(compiled-function () #<bytecode -0x27
  #f(compiled-function () #<bytecode -0x191bb3e9cac0d201>)()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name mml-secure-en-decrypt-sign-2 :documen
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/gnus/mml-sec-tests" 
  command-line()
  normal-top-level()
Test mml-secure-en-decrypt-sign-2 condition:
    (epg-error "Encrypt failed" "Exit")
   FAILED   9/19  mml-secure-en-decrypt-sign-2 (0.629818 sec)
   passed  10/19  mml-secure-find-usable-keys-1 (0.033761 sec)
   passed  11/19  mml-secure-find-usable-keys-2 (0.046812 sec)
   passed  12/19  mml-secure-key-checks (0.064345 sec)
   passed  13/19  mml-secure-select-preferred-keys-1 (0.003727 sec)
   passed  14/19  mml-secure-select-preferred-keys-2 (0.027046 sec)
   passed  15/19  mml-secure-select-preferred-keys-3 (0.027371 sec)
   passed  16/19  mml-secure-select-preferred-keys-4 (0.023920 sec)
   passed  17/19  mml-secure-sign-verify-1 (0.981920 sec)
   passed  18/19  mml-secure-sign-verify-2 (1.427348 sec)
   passed  19/19  mml-secure-sign-verify-3 (0.014458 sec)

Ran 19 tests, 16 results as expected, 1 unexpected, 2 skipped (2020-08-12 18:31:50-0700, 5.894490 sec)
1 expected failures

1 unexpected results:
   FAILED  mml-secure-en-decrypt-sign-2

2 skipped results:
  SKIPPED  mml-secure-en-decrypt-passphrase-cache
  SKIPPED  mml-secure-en-decrypt-passphrase-no-cache-openpgp-todo