unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org
Subject: Re: parallel tests broken on Debian stable
Date: Mon, 20 May 2019 20:44:20 -0400	[thread overview]
Message-ID: <87mujgab6j.fsf@fifthhorseman.net> (raw)
In-Reply-To: <87pnocadqp.fsf@fifthhorseman.net>

[-- Attachment #1: Type: text/plain, Size: 5265 bytes --]

On Mon 2019-05-20 19:49:02 -0400, Daniel Kahn Gillmor wrote:
> However, i tried selectively upgrading all the versions of all of these
> packages *except for gdb* to the version in buster (or to the version
> from backports, in the case of the kernel).  and i'm *still* seeing the
> problem on the stretch system.
>
> So perhaps it's some interaction between timeout and gdb?  I haven't
> managed to test that particular combination yet.

Ok, i just went ahead and tested gdb 8.2.1-2 on a stretch system (after
rolling back my other package upgrades.

Getting gdb installed cleanly required upgrading libc6, libssl1.1, and
libpython (3.5 to 3.7) and associated packages to their versions from
buster, as well as pulling in a few libraries whose sonames weren't
already in stretch at all (libipt2, libncursesw6, libtinfo6).

once gdb was upgraded, the tests passed with moreutils parallel with
timeout enabled.

So the combination that tickles the problem is:

 * GNU timeout (either version)
 * moreutils parallel (either version)
 * gdb 7.12-6 (stretch)

and the problem goes away when gdb is upgraded to 8.2.1-2.

I wasn't able to whittle it down to a smaller reliable test,
unfortunately, but hopefully this is enough for someone else to pick it
up and run with it.


Digging around a bit further, i see back in 2011
9ade8160a6a3f6f55996d7a58dc2cd81a6df8395 from amdragon (cc'ed, though i
haven't heard from him since 2016) that introduced this comment (now in
tests/T380-atomicity.sh):

    # -tty /dev/null works around a conflict between the 'timeout' wrapper
    # and gdb's attempt to control the TTY.
     […]
    gdb -tty /dev/null -batch -x $TEST_DIRECTORY/atomicity.gdb notmuch >/dev/null 2>/dev/null


I tried adding "-tty /dev/null" to every place where ${TEST_GDB} was
invoked (in T050, T060, and T070), and i found that it produced test
failures in T050 and T060.  However, just directing stdin from /dev/null
on those runs of ${TEST_GDB} solved the problem for me. (see patch below)

Leaving any of those three gdb invocations without anything changed
meant that "make -j4 check" would hang pretty reliably with processes
stopped by SIGTTOU again with gdb from stretch.

Again, i don't understand any of this mechanism in detail, but the
following patch appears to avoid hangs of the test suite for me on a
stock debian stretch installation, and is (i think) a narrower change
than the earlier proposal that forced stdin from /dev/null on all the
tests uniformly.

If no one objects, i can propose this as an actual patch, but i'd feel
much happier if someone who understands this particular situation can
propose it with a clearer justifiation.

Regards,

        --dkg


diff --git a/test/T050-new.sh b/test/T050-new.sh
index dfc8508f..9b3d41df 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -366,7 +366,7 @@ run
 EOF
 
 ${TEST_GDB} --batch-silent --return-child-result -x notmuch-new-vanish.gdb \
-    --args notmuch new 2>OUTPUT 1>/dev/null
+    --args notmuch new 2>OUTPUT 1>/dev/null </dev/null
 echo "exit status: $?" >> OUTPUT
 
 # Clean up the file in case gdb isn't available.
diff --git a/test/T060-count.sh b/test/T060-count.sh
index 0c0bf473..a393edb6 100755
--- a/test/T060-count.sh
+++ b/test/T060-count.sh
@@ -116,7 +116,7 @@ EOF
 backup_database
 test_begin_subtest "error message from query_search_messages"
 ${TEST_GDB} --batch-silent --return-child-result -x count-files.gdb \
-    --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null
+    --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null < /dev/null
 cat <<EOF > EXPECTED
 notmuch count: A Xapian exception occurred
 A Xapian exception occurred performing query
diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index 05be473a..2d281fe6 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -267,12 +267,14 @@ for code in  FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR; do
     test_begin_subtest "EXIT_FAILURE when index_file returns $code"
     test_expect_code 1 \
          "${TEST_GDB} --batch-silent --return-child-result \
+ 	     -tty /dev/null \
 	     -ex 'set args insert < $gen_msg_filename' \
 	     -x index-file-$code.gdb notmuch"
 
     test_begin_subtest "success exit with --keep when index_file returns $code"
     test_expect_code 0 \
          "${TEST_GDB} --batch-silent --return-child-result \
+ 	     -tty /dev/null \
 	     -ex 'set args insert --keep < $gen_msg_filename' \
 	     -x index-file-$code.gdb notmuch"
 done
@@ -281,12 +283,14 @@ for code in OUT_OF_MEMORY XAPIAN_EXCEPTION ; do
     test_begin_subtest "EX_TEMPFAIL when index_file returns $code"
     test_expect_code 75 \
          "${TEST_GDB} --batch-silent --return-child-result \
+ 	     -tty /dev/null \
 	     -ex 'set args insert < $gen_msg_filename' \
 	     -x index-file-$code.gdb notmuch"
 
     test_begin_subtest "success exit with --keep when index_file returns $code"
     test_expect_code 0 \
          "${TEST_GDB} --batch-silent --return-child-result \
+ 	     -tty /dev/null \
 	     -ex 'set args insert --keep < $gen_msg_filename' \
 	     -x index-file-$code.gdb notmuch"
 done

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

  reply	other threads:[~2019-05-21  0:44 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-10 10:45 parallel tests broken on Debian stable David Bremner
2019-05-20 17:27 ` Daniel Kahn Gillmor
2019-05-20 18:55   ` Tomi Ollila
2019-05-20 21:45     ` Daniel Kahn Gillmor
2019-05-20 23:49   ` Daniel Kahn Gillmor
2019-05-21  0:44     ` Daniel Kahn Gillmor [this message]
2019-05-21  1:03       ` [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel Daniel Kahn Gillmor
2019-05-21  2:32         ` David Bremner
2019-05-21 15:45           ` Daniel Kahn Gillmor
2019-05-21  6:12         ` Tomi Ollila
2019-05-21 15:50           ` Daniel Kahn Gillmor
2019-05-21 20:23             ` Tomi Ollila
2021-10-30 19:30 ` parallel tests broken on Debian stable David Bremner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://notmuchmail.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87mujgab6j.fsf@fifthhorseman.net \
    --to=dkg@fifthhorseman.net \
    --cc=david@tethera.net \
    --cc=notmuch@notmuchmail.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).