* parallel tests broken on Debian stable
@ 2019-05-10 10:45 David Bremner
2019-05-20 17:27 ` Daniel Kahn Gillmor
2021-10-30 19:30 ` parallel tests broken on Debian stable David Bremner
0 siblings, 2 replies; 13+ messages in thread
From: David Bremner @ 2019-05-10 10:45 UTC (permalink / raw)
To: notmuch; +Cc: Jameson Graef Rollins, Daniel Kahn Gillmor
[-- Attachment #1: Type: text/plain, Size: 1200 bytes --]
In a debian stretch neither gnu parallel nor moreutils parallel
successfully completes the tests. Logs are attached. One thing worth
investigating is that gnu parallel inside timeout doesn't really work
right, since it says something like
parallel: SIGTERM received. No new jobs will be started.
parallel: Waiting for these 8 jobs to finish. Send SIGTERM again to stop now.
parallel: /home/bremner/notmuch/test/T357-index-decryption.sh
parallel: /home/bremner/notmuch/test/T330-emacs-subject-to-filename.sh
parallel: /home/bremner/notmuch/test/T310-emacs.sh
parallel: /home/bremner/notmuch/test/T350-crypto.sh
parallel: /home/bremner/notmuch/test/T355-smime.sh
parallel: /home/bremner/notmuch/test/T160-json.sh
parallel: /home/bremner/notmuch/test/T320-emacs-large-search-buffer.sh
parallel: /home/bremner/notmuch/test/T170-sexp.sh
and then doesn't stop. But the real question is why the jobs don't
complete within a 2 minute timeout.
The attached logs are from a stretch chroot running under buster. I also
saw something similar on a stretch system running on real
hardware. Unsurprisingly, the number of tests completed is
non-determinstic, and varies from machine to machine and run to run.
[-- Attachment #2: moreutils.log --]
[-- Type: application/octet-stream, Size: 6418 bytes --]
INFO: using 2 minute timeout for tests
INFO: running tests with moreutils parallel
T010-help-test: Testing online help
T020-compact: Testing "notmuch compact"
T060-count: Testing "notmuch count" for messages and threads
T030-config: Testing "notmuch config"
T040-setup: Testing "notmuch setup"
T000-basic: Testing the test framework itself.
T050-new: Testing "notmuch new" in several variations
T070-insert: Testing "notmuch insert"
PASS success is reported like this
PASS test runs if prerequisite is satisfied
PASS tests clean up after themselves
PASS notmuch --help
PASS tests clean up even after a failure
PASS Get string value
PASS No new messages
PASS PASS failure to clean up causes the test to fail
Notmuch new without a config suggests notmuch setup
PASS Get list value
PASS Single new message
PASS Insert zero-length file
PASS Set string value
PASS Insert non-message
PASS Single message (full-scan)
PASS Set string value again
PASS Database empty so far
PASS Set list value
PASS Multiple new messages
PASS Set list value again
PASS Insert message
PASS Permissions on inserted message should be 0600
PASS Multiple new messages (full-scan)
PASS Running compact
PASS Remove key
PASS No new messages (non-empty DB)
PASS Compact preserves database
PASS Ensure that test output is suppressed unless the test fails
PASS No new messages (full-scan)
PASS Restoring Backup
PASS Remove non-existent key
PASS Checking restored backup
PASS List all items
PASS New directories
T080-search: Testing "notmuch search" in several variations
PASS Top level --config=FILE option
PASS Top level --config:FILE option
PASS Alternate inode order
PASS Top level --config<space>FILE option
PASS Top level --config=FILE option changed the right file
PASS Ensure that -v does not suppress test output
PASS test that mail store was created
PASS Read config file through a symlink
PASS Insert message adds default tags
PASS mail store should be empty
PASS NOTMUCH_CONFIG is set and points to an existing file
PASS PATH is set to build directory
PASS notmuch is compiled with debugging symbols
PASS Write config file through a symlink
PASS message count is the default for notmuch count
PASS Writing config file through symlink follows symlink
PASS Insert duplicate message
PASS Message moved in
T090-search-output: Testing various settings for "notmuch search --output="
PASS Absolute database path returned
PASS message count with --output=messages
PASS Renamed message
PASS Relative database path properly expanded
PASS Deleted message
PASS thread count with --output=threads
T095-address: Testing "notmuch address" in several variants
PASS thread count is the default for notmuch search
PASS Duplicate message does not change tags
PASS files count
PASS Renamed directory
PASS files count for a duplicate message-id
PASS Insert message, add tag
PASS count with no matching messages
PASS Deleted directory
PASS count with no matching threads
PASS Insert tagged world-readable message
PASS Permissions on inserted world-readable message should be 0644
PASS New directory (at end of list)
PASS Insert tagged world-readable message with group-only umask
PASS message count is the default for batch count
PASS Permissions on inserted world-readable message with funny umask should be 0640
PASS Deleted directory (end of list)
PASS Search body
PASS Insert message, add/remove tags
PASS New symlink to directory
PASS batch message count
PASS Search by from:
PASS Insert message with default tags stays in new/
PASS New symlink to a file
PASS Search by to:
PASS Broken symlink aborts
PASS --output=threads
PASS Insert message with non-maildir synced tags stays in new/
PASS batch thread count
PASS Search by subject:
PASS --output=sender
PASS New two-level directory
PASS Search by subject (utf-8):
PASS batch message count with input file
PASS without --output
PASS Deleted two-level directory
PASS --output=sender --format=json
PASS Insert message with custom new.tags goes to cur/
PASS error message for database open
PASS Insert message with custom new.tags actually gets the tags
PASS Search by id:
PASS --output=recipients
PASS Insert message with maildir synced tags goes to cur/
PASS One character directory at top level
PASS Search by mid:
PASS Support single-message mbox
PASS --output=sender --output=recipients
PASS --output=sender --output=count
PASS --output=threads --format=json
PASS Skip and report non-mail files
PASS notmuch help
PASS notmuch --version
PASS Search by tag:
PASS Insert message with maildir sync off goes to new/
PASS --output=messages
PASS --output=recipients --output=address
PASS --output=messages --duplicate=1
PASS --output=sender --output=address --output=count
PASS Insert message into folder
PASS Ignore files and directories specified in new.ignore
PASS Search by thread:
PASS --output=count --format=json
PASS --output=messages --duplicate=2
PASS --deduplicate=no --sort=oldest-first --output=sender
PASS notmuch --help tag
PASS --output=messages --duplicate=3
PASS Insert message into top level folder
PASS Ignore files and directories specified in new.ignore (full-scan)
PASS --output=messages --format=json
PASS Search body (phrase)
PASS --deduplicate=no --sort=newest-first --output=sender --output=recipients
PASS Insert message into folder with trailing /
PASS --output=messages --format=json --duplicate=1
PASS Ignore files and directories specified in new.ignore (multiple occurrences)
PASS Search by from: (address)
PASS --output=messages --format=json --duplicate=2
PASS --deduplicate=address --output=sender --output=recipients
PASS Don't stop for ignored broken symlinks
PASS --output=messages --format=json --duplicate=3
PASS Insert message into folder, add/remove tags
PASS Search by from: (name)
PASS Ignore files and directories specified in new.ignore (regexp)
PASS Insert message into non-existent folder
PASS Search by from: (name and address)
PASS --output=files
PASS Quiet: No new mail.
Notmuch test suite complete.
All 34 tests passed.
[-- Attachment #3: gnuparallel.log --]
[-- Type: application/octet-stream, Size: 26632 bytes --]
INFO: using 2 minute timeout for tests
INFO: running tests with GNU parallel
T020-compact: Testing "notmuch compact"
PASS Running compact
PASS Compact preserves database
PASS Restoring Backup
PASS Checking restored backup
T000-basic: Testing the test framework itself.
PASS success is reported like this
PASS test runs if prerequisite is satisfied
PASS tests clean up after themselves
PASS tests clean up even after a failure
PASS failure to clean up causes the test to fail
PASS Ensure that test output is suppressed unless the test fails
PASS Ensure that -v does not suppress test output
PASS test that mail store was created
PASS mail store should be empty
PASS NOTMUCH_CONFIG is set and points to an existing file
PASS PATH is set to build directory
PASS notmuch is compiled with debugging symbols
T030-config: Testing "notmuch config"
PASS Get string value
PASS Get list value
PASS Set string value
PASS Set string value again
PASS Set list value
PASS Set list value again
PASS Remove key
PASS Remove non-existent key
PASS List all items
PASS Top level --config=FILE option
PASS Top level --config:FILE option
PASS Top level --config<space>FILE option
PASS Top level --config=FILE option changed the right file
PASS Read config file through a symlink
PASS Write config file through a symlink
PASS Writing config file through symlink follows symlink
PASS Absolute database path returned
PASS Relative database path properly expanded
T040-setup: Testing "notmuch setup"
PASS Notmuch new without a config suggests notmuch setup
PASS Create a new config interactively
T095-address: Testing "notmuch address" in several variants
PASS --output=sender
PASS without --output
PASS --output=sender --format=json
PASS --output=recipients
PASS --output=sender --output=recipients
PASS --output=sender --output=count
PASS --output=recipients --output=address
PASS --output=sender --output=address --output=count
PASS --output=count --format=json
PASS --deduplicate=no --sort=oldest-first --output=sender
PASS --deduplicate=no --sort=newest-first --output=sender --output=recipients
PASS --deduplicate=address --output=sender --output=recipients
PASS --deduplicate=no --output=sender
PASS --deduplicate=mailbox --output=sender --output=count
PASS --deduplicate=address --output=sender --output=count
T110-search-position-overlap-bug: Testing that notmuch does not overlap term positions
PASS Search for a@b.c matches
PASS Search for x@y.z matches
PASS Search for a@y.c must not match
T080-search: Testing "notmuch search" in several variations
PASS Search body
PASS Search by from:
PASS Search by to:
PASS Search by subject:
PASS Search by subject (utf-8):
PASS Search by id:
PASS Search by mid:
PASS Search by tag:
PASS Search by thread:
PASS Search body (phrase)
PASS Search by from: (address)
PASS Search by from: (name)
PASS Search by from: (name and address)
PASS Search by from: without prefix (name and address)
PASS Search by to: (address)
PASS Search by to: (name)
PASS Search by to: (name and address)
PASS Search by to: without prefix (name and address)
PASS Search by subject: (phrase)
PASS Search for all messages ("*")
PASS Search body (utf-8):
PASS headers do not have adjacent term positions
PASS parts have non-overlapping term positions
PASS parts do not have adjacent term positions
T090-search-output: Testing various settings for "notmuch search --output="
PASS --output=threads
PASS --output=threads --format=json
PASS --output=messages
PASS --output=messages --duplicate=1
PASS --output=messages --duplicate=2
PASS --output=messages --duplicate=3
PASS --output=messages --format=json
PASS --output=messages --format=json --duplicate=1
PASS --output=messages --format=json --duplicate=2
PASS --output=messages --format=json --duplicate=3
PASS --output=files
PASS --output=files --duplicate=1
PASS --output=files --format=json
PASS --output=files --format=json --duplicate=2
PASS --output=tags
PASS --output=tags --format=json
PASS sanitize output for quoted-printable line-breaks in author and subject
PASS search for non-existent message prints nothing
PASS search --format=json for non-existent message prints proper empty json
T120-search-insufficient-from-quoting: Testing messages with unquoted . in name
PASS Search by first name
PASS Search by last name:
PASS Search by address:
PASS Search for all messages:
T010-help-test: Testing online help
PASS notmuch --help
PASS notmuch help
PASS notmuch --version
PASS notmuch --help tag
PASS notmuch help tag
T060-count: Testing "notmuch count" for messages and threads
PASS message count is the default for notmuch count
PASS message count with --output=messages
PASS thread count with --output=threads
PASS thread count is the default for notmuch search
PASS files count
PASS files count for a duplicate message-id
PASS count with no matching messages
PASS count with no matching threads
PASS message count is the default for batch count
PASS batch message count
PASS batch thread count
PASS batch message count with input file
PASS error message for database open
PASS error message from query_search_messages
PASS count library function is non-destructive
T100-search-by-folder: Testing "notmuch search" by folder: and path: (with variations)
PASS Single-world folder: specification (multiple results)
PASS Top level folder
PASS Two-word path to narrow results to one
PASS Folder search with --output=files
PASS After removing duplicate instance of matching path
PASS Folder search with --output=files part #2
PASS After removing duplicate instance of matching path part #2
PASS After rename, old path returns nothing
PASS After rename, new path returns result
PASS folder: search
PASS top level folder: search
PASS path: search
PASS top level path: search
PASS recursive path: search
T050-new: Testing "notmuch new" in several variations
PASS No new messages
PASS Single new message
PASS Single message (full-scan)
PASS Multiple new messages
PASS Multiple new messages (full-scan)
PASS No new messages (non-empty DB)
PASS No new messages (full-scan)
PASS New directories
PASS Alternate inode order
PASS Message moved in
PASS Renamed message
PASS Deleted message
PASS Renamed directory
PASS Deleted directory
PASS New directory (at end of list)
PASS Deleted directory (end of list)
PASS New symlink to directory
PASS New symlink to a file
PASS Broken symlink aborts
PASS New two-level directory
PASS Deleted two-level directory
PASS One character directory at top level
PASS Support single-message mbox
PASS Skip and report non-mail files
PASS Ignore files and directories specified in new.ignore
PASS Ignore files and directories specified in new.ignore (full-scan)
PASS Ignore files and directories specified in new.ignore (multiple occurrences)
PASS Don't stop for ignored broken symlinks
PASS Ignore files and directories specified in new.ignore (regexp)
PASS Quiet: No new mail.
PASS Quiet: new, removed and renamed messages.
PASS Empty tags in new.tags are forbidden
PASS Tags starting with '-' in new.tags are forbidden
PASS Invalid tags set exit code
PASS Xapian exception: read only files
PASS Handle files vanishing between scandir and add_file
PASS reference loop does not crash
PASS reference loop ordered by date
T180-text: Testing --format=text output
PASS Show message: text
PASS Search message: text
PASS Show message: text, utf-8
PASS Search message: text, utf-8
PASS Search message tags: text0
PASS Compare text vs. text0 for threads
PASS Compare text vs. text0 for messages
PASS Compare text vs. text0 for files
PASS Compare text vs. text0 for tags
T130-search-limiting: Testing "notmuch search" --offset and --limit parameters
PASS messages: limit does the right thing
PASS messages: concatenation of limited searches
PASS messages: limit larger than result set
PASS messages: limit = 0
PASS messages: offset does the right thing
PASS messages: offset = 0
PASS messages: negative offset
PASS messages: negative offset
PASS messages: negative offset combined with limit
PASS messages: negative offset combined with equal limit
PASS messages: negative offset combined with large limit
PASS messages: negative offset larger then results
PASS threads: limit does the right thing
PASS threads: concatenation of limited searches
PASS threads: limit larger than result set
PASS threads: limit = 0
PASS threads: offset does the right thing
PASS threads: offset = 0
PASS threads: negative offset
PASS threads: negative offset
PASS threads: negative offset combined with limit
PASS threads: negative offset combined with equal limit
PASS threads: negative offset combined with large limit
PASS threads: negative offset larger then results
T205-author-naming: Testing naming of authors with unusual addresses
PASS Add author with empty quoted real name
T140-excludes: Testing "notmuch search, count and show" with excludes in several variations
PASS Search, exclude "deleted" messages from search
PASS Search, exclude "deleted" messages from message search
PASS Search, exclude "deleted" messages from message search --exclude=false
PASS Search, exclude "deleted" messages from message search (non-existent exclude-tag)
PASS Search, exclude "deleted" messages from search, overridden
PASS Search, exclude "deleted" messages from threads
PASS Search, don't exclude "deleted" messages when --exclude=flag specified
PASS Search, don't exclude "deleted" messages from search if not configured
PASS Search, default exclusion (thread summary)
PASS Search, default exclusion (messages)
PASS Search, exclude=true (thread summary)
PASS Search, exclude=true (messages)
PASS Search, exclude=false (thread summary)
PASS Search, exclude=false (messages)
PASS Search, exclude=flag (thread summary)
PASS Search, exclude=flag (messages)
PASS Search, exclude=all (thread summary)
PASS Search, exclude=all (messages)
PASS Search, default exclusion: tag in query (thread summary)
PASS Search, default exclusion: tag in query (messages)
PASS Search, exclude=true: tag in query (thread summary)
PASS Search, exclude=true: tag in query (messages)
PASS Search, exclude=false: tag in query (thread summary)
PASS Search, exclude=false: tag in query (messages)
PASS Search, exclude=flag: tag in query (thread summary)
PASS Search, exclude=flag: tag in query (messages)
PASS Search, exclude=all: tag in query (thread summary)
PASS Search, exclude=all: tag in query (messages)
PASS Count, default exclusion (messages)
PASS Count, default exclusion (threads)
PASS Count, exclude=true (messages)
PASS Count, exclude=true (threads)
PASS Count, exclude=false (messages)
PASS Count, exclude=false (threads)
PASS Count, default exclusion: tag in query (messages)
PASS Count, default exclusion: tag in query (threads)
PASS Count, exclude=true: tag in query (messages)
PASS Count, exclude=true: tag in query (threads)
PASS Count, exclude=false: tag in query (messages)
PASS Count, exclude=false: tag in query (threads)
PASS Show, default exclusion
PASS Show, default exclusion (entire-thread)
PASS Show, exclude=true
PASS Show, exclude=true (entire-thread)
PASS Show, exclude=false
PASS Show, exclude=false (entire-thread)
T200-thread-naming: Testing naming of threads with changing subject
PASS Initial thread name (oldest-first search)
PASS Initial thread name (newest-first search)
PASS Changed thread name (oldest-first search)
PASS Changed thread name (newest-first search)
PASS Ignore added reply prefix (Re:)
PASS Ignore added reply prefix (Aw:)
PASS Ignore added reply prefix (Vs:)
PASS Ignore added reply prefix (Sv:)
PASS Use empty subjects if necessary.
PASS Avoid empty subjects if possible (newest-first).
PASS Avoid empty subjects if possible (oldest-first).
PASS Test order of messages in "notmuch show"
T150-tagging: Testing "notmuch tag"
PASS Adding tags
PASS Removing tags
PASS No tag operations
PASS No query
PASS Redundant tagging
PASS Remove all
PASS Remove all with batch
PASS Remove all with a no-op
PASS Special characters in tags
PASS Tagging order
PASS --batch
PASS --input
PASS --batch --input
PASS --batch --input --remove-all
PASS --batch, dependence on previous line
PASS --batch, blank lines and comments
PASS --batch: checking error messages
PASS --batch: tags with quotes
PASS --batch: tags with punctuation and space
PASS --batch: unicode tags
PASS --batch: only space and % needs to be encoded.
PASS --batch: unicode message-ids
PASS Empty tag names
PASS Tag name beginning with -
PASS Xapian exception: read only files
T230-reply-to-sender: Testing "notmuch reply --reply-to=sender" in several variations
PASS Basic reply-to-sender
PASS From Us, Basic reply to message
PASS Multiple recipients
PASS From Us, Multiple TO recipients
PASS Reply with CC
PASS From Us, Reply with CC
PASS From Us, Reply no TO but with CC
PASS Reply from alternate address
PASS Support for Reply-To
PASS Support for Reply-To with multiple recipients
PASS Un-munging Reply-To
PASS Message with header of exactly 200 bytes
T220-reply: Testing "notmuch reply" in several variations
PASS Basic reply
PASS Multiple recipients
PASS Reply with CC
PASS Reply from alternate address
PASS Reply from address in named group list
PASS Support for Reply-To
PASS Un-munging Reply-To
PASS Un-munging Reply-To With Exact Match
PASS Un-munging Reply-To With Raw addr-spec
PASS Message with header of exactly 200 bytes
PASS From guessing: Envelope-To
PASS From guessing: X-Original-To
PASS From guessing: Delivered-To
PASS Reply with RFC 2047-encoded headers
PASS Reply with RFC 2047-encoded headers (JSON)
PASS Reply to a message with multiple Cc headers
T250-uuencode: Testing handling of uuencoded data
PASS Ensure content before uu data is indexed
PASS Ensure uu data is not indexed
PASS Ensure content after uu data is indexed
T210-raw: Testing notmuch show --format=raw
PASS Attempt to show multiple raw messages
PASS Show a raw message
PASS Show another raw message
PASS content, message of size 0001024
PASS return value, message of size 0001024
PASS content, message of size 0002048
PASS return value, message of size 0002048
PASS content, message of size 0004096
PASS return value, message of size 0004096
PASS content, message of size 0008192
PASS return value, message of size 0008192
PASS content, message of size 0016384
PASS return value, message of size 0016384
PASS content, message of size 0032768
PASS return value, message of size 0032768
PASS content, message of size 0065536
PASS return value, message of size 0065536
PASS content, message of size 0131072
PASS return value, message of size 0131072
PASS content, message of size 0262144
PASS return value, message of size 0262144
PASS content, message of size 0524288
PASS return value, message of size 0524288
PASS content, message of size 1048576
PASS return value, message of size 1048576
T270-author-order: Testing author reordering;
PASS Adding parent message
PASS Adding initial child message
PASS Adding second child message
PASS Searching when all three messages match
PASS Searching when two messages match
PASS Searching when only one message matches
PASS Searching when only first message matches
PASS Adding duplicate author
PASS Searching when all four messages match
PASS Adding non-monotonic child message
PASS Searching non-monotonic messages (oldest-first)
PASS Searching non-monotonic messages (newest-first)
T290-long-id: Testing messages with ridiculously-long message IDs
PASS Referencing long ID before adding
PASS Adding message with long ID
PASS Referencing long ID after adding
PASS Ensure all messages were threaded together
T280-from-guessing: Testing From line heuristics (with multiple configured addresses)
PASS Magic from guessing (nothing to go on)
PASS Magic from guessing (Envelope-to:)
PASS Magic from guessing (X-Original-To:)
PASS Magic from guessing (Received: .. for ..)
PASS Magic from guessing (Received: domain)
PASS Magic from guessing (multiple Received: headers)
PASS Testing From line heuristics (with single configured address)
PASS Magic from guessing (nothing to go on)
PASS Magic from guessing (Envelope-to:)
PASS Magic from guessing (X-Original-To:)
PASS Magic from guessing (Received: .. for ..)
PASS Magic from guessing (Received: domain)
T300-encoding: Testing encoding issues
PASS Message with text of unknown charset
PASS Search for ISO-8859-2 encoded message
PASS RFC 2047 encoded word with spaces
PASS RFC 2047 encoded words back to back
PASS RFC 2047 encoded words without space before or after
PASS Mislabeled Windows-1252 encoding
T190-multipart: Testing output of multipart message
PASS --format=text --part=0, full message
PASS --format=text --part=0 --body=false, message header
PASS --format=text --part=1, message body
PASS --format=text --part=2, multipart/mixed
PASS --format=text --part=3, rfc822 part
PASS --format=text --part=4, rfc822's multipart
PASS --format=text --part=5, rfc822's html part
PASS --format=text --include-html --part=5, rfc822's html part
PASS --format=text --part=6, rfc822's text part
PASS --format=text --part=7, inline attachment
PASS --format=text --part=8, plain text part
PASS --format=text --part=9, pgp signature (unverified)
PASS --format=text --part=8, no part, expect error
PASS --format=json --part=0, full message
PASS --format=json --part=1, message body
PASS --format=json --part=2, multipart/mixed
PASS --format=json --part=3, rfc822 part
PASS --format=json --part=4, rfc822's multipart/alternative
PASS --format=json --part=5, rfc822's html part
PASS --format=json --part=6, rfc822's text part
PASS --format=json --part=7, inline attachment
PASS --format=json --part=8, plain text part
PASS --format=json --part=9, pgp signature (unverified)
PASS --format=json --part=10, no part, expect error
PASS --format=raw
PASS --format=raw --part=0, full message
PASS --format=raw --part=1, message body
PASS --format=raw --part=2, multipart/mixed
PASS --format=raw --part=3, rfc822 part
PASS --format=raw --part=4, rfc822's multipart
PASS --format=raw --part=5, rfc822's html part
PASS --format=raw --part=6, rfc822's text part
PASS --format=raw --part=7, inline attachment
PASS --format=raw --part=8, plain text part
PASS --format=raw --part=9, pgp signature (unverified)
PASS --format=raw --part=10, no part, expect error
PASS --format=mbox
PASS --format=mbox --part=1, incompatible, expect error
PASS 'notmuch reply' to a multipart message
PASS 'notmuch reply' to a multipart message with json format
PASS 'notmuch show --part' does not corrupt a part with CRLF pair
PASS html parts excluded by default
PASS html parts included
PASS indexes mime-type #1
PASS indexes mime-type #2
PASS indexes mime-type #3
PASS case of Content-Disposition doesn't matter for indexing
T260-thread-order: Testing threading when messages received out of order
PASS Messages with one parent get linked in all delivery orders
PASS Messages with all parents get linked in all delivery orders
T240-dump-restore: Testing "notmuch dump" and "notmuch restore"
PASS dump header
PASS Dumping all tags
PASS Dumping all tags II
PASS Clearing all tags
PASS Clearing all tags
PASS Restoring original tags
PASS Restore with nothing to do
PASS Accumulate with existing tags
PASS Accumulate with no tags
PASS Accumulate with new tags
PASS Invalid restore invocation
PASS dump --output=outfile
PASS dump --output=outfile --
PASS dump --gzip
PASS dump --gzip --output=outfile
PASS restoring gzipped stdin
PASS restoring gzipped file
PASS dump -- from:cworth
PASS dump --output=outfile from:cworth
PASS dump --output=outfile -- from:cworth
PASS Check for a safe set of message-ids
PASS format=batch-tag, dump sanity check.
PASS format=batch-tag, missing newline
PASS format=batch-tag, # round-trip
PASS format=batch-tag, # blank lines and comments
PASS format=batch-tag, # reverse-round-trip empty tag
PASS restoring empty file is not an error
PASS file of comments and blank lines is not an error
PASS detect format=batch-tag with leading comments and blanks
PASS detect format=sup with leading comments and blanks
PASS format=batch-tag, round trip with strange tags
PASS format=batch-tag, checking encoded output
PASS restoring sane tags
PASS format=batch-tag, restore=auto
PASS format=sup, restore=auto
PASS format=batch-tag, restore=default
PASS format=sup, restore=default
PASS restore: checking error messages
PASS roundtripping random message-ids and tags
T340-maildir-sync: Testing maildir synchronization
PASS Adding 'S' flag to existing filename removes 'unread' tag
PASS Adding message with 'S' flag prevents 'unread' tag
PASS Adding message with 'S' w/o 'unread' in new.tags prevents 'unread' tag
PASS Adding 'replied' tag adds 'R' flag to filename
PASS notmuch show works with renamed file (without notmuch new)
PASS notmuch reply works with renamed file (without notmuch new)
PASS notmuch new detects no file rename after tag->flag synchronization
PASS When read, message moved from new to cur
PASS No rename should be detected by notmuch new
PASS Adding non-maildir tags does not move message from new to cur
PASS Message in cur lacking maildir info gets one on any tag change
PASS Message in new with maildir info is moved to cur on any tag change
PASS Removing 'S' flag from existing filename adds 'unread' tag
PASS Removing info from filename leaves tags unchanged
PASS Can remove unread tag from message in non-maildir directory
PASS Message in non-maildir directory does not get renamed
PASS notmuch dump/restore re-synchronizes maildir tags with flags
PASS Adding flags to duplicate message tags the mail
PASS Adding duplicate message without flags does not remove tags
PASS Tag changes modify flags of multiple files
PASS Synchronizing tag changes preserves unsupported maildir flags
PASS A file with non-compliant maildir info will not be renamed
PASS Files in new/ get default synchronized tags
PASS draft is valid in new.tags
PASS flagged is valid in new.tags
PASS passed is valid in new.tags
PASS replied is valid in new.tags
T070-insert: Testing "notmuch insert"
PASS Insert zero-length file
PASS Insert non-message
PASS Database empty so far
PASS Insert message
PASS Permissions on inserted message should be 0600
PASS Insert message adds default tags
PASS Insert duplicate message
PASS Duplicate message does not change tags
PASS Insert message, add tag
PASS Insert tagged world-readable message
PASS Permissions on inserted world-readable message should be 0644
PASS Insert tagged world-readable message with group-only umask
PASS Permissions on inserted world-readable message with funny umask should be 0640
PASS Insert message, add/remove tags
PASS Insert message with default tags stays in new/
PASS Insert message with non-maildir synced tags stays in new/
PASS Insert message with custom new.tags goes to cur/
PASS Insert message with custom new.tags actually gets the tags
PASS Insert message with maildir synced tags goes to cur/
PASS Insert message with maildir sync off goes to new/
PASS Insert message into folder
PASS Insert message into top level folder
PASS Insert message into folder with trailing /
PASS Insert message into folder, add/remove tags
PASS Insert message into non-existent folder
PASS Insert message, create folder
PASS Insert message, create subfolder
PASS Created subfolder should have permissions 0700
PASS Created subfolder new/ should also have permissions 0700
PASS Insert message, create world-readable subfolder
PASS Created world-readable subfolder should have permissions 0755
PASS Created world-readable subfolder new/ should also have permissions 0755
PASS Insert message, create existing subfolder
PASS Insert message, create invalid subfolder
PASS Empty tags in new.tags are forbidden
PASS Tags starting with '-' in new.tags are forbidden
PASS Invalid tags set exit code
PASS EXIT_FAILURE when index_file returns FILE_NOT_EMAIL
PASS success exit with --keep when index_file returns FILE_NOT_EMAIL
PASS EXIT_FAILURE when index_file returns READ_ONLY_DATABASE
PASS success exit with --keep when index_file returns READ_ONLY_DATABASE
PASS EXIT_FAILURE when index_file returns UPGRADE_REQUIRED
PASS success exit with --keep when index_file returns UPGRADE_REQUIRED
PASS EXIT_FAILURE when index_file returns PATH_ERROR
PASS success exit with --keep when index_file returns PATH_ERROR
PASS EX_TEMPFAIL when index_file returns OUT_OF_MEMORY
PASS success exit with --keep when index_file returns OUT_OF_MEMORY
PASS EX_TEMPFAIL when index_file returns XAPIAN_EXCEPTION
PASS success exit with --keep when index_file returns XAPIAN_EXCEPTION
parallel: SIGTERM received. No new jobs will be started.
parallel: Waiting for these 8 jobs to finish. Send SIGTERM again to stop now.
parallel: /home/bremner/notmuch/test/T357-index-decryption.sh
parallel: /home/bremner/notmuch/test/T330-emacs-subject-to-filename.sh
parallel: /home/bremner/notmuch/test/T310-emacs.sh
parallel: /home/bremner/notmuch/test/T350-crypto.sh
parallel: /home/bremner/notmuch/test/T355-smime.sh
parallel: /home/bremner/notmuch/test/T160-json.sh
parallel: /home/bremner/notmuch/test/T320-emacs-large-search-buffer.sh
parallel: /home/bremner/notmuch/test/T170-sexp.sh
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: parallel tests broken on Debian stable
2019-05-10 10:45 parallel tests broken on Debian stable David Bremner
@ 2019-05-20 17:27 ` Daniel Kahn Gillmor
2019-05-20 18:55 ` Tomi Ollila
2019-05-20 23:49 ` Daniel Kahn Gillmor
2021-10-30 19:30 ` parallel tests broken on Debian stable David Bremner
1 sibling, 2 replies; 13+ messages in thread
From: Daniel Kahn Gillmor @ 2019-05-20 17:27 UTC (permalink / raw)
To: David Bremner, notmuch
[-- Attachment #1: Type: text/plain, Size: 1343 bytes --]
Thanks for catching this, Bremner.
On Fri 2019-05-10 07:45:18 -0300, David Bremner wrote:
> In a debian stretch neither gnu parallel nor moreutils parallel
> successfully completes the tests.
I have confirmed this misbehavior on debian stretch with moreutils
parallel. on stretch (with gmime 3.0 from stretch-backports), the tests
complete successfully in serial, but with moreutils parallel installed
they consume the full 2 minutes and then time out, but they time out
without failing explicitly.
> But the real question is why the jobs don't complete within a 2 minute
> timeout.
i agree, this is a serious issue, though we should also ensure that
timed-out tests count as failures.
So there are three things to fix:
a) when a timeout hits, the test suite should explicitly fail, even if
no individual tests have failed.
b) we should have a clearer sense of which tests completed and which
did not
c) we should avoid the timeout hanging :)
Looking at the output of "ps" during the hanging test suite, it appears
that parallel is getting blocked by running: T050, T060, T070, T080 (i'm
using a 4-core machine).
I've just sent two patches to the list that aim for part (a), and i'm
going to work on part (c) on stable now. if someone else wants to
improve the test suite to handle (b) that would be great.
--dkg
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: parallel tests broken on Debian stable
2019-05-20 17:27 ` Daniel Kahn Gillmor
@ 2019-05-20 18:55 ` Tomi Ollila
2019-05-20 21:45 ` Daniel Kahn Gillmor
2019-05-20 23:49 ` Daniel Kahn Gillmor
1 sibling, 1 reply; 13+ messages in thread
From: Tomi Ollila @ 2019-05-20 18:55 UTC (permalink / raw)
To: Daniel Kahn Gillmor, David Bremner, notmuch
On Mon, May 20 2019, Daniel Kahn Gillmor wrote:
> Thanks for catching this, Bremner.
>
> On Fri 2019-05-10 07:45:18 -0300, David Bremner wrote:
>> In a debian stretch neither gnu parallel nor moreutils parallel
>> successfully completes the tests.
>
> I have confirmed this misbehavior on debian stretch with moreutils
> parallel. on stretch (with gmime 3.0 from stretch-backports), the tests
> complete successfully in serial, but with moreutils parallel installed
> they consume the full 2 minutes and then time out, but they time out
> without failing explicitly.
>
>> But the real question is why the jobs don't complete within a 2 minute
>> timeout.
>
> i agree, this is a serious issue, though we should also ensure that
> timed-out tests count as failures.
>
> So there are three things to fix:
>
> a) when a timeout hits, the test suite should explicitly fail, even if
> no individual tests have failed.
>
> b) we should have a clearer sense of which tests completed and which
> did not
>
> c) we should avoid the timeout hanging :)
>
>
> Looking at the output of "ps" during the hanging test suite, it appears
> that parallel is getting blocked by running: T050, T060, T070, T080 (i'm
> using a 4-core machine).
>
> I've just sent two patches to the list that aim for part (a), and i'm
> going to work on part (c) on stable now. if someone else wants to
> improve the test suite to handle (b) that would be great.
Simplest would be that after ./notmuch-test --debug exits, test-results/
would still exist :D
>
> --dkg
Tomi
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: parallel tests broken on Debian stable
2019-05-20 17:27 ` Daniel Kahn Gillmor
2019-05-20 18:55 ` Tomi Ollila
@ 2019-05-20 23:49 ` Daniel Kahn Gillmor
2019-05-21 0:44 ` Daniel Kahn Gillmor
1 sibling, 1 reply; 13+ messages in thread
From: Daniel Kahn Gillmor @ 2019-05-20 23:49 UTC (permalink / raw)
To: David Bremner, notmuch
[-- Attachment #1: Type: text/plain, Size: 8409 bytes --]
On Mon 2019-05-20 13:27:03 -0400, Daniel Kahn Gillmor wrote:
> c) we should avoid the timeout hanging :)
I dug into this today, and i'm reporting back my findings.
I have what appears to be a fix (see below), but i don't understand it,
so i'm not advocating for it.
To be clear: my two test cases are two KVM instances, one running
stretch (debian stable) and one running sid (debian unstable). both
systems have 4 virtual CPUs (on a hardware platform that has 4 cores).
The two VMs are otherwise similarly configured. Both have moreutils
installed, and GNU parallel is not installed.
on the stretch system, i can achieve this hang/failure with a simple
"make -j4 check". on the "sid" system, i do not see the failure.
When i disable the use of timeout entirely (with NOTMUCH_TEST_TIMEOUT=0,
see id:20190520232535.4904-1-dkg@fifthhorseman.net), the problem goes
away on the stretch system.
When i inspect the state of the debian stretch system when the tests are
hanging, i see this (from "ps auwx"):
------------------------
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
dkg 7980 1.8 0.5 10228 4348 pts/1 S+ 18:49 0:00 make -j4 check
dkg 8001 0.0 0.3 11228 2884 pts/1 S+ 18:49 0:00 bash /home/dkg/src/notmuch/notmuch/test/notmuch-test
dkg 8011 0.0 0.1 10092 804 pts/1 S 18:49 0:00 timeout 2m parallel -- /home/dkg/src/notmuch/notmuch/test/T000-basic.sh /home/dkg/src/notmuch/notmuch/test/T010-help-test.sh /home/dkg/
dkg 8012 0.0 0.0 4168 744 pts/1 T 18:49 0:00 parallel -- /home/dkg/src/notmuch/notmuch/test/T000-basic.sh /home/dkg/src/notmuch/notmuch/test/T010-help-test.sh /home/dkg/src/notmuch
dkg 8013 0.0 0.0 4168 96 pts/1 T 18:49 0:00 parallel -- /home/dkg/src/notmuch/notmuch/test/T000-basic.sh /home/dkg/src/notmuch/notmuch/test/T010-help-test.sh /home/dkg/src/notmuch
dkg 8014 0.0 0.0 4168 96 pts/1 T 18:49 0:00 parallel -- /home/dkg/src/notmuch/notmuch/test/T000-basic.sh /home/dkg/src/notmuch/notmuch/test/T010-help-test.sh /home/dkg/src/notmuch
dkg 8267 0.0 0.0 4168 96 pts/1 T 18:49 0:00 parallel -- /home/dkg/src/notmuch/notmuch/test/T000-basic.sh /home/dkg/src/notmuch/notmuch/test/T010-help-test.sh /home/dkg/src/notmuch
dkg 8268 0.0 0.0 4276 732 pts/1 T 18:49 0:00 sh -c /home/dkg/src/notmuch/notmuch/test/T050-new.sh
dkg 8270 2.7 0.4 11772 3744 pts/1 T 18:49 0:00 bash /home/dkg/src/notmuch/notmuch/test/T050-new.sh
dkg 8320 0.0 0.0 4168 96 pts/1 T 18:49 0:00 parallel -- /home/dkg/src/notmuch/notmuch/test/T000-basic.sh /home/dkg/src/notmuch/notmuch/test/T010-help-test.sh /home/dkg/src/notmuch
dkg 8321 0.0 0.0 4276 748 pts/1 T 18:49 0:00 sh -c /home/dkg/src/notmuch/notmuch/test/T060-count.sh
dkg 8322 0.7 0.4 11752 3556 pts/1 T 18:49 0:00 bash /home/dkg/src/notmuch/notmuch/test/T060-count.sh
dkg 8345 0.0 0.0 4168 96 pts/1 T 18:49 0:00 parallel -- /home/dkg/src/notmuch/notmuch/test/T000-basic.sh /home/dkg/src/notmuch/notmuch/test/T010-help-test.sh /home/dkg/src/notmuch
dkg 8346 0.0 0.0 4276 744 pts/1 T 18:49 0:00 sh -c /home/dkg/src/notmuch/notmuch/test/T070-insert.sh
dkg 8347 1.7 0.4 11772 3764 pts/1 T 18:49 0:00 bash /home/dkg/src/notmuch/notmuch/test/T070-insert.sh
dkg 8425 0.0 0.0 4168 96 pts/1 T 18:49 0:00 parallel -- /home/dkg/src/notmuch/notmuch/test/T000-basic.sh /home/dkg/src/notmuch/notmuch/test/T010-help-test.sh /home/dkg/src/notmuch
dkg 8426 0.0 0.0 4276 740 pts/1 T 18:49 0:00 sh -c /home/dkg/src/notmuch/notmuch/test/T080-search.sh
dkg 8427 1.5 0.4 11752 3664 pts/1 T 18:49 0:00 bash /home/dkg/src/notmuch/notmuch/test/T080-search.sh
dkg 8763 4.7 2.9 73960 22708 pts/1 T 18:49 0:00 gdb --batch-silent --return-child-result -x count-files.gdb --args notmuch count --output=files *
dkg 8914 0.0 0.8 68508 6228 pts/1 T 18:49 0:00 notmuch search --format=text0 --output=files --offset=1 --limit=1 *
dkg 8915 0.0 0.1 4484 1164 pts/1 T 18:49 0:00 xargs -0 -I {} mv {} /home/dkg/src/notmuch/notmuch/test/tmp.T050-new/mail/moved_messages
dkg 8916 0.0 0.5 68244 3824 pts/1 T 18:49 0:00 notmuch insert --folder=Drafts +draft -unread
dkg 8919 0.0 0.0 13012 704 pts/1 T 18:49 0:00 notmuch new
dkg 8920 0.0 0.0 1412 4 pts/1 t 18:49 0:00 /bin/bash -c exec /home/dkg/src/notmuch/notmuch/notmuch count --output=files \*
------------------------
As you can see in the "STAT" column, nearly all of the hanging processes
are marked with T ("stopped by job control signal" according to ps(1)).
I also note that "t" means "stopped by debugger during the tracing" --
maybe that final line (with "t") is the special one that triggers this?
i don't know.
When i try to connect to any of these stopped processes with "strace -p
$PID", strace reports:
strace: Process 4204 attached
--- stopped by SIGTTOU ---
SIGTTOU is novel to me, and i don't really understand why the test suite
would have this problem. Skimming this guidance:
http://curiousthing.org/sigttin-sigttou-deep-dive-linux
suggested that maybe if i just decoupled the processes from the terminal
"enough" i could get away with a functioning test suite. redirecting
all of stdin, stdout, stderr to /dev/null worked! then i tried pruning
out different pieces, and found that all i needed to do was to redirect
stdin from /dev/null and the test suite would run without problems in
parallel with moreutils parallel. (it also works with GNU parallel, and
if i run the tests serially).
So the patch below is a "fix" but it's not a principled one.
the source for moreutils parallel.c doesn't appear to have changed at
all between stretch and buster. I tried upgrading the version of
moreutils on this stretch system from 0.60-1 to 0.62-1, and i was able
to reproduce the same problem. So i don't believe the problem is with
moreutils.
Some things that might be different between debian stable (stretch) and
testing (buster):
package provides stretch buster
------- -------- ------- ------
GNU coreutils /usr/bin/timeout 8.26-3 8.30-3
GNU bash /bin/bash 4.4-5 5.0-4
GNU dash /bin/sh (via symlink) 0.5.8-2.4 0.5.10.2-5
Linux the kernel 4.9.168-1+deb9u2 4.19.37-3
GNU gdb /usr/bin/gdb 7.12-6 8.2.1-2
I also tried changing the symlink for /bin/sh to point to bash instead
of dash, and was still able to replicate the problem, so i suspect dash
is not the culprit.
However, i tried selectively upgrading all the versions of all of these
packages *except for gdb* to the version in buster (or to the version
from backports, in the case of the kernel). and i'm *still* seeing the
problem on the stretch system.
So perhaps it's some interaction between timeout and gdb? I haven't
managed to test that particular combination yet.
I hope someone else will look into this further, as i'm out of my depth.
--dkg
diff --git a/test/Makefile.local b/test/Makefile.local
index 47244e8f..3a57b6be 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -66,13 +66,13 @@ test-binaries: $(TEST_BINARIES)
test: all test-binaries
ifeq ($V,)
@echo 'Use "$(MAKE) V=1" to see the details for passing and known broken tests.'
- @env NOTMUCH_TEST_QUIET=1 $(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS)
+ @env NOTMUCH_TEST_QUIET=1 $(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS) </dev/null
else
# The user has explicitly enabled quiet execution.
ifeq ($V,0)
- @env NOTMUCH_TEST_QUIET=1 $(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS)
+ @env NOTMUCH_TEST_QUIET=1 $(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS) </dev/null
else
- @$(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS)
+ @$(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS) </dev/null
endif
endif
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: parallel tests broken on Debian stable
2019-05-20 23:49 ` Daniel Kahn Gillmor
@ 2019-05-21 0:44 ` Daniel Kahn Gillmor
2019-05-21 1:03 ` [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel Daniel Kahn Gillmor
0 siblings, 1 reply; 13+ messages in thread
From: Daniel Kahn Gillmor @ 2019-05-21 0:44 UTC (permalink / raw)
To: David Bremner, notmuch
[-- Attachment #1: Type: text/plain, Size: 5265 bytes --]
On Mon 2019-05-20 19:49:02 -0400, Daniel Kahn Gillmor wrote:
> However, i tried selectively upgrading all the versions of all of these
> packages *except for gdb* to the version in buster (or to the version
> from backports, in the case of the kernel). and i'm *still* seeing the
> problem on the stretch system.
>
> So perhaps it's some interaction between timeout and gdb? I haven't
> managed to test that particular combination yet.
Ok, i just went ahead and tested gdb 8.2.1-2 on a stretch system (after
rolling back my other package upgrades.
Getting gdb installed cleanly required upgrading libc6, libssl1.1, and
libpython (3.5 to 3.7) and associated packages to their versions from
buster, as well as pulling in a few libraries whose sonames weren't
already in stretch at all (libipt2, libncursesw6, libtinfo6).
once gdb was upgraded, the tests passed with moreutils parallel with
timeout enabled.
So the combination that tickles the problem is:
* GNU timeout (either version)
* moreutils parallel (either version)
* gdb 7.12-6 (stretch)
and the problem goes away when gdb is upgraded to 8.2.1-2.
I wasn't able to whittle it down to a smaller reliable test,
unfortunately, but hopefully this is enough for someone else to pick it
up and run with it.
Digging around a bit further, i see back in 2011
9ade8160a6a3f6f55996d7a58dc2cd81a6df8395 from amdragon (cc'ed, though i
haven't heard from him since 2016) that introduced this comment (now in
tests/T380-atomicity.sh):
# -tty /dev/null works around a conflict between the 'timeout' wrapper
# and gdb's attempt to control the TTY.
[…]
gdb -tty /dev/null -batch -x $TEST_DIRECTORY/atomicity.gdb notmuch >/dev/null 2>/dev/null
I tried adding "-tty /dev/null" to every place where ${TEST_GDB} was
invoked (in T050, T060, and T070), and i found that it produced test
failures in T050 and T060. However, just directing stdin from /dev/null
on those runs of ${TEST_GDB} solved the problem for me. (see patch below)
Leaving any of those three gdb invocations without anything changed
meant that "make -j4 check" would hang pretty reliably with processes
stopped by SIGTTOU again with gdb from stretch.
Again, i don't understand any of this mechanism in detail, but the
following patch appears to avoid hangs of the test suite for me on a
stock debian stretch installation, and is (i think) a narrower change
than the earlier proposal that forced stdin from /dev/null on all the
tests uniformly.
If no one objects, i can propose this as an actual patch, but i'd feel
much happier if someone who understands this particular situation can
propose it with a clearer justifiation.
Regards,
--dkg
diff --git a/test/T050-new.sh b/test/T050-new.sh
index dfc8508f..9b3d41df 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -366,7 +366,7 @@ run
EOF
${TEST_GDB} --batch-silent --return-child-result -x notmuch-new-vanish.gdb \
- --args notmuch new 2>OUTPUT 1>/dev/null
+ --args notmuch new 2>OUTPUT 1>/dev/null </dev/null
echo "exit status: $?" >> OUTPUT
# Clean up the file in case gdb isn't available.
diff --git a/test/T060-count.sh b/test/T060-count.sh
index 0c0bf473..a393edb6 100755
--- a/test/T060-count.sh
+++ b/test/T060-count.sh
@@ -116,7 +116,7 @@ EOF
backup_database
test_begin_subtest "error message from query_search_messages"
${TEST_GDB} --batch-silent --return-child-result -x count-files.gdb \
- --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null
+ --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null < /dev/null
cat <<EOF > EXPECTED
notmuch count: A Xapian exception occurred
A Xapian exception occurred performing query
diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index 05be473a..2d281fe6 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -267,12 +267,14 @@ for code in FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR; do
test_begin_subtest "EXIT_FAILURE when index_file returns $code"
test_expect_code 1 \
"${TEST_GDB} --batch-silent --return-child-result \
+ -tty /dev/null \
-ex 'set args insert < $gen_msg_filename' \
-x index-file-$code.gdb notmuch"
test_begin_subtest "success exit with --keep when index_file returns $code"
test_expect_code 0 \
"${TEST_GDB} --batch-silent --return-child-result \
+ -tty /dev/null \
-ex 'set args insert --keep < $gen_msg_filename' \
-x index-file-$code.gdb notmuch"
done
@@ -281,12 +283,14 @@ for code in OUT_OF_MEMORY XAPIAN_EXCEPTION ; do
test_begin_subtest "EX_TEMPFAIL when index_file returns $code"
test_expect_code 75 \
"${TEST_GDB} --batch-silent --return-child-result \
+ -tty /dev/null \
-ex 'set args insert < $gen_msg_filename' \
-x index-file-$code.gdb notmuch"
test_begin_subtest "success exit with --keep when index_file returns $code"
test_expect_code 0 \
"${TEST_GDB} --batch-silent --return-child-result \
+ -tty /dev/null \
-ex 'set args insert --keep < $gen_msg_filename' \
-x index-file-$code.gdb notmuch"
done
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel
2019-05-21 0:44 ` Daniel Kahn Gillmor
@ 2019-05-21 1:03 ` Daniel Kahn Gillmor
2019-05-21 2:32 ` David Bremner
2019-05-21 6:12 ` Tomi Ollila
0 siblings, 2 replies; 13+ messages in thread
From: Daniel Kahn Gillmor @ 2019-05-21 1:03 UTC (permalink / raw)
To: Notmuch Mail
This is an empirical fix for a strange interaction we're seeing
between gdb 7.12-6 on debian stretch, GNU timeout, and parallel from
moreutils, reported by Bremner in id:87ef56io5d.fsf@tethera.net.
The problem manifested itself as several concurrent test suite
subprocesses getting stopped by SIGTTOU (including some that don't use
gdb!), which ultimately caused a timeout for the test suite.
This problem itself does not appear to afflict the test suite when
using more modern version of gdb (e.g. 8.2.1-2 in debian buster), and
this fix doesn't cause any problems on newer versions of gdb either,
so it should be safe.
This fix was inspired by a comment from amdragon introduced back in
9ade8160a6a3f6f55996d7a58dc2cd81a6df8395 (now in
tests/T380-atomicity.sh) about an interaction between timeout and gdb.
A marginally cleaner patch be to consistently use gdb with either one
of:
* -tty /dev/null
* < /dev/null
in all of T050, T060, and T070. But T070 can't use "< /dev/null"
because it needs stdin. And when i tried to use "-tty /dev/null" in
T050 and T060, i got test suite failures that i wasn't able to
understand.
So, here is a patch that fixes the hanging problems on the older setup
without understanding them exactly. I welcome any improvements or
actual understanding.
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
---
test/T050-new.sh | 2 +-
test/T060-count.sh | 2 +-
test/T070-insert.sh | 4 ++++
3 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/test/T050-new.sh b/test/T050-new.sh
index dfc8508f..9b3d41df 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -366,7 +366,7 @@ run
EOF
${TEST_GDB} --batch-silent --return-child-result -x notmuch-new-vanish.gdb \
- --args notmuch new 2>OUTPUT 1>/dev/null
+ --args notmuch new 2>OUTPUT 1>/dev/null </dev/null
echo "exit status: $?" >> OUTPUT
# Clean up the file in case gdb isn't available.
diff --git a/test/T060-count.sh b/test/T060-count.sh
index 0c0bf473..a393edb6 100755
--- a/test/T060-count.sh
+++ b/test/T060-count.sh
@@ -116,7 +116,7 @@ EOF
backup_database
test_begin_subtest "error message from query_search_messages"
${TEST_GDB} --batch-silent --return-child-result -x count-files.gdb \
- --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null
+ --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null < /dev/null
cat <<EOF > EXPECTED
notmuch count: A Xapian exception occurred
A Xapian exception occurred performing query
diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index 05be473a..e30f0a43 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -267,12 +267,14 @@ for code in FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR; do
test_begin_subtest "EXIT_FAILURE when index_file returns $code"
test_expect_code 1 \
"${TEST_GDB} --batch-silent --return-child-result \
+ -tty /dev/null \
-ex 'set args insert < $gen_msg_filename' \
-x index-file-$code.gdb notmuch"
test_begin_subtest "success exit with --keep when index_file returns $code"
test_expect_code 0 \
"${TEST_GDB} --batch-silent --return-child-result \
+ -tty /dev/null \
-ex 'set args insert --keep < $gen_msg_filename' \
-x index-file-$code.gdb notmuch"
done
@@ -281,12 +283,14 @@ for code in OUT_OF_MEMORY XAPIAN_EXCEPTION ; do
test_begin_subtest "EX_TEMPFAIL when index_file returns $code"
test_expect_code 75 \
"${TEST_GDB} --batch-silent --return-child-result \
+ -tty /dev/null \
-ex 'set args insert < $gen_msg_filename' \
-x index-file-$code.gdb notmuch"
test_begin_subtest "success exit with --keep when index_file returns $code"
test_expect_code 0 \
"${TEST_GDB} --batch-silent --return-child-result \
+ -tty /dev/null \
-ex 'set args insert --keep < $gen_msg_filename' \
-x index-file-$code.gdb notmuch"
done
--
2.20.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel
2019-05-21 1:03 ` [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel Daniel Kahn Gillmor
@ 2019-05-21 2:32 ` David Bremner
2019-05-21 15:45 ` Daniel Kahn Gillmor
2019-05-21 6:12 ` Tomi Ollila
1 sibling, 1 reply; 13+ messages in thread
From: David Bremner @ 2019-05-21 2:32 UTC (permalink / raw)
To: Daniel Kahn Gillmor, Notmuch Mail
Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:
>
> A marginally cleaner patch be to consistently use gdb with either one
> of:
>
> * -tty /dev/null
> * < /dev/null
>
> in all of T050, T060, and T070. But T070 can't use "< /dev/null"
> because it needs stdin. And when i tried to use "-tty /dev/null" in
> T050 and T060, i got test suite failures that i wasn't able to
> understand.
>
I don't have any understanding to offer, but I wonder if most/all of our
use of gdb could be replaced by writing simple shim libraries and using
LD_PRELOAD. I did a quick proof of concept (at end) for one of the tests in
T070-insert.sh. This would require switching at least some of the test
to use the notmuch-shared binary.
I don't know if relying on LD_PRELOAD is particularly portable. Various
linux distros should be OK, and a quick look at the FreeBSD docs suggest
it might be ok. MacOS would probably need some extra work, as
usual. OTOH, I'm not sure how well (if at all) the gdb based tests work
on MacOS.
In practice the following would presumably be generated by the test
script.
/* gcc -Wall -O2 -fpic -shared -ldl -o shim.so shim.c */
/* LD_PRELOAD=../shim.sh ../../notmuch-shared --config=./notmuch-config insert < foo */
#include "notmuch.h"
#include <stdio.h>
notmuch_status_t
notmuch_database_index_file (notmuch_database_t *notmuch,
const char *filename,
notmuch_indexopts_t *indexopts,
notmuch_message_t **message_ret)
{
printf("calling notmuch_database_index_file\n");
return NOTMUCH_STATUS_XAPIAN_EXCEPTION;
}
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel
2019-05-21 1:03 ` [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel Daniel Kahn Gillmor
2019-05-21 2:32 ` David Bremner
@ 2019-05-21 6:12 ` Tomi Ollila
2019-05-21 15:50 ` Daniel Kahn Gillmor
1 sibling, 1 reply; 13+ messages in thread
From: Tomi Ollila @ 2019-05-21 6:12 UTC (permalink / raw)
To: Daniel Kahn Gillmor, Notmuch Mail
On Mon, May 20 2019, Daniel Kahn Gillmor wrote:
>
> A marginally cleaner patch be to consistently use gdb with either one
> of:
>
> * -tty /dev/null
> * < /dev/null
This looks like a good read:
https://ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_21.html
and is used to speculate below.
I also run the following commands:
trace gdb --batch
strace -ox gdb --batch
strace -oy gdb --batch </dev/null
strace -oz gdb --batch --tty=/dev/null
strace -of gdb --batch </dev/null >/dev/null
strace -og gdb --batch </dev/null >/dev/null 2>/dev/null
and then looked for ioctl() calls in created output file.
the latest (-og) was the only one where gdb(1) was not able
to mess with tty (i.e. even gdb had tty it did not try to
find it on its fd's or open("/dev/tty")...
when run in parallel(1), all the other have possibility to
mess w/ tty *at the same time* if winds are favorable -- but
that means that
1) process had controlling tty before running parallel
2) processes parallel run had controlling tty (same tty)
I did not check this.
> in all of T050, T060, and T070. But T070 can't use "< /dev/null"
> because it needs stdin. And when i tried to use "-tty /dev/null" in
> T050 and T060, i got test suite failures that i wasn't able to
> understand.
In T050 and T060, IIUC the above link, -tty /dev/null makes the inferior
process have /dev/null in its stdin, stdout and stderr -- so there is
no output (I tested T050, not even looked T060).
It is strange that T070 does not work. < /dev/null should only affect
the gdb(1) itself, and not the inside redirection -- but would have to
dig deeper to understand and just now I don't have time...
... anyway all stdin, stdout and stderr should be redirected if --tty
option is not used (or even then, look above), to avoid potential
mess w/ controlling tty possibly shared with other processes...
Tomi
>
> So, here is a patch that fixes the hanging problems on the older setup
> without understanding them exactly. I welcome any improvements or
> actual understanding.
>
> Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
> ---
> test/T050-new.sh | 2 +-
> test/T060-count.sh | 2 +-
> test/T070-insert.sh | 4 ++++
> 3 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/test/T050-new.sh b/test/T050-new.sh
> index dfc8508f..9b3d41df 100755
> --- a/test/T050-new.sh
> +++ b/test/T050-new.sh
> @@ -366,7 +366,7 @@ run
> EOF
>
> ${TEST_GDB} --batch-silent --return-child-result -x notmuch-new-vanish.gdb \
> - --args notmuch new 2>OUTPUT 1>/dev/null
> + --args notmuch new 2>OUTPUT 1>/dev/null </dev/null
> echo "exit status: $?" >> OUTPUT
>
> # Clean up the file in case gdb isn't available.
> diff --git a/test/T060-count.sh b/test/T060-count.sh
> index 0c0bf473..a393edb6 100755
> --- a/test/T060-count.sh
> +++ b/test/T060-count.sh
> @@ -116,7 +116,7 @@ EOF
> backup_database
> test_begin_subtest "error message from query_search_messages"
> ${TEST_GDB} --batch-silent --return-child-result -x count-files.gdb \
> - --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null
> + --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null < /dev/null
> cat <<EOF > EXPECTED
> notmuch count: A Xapian exception occurred
> A Xapian exception occurred performing query
> diff --git a/test/T070-insert.sh b/test/T070-insert.sh
> index 05be473a..e30f0a43 100755
> --- a/test/T070-insert.sh
> +++ b/test/T070-insert.sh
> @@ -267,12 +267,14 @@ for code in FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR; do
> test_begin_subtest "EXIT_FAILURE when index_file returns $code"
> test_expect_code 1 \
> "${TEST_GDB} --batch-silent --return-child-result \
> + -tty /dev/null \
> -ex 'set args insert < $gen_msg_filename' \
> -x index-file-$code.gdb notmuch"
>
> test_begin_subtest "success exit with --keep when index_file returns $code"
> test_expect_code 0 \
> "${TEST_GDB} --batch-silent --return-child-result \
> + -tty /dev/null \
> -ex 'set args insert --keep < $gen_msg_filename' \
> -x index-file-$code.gdb notmuch"
> done
> @@ -281,12 +283,14 @@ for code in OUT_OF_MEMORY XAPIAN_EXCEPTION ; do
> test_begin_subtest "EX_TEMPFAIL when index_file returns $code"
> test_expect_code 75 \
> "${TEST_GDB} --batch-silent --return-child-result \
> + -tty /dev/null \
> -ex 'set args insert < $gen_msg_filename' \
> -x index-file-$code.gdb notmuch"
>
> test_begin_subtest "success exit with --keep when index_file returns $code"
> test_expect_code 0 \
> "${TEST_GDB} --batch-silent --return-child-result \
> + -tty /dev/null \
> -ex 'set args insert --keep < $gen_msg_filename' \
> -x index-file-$code.gdb notmuch"
> done
> --
> 2.20.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel
2019-05-21 6:12 ` Tomi Ollila
@ 2019-05-21 15:50 ` Daniel Kahn Gillmor
2019-05-21 20:23 ` Tomi Ollila
0 siblings, 1 reply; 13+ messages in thread
From: Daniel Kahn Gillmor @ 2019-05-21 15:50 UTC (permalink / raw)
To: Tomi Ollila, Notmuch Mail
[-- Attachment #1: Type: text/plain, Size: 943 bytes --]
Hi Tomi--
On Tue 2019-05-21 09:12:12 +0300, Tomi Ollila wrote:
> This looks like a good read:
> https://ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_21.html
> and is used to speculate below.
Thanks for this pointer, and for the additional analysis.
While i understand this problem much better than i did two days ago, i'm
not sure whether there is an alternate proposal here for how to handle
the situation. Do you have a suggestion for some other fix?
It's tempting to remove the "controlling terminal" from the test suite
entirely (e.g. by running it under "setsid --wait"), but i'm reluctant
to do so because that is not the context in which most instances of
notmuch will themselves be run, and i don't really want the test suite
to be so alienated.
So i'm unsure what to do, but i do want to try to move on beyond all
this "testing the test suite" and start getting more e-mail-related
features into notmuch again :P
--dkg
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel
2019-05-21 15:50 ` Daniel Kahn Gillmor
@ 2019-05-21 20:23 ` Tomi Ollila
0 siblings, 0 replies; 13+ messages in thread
From: Tomi Ollila @ 2019-05-21 20:23 UTC (permalink / raw)
To: Daniel Kahn Gillmor, Notmuch Mail
On Tue, May 21 2019, Daniel Kahn Gillmor wrote:
> Hi Tomi--
>
> On Tue 2019-05-21 09:12:12 +0300, Tomi Ollila wrote:
>> This looks like a good read:
>> https://ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_21.html
>> and is used to speculate below.
>
> Thanks for this pointer, and for the additional analysis.
>
> While i understand this problem much better than i did two days ago, i'm
> not sure whether there is an alternate proposal here for how to handle
> the situation. Do you have a suggestion for some other fix?
I hope id:20190521201702.28754-1-tomi.ollila@iki.fi works
Tomi
>
> It's tempting to remove the "controlling terminal" from the test suite
> entirely (e.g. by running it under "setsid --wait"), but i'm reluctant
> to do so because that is not the context in which most instances of
> notmuch will themselves be run, and i don't really want the test suite
> to be so alienated.
Every test script itself, if not already setsid(1)ed would need to
re-exec itself w/ that, then.
(as I can now run e.g. ./T380-atomicity.sh directly from command line)
>
> So i'm unsure what to do, but i do want to try to move on beyond all
> this "testing the test suite" and start getting more e-mail-related
> features into notmuch again :P
>
> --dkg
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: parallel tests broken on Debian stable
2019-05-10 10:45 parallel tests broken on Debian stable David Bremner
2019-05-20 17:27 ` Daniel Kahn Gillmor
@ 2021-10-30 19:30 ` David Bremner
1 sibling, 0 replies; 13+ messages in thread
From: David Bremner @ 2021-10-30 19:30 UTC (permalink / raw)
To: notmuch; +Cc: Daniel Kahn Gillmor
David Bremner <david@tethera.net> writes:
> In a debian stretch neither gnu parallel nor moreutils parallel
> successfully completes the tests. Logs are attached. One thing worth
> investigating is that gnu parallel inside timeout doesn't really work
> right, since it says something like
I can't figure out if this bug is reproducible in current Debian, so I'm
going to untag it. If someone finds a reproducer, feel free to tag that
as a bug.
d
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2021-10-30 19:30 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-05-10 10:45 parallel tests broken on Debian stable David Bremner
2019-05-20 17:27 ` Daniel Kahn Gillmor
2019-05-20 18:55 ` Tomi Ollila
2019-05-20 21:45 ` Daniel Kahn Gillmor
2019-05-20 23:49 ` Daniel Kahn Gillmor
2019-05-21 0:44 ` Daniel Kahn Gillmor
2019-05-21 1:03 ` [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel Daniel Kahn Gillmor
2019-05-21 2:32 ` David Bremner
2019-05-21 15:45 ` Daniel Kahn Gillmor
2019-05-21 6:12 ` Tomi Ollila
2019-05-21 15:50 ` Daniel Kahn Gillmor
2019-05-21 20:23 ` Tomi Ollila
2021-10-30 19:30 ` parallel tests broken on Debian stable David Bremner
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).