* Notmuch removing messages from the index, when calling `new` two times in a row @ 2015-04-21 16:09 Mayeu 2015-04-23 3:49 ` David Bremner 0 siblings, 1 reply; 15+ messages in thread From: Mayeu @ 2015-04-21 16:09 UTC (permalink / raw) To: notmuch Hi, I have hit a weird behaviour with notmuch. I recently saw a drop of the number of messages in my index, and after doing a backup of the current state of the index, I am able to reproduce this behaviour fairly consistently. ie: notmuch is purgin at least 50% of its index of messages that are still in the maildir. I do the following: 1. remove my notmuch database completely. 2. launching `notmuch new`: Found 42059 total files (that's not much mail). Processed 42059 total files in 3m 24s (205 files/sec.). Added 41331 new messages to the database. 3. launching `notmuch new` right after, without having done anything else on my Maildir folder: Cleaned up 5 of 156 directories (1m 9s remaining). No new mail. Removed 22211 messages. Detected 378 file renames. Then I can loop back to step 1 and reproduce the behaviour every single time. If I keep a window with Notmuch Emacs in it, I can see some e-mail disapearing, and search not returning result anymore after the second notmuch new. Is it a know bug? A feature that I do not understand? Anyway, this make notmuch almost unusable right now, since I can't have acces to my full e-mail history, and I endup with conversations with holes in them. Regards, -- Mayeu a.k.a Matthieu http://6x9.fr GPG: A016 F2D8 0472 1186 1B33 A419 22B1 0496 B00D A693 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row 2015-04-21 16:09 Notmuch removing messages from the index, when calling `new` two times in a row Mayeu @ 2015-04-23 3:49 ` David Bremner 2015-04-23 12:04 ` Mayeu 0 siblings, 1 reply; 15+ messages in thread From: David Bremner @ 2015-04-23 3:49 UTC (permalink / raw) To: Mayeu, notmuch Mayeu <m@6x9.fr> writes: > 3. launching `notmuch new` right after, without having done anything else > on my Maildir folder: > > Cleaned up 5 of 156 directories (1m 9s remaining). > No new mail. Removed 22211 messages. Detected 378 file renames. This message means that notmuch thinks 156 directories have been deleted. Do you have any special mounts / symlinks / file systems that might trigger this? d ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row 2015-04-23 3:49 ` David Bremner @ 2015-04-23 12:04 ` Mayeu 2015-04-23 22:25 ` David Bremner 0 siblings, 1 reply; 15+ messages in thread From: Mayeu @ 2015-04-23 12:04 UTC (permalink / raw) To: David Bremner, notmuch [-- Attachment #1: Type: text/plain, Size: 690 bytes --] David Bremner <david@tethera.net> writes: > This message means that notmuch thinks 156 directories have been > deleted. Do you have any special mounts / symlinks / file systems that > might trigger this? Thank you for your response, my whole drive uses ZFS and I have a ZFS volume for my whole /home/*. My Maildir in itself is just a folder in my ~, and do not have anything special. I have encountered this behaviour also on my previous setup (migrated last week-end from Archlinux to NixOS), and this one was using ext4, so it does not seems to be ZFS related. Regards, -- Mayeu a.k.a Matthieu http://6x9.fr GPG: A016 F2D8 0472 1186 1B33 A419 22B1 0496 B00D A693 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 818 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row 2015-04-23 12:04 ` Mayeu @ 2015-04-23 22:25 ` David Bremner 2015-04-24 9:29 ` Mayeu 0 siblings, 1 reply; 15+ messages in thread From: David Bremner @ 2015-04-23 22:25 UTC (permalink / raw) To: Mayeu, notmuch Mayeu <m@6x9.fr> writes: > David Bremner <david@tethera.net> writes: >> This message means that notmuch thinks 156 directories have been >> deleted. Do you have any special mounts / symlinks / file systems that >> might trigger this? > > Thank you for your response, my whole drive uses ZFS and I have a ZFS > volume for my whole /home/*. My Maildir in itself is just a folder in my > ~, and do not have anything special. > > I have encountered this behaviour also on my previous setup (migrated > last week-end from Archlinux to NixOS), and this one was using ext4, so > it does not seems to be ZFS related. OK. Please try running "notmuch new" (the second time) with --debug. d ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row 2015-04-23 22:25 ` David Bremner @ 2015-04-24 9:29 ` Mayeu 2015-04-26 23:45 ` David Bremner 2015-09-25 18:19 ` Notmuch removing messages from the index, when calling `new` two times in a row Jani Nikula 0 siblings, 2 replies; 15+ messages in thread From: Mayeu @ 2015-04-24 9:29 UTC (permalink / raw) Cc: notmuch On Fri, 24 Apr 2015 07:25:28 +0900 David Bremner <david@tethera.net> wrote: > OK. Please try running "notmuch new" (the second time) with --debug. The output is exactly the same, no other information. I also tried to add --verbose but same output. Both are accepted as valid since I do net get back any error, but the output is not changed. Is it something I should enable during the compilation of Notmuch? I am using 0.19, and from what I see, the notmuch configuration used is the default one on Nixos[1]. But when running ./configure --help I do not see any --{enable,disable}-debug option. [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/mailreaders/notmuch/default.nix -- Mayeu a.k.a Matthieu http://6x9.fr GPG: A016 F2D8 0472 1186 1B33 A419 22B1 0496 B00D A693 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row 2015-04-24 9:29 ` Mayeu @ 2015-04-26 23:45 ` David Bremner 2015-08-02 9:04 ` [PATCH v2] cli/new: add more debugging output David Bremner 2015-09-25 18:19 ` Notmuch removing messages from the index, when calling `new` two times in a row Jani Nikula 1 sibling, 1 reply; 15+ messages in thread From: David Bremner @ 2015-04-26 23:45 UTC (permalink / raw) To: Mayeu; +Cc: notmuch [-- Attachment #1: Type: text/plain, Size: 599 bytes --] Mayeu <ml+notmuch@6x9.fr> writes: > On Fri, 24 Apr 2015 07:25:28 +0900 > David Bremner <david@tethera.net> wrote: >> OK. Please try running "notmuch new" (the second time) with --debug. > > The output is exactly the same, no other information. I also tried to > add --verbose but same output. Both are accepted as valid since I do > net get back any error, but the output is not changed. > I think there is just not enough debugging output in this part of the code. If you can compile from source, please try the attached patch (it's against master, but I guess it should apply to 0.19 as well). [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-cli-new-add-more-debugging-output.patch --] [-- Type: text/x-diff, Size: 1989 bytes --] From 52975ecfdc754863e9ac9b465257a566845377c3 Mon Sep 17 00:00:00 2001 From: David Bremner <david@tethera.net> Date: Mon, 27 Apr 2015 08:42:36 +0900 Subject: [PATCH] cli/new: add more debugging output Try to narrow down what part of the code adds files and directories to the queue(s) to be deleted. --- notmuch-new.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/notmuch-new.c b/notmuch-new.c index e6c283e..39b0b9a 100644 --- a/notmuch-new.c +++ b/notmuch-new.c @@ -528,6 +528,10 @@ add_files (notmuch_database_t *notmuch, "%s/%s", path, notmuch_filenames_get (db_files)); + if (state->debug) + printf ("(D) add_files_recursive, pass 2: queuing passed file %s for deletion\n", + absolute); + _filename_list_add (state->removed_files, absolute); notmuch_filenames_move_to_next (db_files); @@ -542,6 +546,9 @@ add_files (notmuch_database_t *notmuch, { char *absolute = talloc_asprintf (state->removed_directories, "%s/%s", path, filename); + if (state->debug) + printf ("(D) add_files_recursive, pass 2: queuing passed directory %s for deletion\n", + absolute); _filename_list_add (state->removed_directories, absolute); } @@ -610,6 +617,9 @@ add_files (notmuch_database_t *notmuch, char *absolute = talloc_asprintf (state->removed_files, "%s/%s", path, notmuch_filenames_get (db_files)); + if (state->debug) + printf ("(D) add_files_recursive, pass 3: queuing leftover file %s for deletion\n", + absolute); _filename_list_add (state->removed_files, absolute); @@ -622,6 +632,10 @@ add_files (notmuch_database_t *notmuch, "%s/%s", path, notmuch_filenames_get (db_subdirs)); + if (state->debug) + printf ("(D) add_files_recursive, pass 3: queuing leftover directory %s for deletion\n", + absolute); + _filename_list_add (state->removed_directories, absolute); notmuch_filenames_move_to_next (db_subdirs); -- 2.1.4 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v2] cli/new: add more debugging output 2015-04-26 23:45 ` David Bremner @ 2015-08-02 9:04 ` David Bremner 2015-08-02 9:18 ` Tomi Ollila 0 siblings, 1 reply; 15+ messages in thread From: David Bremner @ 2015-08-02 9:04 UTC (permalink / raw) To: David Bremner; +Cc: notmuch Try to narrow down what part of the code adds files and directories to the queue(s) to be deleted. --- - print slightly less scary messages - use git-send-email to send notmuch-new.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/notmuch-new.c b/notmuch-new.c index 8ff1ade..ee786a3 100644 --- a/notmuch-new.c +++ b/notmuch-new.c @@ -528,6 +528,10 @@ add_files (notmuch_database_t *notmuch, "%s/%s", path, notmuch_filenames_get (db_files)); + if (state->debug) + printf ("(D) add_files_recursive, pass 2: queuing passed file %s for deletion from database\n", + absolute); + _filename_list_add (state->removed_files, absolute); notmuch_filenames_move_to_next (db_files); @@ -542,6 +546,9 @@ add_files (notmuch_database_t *notmuch, { char *absolute = talloc_asprintf (state->removed_directories, "%s/%s", path, filename); + if (state->debug) + printf ("(D) add_files_recursive, pass 2: queuing passed directory %s for deletion from database\n", + absolute); _filename_list_add (state->removed_directories, absolute); } @@ -610,6 +617,9 @@ add_files (notmuch_database_t *notmuch, char *absolute = talloc_asprintf (state->removed_files, "%s/%s", path, notmuch_filenames_get (db_files)); + if (state->debug) + printf ("(D) add_files_recursive, pass 3: queuing leftover file %s for deletion from database\n", + absolute); _filename_list_add (state->removed_files, absolute); @@ -622,6 +632,10 @@ add_files (notmuch_database_t *notmuch, "%s/%s", path, notmuch_filenames_get (db_subdirs)); + if (state->debug) + printf ("(D) add_files_recursive, pass 3: queuing leftover directory %s for deletion from database\n", + absolute); + _filename_list_add (state->removed_directories, absolute); notmuch_filenames_move_to_next (db_subdirs); -- 2.1.4 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2] cli/new: add more debugging output 2015-08-02 9:04 ` [PATCH v2] cli/new: add more debugging output David Bremner @ 2015-08-02 9:18 ` Tomi Ollila 2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner 0 siblings, 1 reply; 15+ messages in thread From: Tomi Ollila @ 2015-08-02 9:18 UTC (permalink / raw) To: David Bremner; +Cc: notmuch On Sun, Aug 02 2015, David Bremner <david@tethera.net> wrote: > Try to narrow down what part of the code adds files and directories to > the queue(s) to be deleted. > --- Looks trivial enough to me. +1 Tomi > > - print slightly less scary messages > - use git-send-email to send > > notmuch-new.c | 14 ++++++++++++++ > 1 file changed, 14 insertions(+) > > diff --git a/notmuch-new.c b/notmuch-new.c > index 8ff1ade..ee786a3 100644 > --- a/notmuch-new.c > +++ b/notmuch-new.c > @@ -528,6 +528,10 @@ add_files (notmuch_database_t *notmuch, > "%s/%s", path, > notmuch_filenames_get (db_files)); > > + if (state->debug) > + printf ("(D) add_files_recursive, pass 2: queuing passed file %s for deletion from database\n", > + absolute); > + > _filename_list_add (state->removed_files, absolute); > > notmuch_filenames_move_to_next (db_files); > @@ -542,6 +546,9 @@ add_files (notmuch_database_t *notmuch, > { > char *absolute = talloc_asprintf (state->removed_directories, > "%s/%s", path, filename); > + if (state->debug) > + printf ("(D) add_files_recursive, pass 2: queuing passed directory %s for deletion from database\n", > + absolute); > > _filename_list_add (state->removed_directories, absolute); > } > @@ -610,6 +617,9 @@ add_files (notmuch_database_t *notmuch, > char *absolute = talloc_asprintf (state->removed_files, > "%s/%s", path, > notmuch_filenames_get (db_files)); > + if (state->debug) > + printf ("(D) add_files_recursive, pass 3: queuing leftover file %s for deletion from database\n", > + absolute); > > _filename_list_add (state->removed_files, absolute); > > @@ -622,6 +632,10 @@ add_files (notmuch_database_t *notmuch, > "%s/%s", path, > notmuch_filenames_get (db_subdirs)); > > + if (state->debug) > + printf ("(D) add_files_recursive, pass 3: queuing leftover directory %s for deletion from database\n", > + absolute); > + > _filename_list_add (state->removed_directories, absolute); > > notmuch_filenames_move_to_next (db_subdirs); > -- > 2.1.4 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ^ permalink raw reply [flat|nested] 15+ messages in thread
* test suite: add more debugging output for notmuch-new 2015-08-02 9:18 ` Tomi Ollila @ 2015-08-04 6:48 ` David Bremner 2015-08-04 6:48 ` [PATCH 1/3] test: don't claim single message mbox support is going away David Bremner ` (3 more replies) 0 siblings, 4 replies; 15+ messages in thread From: David Bremner @ 2015-08-04 6:48 UTC (permalink / raw) To: Tomi Ollila, David Bremner; +Cc: notmuch The first patch in the series is a trivial cleanup. I _thought_ (and tricked Tomi ;)) that the next one was trivial too, but now I'm not so sure. First, I forgot to update one test, but I mostly understand (or can at least rationalize) that output. Then I had the bright idea that I should add debugging output to other tests in T050-new. In 5 cases I can't really understand the results. It seems like it takes several runs of notmuch-new to remove directory entries. Maybe this is intended, but I couldn't figure that out in a quick look. ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 1/3] test: don't claim single message mbox support is going away 2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner @ 2015-08-04 6:48 ` David Bremner 2015-08-04 6:48 ` [PATCH 2/3] cli/new: add more debugging output David Bremner ` (2 subsequent siblings) 3 siblings, 0 replies; 15+ messages in thread From: David Bremner @ 2015-08-04 6:48 UTC (permalink / raw) To: Tomi Ollila, David Bremner; +Cc: notmuch We gave up on this. --- test/T050-new.sh | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/test/T050-new.sh b/test/T050-new.sh index e6c3291..66ea10f 100755 --- a/test/T050-new.sh +++ b/test/T050-new.sh @@ -63,15 +63,18 @@ test_begin_subtest "Renamed message" generate_message notmuch new > /dev/null mv "$gen_msg_filename" "${gen_msg_filename}"-renamed -output=$(NOTMUCH_NEW) -test_expect_equal "$output" "No new mail. Detected 1 file rename." +output=$(NOTMUCH_NEW --debug) +test_expect_equal "$output" "(D) add_files_recursive, pass 2: queuing passed file ${gen_msg_filename} for deletion from database +No new mail. Detected 1 file rename." test_begin_subtest "Deleted message" rm "${gen_msg_filename}"-renamed -output=$(NOTMUCH_NEW) -test_expect_equal "$output" "No new mail. Removed 1 message." +output=$(NOTMUCH_NEW --debug) +test_expect_equal "$output" "(D) add_files_recursive, pass 3: queuing leftover file ${gen_msg_filename}-renamed for deletion from database +No new mail. Removed 1 message." + test_begin_subtest "Renamed directory" @@ -163,7 +166,7 @@ rm -rf "${MAIL_DIR}"/two output=$(NOTMUCH_NEW) test_expect_equal "$output" "No new mail. Removed 3 messages." -test_begin_subtest "Support single-message mbox (deprecated)" +test_begin_subtest "Support single-message mbox" cat > "${MAIL_DIR}"/mbox_file1 <<EOF From test_suite@notmuchmail.org Fri Jan 5 15:43:57 2001 From: Notmuch Test Suite <test_suite@notmuchmail.org> -- 2.1.4 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/3] cli/new: add more debugging output 2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner 2015-08-04 6:48 ` [PATCH 1/3] test: don't claim single message mbox support is going away David Bremner @ 2015-08-04 6:48 ` David Bremner 2015-08-04 6:48 ` [PATCH 3/3] test: add debugging output to notmuch-new tests, mark 5 as broken David Bremner 2015-08-04 8:25 ` test suite: add more debugging output for notmuch-new Tomi Ollila 3 siblings, 0 replies; 15+ messages in thread From: David Bremner @ 2015-08-04 6:48 UTC (permalink / raw) To: Tomi Ollila, David Bremner; +Cc: notmuch Try to narrow down what part of the code adds files and directories to the queue(s) to be deleted. Update one test. The output is slightly confusing, but I believe it is correct, resulting from a directory being discovered but containing only ignored files. --- notmuch-new.c | 14 ++++++++++++++ test/T050-new.sh | 1 + 2 files changed, 15 insertions(+) diff --git a/notmuch-new.c b/notmuch-new.c index 8ff1ade..ee786a3 100644 --- a/notmuch-new.c +++ b/notmuch-new.c @@ -528,6 +528,10 @@ add_files (notmuch_database_t *notmuch, "%s/%s", path, notmuch_filenames_get (db_files)); + if (state->debug) + printf ("(D) add_files_recursive, pass 2: queuing passed file %s for deletion from database\n", + absolute); + _filename_list_add (state->removed_files, absolute); notmuch_filenames_move_to_next (db_files); @@ -542,6 +546,9 @@ add_files (notmuch_database_t *notmuch, { char *absolute = talloc_asprintf (state->removed_directories, "%s/%s", path, filename); + if (state->debug) + printf ("(D) add_files_recursive, pass 2: queuing passed directory %s for deletion from database\n", + absolute); _filename_list_add (state->removed_directories, absolute); } @@ -610,6 +617,9 @@ add_files (notmuch_database_t *notmuch, char *absolute = talloc_asprintf (state->removed_files, "%s/%s", path, notmuch_filenames_get (db_files)); + if (state->debug) + printf ("(D) add_files_recursive, pass 3: queuing leftover file %s for deletion from database\n", + absolute); _filename_list_add (state->removed_files, absolute); @@ -622,6 +632,10 @@ add_files (notmuch_database_t *notmuch, "%s/%s", path, notmuch_filenames_get (db_subdirs)); + if (state->debug) + printf ("(D) add_files_recursive, pass 3: queuing leftover directory %s for deletion from database\n", + absolute); + _filename_list_add (state->removed_directories, absolute); notmuch_filenames_move_to_next (db_subdirs); diff --git a/test/T050-new.sh b/test/T050-new.sh index 66ea10f..ab15ca1 100755 --- a/test/T050-new.sh +++ b/test/T050-new.sh @@ -238,6 +238,7 @@ test_expect_equal "$output" \ (D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/ignored_file (D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/three/.git (D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/three/ignored_file +(D) add_files_recursive, pass 3: queuing leftover directory ${MAIL_DIR}/two for deletion from database No new mail." -- 2.1.4 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 3/3] test: add debugging output to notmuch-new tests, mark 5 as broken 2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner 2015-08-04 6:48 ` [PATCH 1/3] test: don't claim single message mbox support is going away David Bremner 2015-08-04 6:48 ` [PATCH 2/3] cli/new: add more debugging output David Bremner @ 2015-08-04 6:48 ` David Bremner 2015-08-04 8:25 ` test suite: add more debugging output for notmuch-new Tomi Ollila 3 siblings, 0 replies; 15+ messages in thread From: David Bremner @ 2015-08-04 6:48 UTC (permalink / raw) To: Tomi Ollila, David Bremner; +Cc: notmuch I marked the tests where I really couldn't understand the output as broken. It could also be that I don't understand how directory removal is supposed to work. --- test/T050-new.sh | 62 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 35 insertions(+), 27 deletions(-) diff --git a/test/T050-new.sh b/test/T050-new.sh index ab15ca1..ff1c354 100755 --- a/test/T050-new.sh +++ b/test/T050-new.sh @@ -3,25 +3,25 @@ test_description='"notmuch new" in several variations' . ./test-lib.sh test_begin_subtest "No new messages" -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "No new mail." test_begin_subtest "Single new message" generate_message -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 1 new message to the database." test_begin_subtest "Multiple new messages" generate_message generate_message -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 2 new messages to the database." test_begin_subtest "No new messages (non-empty DB)" -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "No new mail." @@ -31,7 +31,7 @@ mkdir "${MAIL_DIR}"/def mkdir "${MAIL_DIR}"/ghi generate_message [dir]=def -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 1 new message to the database." @@ -42,7 +42,7 @@ mv "${MAIL_DIR}"/ghi "${MAIL_DIR}"/abc rm "${MAIL_DIR}"/def/* generate_message [dir]=abc -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 1 new message to the database." @@ -54,7 +54,7 @@ mkdir -p "$(dirname "$tmp_msg_filename")" mv "$gen_msg_filename" "$tmp_msg_filename" notmuch new > /dev/null mv "$tmp_msg_filename" "$gen_msg_filename" -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 1 new message to the database." @@ -87,34 +87,39 @@ notmuch new > /dev/null mv "${MAIL_DIR}"/dir "${MAIL_DIR}"/dir-renamed -output=$(NOTMUCH_NEW) -test_expect_equal "$output" "No new mail. Detected 3 file renames." +output=$(NOTMUCH_NEW --debug) +test_expect_equal "$output" "(D) add_files_recursive, pass 2: queuing passed directory ${MAIL_DIR}/dir for deletion from database +No new mail. Detected 3 file renames." test_begin_subtest "Deleted directory" - +test_subtest_known_broken rm -rf "${MAIL_DIR}"/dir-renamed -output=$(NOTMUCH_NEW) -test_expect_equal "$output" "No new mail. Removed 3 messages." +output=$(NOTMUCH_NEW --debug) +test_expect_equal "$output" "(D) add_files_recursive, pass 2: queuing passed directory ${MAIL_DIR}/dir-renamed for deletion from database +No new mail. Removed 3 messages." test_begin_subtest "New directory (at end of list)" +test_subtest_known_broken generate_message [dir]=zzz generate_message [dir]=zzz generate_message [dir]=zzz -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 3 new messages to the database." test_begin_subtest "Deleted directory (end of list)" +test_subtest_known_broken rm -rf "${MAIL_DIR}"/zzz -output=$(NOTMUCH_NEW) -test_expect_equal "$output" "No new mail. Removed 3 messages." +output=$(NOTMUCH_NEW --debug) +test_expect_equal "$output" "(D) add_files_recursive, pass 3: queuing leftover directory ${MAIL_DIR}/zzz for deletion from database +No new mail. Removed 3 messages." test_begin_subtest "New symlink to directory" @@ -125,7 +130,7 @@ mv "${MAIL_DIR}" "${TMP_DIRECTORY}"/actual_maildir mkdir "${MAIL_DIR}" ln -s "${TMP_DIRECTORY}"/actual_maildir "${MAIL_DIR}"/symlink -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 1 new message to the database." @@ -135,13 +140,13 @@ external_msg_filename="${TMP_DIRECTORY}"/external/"$(basename "$gen_msg_filename mkdir -p "$(dirname "$external_msg_filename")" mv "$gen_msg_filename" "$external_msg_filename" ln -s "$external_msg_filename" "$gen_msg_filename" -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 1 new message to the database." test_begin_subtest "Broken symlink aborts" ln -s does-not-exist "${MAIL_DIR}/broken" -output=$(NOTMUCH_NEW 2>&1) +output=$(NOTMUCH_NEW --debug 2>&1) test_expect_equal "$output" \ "Error reading file ${MAIL_DIR}/broken: No such file or directory Note: A fatal error was encountered: Something went wrong trying to read or write a file @@ -155,7 +160,7 @@ generate_message [dir]=two/levels generate_message [dir]=two/levels generate_message [dir]=two/levels -output=$(NOTMUCH_NEW) +output=$(NOTMUCH_NEW --debug) test_expect_equal "$output" "Added 3 new messages to the database." @@ -163,10 +168,12 @@ test_begin_subtest "Deleted two-level directory" rm -rf "${MAIL_DIR}"/two -output=$(NOTMUCH_NEW) -test_expect_equal "$output" "No new mail. Removed 3 messages." +output=$(NOTMUCH_NEW --debug) +test_expect_equal "$output" "(D) add_files_recursive, pass 3: queuing leftover directory ${MAIL_DIR}/two for deletion from database +No new mail. Removed 3 messages." test_begin_subtest "Support single-message mbox" +test_subtest_known_broken cat > "${MAIL_DIR}"/mbox_file1 <<EOF From test_suite@notmuchmail.org Fri Jan 5 15:43:57 2001 From: Notmuch Test Suite <test_suite@notmuchmail.org> @@ -175,11 +182,12 @@ Subject: Test mbox message 1 Body. EOF -output=$(NOTMUCH_NEW 2>&1) +output=$(NOTMUCH_NEW --debug 2>&1) test_expect_equal "$output" "Added 1 new message to the database." # This test requires that notmuch new has been run at least once. test_begin_subtest "Skip and report non-mail files" +test_subtest_known_broken generate_message mkdir -p "${MAIL_DIR}"/.git && touch "${MAIL_DIR}"/.git/config touch "${MAIL_DIR}"/ignored_file @@ -199,7 +207,7 @@ Subject: Test mbox message 2 Body 2. EOF -output=$(NOTMUCH_NEW 2>&1) +output=$(NOTMUCH_NEW --debug 2>&1) test_expect_equal "$output" \ "Note: Ignoring non-mail file: ${MAIL_DIR}/.git/config Note: Ignoring non-mail file: ${MAIL_DIR}/.ignored_hidden_file @@ -267,23 +275,23 @@ OLDCONFIG=$(notmuch config get new.tags) test_begin_subtest "Empty tags in new.tags are forbidden" notmuch config set new.tags "foo;;bar" -output=$(NOTMUCH_NEW 2>&1) +output=$(NOTMUCH_NEW --debug 2>&1) test_expect_equal "$output" "Error: tag '' in new.tags: empty tag forbidden" test_begin_subtest "Tags starting with '-' in new.tags are forbidden" notmuch config set new.tags "-foo;bar" -output=$(NOTMUCH_NEW 2>&1) +output=$(NOTMUCH_NEW --debug 2>&1) test_expect_equal "$output" "Error: tag '-foo' in new.tags: tag starting with '-' forbidden" test_expect_code 1 "Invalid tags set exit code" \ - "NOTMUCH_NEW 2>&1" + "NOTMUCH_NEW --debug 2>&1" notmuch config set new.tags $OLDCONFIG test_begin_subtest "Xapian exception: read only files" chmod u-w ${MAIL_DIR}/.notmuch/xapian/*.DB -output=$(NOTMUCH_NEW 2>&1 | sed 's/: .*$//' ) +output=$(NOTMUCH_NEW --debug 2>&1 | sed 's/: .*$//' ) chmod u+w ${MAIL_DIR}/.notmuch/xapian/*.DB test_expect_equal "$output" "A Xapian exception occurred opening database" -- 2.1.4 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: test suite: add more debugging output for notmuch-new 2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner ` (2 preceding siblings ...) 2015-08-04 6:48 ` [PATCH 3/3] test: add debugging output to notmuch-new tests, mark 5 as broken David Bremner @ 2015-08-04 8:25 ` Tomi Ollila 2015-08-04 19:12 ` David Bremner 3 siblings, 1 reply; 15+ messages in thread From: Tomi Ollila @ 2015-08-04 8:25 UTC (permalink / raw) To: David Bremner, David Bremner; +Cc: notmuch On Tue, Aug 04 2015, David Bremner <david@tethera.net> wrote: > The first patch in the series is a trivial cleanup. > > I _thought_ (and tricked Tomi ;)) that the next one was trivial too, > but now I'm not so sure. > > First, I forgot to update one test, but I mostly understand (or can at > least rationalize) that output. Then I had the bright idea that I > should add debugging output to other tests in T050-new. In 5 cases I > can't really understand the results. It seems like it takes several > runs of notmuch-new to remove directory entries. Maybe this is > intended, but I couldn't figure that out in a quick look. I got the same test results -- and could not figure out the reason through a brief look at the notmuch-new.c code. +1 for the series for now, hopefully we soon find out the resolution to the known broken tests. Tomi ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: test suite: add more debugging output for notmuch-new 2015-08-04 8:25 ` test suite: add more debugging output for notmuch-new Tomi Ollila @ 2015-08-04 19:12 ` David Bremner 0 siblings, 0 replies; 15+ messages in thread From: David Bremner @ 2015-08-04 19:12 UTC (permalink / raw) To: Tomi Ollila; +Cc: notmuch Tomi Ollila <tomi.ollila@iki.fi> writes: > > I got the same test results -- and could not figure out the reason through > a brief look at the notmuch-new.c code. > > +1 for the series for now, hopefully we soon find out the resolution to the > known broken tests. OK, I pushed it, we'll see what happens. It'd be niced to sort this out before the next release anyway. d ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row 2015-04-24 9:29 ` Mayeu 2015-04-26 23:45 ` David Bremner @ 2015-09-25 18:19 ` Jani Nikula 1 sibling, 0 replies; 15+ messages in thread From: Jani Nikula @ 2015-09-25 18:19 UTC (permalink / raw) To: Mayeu; +Cc: notmuch On Fri, 24 Apr 2015, Mayeu <ml+notmuch@6x9.fr> wrote: > On Fri, 24 Apr 2015 07:25:28 +0900 > David Bremner <david@tethera.net> wrote: >> OK. Please try running "notmuch new" (the second time) with --debug. > > The output is exactly the same, no other information. I also tried to > add --verbose but same output. Both are accepted as valid since I do > net get back any error, but the output is not changed. > > Is it something I should enable during the compilation of Notmuch? I > am using 0.19, and from what I see, the notmuch configuration used is > the default one on Nixos[1]. But when running ./configure --help I do > not see any --{enable,disable}-debug option. I don't think we've resolved this bug yet. Do you still see it? Do all the tests pass if you take current git, and do 'make test'? BR, Jani. > > [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/mailreaders/notmuch/default.nix > > -- > Mayeu a.k.a Matthieu > http://6x9.fr > GPG: A016 F2D8 0472 1186 1B33 A419 22B1 0496 B00D A693 > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2015-09-25 18:20 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-04-21 16:09 Notmuch removing messages from the index, when calling `new` two times in a row Mayeu 2015-04-23 3:49 ` David Bremner 2015-04-23 12:04 ` Mayeu 2015-04-23 22:25 ` David Bremner 2015-04-24 9:29 ` Mayeu 2015-04-26 23:45 ` David Bremner 2015-08-02 9:04 ` [PATCH v2] cli/new: add more debugging output David Bremner 2015-08-02 9:18 ` Tomi Ollila 2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner 2015-08-04 6:48 ` [PATCH 1/3] test: don't claim single message mbox support is going away David Bremner 2015-08-04 6:48 ` [PATCH 2/3] cli/new: add more debugging output David Bremner 2015-08-04 6:48 ` [PATCH 3/3] test: add debugging output to notmuch-new tests, mark 5 as broken David Bremner 2015-08-04 8:25 ` test suite: add more debugging output for notmuch-new Tomi Ollila 2015-08-04 19:12 ` David Bremner 2015-09-25 18:19 ` Notmuch removing messages from the index, when calling `new` two times in a row Jani Nikula
Code repositories for project(s) associated with this public inbox https://yhetil.org/notmuch.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).