* Notmuch removing messages from the index, when calling `new` two times in a row
@ 2015-04-21 16:09 Mayeu
2015-04-23 3:49 ` David Bremner
0 siblings, 1 reply; 15+ messages in thread
From: Mayeu @ 2015-04-21 16:09 UTC (permalink / raw)
To: notmuch
Hi,
I have hit a weird behaviour with notmuch.
I recently saw a drop of the number of messages in my index, and after
doing a backup of the current state of the index, I am able to reproduce
this behaviour fairly consistently. ie: notmuch is purgin at least 50%
of its index of messages that are still in the maildir.
I do the following:
1. remove my notmuch database completely.
2. launching `notmuch new`:
Found 42059 total files (that's not much mail).
Processed 42059 total files in 3m 24s (205 files/sec.).
Added 41331 new messages to the database.
3. launching `notmuch new` right after, without having done anything else
on my Maildir folder:
Cleaned up 5 of 156 directories (1m 9s remaining).
No new mail. Removed 22211 messages. Detected 378 file renames.
Then I can loop back to step 1 and reproduce the behaviour every single time.
If I keep a window with Notmuch Emacs in it, I can see some e-mail
disapearing, and search not returning result anymore after the second
notmuch new.
Is it a know bug? A feature that I do not understand? Anyway, this make
notmuch almost unusable right now, since I can't have acces to my full
e-mail history, and I endup with conversations with holes in them.
Regards,
--
Mayeu a.k.a Matthieu
http://6x9.fr
GPG: A016 F2D8 0472 1186 1B33 A419 22B1 0496 B00D A693
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row
2015-04-21 16:09 Notmuch removing messages from the index, when calling `new` two times in a row Mayeu
@ 2015-04-23 3:49 ` David Bremner
2015-04-23 12:04 ` Mayeu
0 siblings, 1 reply; 15+ messages in thread
From: David Bremner @ 2015-04-23 3:49 UTC (permalink / raw)
To: Mayeu, notmuch
Mayeu <m@6x9.fr> writes:
> 3. launching `notmuch new` right after, without having done anything else
> on my Maildir folder:
>
> Cleaned up 5 of 156 directories (1m 9s remaining).
> No new mail. Removed 22211 messages. Detected 378 file renames.
This message means that notmuch thinks 156 directories have been
deleted. Do you have any special mounts / symlinks / file systems that
might trigger this?
d
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row
2015-04-23 3:49 ` David Bremner
@ 2015-04-23 12:04 ` Mayeu
2015-04-23 22:25 ` David Bremner
0 siblings, 1 reply; 15+ messages in thread
From: Mayeu @ 2015-04-23 12:04 UTC (permalink / raw)
To: David Bremner, notmuch
[-- Attachment #1: Type: text/plain, Size: 690 bytes --]
David Bremner <david@tethera.net> writes:
> This message means that notmuch thinks 156 directories have been
> deleted. Do you have any special mounts / symlinks / file systems that
> might trigger this?
Thank you for your response, my whole drive uses ZFS and I have a ZFS
volume for my whole /home/*. My Maildir in itself is just a folder in my
~, and do not have anything special.
I have encountered this behaviour also on my previous setup (migrated
last week-end from Archlinux to NixOS), and this one was using ext4, so
it does not seems to be ZFS related.
Regards,
--
Mayeu a.k.a Matthieu
http://6x9.fr
GPG: A016 F2D8 0472 1186 1B33 A419 22B1 0496 B00D A693
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row
2015-04-23 12:04 ` Mayeu
@ 2015-04-23 22:25 ` David Bremner
2015-04-24 9:29 ` Mayeu
0 siblings, 1 reply; 15+ messages in thread
From: David Bremner @ 2015-04-23 22:25 UTC (permalink / raw)
To: Mayeu, notmuch
Mayeu <m@6x9.fr> writes:
> David Bremner <david@tethera.net> writes:
>> This message means that notmuch thinks 156 directories have been
>> deleted. Do you have any special mounts / symlinks / file systems that
>> might trigger this?
>
> Thank you for your response, my whole drive uses ZFS and I have a ZFS
> volume for my whole /home/*. My Maildir in itself is just a folder in my
> ~, and do not have anything special.
>
> I have encountered this behaviour also on my previous setup (migrated
> last week-end from Archlinux to NixOS), and this one was using ext4, so
> it does not seems to be ZFS related.
OK. Please try running "notmuch new" (the second time) with --debug.
d
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row
2015-04-23 22:25 ` David Bremner
@ 2015-04-24 9:29 ` Mayeu
2015-04-26 23:45 ` David Bremner
2015-09-25 18:19 ` Notmuch removing messages from the index, when calling `new` two times in a row Jani Nikula
0 siblings, 2 replies; 15+ messages in thread
From: Mayeu @ 2015-04-24 9:29 UTC (permalink / raw)
Cc: notmuch
On Fri, 24 Apr 2015 07:25:28 +0900
David Bremner <david@tethera.net> wrote:
> OK. Please try running "notmuch new" (the second time) with --debug.
The output is exactly the same, no other information. I also tried to
add --verbose but same output. Both are accepted as valid since I do
net get back any error, but the output is not changed.
Is it something I should enable during the compilation of Notmuch? I
am using 0.19, and from what I see, the notmuch configuration used is
the default one on Nixos[1]. But when running ./configure --help I do
not see any --{enable,disable}-debug option.
[1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/mailreaders/notmuch/default.nix
--
Mayeu a.k.a Matthieu
http://6x9.fr
GPG: A016 F2D8 0472 1186 1B33 A419 22B1 0496 B00D A693
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row
2015-04-24 9:29 ` Mayeu
@ 2015-04-26 23:45 ` David Bremner
2015-08-02 9:04 ` [PATCH v2] cli/new: add more debugging output David Bremner
2015-09-25 18:19 ` Notmuch removing messages from the index, when calling `new` two times in a row Jani Nikula
1 sibling, 1 reply; 15+ messages in thread
From: David Bremner @ 2015-04-26 23:45 UTC (permalink / raw)
To: Mayeu; +Cc: notmuch
[-- Attachment #1: Type: text/plain, Size: 599 bytes --]
Mayeu <ml+notmuch@6x9.fr> writes:
> On Fri, 24 Apr 2015 07:25:28 +0900
> David Bremner <david@tethera.net> wrote:
>> OK. Please try running "notmuch new" (the second time) with --debug.
>
> The output is exactly the same, no other information. I also tried to
> add --verbose but same output. Both are accepted as valid since I do
> net get back any error, but the output is not changed.
>
I think there is just not enough debugging output in this part of the
code. If you can compile from source, please try the attached patch
(it's against master, but I guess it should apply to 0.19 as well).
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-cli-new-add-more-debugging-output.patch --]
[-- Type: text/x-diff, Size: 1989 bytes --]
From 52975ecfdc754863e9ac9b465257a566845377c3 Mon Sep 17 00:00:00 2001
From: David Bremner <david@tethera.net>
Date: Mon, 27 Apr 2015 08:42:36 +0900
Subject: [PATCH] cli/new: add more debugging output
Try to narrow down what part of the code adds files and directories to
the queue(s) to be deleted.
---
notmuch-new.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/notmuch-new.c b/notmuch-new.c
index e6c283e..39b0b9a 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -528,6 +528,10 @@ add_files (notmuch_database_t *notmuch,
"%s/%s", path,
notmuch_filenames_get (db_files));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 2: queuing passed file %s for deletion\n",
+ absolute);
+
_filename_list_add (state->removed_files, absolute);
notmuch_filenames_move_to_next (db_files);
@@ -542,6 +546,9 @@ add_files (notmuch_database_t *notmuch,
{
char *absolute = talloc_asprintf (state->removed_directories,
"%s/%s", path, filename);
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 2: queuing passed directory %s for deletion\n",
+ absolute);
_filename_list_add (state->removed_directories, absolute);
}
@@ -610,6 +617,9 @@ add_files (notmuch_database_t *notmuch,
char *absolute = talloc_asprintf (state->removed_files,
"%s/%s", path,
notmuch_filenames_get (db_files));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 3: queuing leftover file %s for deletion\n",
+ absolute);
_filename_list_add (state->removed_files, absolute);
@@ -622,6 +632,10 @@ add_files (notmuch_database_t *notmuch,
"%s/%s", path,
notmuch_filenames_get (db_subdirs));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 3: queuing leftover directory %s for deletion\n",
+ absolute);
+
_filename_list_add (state->removed_directories, absolute);
notmuch_filenames_move_to_next (db_subdirs);
--
2.1.4
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v2] cli/new: add more debugging output
2015-04-26 23:45 ` David Bremner
@ 2015-08-02 9:04 ` David Bremner
2015-08-02 9:18 ` Tomi Ollila
0 siblings, 1 reply; 15+ messages in thread
From: David Bremner @ 2015-08-02 9:04 UTC (permalink / raw)
To: David Bremner; +Cc: notmuch
Try to narrow down what part of the code adds files and directories to
the queue(s) to be deleted.
---
- print slightly less scary messages
- use git-send-email to send
notmuch-new.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/notmuch-new.c b/notmuch-new.c
index 8ff1ade..ee786a3 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -528,6 +528,10 @@ add_files (notmuch_database_t *notmuch,
"%s/%s", path,
notmuch_filenames_get (db_files));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 2: queuing passed file %s for deletion from database\n",
+ absolute);
+
_filename_list_add (state->removed_files, absolute);
notmuch_filenames_move_to_next (db_files);
@@ -542,6 +546,9 @@ add_files (notmuch_database_t *notmuch,
{
char *absolute = talloc_asprintf (state->removed_directories,
"%s/%s", path, filename);
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 2: queuing passed directory %s for deletion from database\n",
+ absolute);
_filename_list_add (state->removed_directories, absolute);
}
@@ -610,6 +617,9 @@ add_files (notmuch_database_t *notmuch,
char *absolute = talloc_asprintf (state->removed_files,
"%s/%s", path,
notmuch_filenames_get (db_files));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 3: queuing leftover file %s for deletion from database\n",
+ absolute);
_filename_list_add (state->removed_files, absolute);
@@ -622,6 +632,10 @@ add_files (notmuch_database_t *notmuch,
"%s/%s", path,
notmuch_filenames_get (db_subdirs));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 3: queuing leftover directory %s for deletion from database\n",
+ absolute);
+
_filename_list_add (state->removed_directories, absolute);
notmuch_filenames_move_to_next (db_subdirs);
--
2.1.4
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2] cli/new: add more debugging output
2015-08-02 9:04 ` [PATCH v2] cli/new: add more debugging output David Bremner
@ 2015-08-02 9:18 ` Tomi Ollila
2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner
0 siblings, 1 reply; 15+ messages in thread
From: Tomi Ollila @ 2015-08-02 9:18 UTC (permalink / raw)
To: David Bremner; +Cc: notmuch
On Sun, Aug 02 2015, David Bremner <david@tethera.net> wrote:
> Try to narrow down what part of the code adds files and directories to
> the queue(s) to be deleted.
> ---
Looks trivial enough to me. +1
Tomi
>
> - print slightly less scary messages
> - use git-send-email to send
>
> notmuch-new.c | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/notmuch-new.c b/notmuch-new.c
> index 8ff1ade..ee786a3 100644
> --- a/notmuch-new.c
> +++ b/notmuch-new.c
> @@ -528,6 +528,10 @@ add_files (notmuch_database_t *notmuch,
> "%s/%s", path,
> notmuch_filenames_get (db_files));
>
> + if (state->debug)
> + printf ("(D) add_files_recursive, pass 2: queuing passed file %s for deletion from database\n",
> + absolute);
> +
> _filename_list_add (state->removed_files, absolute);
>
> notmuch_filenames_move_to_next (db_files);
> @@ -542,6 +546,9 @@ add_files (notmuch_database_t *notmuch,
> {
> char *absolute = talloc_asprintf (state->removed_directories,
> "%s/%s", path, filename);
> + if (state->debug)
> + printf ("(D) add_files_recursive, pass 2: queuing passed directory %s for deletion from database\n",
> + absolute);
>
> _filename_list_add (state->removed_directories, absolute);
> }
> @@ -610,6 +617,9 @@ add_files (notmuch_database_t *notmuch,
> char *absolute = talloc_asprintf (state->removed_files,
> "%s/%s", path,
> notmuch_filenames_get (db_files));
> + if (state->debug)
> + printf ("(D) add_files_recursive, pass 3: queuing leftover file %s for deletion from database\n",
> + absolute);
>
> _filename_list_add (state->removed_files, absolute);
>
> @@ -622,6 +632,10 @@ add_files (notmuch_database_t *notmuch,
> "%s/%s", path,
> notmuch_filenames_get (db_subdirs));
>
> + if (state->debug)
> + printf ("(D) add_files_recursive, pass 3: queuing leftover directory %s for deletion from database\n",
> + absolute);
> +
> _filename_list_add (state->removed_directories, absolute);
>
> notmuch_filenames_move_to_next (db_subdirs);
> --
> 2.1.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
^ permalink raw reply [flat|nested] 15+ messages in thread
* test suite: add more debugging output for notmuch-new
2015-08-02 9:18 ` Tomi Ollila
@ 2015-08-04 6:48 ` David Bremner
2015-08-04 6:48 ` [PATCH 1/3] test: don't claim single message mbox support is going away David Bremner
` (3 more replies)
0 siblings, 4 replies; 15+ messages in thread
From: David Bremner @ 2015-08-04 6:48 UTC (permalink / raw)
To: Tomi Ollila, David Bremner; +Cc: notmuch
The first patch in the series is a trivial cleanup.
I _thought_ (and tricked Tomi ;)) that the next one was trivial too,
but now I'm not so sure.
First, I forgot to update one test, but I mostly understand (or can at
least rationalize) that output. Then I had the bright idea that I
should add debugging output to other tests in T050-new. In 5 cases I
can't really understand the results. It seems like it takes several
runs of notmuch-new to remove directory entries. Maybe this is
intended, but I couldn't figure that out in a quick look.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 1/3] test: don't claim single message mbox support is going away
2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner
@ 2015-08-04 6:48 ` David Bremner
2015-08-04 6:48 ` [PATCH 2/3] cli/new: add more debugging output David Bremner
` (2 subsequent siblings)
3 siblings, 0 replies; 15+ messages in thread
From: David Bremner @ 2015-08-04 6:48 UTC (permalink / raw)
To: Tomi Ollila, David Bremner; +Cc: notmuch
We gave up on this.
---
test/T050-new.sh | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/test/T050-new.sh b/test/T050-new.sh
index e6c3291..66ea10f 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -63,15 +63,18 @@ test_begin_subtest "Renamed message"
generate_message
notmuch new > /dev/null
mv "$gen_msg_filename" "${gen_msg_filename}"-renamed
-output=$(NOTMUCH_NEW)
-test_expect_equal "$output" "No new mail. Detected 1 file rename."
+output=$(NOTMUCH_NEW --debug)
+test_expect_equal "$output" "(D) add_files_recursive, pass 2: queuing passed file ${gen_msg_filename} for deletion from database
+No new mail. Detected 1 file rename."
test_begin_subtest "Deleted message"
rm "${gen_msg_filename}"-renamed
-output=$(NOTMUCH_NEW)
-test_expect_equal "$output" "No new mail. Removed 1 message."
+output=$(NOTMUCH_NEW --debug)
+test_expect_equal "$output" "(D) add_files_recursive, pass 3: queuing leftover file ${gen_msg_filename}-renamed for deletion from database
+No new mail. Removed 1 message."
+
test_begin_subtest "Renamed directory"
@@ -163,7 +166,7 @@ rm -rf "${MAIL_DIR}"/two
output=$(NOTMUCH_NEW)
test_expect_equal "$output" "No new mail. Removed 3 messages."
-test_begin_subtest "Support single-message mbox (deprecated)"
+test_begin_subtest "Support single-message mbox"
cat > "${MAIL_DIR}"/mbox_file1 <<EOF
From test_suite@notmuchmail.org Fri Jan 5 15:43:57 2001
From: Notmuch Test Suite <test_suite@notmuchmail.org>
--
2.1.4
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/3] cli/new: add more debugging output
2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner
2015-08-04 6:48 ` [PATCH 1/3] test: don't claim single message mbox support is going away David Bremner
@ 2015-08-04 6:48 ` David Bremner
2015-08-04 6:48 ` [PATCH 3/3] test: add debugging output to notmuch-new tests, mark 5 as broken David Bremner
2015-08-04 8:25 ` test suite: add more debugging output for notmuch-new Tomi Ollila
3 siblings, 0 replies; 15+ messages in thread
From: David Bremner @ 2015-08-04 6:48 UTC (permalink / raw)
To: Tomi Ollila, David Bremner; +Cc: notmuch
Try to narrow down what part of the code adds files and directories to
the queue(s) to be deleted.
Update one test. The output is slightly confusing, but I believe it is
correct, resulting from a directory being discovered but containing only ignored files.
---
notmuch-new.c | 14 ++++++++++++++
test/T050-new.sh | 1 +
2 files changed, 15 insertions(+)
diff --git a/notmuch-new.c b/notmuch-new.c
index 8ff1ade..ee786a3 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -528,6 +528,10 @@ add_files (notmuch_database_t *notmuch,
"%s/%s", path,
notmuch_filenames_get (db_files));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 2: queuing passed file %s for deletion from database\n",
+ absolute);
+
_filename_list_add (state->removed_files, absolute);
notmuch_filenames_move_to_next (db_files);
@@ -542,6 +546,9 @@ add_files (notmuch_database_t *notmuch,
{
char *absolute = talloc_asprintf (state->removed_directories,
"%s/%s", path, filename);
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 2: queuing passed directory %s for deletion from database\n",
+ absolute);
_filename_list_add (state->removed_directories, absolute);
}
@@ -610,6 +617,9 @@ add_files (notmuch_database_t *notmuch,
char *absolute = talloc_asprintf (state->removed_files,
"%s/%s", path,
notmuch_filenames_get (db_files));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 3: queuing leftover file %s for deletion from database\n",
+ absolute);
_filename_list_add (state->removed_files, absolute);
@@ -622,6 +632,10 @@ add_files (notmuch_database_t *notmuch,
"%s/%s", path,
notmuch_filenames_get (db_subdirs));
+ if (state->debug)
+ printf ("(D) add_files_recursive, pass 3: queuing leftover directory %s for deletion from database\n",
+ absolute);
+
_filename_list_add (state->removed_directories, absolute);
notmuch_filenames_move_to_next (db_subdirs);
diff --git a/test/T050-new.sh b/test/T050-new.sh
index 66ea10f..ab15ca1 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -238,6 +238,7 @@ test_expect_equal "$output" \
(D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/ignored_file
(D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/three/.git
(D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/three/ignored_file
+(D) add_files_recursive, pass 3: queuing leftover directory ${MAIL_DIR}/two for deletion from database
No new mail."
--
2.1.4
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 3/3] test: add debugging output to notmuch-new tests, mark 5 as broken
2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner
2015-08-04 6:48 ` [PATCH 1/3] test: don't claim single message mbox support is going away David Bremner
2015-08-04 6:48 ` [PATCH 2/3] cli/new: add more debugging output David Bremner
@ 2015-08-04 6:48 ` David Bremner
2015-08-04 8:25 ` test suite: add more debugging output for notmuch-new Tomi Ollila
3 siblings, 0 replies; 15+ messages in thread
From: David Bremner @ 2015-08-04 6:48 UTC (permalink / raw)
To: Tomi Ollila, David Bremner; +Cc: notmuch
I marked the tests where I really couldn't understand the output as
broken. It could also be that I don't understand how directory removal
is supposed to work.
---
test/T050-new.sh | 62 ++++++++++++++++++++++++++++++++------------------------
1 file changed, 35 insertions(+), 27 deletions(-)
diff --git a/test/T050-new.sh b/test/T050-new.sh
index ab15ca1..ff1c354 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -3,25 +3,25 @@ test_description='"notmuch new" in several variations'
. ./test-lib.sh
test_begin_subtest "No new messages"
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "No new mail."
test_begin_subtest "Single new message"
generate_message
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 1 new message to the database."
test_begin_subtest "Multiple new messages"
generate_message
generate_message
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 2 new messages to the database."
test_begin_subtest "No new messages (non-empty DB)"
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "No new mail."
@@ -31,7 +31,7 @@ mkdir "${MAIL_DIR}"/def
mkdir "${MAIL_DIR}"/ghi
generate_message [dir]=def
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 1 new message to the database."
@@ -42,7 +42,7 @@ mv "${MAIL_DIR}"/ghi "${MAIL_DIR}"/abc
rm "${MAIL_DIR}"/def/*
generate_message [dir]=abc
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 1 new message to the database."
@@ -54,7 +54,7 @@ mkdir -p "$(dirname "$tmp_msg_filename")"
mv "$gen_msg_filename" "$tmp_msg_filename"
notmuch new > /dev/null
mv "$tmp_msg_filename" "$gen_msg_filename"
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 1 new message to the database."
@@ -87,34 +87,39 @@ notmuch new > /dev/null
mv "${MAIL_DIR}"/dir "${MAIL_DIR}"/dir-renamed
-output=$(NOTMUCH_NEW)
-test_expect_equal "$output" "No new mail. Detected 3 file renames."
+output=$(NOTMUCH_NEW --debug)
+test_expect_equal "$output" "(D) add_files_recursive, pass 2: queuing passed directory ${MAIL_DIR}/dir for deletion from database
+No new mail. Detected 3 file renames."
test_begin_subtest "Deleted directory"
-
+test_subtest_known_broken
rm -rf "${MAIL_DIR}"/dir-renamed
-output=$(NOTMUCH_NEW)
-test_expect_equal "$output" "No new mail. Removed 3 messages."
+output=$(NOTMUCH_NEW --debug)
+test_expect_equal "$output" "(D) add_files_recursive, pass 2: queuing passed directory ${MAIL_DIR}/dir-renamed for deletion from database
+No new mail. Removed 3 messages."
test_begin_subtest "New directory (at end of list)"
+test_subtest_known_broken
generate_message [dir]=zzz
generate_message [dir]=zzz
generate_message [dir]=zzz
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 3 new messages to the database."
test_begin_subtest "Deleted directory (end of list)"
+test_subtest_known_broken
rm -rf "${MAIL_DIR}"/zzz
-output=$(NOTMUCH_NEW)
-test_expect_equal "$output" "No new mail. Removed 3 messages."
+output=$(NOTMUCH_NEW --debug)
+test_expect_equal "$output" "(D) add_files_recursive, pass 3: queuing leftover directory ${MAIL_DIR}/zzz for deletion from database
+No new mail. Removed 3 messages."
test_begin_subtest "New symlink to directory"
@@ -125,7 +130,7 @@ mv "${MAIL_DIR}" "${TMP_DIRECTORY}"/actual_maildir
mkdir "${MAIL_DIR}"
ln -s "${TMP_DIRECTORY}"/actual_maildir "${MAIL_DIR}"/symlink
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 1 new message to the database."
@@ -135,13 +140,13 @@ external_msg_filename="${TMP_DIRECTORY}"/external/"$(basename "$gen_msg_filename
mkdir -p "$(dirname "$external_msg_filename")"
mv "$gen_msg_filename" "$external_msg_filename"
ln -s "$external_msg_filename" "$gen_msg_filename"
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 1 new message to the database."
test_begin_subtest "Broken symlink aborts"
ln -s does-not-exist "${MAIL_DIR}/broken"
-output=$(NOTMUCH_NEW 2>&1)
+output=$(NOTMUCH_NEW --debug 2>&1)
test_expect_equal "$output" \
"Error reading file ${MAIL_DIR}/broken: No such file or directory
Note: A fatal error was encountered: Something went wrong trying to read or write a file
@@ -155,7 +160,7 @@ generate_message [dir]=two/levels
generate_message [dir]=two/levels
generate_message [dir]=two/levels
-output=$(NOTMUCH_NEW)
+output=$(NOTMUCH_NEW --debug)
test_expect_equal "$output" "Added 3 new messages to the database."
@@ -163,10 +168,12 @@ test_begin_subtest "Deleted two-level directory"
rm -rf "${MAIL_DIR}"/two
-output=$(NOTMUCH_NEW)
-test_expect_equal "$output" "No new mail. Removed 3 messages."
+output=$(NOTMUCH_NEW --debug)
+test_expect_equal "$output" "(D) add_files_recursive, pass 3: queuing leftover directory ${MAIL_DIR}/two for deletion from database
+No new mail. Removed 3 messages."
test_begin_subtest "Support single-message mbox"
+test_subtest_known_broken
cat > "${MAIL_DIR}"/mbox_file1 <<EOF
From test_suite@notmuchmail.org Fri Jan 5 15:43:57 2001
From: Notmuch Test Suite <test_suite@notmuchmail.org>
@@ -175,11 +182,12 @@ Subject: Test mbox message 1
Body.
EOF
-output=$(NOTMUCH_NEW 2>&1)
+output=$(NOTMUCH_NEW --debug 2>&1)
test_expect_equal "$output" "Added 1 new message to the database."
# This test requires that notmuch new has been run at least once.
test_begin_subtest "Skip and report non-mail files"
+test_subtest_known_broken
generate_message
mkdir -p "${MAIL_DIR}"/.git && touch "${MAIL_DIR}"/.git/config
touch "${MAIL_DIR}"/ignored_file
@@ -199,7 +207,7 @@ Subject: Test mbox message 2
Body 2.
EOF
-output=$(NOTMUCH_NEW 2>&1)
+output=$(NOTMUCH_NEW --debug 2>&1)
test_expect_equal "$output" \
"Note: Ignoring non-mail file: ${MAIL_DIR}/.git/config
Note: Ignoring non-mail file: ${MAIL_DIR}/.ignored_hidden_file
@@ -267,23 +275,23 @@ OLDCONFIG=$(notmuch config get new.tags)
test_begin_subtest "Empty tags in new.tags are forbidden"
notmuch config set new.tags "foo;;bar"
-output=$(NOTMUCH_NEW 2>&1)
+output=$(NOTMUCH_NEW --debug 2>&1)
test_expect_equal "$output" "Error: tag '' in new.tags: empty tag forbidden"
test_begin_subtest "Tags starting with '-' in new.tags are forbidden"
notmuch config set new.tags "-foo;bar"
-output=$(NOTMUCH_NEW 2>&1)
+output=$(NOTMUCH_NEW --debug 2>&1)
test_expect_equal "$output" "Error: tag '-foo' in new.tags: tag starting with '-' forbidden"
test_expect_code 1 "Invalid tags set exit code" \
- "NOTMUCH_NEW 2>&1"
+ "NOTMUCH_NEW --debug 2>&1"
notmuch config set new.tags $OLDCONFIG
test_begin_subtest "Xapian exception: read only files"
chmod u-w ${MAIL_DIR}/.notmuch/xapian/*.DB
-output=$(NOTMUCH_NEW 2>&1 | sed 's/: .*$//' )
+output=$(NOTMUCH_NEW --debug 2>&1 | sed 's/: .*$//' )
chmod u+w ${MAIL_DIR}/.notmuch/xapian/*.DB
test_expect_equal "$output" "A Xapian exception occurred opening database"
--
2.1.4
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: test suite: add more debugging output for notmuch-new
2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner
` (2 preceding siblings ...)
2015-08-04 6:48 ` [PATCH 3/3] test: add debugging output to notmuch-new tests, mark 5 as broken David Bremner
@ 2015-08-04 8:25 ` Tomi Ollila
2015-08-04 19:12 ` David Bremner
3 siblings, 1 reply; 15+ messages in thread
From: Tomi Ollila @ 2015-08-04 8:25 UTC (permalink / raw)
To: David Bremner, David Bremner; +Cc: notmuch
On Tue, Aug 04 2015, David Bremner <david@tethera.net> wrote:
> The first patch in the series is a trivial cleanup.
>
> I _thought_ (and tricked Tomi ;)) that the next one was trivial too,
> but now I'm not so sure.
>
> First, I forgot to update one test, but I mostly understand (or can at
> least rationalize) that output. Then I had the bright idea that I
> should add debugging output to other tests in T050-new. In 5 cases I
> can't really understand the results. It seems like it takes several
> runs of notmuch-new to remove directory entries. Maybe this is
> intended, but I couldn't figure that out in a quick look.
I got the same test results -- and could not figure out the reason through
a brief look at the notmuch-new.c code.
+1 for the series for now, hopefully we soon find out the resolution to the
known broken tests.
Tomi
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: test suite: add more debugging output for notmuch-new
2015-08-04 8:25 ` test suite: add more debugging output for notmuch-new Tomi Ollila
@ 2015-08-04 19:12 ` David Bremner
0 siblings, 0 replies; 15+ messages in thread
From: David Bremner @ 2015-08-04 19:12 UTC (permalink / raw)
To: Tomi Ollila; +Cc: notmuch
Tomi Ollila <tomi.ollila@iki.fi> writes:
>
> I got the same test results -- and could not figure out the reason through
> a brief look at the notmuch-new.c code.
>
> +1 for the series for now, hopefully we soon find out the resolution to the
> known broken tests.
OK, I pushed it, we'll see what happens. It'd be niced to sort this out
before the next release anyway.
d
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Notmuch removing messages from the index, when calling `new` two times in a row
2015-04-24 9:29 ` Mayeu
2015-04-26 23:45 ` David Bremner
@ 2015-09-25 18:19 ` Jani Nikula
1 sibling, 0 replies; 15+ messages in thread
From: Jani Nikula @ 2015-09-25 18:19 UTC (permalink / raw)
To: Mayeu; +Cc: notmuch
On Fri, 24 Apr 2015, Mayeu <ml+notmuch@6x9.fr> wrote:
> On Fri, 24 Apr 2015 07:25:28 +0900
> David Bremner <david@tethera.net> wrote:
>> OK. Please try running "notmuch new" (the second time) with --debug.
>
> The output is exactly the same, no other information. I also tried to
> add --verbose but same output. Both are accepted as valid since I do
> net get back any error, but the output is not changed.
>
> Is it something I should enable during the compilation of Notmuch? I
> am using 0.19, and from what I see, the notmuch configuration used is
> the default one on Nixos[1]. But when running ./configure --help I do
> not see any --{enable,disable}-debug option.
I don't think we've resolved this bug yet. Do you still see it? Do all
the tests pass if you take current git, and do 'make test'?
BR,
Jani.
>
> [1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/mailreaders/notmuch/default.nix
>
> --
> Mayeu a.k.a Matthieu
> http://6x9.fr
> GPG: A016 F2D8 0472 1186 1B33 A419 22B1 0496 B00D A693
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2015-09-25 18:20 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-04-21 16:09 Notmuch removing messages from the index, when calling `new` two times in a row Mayeu
2015-04-23 3:49 ` David Bremner
2015-04-23 12:04 ` Mayeu
2015-04-23 22:25 ` David Bremner
2015-04-24 9:29 ` Mayeu
2015-04-26 23:45 ` David Bremner
2015-08-02 9:04 ` [PATCH v2] cli/new: add more debugging output David Bremner
2015-08-02 9:18 ` Tomi Ollila
2015-08-04 6:48 ` test suite: add more debugging output for notmuch-new David Bremner
2015-08-04 6:48 ` [PATCH 1/3] test: don't claim single message mbox support is going away David Bremner
2015-08-04 6:48 ` [PATCH 2/3] cli/new: add more debugging output David Bremner
2015-08-04 6:48 ` [PATCH 3/3] test: add debugging output to notmuch-new tests, mark 5 as broken David Bremner
2015-08-04 8:25 ` test suite: add more debugging output for notmuch-new Tomi Ollila
2015-08-04 19:12 ` David Bremner
2015-09-25 18:19 ` Notmuch removing messages from the index, when calling `new` two times in a row Jani Nikula
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).