* [PATCH] test: test folder renames
2014-02-23 1:08 Weird behaviour in notmuch new Mark Walters
@ 2014-02-23 21:40 ` Jani Nikula
2014-02-23 23:28 ` Tomi Ollila
0 siblings, 1 reply; 5+ messages in thread
From: Jani Nikula @ 2014-02-23 21:40 UTC (permalink / raw)
To: notmuch, Mark Walters
On Sun, 23 Feb 2014, Mark Walters <markwalters1009@gmail.com> wrote:
> I was experimenting with letting notmuch new take an argument to tell it
> to scan only a particular directory (and sub-directories) for new
> messages. I came across the following strange behaviour which is also
> present in master (with a fresh database)
>
> I have a bunch of maildirs in /home/mail: so folders .mail.foo/
> .mail.bar/ each of which has cur/new/tmp and all the messages are in
> cur.
>
> If I do mv .mail.foo .mail.bar/ and run notmuch new I get the expected
> lots of renames (900 or so in the case I was trying). But if I then do
> mv .mail.bar/.mail.foo . and run notmuch new almost all the messages get
> removed (but 30 renames do get detected). If I then do touch .mail.foo/*
> the messages get found again
>
> I am guessing the 30 renames might be because those 30 have duplicates
> somewhere else.
>
> But the other behaviour has me puzzled.
This test reproduces the problem for me, but it's not
deterministic. With the loop, I get roughly one fail per test run:
FAIL Rename folder back
--- T051-new-renames.27.expected 2014-02-23 21:37:10.121774241 +0000
+++ T051-new-renames.27.output 2014-02-23 21:37:10.121774241 +0000
@@ -1 +1 @@
-No new mail. Detected 10 file renames.
+No new mail. Removed 10 messages.
FAIL Files remain the same
--- T051-new-renames.28.expected 2014-02-23 21:37:10.133774652 +0000
+++ T051-new-renames.28.output 2014-02-23 21:37:10.133774652 +0000
@@ -1,13 +1,3 @@
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-121
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-122
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-123
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-124
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-125
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-126
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-127
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-128
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-129
-/path/to/test/tmp.T051-new-renames/mail/foo/msg-130
/path/to/test/tmp.T051-new-renames/mail/bar/msg-131
/path/to/test/tmp.T051-new-renames/mail/bar/msg-132
/path/to/test/tmp.T051-new-renames/mail/bar/msg-133
I'm as puzzled as you are.
BR,
Jani.
---
test/T051-new-renames.sh | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
create mode 100755 test/T051-new-renames.sh
diff --git a/test/T051-new-renames.sh b/test/T051-new-renames.sh
new file mode 100755
index 000000000000..febe006f5888
--- /dev/null
+++ b/test/T051-new-renames.sh
@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+test_description='"notmuch new" with directory renames'
+. ./test-lib.sh
+
+for loop in `seq 10`; do
+
+rm -rf ${MAIL_DIR}
+
+for i in `seq 10`; do
+ generate_message '[dir]=foo' '[subject]="Message foo $i"'
+done
+
+for i in `seq 10`; do
+ generate_message '[dir]=bar' '[subject]="Message bar $i"'
+done
+
+test_begin_subtest "Index the messages, round $loop"
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "Added 20 new messages to the database."
+
+all_files=$(notmuch search --output=files \*)
+count_foo=$(notmuch count folder:foo)
+
+test_begin_subtest "Rename folder"
+mv ${MAIL_DIR}/foo ${MAIL_DIR}/baz
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
+
+test_begin_subtest "Rename folder back"
+mv ${MAIL_DIR}/baz ${MAIL_DIR}/foo
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
+
+test_begin_subtest "Files remain the same"
+output=$(notmuch search --output=files \*)
+test_expect_equal "$output" "$all_files"
+
+done
+
+test_done
--
1.8.5.3
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] test: test folder renames
2014-02-23 21:40 ` [PATCH] test: test folder renames Jani Nikula
@ 2014-02-23 23:28 ` Tomi Ollila
0 siblings, 0 replies; 5+ messages in thread
From: Tomi Ollila @ 2014-02-23 23:28 UTC (permalink / raw)
To: Jani Nikula, notmuch, Mark Walters
On Sun, Feb 23 2014, Jani Nikula <jani@nikula.org> wrote:
> On Sun, 23 Feb 2014, Mark Walters <markwalters1009@gmail.com> wrote:
>> I was experimenting with letting notmuch new take an argument to tell it
>> to scan only a particular directory (and sub-directories) for new
>> messages. I came across the following strange behaviour which is also
>> present in master (with a fresh database)
>>
>> I have a bunch of maildirs in /home/mail: so folders .mail.foo/
>> .mail.bar/ each of which has cur/new/tmp and all the messages are in
>> cur.
>>
>> If I do mv .mail.foo .mail.bar/ and run notmuch new I get the expected
>> lots of renames (900 or so in the case I was trying). But if I then do
>> mv .mail.bar/.mail.foo . and run notmuch new almost all the messages get
>> removed (but 30 renames do get detected). If I then do touch .mail.foo/*
>> the messages get found again
>>
>> I am guessing the 30 renames might be because those 30 have duplicates
>> somewhere else.
>>
>> But the other behaviour has me puzzled.
>
> This test reproduces the problem for me, but it's not
> deterministic. With the loop, I get roughly one fail per test run:
>
> FAIL Rename folder back
> --- T051-new-renames.27.expected 2014-02-23 21:37:10.121774241 +0000
> +++ T051-new-renames.27.output 2014-02-23 21:37:10.121774241 +0000
> @@ -1 +1 @@
> -No new mail. Detected 10 file renames.
> +No new mail. Removed 10 messages.
> FAIL Files remain the same
> --- T051-new-renames.28.expected 2014-02-23 21:37:10.133774652 +0000
> +++ T051-new-renames.28.output 2014-02-23 21:37:10.133774652 +0000
> @@ -1,13 +1,3 @@
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-121
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-122
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-123
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-124
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-125
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-126
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-127
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-128
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-129
> -/path/to/test/tmp.T051-new-renames/mail/foo/msg-130
> /path/to/test/tmp.T051-new-renames/mail/bar/msg-131
> /path/to/test/tmp.T051-new-renames/mail/bar/msg-132
> /path/to/test/tmp.T051-new-renames/mail/bar/msg-133
>
> I'm as puzzled as you are.
find /path/to/test/tmp.T051-new-renames/ might show what is the order
or 'foo' and 'bar' directories there. the order might be arbitrary
-- it surely is not alphabetical and it might not be the order
created...
The order should not matter -- and maybe it didn't and some change
made that matter...
I'd test now but I should be ZZZ :D
>
> BR,
> Jani.
Tomi
> ---
> test/T051-new-renames.sh | 40 ++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 40 insertions(+)
> create mode 100755 test/T051-new-renames.sh
>
> diff --git a/test/T051-new-renames.sh b/test/T051-new-renames.sh
> new file mode 100755
> index 000000000000..febe006f5888
> --- /dev/null
> +++ b/test/T051-new-renames.sh
> @@ -0,0 +1,40 @@
> +#!/usr/bin/env bash
> +test_description='"notmuch new" with directory renames'
> +. ./test-lib.sh
> +
> +for loop in `seq 10`; do
> +
> +rm -rf ${MAIL_DIR}
> +
> +for i in `seq 10`; do
> + generate_message '[dir]=foo' '[subject]="Message foo $i"'
> +done
> +
> +for i in `seq 10`; do
> + generate_message '[dir]=bar' '[subject]="Message bar $i"'
> +done
> +
> +test_begin_subtest "Index the messages, round $loop"
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "Added 20 new messages to the database."
> +
> +all_files=$(notmuch search --output=files \*)
> +count_foo=$(notmuch count folder:foo)
> +
> +test_begin_subtest "Rename folder"
> +mv ${MAIL_DIR}/foo ${MAIL_DIR}/baz
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
> +
> +test_begin_subtest "Rename folder back"
> +mv ${MAIL_DIR}/baz ${MAIL_DIR}/foo
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
> +
> +test_begin_subtest "Files remain the same"
> +output=$(notmuch search --output=files \*)
> +test_expect_equal "$output" "$all_files"
> +
> +done
> +
> +test_done
> --
> 1.8.5.3
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH] test: test folder renames
@ 2021-12-26 11:21 David Bremner
2021-12-26 15:09 ` Tomi Ollila
0 siblings, 1 reply; 5+ messages in thread
From: David Bremner @ 2021-12-26 11:21 UTC (permalink / raw)
To: notmuch
From: Jani Nikula <jani@nikula.org>
In [1] Mark Walters reported a problem with messages being removed
from the database when the parent directory was renamed. Jani Nikula
proposed [2] these tests but observed
This test is not suitable for merging since it's not deterministic.
After applying Jani's patch [3], the tests now pass deterministically,
and could usefully act as regression tests.
[1]: id:87siray6th.fsf@qmul.ac.uk
[2]: id:1393191650-28333-1-git-send-email-jani@nikula.org
[3]: id:1441445731-4362-2-git-send-email-jani@nikula.org
---
test/T051-new-renames.sh | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
create mode 100755 test/T051-new-renames.sh
diff --git a/test/T051-new-renames.sh b/test/T051-new-renames.sh
new file mode 100755
index 00000000..febe006f
--- /dev/null
+++ b/test/T051-new-renames.sh
@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+test_description='"notmuch new" with directory renames'
+. ./test-lib.sh
+
+for loop in `seq 10`; do
+
+rm -rf ${MAIL_DIR}
+
+for i in `seq 10`; do
+ generate_message '[dir]=foo' '[subject]="Message foo $i"'
+done
+
+for i in `seq 10`; do
+ generate_message '[dir]=bar' '[subject]="Message bar $i"'
+done
+
+test_begin_subtest "Index the messages, round $loop"
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "Added 20 new messages to the database."
+
+all_files=$(notmuch search --output=files \*)
+count_foo=$(notmuch count folder:foo)
+
+test_begin_subtest "Rename folder"
+mv ${MAIL_DIR}/foo ${MAIL_DIR}/baz
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
+
+test_begin_subtest "Rename folder back"
+mv ${MAIL_DIR}/baz ${MAIL_DIR}/foo
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
+
+test_begin_subtest "Files remain the same"
+output=$(notmuch search --output=files \*)
+test_expect_equal "$output" "$all_files"
+
+done
+
+test_done
--
2.34.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] test: test folder renames
2021-12-26 11:21 [PATCH] test: test folder renames David Bremner
@ 2021-12-26 15:09 ` Tomi Ollila
2021-12-26 16:34 ` David Bremner
0 siblings, 1 reply; 5+ messages in thread
From: Tomi Ollila @ 2021-12-26 15:09 UTC (permalink / raw)
To: David Bremner, notmuch
On Sun, Dec 26 2021, David Bremner wrote:
> From: Jani Nikula <jani@nikula.org>
>
> In [1] Mark Walters reported a problem with messages being removed
> from the database when the parent directory was renamed. Jani Nikula
> proposed [2] these tests but observed
With that 'From:' line commit message starts to look weird...
>
> This test is not suitable for merging since it's not deterministic.
>
> After applying Jani's patch [3], the tests now pass deterministically,
> and could usefully act as regression tests.
>
> [1]: id:87siray6th.fsf@qmul.ac.uk
> [2]: id:1393191650-28333-1-git-send-email-jani@nikula.org
> [3]: id:1441445731-4362-2-git-send-email-jani@nikula.org
> ---
> test/T051-new-renames.sh | 40 ++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 40 insertions(+)
> create mode 100755 test/T051-new-renames.sh
>
> diff --git a/test/T051-new-renames.sh b/test/T051-new-renames.sh
> new file mode 100755
> index 00000000..febe006f
> --- /dev/null
> +++ b/test/T051-new-renames.sh
> @@ -0,0 +1,40 @@
> +#!/usr/bin/env bash
> +test_description='"notmuch new" with directory renames'
> +. ./test-lib.sh
> +
> +for loop in `seq 10`; do
{1..10}
> +
> +rm -rf ${MAIL_DIR}
> +
> +for i in `seq 10`; do
{1..10}
> + generate_message '[dir]=foo' '[subject]="Message foo $i"'
> +done
> +
> +for i in `seq 10`; do
{1..10}
> + generate_message '[dir]=bar' '[subject]="Message bar $i"'
> +done
> +
> +test_begin_subtest "Index the messages, round $loop"
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "Added 20 new messages to the database."
> +
> +all_files=$(notmuch search --output=files \*)
> +count_foo=$(notmuch count folder:foo)
> +
> +test_begin_subtest "Rename folder"
> +mv ${MAIL_DIR}/foo ${MAIL_DIR}/baz
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
> +
> +test_begin_subtest "Rename folder back"
> +mv ${MAIL_DIR}/baz ${MAIL_DIR}/foo
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
> +
> +test_begin_subtest "Files remain the same"
> +output=$(notmuch search --output=files \*)
> +test_expect_equal "$output" "$all_files"
> +
> +done
> +
> +test_done
> --
> 2.34.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] test: test folder renames
2021-12-26 15:09 ` Tomi Ollila
@ 2021-12-26 16:34 ` David Bremner
0 siblings, 0 replies; 5+ messages in thread
From: David Bremner @ 2021-12-26 16:34 UTC (permalink / raw)
To: Tomi Ollila, notmuch
Tomi Ollila <tomi.ollila@iki.fi> writes:
> On Sun, Dec 26 2021, David Bremner wrote:
>
>> From: Jani Nikula <jani@nikula.org>
>>
>> In [1] Mark Walters reported a problem with messages being removed
>> from the database when the parent directory was renamed. Jani Nikula
>> proposed [2] these tests but observed
>
> With that 'From:' line commit message starts to look weird...
thanks for the review. I applied your suggestions (and also fixed the
inclusion of test-lib.sh to match the other tests), and applied to
master.
One more test to break 1800 tests. I guess that must be a good thing?
d
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-12-26 16:34 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-12-26 11:21 [PATCH] test: test folder renames David Bremner
2021-12-26 15:09 ` Tomi Ollila
2021-12-26 16:34 ` David Bremner
-- strict thread matches above, loose matches on Subject: below --
2014-02-23 1:08 Weird behaviour in notmuch new Mark Walters
2014-02-23 21:40 ` [PATCH] test: test folder renames Jani Nikula
2014-02-23 23:28 ` Tomi Ollila
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).