* [PATCH] introduce new.rename_tags for renamed (moved) messages
@ 2018-07-06 14:04 Michael J Gruber
2018-09-08 0:47 ` David Bremner
2018-09-21 6:48 ` [PATCH] " Gaute Hope
0 siblings, 2 replies; 10+ messages in thread
From: Michael J Gruber @ 2018-07-06 14:04 UTC (permalink / raw)
To: notmuch
IMAP clients (such as webmail) use folders to mark messages as junk
etc., some even to mark messages as trash ("move to trash"). Such a
change is reported by notmuch as a rename; the message is not tagged
with new.tags since it is not new, so that there is no way to act upon a
rename.
Introduce new.rename_tags (default: not set) which are added by `notmuch
new` to renamed messages. This allows to act upon renames, e.g. to keep
the IMAP folder structure in sync with tags with a tool like `afew` or
homecooked scripts simply by filtering for this tag in the same ways as
one would filter for new messages using new.tags.
Signed-off-by: Michael J Gruber <git@grubix.eu>
---
NEWS | 11 +++++++++++
doc/man1/notmuch-config.rst | 6 ++++++
notmuch-client.h | 8 ++++++++
notmuch-config.c | 26 ++++++++++++++++++++++++++
notmuch-new.c | 23 +++++++++++++++++++++++
5 files changed, 74 insertions(+)
diff --git a/NEWS b/NEWS
index 240d594b..e3b75e74 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,14 @@
+Notmuch 0.28 (UNRELEASED)
+=========================
+
+New command-line features
+-------------------------
+
+User-configurable tags for renamed messages
+
+ A new "new.rename_tags" option is available in the configuration file to
+ determine which tags are applied to renamed (moved) messages.
+
Notmuch 0.27 (2018-06-13)
=========================
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 89909808..9e4198a1 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -77,6 +77,12 @@ The available configuration items are described below.
Default: ``unread;inbox``.
+**new.rename_tags**
+ A list of tags that will be added to all messages which
+ **notmuch new** identifies as renamed (moved).
+
+ Default: not set.
+
**new.ignore**
A list to specify files and directories that will not be searched
for messages by **notmuch new**. Each entry in the list is either:
diff --git a/notmuch-client.h b/notmuch-client.h
index 6c84ecc0..5e1e6b66 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -316,6 +316,14 @@ notmuch_config_set_new_tags (notmuch_config_t *config,
const char *new_tags[],
size_t length);
+const char **
+notmuch_config_get_rename_tags (notmuch_config_t *config,
+ size_t *length);
+void
+notmuch_config_set_rename_tags (notmuch_config_t *config,
+ const char *rename_tags[],
+ size_t length);
+
const char **
notmuch_config_get_new_ignore (notmuch_config_t *config,
size_t *length);
diff --git a/notmuch-config.c b/notmuch-config.c
index e1b16609..02f7d247 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -132,6 +132,8 @@ struct _notmuch_config {
size_t user_other_email_length;
const char **new_tags;
size_t new_tags_length;
+ const char **rename_tags;
+ size_t rename_tags_length;
const char **new_ignore;
size_t new_ignore_length;
bool maildir_synchronize_flags;
@@ -712,6 +714,14 @@ notmuch_config_get_new_tags (notmuch_config_t *config, size_t *length)
&(config->new_tags_length), length);
}
+const char **
+notmuch_config_get_rename_tags (notmuch_config_t *config, size_t *length)
+{
+ return _config_get_list (config, "new", "rename_tags",
+ &(config->rename_tags),
+ &(config->rename_tags_length), length);
+}
+
const char **
notmuch_config_get_new_ignore (notmuch_config_t *config, size_t *length)
{
@@ -738,6 +748,15 @@ notmuch_config_set_new_tags (notmuch_config_t *config,
&(config->new_tags));
}
+void
+notmuch_config_set_rename_tags (notmuch_config_t *config,
+ const char *list[],
+ size_t length)
+{
+ _config_set_list (config, "new", "rename_tags", list, length,
+ &(config->rename_tags));
+}
+
void
notmuch_config_set_new_ignore (notmuch_config_t *config,
const char *list[],
@@ -867,6 +886,13 @@ notmuch_config_command_get (notmuch_config_t *config, char *item)
tags = notmuch_config_get_new_tags (config, &length);
for (i = 0; i < length; i++)
printf ("%s\n", tags[i]);
+ } else if (strcmp(item, "new.rename_tags") == 0) {
+ const char **tags;
+ size_t i, length;
+
+ tags = notmuch_config_get_rename_tags (config, &length);
+ for (i = 0; i < length; i++)
+ printf ("%s\n", tags[i]);
} else if (STRNCMP_LITERAL (item, BUILT_WITH_PREFIX) == 0) {
printf ("%s\n",
notmuch_built_with (item + strlen (BUILT_WITH_PREFIX)) ? "true" : "false");
diff --git a/notmuch-new.c b/notmuch-new.c
index 6a54a1a1..e6d3dc82 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -50,6 +50,8 @@ typedef struct {
bool full_scan;
const char **new_tags;
size_t new_tags_length;
+ const char **rename_tags;
+ size_t rename_tags_length;
const char **ignore_verbatim;
size_t ignore_verbatim_length;
regex_t *ignore_regex;
@@ -948,9 +950,18 @@ remove_filename (notmuch_database_t *notmuch,
status = notmuch_database_remove_message (notmuch, path);
if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
+ const char **tag;
add_files_state->renamed_messages++;
+ notmuch_message_freeze (message);
+
+ for (tag = add_files_state->rename_tags; tag != NULL && *tag != NULL; tag++) {
+ notmuch_message_add_tag (message, *tag);
+ }
+
+
if (add_files_state->synchronize_flags == true)
notmuch_message_maildir_flags_to_tags (message);
+ notmuch_message_thaw (message);
status = NOTMUCH_STATUS_SUCCESS;
} else if (status == NOTMUCH_STATUS_SUCCESS) {
add_files_state->removed_messages++;
@@ -1095,6 +1106,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
add_files_state.verbosity = VERBOSITY_VERBOSE;
add_files_state.new_tags = notmuch_config_get_new_tags (config, &add_files_state.new_tags_length);
+ add_files_state.rename_tags = notmuch_config_get_rename_tags (config, &add_files_state.rename_tags_length);
add_files_state.synchronize_flags = notmuch_config_get_maildir_synchronize_flags (config);
db_path = notmuch_config_get_database_path (config);
add_files_state.db_path = db_path;
@@ -1113,6 +1125,17 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
}
}
+ for (i = 0; i < add_files_state.rename_tags_length; i++) {
+ const char *error_msg;
+
+ error_msg = illegal_tag (add_files_state.rename_tags[i], false);
+ if (error_msg) {
+ fprintf (stderr, "Error: tag '%s' in rename.tags: %s\n",
+ add_files_state.rename_tags[i], error_msg);
+ return EXIT_FAILURE;
+ }
+ }
+
if (hooks) {
ret = notmuch_run_hook (db_path, "pre-new");
if (ret)
--
2.18.0.226.g7b49cad896
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] introduce new.rename_tags for renamed (moved) messages
2018-07-06 14:04 [PATCH] introduce new.rename_tags for renamed (moved) messages Michael J Gruber
@ 2018-09-08 0:47 ` David Bremner
2018-09-18 15:32 ` [PATCH v2] " Michael J Gruber
2018-09-21 6:48 ` [PATCH] " Gaute Hope
1 sibling, 1 reply; 10+ messages in thread
From: David Bremner @ 2018-09-08 0:47 UTC (permalink / raw)
To: Michael J Gruber, notmuch
Michael J Gruber <git@grubix.eu> writes:
>
> Introduce new.rename_tags (default: not set) which are added by `notmuch
> new` to renamed messages. This allows to act upon renames, e.g. to keep
> the IMAP folder structure in sync with tags with a tool like `afew` or
> homecooked scripts simply by filtering for this tag in the same ways as
> one would filter for new messages using new.tags.
The idea seems OK to me. I was hoping for some more feedback from
others, but here we are.
> + const char **tag;
> add_files_state->renamed_messages++;
> + notmuch_message_freeze (message);
> +
> + for (tag = add_files_state->rename_tags; tag != NULL && *tag != NULL; tag++) {
> + notmuch_message_add_tag (message, *tag);
> + }
> +
> +
extra blank line
> if (add_files_state->synchronize_flags == true)
> notmuch_message_maildir_flags_to_tags (message);
> + notmuch_message_thaw (message);
Did you have a specific reason for putting the _thaw after the existing
maildir_flags_to_tags? It's probably not important, but if there's no
good reason it seems more natural before.
As a new feature this needs some tests before it can be merged. You can
start by looking at the tests for new.tags for inspiration I guess
(T050-new.sh and T340-maildir-sync.sh). One case that occured to me is
that when duplicate files (with the same message-id) exist, deleting one
of them is detected as a rename.
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2] introduce new.rename_tags for renamed (moved) messages
2018-09-08 0:47 ` David Bremner
@ 2018-09-18 15:32 ` Michael J Gruber
2019-04-02 12:40 ` [PATCH] performance-tests: tests for renamed/copied files in notmuch new David Bremner
2019-04-03 1:28 ` [PATCH v2] introduce new.rename_tags for renamed (moved) messages David Bremner
0 siblings, 2 replies; 10+ messages in thread
From: Michael J Gruber @ 2018-09-18 15:32 UTC (permalink / raw)
To: notmuch
IMAP clients (such as webmail) use folders to mark messages as junk
etc., some even to mark messages as trash ("move to trash"). Such a
change is reported by notmuch as a rename; the message is not tagged
with new.tags since it is not new, so that there is no way to act upon a
rename.
Introduce new.rename_tags (default: not set) which are added by `notmuch
new` to renamed messages. This allows to act upon renames, e.g. to keep
the IMAP folder structure in sync with tags with a tool like `afew` or
homecooked scripts simply by filtering for this tag in the same ways as
one would filter for new messages using new.tags.
Signed-off-by: Michael J Gruber <git@grubix.eu>
---
Changed since v1:
- acted upon review comments (blank line, _thaw position)
- added 3 tests (mv, cp, cp-rm)
- treat copies as renames, too
The reasoning behind the latter is: If you use a mapping between folders
and tags, then a copy to an additional location should alert the
"mapper" to update that mapping; that's what the rename tag is for.
Maybe it should be named "renew" after all? But it's just the
folder/label name that is/needs to be renewed, nothing else about the
message.
Interdiff against v1:
diff --git a/notmuch-new.c b/notmuch-new.c
index e6d3dc82..e893fa21 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -401,6 +401,13 @@ add_file (notmuch_database_t *notmuch, const char *filename,
break;
/* Non-fatal issues (go on to next file). */
case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
+ notmuch_message_freeze (message);
+
+ for (tag = state->rename_tags; tag != NULL && *tag != NULL; tag++) {
+ notmuch_message_add_tag (message, *tag);
+ }
+
+ notmuch_message_thaw (message);
if (state->synchronize_flags)
notmuch_message_maildir_flags_to_tags (message);
break;
@@ -958,10 +965,9 @@ remove_filename (notmuch_database_t *notmuch,
notmuch_message_add_tag (message, *tag);
}
-
+ notmuch_message_thaw (message);
if (add_files_state->synchronize_flags == true)
notmuch_message_maildir_flags_to_tags (message);
- notmuch_message_thaw (message);
status = NOTMUCH_STATUS_SUCCESS;
} else if (status == NOTMUCH_STATUS_SUCCESS) {
add_files_state->removed_messages++;
diff --git a/test/T340-maildir-sync.sh b/test/T340-maildir-sync.sh
index 7fece5f2..44f32ad2 100755
--- a/test/T340-maildir-sync.sh
+++ b/test/T340-maildir-sync.sh
@@ -196,6 +196,36 @@ notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
test_expect_equal "$(< output)" \
"thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (test unread)"
+test_begin_subtest "Renamed files get default renamed tags"
+OLDCONFIG=$(notmuch config get new.rename_tags)
+notmuch config set new.rename_tags "renamed"
+mv $MAIL_DIR/new/file-in-new $MAIL_DIR/new/file-in-new-renamed
+notmuch new
+notmuch config set new.rename_tags $OLDCONFIG
+notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
+test_expect_equal "$(< output)" \
+"thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (renamed test unread)"
+
+test_begin_subtest "Copied files do get new default renamed tags"
+OLDCONFIG=$(notmuch config get new.rename_tags)
+notmuch config set new.rename_tags "copied"
+cp $MAIL_DIR/new/file-in-new-renamed $MAIL_DIR/new/file-in-new-copied
+notmuch new
+notmuch config set new.rename_tags $OLDCONFIG
+notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
+test_expect_equal "$(< output)" \
+"thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; File in new/ (copied renamed test unread)"
+
+test_begin_subtest "Renamed files (cp+rm) get default renamed tags"
+OLDCONFIG=$(notmuch config get new.rename_tags)
+notmuch config set new.rename_tags "cprm"
+rm $MAIL_DIR/new/file-in-new-renamed
+notmuch new
+notmuch config set new.rename_tags $OLDCONFIG
+notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
+test_expect_equal "$(< output)" \
+"thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (copied cprm renamed test unread)"
+
for tag in draft flagged passed replied; do
test_begin_subtest "$tag is valid in new.tags"
OLDCONFIG=$(notmuch config get new.tags)
NEWS | 11 +++++++++++
doc/man1/notmuch-config.rst | 6 ++++++
notmuch-client.h | 8 ++++++++
notmuch-config.c | 26 ++++++++++++++++++++++++++
notmuch-new.c | 29 +++++++++++++++++++++++++++++
test/T340-maildir-sync.sh | 30 ++++++++++++++++++++++++++++++
6 files changed, 110 insertions(+)
diff --git a/NEWS b/NEWS
index 240d594b..e3b75e74 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,14 @@
+Notmuch 0.28 (UNRELEASED)
+=========================
+
+New command-line features
+-------------------------
+
+User-configurable tags for renamed messages
+
+ A new "new.rename_tags" option is available in the configuration file to
+ determine which tags are applied to renamed (moved) messages.
+
Notmuch 0.27 (2018-06-13)
=========================
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 89909808..9e4198a1 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -77,6 +77,12 @@ The available configuration items are described below.
Default: ``unread;inbox``.
+**new.rename_tags**
+ A list of tags that will be added to all messages which
+ **notmuch new** identifies as renamed (moved).
+
+ Default: not set.
+
**new.ignore**
A list to specify files and directories that will not be searched
for messages by **notmuch new**. Each entry in the list is either:
diff --git a/notmuch-client.h b/notmuch-client.h
index 6c84ecc0..5e1e6b66 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -316,6 +316,14 @@ notmuch_config_set_new_tags (notmuch_config_t *config,
const char *new_tags[],
size_t length);
+const char **
+notmuch_config_get_rename_tags (notmuch_config_t *config,
+ size_t *length);
+void
+notmuch_config_set_rename_tags (notmuch_config_t *config,
+ const char *rename_tags[],
+ size_t length);
+
const char **
notmuch_config_get_new_ignore (notmuch_config_t *config,
size_t *length);
diff --git a/notmuch-config.c b/notmuch-config.c
index e1b16609..02f7d247 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -132,6 +132,8 @@ struct _notmuch_config {
size_t user_other_email_length;
const char **new_tags;
size_t new_tags_length;
+ const char **rename_tags;
+ size_t rename_tags_length;
const char **new_ignore;
size_t new_ignore_length;
bool maildir_synchronize_flags;
@@ -712,6 +714,14 @@ notmuch_config_get_new_tags (notmuch_config_t *config, size_t *length)
&(config->new_tags_length), length);
}
+const char **
+notmuch_config_get_rename_tags (notmuch_config_t *config, size_t *length)
+{
+ return _config_get_list (config, "new", "rename_tags",
+ &(config->rename_tags),
+ &(config->rename_tags_length), length);
+}
+
const char **
notmuch_config_get_new_ignore (notmuch_config_t *config, size_t *length)
{
@@ -738,6 +748,15 @@ notmuch_config_set_new_tags (notmuch_config_t *config,
&(config->new_tags));
}
+void
+notmuch_config_set_rename_tags (notmuch_config_t *config,
+ const char *list[],
+ size_t length)
+{
+ _config_set_list (config, "new", "rename_tags", list, length,
+ &(config->rename_tags));
+}
+
void
notmuch_config_set_new_ignore (notmuch_config_t *config,
const char *list[],
@@ -867,6 +886,13 @@ notmuch_config_command_get (notmuch_config_t *config, char *item)
tags = notmuch_config_get_new_tags (config, &length);
for (i = 0; i < length; i++)
printf ("%s\n", tags[i]);
+ } else if (strcmp(item, "new.rename_tags") == 0) {
+ const char **tags;
+ size_t i, length;
+
+ tags = notmuch_config_get_rename_tags (config, &length);
+ for (i = 0; i < length; i++)
+ printf ("%s\n", tags[i]);
} else if (STRNCMP_LITERAL (item, BUILT_WITH_PREFIX) == 0) {
printf ("%s\n",
notmuch_built_with (item + strlen (BUILT_WITH_PREFIX)) ? "true" : "false");
diff --git a/notmuch-new.c b/notmuch-new.c
index 6a54a1a1..e893fa21 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -50,6 +50,8 @@ typedef struct {
bool full_scan;
const char **new_tags;
size_t new_tags_length;
+ const char **rename_tags;
+ size_t rename_tags_length;
const char **ignore_verbatim;
size_t ignore_verbatim_length;
regex_t *ignore_regex;
@@ -399,6 +401,13 @@ add_file (notmuch_database_t *notmuch, const char *filename,
break;
/* Non-fatal issues (go on to next file). */
case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
+ notmuch_message_freeze (message);
+
+ for (tag = state->rename_tags; tag != NULL && *tag != NULL; tag++) {
+ notmuch_message_add_tag (message, *tag);
+ }
+
+ notmuch_message_thaw (message);
if (state->synchronize_flags)
notmuch_message_maildir_flags_to_tags (message);
break;
@@ -948,7 +957,15 @@ remove_filename (notmuch_database_t *notmuch,
status = notmuch_database_remove_message (notmuch, path);
if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
+ const char **tag;
add_files_state->renamed_messages++;
+ notmuch_message_freeze (message);
+
+ for (tag = add_files_state->rename_tags; tag != NULL && *tag != NULL; tag++) {
+ notmuch_message_add_tag (message, *tag);
+ }
+
+ notmuch_message_thaw (message);
if (add_files_state->synchronize_flags == true)
notmuch_message_maildir_flags_to_tags (message);
status = NOTMUCH_STATUS_SUCCESS;
@@ -1095,6 +1112,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
add_files_state.verbosity = VERBOSITY_VERBOSE;
add_files_state.new_tags = notmuch_config_get_new_tags (config, &add_files_state.new_tags_length);
+ add_files_state.rename_tags = notmuch_config_get_rename_tags (config, &add_files_state.rename_tags_length);
add_files_state.synchronize_flags = notmuch_config_get_maildir_synchronize_flags (config);
db_path = notmuch_config_get_database_path (config);
add_files_state.db_path = db_path;
@@ -1113,6 +1131,17 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
}
}
+ for (i = 0; i < add_files_state.rename_tags_length; i++) {
+ const char *error_msg;
+
+ error_msg = illegal_tag (add_files_state.rename_tags[i], false);
+ if (error_msg) {
+ fprintf (stderr, "Error: tag '%s' in rename.tags: %s\n",
+ add_files_state.rename_tags[i], error_msg);
+ return EXIT_FAILURE;
+ }
+ }
+
if (hooks) {
ret = notmuch_run_hook (db_path, "pre-new");
if (ret)
diff --git a/test/T340-maildir-sync.sh b/test/T340-maildir-sync.sh
index 7fece5f2..44f32ad2 100755
--- a/test/T340-maildir-sync.sh
+++ b/test/T340-maildir-sync.sh
@@ -196,6 +196,36 @@ notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
test_expect_equal "$(< output)" \
"thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (test unread)"
+test_begin_subtest "Renamed files get default renamed tags"
+OLDCONFIG=$(notmuch config get new.rename_tags)
+notmuch config set new.rename_tags "renamed"
+mv $MAIL_DIR/new/file-in-new $MAIL_DIR/new/file-in-new-renamed
+notmuch new
+notmuch config set new.rename_tags $OLDCONFIG
+notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
+test_expect_equal "$(< output)" \
+"thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (renamed test unread)"
+
+test_begin_subtest "Copied files do get new default renamed tags"
+OLDCONFIG=$(notmuch config get new.rename_tags)
+notmuch config set new.rename_tags "copied"
+cp $MAIL_DIR/new/file-in-new-renamed $MAIL_DIR/new/file-in-new-copied
+notmuch new
+notmuch config set new.rename_tags $OLDCONFIG
+notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
+test_expect_equal "$(< output)" \
+"thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; File in new/ (copied renamed test unread)"
+
+test_begin_subtest "Renamed files (cp+rm) get default renamed tags"
+OLDCONFIG=$(notmuch config get new.rename_tags)
+notmuch config set new.rename_tags "cprm"
+rm $MAIL_DIR/new/file-in-new-renamed
+notmuch new
+notmuch config set new.rename_tags $OLDCONFIG
+notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
+test_expect_equal "$(< output)" \
+"thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (copied cprm renamed test unread)"
+
for tag in draft flagged passed replied; do
test_begin_subtest "$tag is valid in new.tags"
OLDCONFIG=$(notmuch config get new.tags)
--
2.19.0.612.g94276ab026
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] introduce new.rename_tags for renamed (moved) messages
2018-07-06 14:04 [PATCH] introduce new.rename_tags for renamed (moved) messages Michael J Gruber
2018-09-08 0:47 ` David Bremner
@ 2018-09-21 6:48 ` Gaute Hope
[not found] ` <CAA19uiRXVEp8kPdxUw4yDvVLmG0DSXpxaKKn-ZC0uFC97uJi_Q@mail.gmail.com>
1 sibling, 1 reply; 10+ messages in thread
From: Gaute Hope @ 2018-09-21 6:48 UTC (permalink / raw)
To: Michael J Gruber, notmuch
[-- Attachment #1: Type: text/plain, Size: 1227 bytes --]
Michael J Gruber writes on July 6, 2018 16:04:
> IMAP clients (such as webmail) use folders to mark messages as junk
> etc., some even to mark messages as trash ("move to trash"). Such a
> change is reported by notmuch as a rename; the message is not tagged
> with new.tags since it is not new, so that there is no way to act upon a
> rename.
>
> Introduce new.rename_tags (default: not set) which are added by `notmuch
> new` to renamed messages. This allows to act upon renames, e.g. to keep
> the IMAP folder structure in sync with tags with a tool like `afew` or
> homecooked scripts simply by filtering for this tag in the same ways as
> one would filter for new messages using new.tags.
Hi,
think this would be very useful. I suggested something similar way back[0]
(no doubt bit-rotted by now), and seem to remember there were some
issues with cases where a rename would not be detected. Might be worth
checking out !
[0] id:1396800683-9164-1-git-send-email-eg@gaute.vetsj.com
BTW: My use-case was solved by using `lastmod:` queries in keywsync [1]
(later obsoleted by gmailieer[2]).
[1] https://github.com/gauteh/abunchoftags
[2] https://github.com/gauteh/gmailieer
Regards, Gaute
[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] introduce new.rename_tags for renamed (moved) messages
[not found] ` <CAA19uiRXVEp8kPdxUw4yDvVLmG0DSXpxaKKn-ZC0uFC97uJi_Q@mail.gmail.com>
@ 2018-09-21 8:34 ` Gaute Hope
0 siblings, 0 replies; 10+ messages in thread
From: Gaute Hope @ 2018-09-21 8:34 UTC (permalink / raw)
To: Michael J Gruber; +Cc: notmuch
[-- Attachment #1: Type: text/plain, Size: 697 bytes --]
Michael J Gruber writes on September 21, 2018 10:22:
> Lastmod and (re)new tags serve different purposes, I would thank. The
> latter is a flag that can be cleared to signal that a certain job has
> been done (by whomever). Whereas, if one actor acts upon a lastmod
> change by modifying the db that will change the lastmod and (possibly)
> trigger another actor to act upon that. Different scenarios which look
> the same if you have one "actor" only.
Agreed, very useful if all scenarios can be reliably caught! Maybe it
was the "message changed without rename" that was problematic, since
offlineimap would change the X-Keywords header without renaming the
file.
- gaute
[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH] performance-tests: tests for renamed/copied files in notmuch new
2018-09-18 15:32 ` [PATCH v2] " Michael J Gruber
@ 2019-04-02 12:40 ` David Bremner
2019-04-02 12:45 ` David Bremner
2019-04-03 20:05 ` Tomi Ollila
2019-04-03 1:28 ` [PATCH v2] introduce new.rename_tags for renamed (moved) messages David Bremner
1 sibling, 2 replies; 10+ messages in thread
From: David Bremner @ 2019-04-02 12:40 UTC (permalink / raw)
To: Michael J Gruber, notmuch
Several people have observed that this is surprisingly slow, and we
have a proposal to add tagging into this code path, so we want to make
sure it doesn't imply too much of a performance hit.
---
performance-test/T00-new.sh | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
I added these tests to help evaluate Michael's propesed patch. I'll send the results in a seperate email.
diff --git a/performance-test/T00-new.sh b/performance-test/T00-new.sh
index 68750129..cec28d58 100755
--- a/performance-test/T00-new.sh
+++ b/performance-test/T00-new.sh
@@ -12,4 +12,34 @@ for i in $(seq 2 6); do
time_run "notmuch new #$i" 'notmuch new'
done
+manifest=$(mktemp manifestXXXXXX)
+
+count=0
+total=0
+while read -r name ; do
+ if [ $((total % 4 )) -eq 0 ]; then
+ echo $name >> $manifest
+ count=$((count + 1))
+ fi
+ total=$((total + 1))
+done < <(find mail -type f ! -path 'mail/.notmuch/*' )
+
+while read -r name ; do
+ mv $name ${name}.renamed
+done < $manifest
+
+time_run "new ($count mv)" 'notmuch new'
+
+while read -r name ; do
+ mv ${name}.renamed $name
+done < $manifest
+
+time_run "new ($count mv back)" 'notmuch new'
+
+while read -r name ; do
+ cp ${name} $name.copy
+done < $manifest
+
+time_run "new ($count cp)" 'notmuch new'
+
time_done
--
2.20.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] performance-tests: tests for renamed/copied files in notmuch new
2019-04-02 12:40 ` [PATCH] performance-tests: tests for renamed/copied files in notmuch new David Bremner
@ 2019-04-02 12:45 ` David Bremner
2019-04-03 20:05 ` Tomi Ollila
1 sibling, 0 replies; 10+ messages in thread
From: David Bremner @ 2019-04-02 12:45 UTC (permalink / raw)
To: Michael J Gruber, notmuch
[-- Attachment #1: Type: text/plain, Size: 772 bytes --]
David Bremner <david@tethera.net> writes:
> Several people have observed that this is surprisingly slow, and we
> have a proposal to add tagging into this code path, so we want to make
> sure it doesn't imply too much of a performance hit.
On my SSD / 8th gen i7 / 32G RAM based debian workstation it seems OK,
with a max slowdown of 1.4% (mean of 5 runs). I'd like to see similar
figures for spinning rust and older CPUs. This is with Xapian 1.4.11;
ideally so would other comparisons.
| | Before | After | slowdown % |
| initial | 517.69 | 522.13 | 0.9 |
| mv | 313.16 | 317.57 | 1.4 |
| mv back | 315.21 | 316.73 | 0.5 |
| cp | 170.36 | 171.08 | 0.4 |
#+TBLFM: $4=100*($3/$2-1);f1
Raw data is attached for the curious
[-- Attachment #2: trials.txt --]
[-- Type: text/plain, Size: 7146 bytes --]
with patch
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 521.81 405.90 111.16 273992 0/15876448
notmuch new #2 0.02 0.01 0.00 12632 0/144
notmuch new #3 0.00 0.00 0.00 8872 0/8
notmuch new #4 0.00 0.00 0.00 8900 0/8
notmuch new #5 0.00 0.00 0.00 9004 0/8
notmuch new #6 0.00 0.00 0.00 8728 0/8
new (52374 mv) 316.30 219.89 95.42 146136 0/4611440
new (52374 mv back) 312.67 217.23 94.55 147476 0/4986256
new (52374 cp) 170.76 125.47 43.85 114360 0/4099808
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 523.08 405.39 112.02 274152 0/15871216
notmuch new #2 0.02 0.00 0.01 12372 0/144
notmuch new #3 0.02 0.01 0.00 8984 0/8
notmuch new #4 0.01 0.00 0.00 8820 0/8
notmuch new #5 0.00 0.00 0.00 8776 0/8
notmuch new #6 0.00 0.00 0.00 8776 0/8
new (52374 mv) 317.27 221.69 94.55 146424 0/4864352
new (52374 mv back) 318.88 222.51 95.31 146764 0/5063728
new (52374 cp) 172.09 126.18 44.75 114548 0/4073552
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 523.58 407.64 112.15 274100 0/16211056
notmuch new #2 0.02 0.01 0.00 12644 0/144
notmuch new #3 0.00 0.00 0.00 8828 0/8
notmuch new #4 0.00 0.00 0.00 8884 0/8
notmuch new #5 0.00 0.00 0.00 8916 0/8
notmuch new #6 0.00 0.00 0.00 8896 0/8
new (52374 mv) 318.53 222.16 95.60 146368 0/5056016
new (52374 mv back) 315.98 219.54 95.64 146796 0/4766496
new (52374 cp) 170.79 124.49 44.63 114560 0/4075616
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 519.77 403.60 111.33 274212 0/15964240
notmuch new #2 0.02 0.00 0.01 12456 0/144
notmuch new #3 0.00 0.00 0.00 9080 0/8
notmuch new #4 0.00 0.00 0.00 8884 0/8
notmuch new #5 0.00 0.00 0.00 9004 0/8
notmuch new #6 0.00 0.00 0.00 8840 0/8
new (52374 mv) 318.98 222.26 95.94 146332 0/4854864
new (52374 mv back) 319.39 221.87 96.65 146944 0/4969296
new (52374 cp) 170.72 124.50 44.63 114224 0/3817216
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 522.43 406.40 111.85 273936 0/16188560
notmuch new #2 0.88 0.01 0.01 12512 0/144
notmuch new #3 0.03 0.01 0.01 8756 0/8
notmuch new #4 0.00 0.00 0.00 8852 0/8
notmuch new #5 0.00 0.00 0.00 8832 0/8
notmuch new #6 0.00 0.00 0.00 8864 0/8
new (52374 mv) 316.75 219.96 95.79 146496 0/4866064
new (52374 mv back) 316.73 220.40 95.33 146760 0/5034928
new (52374 cp) 171.03 125.37 44.34 114384 0/4096496
----------------------------------------------------------------------
without patch:
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 517.73 403.06 110.61 284648 0/14723776
notmuch new #2 0.01 0.00 0.00 8760 0/8
notmuch new #3 0.01 0.00 0.01 8928 0/8
notmuch new #4 0.01 0.01 0.00 8812 0/8
notmuch new #5 0.01 0.00 0.01 8928 0/8
notmuch new #6 0.00 0.00 0.00 8832 0/8
new (52374 mv) 314.48 218.67 94.55 146560 0/4774288
new (52374 mv back) 319.43 222.46 96.09 146780 0/4731808
new (52374 cp) 169.92 125.07 43.83 114476 0/3946896
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 516.03 401.91 109.94 284668 0/14725200
notmuch new #2 0.88 0.01 0.01 12420 0/144
notmuch new #3 0.01 0.00 0.00 8888 0/8
notmuch new #4 0.00 0.00 0.00 8760 0/8
notmuch new #5 0.00 0.00 0.00 8848 0/8
notmuch new #6 0.00 0.00 0.00 8864 0/8
new (52374 mv) 311.78 217.74 93.29 146456 8/4774576
new (52374 mv back) 312.88 219.64 92.07 147064 0/4785104
new (52374 cp) 170.95 125.16 44.57 114384 0/3688016
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 517.58 404.56 110.22 284608 0/15282160
notmuch new #2 0.02 0.00 0.01 12664 0/144
notmuch new #3 0.00 0.00 0.00 8956 0/8
notmuch new #4 0.00 0.00 0.00 8804 0/8
notmuch new #5 0.00 0.00 0.00 8892 0/8
notmuch new #6 0.00 0.00 0.00 8844 0/8
new (52374 mv) 313.10 217.43 94.61 146108 0/4980176
new (52374 mv back) 315.64 219.17 95.49 147176 0/4943408
new (52374 cp) 169.27 124.51 43.71 114500 0/3946928
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 518.28 402.97 111.69 284836 0/14991824
notmuch new #2 0.00 0.00 0.00 8804 0/8
notmuch new #3 0.00 0.00 0.00 8764 0/8
notmuch new #4 0.00 0.00 0.00 8792 0/8
notmuch new #5 0.00 0.00 0.00 9068 0/8
notmuch new #6 0.00 0.00 0.00 8820 0/8
new (52374 mv) 314.22 218.72 94.45 146276 0/4761712
new (52374 mv back) 314.39 220.18 93.40 147288 0/4832464
new (52374 cp) 170.49 125.64 43.84 114380 0/3948064
T00-new.sh: Testing notmuch new [0.4 large]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 518.81 405.03 110.53 284696 0/14964736
notmuch new #2 0.94 0.00 0.00 8808 0/144
notmuch new #3 0.01 0.00 0.00 9008 0/8
notmuch new #4 0.00 0.00 0.00 8768 0/8
notmuch new #5 0.00 0.00 0.00 8916 0/8
notmuch new #6 0.00 0.00 0.00 8776 0/8
new (52374 mv) 312.24 217.13 93.96 146492 0/4772608
new (52374 mv back) 313.70 218.13 93.98 147172 0/4753408
new (52374 cp) 171.15 125.01 44.42 114472 0/3970096
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: trials.org --]
[-- Type: text/x-org, Size: 3915 bytes --]
with patch
| Initial notmuch new | 519.77 | 403.60 | 111.33 | 274212 | 0/15964240 |
| Initial notmuch new | 521.81 | 405.90 | 111.16 | 273992 | 0/15876448 |
| Initial notmuch new | 522.43 | 406.40 | 111.85 | 273936 | 0/16188560 |
| Initial notmuch new | 523.08 | 405.39 | 112.02 | 274152 | 0/15871216 |
| Initial notmuch new | 523.58 | 407.64 | 112.15 | 274100 | 0/16211056 |
| | 522.13 | | | | |
#+TBLFM: @6$2=vmean(@1..@5);f2
| new (52374 cp) | 170.72 | 124.50 | 44.63 | 114224 | 0/3817216 |
| new (52374 cp) | 170.76 | 125.47 | 43.85 | 114360 | 0/4099808 |
| new (52374 cp) | 170.79 | 124.49 | 44.63 | 114560 | 0/4075616 |
| new (52374 cp) | 171.03 | 125.37 | 44.34 | 114384 | 0/4096496 |
| new (52374 cp) | 172.09 | 126.18 | 44.75 | 114548 | 0/4073552 |
| | 171.08 | | | | |
#+TBLFM: @6$2=vmean(@1..@5);f2
| new (52374 mv) | 316.30 | 219.89 | 95.42 | 146136 | 0/4611440 |
| new (52374 mv) | 316.75 | 219.96 | 95.79 | 146496 | 0/4866064 |
| new (52374 mv) | 317.27 | 221.69 | 94.55 | 146424 | 0/4864352 |
| new (52374 mv) | 318.53 | 222.16 | 95.60 | 146368 | 0/5056016 |
| new (52374 mv) | 318.98 | 222.26 | 95.94 | 146332 | 0/4854864 |
| | 317.57 | | | | |
#+TBLFM: @6$2=vmean(@1..@5);f2
| new (52374 mv back) | 312.67 | 217.23 | 94.55 | 147476 | 0/4986256 |
| new (52374 mv back) | 315.98 | 219.54 | 95.64 | 146796 | 0/4766496 |
| new (52374 mv back) | 316.73 | 220.40 | 95.33 | 146760 | 0/5034928 |
| new (52374 mv back) | 318.88 | 222.51 | 95.31 | 146764 | 0/5063728 |
| new (52374 mv back) | 319.39 | 221.87 | 96.65 | 146944 | 0/4969296 |
| | 316.73 | | | | |
#+TBLFM: @6$2=vmean(@1..@5);f2
----------------------------------------------------------------------
without patch:
| Initial notmuch new | 516.03 | 401.91 | 109.94 | 284668 | 0/14725200 |
| Initial notmuch new | 517.58 | 404.56 | 110.22 | 284608 | 0/15282160 |
| Initial notmuch new | 517.73 | 403.06 | 110.61 | 284648 | 0/14723776 |
| Initial notmuch new | 518.28 | 402.97 | 111.69 | 284836 | 0/14991824 |
| Initial notmuch new | 518.81 | 405.03 | 110.53 | 284696 | 0/14964736 |
| | 517.69 | | | | |
#+TBLFM: @6$2=vmean(@1..@5);f2
| new (52374 cp) | 169.27 | 124.51 | 43.71 | 114500 | 0/3946928 |
| new (52374 cp) | 169.92 | 125.07 | 43.83 | 114476 | 0/3946896 |
| new (52374 cp) | 170.49 | 125.64 | 43.84 | 114380 | 0/3948064 |
| new (52374 cp) | 170.95 | 125.16 | 44.57 | 114384 | 0/3688016 |
| new (52374 cp) | 171.15 | 125.01 | 44.42 | 114472 | 0/3970096 |
| | 170.36 | | | | |
#+TBLFM: @6$2=vmean(@1..@5);f2
| new (52374 mv) | 311.78 | 217.74 | 93.29 | 146456 | 8/4774576 |
| new (52374 mv) | 312.24 | 217.13 | 93.96 | 146492 | 0/4772608 |
| new (52374 mv) | 313.10 | 217.43 | 94.61 | 146108 | 0/4980176 |
| new (52374 mv) | 314.22 | 218.72 | 94.45 | 146276 | 0/4761712 |
| new (52374 mv) | 314.48 | 218.67 | 94.55 | 146560 | 0/4774288 |
| | 313.16 | | | | |
#+TBLFM: @6$2=vmean(@1..@5);f2
| new (52374 mv back) | 312.88 | 219.64 | 92.07 | 147064 | 0/4785104 |
| new (52374 mv back) | 313.70 | 218.13 | 93.98 | 147172 | 0/4753408 |
| new (52374 mv back) | 314.39 | 220.18 | 93.40 | 147288 | 0/4832464 |
| new (52374 mv back) | 315.64 | 219.17 | 95.49 | 147176 | 0/4943408 |
| new (52374 mv back) | 319.43 | 222.46 | 96.09 | 146780 | 0/4731808 |
| | 315.21 | | | | |
#+TBLFM: @6$2=vmean(@1..@5);f2
| | Before | After | slowdown % |
| initial | 517.69 | 522.13 | 0.9 |
| mv | 313.16 | 317.57 | 1.4 |
| mv back | 315.21 | 316.73 | 0.5 |
| cp | 170.36 | 171.08 | 0.4 |
#+TBLFM: $4=100*($3/$2-1);f1
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] introduce new.rename_tags for renamed (moved) messages
2018-09-18 15:32 ` [PATCH v2] " Michael J Gruber
2019-04-02 12:40 ` [PATCH] performance-tests: tests for renamed/copied files in notmuch new David Bremner
@ 2019-04-03 1:28 ` David Bremner
1 sibling, 0 replies; 10+ messages in thread
From: David Bremner @ 2019-04-03 1:28 UTC (permalink / raw)
To: Michael J Gruber, notmuch
Michael J Gruber <git@grubix.eu> writes:
> Changed since v1:
> - acted upon review comments (blank line, _thaw position)
> - added 3 tests (mv, cp, cp-rm)
> - treat copies as renames, too
Apologies for taking so long to get back to this. As a general comment,
does this same tagging "hook" make sense for notmuch-insert as well?
That's not meant to delay this series, just food for thought.
> The reasoning behind the latter is: If you use a mapping between folders
> and tags, then a copy to an additional location should alert the
> "mapper" to update that mapping; that's what the rename tag is for.
> Maybe it should be named "renew" after all? But it's just the
> folder/label name that is/needs to be renewed, nothing else about the
> message.
It _is_ confusing to use rename to refer to copies as well as actual
renames. I don't find "renew" better though. I wonder about something
like "new.new_path_tags". That's a bit weird with the repeated "new", I
grant you. Maybe "new.path_change_tags"
> +**new.rename_tags**
> + A list of tags that will be added to all messages which
> + **notmuch new** identifies as renamed (moved).
> +
> + Default: not set.
> +
Even if the name stays the same, you'll need to update the blurb to
mention copies.
> +void
> +notmuch_config_set_rename_tags (notmuch_config_t *config,
> + const char *rename_tags[],
> + size_t length);
> +
If this is only used in notmuch-config.c, I don't think it needs to
be exported (at least until they are needed). Of course it's also worth
asking if we want to call them notmuch-setup.c; I suspect not doing so
is OK, but I haven't checked in detail.
> diff --git a/test/T340-maildir-sync.sh b/test/T340-maildir-sync.sh
These tests don't really have to do with maildir syncing (in the sense
that notmuch uses the word), that's about syncing maildir flags to
notmuch tags. I think they'd be better in T050-new.sh.
> index 7fece5f2..44f32ad2 100755
> --- a/test/T340-maildir-sync.sh
> +++ b/test/T340-maildir-sync.sh
> @@ -196,6 +196,36 @@ notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
> test_expect_equal "$(< output)" \
> "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (test unread)"
>
> +test_begin_subtest "Renamed files get default renamed tags"
> +OLDCONFIG=$(notmuch config get new.rename_tags)
> +notmuch config set new.rename_tags "renamed"
> +mv $MAIL_DIR/new/file-in-new $MAIL_DIR/new/file-in-new-renamed
> +notmuch new
> +notmuch config set new.rename_tags $OLDCONFIG
> +notmuch search 'subject:"File in new"' | notmuch_search_sanitize > output
> +test_expect_equal "$(< output)" \
> +"thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (renamed test unread)"
Most of the newer tests use the pattern
cat << EOF > EXPECTED
thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; File in new/ (renamed test unread)
EOF
test_expect_equal_file EXPECTED output
we also mainly use OUTPUT for the file, and $output as a variable
containing output.
Unfortunately the searches would need to be updated as well to work in
T050-new.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] performance-tests: tests for renamed/copied files in notmuch new
2019-04-02 12:40 ` [PATCH] performance-tests: tests for renamed/copied files in notmuch new David Bremner
2019-04-02 12:45 ` David Bremner
@ 2019-04-03 20:05 ` Tomi Ollila
2019-04-05 16:59 ` David Bremner
1 sibling, 1 reply; 10+ messages in thread
From: Tomi Ollila @ 2019-04-03 20:05 UTC (permalink / raw)
To: David Bremner, Michael J Gruber, notmuch
On Tue, Apr 02 2019, David Bremner wrote:
> Several people have observed that this is surprisingly slow, and we
> have a proposal to add tagging into this code path, so we want to make
> sure it doesn't imply too much of a performance hit.
> ---
> performance-test/T00-new.sh | 30 ++++++++++++++++++++++++++++++
> 1 file changed, 30 insertions(+)
>
> I added these tests to help evaluate Michael's propesed patch. I'll send the results in a seperate email.
>
> diff --git a/performance-test/T00-new.sh b/performance-test/T00-new.sh
> index 68750129..cec28d58 100755
> --- a/performance-test/T00-new.sh
> +++ b/performance-test/T00-new.sh
> @@ -12,4 +12,34 @@ for i in $(seq 2 6); do
> time_run "notmuch new #$i" 'notmuch new'
> done
>
> +manifest=$(mktemp manifestXXXXXX)
> +
> +count=0
> +total=0
> +while read -r name ; do
> + if [ $((total % 4 )) -eq 0 ]; then
> + echo $name >> $manifest
> + count=$((count + 1))
> + fi
> + total=$((total + 1))
> +done < <(find mail -type f ! -path 'mail/.notmuch/*' )
// this comment was written last in this email, just for fun >;) //
find mail -type f ! -path 'mail/.notmuch/*' | sed -n '1~4 p' > $manifest
count=`wc $manifest`
(I'd be interested which one of the above were faster -- my suggestion
does quite a many more forks and execve's but abowe read loop 200 000
read(2)'s and [lf]seek(2)s (and then 50 000 opens).
well, probably no-one would notice difference...)
> +
> +while read -r name ; do
> + mv $name ${name}.renamed
> +done < $manifest
--------'12' -- 2 spaces above (and below...)
luckily bash read builtin does not read input byte at a time
(IIRC it read 128 bytes, then scanned for newline and then seeked
-- in this case it can, since file was redirected -- fd is seekable)
50 000 mv(1) executions definitely take time.
perl -nle 'rename $_, "$_.renamed"' $manifest would be significantly faster
> +
> +time_run "new ($count mv)" 'notmuch new'
> +
> +while read -r name ; do
> + mv ${name}.renamed $name
> +done < $manifest
> +
> +time_run "new ($count mv back)" 'notmuch new'
> +
> +while read -r name ; do
> + cp ${name} $name.copy
> +done < $manifest
perl -nle 'link $_, "$_.copy"' $manifest ?
> +
> +time_run "new ($count cp)" 'notmuch new'
> +
> time_done
> --
> 2.20.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] performance-tests: tests for renamed/copied files in notmuch new
2019-04-03 20:05 ` Tomi Ollila
@ 2019-04-05 16:59 ` David Bremner
0 siblings, 0 replies; 10+ messages in thread
From: David Bremner @ 2019-04-05 16:59 UTC (permalink / raw)
To: Tomi Ollila, Michael J Gruber, notmuch
Tomi Ollila <tomi.ollila@iki.fi> writes:
>> +done < <(find mail -type f ! -path 'mail/.notmuch/*' )
>
> // this comment was written last in this email, just for fun >;) //
>
> find mail -type f ! -path 'mail/.notmuch/*' | sed -n '1~4 p' > $manifest
> count=`wc $manifest`
>
> (I'd be interested which one of the above were faster -- my suggestion
> does quite a many more forks and execve's but abowe read loop 200 000
> read(2)'s and [lf]seek(2)s (and then 50 000 opens).
> well, probably no-one would notice difference...)
Your version is _much_ faster so I used it, along with the other
suggestions in the version I just pushed to master.
d
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2019-04-05 16:59 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-07-06 14:04 [PATCH] introduce new.rename_tags for renamed (moved) messages Michael J Gruber
2018-09-08 0:47 ` David Bremner
2018-09-18 15:32 ` [PATCH v2] " Michael J Gruber
2019-04-02 12:40 ` [PATCH] performance-tests: tests for renamed/copied files in notmuch new David Bremner
2019-04-02 12:45 ` David Bremner
2019-04-03 20:05 ` Tomi Ollila
2019-04-05 16:59 ` David Bremner
2019-04-03 1:28 ` [PATCH v2] introduce new.rename_tags for renamed (moved) messages David Bremner
2018-09-21 6:48 ` [PATCH] " Gaute Hope
[not found] ` <CAA19uiRXVEp8kPdxUw4yDvVLmG0DSXpxaKKn-ZC0uFC97uJi_Q@mail.gmail.com>
2018-09-21 8:34 ` Gaute Hope
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).