* [PATCH 1/4] lib: Return added message even if it already was in the database
2010-10-13 12:13 [PATCH 0/4] Maildir synchronization v2 Michal Sojka
@ 2010-10-13 12:13 ` Michal Sojka
2010-10-13 12:13 ` [PATCH 2/4] Maildir synchronization Michal Sojka
` (5 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Michal Sojka @ 2010-10-13 12:13 UTC (permalink / raw)
To: notmuch
---
lib/database.cc | 3 ++-
lib/notmuch.h | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index e4ac970..9a4f715 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1671,7 +1671,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
DONE:
if (message) {
- if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
+ if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
*message_ret = message;
else
notmuch_message_destroy (message);
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 505ad19..0ba1416 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -238,7 +238,8 @@ notmuch_database_get_directory (notmuch_database_t *database,
* notmuch database will reference the filename, and will not copy the
* entire contents of the file.
*
- * If 'message' is not NULL, then, on successful return '*message'
+ * If 'message' is not NULL, then, on successful return
+ * (NOTMUCH_STATUS_SUCCESS or NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) '*message'
* will be initialized to a message object that can be used for things
* such as adding tags to the just-added message. The user should call
* notmuch_message_destroy when done with the message. On any failure
--
1.7.1.3.g75e44
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH 2/4] Maildir synchronization
2010-10-13 12:13 [PATCH 0/4] Maildir synchronization v2 Michal Sojka
2010-10-13 12:13 ` [PATCH 1/4] lib: Return added message even if it already was in the database Michal Sojka
@ 2010-10-13 12:13 ` Michal Sojka
2010-10-13 13:56 ` Sebastian Spaeth
2010-10-13 12:13 ` [PATCH 3/4] Make maildir synchronization configurable Michal Sojka
` (4 subsequent siblings)
6 siblings, 1 reply; 20+ messages in thread
From: Michal Sojka @ 2010-10-13 12:13 UTC (permalink / raw)
To: notmuch
This patch allows bi-directional synchronization between maildir
flags and certain tags. The flag-to-tag mapping is defined by flag2tag
array.
The synchronization works this way:
1) Whenever notmuch new is executed, the following happens:
o New messages are tagged with configured new_tags.
o For new or renamed messages with maildir info present in the file
name, the tags defined in flag2tag are either added or removed
depending on the flags from the file name.
2) Whenever notmuch tag (or notmuch restore) is executed, a new set of
flags based on the tags is constructed for every message and a new
file name is prepared based on the old file name but with the new
flags. If the flags differs and the old message was in 'new'
directory then this is replaced with 'cur' in the new file name. If
the new and old file names differ, the file is renamed and notmuch
database is updated accordingly.
The rename happens before the database is updated. In case of crash
between rename and database update, the next run of notmuch new
brings the database in sync with the mail store again.
There is currenlty one known issue: Viewing/storing of attachments of
unread messages doesn't work. The reason is that when you view the
message its unread tag is removed which leads to rename of the file,
but Emacs still uses the original name to access the attachment.
Workaround: close the message and open it again.
---
lib/database.cc | 7 ++
lib/message.cc | 226 +++++++++++++++++++++++++++++++++++++++++++++++++
lib/notmuch-private.h | 4 +
lib/notmuch.h | 7 ++
notmuch-new.c | 3 +-
5 files changed, 246 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 9a4f715..9652013 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1643,6 +1643,13 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
_notmuch_message_add_filename (message, filename);
+ /* This is a new message or it has a new filename and as such,
+ * its tags in database either do not exists or might be out
+ * of date. We assign the tags later in notmuch new, but until
+ * then we should not synchronize the tags back to the maildir
+ * flags (if notmuch is configured to do so). */
+ notmuch_message_set_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, TRUE);
+
/* Is this a newly created message object? */
if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
_notmuch_message_add_term (message, "type", "mail");
diff --git a/lib/message.cc b/lib/message.cc
index 71f5619..96c402c 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -43,6 +43,24 @@ struct _notmuch_message {
Xapian::Document doc;
};
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
+
+struct maildir_flag_tag {
+ char flag;
+ const char *tag;
+ bool inverse;
+};
+
+/* ASCII ordered table of Maildir flags and associated tags */
+struct maildir_flag_tag flag2tag[] = {
+ { 'D', "draft", false},
+ { 'F', "flagged", false},
+ { 'P', "passed", false},
+ { 'R', "replied", false},
+ { 'S', "unread", true },
+ { 'T', "delete", false},
+};
+
/* We end up having to call the destructor explicitly because we had
* to use "placement new" in order to initialize C++ objects within a
* block that we allocated with talloc. So C++ is making talloc
@@ -595,15 +613,29 @@ _notmuch_message_set_date (notmuch_message_t *message,
Xapian::sortable_serialise (time_value));
}
+static notmuch_private_status_t
+_notmuch_message_tags_to_maildir (notmuch_message_t *message);
+
/* Synchronize changes made to message->doc out into the database. */
void
_notmuch_message_sync (notmuch_message_t *message)
{
Xapian::WritableDatabase *db;
+ notmuch_private_status_t status;
if (message->notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY)
return;
+ if (// todo_sync_enabled &&
+ !notmuch_message_get_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID)) {
+ status = _notmuch_message_tags_to_maildir (message);
+ if (status != NOTMUCH_PRIVATE_STATUS_SUCCESS) {
+ fprintf (stderr, "Error: Cannot sync tags to maildir (%s)\n",
+ notmuch_status_to_string ((notmuch_status_t)status));
+ /* Exit to avoid unsynchronized mailstore. */
+ exit(1);
+ }
+ }
db = static_cast <Xapian::WritableDatabase *> (message->notmuch->xapian_db);
db->replace_document (message->doc_id, message->doc);
}
@@ -715,6 +747,44 @@ _notmuch_message_remove_term (notmuch_message_t *message,
return NOTMUCH_PRIVATE_STATUS_SUCCESS;
}
+/* Change the message filename stored in the database.
+ *
+ * This change will not be reflected in the database until the next
+ * call to _notmuch_message_sync.
+ */
+notmuch_private_status_t
+_notmuch_message_rename (notmuch_message_t *message,
+ const char *new_filename)
+{
+ void *local = talloc_new (message);
+ char *direntry;
+ Xapian::PostingIterator i, end;
+ Xapian::Document document;
+ notmuch_private_status_t pstatus;
+ notmuch_status_t status;
+ const char *old_filename;
+
+ old_filename = notmuch_message_get_filename(message);
+ old_filename = talloc_reference(local, old_filename);
+ if (unlikely(!old_filename))
+ return NOTMUCH_PRIVATE_STATUS_OUT_OF_MEMORY;
+
+ status = _notmuch_message_add_filename (message, new_filename);
+ if (status)
+ return (notmuch_private_status_t)status;
+
+ status = _notmuch_database_filename_to_direntry (local, message->notmuch,
+ old_filename, &direntry);
+ if (status)
+ return (notmuch_private_status_t)status;
+
+ pstatus = _notmuch_message_remove_term (message, "file-direntry", direntry);
+
+ talloc_free (local);
+
+ return pstatus;
+}
+
notmuch_status_t
notmuch_message_add_tag (notmuch_message_t *message, const char *tag)
{
@@ -772,6 +842,162 @@ notmuch_message_remove_tag (notmuch_message_t *message, const char *tag)
}
notmuch_status_t
+notmuch_message_maildir_to_tags (notmuch_message_t *message, const char *filename)
+{
+ const char *flags, *p;
+ char f;
+ bool valid, unread;
+ unsigned i;
+ notmuch_status_t status;
+
+ flags = strstr (filename, ":2,");
+ if (!flags)
+ return NOTMUCH_STATUS_FILE_NOT_EMAIL;
+ flags += 3;
+
+ /* Check that the letters are valid Maildir flags */
+ f = 0;
+ valid = true;
+ for (p=flags; valid && *p; p++) {
+ switch (*p) {
+ case 'P':
+ case 'R':
+ case 'S':
+ case 'T':
+ case 'D':
+ case 'F':
+ if (*p > f) f=*p;
+ else valid = false;
+ break;
+ default:
+ valid = false;
+ }
+ }
+ if (!valid) {
+ fprintf (stderr, "Warning: Invalid maildir flags in filename %s\n", filename);
+ return NOTMUCH_STATUS_FILE_NOT_EMAIL;
+ }
+
+ status = notmuch_message_freeze (message);
+ if (status)
+ return status;
+ unread = true;
+ for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
+ if ((strchr (flags, flag2tag[i].flag) != NULL) ^ flag2tag[i].inverse) {
+ status = notmuch_message_add_tag (message, flag2tag[i].tag);
+ } else {
+ status = notmuch_message_remove_tag (message, flag2tag[i].tag);
+ }
+ if (status)
+ return status;
+ }
+ status = notmuch_message_thaw (message);
+
+ /* From now on, we can synchronize the tags from the database to
+ * the mailstore. */
+ notmuch_message_set_flag (message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, FALSE);
+ return status;
+}
+
+static void
+maildir_get_new_flags(notmuch_message_t *message, char *flags)
+{
+ notmuch_tags_t *tags;
+ const char *tag;
+ unsigned i;
+ char *p;
+
+ for (i = 0; i < ARRAY_SIZE(flag2tag); i++)
+ flags[i] = flag2tag[i].inverse ? flag2tag[i].flag : '\0';
+
+ for (tags = notmuch_message_get_tags (message);
+ notmuch_tags_valid (tags);
+ notmuch_tags_move_to_next (tags))
+ {
+ tag = notmuch_tags_get (tags);
+ for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
+ if (strcmp(tag, flag2tag[i].tag) == 0)
+ flags[i] = flag2tag[i].inverse ? '\0' : flag2tag[i].flag;
+ }
+ }
+
+ p = flags;
+ for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
+ if (flags[i])
+ *p++ = flags[i];
+ }
+ *p = '\0';
+}
+
+static char *
+maildir_get_subdir (char *filename)
+{
+ char *p, *subdir = NULL;
+
+ p = filename + strlen (filename) - 1;
+ while (p > filename + 3 && *p != '/')
+ p--;
+ if (*p == '/') {
+ subdir = p - 3;
+ if (subdir > filename && *(subdir - 1) != '/')
+ subdir = NULL;
+ }
+ return subdir;
+}
+
+/* Rename the message file so that maildir flags corresponds to the
+ * tags and, if aplicable, move the message from new/ to cur/. */
+static notmuch_private_status_t
+_notmuch_message_tags_to_maildir (notmuch_message_t *message)
+{
+ char flags[ARRAY_SIZE(flag2tag)+1];
+ const char *filename, *p;
+ char *filename_new, *subdir = NULL;
+ int ret;
+
+ maildir_get_new_flags (message, flags);
+
+ filename = notmuch_message_get_filename (message);
+ /* TODO: Iterate over all file names. */
+ p = strstr(filename, ":2,");
+ if ((p && strcmp (p+3, flags) == 0) ||
+ (!p && flags[0] == '\0')) {
+ // Return if flags are not to be changed - this suppresses
+ // moving the message from new/ to cur/ during initial
+ // tagging.
+ return NOTMUCH_PRIVATE_STATUS_SUCCESS;
+ }
+ if (!p)
+ p = filename + strlen(filename);
+
+ filename_new = (char*)talloc_size(message, (p-filename) + 3 + sizeof(flags));
+ if (unlikely (filename_new == NULL))
+ return NOTMUCH_PRIVATE_STATUS_OUT_OF_MEMORY;
+ memcpy(filename_new, filename, p-filename);
+ filename_new[p-filename] = '\0';
+
+ /* If message is in new/ move it under cur/. */
+ subdir = maildir_get_subdir (filename_new);
+ if (subdir && memcmp (subdir, "new/", 4) == 0)
+ memcpy (subdir, "cur/", 4);
+
+ strcpy (filename_new+(p-filename), ":2,");
+ strcpy (filename_new+(p-filename)+3, flags);
+
+ if (strcmp (filename, filename_new) != 0) {
+ ret = rename (filename, filename_new);
+ if (ret == -1) {
+ perror (talloc_asprintf (message, "rename of %s to %s failed",
+ filename, filename_new));
+ exit (1);
+ }
+ return _notmuch_message_rename (message, filename_new);
+ /* _notmuch_message_sync is our caller. Do not call it here. */
+ }
+ return NOTMUCH_PRIVATE_STATUS_SUCCESS;
+}
+
+notmuch_status_t
notmuch_message_remove_all_tags (notmuch_message_t *message)
{
notmuch_private_status_t private_status;
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 5a0cf92..1dc0a20 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -259,6 +259,10 @@ notmuch_status_t
_notmuch_message_add_filename (notmuch_message_t *message,
const char *filename);
+notmuch_private_status_t
+_notmuch_message_rename (notmuch_message_t *message,
+ const char *new_filename);
+
void
_notmuch_message_ensure_thread_id (notmuch_message_t *message);
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 0ba1416..408d633 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -770,6 +770,7 @@ notmuch_message_get_filename (notmuch_message_t *message);
/* Message flags */
typedef enum _notmuch_message_flag {
NOTMUCH_MESSAGE_FLAG_MATCH,
+ NOTMUCH_MESSAGE_FLAG_TAGS_INVALID,
} notmuch_message_flag_t;
/* Get a value of a flag for the email corresponding to 'message'. */
@@ -886,6 +887,12 @@ notmuch_message_remove_tag (notmuch_message_t *message, const char *tag);
notmuch_status_t
notmuch_message_remove_all_tags (notmuch_message_t *message);
+/* Add or remove tags based on the maildir flags in the file name.
+ */
+notmuch_status_t
+notmuch_message_maildir_to_tags (notmuch_message_t *message,
+ const char *filename);
+
/* Freeze the current state of 'message' within the database.
*
* This means that changes to the message state, (via
diff --git a/notmuch-new.c b/notmuch-new.c
index 8818728..ed3f944 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -410,10 +410,11 @@ add_files_recursive (notmuch_database_t *notmuch,
state->added_messages++;
for (tag=state->new_tags; *tag != NULL; tag++)
notmuch_message_add_tag (message, *tag);
+ notmuch_message_maildir_to_tags (message, next);
break;
/* Non-fatal issues (go on to next file) */
case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
- /* Stay silent on this one. */
+ notmuch_message_maildir_to_tags (message, next);
break;
case NOTMUCH_STATUS_FILE_NOT_EMAIL:
fprintf (stderr, "Note: Ignoring non-mail file: %s\n",
--
1.7.1.3.g75e44
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH 2/4] Maildir synchronization
2010-10-13 12:13 ` [PATCH 2/4] Maildir synchronization Michal Sojka
@ 2010-10-13 13:56 ` Sebastian Spaeth
2010-10-22 5:03 ` Michal Sojka
0 siblings, 1 reply; 20+ messages in thread
From: Sebastian Spaeth @ 2010-10-13 13:56 UTC (permalink / raw)
To: Michal Sojka, notmuch
[-- Attachment #1: Type: text/plain, Size: 810 bytes --]
> +/* ASCII ordered table of Maildir flags and associated tags */
> +struct maildir_flag_tag flag2tag[] = {
> + { 'D', "draft", false},
> + { 'F', "flagged", false},
> + { 'P', "passed", false},
> + { 'R', "replied", false},
> + { 'S', "unread", true },
> + { 'T', "delete", false},
> +};
for all of the 4 patches a:
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
with one caveat: you use the tag "delete" to mark the state "trashed". I
think people have slowly settled on the convention "deleted" which is
also more consistent with a state, such as unread, replied, etc...
They work fine and obsolete my notmuchsync (except for the pruning of
deleted mails and the tagging based on folder names). +1 for merging in
notmuch-next or wherever we collect that stuff.
Sebastian
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH 2/4] Maildir synchronization
2010-10-13 13:56 ` Sebastian Spaeth
@ 2010-10-22 5:03 ` Michal Sojka
0 siblings, 0 replies; 20+ messages in thread
From: Michal Sojka @ 2010-10-22 5:03 UTC (permalink / raw)
To: Sebastian Spaeth, notmuch
On Wed, 13 Oct 2010, Sebastian Spaeth wrote:
> > +/* ASCII ordered table of Maildir flags and associated tags */
> > +struct maildir_flag_tag flag2tag[] = {
> > + { 'D', "draft", false},
> > + { 'F', "flagged", false},
> > + { 'P', "passed", false},
> > + { 'R', "replied", false},
> > + { 'S', "unread", true },
> > + { 'T', "delete", false},
> > +};
>
> [...]
>
> with one caveat: you use the tag "delete" to mark the state "trashed". I
> think people have slowly settled on the convention "deleted" which is
> also more consistent with a state, such as unread, replied, etc...
Hi,
I changed "delete" to "deleted" and the updated patch series is
reachable by:
git pull git://rtime.felk.cvut.cz/notmuch maildir-sync-v3
-Michal
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 3/4] Make maildir synchronization configurable
2010-10-13 12:13 [PATCH 0/4] Maildir synchronization v2 Michal Sojka
2010-10-13 12:13 ` [PATCH 1/4] lib: Return added message even if it already was in the database Michal Sojka
2010-10-13 12:13 ` [PATCH 2/4] Maildir synchronization Michal Sojka
@ 2010-10-13 12:13 ` Michal Sojka
2010-10-13 12:13 ` [PATCH 4/4] Tests for maildir synchronization Michal Sojka
` (3 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Michal Sojka @ 2010-10-13 12:13 UTC (permalink / raw)
To: notmuch
This adds group [maildir] and key 'sync_level' to the configuration file.
The value of sync_level is used to control how the synchronization happens.
The default value is no synchronization.
---
lib/database-private.h | 2 +-
lib/database.cc | 9 +++++++++
lib/message.cc | 2 +-
lib/notmuch.h | 19 +++++++++++++++++++
notmuch-client.h | 7 +++++++
notmuch-config.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
notmuch-new.c | 8 ++++++--
notmuch-restore.c | 2 ++
notmuch-setup.c | 17 +++++++++++++++++
notmuch-tag.c | 2 ++
10 files changed, 112 insertions(+), 4 deletions(-)
diff --git a/lib/database-private.h b/lib/database-private.h
index bd72f67..3c80f99 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,7 +49,7 @@ struct _notmuch_database {
Xapian::QueryParser *query_parser;
Xapian::TermGenerator *term_gen;
Xapian::ValueRangeProcessor *value_range_processor;
-
+ enum notmuch_maildir_sync maildir_sync;
};
/* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index 9652013..d6b6f47 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -689,6 +689,8 @@ notmuch_database_open (const char *path,
notmuch = NULL;
}
+ notmuch_database_set_maildir_sync (notmuch, NOTMUCH_MAILDIR_SYNC_NONE);
+
DONE:
if (notmuch_path)
free (notmuch_path);
@@ -718,6 +720,13 @@ notmuch_database_close (notmuch_database_t *notmuch)
talloc_free (notmuch);
}
+void
+notmuch_database_set_maildir_sync (notmuch_database_t *database,
+ enum notmuch_maildir_sync maildir_sync)
+{
+ database->maildir_sync = maildir_sync;
+}
+
const char *
notmuch_database_get_path (notmuch_database_t *notmuch)
{
diff --git a/lib/message.cc b/lib/message.cc
index 96c402c..1b945c6 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -626,7 +626,7 @@ _notmuch_message_sync (notmuch_message_t *message)
if (message->notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY)
return;
- if (// todo_sync_enabled &&
+ if (message->notmuch->maildir_sync == NOTMUCH_MAILDIR_SYNC_NEW_RENAMED_TAGGED &&
!notmuch_message_get_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID)) {
status = _notmuch_message_tags_to_maildir (message);
if (status != NOTMUCH_PRIVATE_STATUS_SUCCESS) {
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 408d633..34b3bcf 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -108,6 +108,19 @@ typedef enum _notmuch_status {
const char *
notmuch_status_to_string (notmuch_status_t status);
+/* Level of synchronization between notmuch tags and maildir flags. */
+enum notmuch_maildir_sync {
+ NOTMUCH_MAILDIR_SYNC_INVALID = 0,
+ /* No synchronization */
+ NOTMUCH_MAILDIR_SYNC_NONE,
+ /* Tag new messages accoring to maildir flags */
+ NOTMUCH_MAILDIR_SYNC_NEW,
+ /* The above + update tags for renamed messages */
+ NOTMUCH_MAILDIR_SYNC_NEW_RENAMED,
+ /* The above + update flags when tags are added/removed. */
+ NOTMUCH_MAILDIR_SYNC_NEW_RENAMED_TAGGED,
+};
+
/* Various opaque data types. For each notmuch_<foo>_t see the various
* notmuch_<foo> functions below. */
typedef struct _notmuch_database notmuch_database_t;
@@ -176,6 +189,12 @@ notmuch_database_open (const char *path,
void
notmuch_database_close (notmuch_database_t *database);
+/* Sets the level of synchronization between maildir flags and notmuch
+ * tags. */
+void
+notmuch_database_set_maildir_sync (notmuch_database_t *database,
+ enum notmuch_maildir_sync maildir_sync);
+
/* Return the database path of the given database.
*
* The return value is a string owned by notmuch so should not be
diff --git a/notmuch-client.h b/notmuch-client.h
index 20be43b..50be95c 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -191,6 +191,13 @@ notmuch_config_set_new_tags (notmuch_config_t *config,
const char *new_tags[],
size_t length);
+enum notmuch_maildir_sync
+notmuch_config_get_maildir_sync (notmuch_config_t *config);
+
+void
+notmuch_config_set_maildir_sync (notmuch_config_t *config,
+ enum notmuch_maildir_sync maildir_sync);
+
notmuch_bool_t
debugger_is_active (void);
diff --git a/notmuch-config.c b/notmuch-config.c
index cf30603..60d5730 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -61,6 +61,21 @@ static const char user_config_comment[] =
" recipient list of replies, and will set the From address based on the\n"
" address to which the original email was addressed.\n";
+static const char maildir_config_comment[] =
+ " Maildir compatibility configuration\n"
+ "\n"
+ " Here you can configure whether and how will notmuch synchronize its\n"
+ " tags with maildir flags."
+ "\n"
+ "\tsync_level Integer in the range 1 - 4 with the following meaning:\n"
+ "\t\t1 - No synchronization at all.\n"
+ "\t\t2 - 'notmuch new' tags the messages based on their maildir flags\n"
+ "\t\t only when it sees them for the first time.\n"
+ "\t\t3 - Same as 2 plus 'notmuch new' updates tags when it detects the\n"
+ "\t\t message was renamed.\n"
+ "\t\t4 - Same as 3 plus whenever message tags are changed, maildir\n"
+ "\t\t flags are updated accordingly.\n";
+
struct _notmuch_config {
char *filename;
GKeyFile *key_file;
@@ -72,6 +87,7 @@ struct _notmuch_config {
size_t user_other_email_length;
const char **new_tags;
size_t new_tags_length;
+ enum notmuch_maildir_sync maildir_sync;
};
static int
@@ -191,6 +207,7 @@ notmuch_config_open (void *ctx,
int file_had_database_group;
int file_had_new_group;
int file_had_user_group;
+ int file_had_maildir_group;
if (is_new_ret)
*is_new_ret = 0;
@@ -221,6 +238,7 @@ notmuch_config_open (void *ctx,
config->user_other_email_length = 0;
config->new_tags = NULL;
config->new_tags_length = 0;
+ config->maildir_sync = NOTMUCH_MAILDIR_SYNC_INVALID;
if (! g_key_file_load_from_file (config->key_file,
config->filename,
@@ -263,6 +281,7 @@ notmuch_config_open (void *ctx,
"database");
file_had_new_group = g_key_file_has_group (config->key_file, "new");
file_had_user_group = g_key_file_has_group (config->key_file, "user");
+ file_had_maildir_group = g_key_file_has_group (config->key_file, "maildir");
if (notmuch_config_get_database_path (config) == NULL) {
@@ -313,6 +332,10 @@ notmuch_config_open (void *ctx,
notmuch_config_set_new_tags (config, tags, 2);
}
+ if (notmuch_config_get_maildir_sync (config) == NOTMUCH_MAILDIR_SYNC_INVALID) {
+ notmuch_config_set_maildir_sync (config, NOTMUCH_MAILDIR_SYNC_NONE);
+ }
+
/* Whenever we know of configuration sections that don't appear in
* the configuration file, we add some comments to help the user
* understand what can be done. */
@@ -340,6 +363,12 @@ notmuch_config_open (void *ctx,
user_config_comment, NULL);
}
+ if (! file_had_maildir_group)
+ {
+ g_key_file_set_comment (config->key_file, "maildir", NULL,
+ maildir_config_comment, NULL);
+ }
+
if (is_new_ret)
*is_new_ret = is_new;
@@ -562,3 +591,22 @@ notmuch_config_set_new_tags (notmuch_config_t *config,
config->new_tags = NULL;
}
+enum notmuch_maildir_sync
+notmuch_config_get_maildir_sync (notmuch_config_t *config)
+{
+ if (config->maildir_sync == NOTMUCH_MAILDIR_SYNC_INVALID) {
+ config->maildir_sync =
+ g_key_file_get_integer (config->key_file,
+ "maildir", "sync_level", NULL);
+ }
+ return config->maildir_sync;
+}
+
+void
+notmuch_config_set_maildir_sync (notmuch_config_t *config,
+ enum notmuch_maildir_sync maildir_sync)
+{
+ g_key_file_set_integer (config->key_file,
+ "maildir", "sync_level", maildir_sync);
+ config->maildir_sync = maildir_sync;
+}
diff --git a/notmuch-new.c b/notmuch-new.c
index ed3f944..40ea610 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -45,6 +45,7 @@ typedef struct {
_filename_list_t *removed_files;
_filename_list_t *removed_directories;
+ enum notmuch_maildir_sync maildir_sync;
} add_files_state_t;
static volatile sig_atomic_t do_add_files_print_progress = 0;
@@ -410,11 +411,13 @@ add_files_recursive (notmuch_database_t *notmuch,
state->added_messages++;
for (tag=state->new_tags; *tag != NULL; tag++)
notmuch_message_add_tag (message, *tag);
- notmuch_message_maildir_to_tags (message, next);
+ if (state->maildir_sync >= NOTMUCH_MAILDIR_SYNC_NEW)
+ notmuch_message_maildir_to_tags (message, next);
break;
/* Non-fatal issues (go on to next file) */
case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
- notmuch_message_maildir_to_tags (message, next);
+ if (state->maildir_sync >= NOTMUCH_MAILDIR_SYNC_NEW_RENAMED)
+ notmuch_message_maildir_to_tags (message, next);
break;
case NOTMUCH_STATUS_FILE_NOT_EMAIL:
fprintf (stderr, "Note: Ignoring non-mail file: %s\n",
@@ -738,6 +741,7 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
return 1;
add_files_state.new_tags = notmuch_config_get_new_tags (config, &add_files_state.new_tags_length);
+ add_files_state.maildir_sync = notmuch_config_get_maildir_sync (config);
db_path = notmuch_config_get_database_path (config);
dot_notmuch_path = talloc_asprintf (ctx, "%s/%s", db_path, ".notmuch");
diff --git a/notmuch-restore.c b/notmuch-restore.c
index b0a4e1c..b5c5c48 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -41,6 +41,8 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
if (notmuch == NULL)
return 1;
+ notmuch_database_set_maildir_sync (notmuch,
+ notmuch_config_get_maildir_sync (config));
if (argc) {
input = fopen (argv[0], "r");
if (input == NULL) {
diff --git a/notmuch-setup.c b/notmuch-setup.c
index 955deb7..8364854 100644
--- a/notmuch-setup.c
+++ b/notmuch-setup.c
@@ -195,6 +195,23 @@ notmuch_setup_command (unused (void *ctx),
g_ptr_array_free (tags, TRUE);
}
+ printf ("\n"
+ "Notmuch can synchronize certain tags with maildir flags. You can\n"
+ "select between several levels of synchronization:\n"
+ "1 - No synchronization at all.\n"
+ "2 - 'notmuch new' tags the messages based on their maildir flags\n"
+ " only when it sees them for the first time.\n"
+ "3 - Same as 2 plus 'notmuch new' updates tags when it detects the\n"
+ " message was renamed.\n"
+ "4 - Same as 3 plus whenever message tags are changed, maildir\n"
+ " flags are updated accordingly.\n");
+
+ prompt ("Maildir synchronization level [%d]: ", notmuch_config_get_maildir_sync (config));
+ if (strlen (response) == 1 &&
+ response[0] >= '1' &&
+ response[0] <= '4')
+ notmuch_config_set_maildir_sync (config, atoi (response));
+
if (! notmuch_config_save (config)) {
if (is_new)
welcome_message_post_setup ();
diff --git a/notmuch-tag.c b/notmuch-tag.c
index fd54bc7..3a489a9 100644
--- a/notmuch-tag.c
+++ b/notmuch-tag.c
@@ -100,6 +100,8 @@ notmuch_tag_command (void *ctx, unused (int argc), unused (char *argv[]))
NOTMUCH_DATABASE_MODE_READ_WRITE);
if (notmuch == NULL)
return 1;
+ notmuch_database_set_maildir_sync (notmuch,
+ notmuch_config_get_maildir_sync (config));
query = notmuch_query_create (notmuch, query_string);
if (query == NULL) {
--
1.7.1.3.g75e44
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH 4/4] Tests for maildir synchronization
2010-10-13 12:13 [PATCH 0/4] Maildir synchronization v2 Michal Sojka
` (2 preceding siblings ...)
2010-10-13 12:13 ` [PATCH 3/4] Make maildir synchronization configurable Michal Sojka
@ 2010-10-13 12:13 ` Michal Sojka
2010-10-13 13:31 ` [PATCH 0/4] Maildir synchronization v2 Mike Kelly
` (2 subsequent siblings)
6 siblings, 0 replies; 20+ messages in thread
From: Michal Sojka @ 2010-10-13 12:13 UTC (permalink / raw)
To: notmuch
Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
---
test/maildir-sync | 216 +++++++++++++++++++++++++++++++++++++++++++++++++++++
test/notmuch-test | 2 +-
test/test-lib.sh | 14 +++-
3 files changed, 229 insertions(+), 3 deletions(-)
create mode 100755 test/maildir-sync
diff --git a/test/maildir-sync b/test/maildir-sync
new file mode 100755
index 0000000..a5f8c36
--- /dev/null
+++ b/test/maildir-sync
@@ -0,0 +1,216 @@
+#!/bin/bash
+
+test_description="Test maildir synchronization"
+
+. ./test-lib.sh
+
+filter_show() {
+ sed -e 's/, /,\n/g'|sed -e "s|${MAIL_DIR}/||" -e '/^"tags"/d'
+ echo
+}
+
+cat >> "$NOTMUCH_CONFIG" <<EOF
+[maildir]
+sync_level=4
+EOF
+
+test_begin_subtest "No new messages"
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail."
+
+cat > expected <<EOF
+Added 1 new message to the database.
+EOF
+test_expect_success "Add a message, no flags" '
+generate_message [subject]="\"test message\"" [date]="\"Sat, 01 Jan 2000 12:00:00 -0000\"" [filename]="\"msg-001:2,\"" &&
+NOTMUCH_NEW > actual &&
+test_cmp expected actual
+#emacs --eval "(gdb \"gdb --annotate=3 --args $(which notmuch) new\")"
+'
+cat > expected <<EOF
+thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test message (inbox unread)
+EOF
+test_expect_success 'Search for the message' '
+notmuch search tag:inbox and tag:unread | notmuch_search_sanitize > actual &&
+test_cmp expected actual
+'
+cat > expected <<EOF
+No new mail. Detected 1 file rename.
+EOF
+test_expect_success 'Add seen flag' '
+mv "${gen_msg_filename}" "${gen_msg_filename}S" &&
+increment_mtime "$(dirname "${gen_msg_filename}")" &&
+NOTMUCH_NEW > actual &&
+test_cmp expected actual
+'
+cat > expected <<EOF
+thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test message (inbox)
+EOF
+test_expect_success 'Check that tags were updated' '
+notmuch search tag:inbox and not tag:unread | notmuch_search_sanitize > actual &&
+test_cmp expected actual
+'
+cat > expected <<EOF
+Added 1 new message to the database.
+EOF
+test_expect_success "Add a seen message" '
+generate_message [subject]="\"test message 2\"" [date]="\"Sat, 01 Jan 2000 12:00:00 -0000\"" [filename]="\"msg-002:2,S\"" &&
+NOTMUCH_NEW > actual &&
+test_cmp expected actual
+'
+cat > expected <<EOF
+thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test message (inbox)
+thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test message 2 (inbox)
+EOF
+test_expect_success 'Check that the seen message is not tagged unread' '
+notmuch search tag:inbox and not tag:unread | notmuch_search_sanitize > actual &&
+test_cmp expected actual
+'
+test_expect_success 'Tag the seen messages as replied' '
+notmuch tag +replied -inbox tag:inbox and not tag:unread
+'
+
+cat > expected <<EOF
+msg-001:2,RS
+msg-002:2,RS
+EOF
+test_expect_success 'Check that R flag was added' '
+ls -1 "${MAIL_DIR}" > actual &&
+test_cmp expected actual
+'
+cat <<EOF > show-expected
+[[[{"id": "msg-001@notmuch-test-suite",
+"match": true,
+"filename": "msg-001:2,RS",
+"timestamp": 946728000,
+"date_relative": "2000-01-01",
+"headers": {"Subject": "test message",
+"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+"To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+"Cc": "",
+"Bcc": "",
+"Date": "Sat,
+01 Jan 2000 12:00:00 -0000"},
+"body": [{"id": 1,
+"content-type": "text/plain",
+"content": "This is just a test message (#1)\n"}]},
+[]]]]
+EOF
+
+test_expect_success 'Renamed message can be shown without running notmuch new' '
+notmuch show --format=json id:msg-001@notmuch-test-suite | filter_show > show-actual &&
+test_cmp show-expected show-actual
+'
+
+test_expect_success 'Test that we can reply to the renamed message' '
+notmuch reply id:msg-001@notmuch-test-suite
+'
+
+echo "No new mail." > expected
+test_expect_success 'No rename should be detected by notmuch new' '
+increment_mtime "$(dirname "${gen_msg_filename}")" &&
+notmuch new > actual &&
+test_cmp expected actual
+'
+test_expect_success "Add a message to new/ without info" '
+generate_message [subject]="\"test message 3\"" [date]="\"Sat, 01 Jan 2000 12:00:00 -0000\"" [dir]=new &&
+NOTMUCH_NEW > actual &&
+test_cmp - actual <<EOF
+Added 1 new message to the database.
+EOF
+'
+test_expect_success "Check that the message has inbox and unread tags" '
+notmuch search tag:inbox and tag:unread | notmuch_search_sanitize > actual &&
+test_cmp - actual <<EOF
+thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test message 3 (inbox unread)
+EOF
+'
+test_expect_success "Tag the message with 'tmp' tag" '
+notmuch tag +tmp tag:inbox and tag:unread'
+
+test_expect_success "Check that the message was not moved from new/ to cur/" '
+echo filename:$gen_msg_filename > expected &&
+notmuch show id:$gen_msg_id|grep -o "filename:.*$" > actual &&
+test_cmp expected actual &&
+test -f "$gen_msg_filename"
+'
+test_expect_success "Check that the message was not renamed" '
+ls "${MAIL_DIR}/new" > actual &&
+test_cmp - actual <<EOF
+msg-003
+EOF
+'
+test_expect_success 'Removing of unread tag should fail without cur/' '
+test_must_fail notmuch tag -unread tag:inbox and tag:unread
+'
+test_expect_success "Check that the tags were not changed" '
+notmuch search tag:inbox and tag:unread | notmuch_search_sanitize > actual &&
+test_cmp - actual <<EOF
+thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test message 3 (inbox tmp unread)
+EOF
+'
+
+# notmuch new is not necessary here, but we run it in order to check
+# for 'no rename' later (*).
+test_expect_success 'Create cur/ and let notmuch know about it' '
+mkdir "$MAIL_DIR/cur" &&
+notmuch new
+'
+test_expect_success 'Removing of unread tag should pass with cur/' '
+notmuch tag -unread tag:inbox and tag:unread
+'
+test_expect_success 'Check that the message was moved to cur/' '\
+ls "$MAIL_DIR/cur" > actual &&
+test_cmp - actual <<EOF
+msg-003:2,S
+EOF
+'
+test_expect_success 'No rename should be detected by notmuch new' '
+increment_mtime "$MAIL_DIR/cur" &&
+notmuch new > actual &&
+test_cmp - actual <<EOF
+No new mail.
+EOF
+'
+# (*) If notmuch new was not run we've got "Processed 1 file in almost
+# no time" here. The reason is that removing unread tag in a previous
+# test created directory document in the database but this document
+# was not linked as subdirectory of $MAIL_DIR. Therefore notmuch new
+# could not reach the cur/ directory and its files in it during
+# recurive traversal.
+test_expect_success 'Remove info from file name' '
+mv "$MAIL_DIR/cur/msg-003:2,S" "$MAIL_DIR/cur/msg-003" &&
+increment_mtime "$MAIL_DIR/cur" &&
+NOTMUCH_NEW > actual
+test_cmp - actual <<EOF
+No new mail. Detected 1 file rename.
+EOF
+'
+test_expect_success "Check that removing info did not change tags" '
+notmuch search tag:inbox | notmuch_search_sanitize > actual &&
+test_cmp - actual <<EOF
+thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test message 3 (inbox tmp)
+EOF
+'
+test_expect_success "Add a message to fakenew/ without info" '
+generate_message [subject]="\"test message 4\"" [date]="\"Sat, 01 Jan 2000 12:00:00 -0000\"" [dir]=fakenew &&
+NOTMUCH_NEW > actual &&
+test_cmp - actual <<EOF
+Added 1 new message to the database.
+EOF
+'
+test_expect_success "Check that the message has inbox and unread tags" '
+notmuch search tag:inbox and tag:unread | notmuch_search_sanitize > actual &&
+test_cmp - actual <<EOF
+thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test message 4 (inbox unread)
+EOF
+'
+test_expect_success 'Removing of unread tag should leave the message in fakenew/' '
+notmuch tag -unread tag:inbox and tag:unread &&
+ls "$MAIL_DIR/fakenew" > actual &&
+test_cmp - actual <<EOF
+msg-004:2,S
+EOF
+'
+
+test_done
diff --git a/test/notmuch-test b/test/notmuch-test
index 13c5d80..39e0fea 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -8,7 +8,7 @@
cd $(dirname "$0")
-TESTS="basic new search json thread-naming reply dump-restore uuencode thread-order author-order from-guessing long-id encoding"
+TESTS="basic new search json thread-naming reply dump-restore uuencode thread-order author-order from-guessing long-id encoding maildir-sync"
# Clean up any results from a previous run
rm -r test-results >/dev/null 2>/dev/null
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 5f7fa14..2036669 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -237,6 +237,12 @@ increment_mtime ()
# Generate the message in directory 'directory/of/choice' within
# the mail store. The directory will be created if necessary.
#
+# [filename]=name
+#
+# Store the message in file 'name'. The default is to store it
+# in 'msg-<count>', where <count> is three-digit number of the
+# message.
+#
# [body]=text
#
# Text to use as the body of the email message
@@ -273,10 +279,14 @@ generate_message ()
local additional_headers
gen_msg_cnt=$((gen_msg_cnt + 1))
- gen_msg_name=msg-$(printf "%03d" $gen_msg_cnt)
+ if [ -z "${template[filename]}" ]; then
+ gen_msg_name="msg-$(printf "%03d" $gen_msg_cnt)"
+ else
+ gen_msg_name=${template[filename]}
+ fi
if [ -z "${template[id]}" ]; then
- gen_msg_id="${gen_msg_name}@notmuch-test-suite"
+ gen_msg_id="${gen_msg_name%:2,*}@notmuch-test-suite"
else
gen_msg_id="${template[id]}"
fi
--
1.7.1.3.g75e44
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 12:13 [PATCH 0/4] Maildir synchronization v2 Michal Sojka
` (3 preceding siblings ...)
2010-10-13 12:13 ` [PATCH 4/4] Tests for maildir synchronization Michal Sojka
@ 2010-10-13 13:31 ` Mike Kelly
2010-10-13 14:24 ` Mike Kelly
2010-10-13 14:50 ` Servilio Afre Puentes
2010-10-18 14:12 ` Mike Kelly
6 siblings, 1 reply; 20+ messages in thread
From: Mike Kelly @ 2010-10-13 13:31 UTC (permalink / raw)
To: notmuch
On Wed, 13 Oct 2010 14:13:54 +0200
Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> Hi,
>
> this is an updated version of patches sent in
> id:1273580061-22580-1-git-send-email-sojkam1@fel.cvut.cz. Only the
> last patch (tests) was actually updated to work with new test suite.
> These patches has already been tested by several people and I think
> they are ready to be merged. They can be pulled by:
>
> git pull git://rtime.felk.cvut.cz/notmuch maildir-sync-v2
This sounds like just the sort of feature set I'd want from this sort
of patch, except unfortunately it won't compile for me on FreeBSD:
gcc -c -DNOTMUCH_VERSION=0.3.1-96-g6e0dca9 -O2 -Wall -Wextra -Wwrite-strings -Wswitch-enum -Wmissing-declarations -DHAVE_GETLINE=1 -D_REENTRANT -I/usr/local/include/gmime-2.4 -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include -DHAVE_VALGRIND=0 -DHAVE_STRCASESTR=1 -Icompat -Ilib -fPIC notmuch-config.c -o notmuch-config.o
notmuch-config.c:90: error: field 'maildir_sync' has incomplete type
notmuch-config.c: In function 'notmuch_config_open':
notmuch-config.c:241: error: 'NOTMUCH_MAILDIR_SYNC_INVALID' undeclared (first use in this function)
notmuch-config.c:241: error: (Each undeclared identifier is reported only once
notmuch-config.c:241: error: for each function it appears in.)
notmuch-config.c:335: error: invalid use of undefined type 'enum notmuch_maildir_sync'
notmuch-config.c:336: error: 'NOTMUCH_MAILDIR_SYNC_NONE' undeclared (first use in this function)
notmuch-config.c:336: error: type of formal parameter 2 is incomplete
notmuch-config.c: At top level:
notmuch-config.c:596: error: return type is an incomplete type
notmuch-config.c:596: error: conflicting types for 'notmuch_config_get_maildir_sync'
notmuch-client.h:195: error: previous declaration of 'notmuch_config_get_maildir_sync' was here
notmuch-config.c: In function 'notmuch_config_get_maildir_sync':
notmuch-config.c:597: error: 'NOTMUCH_MAILDIR_SYNC_INVALID' undeclared (first use in this function)
notmuch-config.c:602: warning: 'return' with a value, in function returning void
notmuch-config.c: At top level:
notmuch-config.c:607: error: parameter 2 ('maildir_sync') has incomplete type
notmuch-config.c: In function 'notmuch_config_set_maildir_sync':
notmuch-config.c:607: warning: unused parameter 'maildir_sync'
gmake: *** [notmuch-config.o] Error 1
This is with "gcc (GCC) 4.2.1 20070719 [FreeBSD]"
Switching to gcc44 seems to allow it to compile correctly, however notmuch didn't used to have that dependency.
--
Mike Kelly
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 13:31 ` [PATCH 0/4] Maildir synchronization v2 Mike Kelly
@ 2010-10-13 14:24 ` Mike Kelly
2010-10-13 14:35 ` Mike Kelly
0 siblings, 1 reply; 20+ messages in thread
From: Mike Kelly @ 2010-10-13 14:24 UTC (permalink / raw)
To: notmuch
On Wed, 13 Oct 2010 09:31:58 -0400
Mike Kelly <pioto@pioto.org> wrote:
> On Wed, 13 Oct 2010 14:13:54 +0200
> Michal Sojka <sojkam1@fel.cvut.cz> wrote:
>
> > Hi,
> >
> > this is an updated version of patches sent in
> > id:1273580061-22580-1-git-send-email-sojkam1@fel.cvut.cz. Only the
> > last patch (tests) was actually updated to work with new test suite.
> > These patches has already been tested by several people and I think
> > they are ready to be merged. They can be pulled by:
> >
> > git pull git://rtime.felk.cvut.cz/notmuch maildir-sync-v2
>
> This sounds like just the sort of feature set I'd want from this sort
> of patch, except unfortunately it won't compile for me on FreeBSD:
Looks like this may also require a newer xapian than i have now
(xapian-core 1.0.18), as notmuch new aborts with:
terminate called after throwing an instance of 'Xapian::InvalidArgumentError'
I'll see if I can track down anything more concrete as a cause.
--
Mike Kelly
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 14:24 ` Mike Kelly
@ 2010-10-13 14:35 ` Mike Kelly
2010-10-13 20:59 ` Michal Sojka
2010-12-09 15:11 ` Mike Kelly
0 siblings, 2 replies; 20+ messages in thread
From: Mike Kelly @ 2010-10-13 14:35 UTC (permalink / raw)
To: notmuch
On Wed, 13 Oct 2010 10:24:25 -0400
Mike Kelly <pioto@pioto.org> wrote:
> Looks like this may also require a newer xapian than i have now
> (xapian-core 1.0.18), as notmuch new aborts with:
>
> terminate called after throwing an instance of
> 'Xapian::InvalidArgumentError'
>
> I'll see if I can track down anything more concrete as a cause.
Well, the full backtrace looks like this:
#0 __cxa_throw (obj=0x2883d560, tinfo=0x8070278, dest=0x806132e <~InvalidArgumentError>)
at ../../.././../gcc-4.4-20100309/libstdc++-v3/libsupc++/eh_throw.cc:67
header = (__cxxabiv1::__cxa_refcounted_exception *) 0x2883d500
#1 0x28353b72 in Xapian::Document::Internal::remove_term () from /usr/local/lib/libxapian.so.21
No symbol table info available.
#2 0x28353c96 in Xapian::Document::remove_term () from /usr/local/lib/libxapian.so.21
No symbol table info available.
#3 0x080604bc in _notmuch_message_remove_term (message=0x28a85c10, prefix_name=0x806c887 "tag", value=0x806c564 "draft")
at lib/message.cc:737
term = 0x28806ab0 "Kdraft"
#4 0x0806086a in notmuch_message_remove_tag (message=0x28a85c10, tag=0x806c564 "draft") at lib/message.cc:832
private_status = 134662454
status = NOTMUCH_STATUS_SUCCESS
#5 0x08060a56 in notmuch_message_maildir_to_tags (message=0x28a85c10,
filename=0x28a860f0 "/usr/home/staff/mike/mail/staff-support/cur/1286944232_2.71920.pit84.pair.com,U=26762,FMD5=74eb4e66bae4700f6b79b81477ef9cfa:2,S") at lib/message.cc:889
flags = 0x28a8616e "S"
i = 0
status = NOTMUCH_STATUS_SUCCESS
p = 0x28a8616f ""
f = 83 'S'
valid = true
unread = true
#6 0x08050bbe in add_files_recursive (notmuch=0x2881e7f0, path=0x2881ec70 "/usr/home/staff/mike/mail/staff-support/cur",
state=0xbfbfe238) at notmuch-new.c:420
err = 32
dir = (DIR *) 0x0
entry = (struct dirent *) 0x28a68640
next = 0x28a860f0 "/usr/home/staff/mike/mail/staff-support/cur/1286944232_2.71920.pit84.pair.com,U=26762,FMD5=74eb4e66bae4700f6b79b81477ef9cfa:2,S"
fs_mtime = 1286978382
db_mtime = 1286944232
status = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID
ret = NOTMUCH_STATUS_SUCCESS
message = (notmuch_message_t *) 0x28a85c10
fs_entries = (struct dirent **) 0x28a6f000
i = 4915
num_fs_entries = 4922
directory = (notmuch_directory_t *) 0x288069f0
db_files = (notmuch_filenames_t *) 0x28806af0
db_subdirs = (notmuch_filenames_t *) 0x28806b30
st = {st_dev = 89, st_ino = 4172664, st_mode = 16832, st_nlink = 2, st_uid = 1118, st_gid = 0, st_rdev = 16689352,
st_atimespec = {tv_sec = 1286979943, tv_nsec = 0}, st_mtimespec = {tv_sec = 1286978382, tv_nsec = 0}, st_ctimespec = {
tv_sec = 1286978382, tv_nsec = 0}, st_size = 1205760, st_blocks = 2400, st_blksize = 4096, st_flags = 0, st_gen = 0, st_lspare = 0,
st_birthtimespec = {tv_sec = 1264441077, tv_nsec = 0}}
is_maildir = 0
new_directory = 0
tag = (const char **) 0x8069d62
#7 0x08050780 in add_files_recursive (notmuch=0x2881e7f0, path=0x2881e790 "/usr/home/staff/mike/mail/staff-support", state=0xbfbfe238)
at notmuch-new.c:302
dir = (DIR *) 0x0
entry = (struct dirent *) 0x288cc5b0
next = 0x2881ec70 "/usr/home/staff/mike/mail/staff-support/cur"
fs_mtime = 1264003223
db_mtime = 1264003223
status = NOTMUCH_STATUS_SUCCESS
ret = NOTMUCH_STATUS_SUCCESS
message = (notmuch_message_t *) 0x0
fs_entries = (struct dirent **) 0x2883d480
i = 2
num_fs_entries = 5
directory = (notmuch_directory_t *) 0x28806870
db_files = (notmuch_filenames_t *) 0x28806a30
db_subdirs = (notmuch_filenames_t *) 0x28806a70
st = {st_dev = 89, st_ino = 3889110, st_mode = 16832, st_nlink = 5, st_uid = 1118, st_gid = 0, st_rdev = 15524535,
st_atimespec = {tv_sec = 1286979943, tv_nsec = 0}, st_mtimespec = {tv_sec = 1264003223, tv_nsec = 0}, st_ctimespec = {
tv_sec = 1264473647, tv_nsec = 0}, st_size = 512, st_blocks = 4, st_blksize = 4096, st_flags = 0, st_gen = 0, st_lspare = 0,
st_birthtimespec = {tv_sec = 1264003223, tv_nsec = 0}}
is_maildir = 1
new_directory = 0
tag = (const char **) 0x8069d62
#8 0x08050780 in add_files_recursive (notmuch=0x2881e7f0, path=0x2881f680 "/usr/home/staff/mike/mail", state=0xbfbfe238)
at notmuch-new.c:302
dir = (DIR *) 0x0
entry = (struct dirent *) 0x28808900
next = 0x2881e790 "/usr/home/staff/mike/mail/staff-support"
fs_mtime = 1284781182
db_mtime = 1284781182
status = NOTMUCH_STATUS_SUCCESS
ret = NOTMUCH_STATUS_SUCCESS
message = (notmuch_message_t *) 0x0
fs_entries = (struct dirent **) 0x28825000
i = 40
num_fs_entries = 47
directory = (notmuch_directory_t *) 0x28806730
db_files = (notmuch_filenames_t *) 0x288068b0
db_subdirs = (notmuch_filenames_t *) 0x28806970
st = {st_dev = 89, st_ino = 3889107, st_mode = 16832, st_nlink = 47, st_uid = 1118, st_gid = 0, st_rdev = 15524532,
st_atimespec = {tv_sec = 1286979943, tv_nsec = 0}, st_mtimespec = {tv_sec = 1284781182, tv_nsec = 0}, st_ctimespec = {
tv_sec = 1284781182, tv_nsec = 0}, st_size = 1024, st_blocks = 4, st_blksize = 4096, st_flags = 0, st_gen = 0, st_lspare = 0,
st_birthtimespec = {tv_sec = 1264181508, tv_nsec = 0}}
is_maildir = 0
new_directory = 0
tag = (const char **) 0xbfbfe178
#9 0x08050fb5 in add_files (notmuch=0x2881e7f0, path=0x2881f680 "/usr/home/staff/mike/mail", state=0xbfbfe238) at notmuch-new.c:553
status = NOTMUCH_STATUS_SUCCESS
action = {__sigaction_u = {__sa_handler = 0x8050166 <handle_sigalrm>, __sa_sigaction = 0x8050166 <handle_sigalrm>}, sa_flags = 2,
sa_mask = {__bits = {0, 0, 0, 0}}}
timerval = {it_interval = {tv_sec = 1, tv_usec = 0}, it_value = {tv_sec = 1, tv_usec = 0}}
timer_is_active = 1
st = {st_dev = 89, st_ino = 3889107, st_mode = 16832, st_nlink = 47, st_uid = 1118, st_gid = 0, st_rdev = 15524532,
st_atimespec = {tv_sec = 1286979943, tv_nsec = 0}, st_mtimespec = {tv_sec = 1284781182, tv_nsec = 0}, st_ctimespec = {
tv_sec = 1284781182, tv_nsec = 0}, st_size = 1024, st_blocks = 4, st_blksize = 4096, st_flags = 0, st_gen = 0, st_lspare = 0,
st_birthtimespec = {tv_sec = 1264181508, tv_nsec = 0}}
#10 0x08051809 in notmuch_new_command (ctx=0x288090a0, argc=0, argv=0xbfbfe320) at notmuch-new.c:800
config = (notmuch_config_t *) 0x2881e490
notmuch = (notmuch_database_t *) 0x2881e7f0
add_files_state = {output_is_a_tty = 1, verbose = 0, new_tags = 0x288064b0, new_tags_length = 0, total_files = 0,
processed_files = 1, added_messages = 0, tv_start = {tv_sec = 1286980142, tv_usec = 691823}, removed_files = 0x288066b0,
removed_directories = 0x288066f0, maildir_sync = NOTMUCH_MAILDIR_SYNC_NEW_RENAMED_TAGGED}
elapsed = -0.12455515557656988
tv_now = {tv_sec = 671753656, tv_usec = -1077943692}
ret = 0
st = {st_dev = 89, st_ino = 3886724, st_mode = 16877, st_nlink = 3, st_uid = 1118, st_gid = 0, st_rdev = 15527616,
st_atimespec = {tv_sec = 1286953297, tv_nsec = 0}, st_mtimespec = {tv_sec = 1273174498, tv_nsec = 0}, st_ctimespec = {
tv_sec = 1273174498, tv_nsec = 0}, st_size = 512, st_blocks = 4, st_blksize = 4096, st_flags = 0, st_gen = 0, st_lspare = 0,
st_birthtimespec = {tv_sec = 1273174498, tv_nsec = 0}}
db_path = 0x2881f680 "/usr/home/staff/mike/mail"
dot_notmuch_path = 0x0
action = {__sigaction_u = {__sa_handler = 0x8050181 <handle_sigint>, __sa_sigaction = 0x8050181 <handle_sigint>}, sa_flags = 2,
sa_mask = {__bits = {0, 0, 0, 0}}}
f = (_filename_node_t *) 0x2d
renamed_files = 672185856
removed_files = 0
status = 134639189
i = 0
#11 0x0804edd6 in main (argc=2, argv=0xbfbfe318) at notmuch.c:499
local = (void *) 0x288090a0
command = (command_t *) 0x80708f4
i = 1
--
Mike Kelly
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 14:35 ` Mike Kelly
@ 2010-10-13 20:59 ` Michal Sojka
2010-10-13 22:03 ` Mike Kelly
2010-12-09 15:11 ` Mike Kelly
1 sibling, 1 reply; 20+ messages in thread
From: Michal Sojka @ 2010-10-13 20:59 UTC (permalink / raw)
To: Mike Kelly, notmuch
On Wed, 13 Oct 2010, Mike Kelly wrote:
> On Wed, 13 Oct 2010 10:24:25 -0400
> Mike Kelly <pioto@pioto.org> wrote:
>
> > Looks like this may also require a newer xapian than i have now
> > (xapian-core 1.0.18), as notmuch new aborts with:
> >
> > terminate called after throwing an instance of
> > 'Xapian::InvalidArgumentError'
> >
> > I'll see if I can track down anything more concrete as a cause.
>
> Well, the full backtrace looks like this:
So you were able to compile it? What was the problem?
>
> #0 __cxa_throw (obj=0x2883d560, tinfo=0x8070278, dest=0x806132e <~InvalidArgumentError>)
> at ../../.././../gcc-4.4-20100309/libstdc++-v3/libsupc++/eh_throw.cc:67
> header = (__cxxabiv1::__cxa_refcounted_exception *) 0x2883d500
> #1 0x28353b72 in Xapian::Document::Internal::remove_term () from /usr/local/lib/libxapian.so.21
> No symbol table info available.
> #2 0x28353c96 in Xapian::Document::remove_term () from /usr/local/lib/libxapian.so.21
> No symbol table info available.
> #3 0x080604bc in _notmuch_message_remove_term (message=0x28a85c10, prefix_name=0x806c887 "tag", value=0x806c564 "draft")
> at lib/message.cc:737
> term = 0x28806ab0 "Kdraft"
> #4 0x0806086a in notmuch_message_remove_tag (message=0x28a85c10, tag=0x806c564 "draft") at lib/message.cc:832
> private_status = 134662454
> status = NOTMUCH_STATUS_SUCCESS
I guess the problem would be that it removes nonexistent tag. I do not
experience such a problem on Linux with Xapian 1.2.3.
-Michal
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 14:35 ` Mike Kelly
2010-10-13 20:59 ` Michal Sojka
@ 2010-12-09 15:11 ` Mike Kelly
1 sibling, 0 replies; 20+ messages in thread
From: Mike Kelly @ 2010-12-09 15:11 UTC (permalink / raw)
To: notmuch
On Wed, Oct 13, 2010 at 10:35:15AM -0400, Mike Kelly wrote:
> On Wed, 13 Oct 2010 10:24:25 -0400
> Mike Kelly <pioto@pioto.org> wrote:
>
> > Looks like this may also require a newer xapian than i have now
> > (xapian-core 1.0.18), as notmuch new aborts with:
> >
> > terminate called after throwing an instance of
> > 'Xapian::InvalidArgumentError'
> >
> > I'll see if I can track down anything more concrete as a cause.
As a bit of a followup... I wasn't able to reproduce this on a different
FreeBSD 8.0 system, so I think my workstation must be subtly broken,
which is lame. And, if I set synchronize_flags=false in my
~/.notmuch-config, I don't get this error on my workstation, either.
--
Mike Kelly
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 12:13 [PATCH 0/4] Maildir synchronization v2 Michal Sojka
` (4 preceding siblings ...)
2010-10-13 13:31 ` [PATCH 0/4] Maildir synchronization v2 Mike Kelly
@ 2010-10-13 14:50 ` Servilio Afre Puentes
2010-10-13 17:26 ` David Bremner
2010-10-13 20:34 ` Michal Sojka
2010-10-18 14:12 ` Mike Kelly
6 siblings, 2 replies; 20+ messages in thread
From: Servilio Afre Puentes @ 2010-10-13 14:50 UTC (permalink / raw)
To: Michal Sojka; +Cc: notmuch
On 13 October 2010 08:13, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
[...]
> THERE IS CURRENTLY ONE KNOWN ISSUE: Viewing/storing of attachments of
> unread messages doesn't work. The reason is that when you view the
> message its unread tag is removed which causes the file to be renamed,
> but Emacs still uses the original name to access the attachment. You can
> workaround this by closing the message and opening it again. This issue
> will be fixed after we (I) implement "notmuch cat" command. With this
> command, emacs would not access the messages by the file name, but by
> running notmuch cat id:<message-id> which will always give the correct
> content.
Wouldn't it be more efficient to query notmuch for the filename using
the message ID we store in the DB? When network usage is implemented,
tramp can give us transparent remote file access in emacs.
Regards,
Servilio
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 14:50 ` Servilio Afre Puentes
@ 2010-10-13 17:26 ` David Bremner
2010-10-13 20:34 ` Michal Sojka
1 sibling, 0 replies; 20+ messages in thread
From: David Bremner @ 2010-10-13 17:26 UTC (permalink / raw)
To: Servilio Afre Puentes; +Cc: notmuch
On Wed, 13 Oct 2010 10:50:36 -0400, Servilio Afre Puentes <servilio@gmail.com> wrote:
>
> Wouldn't it be more efficient to query notmuch for the filename using
> the message ID we store in the DB? When network usage is implemented,
> tramp can give us transparent remote file access in emacs.
>
We can of course do both. From an implementation point of view, they
would both fall under "selective output in plain text" which various
people have been meaning to do, and I think one is no harder than the
other. On the other hand I doubt that there is an efficiency advantage
to using tramp, it is is more fragile (for example it depends on the
remote shell prompt).
d
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 14:50 ` Servilio Afre Puentes
2010-10-13 17:26 ` David Bremner
@ 2010-10-13 20:34 ` Michal Sojka
2010-10-14 2:26 ` Dirk Hohndel
1 sibling, 1 reply; 20+ messages in thread
From: Michal Sojka @ 2010-10-13 20:34 UTC (permalink / raw)
To: Servilio Afre Puentes; +Cc: notmuch
On Wed, 13 Oct 2010, Servilio Afre Puentes wrote:
> On 13 October 2010 08:13, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> [...]
> > THERE IS CURRENTLY ONE KNOWN ISSUE: Viewing/storing of attachments of
> > unread messages doesn't work. The reason is that when you view the
> > message its unread tag is removed which causes the file to be renamed,
> > but Emacs still uses the original name to access the attachment. You can
> > workaround this by closing the message and opening it again. This issue
> > will be fixed after we (I) implement "notmuch cat" command. With this
> > command, emacs would not access the messages by the file name, but by
> > running notmuch cat id:<message-id> which will always give the correct
> > content.
>
> Wouldn't it be more efficient to query notmuch for the filename using
> the message ID we store in the DB? When network usage is implemented,
> tramp can give us transparent remote file access in emacs.
Tramp would not be available in non-emacs GUIs. Also, when notmuch cat
is implemented, emacs gui will be usable remotely by simply defining
notmuch-command variable to contain the name of the script containing:
ssh user@host notmuch "$@"
-Michal
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 20:34 ` Michal Sojka
@ 2010-10-14 2:26 ` Dirk Hohndel
2010-10-14 7:11 ` Michal Sojka
0 siblings, 1 reply; 20+ messages in thread
From: Dirk Hohndel @ 2010-10-14 2:26 UTC (permalink / raw)
To: Michal Sojka, Servilio Afre Puentes; +Cc: notmuch
On Wed, 13 Oct 2010 22:34:34 +0200, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> On Wed, 13 Oct 2010, Servilio Afre Puentes wrote:
> > On 13 October 2010 08:13, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> > [...]
> > > THERE IS CURRENTLY ONE KNOWN ISSUE: Viewing/storing of attachments of
> > > unread messages doesn't work. The reason is that when you view the
> > > message its unread tag is removed which causes the file to be renamed,
> > > but Emacs still uses the original name to access the attachment. You can
> > > workaround this by closing the message and opening it again. This issue
> > > will be fixed after we (I) implement "notmuch cat" command. With this
> > > command, emacs would not access the messages by the file name, but by
> > > running notmuch cat id:<message-id> which will always give the correct
> > > content.
> >
> > Wouldn't it be more efficient to query notmuch for the filename using
> > the message ID we store in the DB? When network usage is implemented,
> > tramp can give us transparent remote file access in emacs.
>
> Tramp would not be available in non-emacs GUIs. Also, when notmuch cat
> is implemented, emacs gui will be usable remotely by simply defining
> notmuch-command variable to contain the name of the script containing:
>
> ssh user@host notmuch "$@"
That to me is certainly a very elegant solution... so what's stopping us
from implementing notmuch cat? No one taken the time to do it? Or is
there a design issue left to be resolved?
/D
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-14 2:26 ` Dirk Hohndel
@ 2010-10-14 7:11 ` Michal Sojka
0 siblings, 0 replies; 20+ messages in thread
From: Michal Sojka @ 2010-10-14 7:11 UTC (permalink / raw)
To: Dirk Hohndel, Servilio Afre Puentes; +Cc: notmuch
On Thu, 14 Oct 2010, Dirk Hohndel wrote:
> On Wed, 13 Oct 2010 22:34:34 +0200, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> >
> > ssh user@host notmuch "$@"
>
> That to me is certainly a very elegant solution... so what's stopping us
> from implementing notmuch cat? No one taken the time to do it?
Yes. I plan to work on this during the next days.
-Michal
> Or is there a design issue left to be resolved?
>
> /D
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-13 12:13 [PATCH 0/4] Maildir synchronization v2 Michal Sojka
` (5 preceding siblings ...)
2010-10-13 14:50 ` Servilio Afre Puentes
@ 2010-10-18 14:12 ` Mike Kelly
2010-10-22 4:43 ` Michal Sojka
6 siblings, 1 reply; 20+ messages in thread
From: Mike Kelly @ 2010-10-18 14:12 UTC (permalink / raw)
To: notmuch
On Wed, 13 Oct 2010 14:13:54 +0200
Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> Hi,
>
> this is an updated version of patches sent in
> id:1273580061-22580-1-git-send-email-sojkam1@fel.cvut.cz. Only the
> last patch (tests) was actually updated to work with new test suite.
> These patches has already been tested by several people and I think
> they are ready to be merged. They can be pulled by:
>
> git pull git://rtime.felk.cvut.cz/notmuch maildir-sync-v2
I've tested these patches on Linux, and they seem to work as expected.
However, I'd like to suggest/request that the level-4 syncing of a
filename should happen in notmuch_(message|thread)_(add|remove)_tag(),
not in notmuch-tag.cc. Otherwise, anyone using libnotmuch directly
(e.g. alternative clients, things using python bindings, etc) will not
benefit from this.
--
Mike Kelly
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH 0/4] Maildir synchronization v2
2010-10-18 14:12 ` Mike Kelly
@ 2010-10-22 4:43 ` Michal Sojka
0 siblings, 0 replies; 20+ messages in thread
From: Michal Sojka @ 2010-10-22 4:43 UTC (permalink / raw)
To: Mike Kelly, notmuch
On Mon, 18 Oct 2010, Mike Kelly wrote:
> Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> > they are ready to be merged. They can be pulled by:
> >
> > git pull git://rtime.felk.cvut.cz/notmuch maildir-sync-v2
>
> I've tested these patches on Linux, and they seem to work as expected.
> However, I'd like to suggest/request that the level-4 syncing of a
> filename should happen in notmuch_(message|thread)_(add|remove)_tag(),
> not in notmuch-tag.cc. Otherwise, anyone using libnotmuch directly
> (e.g. alternative clients, things using python bindings, etc) will not
> benefit from this.
Hi, I'd agree but these pathes don't touch notmuch-tag.c in a
significant way. What's done there is to read the value from config and
to call a function (from libnotmuch) to setup an internal variable
accordingly. Other users of libnotmuch have probably different means of
configuration than notmuch itself and should call this function in a
similar way.
-Michal
^ permalink raw reply [flat|nested] 20+ messages in thread