* [PATCH 1/4] Mailstore abstraction interface
2010-03-18 15:39 Mailstore abstraction & maildir synchronization Michal Sojka
@ 2010-03-18 15:39 ` Michal Sojka
0 siblings, 0 replies; 10+ messages in thread
From: Michal Sojka @ 2010-03-18 15:39 UTC (permalink / raw)
To: notmuch
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 23524 bytes --]
The goal of mailstore abstraction is to allow notmuch to store tags
together with email messages. The abstract interface is needed because
people want to use different ways of storing their emails. This
patchseries implements two types of mailstore - plain files and
maildir. It is expected that additional git-based mailstore will
follow.
This patch contains only the interface changes. No functionality is
added, removed or changed.
A new configuration group [mailstore] is defined. Currently, there is
only one key 'type' whose value determines the used mailstore. The
default value of this option is the plain-file mailstore currently
implemented in notmuch.
Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
---
lib/Makefile.local | 2 +
lib/database-private.h | 1 +
lib/database.cc | 15 ++++++--
lib/mailstore-files.c | 43 ++++++++++++++++++++++++
lib/mailstore-private.h | 59 ++++++++++++++++++++++++++++++++
lib/mailstore.c | 77 ++++++++++++++++++++++++++++++++++++++++++
lib/message.cc | 13 +++++++
lib/notmuch.h | 85 ++++++++++++++++++++++++++++++++++++++++++++--
notmuch-client.h | 7 ++++
notmuch-config.c | 34 +++++++++++++++++++
notmuch-count.c | 3 +-
notmuch-dump.c | 3 +-
notmuch-new.c | 6 ++-
notmuch-reply.c | 3 +-
notmuch-restore.c | 3 +-
notmuch-search-tags.c | 3 +-
notmuch-search.c | 3 +-
notmuch-show.c | 3 +-
notmuch-tag.c | 3 +-
19 files changed, 348 insertions(+), 18 deletions(-)
create mode 100644 lib/mailstore-files.c
create mode 100644 lib/mailstore-private.h
create mode 100644 lib/mailstore.c
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 495b27e..fc8883d 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -3,6 +3,8 @@ extra_cflags += -I$(dir)
libnotmuch_c_srcs = \
$(dir)/libsha1.c \
+ $(dir)/mailstore.c \
+ $(dir)/mailstore-files.c\
$(dir)/message-file.c \
$(dir)/messages.c \
$(dir)/sha1.c \
diff --git a/lib/database-private.h b/lib/database-private.h
index 41918d7..4499b1a 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,6 +49,7 @@ struct _notmuch_database {
Xapian::TermGenerator *term_gen;
Xapian::ValueRangeProcessor *value_range_processor;
+ notmuch_mailstore_t *mailstore;
};
/* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index c91e97c..93c8d0f 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
*/
#include "database-private.h"
+#include "mailstore-private.h"
#include <iostream>
@@ -438,7 +439,7 @@ parse_references (void *ctx,
}
notmuch_database_t *
-notmuch_database_create (const char *path)
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL;
@@ -474,7 +475,8 @@ notmuch_database_create (const char *path)
}
notmuch = notmuch_database_open (path,
- NOTMUCH_DATABASE_MODE_READ_WRITE);
+ NOTMUCH_DATABASE_MODE_READ_WRITE,
+ mailstore);
notmuch_database_upgrade (notmuch, NULL, NULL);
DONE:
@@ -497,7 +499,8 @@ _notmuch_database_ensure_writable (notmuch_database_t *notmuch)
notmuch_database_t *
notmuch_database_open (const char *path,
- notmuch_database_mode_t mode)
+ notmuch_database_mode_t mode,
+ notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL, *xapian_path = NULL;
@@ -605,6 +608,9 @@ notmuch_database_open (const char *path,
prefix_t *prefix = &PROBABILISTIC_PREFIX[i];
notmuch->query_parser->add_prefix (prefix->name, prefix->prefix);
}
+
+ notmuch->mailstore = mailstore;
+ mailstore->notmuch = notmuch;
} catch (const Xapian::Error &error) {
fprintf (stderr, "A Xapian exception occurred opening database: %s\n",
error.get_msg().c_str());
@@ -1493,7 +1499,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
DONE:
if (message) {
- if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
+ if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
*message_ret = message;
else
notmuch_message_destroy (message);
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
new file mode 100644
index 0000000..92d7f5d
--- /dev/null
+++ b/lib/mailstore-files.c
@@ -0,0 +1,43 @@
+/* mailstore-files.c - Original notmuch mail store - a collection of
+ * plain-text email messages (one message per file).
+ *
+ * Copyright © 2009 Carl Worth
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Authors: Carl Worth <cworth@cworth.org>
+ * Michal Sojka <sojkam1@fel.cvut.cz>
+ */
+
+#include "notmuch.h"
+#include "mailstore-private.h"
+
+
+static FILE *
+open_file(notmuch_mailstore_t *mailstore, const char *filename)
+{
+ const char *db_path, *abs_filename;
+
+ db_path = notmuch_database_get_path(mailstore->notmuch);
+ abs_filename = talloc_asprintf(mailstore, "%s/%s", db_path, filename);
+ if (unlikely(abs_filename == NULL))
+ return NULL;
+ return fopen (abs_filename, "r");
+}
+
+/* Original notmuch mail store */
+struct _notmuch_mailstore mailstore_files = {
+ .type = "files",
+ .open_file = open_file,
+};
diff --git a/lib/mailstore-private.h b/lib/mailstore-private.h
new file mode 100644
index 0000000..f606e3f
--- /dev/null
+++ b/lib/mailstore-private.h
@@ -0,0 +1,59 @@
+/* mailstore-private.h - Mailstore abstraction
+ *
+ * Copyright © 2010 Michal Sojka
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: Michal Sojka <sojkam1@fel.cvut.cz>
+ */
+
+#ifndef MAILSTORE_PRIVATE_H
+#define MAILSTORE_PRIVATE_H
+
+#include "notmuch-private.h"
+
+struct _notmuch_mailstore {
+ /* Constant fields */
+ const char *type; /* identification in .notmuch-config */
+ void *priv;
+
+ /* Count files in mailstore so that initial invocation of notmuch
+ * new can report remaining time */
+ void (*count_files)(notmuch_mailstore_t *mailstore,
+ const char *path, int *count,
+ volatile sig_atomic_t *interrupted);
+ /* Scan for new messages and add them to the database. */
+ notmuch_private_status_t (*index_new)(notmuch_mailstore_t *mailstore,
+ const char* path,
+ notmuch_indexing_context_t *ctx);
+ /* Store the tags assigned to the 'messages' to the
+ * 'mailstore'. */
+ notmuch_private_status_t (*sync_tags)(notmuch_mailstore_t *mailstore,
+ notmuch_message_t *message);
+ /* Return file handle from which the message can be read.
+ *
+ * Currently, this is not used and all message access is hardcoded
+ * to notmuch. We will need this operation for git-based and other
+ * non-filesystem mailstores. */
+ FILE * (*open_file)(notmuch_mailstore_t *mailstore,
+ const char *filename);
+
+ /* Run-time fields */
+ notmuch_database_t *notmuch;
+};
+
+extern struct _notmuch_mailstore mailstore_files;
+extern struct _notmuch_mailstore mailstore_maildir;
+
+#endif /* MAILSTORE_PRIVATE_H */
diff --git a/lib/mailstore.c b/lib/mailstore.c
new file mode 100644
index 0000000..cdac9a6
--- /dev/null
+++ b/lib/mailstore.c
@@ -0,0 +1,77 @@
+/* mailstore.c - Mailstore abstraction
+ *
+ * Copyright © 2010 Michal Sojka
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: Michal Sojka <sojkam1@fel.cvut.cz>
+ */
+
+#include <notmuch.h>
+#include "mailstore-private.h"
+
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
+
+static notmuch_mailstore_t *available[] = {
+ &mailstore_files,
+};
+
+notmuch_mailstore_t *
+notmuch_mailstore_get_by_type (const char *type)
+{
+ unsigned i;
+
+ if (type == NULL)
+ return available[0];
+
+ for (i = 0; i < ARRAY_SIZE(available); i++) {
+ if (strcasecmp (type, available[i]->type) == 0)
+ return available[i];
+ }
+ return NULL;
+}
+
+const char *
+notmuch_mailstore_get_type (notmuch_mailstore_t *mailstore)
+{
+ if (!mailstore)
+ return NULL;
+
+ return mailstore->type;
+}
+
+void
+notmuch_mailstore_count_files (notmuch_mailstore_t *mailstore,
+ const char *path, int *count,
+ volatile sig_atomic_t *interrupted)
+{
+ mailstore->count_files (mailstore, path, count, interrupted);
+}
+
+notmuch_status_t
+notmuch_mailstore_index_new (notmuch_mailstore_t *mailstore,
+ const char *path,
+ notmuch_indexing_context_t *ctx)
+{
+ notmuch_private_status_t status;
+ status = mailstore->index_new (mailstore, path, ctx);
+ return COERCE_STATUS (status, "Index new");
+}
+
+FILE *
+notmuch_mailstore_open_file (notmuch_mailstore_t *mailstore,
+ const char *filename)
+{
+ return mailstore->open_file (mailstore, filename);
+}
diff --git a/lib/message.cc b/lib/message.cc
index 721c9a6..c32ee7d 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -20,6 +20,7 @@
#include "notmuch-private.h"
#include "database-private.h"
+#include "mailstore-private.h"
#include <stdint.h>
@@ -555,10 +556,22 @@ void
_notmuch_message_sync (notmuch_message_t *message)
{
Xapian::WritableDatabase *db;
+ notmuch_private_status_t status;
if (message->notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY)
return;
+ if (message->notmuch->mailstore->sync_tags) {
+ status = message->notmuch->mailstore->sync_tags (message->notmuch->mailstore,
+ message);
+ if (status != NOTMUCH_PRIVATE_STATUS_SUCCESS) {
+ fprintf (stderr, "Error: Cannot sync tags to mailstore (%s)\n",
+ notmuch_status_to_string ((notmuch_status_t)status));
+ /* Exit to avoid unsynchronized mailstore. */
+ exit(1);
+ }
+ }
+
db = static_cast <Xapian::WritableDatabase *> (message->notmuch->xapian_db);
db->replace_document (message->doc_id, message->doc);
}
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 0d9cb0f..4f4cc8b 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -32,6 +32,9 @@
NOTMUCH_BEGIN_DECLS
#include <time.h>
+#include <sys/time.h> /* For struct timeval */
+#include <signal.h> /* For sig_atomic_t */
+#include <stdio.h> /* For FILE */
#ifndef FALSE
#define FALSE 0
@@ -119,6 +122,8 @@ typedef struct _notmuch_message notmuch_message_t;
typedef struct _notmuch_tags notmuch_tags_t;
typedef struct _notmuch_directory notmuch_directory_t;
typedef struct _notmuch_filenames notmuch_filenames_t;
+typedef struct _notmuch_mailstore notmuch_mailstore_t;
+
/* Create a new, empty notmuch database located at 'path'.
*
@@ -127,6 +132,8 @@ typedef struct _notmuch_filenames notmuch_filenames_t;
* create a new ".notmuch" directory within 'path' where notmuch will
* store its data.
*
+ * FIXME: Mailstore...
+ *
* After a successful call to notmuch_database_create, the returned
* database will be open so the caller should call
* notmuch_database_close when finished with it.
@@ -140,7 +147,7 @@ typedef struct _notmuch_filenames notmuch_filenames_t;
* an error message on stderr).
*/
notmuch_database_t *
-notmuch_database_create (const char *path);
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore);
typedef enum {
NOTMUCH_DATABASE_MODE_READ_ONLY = 0,
@@ -152,6 +159,8 @@ typedef enum {
/* Open an existing notmuch database located at 'path'.
*
+ * The database will index mails stored in 'mailstore'.
+ *
* The database should have been created at some time in the past,
* (not necessarily by this process), by calling
* notmuch_database_create with 'path'. By default the database should be
@@ -169,7 +178,8 @@ typedef enum {
*/
notmuch_database_t *
notmuch_database_open (const char *path,
- notmuch_database_mode_t mode);
+ notmuch_database_mode_t mode,
+ notmuch_mailstore_t *mailstore);
/* Close the given notmuch database, freeing all associated
* resources. See notmuch_database_open. */
@@ -236,7 +246,8 @@ notmuch_database_get_directory (notmuch_database_t *database,
* notmuch database will reference the filename, and will not copy the
* entire contents of the file.
*
- * If 'message' is not NULL, then, on successful return '*message'
+ * If 'message' is not NULL, then, on successful return
+ * (NOTMUCH_STATUS_SUCCESS or NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) '*message'
* will be initialized to a message object that can be used for things
* such as adding tags to the just-added message. The user should call
* notmuch_message_destroy when done with the message. On any failure
@@ -495,7 +506,7 @@ notmuch_threads_destroy (notmuch_threads_t *threads);
*/
unsigned
notmuch_query_count_messages (notmuch_query_t *query);
-
+
/* Get the thread ID of 'thread'.
*
* The returned string belongs to 'thread' and as such, should not be
@@ -1104,6 +1115,72 @@ notmuch_filenames_move_to_next (notmuch_filenames_t *filenames);
void
notmuch_filenames_destroy (notmuch_filenames_t *filenames);
+/* Returns notmuch_mailstore_t object of a given 'type'.
+ *
+ * The 'type' is case insensitive. If 'type' is NULL, a default
+ * mailstore is returned.
+ *
+ * When no available mailstore matches 'type', NULL is returned.
+ */
+notmuch_mailstore_t *
+notmuch_mailstore_get_by_type (const char *type);
+
+/* Returns the type of a mailstore.
+ *
+ * The returned string is owned by mailstore so should not be modified
+ * nor freed by the caller.
+ */
+const char *
+notmuch_mailstore_get_type (notmuch_mailstore_t *mailstore);
+
+/* Count emails in mailstore.
+ *
+ * This function returns when interrupted is set to non-zero value.
+ */
+void
+notmuch_mailstore_count_files (notmuch_mailstore_t *mailstore,
+ const char *path, int *count,
+ volatile sig_atomic_t *interrupted);
+
+typedef struct notmuch_indexing_context {
+ int output_is_a_tty;
+ int verbose;
+
+ int total_files;
+ int processed_files;
+ int added_messages;
+ int renamed_files;
+ int removed_files;
+ struct timeval tv_start;
+
+ void *priv;
+
+ volatile sig_atomic_t interrupted;
+ volatile sig_atomic_t print_progress;
+ void (*print_progress_cb)(struct notmuch_indexing_context *ctx);
+
+} notmuch_indexing_context_t;
+
+
+/* Index new mails in the mailstore.
+ *
+ * Mailstore is responsible for detecting new emails, adding them to
+ * the database and tagging them with tags stored in the mailstore.
+ */
+notmuch_status_t
+notmuch_mailstore_index_new (notmuch_mailstore_t *mailstore,
+ const char *path,
+ notmuch_indexing_context_t *ctx);
+
+/* Returns file handle to the file in the mailstore.
+ *
+ * TODO: This function is currently not used and access to mails is
+ * hardcoded in other parts of notmuch.
+ */
+FILE *
+notmuch_mailstore_open_file (notmuch_mailstore_t *mailstore,
+ const char *filename);
+
NOTMUCH_END_DECLS
#endif
diff --git a/notmuch-client.h b/notmuch-client.h
index c80b39c..4794b8e 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -171,6 +171,13 @@ notmuch_config_set_user_other_email (notmuch_config_t *config,
const char *other_email[],
size_t length);
+notmuch_mailstore_t *
+notmuch_config_get_mailstore(notmuch_config_t *config);
+
+void
+notmuch_config_set_mailstore(notmuch_config_t *config,
+ notmuch_mailstore_t *mailstore);
+
notmuch_bool_t
debugger_is_active (void);
diff --git a/notmuch-config.c b/notmuch-config.c
index 95430db..299423a 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -62,6 +62,7 @@ struct _notmuch_config {
char *user_primary_email;
char **user_other_email;
size_t user_other_email_length;
+ notmuch_mailstore_t *mailstore;
};
static int
@@ -199,6 +200,7 @@ notmuch_config_open (void *ctx,
config->user_primary_email = NULL;
config->user_other_email = NULL;
config->user_other_email_length = 0;
+ config->mailstore = NULL;
if (! g_key_file_load_from_file (config->key_file,
config->filename,
@@ -264,6 +266,10 @@ notmuch_config_open (void *ctx,
}
}
+ if (notmuch_config_get_mailstore (config) == NULL) {
+ notmuch_config_set_mailstore(config, notmuch_mailstore_get_by_type(NULL));
+ }
+
/* When we create a new configuration file here, we add some
* comments to help the user understand what can be done. */
if (is_new) {
@@ -452,3 +458,31 @@ notmuch_config_set_user_other_email (notmuch_config_t *config,
talloc_free (config->user_other_email);
config->user_other_email = NULL;
}
+
+notmuch_mailstore_t *
+notmuch_config_get_mailstore(notmuch_config_t *config)
+{
+ char *type;
+
+ if (config->mailstore == NULL) {
+ type = g_key_file_get_string (config->key_file,
+ "mailstore", "type", NULL);
+ if (type) {
+ config->mailstore = notmuch_mailstore_get_by_type(type);
+ free (type);
+ }
+ }
+
+ return config->mailstore;
+}
+
+void
+notmuch_config_set_mailstore(notmuch_config_t *config,
+ notmuch_mailstore_t *mailstore)
+{
+ g_key_file_set_string (config->key_file,
+ "mailstore", "type",
+ notmuch_mailstore_get_type(mailstore));
+
+ config->mailstore = NULL;
+}
diff --git a/notmuch-count.c b/notmuch-count.c
index 77aa433..631b8f8 100644
--- a/notmuch-count.c
+++ b/notmuch-count.c
@@ -81,7 +81,8 @@ notmuch_count_command (void *ctx, int argc, char *argv[])
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore(config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-dump.c b/notmuch-dump.c
index 7e7bc17..e74dfcb 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -36,7 +36,8 @@ notmuch_dump_command (unused (void *ctx), int argc, char *argv[])
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-new.c b/notmuch-new.c
index 44b50aa..2d0ba6c 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -749,11 +749,13 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
return 1;
printf ("Found %d total files (that's not much mail).\n", count);
- notmuch = notmuch_database_create (db_path);
+ notmuch = notmuch_database_create (db_path,
+ notmuch_config_get_mailstore (config));
add_files_state.total_files = count;
} else {
notmuch = notmuch_database_open (db_path,
- NOTMUCH_DATABASE_MODE_READ_WRITE);
+ NOTMUCH_DATABASE_MODE_READ_WRITE,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 6c15536..0d6b7cf 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -468,7 +468,8 @@ notmuch_reply_command (void *ctx, int argc, char *argv[])
}
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-restore.c b/notmuch-restore.c
index b0a4e1c..97d817b 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -37,7 +37,8 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_WRITE);
+ NOTMUCH_DATABASE_MODE_READ_WRITE,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-search-tags.c b/notmuch-search-tags.c
index 6f3cfcc..323f416 100644
--- a/notmuch-search-tags.c
+++ b/notmuch-search-tags.c
@@ -52,7 +52,8 @@ notmuch_search_tags_command (void *ctx, int argc, char *argv[])
}
db = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (db == NULL) {
goto error;
}
diff --git a/notmuch-search.c b/notmuch-search.c
index 4e3514b..d57479d 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -245,7 +245,8 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-show.c b/notmuch-show.c
index ff1fecb..26581c1 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -458,7 +458,8 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[]))
}
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-tag.c b/notmuch-tag.c
index 8b6f7dc..5e1a5c3 100644
--- a/notmuch-tag.c
+++ b/notmuch-tag.c
@@ -97,7 +97,8 @@ notmuch_tag_command (void *ctx, unused (int argc), unused (char *argv[]))
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_WRITE);
+ NOTMUCH_DATABASE_MODE_READ_WRITE,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
--
1.7.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 0/4] Mailstore abstraction v4
@ 2010-04-08 14:42 Michal Sojka
2010-04-08 14:42 ` [PATCH 1/4] Mailstore abstraction interface Michal Sojka
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: Michal Sojka @ 2010-04-08 14:42 UTC (permalink / raw)
To: notmuch
Hi all,
this is the fourth version of my mailstore abstraction series. I split
it into two parts, with the first part being sent here. I think that
this part becomes mostly ready for merging.
From the user's point of view, it adds only the 'cat' subcommand and
modifies the .el to use it. Thanks to this change it is easy to use
Emacs client with the database accessed remotely over SSH. The only
additional patch that must be applied is
id:m1my03gsmu.fsf@watt.gilman.jhu.edu. The rebased version is at
http://rtime.felk.cvut.cz/gitweb/notmuch.git/shortlog/refs/heads/jr/quote-args-in-notmuch-show
and can be pulled by 'git pull git://rtime.felk.cvut.cz/notmuch.git
jr/quote-args-in-notmuch-show').
More importantly, there are bigger changes for developers. Since it
might be quite painful to rebase these patches to quickly changing
master (as of the last few days) I'd like to have these merged ASAP.
So the patches here are the following:
Michal Sojka (4):
Mailstore abstraction interface
Conversion to mailstore abstraction
Access messages through mail store interface
Add 'cat' subcommand
My biggest question relates to the first patch, which does an
incompatible change to libnotmuch API. After reading RELEASING file, I
found that this change is probably not what Carl wants to merge (and I
understand that) so I'd like to get some feedback on my suggestion in
that patch.
The subsequent patches are quite straightforward and the comments
there should describe what these patches do.
These patches are also located at git://rtime.felk.cvut.cz/notmuch.git
and tagged by mailstore-abstraction-v4-part1.
This is the overall diffstat:
NEWS | 3 +
emacs/notmuch-show.el | 11 +-
lib/Makefile.local | 2 +
lib/database-private.h | 1 +
lib/database.cc | 29 +-
lib/index.cc | 8 +-
notmuch-new.c => lib/mailstore-files.c | 418 +++++----------------
lib/mailstore-private.h | 59 +++
lib/mailstore.c | 80 ++++
lib/message-file.c | 8 +-
lib/message.cc | 46 ++-
lib/notmuch-private.h | 6 +-
lib/notmuch.h | 98 +++++-
lib/sha1.c | 6 +-
notmuch-client.h | 12 +-
notmuch-config.c | 34 ++
notmuch-count.c | 3 +-
notmuch-dump.c | 3 +-
notmuch-new.c | 658 +++-----------------------------
notmuch-reply.c | 13 +-
notmuch-restore.c | 3 +-
notmuch-search-tags.c | 3 +-
notmuch-search.c | 3 +-
notmuch-show.c | 82 ++++-
notmuch-tag.c | 3 +-
notmuch.c | 4 +
show-message.c | 14 +-
27 files changed, 621 insertions(+), 989 deletions(-)
copy notmuch-new.c => lib/mailstore-files.c (61%)
create mode 100644 lib/mailstore-private.h
create mode 100644 lib/mailstore.c
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/4] Mailstore abstraction interface
2010-04-08 14:42 [PATCH 0/4] Mailstore abstraction v4 Michal Sojka
@ 2010-04-08 14:42 ` Michal Sojka
2010-04-13 17:53 ` Carl Worth
2010-04-08 14:42 ` [PATCH 2/4] Conversion to mailstore abstraction Michal Sojka
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Michal Sojka @ 2010-04-08 14:42 UTC (permalink / raw)
To: notmuch
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 22672 bytes --]
The goal of mailstore abstraction is to allow notmuch to store tags
together with email messages. The abstract interface is needed because
people want to use different ways of storing their emails. Currently,
there exists implementation for two types of mailstore - plain files
and maildir. It is expected that additional git-based mailstore will
be developed later.
This patch contains only the interface changes. No functionality is
added, removed or changed.
A new configuration group [mailstore] is defined. Currently, there is
only one key called 'type' whose value determines the used mailstore.
The default value of this option (files) is the plain-file mailstore
currently implemented in notmuch.
This patch changes libnotmuch API (and ABI). The functions
notmuch_database_create() and notmuch_database_open() need additional
parameter to identify the type of mailstore, which is used to access
the messages.
If we want to preserve the API, it would be necessary to encode the
mailstore type into the 'path' parameter. For example the value
<path>#<type> (or <type>://<path>?) would mean use the mailstore of
given 'type' located at 'path'. If we cannot find the type we would
assuse the default mailstore.
Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
---
lib/Makefile.local | 1 +
lib/database-private.h | 1 +
lib/database.cc | 15 ++++++--
lib/mailstore-private.h | 59 ++++++++++++++++++++++++++++++++
lib/mailstore.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++
lib/message.cc | 13 +++++++
lib/notmuch.h | 82 ++++++++++++++++++++++++++++++++++++++++----
notmuch-client.h | 7 ++++
notmuch-config.c | 34 +++++++++++++++++++
notmuch-count.c | 3 +-
notmuch-dump.c | 3 +-
notmuch-new.c | 6 ++-
notmuch-reply.c | 3 +-
notmuch-restore.c | 3 +-
notmuch-search-tags.c | 3 +-
notmuch-search.c | 3 +-
notmuch-show.c | 6 ++-
notmuch-tag.c | 3 +-
18 files changed, 307 insertions(+), 23 deletions(-)
create mode 100644 lib/mailstore-private.h
create mode 100644 lib/mailstore.c
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 0e3a4d1..6243af1 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -31,6 +31,7 @@ extra_cflags += -I$(dir) -fPIC
libnotmuch_c_srcs = \
$(dir)/libsha1.c \
+ $(dir)/mailstore.c \
$(dir)/message-file.c \
$(dir)/messages.c \
$(dir)/sha1.c \
diff --git a/lib/database-private.h b/lib/database-private.h
index 41918d7..4499b1a 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,6 +49,7 @@ struct _notmuch_database {
Xapian::TermGenerator *term_gen;
Xapian::ValueRangeProcessor *value_range_processor;
+ notmuch_mailstore_t *mailstore;
};
/* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index c91e97c..93c8d0f 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
*/
#include "database-private.h"
+#include "mailstore-private.h"
#include <iostream>
@@ -438,7 +439,7 @@ parse_references (void *ctx,
}
notmuch_database_t *
-notmuch_database_create (const char *path)
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL;
@@ -474,7 +475,8 @@ notmuch_database_create (const char *path)
}
notmuch = notmuch_database_open (path,
- NOTMUCH_DATABASE_MODE_READ_WRITE);
+ NOTMUCH_DATABASE_MODE_READ_WRITE,
+ mailstore);
notmuch_database_upgrade (notmuch, NULL, NULL);
DONE:
@@ -497,7 +499,8 @@ _notmuch_database_ensure_writable (notmuch_database_t *notmuch)
notmuch_database_t *
notmuch_database_open (const char *path,
- notmuch_database_mode_t mode)
+ notmuch_database_mode_t mode,
+ notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL, *xapian_path = NULL;
@@ -605,6 +608,9 @@ notmuch_database_open (const char *path,
prefix_t *prefix = &PROBABILISTIC_PREFIX[i];
notmuch->query_parser->add_prefix (prefix->name, prefix->prefix);
}
+
+ notmuch->mailstore = mailstore;
+ mailstore->notmuch = notmuch;
} catch (const Xapian::Error &error) {
fprintf (stderr, "A Xapian exception occurred opening database: %s\n",
error.get_msg().c_str());
@@ -1493,7 +1499,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
DONE:
if (message) {
- if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
+ if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
*message_ret = message;
else
notmuch_message_destroy (message);
diff --git a/lib/mailstore-private.h b/lib/mailstore-private.h
new file mode 100644
index 0000000..f606e3f
--- /dev/null
+++ b/lib/mailstore-private.h
@@ -0,0 +1,59 @@
+/* mailstore-private.h - Mailstore abstraction
+ *
+ * Copyright © 2010 Michal Sojka
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: Michal Sojka <sojkam1@fel.cvut.cz>
+ */
+
+#ifndef MAILSTORE_PRIVATE_H
+#define MAILSTORE_PRIVATE_H
+
+#include "notmuch-private.h"
+
+struct _notmuch_mailstore {
+ /* Constant fields */
+ const char *type; /* identification in .notmuch-config */
+ void *priv;
+
+ /* Count files in mailstore so that initial invocation of notmuch
+ * new can report remaining time */
+ void (*count_files)(notmuch_mailstore_t *mailstore,
+ const char *path, int *count,
+ volatile sig_atomic_t *interrupted);
+ /* Scan for new messages and add them to the database. */
+ notmuch_private_status_t (*index_new)(notmuch_mailstore_t *mailstore,
+ const char* path,
+ notmuch_indexing_context_t *ctx);
+ /* Store the tags assigned to the 'messages' to the
+ * 'mailstore'. */
+ notmuch_private_status_t (*sync_tags)(notmuch_mailstore_t *mailstore,
+ notmuch_message_t *message);
+ /* Return file handle from which the message can be read.
+ *
+ * Currently, this is not used and all message access is hardcoded
+ * to notmuch. We will need this operation for git-based and other
+ * non-filesystem mailstores. */
+ FILE * (*open_file)(notmuch_mailstore_t *mailstore,
+ const char *filename);
+
+ /* Run-time fields */
+ notmuch_database_t *notmuch;
+};
+
+extern struct _notmuch_mailstore mailstore_files;
+extern struct _notmuch_mailstore mailstore_maildir;
+
+#endif /* MAILSTORE_PRIVATE_H */
diff --git a/lib/mailstore.c b/lib/mailstore.c
new file mode 100644
index 0000000..eb27952
--- /dev/null
+++ b/lib/mailstore.c
@@ -0,0 +1,85 @@
+/* mailstore.c - Mailstore abstraction
+ *
+ * Copyright © 2010 Michal Sojka
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: Michal Sojka <sojkam1@fel.cvut.cz>
+ */
+
+#include <notmuch.h>
+#include "mailstore-private.h"
+
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
+
+/* Original notmuch mail store */
+struct _notmuch_mailstore mailstore_files = {
+ .type = "files",
+};
+
+static notmuch_mailstore_t *available[] = {
+ &mailstore_files,
+};
+
+notmuch_mailstore_t *
+notmuch_mailstore_get_by_type (const char *type)
+{
+ unsigned i;
+
+ if (type == NULL)
+ return available[0];
+
+ for (i = 0; i < ARRAY_SIZE(available); i++) {
+ if (strcasecmp (type, available[i]->type) == 0)
+ return available[i];
+ }
+ return NULL;
+}
+
+const char *
+notmuch_mailstore_get_type (notmuch_mailstore_t *mailstore)
+{
+ if (!mailstore)
+ return NULL;
+
+ return mailstore->type;
+}
+
+void
+notmuch_mailstore_count_files (notmuch_mailstore_t *mailstore,
+ const char *path, int *count,
+ volatile sig_atomic_t *interrupted)
+{
+ mailstore->count_files (mailstore, path, count, interrupted);
+}
+
+notmuch_status_t
+notmuch_mailstore_index_new (notmuch_mailstore_t *mailstore,
+ const char *path,
+ notmuch_indexing_context_t *ctx)
+{
+ notmuch_private_status_t status;
+ status = mailstore->index_new (mailstore, path, ctx);
+ return COERCE_STATUS (status, "Index new");
+}
+
+FILE *
+notmuch_mailstore_open_file (notmuch_mailstore_t *mailstore,
+ const char *filename)
+{
+ if (mailstore->open_file)
+ return mailstore->open_file (mailstore, filename);
+ else
+ return NULL;
+}
diff --git a/lib/message.cc b/lib/message.cc
index 721c9a6..c32ee7d 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -20,6 +20,7 @@
#include "notmuch-private.h"
#include "database-private.h"
+#include "mailstore-private.h"
#include <stdint.h>
@@ -555,10 +556,22 @@ void
_notmuch_message_sync (notmuch_message_t *message)
{
Xapian::WritableDatabase *db;
+ notmuch_private_status_t status;
if (message->notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY)
return;
+ if (message->notmuch->mailstore->sync_tags) {
+ status = message->notmuch->mailstore->sync_tags (message->notmuch->mailstore,
+ message);
+ if (status != NOTMUCH_PRIVATE_STATUS_SUCCESS) {
+ fprintf (stderr, "Error: Cannot sync tags to mailstore (%s)\n",
+ notmuch_status_to_string ((notmuch_status_t)status));
+ /* Exit to avoid unsynchronized mailstore. */
+ exit(1);
+ }
+ }
+
db = static_cast <Xapian::WritableDatabase *> (message->notmuch->xapian_db);
db->replace_document (message->doc_id, message->doc);
}
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 88da078..31e47a4 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -32,6 +32,8 @@
NOTMUCH_BEGIN_DECLS
#include <time.h>
+#include <signal.h> /* For sig_atomic_t */
+#include <stdio.h> /* For FILE */
#ifndef FALSE
#define FALSE 0
@@ -119,13 +121,15 @@ typedef struct _notmuch_message notmuch_message_t;
typedef struct _notmuch_tags notmuch_tags_t;
typedef struct _notmuch_directory notmuch_directory_t;
typedef struct _notmuch_filenames notmuch_filenames_t;
+typedef struct _notmuch_mailstore notmuch_mailstore_t;
+
/* Create a new, empty notmuch database located at 'path'.
*
- * The path should be a top-level directory to a collection of
- * plain-text email messages (one message per file). This call will
- * create a new ".notmuch" directory within 'path' where notmuch will
- * store its data.
+ * The path should be a top-level directory to a mail store e.g. to a
+ * collection of plain-text email messages (one message per file).
+ * This call will create a new ".notmuch" directory within 'path'
+ * where notmuch will store its data.
*
* After a successful call to notmuch_database_create, the returned
* database will be open so the caller should call
@@ -140,7 +144,7 @@ typedef struct _notmuch_filenames notmuch_filenames_t;
* an error message on stderr).
*/
notmuch_database_t *
-notmuch_database_create (const char *path);
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore);
typedef enum {
NOTMUCH_DATABASE_MODE_READ_ONLY = 0,
@@ -169,7 +173,8 @@ typedef enum {
*/
notmuch_database_t *
notmuch_database_open (const char *path,
- notmuch_database_mode_t mode);
+ notmuch_database_mode_t mode,
+ notmuch_mailstore_t *mailstore);
/* Close the given notmuch database, freeing all associated
* resources. See notmuch_database_open. */
@@ -236,7 +241,8 @@ notmuch_database_get_directory (notmuch_database_t *database,
* notmuch database will reference the filename, and will not copy the
* entire contents of the file.
*
- * If 'message' is not NULL, then, on successful return '*message'
+ * If 'message' is not NULL, then, on successful return
+ * (NOTMUCH_STATUS_SUCCESS or NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) '*message'
* will be initialized to a message object that can be used for things
* such as adding tags to the just-added message. The user should call
* notmuch_message_destroy when done with the message. On any failure
@@ -496,7 +502,7 @@ notmuch_threads_destroy (notmuch_threads_t *threads);
*/
unsigned
notmuch_query_count_messages (notmuch_query_t *query);
-
+
/* Get the thread ID of 'thread'.
*
* The returned string belongs to 'thread' and as such, should not be
@@ -1105,6 +1111,66 @@ notmuch_filenames_move_to_next (notmuch_filenames_t *filenames);
void
notmuch_filenames_destroy (notmuch_filenames_t *filenames);
+/* Returns notmuch_mailstore_t object of a given 'type'.
+ *
+ * The 'type' is case insensitive. If 'type' is NULL, a default
+ * mailstore is returned.
+ *
+ * When no available mailstore matches 'type', NULL is returned.
+ */
+notmuch_mailstore_t *
+notmuch_mailstore_get_by_type (const char *type);
+
+/* Returns the type of a mailstore.
+ *
+ * The returned string is owned by mailstore so should not be modified
+ * nor freed by the caller.
+ */
+const char *
+notmuch_mailstore_get_type (notmuch_mailstore_t *mailstore);
+
+/* Count emails in mailstore.
+ *
+ * This function returns when interrupted is set to non-zero value.
+ */
+void
+notmuch_mailstore_count_files (notmuch_mailstore_t *mailstore,
+ const char *path, int *count,
+ volatile sig_atomic_t *interrupted);
+
+typedef struct notmuch_indexing_context {
+ int processed_files;
+ int added_messages;
+ int renamed_files;
+ int removed_files;
+
+ void *priv; /* Private field for indexing function */
+
+ int verbose;
+ volatile sig_atomic_t interrupted;
+ volatile sig_atomic_t print_progress;
+ void (*print_progress_cb)(struct notmuch_indexing_context *ctx);
+ void (*print_verbose_cb)(struct notmuch_indexing_context *ctx, const char *filename);
+ void *print_ctx; /* Private field for printing functions */
+} notmuch_indexing_context_t;
+
+
+/* Index new mails in the mailstore.
+ *
+ * Mailstore is responsible for detecting new emails, adding them to
+ * the database and tagging them with tags stored in the mailstore.
+ */
+notmuch_status_t
+notmuch_mailstore_index_new (notmuch_mailstore_t *mailstore,
+ const char *path,
+ notmuch_indexing_context_t *ctx);
+
+/* Returns file handle to the file in the mailstore.
+ */
+FILE *
+notmuch_mailstore_open_file (notmuch_mailstore_t *mailstore,
+ const char *filename);
+
NOTMUCH_END_DECLS
#endif
diff --git a/notmuch-client.h b/notmuch-client.h
index d36b9ec..d8c8df4 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -183,6 +183,13 @@ notmuch_config_set_user_other_email (notmuch_config_t *config,
const char *other_email[],
size_t length);
+notmuch_mailstore_t *
+notmuch_config_get_mailstore(notmuch_config_t *config);
+
+void
+notmuch_config_set_mailstore(notmuch_config_t *config,
+ notmuch_mailstore_t *mailstore);
+
notmuch_bool_t
debugger_is_active (void);
diff --git a/notmuch-config.c b/notmuch-config.c
index 95430db..299423a 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -62,6 +62,7 @@ struct _notmuch_config {
char *user_primary_email;
char **user_other_email;
size_t user_other_email_length;
+ notmuch_mailstore_t *mailstore;
};
static int
@@ -199,6 +200,7 @@ notmuch_config_open (void *ctx,
config->user_primary_email = NULL;
config->user_other_email = NULL;
config->user_other_email_length = 0;
+ config->mailstore = NULL;
if (! g_key_file_load_from_file (config->key_file,
config->filename,
@@ -264,6 +266,10 @@ notmuch_config_open (void *ctx,
}
}
+ if (notmuch_config_get_mailstore (config) == NULL) {
+ notmuch_config_set_mailstore(config, notmuch_mailstore_get_by_type(NULL));
+ }
+
/* When we create a new configuration file here, we add some
* comments to help the user understand what can be done. */
if (is_new) {
@@ -452,3 +458,31 @@ notmuch_config_set_user_other_email (notmuch_config_t *config,
talloc_free (config->user_other_email);
config->user_other_email = NULL;
}
+
+notmuch_mailstore_t *
+notmuch_config_get_mailstore(notmuch_config_t *config)
+{
+ char *type;
+
+ if (config->mailstore == NULL) {
+ type = g_key_file_get_string (config->key_file,
+ "mailstore", "type", NULL);
+ if (type) {
+ config->mailstore = notmuch_mailstore_get_by_type(type);
+ free (type);
+ }
+ }
+
+ return config->mailstore;
+}
+
+void
+notmuch_config_set_mailstore(notmuch_config_t *config,
+ notmuch_mailstore_t *mailstore)
+{
+ g_key_file_set_string (config->key_file,
+ "mailstore", "type",
+ notmuch_mailstore_get_type(mailstore));
+
+ config->mailstore = NULL;
+}
diff --git a/notmuch-count.c b/notmuch-count.c
index 77aa433..631b8f8 100644
--- a/notmuch-count.c
+++ b/notmuch-count.c
@@ -81,7 +81,8 @@ notmuch_count_command (void *ctx, int argc, char *argv[])
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore(config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-dump.c b/notmuch-dump.c
index 7e7bc17..e74dfcb 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -36,7 +36,8 @@ notmuch_dump_command (unused (void *ctx), int argc, char *argv[])
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-new.c b/notmuch-new.c
index 44b50aa..2d0ba6c 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -749,11 +749,13 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
return 1;
printf ("Found %d total files (that's not much mail).\n", count);
- notmuch = notmuch_database_create (db_path);
+ notmuch = notmuch_database_create (db_path,
+ notmuch_config_get_mailstore (config));
add_files_state.total_files = count;
} else {
notmuch = notmuch_database_open (db_path,
- NOTMUCH_DATABASE_MODE_READ_WRITE);
+ NOTMUCH_DATABASE_MODE_READ_WRITE,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 39377e1..1ec28cd 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -533,7 +533,8 @@ notmuch_reply_command (void *ctx, int argc, char *argv[])
}
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-restore.c b/notmuch-restore.c
index b0a4e1c..97d817b 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -37,7 +37,8 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_WRITE);
+ NOTMUCH_DATABASE_MODE_READ_WRITE,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-search-tags.c b/notmuch-search-tags.c
index 6f3cfcc..323f416 100644
--- a/notmuch-search-tags.c
+++ b/notmuch-search-tags.c
@@ -52,7 +52,8 @@ notmuch_search_tags_command (void *ctx, int argc, char *argv[])
}
db = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (db == NULL) {
goto error;
}
diff --git a/notmuch-search.c b/notmuch-search.c
index 4e3514b..d57479d 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -245,7 +245,8 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-show.c b/notmuch-show.c
index 76873a1..6dca672 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -461,7 +461,8 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[]))
}
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
@@ -547,7 +548,8 @@ notmuch_part_command (void *ctx, unused (int argc), unused (char *argv[]))
}
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_ONLY);
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ notmuch_config_get_mailstore(config));
if (notmuch == NULL)
return 1;
diff --git a/notmuch-tag.c b/notmuch-tag.c
index 8b6f7dc..5e1a5c3 100644
--- a/notmuch-tag.c
+++ b/notmuch-tag.c
@@ -97,7 +97,8 @@ notmuch_tag_command (void *ctx, unused (int argc), unused (char *argv[]))
return 1;
notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
- NOTMUCH_DATABASE_MODE_READ_WRITE);
+ NOTMUCH_DATABASE_MODE_READ_WRITE,
+ notmuch_config_get_mailstore (config));
if (notmuch == NULL)
return 1;
--
1.7.0.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 2/4] Conversion to mailstore abstraction
2010-04-08 14:42 [PATCH 0/4] Mailstore abstraction v4 Michal Sojka
2010-04-08 14:42 ` [PATCH 1/4] Mailstore abstraction interface Michal Sojka
@ 2010-04-08 14:42 ` Michal Sojka
2010-04-08 14:42 ` [PATCH 3/4] Access messages through mail store interface Michal Sojka
` (2 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Michal Sojka @ 2010-04-08 14:42 UTC (permalink / raw)
To: notmuch
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 45118 bytes --]
The code for detection of new files in the mailstore and their
addition to the database is moved from notmuch-new.c to
lib/mailstore-files.c, where it is called by the abstract mailstore
interface.
The code was only changed to allow the progress reporting functions to
be implemented outside of notmuch library, as can be seen by
git diff HEAD^ -C
Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
---
lib/Makefile.local | 1 +
notmuch-new.c => lib/mailstore-files.c | 406 ++++----------------
lib/mailstore.c | 5 -
notmuch-new.c | 660 +++-----------------------------
4 files changed, 141 insertions(+), 931 deletions(-)
copy notmuch-new.c => lib/mailstore-files.c (61%)
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 6243af1..863df4c 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -32,6 +32,7 @@ extra_cflags += -I$(dir) -fPIC
libnotmuch_c_srcs = \
$(dir)/libsha1.c \
$(dir)/mailstore.c \
+ $(dir)/mailstore-files.c\
$(dir)/message-file.c \
$(dir)/messages.c \
$(dir)/sha1.c \
diff --git a/notmuch-new.c b/lib/mailstore-files.c
similarity index 61%
copy from notmuch-new.c
copy to lib/mailstore-files.c
index 2d0ba6c..8b5e1d5 100644
--- a/notmuch-new.c
+++ b/lib/mailstore-files.c
@@ -1,4 +1,5 @@
-/* notmuch - Not much of an email program, (just index and search)
+/* mailstore-files.c - Original notmuch mail store - a collection of
+ * plain-text email messages (one message per file).
*
* Copyright © 2009 Carl Worth
*
@@ -15,12 +16,14 @@
* You should have received a copy of the GNU General Public License
* along with this program. If not, see http://www.gnu.org/licenses/ .
*
- * Author: Carl Worth <cworth@cworth.org>
+ * Authors: Carl Worth <cworth@cworth.org>
+ * Michal Sojka <sojkam1@fel.cvut.cz>
*/
-#include "notmuch-client.h"
-
-#include <unistd.h>
+#define _GNU_SOURCE /* For asprintf() */
+#include "notmuch.h"
+#include "mailstore-private.h"
+#include <dirent.h>
typedef struct _filename_node {
char *filename;
@@ -32,38 +35,10 @@ typedef struct _filename_list {
_filename_node_t **tail;
} _filename_list_t;
-typedef struct {
- int output_is_a_tty;
- int verbose;
-
- int total_files;
- int processed_files;
- int added_messages;
- struct timeval tv_start;
-
+typedef struct _indexing_context_priv {
_filename_list_t *removed_files;
_filename_list_t *removed_directories;
-} add_files_state_t;
-
-static volatile sig_atomic_t do_add_files_print_progress = 0;
-
-static void
-handle_sigalrm (unused (int signal))
-{
- do_add_files_print_progress = 1;
-}
-
-static volatile sig_atomic_t interrupted;
-
-static void
-handle_sigint (unused (int sig))
-{
- ssize_t ignored;
- static char msg[] = "Stopping... \n";
-
- ignored = write(2, msg, sizeof(msg)-1);
- interrupted = 1;
-}
+} _indexing_context_priv_t;
static _filename_list_t *
_filename_list_create (const void *ctx)
@@ -100,34 +75,6 @@ tag_inbox_and_unread (notmuch_message_t *message)
notmuch_message_add_tag (message, "unread");
}
-static void
-add_files_print_progress (add_files_state_t *state)
-{
- struct timeval tv_now;
- double elapsed_overall, rate_overall;
-
- gettimeofday (&tv_now, NULL);
-
- elapsed_overall = notmuch_time_elapsed (state->tv_start, tv_now);
- rate_overall = (state->processed_files) / elapsed_overall;
-
- printf ("Processed %d", state->processed_files);
-
- if (state->total_files) {
- double time_remaining;
-
- time_remaining = ((state->total_files - state->processed_files) /
- rate_overall);
- printf (" of %d files (", state->total_files);
- notmuch_time_print_formatted_seconds (time_remaining);
- printf (" remaining). \r");
- } else {
- printf (" files (%d files/sec.) \r", (int) rate_overall);
- }
-
- fflush (stdout);
-}
-
static int
dirent_sort_inode (const struct dirent **a, const struct dirent **b)
{
@@ -169,6 +116,7 @@ _entries_resemble_maildir (struct dirent **entries, int count)
return 0;
}
+
/* Examine 'path' recursively as follows:
*
* o Ask the filesystem for the mtime of 'path' (fs_mtime)
@@ -205,9 +153,9 @@ _entries_resemble_maildir (struct dirent **entries, int count)
* o Tell the database to update its time of 'path' to 'fs_mtime'
*/
static notmuch_status_t
-add_files_recursive (notmuch_database_t *notmuch,
+add_files_recursive (notmuch_mailstore_t *mailstore,
const char *path,
- add_files_state_t *state)
+ notmuch_indexing_context_t *state)
{
DIR *dir = NULL;
struct dirent *entry = NULL;
@@ -222,6 +170,8 @@ add_files_recursive (notmuch_database_t *notmuch,
notmuch_filenames_t *db_subdirs = NULL;
struct stat st;
notmuch_bool_t is_maildir, new_directory;
+ _indexing_context_priv_t *priv = state->priv;
+ notmuch_database_t *notmuch = mailstore->notmuch;
if (stat (path, &st)) {
fprintf (stderr, "Error reading directory %s: %s\n",
@@ -268,7 +218,7 @@ add_files_recursive (notmuch_database_t *notmuch,
is_maildir = _entries_resemble_maildir (fs_entries, num_fs_entries);
for (i = 0; i < num_fs_entries; i++) {
- if (interrupted)
+ if (state->interrupted)
break;
entry = fs_entries[i];
@@ -302,7 +252,7 @@ add_files_recursive (notmuch_database_t *notmuch,
}
next = talloc_asprintf (notmuch, "%s/%s", path, entry->d_name);
- status = add_files_recursive (notmuch, next, state);
+ status = add_files_recursive (mailstore, next, state);
if (status && ret == NOTMUCH_STATUS_SUCCESS)
ret = status;
talloc_free (next);
@@ -317,7 +267,7 @@ add_files_recursive (notmuch_database_t *notmuch,
/* Pass 2: Scan for new files, removed files, and removed directories. */
for (i = 0; i < num_fs_entries; i++)
{
- if (interrupted)
+ if (state->interrupted)
break;
entry = fs_entries[i];
@@ -327,11 +277,11 @@ add_files_recursive (notmuch_database_t *notmuch,
while (notmuch_filenames_valid (db_files) &&
strcmp (notmuch_filenames_get (db_files), entry->d_name) < 0)
{
- char *absolute = talloc_asprintf (state->removed_files,
+ char *absolute = talloc_asprintf (priv->removed_files,
"%s/%s", path,
notmuch_filenames_get (db_files));
- _filename_list_add (state->removed_files, absolute);
+ _filename_list_add (priv->removed_files, absolute);
notmuch_filenames_move_to_next (db_files);
}
@@ -343,10 +293,10 @@ add_files_recursive (notmuch_database_t *notmuch,
if (strcmp (filename, entry->d_name) < 0)
{
- char *absolute = talloc_asprintf (state->removed_directories,
+ char *absolute = talloc_asprintf (priv->removed_directories,
"%s/%s", path, filename);
- _filename_list_add (state->removed_directories, absolute);
+ _filename_list_add (priv->removed_directories, absolute);
}
notmuch_filenames_move_to_next (db_subdirs);
@@ -394,18 +344,8 @@ add_files_recursive (notmuch_database_t *notmuch,
state->processed_files++;
- if (state->verbose) {
- if (state->output_is_a_tty)
- printf("\r\033[K");
-
- printf ("%i/%i: %s",
- state->processed_files,
- state->total_files,
- next);
-
- putchar((state->output_is_a_tty) ? '\r' : '\n');
- fflush (stdout);
- }
+ if (state->verbose && state->print_verbose_cb)
+ state->print_verbose_cb(state, next);
status = notmuch_database_add_message (notmuch, next, &message);
switch (status) {
@@ -445,46 +385,50 @@ add_files_recursive (notmuch_database_t *notmuch,
message = NULL;
}
- if (do_add_files_print_progress) {
- do_add_files_print_progress = 0;
- add_files_print_progress (state);
+ if (state->print_progress &&
+ state->print_progress_cb) {
+ state->print_progress = 0;
+ state->print_progress_cb (state);
}
talloc_free (next);
next = NULL;
}
+ if (state->interrupted)
+ goto DONE;
+
/* Now that we've walked the whole filesystem list, anything left
* over in the database lists has been deleted. */
while (notmuch_filenames_valid (db_files))
{
- char *absolute = talloc_asprintf (state->removed_files,
+ char *absolute = talloc_asprintf (priv->removed_files,
"%s/%s", path,
notmuch_filenames_get (db_files));
- _filename_list_add (state->removed_files, absolute);
+ _filename_list_add (priv->removed_files, absolute);
notmuch_filenames_move_to_next (db_files);
}
while (notmuch_filenames_valid (db_subdirs))
{
- char *absolute = talloc_asprintf (state->removed_directories,
+ char *absolute = talloc_asprintf (priv->removed_directories,
"%s/%s", path,
notmuch_filenames_get (db_subdirs));
- _filename_list_add (state->removed_directories, absolute);
+ _filename_list_add (priv->removed_directories, absolute);
notmuch_filenames_move_to_next (db_subdirs);
}
- if (! interrupted) {
+ if (! state->interrupted) {
status = notmuch_directory_set_mtime (directory, fs_mtime);
if (status && ret == NOTMUCH_STATUS_SUCCESS)
ret = status;
}
- DONE:
+DONE:
if (next)
talloc_free (next);
if (entry)
@@ -503,67 +447,6 @@ add_files_recursive (notmuch_database_t *notmuch,
return ret;
}
-/* This is the top-level entry point for add_files. It does a couple
- * of error checks, sets up the progress-printing timer and then calls
- * into the recursive function. */
-static notmuch_status_t
-add_files (notmuch_database_t *notmuch,
- const char *path,
- add_files_state_t *state)
-{
- notmuch_status_t status;
- struct sigaction action;
- struct itimerval timerval;
- notmuch_bool_t timer_is_active = FALSE;
- struct stat st;
-
- if (state->output_is_a_tty && ! debugger_is_active () && ! state->verbose) {
- /* Setup our handler for SIGALRM */
- memset (&action, 0, sizeof (struct sigaction));
- action.sa_handler = handle_sigalrm;
- sigemptyset (&action.sa_mask);
- action.sa_flags = SA_RESTART;
- sigaction (SIGALRM, &action, NULL);
-
- /* Then start a timer to send SIGALRM once per second. */
- timerval.it_interval.tv_sec = 1;
- timerval.it_interval.tv_usec = 0;
- timerval.it_value.tv_sec = 1;
- timerval.it_value.tv_usec = 0;
- setitimer (ITIMER_REAL, &timerval, NULL);
-
- timer_is_active = TRUE;
- }
-
- if (stat (path, &st)) {
- fprintf (stderr, "Error reading directory %s: %s\n",
- path, strerror (errno));
- return NOTMUCH_STATUS_FILE_ERROR;
- }
-
- if (! S_ISDIR (st.st_mode)) {
- fprintf (stderr, "Error: %s is not a directory.\n", path);
- return NOTMUCH_STATUS_FILE_ERROR;
- }
-
- status = add_files_recursive (notmuch, path, state);
-
- if (timer_is_active) {
- /* Now stop the timer. */
- timerval.it_interval.tv_sec = 0;
- timerval.it_interval.tv_usec = 0;
- timerval.it_value.tv_sec = 0;
- timerval.it_value.tv_usec = 0;
- setitimer (ITIMER_REAL, &timerval, NULL);
-
- /* And disable the signal handler. */
- action.sa_handler = SIG_IGN;
- sigaction (SIGALRM, &action, NULL);
- }
-
- return status;
-}
-
/* XXX: This should be merged with the add_files function since it
* shares a lot of logic with it. */
/* Recursively count all regular files in path and all sub-directories
@@ -571,7 +454,9 @@ add_files (notmuch_database_t *notmuch,
* initialized to zero by the top-level caller before calling
* count_files). */
static void
-count_files (const char *path, int *count)
+count_files (notmuch_mailstore_t *mailstore,
+ const char *path, int *count,
+ volatile sig_atomic_t *interrupted)
{
struct dirent *entry = NULL;
char *next;
@@ -580,13 +465,14 @@ count_files (const char *path, int *count)
int num_fs_entries = scandir (path, &fs_entries, 0, dirent_sort_inode);
int i = 0;
+ (void)mailstore;
if (num_fs_entries == -1) {
fprintf (stderr, "Warning: failed to open directory %s: %s\n",
path, strerror (errno));
goto DONE;
}
- while (!interrupted) {
+ while (!*interrupted) {
if (i == num_fs_entries)
break;
@@ -620,45 +506,19 @@ count_files (const char *path, int *count)
fflush (stdout);
}
} else if (S_ISDIR (st.st_mode)) {
- count_files (next, count);
+ count_files (mailstore, next, count, interrupted);
}
free (next);
}
- DONE:
+DONE:
if (entry)
free (entry);
if (fs_entries)
free (fs_entries);
}
-static void
-upgrade_print_progress (void *closure,
- double progress)
-{
- add_files_state_t *state = closure;
-
- printf ("Upgrading database: %.2f%% complete", progress * 100.0);
-
- if (progress > 0) {
- struct timeval tv_now;
- double elapsed, time_remaining;
-
- gettimeofday (&tv_now, NULL);
-
- elapsed = notmuch_time_elapsed (state->tv_start, tv_now);
- time_remaining = (elapsed / progress) * (1.0 - progress);
- printf (" (");
- notmuch_time_print_formatted_seconds (time_remaining);
- printf (" remaining)");
- }
-
- printf (". \r");
-
- fflush (stdout);
-}
-
/* Recursively remove all filenames from the database referring to
* 'path' (or to any of its children). */
static void
@@ -702,163 +562,51 @@ _remove_directory (void *ctx,
notmuch_directory_destroy (directory);
}
-int
-notmuch_new_command (void *ctx, int argc, char *argv[])
+static notmuch_private_status_t
+index_new(notmuch_mailstore_t *mailstore, const char* path,
+ notmuch_indexing_context_t *indexing_ctx)
{
- notmuch_config_t *config;
- notmuch_database_t *notmuch;
- add_files_state_t add_files_state;
- double elapsed;
- struct timeval tv_now;
- int ret = 0;
- struct stat st;
- const char *db_path;
- char *dot_notmuch_path;
- struct sigaction action;
+ _indexing_context_priv_t *priv;
_filename_node_t *f;
- int renamed_files, removed_files;
- notmuch_status_t status;
- int i;
-
- add_files_state.verbose = 0;
- add_files_state.output_is_a_tty = isatty (fileno (stdout));
-
- for (i = 0; i < argc && argv[i][0] == '-'; i++) {
- if (STRNCMP_LITERAL (argv[i], "--verbose") == 0) {
- add_files_state.verbose = 1;
- } else {
- fprintf (stderr, "Unrecognized option: %s\n", argv[i]);
- return 1;
- }
- }
-
- config = notmuch_config_open (ctx, NULL, NULL);
- if (config == NULL)
- return 1;
+ notmuch_status_t status, ret;
+ notmuch_database_t *notmuch = mailstore->notmuch;
- db_path = notmuch_config_get_database_path (config);
+ priv = talloc(NULL, _indexing_context_priv_t);
+ indexing_ctx->priv = priv;
+ if (priv == NULL)
+ return NOTMUCH_STATUS_OUT_OF_MEMORY;
- dot_notmuch_path = talloc_asprintf (ctx, "%s/%s", db_path, ".notmuch");
+ priv->removed_files = _filename_list_create (priv);
+ priv->removed_directories = _filename_list_create (priv);
- if (stat (dot_notmuch_path, &st)) {
- int count;
+ ret = add_files_recursive(mailstore, path, indexing_ctx);
- count = 0;
- count_files (db_path, &count);
- if (interrupted)
- return 1;
-
- printf ("Found %d total files (that's not much mail).\n", count);
- notmuch = notmuch_database_create (db_path,
- notmuch_config_get_mailstore (config));
- add_files_state.total_files = count;
- } else {
- notmuch = notmuch_database_open (db_path,
- NOTMUCH_DATABASE_MODE_READ_WRITE,
- notmuch_config_get_mailstore (config));
- if (notmuch == NULL)
- return 1;
-
- if (notmuch_database_needs_upgrade (notmuch)) {
- printf ("Welcome to a new version of notmuch! Your database will now be upgraded.\n");
- gettimeofday (&add_files_state.tv_start, NULL);
- notmuch_database_upgrade (notmuch, upgrade_print_progress,
- &add_files_state);
- printf ("Your notmuch database has now been upgraded to database format version %u.\n",
- notmuch_database_get_version (notmuch));
- }
-
- add_files_state.total_files = 0;
- }
-
- if (notmuch == NULL)
- return 1;
-
- /* Setup our handler for SIGINT. We do this after having
- * potentially done a database upgrade we this interrupt handler
- * won't support. */
- memset (&action, 0, sizeof (struct sigaction));
- action.sa_handler = handle_sigint;
- sigemptyset (&action.sa_mask);
- action.sa_flags = SA_RESTART;
- sigaction (SIGINT, &action, NULL);
-
- talloc_free (dot_notmuch_path);
- dot_notmuch_path = NULL;
-
- add_files_state.processed_files = 0;
- add_files_state.added_messages = 0;
- gettimeofday (&add_files_state.tv_start, NULL);
-
- add_files_state.removed_files = _filename_list_create (ctx);
- add_files_state.removed_directories = _filename_list_create (ctx);
-
- ret = add_files (notmuch, db_path, &add_files_state);
-
- removed_files = 0;
- renamed_files = 0;
- for (f = add_files_state.removed_files->head; f; f = f->next) {
+ indexing_ctx->removed_files = 0;
+ indexing_ctx->renamed_files = 0;
+ for (f = priv->removed_files->head; f; f = f->next) {
status = notmuch_database_remove_message (notmuch, f->filename);
if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID)
- renamed_files++;
+ indexing_ctx->renamed_files++;
else
- removed_files++;
- }
-
- for (f = add_files_state.removed_directories->head; f; f = f->next) {
- _remove_directory (ctx, notmuch, f->filename,
- &renamed_files, &removed_files);
- }
-
- talloc_free (add_files_state.removed_files);
- talloc_free (add_files_state.removed_directories);
-
- gettimeofday (&tv_now, NULL);
- elapsed = notmuch_time_elapsed (add_files_state.tv_start,
- tv_now);
-
- if (add_files_state.processed_files) {
- printf ("Processed %d %s in ", add_files_state.processed_files,
- add_files_state.processed_files == 1 ?
- "file" : "total files");
- notmuch_time_print_formatted_seconds (elapsed);
- if (elapsed > 1) {
- printf (" (%d files/sec.). \n",
- (int) (add_files_state.processed_files / elapsed));
- } else {
- printf (". \n");
- }
- }
-
- if (add_files_state.added_messages) {
- printf ("Added %d new %s to the database.",
- add_files_state.added_messages,
- add_files_state.added_messages == 1 ?
- "message" : "messages");
- } else {
- printf ("No new mail.");
- }
-
- if (removed_files) {
- printf (" Removed %d %s.",
- removed_files,
- removed_files == 1 ? "message" : "messages");
- }
-
- if (renamed_files) {
- printf (" Detected %d file %s.",
- renamed_files,
- renamed_files == 1 ? "rename" : "renames");
+ indexing_ctx->removed_files++;
}
- printf ("\n");
-
- if (ret) {
- printf ("\nNote: At least one error was encountered: %s\n",
- notmuch_status_to_string (ret));
+ for (f = priv->removed_directories->head; f; f = f->next) {
+ _remove_directory (priv, notmuch, f->filename,
+ &indexing_ctx->renamed_files,
+ &indexing_ctx->removed_files);
}
- notmuch_database_close (notmuch);
+ talloc_free(priv);
- return ret || interrupted;
+ return ret;
}
+
+/* Original notmuch mail store */
+struct _notmuch_mailstore mailstore_files = {
+ .type = "files",
+ .count_files = count_files,
+ .index_new = index_new,
+ .sync_tags = NULL, /* We cannot store tags in this mailstore. */
+ .open_file = NULL, /* Currently not implemented */
+};
diff --git a/lib/mailstore.c b/lib/mailstore.c
index eb27952..709db72 100644
--- a/lib/mailstore.c
+++ b/lib/mailstore.c
@@ -23,11 +23,6 @@
#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
-/* Original notmuch mail store */
-struct _notmuch_mailstore mailstore_files = {
- .type = "files",
-};
-
static notmuch_mailstore_t *available[] = {
&mailstore_files,
};
diff --git a/notmuch-new.c b/notmuch-new.c
index 2d0ba6c..6c527e9 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -22,35 +22,12 @@
#include <unistd.h>
-typedef struct _filename_node {
- char *filename;
- struct _filename_node *next;
-} _filename_node_t;
-
-typedef struct _filename_list {
- _filename_node_t *head;
- _filename_node_t **tail;
-} _filename_list_t;
-
-typedef struct {
- int output_is_a_tty;
- int verbose;
-
- int total_files;
- int processed_files;
- int added_messages;
- struct timeval tv_start;
-
- _filename_list_t *removed_files;
- _filename_list_t *removed_directories;
-} add_files_state_t;
-
-static volatile sig_atomic_t do_add_files_print_progress = 0;
+notmuch_indexing_context_t *indexing_ctx;
static void
handle_sigalrm (unused (int signal))
{
- do_add_files_print_progress = 1;
+ indexing_ctx->print_progress = 1;
}
static volatile sig_atomic_t interrupted;
@@ -62,63 +39,35 @@ handle_sigint (unused (int sig))
static char msg[] = "Stopping... \n";
ignored = write(2, msg, sizeof(msg)-1);
- interrupted = 1;
-}
-
-static _filename_list_t *
-_filename_list_create (const void *ctx)
-{
- _filename_list_t *list;
-
- list = talloc (ctx, _filename_list_t);
- if (list == NULL)
- return NULL;
-
- list->head = NULL;
- list->tail = &list->head;
-
- return list;
-}
-
-static void
-_filename_list_add (_filename_list_t *list,
- const char *filename)
-{
- _filename_node_t *node = talloc (list, _filename_node_t);
-
- node->filename = talloc_strdup (list, filename);
- node->next = NULL;
-
- *(list->tail) = node;
- list->tail = &node->next;
+ indexing_ctx->interrupted = 1;
}
-static void
-tag_inbox_and_unread (notmuch_message_t *message)
-{
- notmuch_message_add_tag (message, "inbox");
- notmuch_message_add_tag (message, "unread");
-}
+struct print_ctx {
+ int total_files;
+ struct timeval tv_start;
+ int output_is_a_tty;
+};
static void
-add_files_print_progress (add_files_state_t *state)
+add_files_print_progress (notmuch_indexing_context_t *state)
{
+ struct print_ctx *print_ctx = state->print_ctx;
struct timeval tv_now;
double elapsed_overall, rate_overall;
gettimeofday (&tv_now, NULL);
- elapsed_overall = notmuch_time_elapsed (state->tv_start, tv_now);
+ elapsed_overall = notmuch_time_elapsed (print_ctx->tv_start, tv_now);
rate_overall = (state->processed_files) / elapsed_overall;
printf ("Processed %d", state->processed_files);
- if (state->total_files) {
+ if (print_ctx->total_files) {
double time_remaining;
- time_remaining = ((state->total_files - state->processed_files) /
+ time_remaining = ((print_ctx->total_files - state->processed_files) /
rate_overall);
- printf (" of %d files (", state->total_files);
+ printf (" of %d files (", print_ctx->total_files);
notmuch_time_print_formatted_seconds (time_remaining);
printf (" remaining). \r");
} else {
@@ -128,396 +77,42 @@ add_files_print_progress (add_files_state_t *state)
fflush (stdout);
}
-static int
-dirent_sort_inode (const struct dirent **a, const struct dirent **b)
-{
- return ((*a)->d_ino < (*b)->d_ino) ? -1 : 1;
-}
-
-static int
-dirent_sort_strcmp_name (const struct dirent **a, const struct dirent **b)
-{
- return strcmp ((*a)->d_name, (*b)->d_name);
-}
-
-/* Test if the directory looks like a Maildir directory.
- *
- * Search through the array of directory entries to see if we can find all
- * three subdirectories typical for Maildir, that is "new", "cur", and "tmp".
- *
- * Return 1 if the directory looks like a Maildir and 0 otherwise.
- */
-static int
-_entries_resemble_maildir (struct dirent **entries, int count)
-{
- int i, found = 0;
-
- for (i = 0; i < count; i++) {
- if (entries[i]->d_type != DT_DIR && entries[i]->d_type != DT_UNKNOWN)
- continue;
-
- if (strcmp(entries[i]->d_name, "new") == 0 ||
- strcmp(entries[i]->d_name, "cur") == 0 ||
- strcmp(entries[i]->d_name, "tmp") == 0)
- {
- found++;
- if (found == 3)
- return 1;
- }
- }
-
- return 0;
-}
-
-/* Examine 'path' recursively as follows:
- *
- * o Ask the filesystem for the mtime of 'path' (fs_mtime)
- * o Ask the database for its timestamp of 'path' (db_mtime)
- *
- * o Ask the filesystem for files and directories within 'path'
- * (via scandir and stored in fs_entries)
- * o Ask the database for files and directories within 'path'
- * (db_files and db_subdirs)
- *
- * o Pass 1: For each directory in fs_entries, recursively call into
- * this same function.
- *
- * o Pass 2: If 'fs_mtime' > 'db_mtime', then walk fs_entries
- * simultaneously with db_files and db_subdirs. Look for one of
- * three interesting cases:
- *
- * 1. Regular file in fs_entries and not in db_files
- * This is a new file to add_message into the database.
- *
- * 2. Filename in db_files not in fs_entries.
- * This is a file that has been removed from the mail store.
- *
- * 3. Directory in db_subdirs not in fs_entries
- * This is a directory that has been removed from the mail store.
- *
- * Note that the addition of a directory is not interesting here,
- * since that will have been taken care of in pass 1. Also, we
- * don't immediately act on file/directory removal since we must
- * ensure that in the case of a rename that the new filename is
- * added before the old filename is removed, (so that no
- * information is lost from the database).
- *
- * o Tell the database to update its time of 'path' to 'fs_mtime'
- */
-static notmuch_status_t
-add_files_recursive (notmuch_database_t *notmuch,
- const char *path,
- add_files_state_t *state)
+static void
+add_files_print_verbose (notmuch_indexing_context_t *state, const char *filename)
{
- DIR *dir = NULL;
- struct dirent *entry = NULL;
- char *next = NULL;
- time_t fs_mtime, db_mtime;
- notmuch_status_t status, ret = NOTMUCH_STATUS_SUCCESS;
- notmuch_message_t *message = NULL;
- struct dirent **fs_entries = NULL;
- int i, num_fs_entries;
- notmuch_directory_t *directory;
- notmuch_filenames_t *db_files = NULL;
- notmuch_filenames_t *db_subdirs = NULL;
- struct stat st;
- notmuch_bool_t is_maildir, new_directory;
-
- if (stat (path, &st)) {
- fprintf (stderr, "Error reading directory %s: %s\n",
- path, strerror (errno));
- return NOTMUCH_STATUS_FILE_ERROR;
- }
-
- /* This is not an error since we may have recursed based on a
- * symlink to a regular file, not a directory, and we don't know
- * that until this stat. */
- if (! S_ISDIR (st.st_mode))
- return NOTMUCH_STATUS_SUCCESS;
-
- fs_mtime = st.st_mtime;
-
- directory = notmuch_database_get_directory (notmuch, path);
- db_mtime = notmuch_directory_get_mtime (directory);
-
- if (db_mtime == 0) {
- new_directory = TRUE;
- db_files = NULL;
- db_subdirs = NULL;
- } else {
- new_directory = FALSE;
- db_files = notmuch_directory_get_child_files (directory);
- db_subdirs = notmuch_directory_get_child_directories (directory);
- }
-
- /* If the database knows about this directory, then we sort based
- * on strcmp to match the database sorting. Otherwise, we can do
- * inode-based sorting for faster filesystem operation. */
- num_fs_entries = scandir (path, &fs_entries, 0,
- new_directory ?
- dirent_sort_inode : dirent_sort_strcmp_name);
-
- if (num_fs_entries == -1) {
- fprintf (stderr, "Error opening directory %s: %s\n",
- path, strerror (errno));
- ret = NOTMUCH_STATUS_FILE_ERROR;
- goto DONE;
- }
-
- /* Pass 1: Recurse into all sub-directories. */
- is_maildir = _entries_resemble_maildir (fs_entries, num_fs_entries);
-
- for (i = 0; i < num_fs_entries; i++) {
- if (interrupted)
- break;
-
- entry = fs_entries[i];
-
- /* We only want to descend into directories.
- * But symlinks can be to directories too, of course.
- *
- * And if the filesystem doesn't tell us the file type in the
- * scandir results, then it might be a directory (and if not,
- * then we'll stat and return immediately in the next level of
- * recursion). */
- if (entry->d_type != DT_DIR &&
- entry->d_type != DT_LNK &&
- entry->d_type != DT_UNKNOWN)
- {
- continue;
- }
-
- /* Ignore special directories to avoid infinite recursion.
- * Also ignore the .notmuch directory and any "tmp" directory
- * that appears within a maildir.
- */
- /* XXX: Eventually we'll want more sophistication to let the
- * user specify files to be ignored. */
- if (strcmp (entry->d_name, ".") == 0 ||
- strcmp (entry->d_name, "..") == 0 ||
- (is_maildir && strcmp (entry->d_name, "tmp") == 0) ||
- strcmp (entry->d_name, ".notmuch") ==0)
- {
- continue;
- }
-
- next = talloc_asprintf (notmuch, "%s/%s", path, entry->d_name);
- status = add_files_recursive (notmuch, next, state);
- if (status && ret == NOTMUCH_STATUS_SUCCESS)
- ret = status;
- talloc_free (next);
- next = NULL;
- }
+ struct print_ctx *print_ctx = state->print_ctx;
+ if (print_ctx->output_is_a_tty)
+ printf("\r\033[K");
- /* If this directory hasn't been modified since the last
- * "notmuch new", then we can skip the second pass entirely. */
- if (fs_mtime <= db_mtime)
- goto DONE;
+ printf ("%i/%i: %s",
+ state->processed_files,
+ print_ctx->total_files,
+ filename);
- /* Pass 2: Scan for new files, removed files, and removed directories. */
- for (i = 0; i < num_fs_entries; i++)
- {
- if (interrupted)
- break;
-
- entry = fs_entries[i];
-
- /* Check if we've walked past any names in db_files or
- * db_subdirs. If so, these have been deleted. */
- while (notmuch_filenames_valid (db_files) &&
- strcmp (notmuch_filenames_get (db_files), entry->d_name) < 0)
- {
- char *absolute = talloc_asprintf (state->removed_files,
- "%s/%s", path,
- notmuch_filenames_get (db_files));
-
- _filename_list_add (state->removed_files, absolute);
-
- notmuch_filenames_move_to_next (db_files);
- }
-
- while (notmuch_filenames_valid (db_subdirs) &&
- strcmp (notmuch_filenames_get (db_subdirs), entry->d_name) <= 0)
- {
- const char *filename = notmuch_filenames_get (db_subdirs);
-
- if (strcmp (filename, entry->d_name) < 0)
- {
- char *absolute = talloc_asprintf (state->removed_directories,
- "%s/%s", path, filename);
-
- _filename_list_add (state->removed_directories, absolute);
- }
-
- notmuch_filenames_move_to_next (db_subdirs);
- }
-
- /* If we're looking at a symlink, we only want to add it if it
- * links to a regular file, (and not to a directory, say).
- *
- * Similarly, if the file is of unknown type (due to filesytem
- * limitations), then we also need to look closer.
- *
- * In either case, a stat does the trick.
- */
- if (entry->d_type == DT_LNK || entry->d_type == DT_UNKNOWN) {
- int err;
-
- next = talloc_asprintf (notmuch, "%s/%s", path, entry->d_name);
- err = stat (next, &st);
- talloc_free (next);
- next = NULL;
-
- /* Don't emit an error for a link pointing nowhere, since
- * the directory-traversal pass will have already done
- * that. */
- if (err)
- continue;
-
- if (! S_ISREG (st.st_mode))
- continue;
- } else if (entry->d_type != DT_REG) {
- continue;
- }
-
- /* Don't add a file that we've added before. */
- if (notmuch_filenames_valid (db_files) &&
- strcmp (notmuch_filenames_get (db_files), entry->d_name) == 0)
- {
- notmuch_filenames_move_to_next (db_files);
- continue;
- }
-
- /* We're now looking at a regular file that doesn't yet exist
- * in the database, so add it. */
- next = talloc_asprintf (notmuch, "%s/%s", path, entry->d_name);
-
- state->processed_files++;
-
- if (state->verbose) {
- if (state->output_is_a_tty)
- printf("\r\033[K");
-
- printf ("%i/%i: %s",
- state->processed_files,
- state->total_files,
- next);
-
- putchar((state->output_is_a_tty) ? '\r' : '\n');
- fflush (stdout);
- }
-
- status = notmuch_database_add_message (notmuch, next, &message);
- switch (status) {
- /* success */
- case NOTMUCH_STATUS_SUCCESS:
- state->added_messages++;
- tag_inbox_and_unread (message);
- break;
- /* Non-fatal issues (go on to next file) */
- case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
- /* Stay silent on this one. */
- break;
- case NOTMUCH_STATUS_FILE_NOT_EMAIL:
- fprintf (stderr, "Note: Ignoring non-mail file: %s\n",
- next);
- break;
- /* Fatal issues. Don't process anymore. */
- case NOTMUCH_STATUS_READ_ONLY_DATABASE:
- case NOTMUCH_STATUS_XAPIAN_EXCEPTION:
- case NOTMUCH_STATUS_OUT_OF_MEMORY:
- fprintf (stderr, "Error: %s. Halting processing.\n",
- notmuch_status_to_string (status));
- ret = status;
- goto DONE;
- default:
- case NOTMUCH_STATUS_FILE_ERROR:
- case NOTMUCH_STATUS_NULL_POINTER:
- case NOTMUCH_STATUS_TAG_TOO_LONG:
- case NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW:
- case NOTMUCH_STATUS_LAST_STATUS:
- INTERNAL_ERROR ("add_message returned unexpected value: %d", status);
- goto DONE;
- }
-
- if (message) {
- notmuch_message_destroy (message);
- message = NULL;
- }
-
- if (do_add_files_print_progress) {
- do_add_files_print_progress = 0;
- add_files_print_progress (state);
- }
-
- talloc_free (next);
- next = NULL;
- }
-
- /* Now that we've walked the whole filesystem list, anything left
- * over in the database lists has been deleted. */
- while (notmuch_filenames_valid (db_files))
- {
- char *absolute = talloc_asprintf (state->removed_files,
- "%s/%s", path,
- notmuch_filenames_get (db_files));
-
- _filename_list_add (state->removed_files, absolute);
-
- notmuch_filenames_move_to_next (db_files);
- }
-
- while (notmuch_filenames_valid (db_subdirs))
- {
- char *absolute = talloc_asprintf (state->removed_directories,
- "%s/%s", path,
- notmuch_filenames_get (db_subdirs));
-
- _filename_list_add (state->removed_directories, absolute);
-
- notmuch_filenames_move_to_next (db_subdirs);
- }
-
- if (! interrupted) {
- status = notmuch_directory_set_mtime (directory, fs_mtime);
- if (status && ret == NOTMUCH_STATUS_SUCCESS)
- ret = status;
- }
-
- DONE:
- if (next)
- talloc_free (next);
- if (entry)
- free (entry);
- if (dir)
- closedir (dir);
- if (fs_entries)
- free (fs_entries);
- if (db_subdirs)
- notmuch_filenames_destroy (db_subdirs);
- if (db_files)
- notmuch_filenames_destroy (db_files);
- if (directory)
- notmuch_directory_destroy (directory);
-
- return ret;
+ putchar((print_ctx->output_is_a_tty) ? '\r' : '\n');
+ fflush (stdout);
}
/* This is the top-level entry point for add_files. It does a couple
* of error checks, sets up the progress-printing timer and then calls
* into the recursive function. */
static notmuch_status_t
-add_files (notmuch_database_t *notmuch,
+add_files (notmuch_mailstore_t *mailstore,
const char *path,
- add_files_state_t *state)
+ notmuch_indexing_context_t *state) /* FIXME: rename */
{
+ struct print_ctx *print_ctx = state->print_ctx;
notmuch_status_t status;
struct sigaction action;
struct itimerval timerval;
notmuch_bool_t timer_is_active = FALSE;
struct stat st;
- if (state->output_is_a_tty && ! debugger_is_active () && ! state->verbose) {
+ state->print_progress = 0;
+ state->print_progress_cb = add_files_print_progress;
+ state->print_verbose_cb = add_files_print_verbose;
+
+ if (print_ctx->output_is_a_tty && ! debugger_is_active () && ! state->verbose) {
/* Setup our handler for SIGALRM */
memset (&action, 0, sizeof (struct sigaction));
action.sa_handler = handle_sigalrm;
@@ -546,7 +141,7 @@ add_files (notmuch_database_t *notmuch,
return NOTMUCH_STATUS_FILE_ERROR;
}
- status = add_files_recursive (notmuch, path, state);
+ status = notmuch_mailstore_index_new (mailstore, path, state);
if (timer_is_active) {
/* Now stop the timer. */
@@ -564,80 +159,12 @@ add_files (notmuch_database_t *notmuch,
return status;
}
-/* XXX: This should be merged with the add_files function since it
- * shares a lot of logic with it. */
-/* Recursively count all regular files in path and all sub-directories
- * of path. The result is added to *count (which should be
- * initialized to zero by the top-level caller before calling
- * count_files). */
-static void
-count_files (const char *path, int *count)
-{
- struct dirent *entry = NULL;
- char *next;
- struct stat st;
- struct dirent **fs_entries = NULL;
- int num_fs_entries = scandir (path, &fs_entries, 0, dirent_sort_inode);
- int i = 0;
-
- if (num_fs_entries == -1) {
- fprintf (stderr, "Warning: failed to open directory %s: %s\n",
- path, strerror (errno));
- goto DONE;
- }
-
- while (!interrupted) {
- if (i == num_fs_entries)
- break;
-
- entry = fs_entries[i++];
-
- /* Ignore special directories to avoid infinite recursion.
- * Also ignore the .notmuch directory.
- */
- /* XXX: Eventually we'll want more sophistication to let the
- * user specify files to be ignored. */
- if (strcmp (entry->d_name, ".") == 0 ||
- strcmp (entry->d_name, "..") == 0 ||
- strcmp (entry->d_name, ".notmuch") == 0)
- {
- continue;
- }
-
- if (asprintf (&next, "%s/%s", path, entry->d_name) == -1) {
- next = NULL;
- fprintf (stderr, "Error descending from %s to %s: Out of memory\n",
- path, entry->d_name);
- continue;
- }
-
- stat (next, &st);
-
- if (S_ISREG (st.st_mode)) {
- *count = *count + 1;
- if (*count % 1000 == 0) {
- printf ("Found %d files so far.\r", *count);
- fflush (stdout);
- }
- } else if (S_ISDIR (st.st_mode)) {
- count_files (next, count);
- }
-
- free (next);
- }
-
- DONE:
- if (entry)
- free (entry);
- if (fs_entries)
- free (fs_entries);
-}
-
static void
upgrade_print_progress (void *closure,
double progress)
{
- add_files_state_t *state = closure;
+ notmuch_indexing_context_t *state = closure;
+ struct print_ctx *print_ctx = state->print_ctx;
printf ("Upgrading database: %.2f%% complete", progress * 100.0);
@@ -647,7 +174,7 @@ upgrade_print_progress (void *closure,
gettimeofday (&tv_now, NULL);
- elapsed = notmuch_time_elapsed (state->tv_start, tv_now);
+ elapsed = notmuch_time_elapsed (print_ctx->tv_start, tv_now);
time_remaining = (elapsed / progress) * (1.0 - progress);
printf (" (");
notmuch_time_print_formatted_seconds (time_remaining);
@@ -659,55 +186,13 @@ upgrade_print_progress (void *closure,
fflush (stdout);
}
-/* Recursively remove all filenames from the database referring to
- * 'path' (or to any of its children). */
-static void
-_remove_directory (void *ctx,
- notmuch_database_t *notmuch,
- const char *path,
- int *renamed_files,
- int *removed_files)
-{
- notmuch_directory_t *directory;
- notmuch_filenames_t *files, *subdirs;
- notmuch_status_t status;
- char *absolute;
-
- directory = notmuch_database_get_directory (notmuch, path);
-
- for (files = notmuch_directory_get_child_files (directory);
- notmuch_filenames_valid (files);
- notmuch_filenames_move_to_next (files))
- {
- absolute = talloc_asprintf (ctx, "%s/%s", path,
- notmuch_filenames_get (files));
- status = notmuch_database_remove_message (notmuch, absolute);
- if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID)
- *renamed_files = *renamed_files + 1;
- else
- *removed_files = *removed_files + 1;
- talloc_free (absolute);
- }
-
- for (subdirs = notmuch_directory_get_child_directories (directory);
- notmuch_filenames_valid (subdirs);
- notmuch_filenames_move_to_next (subdirs))
- {
- absolute = talloc_asprintf (ctx, "%s/%s", path,
- notmuch_filenames_get (subdirs));
- _remove_directory (ctx, notmuch, absolute, renamed_files, removed_files);
- talloc_free (absolute);
- }
-
- notmuch_directory_destroy (directory);
-}
-
int
notmuch_new_command (void *ctx, int argc, char *argv[])
{
notmuch_config_t *config;
notmuch_database_t *notmuch;
- add_files_state_t add_files_state;
+ struct print_ctx print_ctx;
+ notmuch_indexing_context_t add_files_state; /* FIXME: Rename */
double elapsed;
struct timeval tv_now;
int ret = 0;
@@ -715,13 +200,15 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
const char *db_path;
char *dot_notmuch_path;
struct sigaction action;
- _filename_node_t *f;
- int renamed_files, removed_files;
- notmuch_status_t status;
int i;
+ notmuch_mailstore_t *mailstore;
+
+ memset (&add_files_state, 0, sizeof(add_files_state));
+ indexing_ctx = &add_files_state;
add_files_state.verbose = 0;
- add_files_state.output_is_a_tty = isatty (fileno (stdout));
+ add_files_state.print_ctx = &print_ctx;
+ print_ctx.output_is_a_tty = isatty (fileno (stdout));
for (i = 0; i < argc && argv[i][0] == '-'; i++) {
if (STRNCMP_LITERAL (argv[i], "--verbose") == 0) {
@@ -737,6 +224,7 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
return 1;
db_path = notmuch_config_get_database_path (config);
+ mailstore = notmuch_config_get_mailstore (config);
dot_notmuch_path = talloc_asprintf (ctx, "%s/%s", db_path, ".notmuch");
@@ -744,31 +232,29 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
int count;
count = 0;
- count_files (db_path, &count);
+ notmuch_mailstore_count_files (mailstore, db_path, &count, &interrupted);
if (interrupted)
return 1;
printf ("Found %d total files (that's not much mail).\n", count);
- notmuch = notmuch_database_create (db_path,
- notmuch_config_get_mailstore (config));
- add_files_state.total_files = count;
+ notmuch = notmuch_database_create (db_path, mailstore);
+ print_ctx.total_files = count;
} else {
- notmuch = notmuch_database_open (db_path,
- NOTMUCH_DATABASE_MODE_READ_WRITE,
- notmuch_config_get_mailstore (config));
+ notmuch = notmuch_database_open (db_path, NOTMUCH_DATABASE_MODE_READ_WRITE,
+ mailstore);
if (notmuch == NULL)
return 1;
if (notmuch_database_needs_upgrade (notmuch)) {
printf ("Welcome to a new version of notmuch! Your database will now be upgraded.\n");
- gettimeofday (&add_files_state.tv_start, NULL);
+ gettimeofday (&print_ctx.tv_start, NULL);
notmuch_database_upgrade (notmuch, upgrade_print_progress,
&add_files_state);
printf ("Your notmuch database has now been upgraded to database format version %u.\n",
notmuch_database_get_version (notmuch));
}
- add_files_state.total_files = 0;
+ print_ctx.total_files = 0;
}
if (notmuch == NULL)
@@ -777,6 +263,7 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
/* Setup our handler for SIGINT. We do this after having
* potentially done a database upgrade we this interrupt handler
* won't support. */
+ add_files_state.interrupted = 0;
memset (&action, 0, sizeof (struct sigaction));
action.sa_handler = handle_sigint;
sigemptyset (&action.sa_mask);
@@ -788,33 +275,12 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
add_files_state.processed_files = 0;
add_files_state.added_messages = 0;
- gettimeofday (&add_files_state.tv_start, NULL);
-
- add_files_state.removed_files = _filename_list_create (ctx);
- add_files_state.removed_directories = _filename_list_create (ctx);
-
- ret = add_files (notmuch, db_path, &add_files_state);
-
- removed_files = 0;
- renamed_files = 0;
- for (f = add_files_state.removed_files->head; f; f = f->next) {
- status = notmuch_database_remove_message (notmuch, f->filename);
- if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID)
- renamed_files++;
- else
- removed_files++;
- }
-
- for (f = add_files_state.removed_directories->head; f; f = f->next) {
- _remove_directory (ctx, notmuch, f->filename,
- &renamed_files, &removed_files);
- }
+ gettimeofday (&print_ctx.tv_start, NULL);
- talloc_free (add_files_state.removed_files);
- talloc_free (add_files_state.removed_directories);
+ ret = add_files (mailstore, db_path, &add_files_state);
gettimeofday (&tv_now, NULL);
- elapsed = notmuch_time_elapsed (add_files_state.tv_start,
+ elapsed = notmuch_time_elapsed (print_ctx.tv_start,
tv_now);
if (add_files_state.processed_files) {
@@ -839,16 +305,16 @@ notmuch_new_command (void *ctx, int argc, char *argv[])
printf ("No new mail.");
}
- if (removed_files) {
+ if (add_files_state.removed_files) {
printf (" Removed %d %s.",
- removed_files,
- removed_files == 1 ? "message" : "messages");
+ add_files_state.removed_files,
+ add_files_state.removed_files == 1 ? "message" : "messages");
}
- if (renamed_files) {
+ if (add_files_state.renamed_files) {
printf (" Detected %d file %s.",
- renamed_files,
- renamed_files == 1 ? "rename" : "renames");
+ add_files_state.renamed_files,
+ add_files_state.renamed_files == 1 ? "rename" : "renames");
}
printf ("\n");
--
1.7.0.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 3/4] Access messages through mail store interface
2010-04-08 14:42 [PATCH 0/4] Mailstore abstraction v4 Michal Sojka
2010-04-08 14:42 ` [PATCH 1/4] Mailstore abstraction interface Michal Sojka
2010-04-08 14:42 ` [PATCH 2/4] Conversion to mailstore abstraction Michal Sojka
@ 2010-04-08 14:42 ` Michal Sojka
2010-04-08 14:42 ` [PATCH 4/4] Add 'cat' subcommand Michal Sojka
2010-04-13 18:43 ` [PATCH 0/4] Mailstore abstraction v4 Carl Worth
4 siblings, 0 replies; 10+ messages in thread
From: Michal Sojka @ 2010-04-08 14:42 UTC (permalink / raw)
To: notmuch
This patch modifies notmuch binary to access the messages through mail
store interface, so that non-file based mail stores can also be
implemented.
The API of notmuch library was changed. Now,
notmuch_message_get_filename() returns relative file name with respect
to the database path. As a result, notmuch show also outputs relative
paths so that MUAs need to be changed.
Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
---
lib/database.cc | 14 +++++++++++---
lib/index.cc | 8 ++++++--
lib/mailstore-files.c | 18 +++++++++++++++++-
lib/message-file.c | 8 ++++----
lib/message.cc | 33 +++++++++++++++++++++++++--------
lib/notmuch-private.h | 6 +++---
lib/notmuch.h | 16 ++++++++++++++--
lib/sha1.c | 6 +-----
notmuch-client.h | 2 +-
notmuch-reply.c | 10 +++++++++-
notmuch-show.c | 14 ++++++++++++--
show-message.c | 14 +-------------
12 files changed, 104 insertions(+), 45 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 93c8d0f..bd64ed3 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1374,6 +1374,7 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
notmuch_message_t *message = NULL;
notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
notmuch_private_status_t private_status;
+ const char *relative;
const char *date, *header;
const char *from, *to, *subject;
@@ -1386,7 +1387,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
if (ret)
return ret;
- message_file = notmuch_message_file_open (filename);
+ relative = _notmuch_database_relative_path (notmuch, filename);
+ message_file = notmuch_message_file_open (notmuch->mailstore, relative);
if (message_file == NULL)
return NOTMUCH_STATUS_FILE_ERROR;
@@ -1438,9 +1440,15 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
}
if (message_id == NULL ) {
+ FILE *file;
+ char *sha1 = NULL;
/* No message-id at all, let's generate one by taking a
* hash over the file's contents. */
- char *sha1 = notmuch_sha1_of_file (filename);
+ file = notmuch_mailstore_open_file (notmuch->mailstore, relative);
+ if (file) {
+ sha1 = notmuch_sha1_of_file (file);
+ fclose (file);
+ }
/* If that failed too, something is really wrong. Give up. */
if (sha1 == NULL) {
@@ -1483,7 +1491,7 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
date = notmuch_message_file_get_header (message_file, "date");
_notmuch_message_set_date (message, date);
- _notmuch_message_index_file (message, filename);
+ _notmuch_message_index_file (message, relative);
} else {
ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
}
diff --git a/lib/index.cc b/lib/index.cc
index cf93025..4d8c4dd 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -425,15 +425,19 @@ _notmuch_message_index_file (notmuch_message_t *message,
const char *from, *subject;
notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
static int initialized = 0;
+ notmuch_mailstore_t *mailstore;
if (! initialized) {
g_mime_init (0);
initialized = 1;
}
- file = fopen (filename, "r");
+ mailstore = notmuch_message_get_mailstore(message);
+ file = notmuch_mailstore_open_file (mailstore, filename);
if (! file) {
- fprintf (stderr, "Error opening %s: %s\n", filename, strerror (errno));
+ fprintf (stderr, "Error opening %s: %s\n",
+ notmuch_message_get_filename (message),
+ strerror (errno));
ret = NOTMUCH_STATUS_FILE_ERROR;
goto DONE;
}
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
index 8b5e1d5..f2cb8d7 100644
--- a/lib/mailstore-files.c
+++ b/lib/mailstore-files.c
@@ -602,11 +602,27 @@ index_new(notmuch_mailstore_t *mailstore, const char* path,
return ret;
}
+static FILE *
+open_file(notmuch_mailstore_t *mailstore, const char *filename)
+{
+ const char *db_path;
+ char *abs_filename;
+ FILE *file;
+
+ db_path = notmuch_database_get_path(mailstore->notmuch);
+ abs_filename = talloc_asprintf(NULL, "%s/%s", db_path, filename);
+ if (unlikely(abs_filename == NULL))
+ return NULL;
+ file = fopen (abs_filename, "r");
+ talloc_free(abs_filename);
+ return file;
+}
+
/* Original notmuch mail store */
struct _notmuch_mailstore mailstore_files = {
.type = "files",
.count_files = count_files,
.index_new = index_new,
.sync_tags = NULL, /* We cannot store tags in this mailstore. */
- .open_file = NULL, /* Currently not implemented */
+ .open_file = open_file,
};
diff --git a/lib/message-file.c b/lib/message-file.c
index 0c152a3..13c9f4c 100644
--- a/lib/message-file.c
+++ b/lib/message-file.c
@@ -94,7 +94,7 @@ _notmuch_message_file_destructor (notmuch_message_file_t *message)
/* Create a new notmuch_message_file_t for 'filename' with 'ctx' as
* the talloc owner. */
notmuch_message_file_t *
-_notmuch_message_file_open_ctx (void *ctx, const char *filename)
+_notmuch_message_file_open_ctx (void *ctx, notmuch_mailstore_t *mailstore, const char *filename)
{
notmuch_message_file_t *message;
@@ -104,7 +104,7 @@ _notmuch_message_file_open_ctx (void *ctx, const char *filename)
talloc_set_destructor (message, _notmuch_message_file_destructor);
- message->file = fopen (filename, "r");
+ message->file = notmuch_mailstore_open_file(mailstore, filename);
if (message->file == NULL)
goto FAIL;
@@ -126,9 +126,9 @@ _notmuch_message_file_open_ctx (void *ctx, const char *filename)
}
notmuch_message_file_t *
-notmuch_message_file_open (const char *filename)
+notmuch_message_file_open (notmuch_mailstore_t *mailstore, const char *filename)
{
- return _notmuch_message_file_open_ctx (NULL, filename);
+ return _notmuch_message_file_open_ctx (NULL, mailstore, filename);
}
void
diff --git a/lib/message.cc b/lib/message.cc
index c32ee7d..c7eff7c 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -247,6 +247,7 @@ static void
_notmuch_message_ensure_message_file (notmuch_message_t *message)
{
const char *filename;
+ notmuch_mailstore_t *mailstore;
if (message->message_file)
return;
@@ -255,7 +256,9 @@ _notmuch_message_ensure_message_file (notmuch_message_t *message)
if (unlikely (filename == NULL))
return;
- message->message_file = _notmuch_message_file_open_ctx (message, filename);
+ mailstore = notmuch_message_get_mailstore (message);
+
+ message->message_file = _notmuch_message_file_open_ctx (message, mailstore, filename);
}
const char *
@@ -429,7 +432,7 @@ notmuch_message_get_filename (notmuch_message_t *message)
int prefix_len = strlen (prefix);
Xapian::TermIterator i;
char *colon, *direntry = NULL;
- const char *db_path, *directory, *basename;
+ const char *directory, *basename;
unsigned int directory_id;
void *local = talloc_new (message);
@@ -474,18 +477,16 @@ notmuch_message_get_filename (notmuch_message_t *message)
*colon = '\0';
- db_path = notmuch_database_get_path (message->notmuch);
-
directory = _notmuch_database_get_directory_path (local,
message->notmuch,
directory_id);
if (strlen (directory))
- message->filename = talloc_asprintf (message, "%s/%s/%s",
- db_path, directory, basename);
- else
message->filename = talloc_asprintf (message, "%s/%s",
- db_path, basename);
+ directory, basename);
+ else
+ message->filename = talloc_asprintf (message, "%s",
+ basename);
talloc_free ((void *) directory);
talloc_free (local);
@@ -493,6 +494,22 @@ notmuch_message_get_filename (notmuch_message_t *message)
return message->filename;
}
+FILE *
+notmuch_message_fopen (notmuch_message_t *message)
+{
+ const char *filename;
+ filename = notmuch_message_get_filename (message);
+ return notmuch_mailstore_open_file (message->notmuch->mailstore,
+ filename);
+}
+
+notmuch_mailstore_t *
+notmuch_message_get_mailstore (notmuch_message_t *message)
+{
+ return message->notmuch->mailstore;
+}
+
+
notmuch_bool_t
notmuch_message_get_flag (notmuch_message_t *message,
notmuch_message_flag_t flag)
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index d52d84d..bab2090 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -295,11 +295,11 @@ typedef struct _notmuch_message_file notmuch_message_file_t;
* Returns NULL if any error occurs.
*/
notmuch_message_file_t *
-notmuch_message_file_open (const char *filename);
+notmuch_message_file_open (notmuch_mailstore_t *mailstore, const char *filename);
/* Like notmuch_message_file_open but with 'ctx' as the talloc owner. */
notmuch_message_file_t *
-_notmuch_message_file_open_ctx (void *ctx, const char *filename);
+_notmuch_message_file_open_ctx (void *ctx, notmuch_mailstore_t *mailstore, const char *filename);
/* Close a notmuch message previously opened with notmuch_message_open. */
void
@@ -402,7 +402,7 @@ char *
notmuch_sha1_of_string (const char *str);
char *
-notmuch_sha1_of_file (const char *filename);
+notmuch_sha1_of_file (FILE *file);
/* tags.c */
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 31e47a4..54de0bd 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -739,8 +739,8 @@ notmuch_message_get_replies (notmuch_message_t *message);
/* Get a filename for the email corresponding to 'message'.
*
- * The returned filename is an absolute filename, (the initial
- * component will match notmuch_database_get_path() ).
+ * The returned filename is an relative filename of the message within
+ * the mail store.
*
* The returned string belongs to the message so should not be
* modified or freed by the caller (nor should it be referenced after
@@ -754,6 +754,18 @@ notmuch_message_get_replies (notmuch_message_t *message);
const char *
notmuch_message_get_filename (notmuch_message_t *message);
+/* Return file handle to read the content of the message.
+ *
+ * This is a helper function which determines message filename and
+ * calls notmuch_mailstore_open_file().
+ */
+FILE *
+notmuch_message_fopen (notmuch_message_t *message);
+
+/* Get a pointer to the mailstore where the message is stored */
+notmuch_mailstore_t *
+notmuch_message_get_mailstore (notmuch_message_t *message);
+
/* Message flags */
typedef enum _notmuch_message_flag {
NOTMUCH_MESSAGE_FLAG_MATCH,
diff --git a/lib/sha1.c b/lib/sha1.c
index cc48108..a8991b1 100644
--- a/lib/sha1.c
+++ b/lib/sha1.c
@@ -74,9 +74,8 @@ notmuch_sha1_of_string (const char *str)
* file not found, etc.), this function returns NULL.
*/
char *
-notmuch_sha1_of_file (const char *filename)
+notmuch_sha1_of_file (FILE *file)
{
- FILE *file;
#define BLOCK_SIZE 4096
unsigned char block[BLOCK_SIZE];
size_t bytes_read;
@@ -84,7 +83,6 @@ notmuch_sha1_of_file (const char *filename)
unsigned char digest[SHA1_DIGEST_SIZE];
char *result;
- file = fopen (filename, "r");
if (file == NULL)
return NULL;
@@ -108,8 +106,6 @@ notmuch_sha1_of_file (const char *filename)
result = _hex_of_sha1_digest (digest);
- fclose (file);
-
return result;
}
diff --git a/notmuch-client.h b/notmuch-client.h
index d8c8df4..728e448 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -126,7 +126,7 @@ char *
query_string_from_args (void *ctx, int argc, char *argv[]);
notmuch_status_t
-show_message_body (const char *filename,
+show_message_body (FILE *file,
void (*show_part) (GMimeObject *part, int *part_count));
notmuch_status_t
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 1ec28cd..32d17ef 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -347,6 +347,7 @@ guess_from_received_header (notmuch_config_t *config, notmuch_message_t *message
static int
notmuch_reply_format_default(void *ctx, notmuch_config_t *config, notmuch_query_t *query)
{
+ FILE *file;
GMimeMessage *reply;
notmuch_messages_t *messages;
notmuch_message_t *message;
@@ -415,7 +416,14 @@ notmuch_reply_format_default(void *ctx, notmuch_config_t *config, notmuch_query_
notmuch_message_get_header (message, "date"),
notmuch_message_get_header (message, "from"));
- show_message_body (notmuch_message_get_filename (message), reply_part);
+ file = notmuch_message_fopen (message);
+ if (file) {
+ show_message_body (file, reply_part);
+ fclose (file);
+ } else
+ fprintf (stderr, "Error opening %s: %s\n",
+ notmuch_message_get_filename (message),
+ strerror (errno));
notmuch_message_destroy (message);
}
diff --git a/notmuch-show.c b/notmuch-show.c
index 6dca672..66fd773 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -339,6 +339,8 @@ format_part_json (GMimeObject *part, int *part_count)
static void
show_message (void *ctx, const show_format_t *format, notmuch_message_t *message, int indent)
{
+ FILE *file;
+
fputs (format->message_start, stdout);
if (format->message)
format->message(ctx, message, indent);
@@ -349,8 +351,16 @@ show_message (void *ctx, const show_format_t *format, notmuch_message_t *message
fputs (format->header_end, stdout);
fputs (format->body_start, stdout);
- if (format->part)
- show_message_body (notmuch_message_get_filename (message), format->part);
+ if (format->part) {
+ file = notmuch_message_fopen (message);
+ if (file) {
+ show_message_body (file, format->part);
+ fclose (file);
+ } else
+ fprintf (stderr, "Error opening %s: %s\n",
+ notmuch_message_get_filename (message),
+ strerror (errno));
+ }
fputs (format->body_end, stdout);
fputs (format->message_end, stdout);
diff --git a/show-message.c b/show-message.c
index b1b61be..79911a7 100644
--- a/show-message.c
+++ b/show-message.c
@@ -60,23 +60,15 @@ show_message_part (GMimeObject *part, int *part_count,
}
notmuch_status_t
-show_message_body (const char *filename,
+show_message_body (FILE *file,
void (*show_part) (GMimeObject *part, int *part_count))
{
GMimeStream *stream = NULL;
GMimeParser *parser = NULL;
GMimeMessage *mime_message = NULL;
notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
- FILE *file = NULL;
int part_count = 0;
- file = fopen (filename, "r");
- if (! file) {
- fprintf (stderr, "Error opening %s: %s\n", filename, strerror (errno));
- ret = NOTMUCH_STATUS_FILE_ERROR;
- goto DONE;
- }
-
stream = g_mime_stream_file_new (file);
g_mime_stream_file_set_owner (GMIME_STREAM_FILE (stream), FALSE);
@@ -87,7 +79,6 @@ show_message_body (const char *filename,
show_message_part (g_mime_message_get_mime_part (mime_message),
&part_count, show_part);
- DONE:
if (mime_message)
g_object_unref (mime_message);
@@ -97,9 +88,6 @@ show_message_body (const char *filename,
if (stream)
g_object_unref (stream);
- if (file)
- fclose (file);
-
return ret;
}
--
1.7.0.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 4/4] Add 'cat' subcommand
2010-04-08 14:42 [PATCH 0/4] Mailstore abstraction v4 Michal Sojka
` (2 preceding siblings ...)
2010-04-08 14:42 ` [PATCH 3/4] Access messages through mail store interface Michal Sojka
@ 2010-04-08 14:42 ` Michal Sojka
2010-04-13 18:43 ` [PATCH 0/4] Mailstore abstraction v4 Carl Worth
4 siblings, 0 replies; 10+ messages in thread
From: Michal Sojka @ 2010-04-08 14:42 UTC (permalink / raw)
To: notmuch
This command dumps raw message identified by filename to standard
output. It uses mail store interface to get the message from the right
place.
notmuch.el was modified to use this command to access the raw message
content (view/save attachments, view raw message and pipe message to
command).
With this patch, it is straightforward to use notmuch emacs interface
remotely over SSH. To do this, it is sufficient to redefine
notmuch-command variable to contain the name of a script containing:
ssh user@host notmuch "$@"
Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
---
NEWS | 3 ++
emacs/notmuch-show.el | 11 ++++++--
notmuch-client.h | 3 ++
notmuch-show.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++
notmuch.c | 4 +++
5 files changed, 80 insertions(+), 3 deletions(-)
diff --git a/NEWS b/NEWS
index f29ac27..3bd21fa 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,6 @@
+A new subcommand 'cat' was added, which simplifies use of Emacs
+interface with remote database (accessed over SSH).
+
Notmuch 0.1 (2010-04-05)
========================
This is the first release of the notmuch mail system.
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6f5a55d..45d49d1 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -328,7 +328,11 @@ buffer."
(defun notmuch-show-view-raw-message ()
"View the raw email of the current message."
(interactive)
- (view-file (notmuch-show-get-filename)))
+ (let ((filename (notmuch-show-get-filename)))
+ (let ((buf (get-buffer-create (concat "*notmuch-raw-" filename "*"))))
+ (switch-to-buffer buf)
+ (save-excursion
+ (call-process notmuch-command nil t nil "cat" filename)))))
(defmacro with-current-notmuch-show-message (&rest body)
"Evaluate body with current buffer set to the text of current message"
@@ -336,7 +340,7 @@ buffer."
(let ((filename (notmuch-show-get-filename)))
(let ((buf (generate-new-buffer (concat "*notmuch-msg-" filename "*"))))
(with-current-buffer buf
- (insert-file-contents filename nil nil nil t)
+ (call-process notmuch-command nil t nil "cat" filename)
,@body)
(kill-buffer buf)))))
@@ -390,7 +394,8 @@ current email message as stdin. Anything printed by the command
to stdout or stderr will appear in the *Messages* buffer."
(interactive "sPipe message to command: ")
(apply 'start-process-shell-command "notmuch-pipe-command" "*notmuch-pipe*"
- (list command " < " (shell-quote-argument (notmuch-show-get-filename)))))
+ (list notmuch-command "cat"
+ (shell-quote-argument (notmuch-show-get-filename) " | " command))))
(defun notmuch-show-move-to-current-message-summary-line ()
"Move to the beginning of the one-line summary of the current message.
diff --git a/notmuch-client.h b/notmuch-client.h
index 728e448..666e70d 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -111,6 +111,9 @@ int
notmuch_search_tags_command (void *ctx, int argc, char *argv[]);
int
+notmuch_cat_command (void *ctx, int argc, char *argv[]);
+
+int
notmuch_part_command (void *ctx, int argc, char *argv[]);
const char *
diff --git a/notmuch-show.c b/notmuch-show.c
index 66fd773..dbab9a7 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -515,6 +515,68 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[]))
}
int
+notmuch_cat_command (void *ctx, unused (int argc), unused (char *argv[]))
+{
+ notmuch_config_t *config;
+ notmuch_database_t *notmuch;
+ notmuch_mailstore_t *mailstore;
+ int i;
+ FILE *file;
+ const char *filename;
+ size_t size;
+ char buf[4096];
+
+ for (i = 0; i < argc && argv[i][0] == '-'; i++) {
+/* if (STRNCMP_LITERAL (argv[i], "--part=") == 0) { */
+/* part = atoi(argv[i] + sizeof ("--part=") - 1); */
+/* } else { */
+ fprintf (stderr, "Unrecognized option: %s\n", argv[i]);
+/* return 1; */
+/* } */
+ }
+
+ argc -= i;
+ argv += i;
+
+ if (argc == 0) {
+ fprintf (stderr, "Error: No filename given\n");
+ return 1;
+ }
+ filename = argv[0];
+
+ config = notmuch_config_open (ctx, NULL, NULL);
+ if (config == NULL)
+ return 1;
+
+ mailstore = notmuch_config_get_mailstore (config);
+
+ if (mailstore == NULL) {
+ fprintf (stderr, "Error: I have no mailstore\n");
+ return 1;
+ }
+
+ notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
+ NOTMUCH_DATABASE_MODE_READ_ONLY,
+ mailstore);
+
+ file = notmuch_mailstore_open_file(mailstore, filename);
+ if (file == NULL) {
+ fprintf (stderr, "Error: Cannot open %s in %s: %s\n", filename,
+ notmuch_mailstore_get_type (mailstore), strerror (errno));
+ return 1;
+ }
+ while (!feof (file)) {
+ size = fread(buf, 1, sizeof(buf), file);
+ fwrite (buf, size, 1, stdout);
+ }
+ fclose (file);
+
+ notmuch_database_close (notmuch);
+
+ return 0;
+}
+
+int
notmuch_part_command (void *ctx, unused (int argc), unused (char *argv[]))
{
notmuch_config_t *config;
diff --git a/notmuch.c b/notmuch.c
index f5669fc..bcebcdc 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -294,6 +294,10 @@ command_t commands[] = {
"\tcontain tags only from messages that match the search-term(s).\n"
"\n"
"\tIn both cases the list will be alphabetically sorted." },
+ { "cat", notmuch_cat_command,
+ "<path>",
+ "Dump raw message identified by path to standard output.",
+ "" },
{ "part", notmuch_part_command,
"--part=<num> <search-terms>",
"Output a single MIME part of a message.",
--
1.7.0.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 1/4] Mailstore abstraction interface
2010-04-08 14:42 ` [PATCH 1/4] Mailstore abstraction interface Michal Sojka
@ 2010-04-13 17:53 ` Carl Worth
2010-04-13 21:49 ` Stewart Smith
2010-04-14 8:50 ` Michal Sojka
0 siblings, 2 replies; 10+ messages in thread
From: Carl Worth @ 2010-04-13 17:53 UTC (permalink / raw)
To: Michal Sojka, notmuch
[-- Attachment #1: Type: text/plain, Size: 1638 bytes --]
On Thu, 8 Apr 2010 16:42:43 +0200, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> The goal of mailstore abstraction is to allow notmuch to store tags
> together with email messages. The abstract interface is needed because
> people want to use different ways of storing their emails. Currently,
> there exists implementation for two types of mailstore - plain files
> and maildir. It is expected that additional git-based mailstore will
> be developed later.
I don't agree with the approach being taken here.
I don't think that the expectation of future need is a good basis for
adding a level of abstraction now. This gives us current costs without
benefit.
Meanwhile, the two different mail stores that you are currently support
("plain" and "maildir") aren't really different types. We do already
have code within notmuch to detect whether any particular directory is a
maildir, (with the heuristic of looking for child directories named
"cur", "new", and "tmp"). So I think we can and should support both
maildir and non-maildir mail storage just fine.
The question of "once you have maildir storage, should you synchronize
that tags" is quite separate. The answer there might be "yes", "no", or
"let the user decide". But if it is configurable, then the configuration
should be something like
[maildir]
synchronize_flags=yes
rather than
[mailstore]
type=maildir
This series is looking like one of the most complete approaches to
maildir-flag synchronization, (and I like some of the motivation that
leads to "notmuch cat"). But I think the mailstore abstraction is
largely a distraction from the real features here.
-Carl
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/4] Mailstore abstraction v4
2010-04-08 14:42 [PATCH 0/4] Mailstore abstraction v4 Michal Sojka
` (3 preceding siblings ...)
2010-04-08 14:42 ` [PATCH 4/4] Add 'cat' subcommand Michal Sojka
@ 2010-04-13 18:43 ` Carl Worth
4 siblings, 0 replies; 10+ messages in thread
From: Carl Worth @ 2010-04-13 18:43 UTC (permalink / raw)
To: Michal Sojka, notmuch
[-- Attachment #1: Type: text/plain, Size: 920 bytes --]
On Thu, 8 Apr 2010 16:42:42 +0200, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> My biggest question relates to the first patch, which does an
> incompatible change to libnotmuch API. After reading RELEASING file, I
> found that this change is probably not what Carl wants to merge (and I
> understand that) so I'd like to get some feedback on my suggestion in
> that patch.
[I composed this message a while ago, but failed to get it through the
mailing-list moderation until now. It may be largely irrelevant in light
of my more recent review of the mailstore abstraction, but here it is.]
I haven't looked closely at the implementation here, but a quick glance
at the API/ABI change suggests an easy answer:
Why not just add a new function that accepts the mailstore type, and
preserve the original function, (which would then simply call the new
function with an argument of "files" for the mailstore type).
-Carl
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/4] Mailstore abstraction interface
2010-04-13 17:53 ` Carl Worth
@ 2010-04-13 21:49 ` Stewart Smith
2010-04-14 8:50 ` Michal Sojka
1 sibling, 0 replies; 10+ messages in thread
From: Stewart Smith @ 2010-04-13 21:49 UTC (permalink / raw)
To: Carl Worth, Michal Sojka, notmuch
On Tue, 13 Apr 2010 10:53:12 -0700, Carl Worth <cworth@cworth.org> wrote:
> This series is looking like one of the most complete approaches to
> maildir-flag synchronization, (and I like some of the motivation that
> leads to "notmuch cat"). But I think the mailstore abstraction is
> largely a distraction from the real features here.
For my case (of wanting to have backup of my mailstore complete in
reasonable time, preferably using less disk space) of wanting mail in
git packs, 'notmuch cat' being used everywhere removes a lot of the
issues of doing this.
(pluggin in an alternative to readdir is fairly simple... but the emacs
UI needs to read from it too :)
--
Stewart Smith
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/4] Mailstore abstraction interface
2010-04-13 17:53 ` Carl Worth
2010-04-13 21:49 ` Stewart Smith
@ 2010-04-14 8:50 ` Michal Sojka
1 sibling, 0 replies; 10+ messages in thread
From: Michal Sojka @ 2010-04-14 8:50 UTC (permalink / raw)
To: Carl Worth, notmuch
On Tue, 13 Apr 2010, Carl Worth wrote:
> On Thu, 8 Apr 2010 16:42:43 +0200, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> > The goal of mailstore abstraction is to allow notmuch to store tags
> > together with email messages. The abstract interface is needed because
> > people want to use different ways of storing their emails. Currently,
> > there exists implementation for two types of mailstore - plain files
> > and maildir. It is expected that additional git-based mailstore will
> > be developed later.
>
> I don't agree with the approach being taken here.
>
> I don't think that the expectation of future need is a good basis for
> adding a level of abstraction now. This gives us current costs without
> benefit.
Thanks for the review, Carl. Since I'm interested in further development
of mailstore abstraction until it is really useful, I'd like to make the
patch series as small as possible to reduce maintenance burden. I think
I can extract some "real features" such as cat subcommand from my
patches and send them separately, perhaps in 0.3 merge window.
> Meanwhile, the two different mail stores that you are currently support
> ("plain" and "maildir") aren't really different types. We do already
> have code within notmuch to detect whether any particular directory is a
> maildir, (with the heuristic of looking for child directories named
> "cur", "new", and "tmp"). So I think we can and should support both
> maildir and non-maildir mail storage just fine.
>
> The question of "once you have maildir storage, should you synchronize
> that tags" is quite separate. The answer there might be "yes", "no", or
> "let the user decide". But if it is configurable, then the configuration
> should be something like
>
> [maildir]
> synchronize_flags=yes
>
> rather than
>
> [mailstore]
> type=maildir
Yes, these two mail stores share a lot of code and there is only a small
difference between them. I agree that it is cleaner for users to see
them as a single mailstore type and use configuration options to change
the behavior.
-Michal
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2010-04-14 14:17 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-04-08 14:42 [PATCH 0/4] Mailstore abstraction v4 Michal Sojka
2010-04-08 14:42 ` [PATCH 1/4] Mailstore abstraction interface Michal Sojka
2010-04-13 17:53 ` Carl Worth
2010-04-13 21:49 ` Stewart Smith
2010-04-14 8:50 ` Michal Sojka
2010-04-08 14:42 ` [PATCH 2/4] Conversion to mailstore abstraction Michal Sojka
2010-04-08 14:42 ` [PATCH 3/4] Access messages through mail store interface Michal Sojka
2010-04-08 14:42 ` [PATCH 4/4] Add 'cat' subcommand Michal Sojka
2010-04-13 18:43 ` [PATCH 0/4] Mailstore abstraction v4 Carl Worth
-- strict thread matches above, loose matches on Subject: below --
2010-03-18 15:39 Mailstore abstraction & maildir synchronization Michal Sojka
2010-03-18 15:39 ` [PATCH 1/4] Mailstore abstraction interface Michal Sojka
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).