From: Austin Clements <amdragon@MIT.EDU>
To: notmuch@notmuchmail.org
Subject: [PATCH 3/3] lib: Reject multi-message mboxes and deprecate single-message mbox
Date: Sun, 25 Nov 2012 01:16:01 -0500 [thread overview]
Message-ID: <1353824161-31717-3-git-send-email-amdragon@mit.edu> (raw)
In-Reply-To: <1353824161-31717-1-git-send-email-amdragon@mit.edu>
Previously, we would treat multi-message mboxes as one giant email,
which, besides the obvious incorrect indexing, often led to
out-of-memory errors for archival mboxes. Now we explicitly reject
multi-message mboxes. For historical reasons, we retain support for
single-message mboxes, but official deprecate this behavior.
---
lib/database.cc | 4 +++-
lib/index.cc | 28 ++++++++++++++++++++++++++++
test/new | 8 +++++---
3 files changed, 36 insertions(+), 4 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 4df3217..91d4329 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1821,7 +1821,9 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
date = notmuch_message_file_get_header (message_file, "date");
_notmuch_message_set_header_values (message, date, from, subject);
- _notmuch_message_index_file (message, filename);
+ ret = _notmuch_message_index_file (message, filename);
+ if (ret)
+ goto DONE;
} else {
ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
}
diff --git a/lib/index.cc b/lib/index.cc
index e377732..da0e6ce 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -435,6 +435,9 @@ _notmuch_message_index_file (notmuch_message_t *message,
const char *from, *subject;
notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
static int initialized = 0;
+ char from_buf[5];
+ bool is_mbox = false;
+ static bool mbox_warning = false;
if (! initialized) {
g_mime_init (0);
@@ -448,13 +451,38 @@ _notmuch_message_index_file (notmuch_message_t *message,
goto DONE;
}
+ /* Is this mbox? */
+ if (fread (from_buf, sizeof (from_buf), 1, file) == 1 &&
+ strncmp (from_buf, "From ", 5) == 0)
+ is_mbox = true;
+ rewind (file);
+
/* Evil GMime steals my FILE* here so I won't fclose it. */
stream = g_mime_stream_file_new (file);
parser = g_mime_parser_new_with_stream (stream);
+ g_mime_parser_set_scan_from (parser, is_mbox);
mime_message = g_mime_parser_construct_message (parser);
+ if (is_mbox) {
+ if (!g_mime_parser_eos (parser)) {
+ /* This is a multi-message mbox. */
+ ret = NOTMUCH_STATUS_FILE_NOT_EMAIL;
+ goto DONE;
+ }
+ /* For historical reasons, we support single-message mboxes,
+ * but this behavior is likely to change in the future, so
+ * warn. */
+ if (!mbox_warning) {
+ mbox_warning = true;
+ fprintf (stderr, "\
+Warning: %s is an mbox containing a single message,\n\
+likely caused by misconfigured mail delivery. Support for single-message\n\
+mboxes is deprecated and may be removed in the future.\n", filename);
+ }
+ }
+
from = g_mime_message_get_sender (mime_message);
addresses = internet_address_list_parse_string (from);
diff --git a/test/new b/test/new
index 29f9aff..f562cec 100755
--- a/test/new
+++ b/test/new
@@ -163,7 +163,7 @@ rm -rf "${MAIL_DIR}"/two
output=$(NOTMUCH_NEW)
test_expect_equal "$output" "No new mail. Removed 3 messages."
-test_begin_subtest "Support single-message mbox"
+test_begin_subtest "Support single-message mbox (deprecated)"
cat > "${MAIL_DIR}"/mbox_file1 <<EOF
From test_suite@notmuchmail.org Fri Jan 5 15:43:57 2001
From: Notmuch Test Suite <test_suite@notmuchmail.org>
@@ -174,11 +174,13 @@ Body.
EOF
output=$(NOTMUCH_NEW 2>&1)
test_expect_equal "$output" \
-"Added 1 new message to the database."
+"Warning: ${MAIL_DIR}/mbox_file1 is an mbox containing a single message,
+likely caused by misconfigured mail delivery. Support for single-message
+mboxes is deprecated and may be removed in the future.
+Added 1 new message to the database."
# This test requires that notmuch new has been run at least once.
test_begin_subtest "Skip and report non-mail files"
-test_subtest_known_broken
generate_message
mkdir -p "${MAIL_DIR}"/.git && touch "${MAIL_DIR}"/.git/config
touch "${MAIL_DIR}"/ignored_file
--
1.7.10.4
next prev parent reply other threads:[~2012-11-25 6:17 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-25 6:15 [PATCH 1/3] test: Test notmuch new for single-message mbox Austin Clements
2012-11-25 6:16 ` [PATCH 2/3] test: Test for ignoring multi-message mbox Austin Clements
2012-11-25 6:16 ` Austin Clements [this message]
2012-11-25 13:26 ` [PATCH 3/3] lib: Reject multi-message mboxes and deprecate single-message mbox Tomi Ollila
2012-11-25 18:05 ` Austin Clements
2012-11-25 19:57 ` Tomi Ollila
2012-11-25 11:33 ` [PATCH 1/3] test: Test notmuch new for " Mark Walters
2012-11-27 1:14 ` David Bremner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://notmuchmail.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1353824161-31717-3-git-send-email-amdragon@mit.edu \
--to=amdragon@mit.edu \
--cc=notmuch@notmuchmail.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).