unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
From: Mark Walters <markwalters1009@gmail.com>
To: notmuch@notmuchmail.org
Subject: [PATCH] dump: make dump take Xapian write lock
Date: Mon, 23 Jun 2014 21:12:29 +0100	[thread overview]
Message-ID: <1403554349-8888-1-git-send-email-markwalters1009@gmail.com> (raw)
In-Reply-To: <87zjhh67e7.fsf@qmul.ac.uk>

Dump currently only takes the read lock. Xapian can cope with some
changes while maintaining a read snapshot but with more changes it
fails. Currently notmuch just gives a xapian error.

To avoid this we take the write lock when dumping. This prevents other
notmuch processes from modifying the xapian database preventing this
error.

Discussion with Olly on irc indicates that this is currently the best
solution: in xapian trunk there may be better possibilities using
snapshots but they need to make it to a release and propogate out to
users before we can switch approach.

Finally, this breaks one use case: pipelines of the form

notmuch dump | ... | notmuch restore

According to Olly this is already very fragile: it will only work on
small databases. One of the tests relies on this behaviour so fix that
to store the dump rather than use a pipe.
---
I haven't tested this much: all tests pass (with the one fix) and it is simple.

Best wishes

Mark


 notmuch-dump.c       |    2 +-
 test/T150-tagging.sh |    4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/notmuch-dump.c b/notmuch-dump.c
index 887a208..9c6ad7f 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -212,7 +212,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[])
     int ret;
 
     if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_ONLY, &notmuch))
+			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
 	return EXIT_FAILURE;
 
     char *output_file_name = NULL;
diff --git a/test/T150-tagging.sh b/test/T150-tagging.sh
index dc118f3..45471ac 100755
--- a/test/T150-tagging.sh
+++ b/test/T150-tagging.sh
@@ -247,8 +247,8 @@ ${TEST_DIRECTORY}/random-corpus --config-path=${NOTMUCH_CONFIG} \
 notmuch dump --format=batch-tag | sed 's/^.* -- /+common_tag -- /' | \
     sort > EXPECTED
 
-notmuch dump --format=batch-tag | sed 's/^.* -- /  -- /' | \
-    notmuch restore --format=batch-tag
+notmuch dump --format=batch-tag | sed 's/^.* -- /  -- /' > INTERMEDIATE_STEP
+notmuch restore --format=batch-tag < INTERMEDIATE_STEP
 
 notmuch tag --batch < EXPECTED
 
-- 
1.7.10.4

  reply	other threads:[~2014-06-23 20:12 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-06-06  8:03 notmuch dump: taking write-lock to protect from concurrent (cronned) notmuch new? Maarten Aertsen
2014-06-06 11:46 ` Mark Walters
2014-06-12 22:56   ` David Bremner
2014-06-12 23:21     ` Mark Walters
2014-06-23 20:12       ` Mark Walters [this message]
2014-07-15 14:10         ` [PATCH] dump: make dump take Xapian write lock David Bremner
2014-07-16  6:50           ` Mark Walters
2014-07-16 22:36         ` David Bremner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://notmuchmail.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1403554349-8888-1-git-send-email-markwalters1009@gmail.com \
    --to=markwalters1009@gmail.com \
    --cc=notmuch@notmuchmail.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).