From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 1F79F431FC0 for ; Tue, 15 Jul 2014 23:51:15 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: 0.502 X-Spam-Level: X-Spam-Status: No, score=0.502 tagged_above=-999 required=5 tests=[DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001, NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hI9ybJZr3EGN for ; Tue, 15 Jul 2014 23:51:07 -0700 (PDT) Received: from mail1.qmul.ac.uk (mail1.qmul.ac.uk [138.37.6.7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id DBFCD431FBD for ; Tue, 15 Jul 2014 23:51:06 -0700 (PDT) Received: from smtp.qmul.ac.uk ([138.37.6.40]) by mail1.qmul.ac.uk with esmtp (Exim 4.71) (envelope-from ) id 1X7J3N-0005fj-Ba; Wed, 16 Jul 2014 07:51:01 +0100 Received: from [194.42.225.51] (helo=localhost) by smtp.qmul.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.71) (envelope-from ) id 1X7J3N-0004w5-5d; Wed, 16 Jul 2014 07:51:01 +0100 From: Mark Walters To: David Bremner , notmuch@notmuchmail.org Subject: Re: [PATCH] dump: make dump take Xapian write lock In-Reply-To: <87y4vuaf10.fsf@maritornes.cs.unb.ca> References: <87zjhh67e7.fsf@qmul.ac.uk> <1403554349-8888-1-git-send-email-markwalters1009@gmail.com> <87y4vuaf10.fsf@maritornes.cs.unb.ca> User-Agent: Notmuch/0.15.2+615~g78e3a93 (http://notmuchmail.org) Emacs/23.4.1 (x86_64-pc-linux-gnu) Date: Wed, 16 Jul 2014 07:50:41 +0100 Message-ID: <871ttlztj2.fsf@qmul.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-QM-SPAM-Info: 194.42.225.51 is in Janet mirror of Spamhaus XBL; see http://www.spamhaus.org/xbl/ X-Sender-Host-Address: 194.42.225.51 X-QM-Geographic: According to ripencc, this message was delivered by a machine in Britain (UK) (GB). X-QM-SPAM-Info: Sender has good ham record. :) X-QM-Body-MD5: 402ab17fd2c5d35e4d7b1c38f424c9e1 (of first 20000 bytes) X-SpamAssassin-Score: -0.1 X-SpamAssassin-SpamBar: / X-SpamAssassin-Report: The QM spam filters have analysed this message to determine if it is spam. We require at least 5.0 points to mark a message as spam. This message scored -0.1 points. Summary of the scoring: * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider * (markwalters1009[at]gmail.com) * -0.1 AWL AWL: From: address is in the auto white-list X-QM-Scan-Virus: ClamAV says the message is clean X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 06:51:15 -0000 On Tue, 15 Jul 2014, David Bremner wrote: > Mark Walters writes: > >> >> Discussion with Olly on irc indicates that this is currently the best >> solution: in xapian trunk there may be better possibilities using >> snapshots but they need to make it to a release and propogate out to >> users before we can switch approach. > > I agree that this seems to be the only feasible approach to make dump > atomic. I'm a little unsure about the benefits to the end user though. > Currently, the user has to check the return code of dump to ensure it > completed correctly. With this change, the user will still have to > check that dump did not error out when trying to acquire a write lock. > The following ticket > > http://trac.xapian.org/ticket/275 > > suggests that if we wanted to a blocking open, we'd be on our own. > > Did I miss something here about the benefits? I agree that failing > earlier is nicer, but is that it? You are correct that we may still fail. I think there are two advantages in addition to failing earlier. First, I think that if the dump fails there will not be any output file: I think the old version would leave a partial dump (note I haven't checked that carefully). This makes it more likely the failing dump is noticed. Secondly, this version will only fail if the database is being written to when it starts: the old version would fail if something attempted to write to it at any point during the dump, which can be relatively lengthy. This should at least reduce the frequency of problems. Best wishes Mark