From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 940776DE1D34 for ; Thu, 23 Feb 2017 18:49:07 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.005 X-Spam-Level: X-Spam-Status: No, score=-0.005 tagged_above=-999 required=5 tests=[AWL=0.006, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x-DUxMZ_lb3r for ; Thu, 23 Feb 2017 18:49:06 -0800 (PST) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by arlo.cworth.org (Postfix) with ESMTPS id E87906DE1C7F for ; Thu, 23 Feb 2017 18:49:05 -0800 (PST) Received: from remotemail by fethera.tethera.net with local (Exim 4.84_2) (envelope-from ) id 1ch5vm-0004RY-KU; Thu, 23 Feb 2017 21:48:26 -0500 Received: (nullmailer pid 13389 invoked by uid 1000); Fri, 24 Feb 2017 02:49:02 -0000 From: David Bremner To: Gaute Hope , notmuch@notmuchmail.org Subject: Re: [RFC patch 2/2] lib: handle DatabaseModifiedError in _n_message_ensure_metadata In-Reply-To: <20170224020048.10718-2-david@tethera.net> References: <87y3wx2k3s.fsf@tethera.net> <20170224020048.10718-1-david@tethera.net> <20170224020048.10718-2-david@tethera.net> Date: Thu, 23 Feb 2017 22:49:02 -0400 Message-ID: <87efyo2ti9.fsf@tethera.net> MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Feb 2017 02:49:07 -0000 David Bremner writes: > The error handling here still needs work. The retry count should be > handled in more sane way, and both running out of retries and an error > return from notmuch_database_reopen should be handled. Probably the number of retries can be limited to 1, since afaict, reopen is not going to return any recoverable errors. Gaute's test program seems run to completion without any source modifications. One general comment about this approach is that it does not do anything about any the metadata that may be cached by notmuch for other messages. Roughly speaking the behaviour should be atomic with respect to messages, but not e.g. within threads. A message read earlier may not reflect changes seen in a later one. That potential inconsistency would only occur in cases currently resulting in crashes. It is possible there is some scheme we can use to invalidate all of the cached metadata. Here's an initial idea. The notmuch_database_t object keeps a counter/timestamp which indicates the last time it was invalidated. Each message object has a counter/timestamp indicating the last time it it was synced/validated. Instead of checking if some pointer is null force a read, we'd have to compare these two counters. I think the main counter could be set to zero by database_open and incremented by database_reopen