From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 4Gh5BhvpumATOwAAgWs5BA (envelope-from ) for ; Sat, 05 Jun 2021 05:01:47 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id QEXEARvpumApVgAAbx9fmQ (envelope-from ) for ; Sat, 05 Jun 2021 03:01:47 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7790320C16 for ; Sat, 5 Jun 2021 05:01:46 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 121FB2B253; Fri, 4 Jun 2021 23:01:40 -0400 (EDT) X-Greylist: delayed 1097 seconds by postgrey-1.36 at nmbug; Fri, 04 Jun 2021 23:01:36 EDT Received: from thyestes.tartarus.org (thyestes.tartarus.org [5.196.91.86]) by mail.notmuchmail.org (Postfix) with ESMTPS id A77CB2B246 for ; Fri, 4 Jun 2021 23:01:36 -0400 (EDT) Received: from olly by thyestes.tartarus.org with local (Exim 4.92) (envelope-from ) id 1lpMHM-0001rB-26; Sat, 05 Jun 2021 03:43:16 +0100 Date: Sat, 5 Jun 2021 03:43:16 +0100 From: Olly Betts To: Felipe Contreras Subject: Re: How to recover from this permanent fatal error? Message-ID: <20210605024316.GC9932@survex.com> Mail-Followup-To: Felipe Contreras , David Bremner , "notmuch@notmuchmail.org" , xapian-discuss@lists.xapian.org References: <8735txjdmc.fsf@tethera.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Message-ID-Hash: QOI2HFUKTH6H7W2KWXG4JCXNIDYBKISN X-Message-ID-Hash: QOI2HFUKTH6H7W2KWXG4JCXNIDYBKISN X-MailFrom: olly@survex.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: "notmuch@notmuchmail.org" , xapian-discuss@lists.xapian.org X-Mailman-Version: 3.2.1 Precedence: list Reply-To: Xapian Discussion List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1622862106; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=8A1hS2ujbC3z0Odt2M7JLwuw5HA6LXR87Cxv2Uw3p4Y=; b=LFthrSHzI9d+ICXKOPunW5izPbsFYAJPzQcIBPmEY78WbjKHAwVmu20BCXUc+S/p2ifVcS hDNIZ9kIYFymqgzX7ZGj7anK0q2htB35cFjDklwNPdnUMyEXvYzcNB3/Cqq0fylmbXuAdw myeOSUjViAgPhsKRfNbDD6a9VYXMRu40h0mkdEpNg/SLqCB7jaJgMWJh9DTZ0QuAZ4jE+2 uogKq5EiD3EcBNuvw5I25x4JkA/ytQdLTPs0MUUDmDtbOGPvifgvLUugCTD2Y2AILMFMdU BpgJe+1MQNfZzM642IGW6BV/lHsi95WunDpPxUnkzvMweYTUUw9H/EFNRvlxTA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1622862106; a=rsa-sha256; cv=none; b=C5vDTyjMiBku9StxQX4yXLVWO4aDyLGnSP559T784bvy5gnA5COpDsjPyBF+sKAXE+JONv GZm3LGwhyoM68KDwgul6ThCjfC9LirlknkFT3T2iU1e28qAhwa62wcMqrRLlFCytaKs1ES 2aeA7lllANrzvBN5KETMMTOuZBhTNtzRyStUq7SR6ntKB5lA1F8D3JaJKfgfWkzfxwwiJx cmRGxSJU1Q71dP+4IskHFMfJjNzf0MIO7qu5POTERR+F+AceTamn+sbCdw6XGO/Vf7MNGd 4h+OqPDXOWX6wafpYp0FJlCya3c8D3q2il+lRnG+VtwTpicfZ2GVXkjdYx2ewQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -1.01 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 7790320C16 X-Spam-Score: -1.01 X-Migadu-Scanner: scn0.migadu.com X-TUID: eJD9BzFcCV4t On Fri, Jun 04, 2021 at 08:40:56PM -0500, Felipe Contreras wrote: > On Fri, Jun 4, 2021 at 8:37 PM David Bremner wrote: > > Felipe Contreras writes: > > > > I can't use notmuch anymore, I get this error: > > > > > > A Xapian exception occurred opening database: The revision being read > > > has been discarded - you should call Xapian::Database::reopen() and > > > retry the operation > > > > > > Context. In order to investigate a bug about mbsync I moved away the > > > folder ~/mail/.notmuch. I have a timer that calls notmuch new after > > > mbsync, so I paused that timer. > > > > > > Initially I used notmuch, only to see everything empty. Then I > > > recalled what I did, removed all the files, and moved back the .nomuch > > > directory. Perhaps a process had the database or the empty replacement open for writing over the moving aside or the moving back? That could result in a broken database. > `xapian-check ~/mail/.notmuch/xapian F` doesn't seem to change anything. With some filing systems and older format (chert) Xapian databases a system crash or power failure could result in truncating to zero size the files which tracked which blocks were in use and where the root of a particular revision of the tree; the xapian-check's "fix" mode was added to recreate those files by scanning the whole database to work out what they should contain. In newer format databases (glass) we eliminated these files and currently the "fix" mode doesn't actually do anything for glass. The plan was to teach xapian-check how to recreate the `iamglass` file, but that doesn't seem to suffer from the truncation problem and so it hasn't actually been implemented yet and so "F" currently does nothing for glass databases. > > > IIRC I was able to use notmuch without problems once, and then I got the issue. > > > > Maybe the Xapian folk will have a more concrete suggestion, but I would > > start by running xapian-check on the database. In your case I guess that > > should be "xapian-check ~/mail/.notmuch". I'd suggest trying this simple tool I wrote that can probably rescue the tags from a broken notmuch database (the tags are the part notmuch can't just recreate by reindexing): https://git.xapian.org/?p=xapian;a=blob;f=README.notmuch;hb=refs/heads/notmuch-tag-rescue-hack Once you have those, you can reindex your mail and then restore the tags. Cheers, Olly