From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id oEnQNsCevGA7GAAAgWs5BA (envelope-from ) for ; Sun, 06 Jun 2021 12:09:04 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id iD4vMsCevGCobgAAB5/wlQ (envelope-from ) for ; Sun, 06 Jun 2021 10:09:04 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8FEC437224 for ; Sun, 6 Jun 2021 12:09:04 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 8763127185; Sun, 6 Jun 2021 06:08:56 -0400 (EDT) Received: from thyestes.tartarus.org (thyestes.tartarus.org [5.196.91.86]) by mail.notmuchmail.org (Postfix) with ESMTPS id 5C7131FFD8 for ; Sun, 6 Jun 2021 06:08:54 -0400 (EDT) Received: from olly by thyestes.tartarus.org with local (Exim 4.92) (envelope-from ) id 1lppi7-0002Px-5G; Sun, 06 Jun 2021 11:08:51 +0100 Date: Sun, 6 Jun 2021 11:08:51 +0100 From: Olly Betts To: Felipe Contreras Subject: Re: How to recover from this permanent fatal error? Message-ID: <20210606100851.GA18586@survex.com> Mail-Followup-To: Felipe Contreras , Xapian Discussion , David Bremner , "notmuch@notmuchmail.org" References: <8735txjdmc.fsf@tethera.net> <20210605024316.GC9932@survex.com> <20210606014517.GD9932@survex.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Message-ID-Hash: BTXUBK65EIPOV3ZNQ4J7D3QF5M3YMHUD X-Message-ID-Hash: BTXUBK65EIPOV3ZNQ4J7D3QF5M3YMHUD X-MailFrom: olly@survex.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Xapian Discussion , "notmuch@notmuchmail.org" X-Mailman-Version: 3.2.1 Precedence: list Reply-To: Xapian Discussion List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1622974144; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Q3Dia7Ed+TAcu7K5SFp3uAZY3e5SBQUtp7jQNz13L4Q=; b=C8AFpD6plIr25Jt1lwyb1Zc/7K6RqEe+mIRztp0JkWjHXM0geb9U1iZQbLRoHMWW5Udfqn pVOyWwXAcYwOEqQX3VyDFUSII3EqeVT+YKplgM+o2E94msrD8WQ8dHDPKAtvfn2p2Wrles XxSQw45Bn6iAtEWJE2xaUWCWVxJrBVOtHAKOBNYau0ModpVsZjw7/Vxrnxkn1JcFkL+GWe dYPSMvNKxzJ3XWeNbxV8bB0im/zQw6BdRKvj5Ip0pb1HzNdMWI9V5KDKb3QYKQ5fvWY7u2 tc7/VE/PiXmLNVAS20I8ox9CKHbvcmlz0H9nsBoXGrsXyTNUcNJz951TUbvvZA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1622974144; a=rsa-sha256; cv=none; b=QZlabqLozvSfiRsJMae9GMPJCLSjBf5OPA4HGdrJHD62uoB0pSXOsyn16GwUR3mXFCgtJo YZ4NPCjXq+kWNQU7os3OkQHqOhQYyH89Z9Z+EHmXdjKQhJFONduu8IMDbhaZtBlbij4iVQ 2gp35la6qErdp7kzawTbjirJrJWGkDJYQymt/TNsVG1b20keiWBI6E46NZpQNUGa5ItKMR 13YdzqgIjyGWHL5t8elREjzYnnqrffKrG1KNCjVdU4aEB7NS3w9R78JZEZAlVDSTKS6NaU 8+78AiDZPw7kZMNTpKzEygw6DnL+Ddf0+PaeWYqQAl/bOG6dIUuWvRaCPsqx4g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -1.03 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 8FEC437224 X-Spam-Score: -1.03 X-Migadu-Scanner: scn0.migadu.com X-TUID: p09N4EpbHuhs On Sat, Jun 05, 2021 at 11:40:58PM -0500, Felipe Contreras wrote: > % xapian-core/bin/xapian-check ~/mail/.notmuch/xapian/termlist.glass > termlist: > xapian-core/bin/.libs/lt-xapian-check: DatabaseCorruptError: Db block > overwritten - are there multiple writers? Ah - this tool currently requires the termlist table to be undamaged enough to at least scan through. You could try commenting out the body of GlassTable::set_overwritten() in xapian-core/backends/glass/glass_table.cc so it keeps going instead of throwing this exception, which might allow it to usefully recover some or all tags. If you (or anyone) try that and it works let me know and I can patch the branch to emit a warning message and continue there. If the postlist table is readable it'd be possible to rescue the tag data from there instead, but that's more complicated to do because the tags would need collating for each message. Cheers, Olly