From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id OOuNJr9to2QYOAAASxT56A (envelope-from ) for ; Tue, 04 Jul 2023 02:54:23 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id ePtRJr9to2RcgAAA9RJhRA (envelope-from ) for ; Tue, 04 Jul 2023 02:54:23 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [IPv6:2a01:4f9:c011:7a79::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 012FC9053 for ; Tue, 4 Jul 2023 02:54:23 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2a01:4f9:c011:7a79::1 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org ARC-Seal: i=1; s=key1; d=yhetil.org; t=1688432063; a=rsa-sha256; cv=none; b=LjNu6qmhwTEupsfvoVQrDUlW1SsjoqiMZfE0BeSsnC50eGxwG6SsKB57pibA/2cC5DxFpw xmSIMleL+sbRPJOJdgSYNAOqReA8lGLNXGMF5N7yMVUkn/xSi0rm4iK8GdC9NFTZWlwIdx 8LONXpIVv1xAvqR4hAlfIwNTbOvIlFAMy9eIwmdIsjutvuMs2dXkT8lNuE6YXNNAsYemLs WQW3tllrA5+jLZpvLvjTYOggOJxoXDzEWx2S2OqVH2G9DLwlSIxvxvoeSDqnTXhlOFGHtX wV4h41jKOehPm0GRBcx4W9+byNwn8RG/B7Llo87pQWvhjyGYEAqpS1BW05Pj5Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2a01:4f9:c011:7a79::1 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1688432063; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post; bh=murRZheGwI0cLAQpop7uHLYsHTN8PAjriABydBgomjo=; b=UmVjNIVMFlq+9tIfqFmdSAA4M48qWwSK1Cehx48Zm4wqEzNZobh0cBcJ6tcfQmt+g9mYxi K5f9mI9B7Ggor5kDb//HldblwBA03C4oLidOOZe9qUzEcLy0ar9tmjTGsUkBVhfJVMfw82 FLz9+dkPIw1uHUoCzLSEEVSDPmCnq/zzOSwmpEzpZFUJZiw+cTe86HUgHAPS2JmoKxnGy9 /UaTr4L/Fs552YLg0O5HlxsWcCL4WfsggBU1R/nTvjtGh8TWDA7AFWZAbJoQGOiRmMLApN WUPvVxmUqJLKh3m0kEXzNenU+7sFLwV1aG0ICgw42eOwXMjjK7YfWROL2rnCBg== Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 5BDD95F420; Tue, 4 Jul 2023 00:54:20 +0000 (UTC) X-Greylist: delayed 2047 seconds by postgrey-1.36 at yantan; Tue, 04 Jul 2023 00:54:17 UTC Received: from thyestes.tartarus.org (thyestes.tartarus.org [5.196.91.86]) by mail.notmuchmail.org (Postfix) with ESMTPS id 8D8D35F3D0 for ; Tue, 4 Jul 2023 00:54:17 +0000 (UTC) Received: from olly by thyestes.tartarus.org with local (Exim 4.92) (envelope-from ) id 1qGTm3-0000b0-D6; Tue, 04 Jul 2023 01:20:07 +0100 Date: Tue, 4 Jul 2023 01:20:07 +0100 From: Olly Betts To: David Bremner Subject: Re: Internal error: Message without type term Message-ID: <20230704002007.GS26907@survex.com> Mail-Followup-To: David Bremner , "Peter P." , notmuch@notmuchmail.org, xapian-discuss@lists.xapian.org References: <20230703074700.dq52buzslq4mr5cr@fastmail.com> <87bkgtjnb9.fsf@tethera.net> <20230703104739.lgoxdghpiy2pqkdg@fastmail.com> <873525ji5g.fsf@tethera.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <873525ji5g.fsf@tethera.net> User-Agent: Mutt/1.10.1 (2018-07-13) Message-ID-Hash: OL6ZG6S5ACBMK756NWETSQPW2UZBUJFP X-Message-ID-Hash: OL6ZG6S5ACBMK756NWETSQPW2UZBUJFP X-MailFrom: olly@survex.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: "Peter P." , notmuch@notmuchmail.org, xapian-discuss@lists.xapian.org X-Mailman-Version: 3.3.3 Precedence: list Reply-To: Xapian Discussion List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Country: DE X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -1.68 X-Migadu-Queue-Id: 012FC9053 X-Spam-Score: -1.68 X-TUID: mW9Kw92wCnOb On Mon, Jul 03, 2023 at 02:26:03PM +0200, David Bremner wrote: > "Peter P." writes: > > > I ran xapian-check on ~/.notmuch/xapian and include its messages > > below at the end of this mail. Everyone please forgive me for > > pasting 1121 there. :) > > H'mm. It doesn't look familiar to me, but I will check with xapian > experts to see if the failure mode is known/fixable. I'd guess probably > not fixable. Currently we don't have a database fixing tool for glass databases (the "fix" mode in xapian-check can recreate base files for the older chert database format, but glass doesn't have these base files which eradicated the failure mode of them sometimes getting truncated to zero size on power failure or OS crash). Some of the problems reported have an obvious fix, but we don't have existing code to fix them, and some look like they are probably due to data being overwritten so fixing everything to be consistent probably wouldn't actually give a database that entirely matches your email anyway. Was this database originally created by Xapian < 1.4.22? It looks like it could be the result of the bug fixed in 1.4.22 with handling commit() failure on disk full. > >> 2) Move the database out of the way, re-run notmuch new, > >> and restore your state using "notmuch restore < notmuch-db.txt" > > > > I'd be fine regenerating the entire database without a backup dump even, > > I don't think there is anything in there that can't be regernerated, > > no? > > The main thing that would be lost is tags that are not synched to > maildir flags. In the "standard" workflow "inbox" is such a tag. If there's tag data in the database which isn't backed up or synced to maildir flags, you may be able to rescue it using: https://git.xapian.org/?p=xapian;a=blob;f=README.notmuch;hb=refs/heads/notmuch-tag-rescue-hack This creates a file with the tag data in the format `notmuch restore` expects. I'd expect this would work for your database as the termlist table is mostly OK. Cheers, Olly