From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id iPvZBnCH1l+vSQAA0tVLHw (envelope-from ) for ; Sun, 13 Dec 2020 21:28:16 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id QBecAnCH1l96JAAAbx9fmQ (envelope-from ) for ; Sun, 13 Dec 2020 21:28:16 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (2048 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8256394038E for ; Sun, 13 Dec 2020 21:28:15 +0000 (UTC) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 0515A29C7A; Sun, 13 Dec 2020 16:28:07 -0500 (EST) X-Greylist: delayed 303 seconds by postgrey-1.36 at nmbug; Sun, 13 Dec 2020 16:28:04 EST Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) by mail.notmuchmail.org (Postfix) with ESMTPS id A1BA12853A for ; Sun, 13 Dec 2020 16:28:04 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1607894883; bh=cBuEvu9Z3xq9B2kL/1+amD+uerhLw0MbPaMmGIPawhw=; h=X-UI-Sender-Class:Date:From:To:Subject:References:In-Reply-To; b=Wvba62v5ghGU2AOvw/O4xyKY9XiHL/spjyq/V7GkKxqiARM3VQ+JDBbtszbaQ8Whr pgXjKHe+bMZKHP9SttJZKLu4KMHz0Jg1A8zO6x0YYjRk1ecR4faekyJz0IHvEW592z 1nDnLk4UWPkU8UroRrT/P+uq0SJt1ws1GPrOKqV8= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from no.speedport.ip ([84.185.97.253]) by mail.gmx.com (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1M4s0j-1knBFm2IFj-001yiR for ; Sun, 13 Dec 2020 22:22:57 +0100 Date: Sun, 13 Dec 2020 22:22:52 +0100 From: Gregor Zattler To: notmuch Subject: Re: consistent database corruption with notmuch new Message-ID: <20201213212252.GH21521@no.workgroup> Mail-Followup-To: notmuch References: <20201213131909.GD21521@no.workgroup> <87zh2hhk15.fsf@tethera.net> <20201213141543.GE21521@no.workgroup> <20201213151336.GF21521@no.workgroup> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20201213151336.GF21521@no.workgroup> X-Provags-ID: V03:K1:77FN+EjpG00a02ZXFataXLKOPplwVI4DSljU8Y1VAWep2p0PAxP QUvNuR4vFOTu7TvNPUtHeyqQKh89K8DXALoKqA9Ffjjq2T7Z62oFKzVAq2RbjGjyWfrl/lV Q8W0SczYGtEksInHNgh6eya7QL6VTnJzqhfSJME7Nk9ipSuecfLT1T/7F7s/RFtqDCKIwA4 EOZijMl8lRsxDtzjWbh0Q== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:OH2W1tphzzY=:0OJwztUlKfZZxmZxCWXpaY w/UYOprIAWhsuULH2YoYr9ggY/xccParfWxLvQ0GTfChkQLao0SQbK3E/XWlghbzW94kGyfuJ WIkn9I3dtyRvLnyilL49CRH91Z+WYWb2ffyJ6HjjU7D0hFfiF0WhKqzTaCeF6Q2zrozfVg1Td 1sY8+xf5K5qbzg1dC7d5VqiSGxkZtC7bUjzq5q66VzoWsa6TmNtAFq4bS+VJZXEa9ZfVq4R0E lIZCCeb8DH0VzOIIw3nOwJwa4igw5dP/e85GHYe6QsOiK6sVgjWdvT0caciZBq3GINY8nyMOL NoqPsrH9uWV70PF5ycJMB4CFPvDGPizNOX+tVcODPQ7cp+bLe0O9z7RKrnG6xlQnYz0F8KEU8 H9NNjb5+tJVoGs3tt3SHla8GsQBgVSdCK3JnFDop+Td3Bz2AcnKfgPhPl6hKIeHuvMNtaa4dD vELuHT6omhVxkZzQAwXrwuKbKA6pO+QUxQDJoqp68kQsMrYjbq9TNb1wJamnm7NZOcd+J2uIk xm9HO7j1RllO7lN5+enziIOopLCPWMRA5TumoslXpEzdVBrOHaXPaes0VrbEmpuZjqGv7y916 ASjG/vARtDhIB6y9z7wXxD04ALbxn2rkft1drAc9Utie6Vww1xB39t7jSEgmEm4bD7cQgvErC Yv5niLUBhwoCysmGVFfFPjdK70irz8m7F4IwYqhbIudccU7CICnCG2yFB9QCujMhPWGmuiu8O FHFgVSGmwW65I/7HYedfww8rvm3BuXV7oTcRfMlknQabojeDzTkPb5K5lU1/1FJOP1Ae3DhWW qX6ieLWFHKpWII2YZMI1/c9QTmjbN9oAw99IR7oUaPT2ixIm+RzICK1zb/Gr9PvGgV+G4AGqM jyxjnJ5wCPvxZ9Mql3AQ== Message-ID-Hash: ICLYHN4ZQ7WXXKXUVAPCJWB7YPULI574 X-Message-ID-Hash: ICLYHN4ZQ7WXXKXUVAPCJWB7YPULI574 X-MailFrom: telegraph@gmx.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: 0.72 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmx.net header.s=badeba3b8450 header.b=Wvba62v5; dmarc=fail reason="No valid SPF" header.from=gmx.net (policy=none); spf=temperror (aspmx1.migadu.com: error in processing during lookup of notmuch-bounces@notmuchmail.org: DNS error) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 8256394038E X-Spam-Score: 0.72 X-Migadu-Scanner: scn1.migadu.com X-TUID: S/UQ54FDVsbL Hi David, notmuch developers, * Gregor Zattler [13. Dez. 2020]: > I now got the bullseye version of xapian-core source > package, build it on my buster machine (all 4 tests > passed, expected tests failed). > > xapian-check F version 1.4.17-1 did not succed in fixing > the last corrupted database > > I now do a notmuch new with libxapian30 version 1.4.17-1 > and will report back in a few hours. The result is only slightly different from version 1.4.11: 0 (master *) grfz@no:~/Mail/.notmuch$ nice ionice -c3 notmuch new ; nice ionice -c3 notmuch new ; nice ionice -c3 notmuch new Welcome to a new version of notmuch! Your database will now be upgraded. This process is safe to interrupt. Backing up tags to /home/grfz/Mail/.notmuch/dump-20201213T144839.gz... Your notmuch database has now been upgraded. Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox Processed 1175849 total files in 3h 1m 40s (107 files/sec.). Added 1102076 new messages to the database. Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox Processed 143329 total files in 1h 21m 23s (29 files/sec.). Added 83756 new messages to the database. Removed 18389 messages. Detected 96 file renames. Processed 124614 total files in 39m 23s (52 files/sec.). Added 83712 new messages to the database. 0 (master *) grfz@no:~/Mail/.notmuch$ nice ionice -c3 notmuch new ; nice ionice -c3 notmuch new ; nice ionice -c3 notmuch new add_file: A Xapian exception occurred A Xapian exception occurred at lib/message.cc:1182: No termlist for document 1114342 Processed 697 total files in 14s (46 files/sec.). Added 299 new messages to the database. Note: A fatal error was encountered: A Xapian exception occurred add_file: A Xapian exception occurred A Xapian exception occurred at lib/message.cc:1182: No termlist for document 1114342 Processed 697 total files in 2s (250 files/sec.). Added 299 new messages to the database. Note: A fatal error was encountered: A Xapian exception occurred add_file: A Xapian exception occurred A Xapian exception occurred at lib/message.cc:1182: No termlist for document 1114342 Processed 697 total files in 2s (252 files/sec.). Added 299 new messages to the database. Note: A fatal error was encountered: A Xapian exception occurred 1 (master *) grfz@no:~/Mail/.notmuch$ This time email werde delivered while notmuch new was running but there were certainly not 83756 new messages or 18389 messages removed ones. Since I now for the frist time in years read email in mutt emails were moved from new/ to cur/ and renamed at the same time, but there were neither such many new ones nore were thousands removed. Next I will test with notmuch from git master, but this will have to wait a day or two. Ciao, Gregor -- -... --- .-. . -.. ..--.. ...-.-