From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 2P3WOaq711+7FQAA0tVLHw (envelope-from ) for ; Mon, 14 Dec 2020 19:23:22 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id sN6aNaq7119CGAAAbx9fmQ (envelope-from ) for ; Mon, 14 Dec 2020 19:23:22 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (2048 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 82AD89402D6 for ; Mon, 14 Dec 2020 19:23:22 +0000 (UTC) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 25DCD29C78; Mon, 14 Dec 2020 14:23:17 -0500 (EST) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) by mail.notmuchmail.org (Postfix) with ESMTPS id EF01A270DE for ; Mon, 14 Dec 2020 14:23:13 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1607973792; bh=S9Gv8LD/J7HHzPuzpO/uUsAGkg/Qhs9oCx4qNNoLkiM=; h=X-UI-Sender-Class:Date:From:To:Subject:References:In-Reply-To; b=dHDXVHoGOc848a2KwLcPv2BdCZKnYPDTg4gA9CJKjMBmd6Mc1CcpD0CkGCe7mAJAR 6J7sfwxxFhnhajXIt1xqRa+2i5OWXimJQNcseUhDRahXAnYrtnLr7PJwKu5W7BLA6Y Yz3IfQzbL2IJPXXsCYaVh/ipie3EiwmbTB5+KbbA= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from no.workgroup ([84.185.97.253]) by mail.gmx.com (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1N2mBa-1k7t8V3iHs-0138sr for ; Mon, 14 Dec 2020 20:23:11 +0100 Date: Mon, 14 Dec 2020 20:22:51 +0100 From: Gregor Zattler To: notmuch Subject: Re: consistent database corruption with notmuch new Message-ID: <20201214192251.GA7858@no.workgroup> Mail-Followup-To: notmuch References: <20201213131909.GD21521@no.workgroup> <87zh2hhk15.fsf@tethera.net> <20201213141543.GE21521@no.workgroup> <20201213151336.GF21521@no.workgroup> <20201213212252.GH21521@no.workgroup> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20201213212252.GH21521@no.workgroup> X-Provags-ID: V03:K1:93MPs34Hts+rNEDrRN4tZHLfRlXMLkcEIGq46Gxx42MvWVl/26S 4CxRTe7eIyQOmkDGS/FMeI4GaL+6UHpFzB5KTKdXVzwkYxzAgzQs17/3Oe+K2fQkqLe35vW Ec4AvVFJPXq768G9sToG77ivPx5AG1r4/03v0PAbQizr0TeEEqmcI625dWYkh0ciVD0hwGq FDCCaB40A4mBve7/Q4ImQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:XjgsuhZy/Vc=:OCXb5QdeqatzVDUBnf1eFI JtddYpcfbsJLtbdyfRLwqlA14pwUs2xxQMOhEPqQAs1qycoaN0BwRxz7gddA41id0ecjH31Yf y954NNpl8Y6RvN+ou9mxLZjUcwIhTrPbxp3VS8I8tkZUBFzZ+12E6at8PQxCAT+8Njz4hmdhb QYZs3KJHI/CLXH33HHsl3qxDIrxORozkmLP9fT+qerel3gwXclGSF6efekSOAIvY/+ttKqPzS ewKUPCVRA9M9aLMxS7MIo+ajyI+Qkre/U5SgMMeVjUeYdSGehkd0hw3ZueGWJ5GBm0uL4yQA6 OyXRDNmLS6QY/l49tsTFrZThXcFRA0DPNFJ41VlA+yRu+bIzdCNcXkeqlUDn1ul9SI+bV7Q++ Ldgj/4JA8JZ4M7m0rZ+bLNenljoLbu2Bsc4zzlDHi/HBVfITXw79/2IwoPEVjyzhFt2UP6qdh xLcT2d64r/MY5QF82t9xVpMlkT+OR+izjnj0MsuqDFCwvKlYol38pgzPx/DTJv08b8RHlaBY3 GOpUYdhvPzd0Zck5lBF8zrCYKT5etyZPMMw1jnjNsC2bznd/mnO+8Zb0Hd+YH0Bdb5O0oXUfb 1PpcBP3FrRDWgaD5GcqwMUtiIWFADkSipuJHF2b9n1akuqxhCJxW7tW3MAkvPlBoRr/FaZn6U yYaPp/4Jay9CXuCyz5t0+8/owtgGpZ7bPCvHJCa+XmPk9k8Xh8tzJdWLQ7gV4GmZsu6byZiKN QzQDxe6eIBSZEcuwdiFvSPpFn+gAKXoZk2E0R4mQBdsA5C8RGLtukTQngUrXY1/NWY9CDT2Jn NrVmiIMsx0G3zpYUQKCma8jq2vJanEJotvzwM1xInfb1iGD6oOSF/EZVTYVAU6mXbC2ecjkaK qmse6gGrmoeJ8OZTZdig== Message-ID-Hash: Y7QMSYZTB6Y33MVJ7INH4X42QWB7BHBM X-Message-ID-Hash: Y7QMSYZTB6Y33MVJ7INH4X42QWB7BHBM X-MailFrom: telegraph@gmx.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -0.98 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmx.net header.s=badeba3b8450 header.b=dHDXVHoG; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmx.net (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 82AD89402D6 X-Spam-Score: -0.98 X-Migadu-Scanner: scn0.migadu.com X-TUID: IivGOvNh65SC Hi notmuch developers, * Gregor Zattler [13. Dez. 2020]: > * Gregor Zattler [13. Dez. 2020]: >> I now do a notmuch new with libxapian30 version 1.4.17-1 >> and will report back in a few hours. > > The result is only slightly different from version 1.4.11: actually now I realized, that notmuch was not linked against libxapian v1.4.17-1. Now I build notmuch from master and linked it with libxapian 1.4.17-1. The result is almost the same, though: 0 (master *) grfz@no:~/Mail/.notmuch$ nice ionice -c3 notmuch new --full-scan ; nice ionice -c3 notmuch new --full-scan ; nice ionice -c3 notmuch new --full-scan Welcome to a new version of notmuch! Your database will now be upgraded. This process is safe to interrupt. Backing up tags to /home/grfz/Mail/.notmuch/dump-20201214T124836.gz... Your notmuch database has now been upgraded. Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox Processed 1176599 total files in 3h 31m 23s (92 files/sec.). Added 1102787 new messages to the database. Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox Processed 125008 total files in 44m 59s (46 files/sec.). Added 83898 new messages to the database. Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2, add_file: A Xapian exception occurred). A Xapian exception occurred at lib/message.cc:1182: Too few chunks of compressed data Processed 122480 total files in 42m 1s (48 files/sec.). Added 83907 new messages to the database. Note: A fatal error was encountered: A Xapian exception occurred 1 (master *) grfz@no:~/Mail/.notmuch$ notmuch new still corrupts the database, the second notmuch new invocation finds emails the first did not find. Ciao, Gregor -- -... --- .-. . -.. ..--.. ...-.-