From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id aPwnHsi4q2FBQgEAgWs5BA (envelope-from ) for ; Sat, 04 Dec 2021 19:51:52 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id uH3OGci4q2EoIAAAB5/wlQ (envelope-from ) for ; Sat, 04 Dec 2021 18:51:52 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 409BA2396 for ; Sat, 4 Dec 2021 19:51:52 +0100 (CET) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id DCFDE290DE; Sat, 4 Dec 2021 13:51:46 -0500 (EST) X-Greylist: delayed 433 seconds by postgrey-1.36 at nmbug; Sat, 04 Dec 2021 13:38:00 EST Received: from pmade.com (mail.pmade.com [45.33.113.117]) by mail.notmuchmail.org (Postfix) with ESMTPS id AA93629080 for ; Sat, 4 Dec 2021 13:38:00 -0500 (EST) Received: from medusa (unknown [IPv6:2600:8800:1701:e52a::1f7c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) (Authenticated sender: pjones@pmade.com) by pmade.com (Postfix) with ESMTPSA id C9D60120677 for ; Sat, 4 Dec 2021 18:30:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pmade.com; s=20150827; t=1638642646; bh=OHo7qULfue9t5eqKP54CivID60fuFi5OHsHTXOYCkrI=; h=From:To:Subject:Date:From; b=QVGaLzAvHzLw9AatKozI7e8vO2iKincB4QXXgVuoP/g/qwLrWR6En6gIh+S3Nmueg ivxdbWWE6+PPDnP7NHGD0nPhfjI1TBlm3rW2Dp9fySfOBOf7NusItzTSMrK0upIApt IyX49Njar9Um8YKcEy4etTmAWP15LT0Q74NT3sXA= From: "Peter J. Jones" To: notmuch@notmuchmail.org Subject: notmuch insert: Cannot open Xapian database (0.34.1) User-Agent: Notmuch/0.31.4 (https://notmuchmail.org) Emacs/27.2 (x86_64-pc-linux-gnu) Date: Sat, 04 Dec 2021 11:30:39 -0700 Message-ID: <87mtlg8bow.fsf@devalot.com> MIME-Version: 1.0 X-Rspamd-Server: moriarty X-Spamd-Result: default: False [-0.10 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:22773, ipnet:2600:8800::/32, country:US] X-Rspamd-Queue-Id: C9D60120677 X-MailFrom: pjones@pmade.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0 Message-ID-Hash: EMEI6TCVJSQM537VVWJJWOB6HBITNXCP X-Message-ID-Hash: EMEI6TCVJSQM537VVWJJWOB6HBITNXCP X-Mailman-Approved-At: Sat, 04 Dec 2021 13:51:44 -0500 X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-Country: FR ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1638643912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=21ByuqkzLWNHbic89rO9ikO5QiDag+er7/Vh5qgNI1k=; b=kTtIJPJfdDGXEzLWII1baSJfJJVHW8NJMDVHoGnKPRy1ydZ6yzrNBMeuU+PLW8qc4lsp0d DGfwIBMZs1PE6Xe7n61yvx1Gk2/ZM0OIMR1CdZFpNpUVPJTvuMTC/KS2XaEBPd4/QG+to/ Vezcvso8eJyj6+NdkohCiwE6CHiEomcY3DqDuIbfvI3nQ/S1o4mZqa2b/kDey07KbNxpNy ym+o65UPjaqdixa59O+x9oz7Zpmqnn7qJJ2BjjcJtDEjGAzda1wH8zeyEssxL5R+AHwNbi KvY1UPuyMbxRfBdnmwNX7Dfi++5MC2PaOVu5ZFVSkoRapCtXNooU2dOmVcdXew== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1638643912; a=rsa-sha256; cv=none; b=pDpQDe4MCevCIIj7K1vCq+Q//96S+zWTWTGJUxUrVVnFnuJgTGU7qvdAtYOK26jTzA2Dec MzMJzoXh3sqZgwqUZH8nDgaNmfjSB45Rbks+xPYBdB0JkpKplRU/HCEQFsZpDBAwQZZLox 8jJiLjO8rT+KtP2ovi3jRe0pV1XnIW4Ri5jaa9fwR2CUmvJgnkDhaGXrG4MBjUgpqAuO1r hkgIXWhMOp5YdDk3RzAu/KaZHoYiREHeSsUAv7lqcKr3y+tZC5NHfbbVDqXYO3aVhoXq7b +5v0U8uqmXcoztuFpTDtBg+4f7Lvi03cKXgCjim3LIAgJkF2o5cP4pWLl6xzuA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=pmade.com header.s=20150827 header.b=QVGaLzAv; dmarc=fail reason="SPF not aligned (relaxed)" header.from=pmade.com (policy=quarantine); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 6.00 X-Spam: Yes Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=pmade.com header.s=20150827 header.b=QVGaLzAv; dmarc=fail reason="SPF not aligned (relaxed)" header.from=pmade.com (policy=quarantine); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 409BA2396 X-Spam-Score: 6.00 X-Migadu-Spam: Yes X-Migadu-Scanner: scn0.migadu.com X-TUID: dyH+LUM7k4++ I'm in the process of upgrading from notmuch 0.31.4 to 0.34.1 due to an OS upgrade (NixOS 21.11). I have tests that verify my mail configuration (postfix, dovecot, notmuch, etc.) and they fail with notmuch 0.34.1. I've tracked it down to (what I believe to be) two possible bugs. First, with 0.31 you could call `notmuch insert` and it would create the database if it didn't exist. This appears to no longer work in 0.34: ``` [nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch config get database.path /home/pjones/src/rc/mailrc/maildir [nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch insert < <(echo foo) Error: Cannot open database at /home/pjones/src/rc/mailrc/maildir/.notmuch: No such file or directory. ``` I can see in `notmuch.c` that the insert command doesn't have the `NOTMUCH_COMMAND_DATABASE_CREATE` bit set, so this is the expected behavior in 0.34 and was probably a bug in 0.31. However, I would argue that it's useful to create the database in this situation. In my tests I bootstrap a new VM with an account that has no mail yet. Then mail comes in via Dovecot which is handed off to `notmuch insert`, which fails. On my end I can fix the test by running `notmuch new` before Dovecot calls `notmuch insert`. The second issue is that if the `.notmuch` directory exists, but the `.notmuch/xapian` directory doesn't, `notmuch new/insert` will fail: ``` [nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch new Found 0 total files (that's not much mail). No new mail. [nix-shell:~/src/rc/mailrc]$ rm -rf maildir/.notmuch/xapian [nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch new Cannot open Xapian database at /home/pjones/src/rc/mailrc/maildir/.notmuch/xapian: Couldn't stat '/home/pjones/src/rc/mailrc/maildir/.notmuch/xapian' ``` The reason I get into this state is because I use Home Manager to manage my configuration files and it creates `~/mail/.notmuch` so it can install hook scripts. Of course, this only happens on a brand new setup, like in my test VM where Home Manager sets up my configuration files first, then notmuch is run to test mail delivery and indexing. This seemed to work in 0.31 (notmuch would just create a new database). Would it be okay to change `notmuch_database_create_with_config` to create a new database if `.notmuch` exists but `.notmuch/xapian` doesn't? Thanks! -- Peter Jones