unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* notmuch insert: Cannot open Xapian database (0.34.1)
@ 2021-12-04 18:30 Peter J. Jones
  2021-12-04 19:01 ` David Bremner
  0 siblings, 1 reply; 3+ messages in thread
From: Peter J. Jones @ 2021-12-04 18:30 UTC (permalink / raw)
  To: notmuch

I'm in the process of upgrading from notmuch 0.31.4 to 0.34.1 due to an
OS upgrade (NixOS 21.11).  I have tests that verify my mail
configuration (postfix, dovecot, notmuch, etc.) and they fail with
notmuch 0.34.1.  I've tracked it down to (what I believe to be) two
possible bugs.

First, with 0.31 you could call `notmuch insert` and it would create the
database if it didn't exist.  This appears to no longer work in 0.34:

```
[nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch config get database.path
/home/pjones/src/rc/mailrc/maildir
[nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch insert < <(echo foo)
Error: Cannot open database at /home/pjones/src/rc/mailrc/maildir/.notmuch: No such file or directory.
```

I can see in `notmuch.c` that the insert command doesn't have the
`NOTMUCH_COMMAND_DATABASE_CREATE` bit set, so this is the expected
behavior in 0.34 and was probably a bug in 0.31.  However, I would argue
that it's useful to create the database in this situation.

In my tests I bootstrap a new VM with an account that has no mail yet.
Then mail comes in via Dovecot which is handed off to `notmuch insert`,
which fails.  On my end I can fix the test by running `notmuch new`
before Dovecot calls `notmuch insert`.

The second issue is that if the `.notmuch` directory exists, but the
`.notmuch/xapian` directory doesn't, `notmuch new/insert` will fail:

```
[nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch new
Found 0 total files (that's not much mail).
No new mail.
[nix-shell:~/src/rc/mailrc]$ rm -rf maildir/.notmuch/xapian
[nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch new
Cannot open Xapian database at /home/pjones/src/rc/mailrc/maildir/.notmuch/xapian: Couldn't stat '/home/pjones/src/rc/mailrc/maildir/.notmuch/xapian'
```

The reason I get into this state is because I use Home Manager to manage
my configuration files and it creates `~/mail/.notmuch` so it can
install hook scripts.  Of course, this only happens on a brand new
setup, like in my test VM where Home Manager sets up my configuration
files first, then notmuch is run to test mail delivery and indexing.

This seemed to work in 0.31 (notmuch would just create a new database).

Would it be okay to change `notmuch_database_create_with_config` to
create a new database if `.notmuch` exists but `.notmuch/xapian`
doesn't?

Thanks!


-- 
Peter Jones

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: notmuch insert: Cannot open Xapian database (0.34.1)
  2021-12-04 18:30 notmuch insert: Cannot open Xapian database (0.34.1) Peter J. Jones
@ 2021-12-04 19:01 ` David Bremner
  2021-12-06 17:57   ` Peter J. Jones
  0 siblings, 1 reply; 3+ messages in thread
From: David Bremner @ 2021-12-04 19:01 UTC (permalink / raw)
  To: Peter J. Jones, notmuch

"Peter J. Jones" <pjones@pmade.com> writes:

> I can see in `notmuch.c` that the insert command doesn't have the
> `NOTMUCH_COMMAND_DATABASE_CREATE` bit set, so this is the expected
> behavior in 0.34 and was probably a bug in 0.31.  However, I would argue
> that it's useful to create the database in this situation.

I have no real objection to that happening; I'm not sure if it would
break anything. 

>
> In my tests I bootstrap a new VM with an account that has no mail yet.
> Then mail comes in via Dovecot which is handed off to `notmuch insert`,
> which fails.  On my end I can fix the test by running `notmuch new`
> before Dovecot calls `notmuch insert`.
>
> The second issue is that if the `.notmuch` directory exists, but the
> `.notmuch/xapian` directory doesn't, `notmuch new/insert` will fail:
> files first, then notmuch is run to test mail delivery and indexing.
>
> This seemed to work in 0.31 (notmuch would just create a new database).
>
> Would it be okay to change `notmuch_database_create_with_config` to
> create a new database if `.notmuch` exists but `.notmuch/xapian`
> doesn't?

There is many different options for database location possible in 0.32
that were not possible in 0.31.  It might be a bit trickier to make this
change (without compromising something else) than the previous one.

Perhaps migrating to XDG database locations (see notmuch-config(1) for
details) would be an option for you?

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: notmuch insert: Cannot open Xapian database (0.34.1)
  2021-12-04 19:01 ` David Bremner
@ 2021-12-06 17:57   ` Peter J. Jones
  0 siblings, 0 replies; 3+ messages in thread
From: Peter J. Jones @ 2021-12-06 17:57 UTC (permalink / raw)
  To: David Bremner, notmuch

On Sat, Dec 04 2021, David Bremner wrote:
> "Peter J. Jones" <pjones@pmade.com> writes:
>>
>> In my tests I bootstrap a new VM with an account that has no mail yet.
>> Then mail comes in via Dovecot which is handed off to `notmuch insert`,
>> which fails.  On my end I can fix the test by running `notmuch new`
>> before Dovecot calls `notmuch insert`.
>>
>> The second issue is that if the `.notmuch` directory exists, but the
>> `.notmuch/xapian` directory doesn't, `notmuch new/insert` will fail:
>> files first, then notmuch is run to test mail delivery and indexing.
>>
>> This seemed to work in 0.31 (notmuch would just create a new database).
>>
>> Would it be okay to change `notmuch_database_create_with_config` to
>> create a new database if `.notmuch` exists but `.notmuch/xapian`
>> doesn't?
>
> There is many different options for database location possible in 0.32
> that were not possible in 0.31.  It might be a bit trickier to make this
> change (without compromising something else) than the previous one.
>
> Perhaps migrating to XDG database locations (see notmuch-config(1) for
> details) would be an option for you?

Thanks for the pointer.  I was able to resolve this issue by moving my
hook scripts out of `database.path` thanks to `database.hook_dir`.

-- 
Peter Jones

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-12-06 17:57 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-04 18:30 notmuch insert: Cannot open Xapian database (0.34.1) Peter J. Jones
2021-12-04 19:01 ` David Bremner
2021-12-06 17:57   ` Peter J. Jones

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).