unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
From: Matthew Lear <matt@bubblegen.co.uk>
To: notmuch@notmuchmail.org
Subject: Database locked when running hooks
Date: Thu, 18 Mar 2021 08:32:53 +0000	[thread overview]
Message-ID: <9C1993DF-84BD-4199-A9C8-BADA98498812@bubblegen.co.uk> (raw)


[-- Attachment #1.1: Type: text/plain, Size: 3495 bytes --]

Hello all,
I recently updated to notmuch 0.31.4+137~g6967dcb and made some changes to the way I sync my mail using lieer.
I wanted to utilise the pre-new and post-new notmuch hooks.
However, when running an operation to import mail in the context of the notmuch pre-new hook (by running gmi pull), I can see that the database is locked, which means that running notmuch new hangs.
From man (5) notmuch-hooks the wording is such that it implies that the database is not locked when the pre-new and post-new hooks are run.

If I strace notmuch new, I can see that the database is opened for writing before the pre-new hook is run, and it is not closed before the hook is run.
This means that any operation which needs write access to database (e.g. lieer’s gmi pull) cannot obtain the database lock, so it hangs.

The problem can be reproduced by creating $DATABASEDIR/.notmuch/hooks/pre-new and having it perform a notmuch CLI command to write to the DB, such as apply a tag.

I described the problem in https://github.com/gauteh/lieer/issues/195 <https://github.com/gauteh/lieer/issues/195>

Assuming the database should be able to be opened for writing in the context of the pre-new (and post-new) hooks, this seems like a bug.
Here is some output...

Running notmuch new before the pre-new hook is executed:

..
openat(AT_FDCWD, "/home/myhome/path/to/maildir//.notmuch/xapian/flintlock", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
fcntl(3, F_OFD_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) = 0
openat(AT_FDCWD, "/home/ml943601/b/mail/brcm//.notmuch/xapian/iamglass", O_RDONLY) = 4
..

It’s never closed before the hook is executed.
What the hang looks like:

 [~]
$ notmuch new
path: /home/myhome/path/to/maildir/gmaildir/
pull: partial synchronization.. (hid: 66446860)
fetching changes ....fetching changes ....fetching changes ....fetching changes ...resolving changes (376) ........................................receiving content (156) ...^C
Traceback (most recent call last):
  File "/home/myhome/git/lieer/gmi", line 24, in <module>
    g.main ()
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 215, in main
    args.func (args)
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 418, in pull
    self.partial_pull ()
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 548, in partial_pull
    updated     = self.get_content (message_gids)
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 754, in get_content
    self.remote.get_messages (need_content, _got_msgs, 'raw')
  File "/home/myhome/git/lieer/lieer/remote.py", line 133, in func_wrap
    return func (self, *args, **kwargs)
  File "/home/myhome/git/lieer/lieer/remote.py", line 384, in get_messages
    cb (msg_batch)
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 749, in _got_msgs
    with notmuch.Database (mode = notmuch.Database.MODE.READ_WRITE) as db:
  File "/usr/local/lib/python3.8/dist-packages/notmuch/database.py", line 164, in __init__
    self.open(path, mode)
  File "/usr/local/lib/python3.8/dist-packages/notmuch/database.py", line 223, in open
    status = Database._open(_str(path), mode, byref(db))

There is no other program running at the time of the hang which accesses the database.
The problem seems to be caused by the database being opened for writing before the hook is executed.

Does notmuch require the database to be open when calling the hook?

Cheers,
—  Matt


[-- Attachment #1.2: Type: text/html, Size: 5177 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]



             reply	other threads:[~2021-03-18  8:40 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-18  8:32 Matthew Lear [this message]
2021-03-18 10:55 ` Database locked when running hooks David Bremner
2021-03-19  2:07 ` [PATCH 1/2] test: Add tests for write access to database from hooks David Bremner
2021-03-19  2:07   ` [PATCH 2/2] CLI/new: drop the write lock to run the pre-new hook David Bremner
2021-03-24 11:03     ` David Bremner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://notmuchmail.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9C1993DF-84BD-4199-A9C8-BADA98498812@bubblegen.co.uk \
    --to=matt@bubblegen.co.uk \
    --cc=notmuch@notmuchmail.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).