Re: [PATCH 4/4] CLI/git: replace calls to notmuch-search with database access

[Tomi Ollila <tomi.ollila@iki.fi>]

On Sun, Jul 03 2022, David Bremner wrote:

> This introduces a dependency on the (new) python bindings, but since
> it also yields a 4x performance improvement on the large performance
> corpus, I think it is worth it.
> ---
>  debian/control   |  1 +
>  notmuch-git.py   | 18 +++++++++---------
>  test/T850-git.sh |  2 ++
>  3 files changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/debian/control b/debian/control
> index 0ffe958c..7099fe97 100644
> --- a/debian/control
> +++ b/debian/control
> @@ -73,6 +73,7 @@ Depends:
>   git,
>   notmuch,
>   python3,
> + python3-notmuch2
>   ${misc:Depends}
>  Description: thread-based email index, search and tagging
>   Notmuch is a system for indexing, searching, reading, and tagging
> diff --git a/notmuch-git.py b/notmuch-git.py
> index a3ae15f7..eac24a46 100644
> --- a/notmuch-git.py
> +++ b/notmuch-git.py
> @@ -701,21 +701,21 @@ def _is_unmerged(ref='@{upstream}'):
>  
>  @timed
>  def get_status():
> +    from notmuch2 import Database
> +
>      status = {
>          'deleted': {},
>          'missing': {},
>          }
>      with PrivateIndex(repo=NOTMUCH_GIT_DIR, prefix=TAG_PREFIX) as index:
>          maybe_deleted = index.diff(filter='D')
> -        for id, tags in maybe_deleted.items():
> -            (_, stdout, stderr) = _spawn(
> -                args=['notmuch', 'search', '--output=files', 'id:{0}'.format(id)],
> -                stdout=_subprocess.PIPE,
> -                wait=True)
> -            if stdout:
> -                status['deleted'][id] = tags
> -            else:
> -                status['missing'][id] = tags
> +        with Database() as notmuch:
> +            for id, tags in maybe_deleted.items():
> +                try:
> +                    _ =  notmuch.find(id)

One extra space above.

For me looking further here stalled to the introduced python bindings
dependency -- it make is much harder to make working notmuch-git
installation...

... currently it is enough that notmuch(1) binary is in PATH -- I
personally just build new versions of notmuch and copy it to $HOME/bin
and it just works(tm). Same with nmbug. To get python bindings work one has
to be able to build the c module, and then copy the set of python files (or
make zip archive) to a directory (along w/ that c module -- where did that
get copied cannot remember now...:)

I've trying to think if there were a way to somehow run only one notmuch
command instead of notmuch search on all maeby-deleted files -- or
alternatively attempt to load python bindings and in case of failure use
the notmuch-search methid...

Tomi

> +                    status['deleted'][id] = tags
> +                except LookupError:
> +                    status['missing'][id] = tags
>          status['added'] = index.diff(filter='A')
>  
>      return status
> diff --git a/test/T850-git.sh b/test/T850-git.sh
> index 81400328..b1bc9e7e 100755
> --- a/test/T850-git.sh
> +++ b/test/T850-git.sh
> @@ -7,6 +7,8 @@ if [ $NOTMUCH_HAVE_SFSEXP -ne 1 ]; then
>      test_done
>  fi
>  
> +export PYTHONPATH="$NOTMUCH_BUILDDIR/bindings/python-cffi/build/stage:${PYTHONPATH:+:$PYTHONPATH}"
> +
>  # be very careful using backup_database / restore_database in this
>  # file, as they fool the cache invalidation checks in notmuch-git.
>  
> -- 
> 2.35.2
>
> _______________________________________________
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-leave@notmuchmail.org