From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id kA8lGBbzxmLKYgEAbAwnHQ (envelope-from ) for ; Thu, 07 Jul 2022 16:52:06 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id iJ5EFxbzxmJHgQAAG6o9tA (envelope-from ) for ; Thu, 07 Jul 2022 16:52:06 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [135.181.149.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CC9F72B720 for ; Thu, 7 Jul 2022 16:52:05 +0200 (CEST) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 271895E545; Thu, 7 Jul 2022 14:52:03 +0000 (UTC) Received: from meesny.iki.fi (meesny.iki.fi [195.140.195.201]) by mail.notmuchmail.org (Postfix) with ESMTPS id DBDB05DD92 for ; Thu, 7 Jul 2022 14:52:00 +0000 (UTC) Received: from c53 (gw1.nor.fi [185.218.193.67]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: too) by meesny.iki.fi (Postfix) with ESMTPSA id 9BEEF200B9; Thu, 7 Jul 2022 17:51:59 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=meesny; t=1657205519; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=A/gB0kCuXan3QoUdwwTkwW/9NKF+dNfrfyASHu35ajQ=; b=JWt2DGTWH4NtyqPsh2hHwuB66mwGwbU87nsAKuEqFp3+Dd21S/cEp5K7+grLx3zf3ncFqU KafCC8DJvr0VGZUbhNn35ycR9vvWr4IqqdAFVB2ea/T4H266SCpyRn7NrkpAifoxDVTi1s u7V5Vs9/DeyN4xxrlRZYSlKefXMrxFw= From: Tomi Ollila To: David Bremner , notmuch@notmuchmail.org Subject: Re: [PATCH 4/4] CLI/git: replace calls to notmuch-search with database access In-Reply-To: <20220703151103.1800726-5-david@tethera.net> References: <20220703151103.1800726-1-david@tethera.net> <20220703151103.1800726-5-david@tethera.net> User-Agent: Notmuch/0.36+55~gfb9633d (https://notmuchmail.org) Emacs/27.1 X-Face: HhBM'cA~ MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=meesny; t=1657205519; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=A/gB0kCuXan3QoUdwwTkwW/9NKF+dNfrfyASHu35ajQ=; b=H/fRnRJ1L2oLI+3N5H6OA2LUHenckfww5jpDy+r9C/59VZJwluMuYyS34tbi7IeN059daQ Mhl2aZYpd8L5hesueiRUSIGUjJEpFZRiQdNzxmJV2GuNACkMMUczjxC4JmzNmLACoJqP6n O5NLilD3VeU1BCDWT204MzWYvbltBNg= ARC-Authentication-Results: i=1; ORIGINATING; auth=pass smtp.auth=too smtp.mailfrom=tomi.ollila@iki.fi ARC-Seal: i=1; s=meesny; d=iki.fi; t=1657205519; a=rsa-sha256; cv=none; b=QN+DBrVuXwmmIdP9wB7p412q6//nwOuc7NM8xB2pR0wTa52kZE6cES8wnJfxv/5+pahMJl IvJ6ZVQnPSEj2JSsU7si+5qwVBZ6QmnPqhK6pLWDHTR5e+CpEICf0eloDLACTkeYhLqZfi cdYYsIQ9f5XyI7mdnGzKdwl0W/R2vno= Message-ID-Hash: HBVV7HVUWVYLS2KQCSWBEUSQZXEX2FNZ X-Message-ID-Hash: HBVV7HVUWVYLS2KQCSWBEUSQZXEX2FNZ X-MailFrom: tomi.ollila@iki.fi X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: DE ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1657205526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=R+fJLFzsqVcVmLZ48njsZcGg/7UWFES15PnQK9ztpUs=; b=jI9PUOwPNEiJCnJjpOJo//xvkls4YadnToStJ4ULz2vK0HaxOH25qmxX4zlHEdI+AuKcQ2 t9J/UnmRtYNu06IYV4iPbhbuL42Ba4/nZpaY7nECSTnrKGPxZKDGDUdIfb/dhqNFh4V+b9 i8DJd3zHiVT3fdCQGb8zJ39Oxl410ydJyzvKhKYq+fsL2YDmmjgUzSedDH1uLNQ/PHABjZ O9d/jLCvD0Pxcd1ghZ1ZcHUiFnObf/+wa3QsDSVxND9BafOZ94v2n2+HqQbOEQr+guEywJ nqX97qzoq3TI6L/mm7rJBCPVA4ceqxzk9B5wEK5I6rIouOlbj3pMmodgFDUpxA== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1657205526; a=rsa-sha256; cv=fail; b=X/MB6uQvLjxpkNDMmo5uMxh9mm3JixJbOgeSnbL4tTTMoWmxfcKitqd4WpmOnAeRhU0nxN LsQKY1kKfEwK3Eg+DSa3hXefe6dUTEKsgVodDPLO1r+kYsDFY210PCfRLKtqeAoqywu/54 cqLI/Z5V20LBORRSppweQEmpBHt5VgbsGR7l3jgbOsXpZodLjuARozB7CnweKEJtIzWGFO 2UdcgG4zGTTvo0d7pbQBBQ08fHQI2HBlrsQleWwdzGrwnCvuQp/l3Ob0bKva1g+Rf22ptw gGhlPEO76FGtnezmR1M51+81YTvLgjyJCWkaUrXO5fXGmWUXBLfyBkwt0QM+mA== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=iki.fi header.s=meesny header.b=JWt2DGTW; arc=reject ("signature check failed: fail, {[1] = sig:iki.fi:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 2.84 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=iki.fi header.s=meesny header.b=JWt2DGTW; arc=reject ("signature check failed: fail, {[1] = sig:iki.fi:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: CC9F72B720 X-Spam-Score: 2.84 X-Migadu-Scanner: scn0.migadu.com X-TUID: YHvx4XQI+Byn On Sun, Jul 03 2022, David Bremner wrote: > This introduces a dependency on the (new) python bindings, but since > it also yields a 4x performance improvement on the large performance > corpus, I think it is worth it. > --- > debian/control | 1 + > notmuch-git.py | 18 +++++++++--------- > test/T850-git.sh | 2 ++ > 3 files changed, 12 insertions(+), 9 deletions(-) > > diff --git a/debian/control b/debian/control > index 0ffe958c..7099fe97 100644 > --- a/debian/control > +++ b/debian/control > @@ -73,6 +73,7 @@ Depends: > git, > notmuch, > python3, > + python3-notmuch2 > ${misc:Depends} > Description: thread-based email index, search and tagging > Notmuch is a system for indexing, searching, reading, and tagging > diff --git a/notmuch-git.py b/notmuch-git.py > index a3ae15f7..eac24a46 100644 > --- a/notmuch-git.py > +++ b/notmuch-git.py > @@ -701,21 +701,21 @@ def _is_unmerged(ref='@{upstream}'): > > @timed > def get_status(): > + from notmuch2 import Database > + > status = { > 'deleted': {}, > 'missing': {}, > } > with PrivateIndex(repo=NOTMUCH_GIT_DIR, prefix=TAG_PREFIX) as index: > maybe_deleted = index.diff(filter='D') > - for id, tags in maybe_deleted.items(): > - (_, stdout, stderr) = _spawn( > - args=['notmuch', 'search', '--output=files', 'id:{0}'.format(id)], > - stdout=_subprocess.PIPE, > - wait=True) > - if stdout: > - status['deleted'][id] = tags > - else: > - status['missing'][id] = tags > + with Database() as notmuch: > + for id, tags in maybe_deleted.items(): > + try: > + _ = notmuch.find(id) One extra space above. For me looking further here stalled to the introduced python bindings dependency -- it make is much harder to make working notmuch-git installation... ... currently it is enough that notmuch(1) binary is in PATH -- I personally just build new versions of notmuch and copy it to $HOME/bin and it just works(tm). Same with nmbug. To get python bindings work one has to be able to build the c module, and then copy the set of python files (or make zip archive) to a directory (along w/ that c module -- where did that get copied cannot remember now...:) I've trying to think if there were a way to somehow run only one notmuch command instead of notmuch search on all maeby-deleted files -- or alternatively attempt to load python bindings and in case of failure use the notmuch-search methid... Tomi > + status['deleted'][id] = tags > + except LookupError: > + status['missing'][id] = tags > status['added'] = index.diff(filter='A') > > return status > diff --git a/test/T850-git.sh b/test/T850-git.sh > index 81400328..b1bc9e7e 100755 > --- a/test/T850-git.sh > +++ b/test/T850-git.sh > @@ -7,6 +7,8 @@ if [ $NOTMUCH_HAVE_SFSEXP -ne 1 ]; then > test_done > fi > > +export PYTHONPATH="$NOTMUCH_BUILDDIR/bindings/python-cffi/build/stage:${PYTHONPATH:+:$PYTHONPATH}" > + > # be very careful using backup_database / restore_database in this > # file, as they fool the cache invalidation checks in notmuch-git. > > -- > 2.35.2 > > _______________________________________________ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-leave@notmuchmail.org