From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id cEV0HAt90WIHCwEAbAwnHQ (envelope-from ) for ; Fri, 15 Jul 2022 16:43:23 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id mO8lGwt90WL8UwAAG6o9tA (envelope-from ) for ; Fri, 15 Jul 2022 16:43:23 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [135.181.149.255]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BD41679D5 for ; Fri, 15 Jul 2022 16:43:22 +0200 (CEST) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 0559760818; Fri, 15 Jul 2022 14:43:20 +0000 (UTC) Received: from fethera.tethera.net (fethera.tethera.net [IPv6:2607:5300:60:c5::1]) by mail.notmuchmail.org (Postfix) with ESMTP id 117995E01E for ; Fri, 15 Jul 2022 14:43:17 +0000 (UTC) Received: by fethera.tethera.net (Postfix, from userid 1001) id 8432D5FBD0; Fri, 15 Jul 2022 10:43:15 -0400 (EDT) Received: (nullmailer pid 793186 invoked by uid 1000); Fri, 15 Jul 2022 14:43:12 -0000 From: David Bremner To: David Bremner , notmuch@notmuchmail.org Subject: [PATCH v2] CLI/git: opportunistically use bindings to check for known messages Date: Fri, 15 Jul 2022 10:43:01 -0400 Message-Id: <20220715144301.793133-1-david@tethera.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220703151103.1800726-5-david@tethera.net> References: <20220703151103.1800726-5-david@tethera.net> MIME-Version: 1.0 Message-ID-Hash: KNNFX6QMQ35Z35RNE5KMRAORXUVAW2M4 X-Message-ID-Hash: KNNFX6QMQ35Z35RNE5KMRAORXUVAW2M4 X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: DE ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1657896203; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post; bh=YoBLytwvXISjFlv/hYO3pz5R7Mhk/F4sUf6U/0C3Ap0=; b=Nz1huhTbBq3kROQGzzM5txjjsi8s9DDbdw3yKMw3NKw9HO5iEQWgsz7anFv04LCY0QgkSH vwvBgJ4kHkWiLyMhbZWJnS6hQOrPBdnNxVGfHXJlHfgfpxUUpnAC+CDA0eCdpVn0bjJfUw rK/+BwTE27slg1UGIPJ82i3OkxI4b6Etxf3rl8xfKUmLxEcNhhyH2vreP6NbSiFLzN9tMt FS6MLzzK2qSwScAhamsyWdBeJkfMV9KilaQCGfolHOLxKhgdpw0xE83tKTX3O28Lcz5IhV 8nDUB/9cUzaGhFjdrNHFX6m1dfZIOY+arFvDmVIXWOZbWdwLiQ8YALBnaIdWiA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1657896203; a=rsa-sha256; cv=none; b=RMOL9+wu+1zMWtXBr/VsYOM5rSpgD4sfqLacBw0bJi/H5edUE47wnXCDNS1hFQ9yO02QAw ulo0bPKyZSjB/HVhHwdadansSZ24r3/8Lv/WApDGDvJ6L/040WjE1QQohd8kBW4U8xBzwk qHjLEA+95SgbfSroUozkB7xaDHMFeeFvMcXUaUUA4Jd+YA66GuFPECA41h2hVsE4e0bc2t uQUe50F5Tbs4jkHg1GOvzriWlMMh0Ars5P2hTmvQxBoKzWZdaYp8vWecH8NgVzRog+kgi+ 3SmncjIfgugKkHlUq4x3k9PCgCsmz2Fg/n5LGbobRf/B35QwddXV/Saq5vhV9w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -2.14 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 135.181.149.255 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: BD41679D5 X-Spam-Score: -2.14 X-Migadu-Scanner: scn0.migadu.com X-TUID: L3xUyVUnsLAF If the bindings are installed, use them to avoid one exec of notmuch search per message. --- notmuch-git.py | 33 ++++++++++++++++++++++++++++----- 1 file changed, 28 insertions(+), 5 deletions(-) I decided to leave the old (slow) path in for now, since it is fast enough for use of nmbug to manage notmuch developement tags. diff --git a/notmuch-git.py b/notmuch-git.py index 4d9887c8..ceb86fbc 100644 --- a/notmuch-git.py +++ b/notmuch-git.py @@ -698,6 +698,32 @@ def _is_unmerged(ref='@{upstream}'): stdout=_subprocess.PIPE, wait=True) return base != fetch_head +class DatabaseCache: + def __init__(self): + try: + from notmuch2 import Database + self._notmuch = Database() + except ImportError: + self._notmuch = None + self._known = {} + + def known(self,id): + if id in self._known: + return self._known[id]; + + if self._notmuch: + try: + _ = self._notmuch.find(id) + self._known[id] = True + except LookupError: + self._known[id] = False + else: + (_, stdout, stderr) = _spawn( + args=['notmuch', 'search', '--output=files', 'id:{0}'.format(id)], + stdout=_subprocess.PIPE, + wait=True) + self._known[id] = stdout != None + return self._known[id] @timed def get_status(): @@ -705,14 +731,11 @@ def get_status(): 'deleted': {}, 'missing': {}, } + db = DatabaseCache() with PrivateIndex(repo=NOTMUCH_GIT_DIR, prefix=TAG_PREFIX) as index: maybe_deleted = index.diff(filter='D') for id, tags in maybe_deleted.items(): - (_, stdout, stderr) = _spawn( - args=['notmuch', 'search', '--output=files', 'id:{0}'.format(id)], - stdout=_subprocess.PIPE, - wait=True) - if stdout: + if db.known(id): status['deleted'][id] = tags else: status['missing'][id] = tags -- 2.35.1