From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id eAQwIayxwWLbKQEAbAwnHQ (envelope-from ) for ; Sun, 03 Jul 2022 17:11:40 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id UEclIKyxwWLtggEAG6o9tA (envelope-from ) for ; Sun, 03 Jul 2022 17:11:40 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [IPv6:2a01:4f9:c011:7a79::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 464433FC65 for ; Sun, 3 Jul 2022 17:11:40 +0200 (CEST) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 431325F525; Sun, 3 Jul 2022 15:11:20 +0000 (UTC) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by mail.notmuchmail.org (Postfix) with ESMTP id 0F4585E226 for ; Sun, 3 Jul 2022 15:11:17 +0000 (UTC) Received: by fethera.tethera.net (Postfix, from userid 1001) id 373B25FC01; Sun, 3 Jul 2022 11:11:15 -0400 (EDT) Received: (nullmailer pid 1800798 invoked by uid 1000); Sun, 03 Jul 2022 15:11:10 -0000 From: David Bremner To: notmuch@notmuchmail.org Subject: [PATCH 4/4] CLI/git: replace calls to notmuch-search with database access Date: Sun, 3 Jul 2022 12:11:03 -0300 Message-Id: <20220703151103.1800726-5-david@tethera.net> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220703151103.1800726-1-david@tethera.net> References: <20220703151103.1800726-1-david@tethera.net> MIME-Version: 1.0 Message-ID-Hash: EVMEB3BLL7DSPEYW5XB4QDA5I7HLS533 X-Message-ID-Hash: EVMEB3BLL7DSPEYW5XB4QDA5I7HLS533 X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: DE ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1656861100; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post; bh=UhWYlX+3lK6cfV6JBqKEjF2jImSPCflumbIGq4X87ws=; b=RwsYZE+hYlCTa7SIr/suCIAAAlrGWfybFhXdLuAEbB34u2W7LFUEtDLRNE4TZnAH0ON3we NhTE3j1sE3FldHJq9IQlGX4HQm35i0q0F08tJi2xbkfYxhfCOYyCiEaBaUvXxlhp1Avnlj J6tKwDJOpJtzkMggN0tfbs2TsRYMrJ4+C5chii72x62/ZTayzA9ceqgcu1tFcQABIRn2zz CvgTEbckp7MNEAeHz9NAwsCV2wVD7U6QHtYc91sazH9+p38s6uEQllA6hpXLo9FBfx17Rz yCXHXwBMHWCLMtmhVUWnFvAr+NoaSg57yHwVGehGsyAvO4FoL34ucxUyzaFUYQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1656861100; a=rsa-sha256; cv=none; b=arW/znfjuiJFHf4WeGVoW39MQoeTKTNjcET7xZg2N6QSbOV0l0V3iVW93i5HCaltFaOerQ jqjjJ0E4QuRM7AU8+YeyaEsZqkVbHBTH5uKuHh6Z8ZP6cIUP/Qz2KdyKvyhDQ6uezIWlvh kLcN1lP3QXHaHQK/hLCjo7RtP7Vi/lkCo6WwSTjZzu4ImiwsV/SWmoZeLlFaKRKs4lHarO lRJM9NAw7OLnBnddoXk9iTtemb+0ldjTB0/4PiCVrxP8SNZM5o7Y/flz3/hI4yKjq33KJm bOYnui1dIJc1ycaNwXQblk9IJ3sIWZh+D5EdykadbsFUrNvaetQGsxHuMChB5A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2a01:4f9:c011:7a79::1 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -1.41 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2a01:4f9:c011:7a79::1 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 464433FC65 X-Spam-Score: -1.41 X-Migadu-Scanner: scn0.migadu.com X-TUID: YtrtCYv840Fk This introduces a dependency on the (new) python bindings, but since it also yields a 4x performance improvement on the large performance corpus, I think it is worth it. --- debian/control | 1 + notmuch-git.py | 18 +++++++++--------- test/T850-git.sh | 2 ++ 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/debian/control b/debian/control index 0ffe958c..7099fe97 100644 --- a/debian/control +++ b/debian/control @@ -73,6 +73,7 @@ Depends: git, notmuch, python3, + python3-notmuch2 ${misc:Depends} Description: thread-based email index, search and tagging Notmuch is a system for indexing, searching, reading, and tagging diff --git a/notmuch-git.py b/notmuch-git.py index a3ae15f7..eac24a46 100644 --- a/notmuch-git.py +++ b/notmuch-git.py @@ -701,21 +701,21 @@ def _is_unmerged(ref='@{upstream}'): @timed def get_status(): + from notmuch2 import Database + status = { 'deleted': {}, 'missing': {}, } with PrivateIndex(repo=NOTMUCH_GIT_DIR, prefix=TAG_PREFIX) as index: maybe_deleted = index.diff(filter='D') - for id, tags in maybe_deleted.items(): - (_, stdout, stderr) = _spawn( - args=['notmuch', 'search', '--output=files', 'id:{0}'.format(id)], - stdout=_subprocess.PIPE, - wait=True) - if stdout: - status['deleted'][id] = tags - else: - status['missing'][id] = tags + with Database() as notmuch: + for id, tags in maybe_deleted.items(): + try: + _ = notmuch.find(id) + status['deleted'][id] = tags + except LookupError: + status['missing'][id] = tags status['added'] = index.diff(filter='A') return status diff --git a/test/T850-git.sh b/test/T850-git.sh index 81400328..b1bc9e7e 100755 --- a/test/T850-git.sh +++ b/test/T850-git.sh @@ -7,6 +7,8 @@ if [ $NOTMUCH_HAVE_SFSEXP -ne 1 ]; then test_done fi +export PYTHONPATH="$NOTMUCH_BUILDDIR/bindings/python-cffi/build/stage:${PYTHONPATH:+:$PYTHONPATH}" + # be very careful using backup_database / restore_database in this # file, as they fool the cache invalidation checks in notmuch-git. -- 2.35.2