From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id 8CifBvuWm2KmewEAbAwnHQ (envelope-from ) for ; Sat, 04 Jun 2022 19:31:39 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id 0MmMBvuWm2J/RQEAauVa8A (envelope-from ) for ; Sat, 04 Jun 2022 19:31:39 +0200 Received: from mail.notmuchmail.org (yantan.tethera.net [IPv6:2a01:4f9:c011:7a79::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0AA1D45AA3 for ; Sat, 4 Jun 2022 19:31:38 +0200 (CEST) Received: from yantan.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id B182A5F7D0; Sat, 4 Jun 2022 17:23:35 +0000 (UTC) Received: from fethera.tethera.net (fethera.tethera.net [IPv6:2607:5300:60:c5::1]) by mail.notmuchmail.org (Postfix) with ESMTP id 49C835F7C4 for ; Sat, 4 Jun 2022 17:23:33 +0000 (UTC) Received: by fethera.tethera.net (Postfix, from userid 1001) id B85A55FBD7; Sat, 4 Jun 2022 13:23:32 -0400 (EDT) Received: (nullmailer pid 1150080 invoked by uid 1000); Sat, 04 Jun 2022 17:23:25 -0000 From: David Bremner To: notmuch@notmuchmail.org Subject: [PATCH v3 10/17] CLI/git: replace enumeration of tags with sexp query. Date: Sat, 4 Jun 2022 14:23:06 -0300 Message-Id: <20220604172313.1149879-11-david@tethera.net> X-Mailer: git-send-email 2.35.2 In-Reply-To: <20220604172313.1149879-1-david@tethera.net> References: <20220604172313.1149879-1-david@tethera.net> MIME-Version: 1.0 Message-ID-Hash: J5BO5KOJLY43SWFBA447C7VL4BKXHYSZ X-Message-ID-Hash: J5BO5KOJLY43SWFBA447C7VL4BKXHYSZ X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.3 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: DE ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1654363898; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-owner:list-unsubscribe:list-subscribe:list-post; bh=yBzvJXZJRIYE7Zls3fh7/aIzg3mw7mNhRsJzWEqFBw8=; b=YNDR//WjF7fPHAJged9qvaANvQWkbHJmZ4Bbmc84gXPz/th1s2z/T98ErrGLgKx3HLi93/ v8QE5EjEk+mup1ZDBKGp8PM9A1Ogl1DjHQ+DPiomtmP3tKFGTvxdfFGziKdU8csPfwWnVG bPPlGBAe45K8l9AqXSrBVY+2Ne7W2gWGryNKI/+KKntRArPi6hkPO2WR8PU5Vld1jc5EoD 8q3aQl/9Aglh6ZJkBjr9x58Bq6hrDxL2rwzjDLk+D8IlRlrAafBFNVZNCB7sNBeIh+VZGW /ZOBg1I2GuflBr9dzkCtrKTobk7Gw9HwDF496+tBiXtpIDEDOUgmRDAskJl/iw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1654363898; a=rsa-sha256; cv=none; b=Ilc19h2Ie+vJfEaluACTeVHlBk9TV2ECFuCr3GIu+W5aM8gMgp0UtMjT/sj7ZY/ppfAKnq T+CSVj+DcpjvcoctpjE4x3cx3HLSGHtx8XBiarPl06lXIAVLhHePzG+a+zn8yjpjQW/A+e mxqxZ0hsY7gOwlSlyyaAoBmfxWeCnC4iIuZjSY0ggMn16bGUWJl1fiEzcAIyvZr1Am1YJy HSJTpoMMJh0fxhzamUhVLcbfbj74IDstu8HaRq8zvh+sIF6oxVW1DT4SwX/LW1KlWZ6/bs g9IVHOpy4Kql5e3nIb/0WbkVsXMOWc/G1YNNXyHKE46ei+9UyluwLpPVVwXuTg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2a01:4f9:c011:7a79::1 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 5.95 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2a01:4f9:c011:7a79::1 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 0AA1D45AA3 X-Spam-Score: 5.95 X-Migadu-Scanner: scn1.migadu.com X-TUID: ChxgMmbXGXbx Unlike the (current) infix query parser provided by Xapian, the notmuch specific sexp query parser supports prefixed wildcard queries, so use those. In addition to being somewhat faster, this avoids needing to escape all of the user's tags to pass via the shell. --- notmuch-git.py | 26 +++++++++++++++++++------- test/T850-git.sh | 6 +++++- 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/notmuch-git.py b/notmuch-git.py index 543ef4f1..badf9000 100644 --- a/notmuch-git.py +++ b/notmuch-git.py @@ -219,16 +219,17 @@ def _get_remote(): stdout=_subprocess.PIPE, wait=True) return remote.strip() +def _tag_query(prefix=None): + if prefix is None: + prefix = TAG_PREFIX + return '(tag (starts-with "{:s}"))'.format(prefix.replace('"','\\\"')) def get_tags(prefix=None): "Get a list of tags with a given prefix." - if prefix is None: - prefix = TAG_PREFIX (status, stdout, stderr) = _spawn( - args=['notmuch', 'search', '--output=tags', '*'], + args=['notmuch', 'search', '--query=sexp', '--output=tags', _tag_query(prefix)], stdout=_subprocess.PIPE, wait=True) - return [tag for tag in stdout.splitlines() if tag.startswith(prefix)] - + return [tag for tag in stdout.splitlines()] def archive(treeish='HEAD', args=()): """ @@ -584,13 +585,12 @@ def get_status(): def _index_tags(): "Write notmuch tags to the nmbug.index." path = _os.path.join(NOTMUCH_GIT_DIR, 'nmbug.index') - query = ' '.join('tag:"{tag}"'.format(tag=tag) for tag in get_tags()) prefix = '+{0}'.format(_ENCODED_TAG_PREFIX) _git( args=['read-tree', '--empty'], additional_env={'GIT_INDEX_FILE': path}, wait=True) with _spawn( - args=['notmuch', 'dump', '--format=batch-tag', '--', query], + args=['notmuch', 'dump', '--format=batch-tag', '--query=sexp', '--', _tag_query()], stdout=_subprocess.PIPE) as notmuch: with _git( args=['update-index', '--index-info'], @@ -690,6 +690,14 @@ def _help(parser, command=None): else: parser.parse_args(['--help']) +def _notmuch_config_get(key): + (status, stdout, stderr) = _spawn( + args=['notmuch', 'config', 'get', key], + stdout=_subprocess.PIPE, wait=True) + if status != 0: + _LOG.error("failed to run notmuch config") + sys.exit(1) + return stdout.rstrip() if __name__ == '__main__': import argparse @@ -834,6 +842,10 @@ if __name__ == '__main__': for var in ['NOTMUCH_GIT_DIR', 'NOTMUCH_GIT_PREFIX', 'NOTMUCH_PROFILE', 'NOTMUCH_CONFIG' ]: _LOG.debug('env {:s} = {:s}'.format(var, _os.getenv(var,'%None%'))) + if _notmuch_config_get('built_with.sexp_queries') != 'true': + _LOG.error("notmuch git needs sexp query support") + sys.exit(1) + if not getattr(args, 'func', None): parser.print_usage() _sys.exit(1) diff --git a/test/T850-git.sh b/test/T850-git.sh index 2badc52d..72091b56 100755 --- a/test/T850-git.sh +++ b/test/T850-git.sh @@ -2,6 +2,11 @@ test_description='"notmuch git" to save and restore tags' . $(dirname "$0")/test-lib.sh || exit 1 +if [ $NOTMUCH_HAVE_SFSEXP -ne 1 ]; then + printf "Skipping due to missing sfsexp library\n" + test_done +fi + add_email_corpus git config --global user.email notmuch@example.org @@ -20,7 +25,6 @@ notmuch search --output=messages '*' | sort > EXPECTED test_expect_equal_file_nonempty EXPECTED OUTPUT test_begin_subtest "commit, with quoted tag" -test_subtest_known_broken notmuch git -C clone2.git -p '' clone tags.git git -C clone2.git ls-tree -r --name-only HEAD | grep /inbox > BEFORE notmuch tag '+"quoted tag"' '*' -- 2.35.2