From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id CJ+VJ7Up7mDeeQAAgWs5BA (envelope-from ) for ; Wed, 14 Jul 2021 02:03:01 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 0KBNI7Up7mCcWwAAB5/wlQ (envelope-from ) for ; Wed, 14 Jul 2021 00:03:01 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D3DF7ADA4 for ; Wed, 14 Jul 2021 02:03:00 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id D5F6129083; Tue, 13 Jul 2021 20:02:52 -0400 (EDT) Received: from fethera.tethera.net (fethera.tethera.net [IPv6:2607:5300:60:c5::1]) by mail.notmuchmail.org (Postfix) with ESMTP id 7BF1829083 for ; Tue, 13 Jul 2021 20:02:44 -0400 (EDT) Received: by fethera.tethera.net (Postfix, from userid 1001) id DE5865FD21; Tue, 13 Jul 2021 20:02:43 -0400 (EDT) Received: (nullmailer pid 804645 invoked by uid 1000); Wed, 14 Jul 2021 00:02:42 -0000 From: David Bremner To: notmuch@notmuchmail.org Subject: Early preview of s-expression based query parser Date: Tue, 13 Jul 2021 21:02:28 -0300 Message-Id: <20210714000239.804384-1-david@tethera.net> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Message-ID-Hash: R7KXHRW6Q3NKP5PZCCYJBGFOCRYT6KNA X-Message-ID-Hash: R7KXHRW6Q3NKP5PZCCYJBGFOCRYT6KNA X-MailFrom: bremner@tethera.net X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1626220981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ApZyqgHadC1Gc0N+q4q9cHXvoOVGH6O+S8SINhKpULo=; b=BqSs4FZFhCn4OZ677+ktzqLflJvMbZ2e6ri3ErLB19o3VWYsXDLULPhcBnUvWc7SshY3lG VvmvwP5ZLxGZAQah6w/c12HAJSIYAfya9W99bbUtSbJOCcYAb9P7Fd9AuE1Sk6cn+owhnR OGCseQaPlfKNCynJ1JPHd5tSF+m/hnQlalSbeoC7msi2pvoh0wpbFpvUmn+YQ5yLvkzyUq CfftVl7cpBXjsdWa4vQzGgzQfEGYuz9vjuM617xAcpob8NbAnJSyvOW9Gwoa4EgxfHCcqn Y72d8xuDKqH6QadtgaSQ5zFWuI1RxX32wbX+nxY1UYyxvTFE6YS7wVf9r3hWoQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1626220981; a=rsa-sha256; cv=none; b=O3x8FvJl/fA/x9yJQ222YWk+jvcgt98iTUdYTjq2GLdNaF201qhGXHK/CLUgvQkXrnJUcJ bgItUPfYyYeTmHnHBTI6Y7UVyRP5d4u+0+Stn+pO5pdZWnrrTJnbQLFJ8tSlBX96RJ0fOt PKQydTc9MV8AjpErZ5otfmxHI0YhPLSQrua+mJv6bMc47IQUjtKTk3hAK/H4HInC3NdISz KfKPesvGEXKzH0ROoOKT/XRdla6aRrFlsmzaR3JDw4YQSpsaDKnQgKVdtULRSMxStoz5nY m6HGvFLeB6KYhwP9kIoLW1/Vh03vSOfazt+knkyKB8E6pHZIpDb8BQ1kOjb6KQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -1.08 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: D3DF7ADA4 X-Spam-Score: -1.08 X-Migadu-Scanner: scn0.migadu.com X-TUID: aCD4g4CPJ0kp I have no plans to replace or deprecate the Xapian query parser based syntax. On the other hand, having full control over the parser makes it easier to support certain features. As an example, once the basic parser is in place, the last patch in the series adds prefixed wildcards with about 10 lines of code. S-expressions are also hand for constructing queries from other queries, which is not really well supported by the Xapian query parser. So it should also make it possible to fix a few bugs resulting from gluing strings together and hoping the result is a valid query. There are quite a few missing features here, and a couple known bugs (the treatment of body: queries is probably mostly broken in the new parser). Finally, when I next re-roll the series I will probably push the error handling earlier in the series. Feedback of any kind is welcome, but particularly on UI / UX issues. You can get a pretty good idea of the supported syntax by looking at the tests. It does add a dependency on sfsexp, a C s-expression parsing library. It's not heavyweight, but it's also not available very many places yet (other than on github). Upstream has been great about merging a few changes for me. One question mark is UTF8 support; it seems to work OK, but upstream is in the process of documenting the support status.