From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#47711: bug#48841: bug#47711: bug#48841: bug#47711: [PATCH VERSION 2] Add new `completion-filter-completions` API and deferred highlighting Date: Thu, 2 Nov 2023 00:45:18 +0200 Message-ID: <31cadbfd-d086-a04f-0ed9-17ce70b4282c@gutov.dev> References: <56ab18b1-4348-9b2c-85bb-af9b76cd429a@daniel-mendler.de> <328f87eb-6474-1442-e1ca-9ae8deb2a84a@yandex.ru> <83fsvcbio7.fsf@gnu.org> <9f432d18-e70f-54c1-0173-1899fb66d176@gutov.dev> <877cnafv39.fsf@gmail.com> <9447dde3-b8e7-2ec0-9a9c-72c4cf9d12a8@gutov.dev> <7d14bc13-4419-816c-5708-c42988c39c02@gutov.dev> <5d0a78cc-4fa0-ef04-3462-1826f17d7d56@gutov.dev> <877cn8asud.fsf@gmail.com> <8734xtauqj.fsf@gmail.com> <5181f95e-61e7-c8c4-6389-44ee57e0c749@gutov.dev> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="9602"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: Daniel Mendler , Eli Zaretskii , Stefan Monnier , 47711@debbugs.gnu.org To: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Nov 01 23:46:48 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qyJz6-0002Fg-Dr for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 01 Nov 2023 23:46:48 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qyJyo-0001UX-C6; Wed, 01 Nov 2023 18:46:30 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qyJym-0001UB-2w for bug-gnu-emacs@gnu.org; Wed, 01 Nov 2023 18:46:28 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qyJyl-0007sW-Qt for bug-gnu-emacs@gnu.org; Wed, 01 Nov 2023 18:46:27 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qyJzK-00089h-Cs for bug-gnu-emacs@gnu.org; Wed, 01 Nov 2023 18:47:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 01 Nov 2023 22:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47711 X-GNU-PR-Package: emacs Original-Received: via spool by 47711-submit@debbugs.gnu.org id=B47711.169887877231254 (code B ref 47711); Wed, 01 Nov 2023 22:47:02 +0000 Original-Received: (at 47711) by debbugs.gnu.org; 1 Nov 2023 22:46:12 +0000 Original-Received: from localhost ([127.0.0.1]:53446 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qyJyR-00087w-HU for submit@debbugs.gnu.org; Wed, 01 Nov 2023 18:46:11 -0400 Original-Received: from out2-smtp.messagingengine.com ([66.111.4.26]:50883) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qyJyL-00087J-Qf for 47711@debbugs.gnu.org; Wed, 01 Nov 2023 18:46:06 -0400 Original-Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 0F1025C0212; Wed, 1 Nov 2023 18:45:22 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 01 Nov 2023 18:45:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1698878722; x=1698965122; bh=ePzhaBKxh8el6GMSrfuuAt8RiLBLPV6ogI9 K8442gAg=; b=jWlTBtOFRpumllmMKFdUPrbI7UybNlOHN+vsLsVajPpCFUgbMC8 +tAeKlJGo4KNMAEGm50AIZqTnknC44UIeFOL7Xa8pvid7u1rgItOFCtbLqTjySLX KofL744YRKJYKdtqedw9AnBN+6zW0UZ4MVOFWlQSCvbpTLI3FaMavFOhzHe/rY6u Ic4Fl515bgZ5j3PqXz/4kBIP2YiACWro8hwGGAtWmC6bjBcrR16EHvVkka+N3o8R xBTGjaoEj1C4g51zg4sw2xB6FO7O1+9U/sB1u0OCVIN7jzs5M6zQ4E+2a7hUmk0B FLh/5LpR5pwQXn106JQtaroZhy/WI/S4k1w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1698878722; x=1698965122; bh=ePzhaBKxh8el6GMSrfuuAt8RiLBLPV6ogI9 K8442gAg=; b=rlntdExTkMDMDbCwz73RCYm/xKb4T4Mgw4eKXTAKCV8DVi4Q610 ML8UzuhKjmkQJpbp7QQsulOeMfcrivdSWBhHpuRuqv4Rj6+lM+yqkemtFHSGxGMG tVFGZeHrwMSDIHLjdNb2z+kGpa8D0nFe4fMR5/mTHYwvqw0ZlcgPc5RARoRyyK/A 0n3FxVlLu7bq4yqcpS2Wuwgo8BACvPrIpIKyLTFFW0VJuWw79jFGLbkkgztis9HX uVmmlRYuEkoOk4M57rlvX/7gXa1MxNM/ljFhC+qqbu3SMoAH/kRwl88VEq8xw99L 0dlE6Yg9pECeKbXCZpDUXs5U3amPEKCbEVg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedruddthedgtddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephfffheeljeffgeffueeghfekkedtfffgheejvdegjeettdduheeufffggfef jeehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 1 Nov 2023 18:45:19 -0400 (EDT) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:273626 Archived-At: On 01/11/2023 20:47, João Távora wrote: > On Tue, Oct 31, 2023 at 8:52 PM Dmitry Gutov wrote: > >> It seems like the only code that would be concerned with it are >> completion styles that also do sorting, or completion tables that would >> do similar things to this "with quoting" business. But I'm not aware of >> any other examples of the latter aside from what is inside Emacs itself. > > If orderless (which I've never tried), does some kind of scoring of > completions, it probably also needs the same complications of flex. Turns out, Orderless doesn't do any scoring or sorting. Only filtering. >>>> Anyway, have you looked into what it would take to solve it? >>> >>> No, naively, I just think it's a similar situation of display and business >>> logic being mixed up. Presumably the quoted stuff is just for insertion >>> (and display?), and the unquoted stuff is what patterns/scoring should >>> operate on. >> >> Apparently it's good for insertion, but according to that comment inside >> the function, the unquoted stuff might actually be better for display. > > No idea what the unquoted stuff is for, so I haven't really tested it. A couple of scenarios: First: 1. sudo mkdir /home/${USER}-foobar 2. C-x C-f /home/${USER} TAB ; it shows both directories inside home as ${USER}-foo/ ${USER}/ Second: 1. mkdir ~/examples/test\ test\ test/ 2. mkdir ~/examples/test\ test/ 3. M-x shell 4. In the shell buffer, type 'ls ~/examples/test\ ' and TAB. See: test\ test/ test\ test\ test/ In the current implementation, both the inputs and the text in the completions buffer that we see, are "quoted". The "unquoted" versions would be the directory name with the variable substitution performed, and the directory names without backslashes. >> I'm not 100% clear which of the versions is better for >> scoring/highlighting, but apparently the unquoted one. >> >>> But, IMO, there's no need to tackle it right now. >>> >>> If the thing holding you back from the lazy-hilit-2023-v4.patch is the >>> completion-score propertization, I can move it to the sorting step >>> in a future v5 and add spread the completion--unquoted thing a little >>> bit more. >> >> I think that's the main blocker, yes. > > Alright, here goes v5 then, with this change. Note I've implemented > this unquoted thing which kicks in in C-x f but I haven't actually > seen any strings that have different "quoted" "non-quoted" versions. > > The performance of the three main patches as measured in yet > another machine: > > ;; C-h v > ;; > ;; Daniel+Dmitry: 0.696340454545 > ;; lazy hilit v4: 0.692849642852 > ;; lazy hilit v5: 0.683088541667 > ;; > ;; completing-read > ;; > ;; Daniel+Dmitry: 0.590994909091 > ;; lazy hilit v4: 0.586523307692 > ;; lazy hilit v5: 0.586165466667 > > Nothing unexpected. Confirm. The "property allocation" spikes are gone too. > So if you're satisfied with the general design now, maybe > we should start looking at finer details, docstrings, style, > etc. LGTM overall, and I see that you compressed the sorting code a little. Both quoting/unquoting scenarios also seem to work as expected (for highlighting, that seems to be thanks to completion--twq-all applying the faces eagerly anyway). Though given the examples (and I think others should be similar) it wouldn't be an end of the world if scoring didn't really work for them -- filtering should have already done most of the job. All of this is to say that any new 3rd party completion styles, even those that do sorting, would be okay without knowing about this text property. Some minor nits for the patch: > +Completion-presenting frontends may opt to bind this variable to > +non-nil value in the context of completion-producing calls (such > +as `completion-all-sorted-completions'). This hints the I suggest mentioning `completion-all-completions' instead, as it is more often used directly by the frontends. > +responsible this fontification. The frontend binds this variable responsible for > +hint and greedily fontify as usual. It is still safe for a "fontify eagerly"? I think that's a more common term than "greedily". > + "Used by completions styles to honouring `completion-lazy-hilit'. "to honour", or "styles honouring" > +(defun completion--flex-score (str re &optional dont-error) Looks like the third argument is unused in both callers. I think it was intended for compose-flex-sort-fn. > +see) for later lazy highlighting" Missing period. > + ;; Lazy highlight not requested, so strings are > + ;; assumed to already contain `completion-> score' > + ;; (and highlighting) and we can freely destroy > + ;; list. Perhaps drop the last two lines, since IIUC the list can be destructively sorted in both cases, lazy highlighting or not. I guess we should wait a few days to see if anyone has more comments, and then install this?