From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#47711: bug#48841: bug#47711: bug#48841: bug#47711: [PATCH VERSION 2] Add new `completion-filter-completions` API and deferred highlighting Date: Tue, 31 Oct 2023 05:20:04 +0200 Message-ID: References: <3d3f894f-a6fa-53ae-5d50-c3aa9bffc73e@daniel-mendler.de> <56ab18b1-4348-9b2c-85bb-af9b76cd429a@daniel-mendler.de> <328f87eb-6474-1442-e1ca-9ae8deb2a84a@yandex.ru> <83fsvcbio7.fsf@gnu.org> <9f432d18-e70f-54c1-0173-1899fb66d176@gutov.dev> <877cnafv39.fsf@gmail.com> <9447dde3-b8e7-2ec0-9a9c-72c4cf9d12a8@gutov.dev> <7d14bc13-4419-816c-5708-c42988c39c02@gutov.dev> <5d0a78cc-4fa0-ef04-3462-1826f17d7d56@gutov.dev> <877cn8asud.fsf@gmail.com> <8734xtauqj.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="13454"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: Daniel Mendler , Eli Zaretskii , Stefan Monnier , 47711@debbugs.gnu.org To: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Oct 31 04:20:47 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qxfJ7-0003I2-F9 for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 31 Oct 2023 04:20:46 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxfIt-0003H6-40; Mon, 30 Oct 2023 23:20:31 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxfIr-0003Gv-Df for bug-gnu-emacs@gnu.org; Mon, 30 Oct 2023 23:20:29 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qxfIr-0005YD-1h for bug-gnu-emacs@gnu.org; Mon, 30 Oct 2023 23:20:29 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qxfJO-0002Ik-7c for bug-gnu-emacs@gnu.org; Mon, 30 Oct 2023 23:21:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 31 Oct 2023 03:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47711 X-GNU-PR-Package: emacs Original-Received: via spool by 47711-submit@debbugs.gnu.org id=B47711.16987224508821 (code B ref 47711); Tue, 31 Oct 2023 03:21:02 +0000 Original-Received: (at 47711) by debbugs.gnu.org; 31 Oct 2023 03:20:50 +0000 Original-Received: from localhost ([127.0.0.1]:47285 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qxfJC-0002ID-2L for submit@debbugs.gnu.org; Mon, 30 Oct 2023 23:20:50 -0400 Original-Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:54831) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qxfJ9-0002Hy-Ss for 47711@debbugs.gnu.org; Mon, 30 Oct 2023 23:20:48 -0400 Original-Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id 1AD1F32009DC; Mon, 30 Oct 2023 23:20:08 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Mon, 30 Oct 2023 23:20:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1698722407; x=1698808807; bh=iZ3Ddus2JGcBW/zvgkI4G6xCAqeLIsQt4Ej fnTeCALk=; b=l0J1UwArIqeOSt47wz9tUQwQDiTcfzfZd0V562c9H+Le7agQJ1v 53f2wJwfLuuQ/GetPzgbrViRrGKezfTmZg7UYxIEnkUX0p3d3L8oL6o0SlP2l/jm NfwHljz7OphLmteohymmxUc50z1GBns+GfHGs+5fotJn2EPtOmRSt7pNmQobn9Xq 0R/w/AyusrRySo52xK2mExX5rvbp07we1eC+uxdXUtne+iohuSo0jAJGNiA/veL6 WH99ZdQ9QsB/2u0jcRbKiFovrEJ8MQGBrE1ltFGjBWosayz1mAIGFhCjRJ8fJ3XV 6GVEMJ72JPJXJiaC40CBSXSNuixEHaRJ7ZA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1698722407; x=1698808807; bh=iZ3Ddus2JGcBW/zvgkI4G6xCAqeLIsQt4Ej fnTeCALk=; b=jUsU+avuqHZo7vrQbPWsgXBdL3D9aC+mdQcbRYgxE6u4UPIW/8d 7YDLZ0qhQS9522SqcfgzHHlb+uKlcM4wuWFZWKLjmD0JelhVcnLyuURSPNGmIPNi bWZGyzSowoQ+AKajMSDVyfibMguxReHaxKoodjKoHEPVzaCGHqYoYyc4Xl2yd5T8 BtMjkUck5rjDjc8PlT3ALAuP9ai59qpnMkoWA4c0x3jdSCwEkE9RCGpVON1BCkDg mZ2VfDNHseAoPbKUxhDyGspadiSyjcEH9gljw3eqN3ucNBzfXfFKNJvq7FwEfYkt meARotjXWnVe+Gdk1uiiFPQzI/ao9LuefVg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedruddtuddgheehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephfffheeljeffgeffueeghfekkedtfffgheejvdegjeettdduheeufffggfef jeehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 30 Oct 2023 23:20:06 -0400 (EDT) Content-Language: en-US In-Reply-To: <8734xtauqj.fsf@gmail.com> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:273556 Archived-At: Hi Joao, On 30/10/2023 01:12, João Távora wrote: >> Thanks. My measurements are similar, except the difference switch the >> other way a little bit. It might depend on the particulars of the >> individual machine anyway. > > Yes, it could, but I've reproduced this in different hardware. > > Check that you're taking enough samples, I take about 15-20 samples. > Maybe the lazy-hilit patch pays a an extra cost upfront for the very > first C-h v or completing-read, to create the properties keys on the > strings, which are then reused. I actually do see that. At first I didn't pay much attention to such outliers. They usually look like the second measurement here: Elapsed time: 0.541624s (0.164396s in 5 GCs) Elapsed time: 0.861175s (0.415142s in 10 GCs) Elapsed time: 0.486012s (0.057915s in 1 GCs) Elapsed time: 0.505339s (0.055759s in 1 GCs) Elapsed time: 0.481024s (0.057757s in 1 GCs) Elapsed time: 0.471350s (0.056383s in 1 GCs) Elapsed time: 0.495125s (0.056129s in 1 GCs) Elapsed time: 0.513310s (0.058437s in 1 GCs) Elapsed time: 0.491978s (0.057144s in 1 GCs) Which keys are those? I only know about one key - 'completion-score'. > This cost is ammortized very quickly, > of course, but if you're taking the measurement immediately after Emacs > -Q and with few samples, it skews the numbers. I always took around 16 samples, and now made sure to take exactly that number, starting with "y" and then "yo", "y", etc. Although the timing for the empty input is usually included (but it doesn't look too different from the rest). Anyway, I retook the numbers a couple of times. One of the patches (the same one) still looks a little faster, but the fluctuations between runs are large enough to avoid making any big conclusions: d+d 300000 completing-read 0.504 C-h v 0.630 lazy-hilit-2023-v4 300000 completing-read 0.517 C-h v 0.661 d+d again 300000 completing-read 0.486 C-h v 0.587 lazy-hilit-2023-v4 again 300000 completing-read 0.519 C-h v 0.651 And to double-check, these are comparison between 0001-Add-new-completion-filter-completions-API-and-deferr-v3.diff and lazy-hilit-2023-v4.diff. 16 samples every time. And I think I dropped the run with the "spike" in all or most of the above. The first of the patches doesn't seem to cause it, though. >> All averages made using 'M-x calc-grab-region' followed by 'u M'. > > Wow, thanks for this tip. I wondered if there was an easier way than > M-x cua-rectangle-mark-mode + hand rolled avg function. No problem! I lost your snippet for avg, so had to google. :-/ rectangle-mark-mode (C-x SPC) is still part of the recipe, though. >>>> Third, it made a principled stance to avoid altering the original >>>> strings, even the non-visual text properties. This approach could be >>>> adopted piecewise from Daniel's patch, especially if the performance >>>> ends up the same or insignificantly different in practical usage. >>> If we really wanted to, we could also adopt the non-propertizing >>> approach in my lazy-hilit patch, by calculating the score "just in >>> time", much like Daniel's patch does it. But it should be clear that >>> what we save in allocation in completion-pcm--hilit-commonality, we'll >>> pay for in the same amount in consing later. So no performance benefit. >> >> Not for performance, no. Although the way it separates the sorting >> into its own phase makes it easier to reason about that particular >> cost. And for 300000 symbols, scoring and sorting really take the most >> time, e.g. about 2/3rds. Which might help with optimizing it further >> down in the future, somehow, > > I think further optimization would be localized to the scoring function > itself, not to the place where it is performed. Most likely, yes. It seems to be the most expensive part. But it still seems easier to measure/tweak when it happens during a separate step, rather than mixed in with the rest of completion-all-completions' business. >> But if course it would be nice to avoid the wart, so if you have any >> better ideas, they are welcome. > > I'm not saying it would necessarily spread even further, but if you want > to do scoring "just in time" like I suggested -- presumably to > completely avoid propertizing strings -- that particular wart spreads a > little more and thus becomes something that is slightly harder to remove > later on. Could you describe the other places you think it might spread to? Other completion styles like Orderless? As long as there's only one place producing this property (as opposed to consuming it), it seems straightforward enough to remove anyway. >> So far you advocated toward avoiding breaking changes while >> implementing the present performance improvement. > > Both patches do that, so what I've been arguing for is simplicity and > coherence. I don't think completion-sorted-completions and the > complexity it brings to minibuffer.el is a step in the rigth direction, > and the performance benefits it does undeniably bring can be achieved > with less drastic means. What I meant is, solving the quote-unquote conundrum might require a larger breaking change than the one that you wanted for this discussion. Anyway, have you looked into what it would take to solve it? Such text propertization might actually work as a cheap replacement to returning "a function to ... requote them". If both highlighting and scoring functions work fine on the "unquoted" strings, then we would only need to make sure the "quoted" is used when the completion is inserted. Could we make a rule that every table-with-quoting would have to call a particular exit-function? Perhaps before the existing exit-function. That might solve a bunch of things, though I don't see a robust way to enforce this practice, given that completion-table-with-quoting works on the level of completion tables, whereas :exit-function is only specified in the capf tuple.