From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Daniel Mendler Newsgroups: gmane.emacs.devel Subject: Re: Improvement proposals for `completing-read' Date: Wed, 7 Apr 2021 21:46:43 +0200 Message-ID: <0342c2d5-02dd-ad9e-5b8e-dfe52f6469c6@daniel-mendler.de> References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12741"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Apr 07 21:48:12 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lUE9s-0003Dc-2G for ged-emacs-devel@m.gmane-mx.org; Wed, 07 Apr 2021 21:48:12 +0200 Original-Received: from localhost ([::1]:47802 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lUE9r-0005se-2L for ged-emacs-devel@m.gmane-mx.org; Wed, 07 Apr 2021 15:48:11 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:48598) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUE8a-0004uD-9i for emacs-devel@gnu.org; Wed, 07 Apr 2021 15:46:52 -0400 Original-Received: from server.qxqx.de ([2a01:4f8:121:346::180]:49411 helo=mail.qxqx.de) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUE8W-0008Pc-Rh for emacs-devel@gnu.org; Wed, 07 Apr 2021 15:46:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=qxqx.de; s=mail1392553390; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=WtkUW73EkaHvDQQK/kB3Qho1kXIdT9ZZNQSUtP4BaKc=; b=OMJU/CK9OuVgil6MGTijDpt0Xb UkDPFsYAvNP5AKr+NiaHNSXw0ffiauTvmSwHUTracq/+EQGR8v2J1XEzdsLWDSEMNSUyDWY7vm3aG 0VkI+LbnWYS23PFeJnKvAIJZ8P1J/rMxgXRzPfMB9vEosfGIL2Ga2FMNK4j6B08PCuRI=; In-Reply-To: Content-Language: en-US Received-SPF: pass client-ip=2a01:4f8:121:346::180; envelope-from=mail@daniel-mendler.de; helo=mail.qxqx.de X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:267565 Archived-At: On 4/7/21 7:20 PM, Stefan Monnier wrote: > BTW, rereading the below, I see I'm sounding a bit too authoritative. > I'm the original author of the minibuffer.el completion code, but > remember that what I wrote is just my opinion: you'll want to > convince the actual maintainers for some of those changes ;-) I am aware that you are the original author of this code. Thank you for your response and your comments. Actually I felt your response sounded more like "I mostly agree with some of the proposals, let's see patches, then we talk!" ;) I hope we will also hear some opinions by the current maintainers. I respond to the points in your previous mail: >> Proposal 1: Disabling the history > [ I'm not sure there's a great benefit here, since in the situation > where simplicity is what matters, using nil seems like a much better > choice, but: ] I agree that the choice is not great. I would have either chosen `nil` or some special value like `'disable-history` instead. But this is what we got from `read-from-minibuffer`. Therefore I would go with `t` for consistency. >> Proposal 3: Sort file names by history > I think this is just a bug in that we don't pay attention to boundaries > when sorting. It's probably just a matter of prepending the prefix > removed from the all-completions output when looking for an element in > the list (the end of this prefix is returned in the last `cdr` of > `completion-all-completions`). Similarly, we may want to compare with > `string-prefix-p` rather than `equal`. I think for files you cannot get away with looking up the prefix+candidate as obtained from the boundaries. It will work mostly but you may want to implement a bit more path normalization to get better results. There are two different ways one can implement this: (1) Either lookup the prefix+candidate in the history or (2) normalize the history first and lookup the plain candidates. Option (2) will usually require fewer allocations. >> Proposal 4: Add support for `group-function' >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Sounds fine to me, yes. > > [ This touches a delicate issue, of course: since some completion styles > have their idea of sorting (e.g. `flex`), some UIs have other ideas, > and then some completion tables have yet more ideas. > I'm not super-happy with all those functions, to be honest, but it's > not clear what an API should look like to better decouple the UIs, > styles, and backends in this respect, so for now the best is probably > to keep piling on stuff until a clearer picture emerges. ] The ad-hoc sorting of flex is something I am actually not happy with. Is it really necessary to have this? My problem with this is two-fold: - There is this metadata adjustment which somehow unexpectedly adjusts/mutates the metadata - Why does a completion style sort? I am mostly happy with styles which don't do that. When I was working on my Consult package I was actually confused by the "disorder" introduced by `flex` and `fido-mode` even with `display/cycle-sort-function` set to the identity. I should probably read a bit up on the history of this feature, then I will know better. But when I first saw this flex adjustment I got confused. > I'd have expected the "list of lists" result to already include the > group titles. Yes, the group function is supposed to return an alist of group titles and group candidates. I only want to emphasize that the completion UI will use the function twice, first to group/sort the candidates, while retaining the original sorting order. And second the function can be used to obtain the titles again for the candidates which are actually displayed, which may not be many, e.g., only 10 in the case of Ivy and friends for example. >> Proposal 5: Forbid the null completions for `REQUIRE-MATCH=t' >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > I generally like the idea (I've also been bothered by this possibility > to return an empty string), but I haven't looked into fixing the > problem. There's already an obvious way to tell `completing-read` > whether the null string is acceptable (by making the completion-table's > `test-completion` return non-nil for it), so maybe we don't need > a special new value. But I don't have a good sense of the scale of the > potential compatibility breakage [I suspect that such a change might > also fix some code which doesn't bother to check for an empty output. ] > IOW, this deserves a bit for `grep`ping through all the ELisp code you > can get your hands on and try to see how/if it would be affected by such > a change. Okay, I think it is reasonable to gather some data regarding the extent of the problem. But then I would strongly prefer to just fix the issue. That being said, it was not obvious to me that you can actually force require match by writing a proper completion table which properly handles the `test-completion` action. However this is a very indirect way to fix the issue and I think I've observed multiple commands in the wild which choked up with null completion. >> Proposal 6: Return text properties from `completing-read' >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > I think what you mean is that the text properties should be stripped > from the text taken from the minibuffer, but should be preserved from > the text taken from the completion tables's candidates (and then > together with that, you'd want to guarantee that when require-match is > non-nil the string object returned is actually one of those candidates > returned by the completion table)? Yes. > This is one of the important differences between "completion" and > "selection" in that completion is about using a completion-table as > a helper to write a chunk of text (but the resulting text in the end > should not be affected by how the user used the completion table to > come up with the final text), whereas selection is about choosing among > a set of candidates and the result is expected to be `eq` to one of > the candidates. Yes, I agree. I am glad for the discussion regarding the difference of selection/completion (Thanks, Philip K!). I hope we can adjust the API such that it caters for the selection use case a bit better. >> Proposal 7: Allow duplicates and retain object identity >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > [I've been resisting this for quite some years, as Drew can testify.] > > While I agree with preserving object identity, the issue of duplicates > is more problematic, because it means the user can't choose between them > using self-insert-command + RET. IOW such completion-tables are > fundamentally making assumptions about the kind of UI with which they > can be used. Yes, that is right. If you care about the actual candidate you get from a set of duplicates, you must use a completion UI which can handle duplicates. If you use an UI which cannot handle this (which rather completes) you cannot get the distinction. I think there is a real need for this if you look at the Swiper command and you want to navigate across equal lines. Then there is the ivy-occur/embark-export/embark-collect command which allows to export the lines to an occur buffer where you can again navigate over the duplicate items. Given how everything is implemented internally (list of candidates, filtering via completion-all-completions etc), it feels like an unnecessary restriction to remove duplicates. It would be more natural to just allow them. If one actually implements proposal 6 it may be even more natural then, since completion UIs which supports deduplication (and an index based selection) will then just do the right thing automatically. These UIs would have to say - Hah, I got a duplicate let's scramble that and just return the first or a random one of the duplicates. Using deduplication prefixes is a really ugly solution to work around the duplicate issue currently. I am using such prefixes and to make matters worse, these prefixes break the basic completion style. >> Proposal 8: Completion style optimization of filtering and highlighting > > That's not a great solution, but as a temporary patch until we have > a better API it's OK. Yes, it was just the first thing we came up with. It may be worth to think about better alternatives. But as a data point if one uses the Orderless completion style which supports this extension it works really well with many candidates and gives a noticeable performance boost. > I suspect "pass the small subset of candidates once again through > `completion-all-completions'" can be tricky to do (e.g. for things like > file-name completion with `partial-completion`). In `vertico--highlight` I am checking the length of the result. If it is not equal to the original length, the UI simply displays the original candidates. This is an ugly hack. Maybe it would be better to have a `completion-style-operation` variable which can be either `'both`, `'filter` or `'highlight`? Daniel Mendler