From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Toby Cubitt Newsgroups: gmane.emacs.devel Subject: Re: Emacs completion matches selection UI Date: Wed, 8 Jan 2014 02:58:40 +0000 Message-ID: <20140108025840.GA7365@c3po> References: <52CC8C69.2030705@yandex.ru> Reply-To: Toby Cubitt NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1389149947 17844 80.91.229.3 (8 Jan 2014 02:59:07 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 8 Jan 2014 02:59:07 +0000 (UTC) Cc: emacs-devel@gnu.org To: Dmitry Gutov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jan 08 03:59:15 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1W0jMQ-0007HR-11 for ged-emacs-devel@m.gmane.org; Wed, 08 Jan 2014 03:59:14 +0100 Original-Received: from localhost ([::1]:44598 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W0jMP-0003E1-ME for ged-emacs-devel@m.gmane.org; Tue, 07 Jan 2014 21:59:13 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:34859) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W0jMI-00037V-KD for emacs-devel@gnu.org; Tue, 07 Jan 2014 21:59:10 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1W0jMA-0006mp-BX for emacs-devel@gnu.org; Tue, 07 Jan 2014 21:59:06 -0500 Original-Received: from sanddollar.geekisp.com ([216.168.135.167]:30778) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1W0jMA-0006ml-5L for emacs-devel@gnu.org; Tue, 07 Jan 2014 21:58:58 -0500 Original-Received: (qmail 3936 invoked by uid 1003); 8 Jan 2014 02:58:57 -0000 Original-Received: from localhost (localhost.geekisp.com [127.0.0.1]) by localhost.geekisp.com (tmda-ofmipd) with ESMTP; Tue, 07 Jan 2014 21:58:52 -0500 Content-Disposition: inline In-Reply-To: <52CC8C69.2030705@yandex.ru> X-PGP-Key: http://www.dr-qubit.org/gpg-toby-pub.asc User-Agent: Mutt/1.5.22 (2013-10-16) X-Delivery-Agent: TMDA/1.1.11 (Ladyburn) X-Primary-Address: toby@dr-qubit.org X-detected-operating-system: by eggs.gnu.org: OpenBSD 4.x-5.x [fuzzy] X-Received-From: 216.168.135.167 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:167690 Archived-At: On Wed, Jan 08, 2014 at 03:23:21AM +0400, Dmitry Gutov wrote: > On 07.01.2014 07:32, Toby Cubitt wrote: > >> Examples: > >> 1. `prefix-replaced' and `common-substring' both seem to be tailored to > >> the dynamic interface. > > > > Not at all. `prefix-replaced' tells you that the buffer substring > > that is being completed has already been replaced with new text in > > the buffer. This is important for non-prefix completion (e.g. pattern > > matches). >=20 > > I'm not sure if it's generally applicable. [...] > > Any UI that modifies the buffer text will almost certainly need to set > > `prefix-replaced'; >=20 > I don't really see why a non-prefix completion UI (or any other one) has= =20 > to modify the buffer text. Company has an inline "preview" frontend, but= =20 > it uses an overlay, and so the buffer text remains unmodified. Ah, you mean using the `before-string' or `after-string' overlay property? That does sound like a better implementation. > AFAIK, neither Company nor CAPF keep this kind of history: if the > buffer was modified, what happens after is a new completion, or no > completion at all (if circumstances told us we should abort). Neither Company nor CAPF support refining the completion list by adding more characters to the prefix/pattern (somewhat similar to isearch). Completion-UI does, and it's very useful. Indeed, this is one of the desired features Ted explicitly asked for in his recent post. In general, this may require storing a bit more history about the completion process, so you can continue from where you left off correctly. At the very least, you need to store which completion backend was used (which I think Company does too; CAPF makes this trickier, since it doesn't have a simple way of identifying completion sources). You've convinced me that prefix-replaced isn't needed, as long as UIs aren't allowed to manipulate the buffer text. I need to think about it more, but perhaps that could be a good restriction to impose as it prevents UI bugs from mangling the buffer text. > > And the core Completion-UI code that ties all the UIs together makes > > heavy use of the `prefix-replaced' property. So even in Completion-UI > > it's not only used by the dynamic interface. >=20 > Could you explain how? I didn't impose any restriction on the UIs in what they're allowed to do with the buffer text, so the core uses this e.g. when figuring out what text it should leave behind in the buffer when a completion is accepted. But I think imposing a restriction that UIs don't modify the buffer text during the completion process is probably a good one, in which case it may be unnecessary. > > Similarly, `common-substring' demarcates the longest common substring if > > completion text has been inserted in the buffer, which is information a > > third-party UI might very well want to make use of. >=20 > Why? How? Who knows how someone might want to make use of it in their UI? They might want to highlight the common substring differently, say. But this one's just for convenience anyway. A UI can recalculate it from the completion list in any case. > > The properties documented in the `completion-ui-register-interface' > > docstring are those that have a standard meaning in Completion-UI. Note > > that nothing prevents a particular UI widget from storing its own data = in > > its own widget-specific overlay properties if so desired. >=20 > Sure. >=20 > >> 2. `completion-source' seems to have to refer to a Completion-UI > >> source, right? > >> > >> So how would one use an UI defined as such, in a different package? > >> Would Company define itself as a new source? Or add a source per each > >> backend? > > > > Depends on what the generic Emacs API for specifying completion > > sources ends up looking like. If we end up using c-a-p-f, > > `completion-source' will need to store something that identifies the > > c-a-p-f entry that returned the completion candidates for this > > completion process. >=20 > Hmm, I was expecting something easier, since IIRC you said Company=20 > should have been able to use it without major changes. Maybe I misunderstood what you were asking. What I meant was, until we've settled on the API for completion sources, we can't decide on the format of the data that records which completion source ended up being used. If you meant what would Company have done in the *current* Completion-UI implementation, then yes: Company would have simply defined its backends via the Completion-UI API, instead of via CAPF or company-backends. But we seem to heading towards a different API for defining and selecting completion sources. Probably CAPF, but perhaps something more similar to company-backends. If the former, we'll very likely want to record which CAPF source returned the completions somehow. If the latter, then the equivalent of the backend name is fine. (In the current Completion-UI implementation, this property stores the completion source name, which modulo implementation differences is directly equivalent to your backend names.) I think you're reading way too much into the details of the current Completion-UI implementation. The details will inevitably change if we integrate it into part of a generic Emacs completion API. But whatever the specifics of the data stored, and whatever we call the variable/property that stores that data, we'll still need some way of recording which completion source returned the completions we're dealing with. Without this information, a number of features become tricky or impossible to implement in the UI. As you surely know, since both Company already records and uses information about which backend was used, just like Completion-UI does. > > If we end up using c-a-p-f, `completion-source' > > will need to store something that identifies the c-a-p-f entry that > > returned the completion candidates for this completion process. >=20 > Probably just the value returned by the successful completion function:= =20 > start, end, collection (aka completion table, which is often a function). But how would we use that in the Customization interface, to allow users to customize the UI in different ways for different sources? It seems useful to have this feature, so users can make use it if they want to. (See the use-case examples I gave in my reply to Stefan.) I'd be sorry to lose the per-source customization feature. It's one I make light but frequent use of myself. And users are bound to ask for something like this later! It's a bummer that CAPF doesn't associate unique names with completion sources, in the way that Company and Completion-UI do. One possibility might be to have CAPF functions optionally set a :name property in the optional PROPS return value if they want to support per-source UI customization for that source. What I don't see at the moment is how Customize would discover what all the possible CAPF source names are, in order to conveniently list them as options in a menu. (In Completion-UI, this is one of the things handled by the macro used to define new sources.) But that's a detail. > >>> The best comparison one can make is that > >>> company-backends loosely corresponds to a mixture of > >>> completion-ui-source-definitions and auto-completion-source-functions. > >> > >> The latter: yes, the former: no. It's never used the same way as you u= se > >> `completion-ui-source-definitions'. > > > > I said "loosely corresponds", and I stand by that. One of the things a > > `company-backends' entry does is to define when a given completion sour= ce > > should be used. In Completion-UI, that part of company-backends' role is > > performed by auto-completion-source-functions. >=20 > Yes, that's why I agreed with this comparison (and also made it earlier,= =20 > IIRC). I disagree with the other comparison. Sorry, my bad. I read what you wrote the wrong way around. > > Could well be. But you seem to be arguing both ways. You also argue that > > the c-a-p-f API is bad because it's opaque and hard for users to > > configure. If backend definitions and selection logic are always suppli= ed > > by package authors, then the fact that c-a-p-f is opaque isn't so > > significant. >=20 > The opaqueness in c-a-p-f is bad because the exact values of=20 > `company-backends' and `completion-at-point-functions' are significant.= =20 > A third-party package author can only push a function in either of these= =20 > lists, but they can be responsible for the eventual order of the elements. > > And the order matters, because it influences which completion backend=20 > will get picked when several are suitable. When we get to the grouped=20 > backends, which I've mentioned several times, which Company supports,=20 > and CAPF will hopefully support in the future, being able to understand= =20 > and maybe change the values of either list becomes even more important. I completely agree. How do you suggest we could improve it, without replacing CAPF or breaking backwards compatibility? This is something you care about more and have more experience of in Company than I do with Predictive. I'm pretty confident that I can adapt the Completion-UI code to whatever API we settle on for defining completion sources and selection logic, without too much pain. > >> Please take a look at `company-etags' and `company-dabbrev' and see if > >> you can point out the situations when the user might find the `prefix' > >> code of either too limiting. > > > > dabbrev and etags are both sources that, if they're useful in a given > > buffer, they're useful in the entire buffer. > > > > Consider the LaTeX math mode source, and the LaTeX environment name >=20 > My question was, if you could point out problems with any of our bundled= =20 > backends (or, failing that, third-party ones). If they look okay, maybe Did some text get cut off here in your reply? > > source, and the LaTeX preamble source, etc. Trying to code the selection > > logic for all of these using the Company API looks very awkward to me, > > compared to the simplicity of setting a few buffer-local variables in > > Completion-UI. >=20 > If performance is the problem, we could solve that by either: >=20 > 1) Adding some pre-completion hook which would allow you to run some=20 > code once, set a buffer-local variable, which all backend functions for= =20 > LaTeX can refer to later in `prefix' action. I don't think this can work, because the choice of source depends on the location of point and the text in the buffer, which changes from completion to completion. For the Predictive LaTeX support, I use a mish mash of piggy-backing on jit-lock where possible, some regexp match tests, and a limited form of incremental parsing of the LaTeX code when the rest aren't enough. (I used to exclusively use the latter, implemented using my auto-overlays package. Surprisingly this was more than efficient enough to cause no noticeable delay when typing and auto-completing, even on very old hardware. But writing and debugging the incremental parser definitions were a nightmare, largely because TeX doesn't have a well-defined grammar. So I replaced as much as possible of the parsing with simpler methods.) This kind of thing is simpler to do in proper languages (assuming you already have an incremental parser like CEDET, or a framework like nxml-mode). > 2) Create a special "merged" backend that would collect the results from= =20 > all LaTeX backend in a carefully defined fashion. This effectively means moving the selection logic out of Company and into Predictive. Which isn't necessarily a bad solution. It's how I used to do it in Predictive, until I generalised the selection logic and moved it into Completion-UI so other sources could make use of it. If we stick with the CAPF API, I suspect I'll end up moving my source selection logic back into Predictive, and making it Predictive-specific again. > 3) Indeed add some hook analogous to `auto-completion-source-functions'.= =20 > But yeah, CAPF already does that. >=20 > > Sure, I could move that logic into Predictive itself, and have a single > > Predictive LaTeX backend. But that serves to demonstrate that the API > > isn't flexible enough to let me do what I want easily. Other markup > > languages and programming languages make similar demands on the API. >=20 > We bundle several backends, some of them for programming modes, and so=20 > far you haven't pointed out specific problems with any of them. Well, so far none of the existing Company backends are trying to do auto-completion from a dictionary of 100,000 words, in a language with no regular grammar, and where even tenths of a second lag are sufficient to make typing unusable. It would be an interesting experiment to stress-test the selection mechanism by reimplementing Predictive's LaTeX support in Company. But it would take some effort to code, and I don't have time (nor really the interest) to code this up, especially when the Emacs API for this is likely to look quite different. > >> Or you can define new backends that would do some common checks in > >> `prefix' (maybe calling an extracted function with common code) and > >> simply delegate all other actions to the respective base backend. > >> Implementing this is trivial. > > > > And once you've finished doing this, and factored out common selection > > mechanisms like regexps, faces and text properties into utilities >=20 > Regexps we have already (company-grab-...), How fast is this if you have to go through, say, 100 moderately complex regexps checking for a match? (See the predictive-latex.el in the old Predictive tarball release for examples.) > instead of faces one should be using `syntax-ppss', font-lock isn't > always available, and text properties... hmm. Font-lock faces are *very* useful when you care about efficiency, because of all the effort that's gone into optimising jit-look. (Font-lock has very similar speed constraints to Predictive: it must not get in the way of typing.) > > functions...you'll have reimplemented something closer the Completion-UI > > API or c-a-p-f :) >=20 > Only if we add a similar hook, see 3) above. >=20 > >> I understand the principle, really. But the more one "cleanly separate= s" > >> code, the harder it can be sometimes to read it, to get the full pictu= re. > > > > Indeed, which is why I listed grouping the completion and selection log= ic > > into one place as one of the things I liked about Company's API. > > > > Perhaps the cleanest and most flexible solution would be to have a list > > with entries of the form (TEST-FUNCTION . COMPLETION-FUNCTION), > > COMPLETION-FUNCTION is used if TEST-FUNCTION returns non-nil. >=20 > I believe this suffers from the LaTeX problem you've described above. If= =20 > you have a dozen of completion functions for LaTeX, this scheme expects= =20 > you to have a dozen of corresponding test functions, Indeed. > and when one fails, the next will be called, and it won't be able to > use the results of the previous call (unless they set and use some > common buffer-local variable, which Company backends could also do; but > that's ugly). In my use-case, there's no useful information from previous tests anyway. But I don't think this is such an important use-case to consider for a generic API. Predictive has unusual requirements (always-on auto-completion as the main use-case imposes extreme efficiency demands). And (La)TeX and also Texinfo are odd cases, because their lack of a regular grammar makes background incremental parsing more difficult compared to programming languages. Probably the best solution is to move the fast-selection logic I implemented for use in LaTeX and Texinfo can back into Predictive. > > And then supply a bunch of standard utility functions for use as > > TEST-FUNCTION's, for testing regexp matches, faces, text-properties, > > etc. >=20 > Like I described previously, such stand-alone tests probably won't be=20 > very useful as values of this alist. One usually has to call several of= =20 > them to see if a completion-function is suitable. I meant to write "for use *in* test functions". Simple utility functions for testing regexps (which you already have), faces, etc. which can be combined to build a test function. > >> It makes certain amount of sense, although it looks like it could make > >> creating a "merged" completion function more difficult. > > > > I doubt it'll be insurmountable. Also, merged completion functions are a > > rather advanced feature that may not belong in core Emacs anyway (though > > it would be good if the API supported them without ugly hacks). >=20 > It's a good feature enabling some kinds of backends that aren't usually= =20 > useful on their own (like Yasnippet). In my book, that's a good argument= =20 > to accept or reject an API. I'd be very happy to see a solution in Emacs if you can come up with one and convince people to merge it into core. As I said, adapting the Completion-UI code to a different completion source API shouldn't be difficult. > >> We'll see. > > > > Indeed, we'll see. Whilst I'd be happy to see the somewhat complicated > > and opaque c-a-p-f API replaced with something cleaner and simpler, I > > don't see us winning that argument. >=20 > Personally, I'd probably be fine with c-a-p-f as long as it's powerful=20 > enough. Moving to a less featureful API is likely out of the question,=20 > but if it's demonstrated that c-a-p-f is fairly unsuitable for=20 > implementing some features, I believe it would be a good reason to rule= =20 > it out. In Completion-UI, I implemented combined completion sources through a `completion-ui-combining-complete' wrapper function. (They're not functionally equivalent to your merged sources, but the requirements are somewhat similar.) Would using something like that to build merged CAPF functions be a solution for Company? I know this isn't particularly user-friendly if you want to allow users to easily define their own merged sources. I don't know if that's something people regularly do in Company. As I said, it's not me you need to persuade if you want to replace CAPF. I would be happy to see a nicer API than CAPF make its way into Emacs core. But I also understand why there will be resistance from Emacs devs to replacing an existing API unless there's a clear and pressing need. > >> But the general approach, while flexible on its surface, complicates > >> things if I intend to use any existing sources, written by third > >> parties. Because their authors are unlikely to have anticipated the > >> logic I'll add in my custom predicate function and to have written any > >> code in their packages I might use. Or, at least, that's considerably > >> less likely. > > > > I don't get your argument here. You have to wrap the third-party > > completion function in Company in order to code the appropriate backend > > selection logic. In Completion-UI, you put that code in an > > `auto-completion-source-function' instead, and probably don't need to >=20 > Which code? There's likely to be none. >=20 > Hence, more effort required on my part. >=20 > Here I'm describing an organizational problem caused by an API. Not a=20 > technical one. I think we probably agree. Not worth pursuing this particular discussion further really, since I'm not suggesting we use the Completion-UI API for source selection. There's one thing we should perhaps think a bit more about. Is it right to say that the majority of the Company backends are selected based on global properties of a buffer (e.g. major-modes)? This seems to be the case for the default `company-backends' list. In Completion-UI I was almost entirely focused on selecting backends based on local properties at different locations within a buffer (regexps, faces, syntax, etc.). How do you envisage supporting local source selection in the new (or enhanced CAPF) API? Would this kind of local completion source selection always be implemented within a single backend, like the Company CEDET backend? Or should there be a convenient way of supporting it in the generic source selection API? This kind of relates to your point about the ordering being important... > >> Like mentioned above, delegating the search for completions to an > >> existing backend is trivial. These are functions, and as such they are > >> stateless. Just call (other-backend 'candidates current-prefix). > > > > But this results is convoluted and somewhat confusing code. If the > > selection logic and completion function were separate, you wouldn't need > > to use such hacks. >=20 > Looks straightforward to me. Function delegation is a rather simple conce= pt. >=20 > We have examples of that in the Company frontends: take a look at=20 > `company-pseudo-tooltip-unless-just-one-frontend' and=20 > `company-preview-if-just-one-frontend'. >=20 > `company-dabbrev-code' also delegates to `company-dabbrev', but it just= =20 > uses a public function from that backend's package, which is also a=20 > valid approach. Following the code path is unnecessarily convoluted. I can't tell that company-dabbrev-code internally delegates to the company-dabbrev backend just from looking at `company-backends'. If it looked more like: ((use-dabbrev-code-p . dabbrev-completion) (use-dabbrev-p . dabbrev-completion)) it would be immediately obvious that these are two different ways of using the same source of completions. > > It understand why you're harping on about the limitations of the > > default hook entries >=20 > I just can't see how you find them useful on their own. I'll shut up > now. :) Look at predictive-latex.el, which uses them on their own :) But I'll shut up too. It seems clear to me from our discussion that we should either use CAPF or something closer to Company for the completion source selection API. =46rom a Completion-UI perspective, I don't much care what we use. Switching between completion sources was never the main focus of Completion-UI (the UI was the main focus). I should be able to adapt the code to whatever we settle on. Most of it's abstracted into accessor functions/macros internally, anyway. > > Indeed, though this is starting to look a lot like defining some standa= rd > > hook functions for auto-completion-source-functions. As you make the > > company-backends API more flexible and convenient, you'll increasingly > > find yourself reimplementing equivalent functionality to that of c-a-p-f > > and the Completion-UI API. If were to start simplifying the Completion-= UI > > API or c-a-p-f, it'll increasingly look more like the Company API. Maybe > > the sweet spot is in the middle? >=20 > Maybe. I can certainly see myself adding a > auto-completion-source-functions analog in Company, as an advanced > feature. The question for now is, what should go in the generic Emacs API? Vanilla CAPF, perhaps with some additional standard PROPS properties? If the generic Emacs API can't replace company-backends in Company, then in my view the API is no good. You're the expert here :) > >> Actually, I was thinking about the former option. Let's define widgets > >> with an API in the usual sense, so it can be used by both > >> `completion-at-point' and external packages. We'll need this kind of A= PI > >> either way, in order to be able to write new widgets. > > > > I'm confused. `completion-at-point' will never use an API for *defining* > > new completion UI widgets (`completion-ui-register-interface'). It will > > need to be modified to *invoke* the new UI widgets. >=20 > Here I'm thinking in terms of Company approach, I guess. The API for=20 > defining a backend or frontend and the way it is used are the same: a=20 > protocol consisting of messages and proper reactions or responses to them. >=20 > Thus, "widget has to respond to..." would be in the API for defining a=20 > widget, and "widgets respond to..." would be in the usage API.=20 > Essentially the same. Sounds pretty close to the Completion-UI widget API. A widget has to provide "function that responds to x", "function that responds to y". > If you like to add convenience macros, extra infrastructure, etc, they=20 > may diverge, but we'll need an API a completion package can use easily=20 > use anyway, not just a bunch of private functions that can change will=20 > every release. Absolutely. The way I imagine it, completion packages like Company would rarely need to use the API for defining new widgets. (Unless they're defining custom widgets that are so package-specific they don't belong in core Emacs. But that seems like a rare case.) What they *will* all use is the API for *invoking* the completion interface, which we began to discuss below. > > Copying and adapting some of the code from `complete-in-buffer' into > > `completion-in-region' would suffice for that. Then > > `completion-in-region' would remain the generic Emacs API for displaying > > and selecting completion candidates (only now it would display them in a > > nicer interface). >=20 > Maybe. At a first glance, `completion-in-region' will need access to=20 > PROPS returned by completion functions, not just COLLECTIONS. Agreed. I figured we could add an additional PROPS optional argument, as this wouldn't break backwards compatibility... > >> Yes, but see above. Using 1. from Company would be the current next st= ep > >> toward integration, as I see it. > > > > Really? Why should we use the Company UI widget API and interfaces in > > Emacs, when the API is less flexible and less customizable than that the > > Completion-UI widgetAPI, and it implements fewer widgets? >=20 > Sorry for being unclear. I meant the reverse: Company would use the new= =20 > widget API defined here, while still retaining its backends, at least=20 > initially. Thanks, makes sense now. > Although while you don't own the popup widget, we do have one. Maybe=20 > porting that code won't be too hard. Great! It would be a good test of the API too, to see how easy it is to port. Sounds like your equivalent of the "dynamic" interface is better, too (you avoid modifying the buffer text). I'd like to either port the Company version, or adapt the Completion-UI one to use the Company mechanism (whichever is simpler). > > Did you really mean that you wanted to throw Completion-UI in the > > garbage, rewrite Company yourself to be suitable as a generic Emacs > > completion API, rewrite/port the missing UIs and features, and persuade > > people to merge it into Emacs core? (If so, great! Let me know when > > you're done, and then I can stop maintaining Completion-UI and switch > > Predictive over to the new interface :-) >=20 > I'll think about it. :) Not sure about the "persuade people to merge in= =20 > into Emacs core", though. Ah, but that's the hard part - the rest is easy ;-) > But you could help kick this process off my filing an issue describing=20 > Company's backend API shortcomings. Is it just the lack of=20 > `auto-completion-source-functions'? Non-prefix completion? Erm, I don't see any major shortcomings in the API. Probably that hasn't come across in the discussion :-) Minor ones that have come here were: - No support for non-prefix completion (but it looks easy enough to extend the API). - Difficult (impossible?) to distinguish between "no completions available" and "backend not applicable". > > Probably I misunderstood what you wrote. >=20 > Yep. :) Shame! I was hoping you were volunteering to do all the work ;-) > > I only really have one strong requirement: if some form of > > auto-completion mode gets included in the Emacs completion UI, I will > > argue hard for it to be as powerful as Completion-UI's > > `auto-completion-mode'. (Take a look at `auto-completion-syntax-alist' > > and `auto-completion-override-syntax-alist', and at the way all the UI > > widgets are integrated.) Anything less powerful, and it will be useless > > to me for Predictive. >=20 > I'll take a look, thanks. >=20 > >> Customizing hooks is a tricky business. I believe the opaqueness of > >> c-a-p-f to the user is the main problem with the current API. > > > > But above you argued that backends and selection logic are supplied and > > configured by package authors, not by users!? In which case package > > authors can simply supply a c-a-p-f function, and users can add the > > functions they want to c-a-p-f. (I believe Stefan made a similar point > > earlier.) >=20 > Customizing a hook is tricky for user. Try `M-x customize-variable RET=20 > find-file-hook'. You'll only see the buffer-local value, and not the=20 > global one. But `run-hooks' uses both if the local value includes `t',=20 > which it usually does. I see the global value in Customize, even when I deliberately add a local hook function. Showing the global value in Customize seems the correct thing to do. Is that not what Emacs does? (I wouldn't know - I've never Customized a hook. I always use `add-hook' in my .emacs for that.) > Users can modify the hooks programmatically, of course, but that's a=20 > step more difficult. And they'll also need to understand the values=20 > already there, to be able to remove or rearrange them. >=20 > We'll also should discourage lambdas there. At the moment, my=20 > find-file-hook contains this beauty, courtesy of autorevert.el: >=20 > #[nil "\302\301!\210\303\304=08!8\211=11\207" > [buffer-file-name auto-revert-tail-pos make-local-variable 7=20 > file-attributes] > 3] >=20 > We should have less of that. :-) > > Don't get me wrong. I'm no fan of the complexity of c-a-p-f. But you > > seem to be arguing both ways. >=20 > Of course I'm arguing toward a middle ground comfortable to me. So:=20 > package authors deciding when their backend is suitable to use -- good;= =20 > users deciding the order of trying backends and their groupings -- also= =20 > good. OK, makes sense. If I understand correctly, CAPF is fine for letting users decide the order of trying backends. It's the groupings part that's less clear, right? > > Swings and roundabouts. I could always add a > > `completion-ui-major-mode-source' function that checks an alist, and th= en > > it would be: >=20 > Yes, well, here you are discarding the "standard, tried-and-tested Emacs= =20 > mechanisms" of major mode hooks and buffer-local values. Which was=20 > exactly my point. Now I'm confused. I thought that was *my* point ;-) > > - foo completion function > > - bar completion function > > - foo predicate function > > - bar predicate function > > - add two entries to the alist >=20 > You forgot adding `completion-ui-major-mode-source' to the relevant list. >=20 > > - code 1st bundled completion function and selection logic > > - code 2nd bundled completion function and selection logic (which > > duplicates the logic in the 1st one, with different parameters) > > ... > > - code 8th bundled completion function and selection logic (which > > duplicates the logic in the last 7, with different parameters) >=20 > Could that be a one-line function call, in each case with different=20 > arguments? Aside from potential performance problems, it sounds rather ne= at. Not really. The selection logic for each source involves a list of regexps, faces, etc. to test, and these are different for each source. Passing lists of hard-coded regexps and faces around would be ugly. I guess you could store the regexps etc. in configuration variables (one variable per source, or a single alist), and just pass an identifier as an argument. > > - add 8 entries to company-backends > > - arrange for package-specific sources to be added to company-backends > > dynamically > > > > Contrast with Completion-UI's, which involves a *lot* less coding: > > > > - register 1st source (no need to code a new completion function, just > > set one parameter when calling `completion-ui-register-source') >=20 > You can likewise delegate to the same completion function in Company=20 > backends. It's the same amount of code (1 line). But in Completion-UI, you don't even have to write the code - the macro does it for you :) > > It would be almost trivial to switch Completion-UI over to > > c-a-p-f. All I'd need to do to replicate the existing functionality > > is add a few additional properties to the PROPS return argument. >=20 > Great. I'd like to see the patch. :) I'll start working on it as soon as we agree on the completion source API, and as soon as I have some spare time. (Which unfortunately isn't likely to be before the end of term here in Cambridge, as I have a lecture course to prepare from scratch. But the feature-freeze makes it less pressing for now.) > > Presumably I could always set a buffer-local `company-backends' from > > predictive-mode that only contains relevant backends if necessary. >=20 > Yes, of course. >=20 > > c-a-p-f is trivially customizable by users, as long as packages / Compa= ny > > supplies suitable c-a-p-f functions for the backends. >=20 > When you say "trivially customizable", do you mean via the Customize=20 > interface? I just meant that it's a straight list of functions, so it's easy for users to understand how to configure it. I take your point about configuring hooks via the customization interface being tricky. But at least configuring hooks is standard Emacs foo. I suspect most users who want to change the order of functions in CAPF will know how to set hooks. It would be nice to define a Customization interface that lists the available CAPF functions in a drop-down menu (whilst also allowing free-form entries, of course). If I find a way of porting the Completion-UI auto-updating customization interface magic to CAPF, this would be feasible... > > *Writing* c-a-p-f is non-trivial, but as you said earlier those > > are/should be supplied by package authors. Or have I misunderstood? >=20 > I don't think I said that about c-a-p-f, but yes, they should be. Good, we're on the same page. > >>> 3. The most popular and useful "list-the-available-completions" UI is > >>> popup.el. Is there any chance of getting copyright assignment for > >>> this? Or will we be forced to code something equivalent from scr= atch? > >> > >> https://github.com/auto-complete/popup-el/issues/50 > > > > Fingers crossed...! >=20 > I wouldn't hold my breath: IME, getting a response from that crowd on=20 > non-trivial issues is hard. Collecting copyright assignments could be=20 > harder still. Shame. I guess we'll be recoding it then. At least it won't be from scratch, since you've already got something similar in Company. > P.S. My email client displays each new message from you as a separate=20 > thread, possibly because your email address is timestamped, or maybe=20 > because they don't contain a "References" header. Could you do something= =20 > about that? Strange. Mutt threads them fine for me. The ones I've checked all contain an "In-Reply-To" header. I haven't changed the mutt reply header settings, so presumably mutt doesn't add "References" headers by default? Perhaps your email client is ignoring the In-Reply-To header? (I somehow doubt the date stamps in the tagged addresses are the issue. Threads always contain messages from different senders, so it threading should work even if your mail client considers the changing dated addresses to be different senders.) All the best, Toby --=20 Dr T. S. Cubitt Royal Society University Research Fellow and Fellow of Churchill College, Cambridge Centre for Quantum Information DAMTP, University of Cambridge email: tsc25@cantab.net web: www.dr-qubit.org