From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Bob Bobeck Newsgroups: gmane.emacs.devel Subject: Re: Emacs completion matches selection UI Date: Tue, 7 Jan 2014 22:38:40 -0500 Message-ID: References: <52CC8C69.2030705@yandex.ru> <20140108025840.GA7365@c3po> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Trace: ger.gmane.org 1389152328 10801 80.91.229.3 (8 Jan 2014 03:38:48 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 8 Jan 2014 03:38:48 +0000 (UTC) Cc: emacs-devel@gnu.org, Dmitry Gutov To: Toby Cubitt Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jan 08 04:38:56 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1W0jyp-00051i-ED for ged-emacs-devel@m.gmane.org; Wed, 08 Jan 2014 04:38:55 +0100 Original-Received: from localhost ([::1]:44718 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W0jyo-0007pi-Re for ged-emacs-devel@m.gmane.org; Tue, 07 Jan 2014 22:38:54 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:41366) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W0jyg-0007o8-E7 for emacs-devel@gnu.org; Tue, 07 Jan 2014 22:38:50 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1W0jyb-0000vT-W8 for emacs-devel@gnu.org; Tue, 07 Jan 2014 22:38:46 -0500 Original-Received: from mail-wi0-x244.google.com ([2a00:1450:400c:c05::244]:60400) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1W0jyb-0000vK-HV for emacs-devel@gnu.org; Tue, 07 Jan 2014 22:38:41 -0500 Original-Received: by mail-wi0-f196.google.com with SMTP id hi5so395519wib.7 for ; Tue, 07 Jan 2014 19:38:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=N6eF/nEUzsVa62c69YjKVOBi0iT0ViL03QgDS6mCtZg=; b=br8lCSSqjRmMASqS3iegwE56EzO0p17SXWb2rxfhprX2wi37R0ai0rsUye0jONGfbp KVcn1XPJf7Uz4gCEODU6vUrIel/iUetWvcEwqvBMGnexYWri0GxqjvrTCF2pbnvRWlku /Ndi4uoCQ4nE87W/IQ+ZDIWtVLeb+DtkDbqU8gx0qtGb2eTl1kSOHdvyQtT7XsG455l2 VqQSfA5/D7mwMQi++XmnzzrEOvT5AKeZgua7xlnKOpIcbrwgogHjNnM+G5TzeT49d34b 7HfOkdztvKYzzpLkhbFpvUWp7qMlGF7iZvSWv4UNRrx1cTRDQjol6SkP+bsvR4CGrJVI 87DQ== X-Received: by 10.194.236.9 with SMTP id uq9mr67743275wjc.31.1389152320628; Tue, 07 Jan 2014 19:38:40 -0800 (PST) Original-Received: by 10.227.216.2 with HTTP; Tue, 7 Jan 2014 19:38:40 -0800 (PST) In-Reply-To: <20140108025840.GA7365@c3po> X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a00:1450:400c:c05::244 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:167697 Archived-At: This has got to be the longest stream of crap I've ever seen pushed out on this mailing list. Are you all a bunch of novelists or are you hackers? Pinch this turd off and start coding or shut the hell up. On 1/7/14, Toby Cubitt wrote: > On Wed, Jan 08, 2014 at 03:23:21AM +0400, Dmitry Gutov wrote: >> On 07.01.2014 07:32, Toby Cubitt wrote: >> >> Examples: >> >> 1. `prefix-replaced' and `common-substring' both seem to be tailored >> >> to >> >> the dynamic interface. >> > >> > Not at all. `prefix-replaced' tells you that the buffer substring >> > that is being completed has already been replaced with new text in >> > the buffer. This is important for non-prefix completion (e.g. pattern >> > matches). >> >> > I'm not sure if it's generally applicable. > [...] >> > Any UI that modifies the buffer text will almost certainly need to set >> > `prefix-replaced'; >> >> I don't really see why a non-prefix completion UI (or any other one) has >> to modify the buffer text. Company has an inline "preview" frontend, but >> it uses an overlay, and so the buffer text remains unmodified. > > Ah, you mean using the `before-string' or `after-string' overlay > property? That does sound like a better implementation. > > >> AFAIK, neither Company nor CAPF keep this kind of history: if the >> buffer was modified, what happens after is a new completion, or no >> completion at all (if circumstances told us we should abort). > > Neither Company nor CAPF support refining the completion list by adding > more characters to the prefix/pattern (somewhat similar to > isearch). Completion-UI does, and it's very useful. Indeed, this is one > of the desired features Ted explicitly asked for in his recent post. > > In general, this may require storing a bit more history about the > completion process, so you can continue from where you left off > correctly. At the very least, you need to store which completion backend > was used (which I think Company does too; CAPF makes this trickier, since > it doesn't have a simple way of identifying completion sources). > > You've convinced me that prefix-replaced isn't needed, as long as UIs > aren't allowed to manipulate the buffer text. I need to think about it > more, but perhaps that could be a good restriction to impose as it > prevents UI bugs from mangling the buffer text. > > >> > And the core Completion-UI code that ties all the UIs together makes >> > heavy use of the `prefix-replaced' property. So even in Completion-UI >> > it's not only used by the dynamic interface. >> >> Could you explain how? > > I didn't impose any restriction on the UIs in what they're allowed to do > with the buffer text, so the core uses this e.g. when figuring out what > text it should leave behind in the buffer when a completion is accepted. > > But I think imposing a restriction that UIs don't modify the buffer text > during the completion process is probably a good one, in which case it > may be unnecessary. > >> > Similarly, `common-substring' demarcates the longest common substring >> > if >> > completion text has been inserted in the buffer, which is information a >> > third-party UI might very well want to make use of. >> >> Why? How? > > Who knows how someone might want to make use of it in their UI? They > might want to highlight the common substring differently, say. But this > one's just for convenience anyway. A UI can recalculate it from the > completion list in any case. > >> > The properties documented in the `completion-ui-register-interface' >> > docstring are those that have a standard meaning in Completion-UI. Note >> > that nothing prevents a particular UI widget from storing its own data >> > in >> > its own widget-specific overlay properties if so desired. >> >> Sure. >> >> >> 2. `completion-source' seems to have to refer to a Completion-UI >> >> source, right? >> >> >> >> So how would one use an UI defined as such, in a different package? >> >> Would Company define itself as a new source? Or add a source per each >> >> backend? >> > >> > Depends on what the generic Emacs API for specifying completion >> > sources ends up looking like. If we end up using c-a-p-f, >> > `completion-source' will need to store something that identifies the >> > c-a-p-f entry that returned the completion candidates for this >> > completion process. >> >> Hmm, I was expecting something easier, since IIRC you said Company >> should have been able to use it without major changes. > > Maybe I misunderstood what you were asking. What I meant was, until we've > settled on the API for completion sources, we can't decide on the format > of the data that records which completion source ended up being used. > > If you meant what would Company have done in the *current* Completion-UI > implementation, then yes: Company would have simply defined its backends > via the Completion-UI API, instead of via CAPF or company-backends. > > But we seem to heading towards a different API for defining and selecting > completion sources. Probably CAPF, but perhaps something more similar to > company-backends. > > If the former, we'll very likely want to record which CAPF source > returned the completions somehow. If the latter, then the equivalent of > the backend name is fine. (In the current Completion-UI implementation, > this property stores the completion source name, which modulo > implementation differences is directly equivalent to your backend names.) > > I think you're reading way too much into the details of the current > Completion-UI implementation. The details will inevitably change if we > integrate it into part of a generic Emacs completion API. But whatever > the specifics of the data stored, and whatever we call the > variable/property that stores that data, we'll still need some way of > recording which completion source returned the completions we're dealing > with. Without this information, a number of features become tricky or > impossible to implement in the UI. As you surely know, since both Company > already records and uses information about which backend was used, just > like Completion-UI does. > >> > If we end up using c-a-p-f, `completion-source' >> > will need to store something that identifies the c-a-p-f entry that >> > returned the completion candidates for this completion process. >> >> Probably just the value returned by the successful completion function: >> start, end, collection (aka completion table, which is often a function). > > But how would we use that in the Customization interface, to allow users > to customize the UI in different ways for different sources? > > It seems useful to have this feature, so users can make use it if they > want to. (See the use-case examples I gave in my reply to Stefan.) I'd be > sorry to lose the per-source customization feature. It's one I make light > but frequent use of myself. And users are bound to ask for something like > this later! > > It's a bummer that CAPF doesn't associate unique names with completion > sources, in the way that Company and Completion-UI do. > > One possibility might be to have CAPF functions optionally set a :name > property in the optional PROPS return value if they want to support > per-source UI customization for that source. What I don't see at the > moment is how Customize would discover what all the possible CAPF source > names are, in order to conveniently list them as options in a menu. (In > Completion-UI, this is one of the things handled by the macro used to > define new sources.) But that's a detail. > > >> >>> The best comparison one can make is that >> >>> company-backends loosely corresponds to a mixture of >> >>> completion-ui-source-definitions and >> >>> auto-completion-source-functions. >> >> >> >> The latter: yes, the former: no. It's never used the same way as you >> >> use >> >> `completion-ui-source-definitions'. >> > >> > I said "loosely corresponds", and I stand by that. One of the things a >> > `company-backends' entry does is to define when a given completion >> > source >> > should be used. In Completion-UI, that part of company-backends' role >> > is >> > performed by auto-completion-source-functions. >> >> Yes, that's why I agreed with this comparison (and also made it earlier, >> IIRC). I disagree with the other comparison. > > Sorry, my bad. I read what you wrote the wrong way around. > >> > Could well be. But you seem to be arguing both ways. You also argue >> > that >> > the c-a-p-f API is bad because it's opaque and hard for users to >> > configure. If backend definitions and selection logic are always >> > supplied >> > by package authors, then the fact that c-a-p-f is opaque isn't so >> > significant. >> >> The opaqueness in c-a-p-f is bad because the exact values of >> `company-backends' and `completion-at-point-functions' are significant. >> A third-party package author can only push a function in either of these >> lists, but they can be responsible for the eventual order of the >> elements. >> >> And the order matters, because it influences which completion backend >> will get picked when several are suitable. When we get to the grouped >> backends, which I've mentioned several times, which Company supports, >> and CAPF will hopefully support in the future, being able to understand >> and maybe change the values of either list becomes even more important. > > I completely agree. > > How do you suggest we could improve it, without replacing CAPF or > breaking backwards compatibility? > > This is something you care about more and have more experience of in > Company than I do with Predictive. I'm pretty confident that I can adapt > the Completion-UI code to whatever API we settle on for defining > completion sources and selection logic, without too much pain. > >> >> Please take a look at `company-etags' and `company-dabbrev' and see if >> >> you can point out the situations when the user might find the `prefix' >> >> code of either too limiting. >> > >> > dabbrev and etags are both sources that, if they're useful in a given >> > buffer, they're useful in the entire buffer. >> > >> > Consider the LaTeX math mode source, and the LaTeX environment name >> >> My question was, if you could point out problems with any of our bundled >> backends (or, failing that, third-party ones). If they look okay, maybe > > Did some text get cut off here in your reply? > >> > source, and the LaTeX preamble source, etc. Trying to code the >> > selection >> > logic for all of these using the Company API looks very awkward to me, >> > compared to the simplicity of setting a few buffer-local variables in >> > Completion-UI. >> >> If performance is the problem, we could solve that by either: >> >> 1) Adding some pre-completion hook which would allow you to run some >> code once, set a buffer-local variable, which all backend functions for >> LaTeX can refer to later in `prefix' action. > > I don't think this can work, because the choice of source depends on the > location of point and the text in the buffer, which changes from > completion to completion. > > For the Predictive LaTeX support, I use a mish mash of piggy-backing on > jit-lock where possible, some regexp match tests, and a limited form of > incremental parsing of the LaTeX code when the rest aren't enough. (I > used to exclusively use the latter, implemented using my auto-overlays > package. Surprisingly this was more than efficient enough to cause no > noticeable delay when typing and auto-completing, even on very old > hardware. But writing and debugging the incremental parser definitions > were a nightmare, largely because TeX doesn't have a well-defined > grammar. So I replaced as much as possible of the parsing with simpler > methods.) > > This kind of thing is simpler to do in proper languages (assuming you > already have an incremental parser like CEDET, or a framework like > nxml-mode). > > >> 2) Create a special "merged" backend that would collect the results from >> all LaTeX backend in a carefully defined fashion. > > This effectively means moving the selection logic out of Company and into > Predictive. Which isn't necessarily a bad solution. It's how I used to do > it in Predictive, until I generalised the selection logic and moved it > into Completion-UI so other sources could make use of it. > > If we stick with the CAPF API, I suspect I'll end up moving my source > selection logic back into Predictive, and making it Predictive-specific > again. > >> 3) Indeed add some hook analogous to `auto-completion-source-functions'. >> But yeah, CAPF already does that. >> >> > Sure, I could move that logic into Predictive itself, and have a single >> > Predictive LaTeX backend. But that serves to demonstrate that the API >> > isn't flexible enough to let me do what I want easily. Other markup >> > languages and programming languages make similar demands on the API. >> >> We bundle several backends, some of them for programming modes, and so >> far you haven't pointed out specific problems with any of them. > > Well, so far none of the existing Company backends are trying to do > auto-completion from a dictionary of 100,000 words, in a language with no > regular grammar, and where even tenths of a second lag are sufficient to > make typing unusable. > > It would be an interesting experiment to stress-test the selection > mechanism by reimplementing Predictive's LaTeX support in Company. But it > would take some effort to code, and I don't have time (nor really the > interest) to code this up, especially when the Emacs API for this is > likely to look quite different. > > >> >> Or you can define new backends that would do some common checks in >> >> `prefix' (maybe calling an extracted function with common code) and >> >> simply delegate all other actions to the respective base backend. >> >> Implementing this is trivial. >> > >> > And once you've finished doing this, and factored out common selection >> > mechanisms like regexps, faces and text properties into utilities >> >> Regexps we have already (company-grab-...), > > How fast is this if you have to go through, say, 100 moderately complex > regexps checking for a match? (See the predictive-latex.el in the old > Predictive tarball release for examples.) > >> instead of faces one should be using `syntax-ppss', font-lock isn't >> always available, and text properties... hmm. > > Font-lock faces are *very* useful when you care about efficiency, because > of all the effort that's gone into optimising jit-look. (Font-lock has > very similar speed constraints to Predictive: it must not get in the way > of typing.) > >> > functions...you'll have reimplemented something closer the >> > Completion-UI >> > API or c-a-p-f :) >> >> Only if we add a similar hook, see 3) above. >> >> >> I understand the principle, really. But the more one "cleanly >> >> separates" >> >> code, the harder it can be sometimes to read it, to get the full >> >> picture. >> > >> > Indeed, which is why I listed grouping the completion and selection >> > logic >> > into one place as one of the things I liked about Company's API. >> > >> > Perhaps the cleanest and most flexible solution would be to have a list >> > with entries of the form (TEST-FUNCTION . COMPLETION-FUNCTION), >> > COMPLETION-FUNCTION is used if TEST-FUNCTION returns non-nil. >> >> I believe this suffers from the LaTeX problem you've described above. If >> you have a dozen of completion functions for LaTeX, this scheme expects >> you to have a dozen of corresponding test functions, > > Indeed. > >> and when one fails, the next will be called, and it won't be able to >> use the results of the previous call (unless they set and use some >> common buffer-local variable, which Company backends could also do; but >> that's ugly). > > In my use-case, there's no useful information from previous tests anyway. > > But I don't think this is such an important use-case to consider for a > generic API. Predictive has unusual requirements (always-on > auto-completion as the main use-case imposes extreme efficiency demands). > And (La)TeX and also Texinfo are odd cases, because their lack of a > regular grammar makes background incremental parsing more difficult > compared to programming languages. > > Probably the best solution is to move the fast-selection logic I > implemented for use in LaTeX and Texinfo can back into Predictive. > > >> > And then supply a bunch of standard utility functions for use as >> > TEST-FUNCTION's, for testing regexp matches, faces, text-properties, >> > etc. >> >> Like I described previously, such stand-alone tests probably won't be >> very useful as values of this alist. One usually has to call several of >> them to see if a completion-function is suitable. > > I meant to write "for use *in* test functions". Simple utility functions > for testing regexps (which you already have), faces, etc. which can be > combined to build a test function. > > >> >> It makes certain amount of sense, although it looks like it could make >> >> creating a "merged" completion function more difficult. >> > >> > I doubt it'll be insurmountable. Also, merged completion functions are >> > a >> > rather advanced feature that may not belong in core Emacs anyway >> > (though >> > it would be good if the API supported them without ugly hacks). >> >> It's a good feature enabling some kinds of backends that aren't usually >> useful on their own (like Yasnippet). In my book, that's a good argument >> to accept or reject an API. > > I'd be very happy to see a solution in Emacs if you can come up with one > and convince people to merge it into core. As I said, adapting the > Completion-UI code to a different completion source API shouldn't be > difficult. > >> >> We'll see. >> > >> > Indeed, we'll see. Whilst I'd be happy to see the somewhat complicated >> > and opaque c-a-p-f API replaced with something cleaner and simpler, I >> > don't see us winning that argument. >> >> Personally, I'd probably be fine with c-a-p-f as long as it's powerful >> enough. Moving to a less featureful API is likely out of the question, >> but if it's demonstrated that c-a-p-f is fairly unsuitable for >> implementing some features, I believe it would be a good reason to rule >> it out. > > In Completion-UI, I implemented combined completion sources through a > `completion-ui-combining-complete' wrapper function. (They're not > functionally equivalent to your merged sources, but the requirements are > somewhat similar.) > > Would using something like that to build merged CAPF functions be a > solution for Company? I know this isn't particularly user-friendly if you > want to allow users to easily define their own merged sources. I don't > know if that's something people regularly do in Company. > > > As I said, it's not me you need to persuade if you want to replace > CAPF. I would be happy to see a nicer API than CAPF make its way into > Emacs core. But I also understand why there will be resistance from Emacs > devs to replacing an existing API unless there's a clear and pressing need. > > >> >> But the general approach, while flexible on its surface, complicates >> >> things if I intend to use any existing sources, written by third >> >> parties. Because their authors are unlikely to have anticipated the >> >> logic I'll add in my custom predicate function and to have written any >> >> code in their packages I might use. Or, at least, that's considerably >> >> less likely. >> > >> > I don't get your argument here. You have to wrap the third-party >> > completion function in Company in order to code the appropriate backend >> > selection logic. In Completion-UI, you put that code in an >> > `auto-completion-source-function' instead, and probably don't need to >> >> Which code? There's likely to be none. >> >> Hence, more effort required on my part. >> >> Here I'm describing an organizational problem caused by an API. Not a >> technical one. > > I think we probably agree. Not worth pursuing this particular discussion > further really, since I'm not suggesting we use the Completion-UI API for > source selection. > > There's one thing we should perhaps think a bit more about. > > Is it right to say that the majority of the Company backends are selected > based on global properties of a buffer (e.g. major-modes)? This seems to > be the case for the default `company-backends' list. In Completion-UI I > was almost entirely focused on selecting backends based on local > properties at different locations within a buffer (regexps, faces, > syntax, etc.). > > How do you envisage supporting local source selection in the new (or > enhanced CAPF) API? Would this kind of local completion source selection > always be implemented within a single backend, like the Company CEDET > backend? Or should there be a convenient way of supporting it in the > generic source selection API? > > This kind of relates to your point about the ordering being important... > > >> >> Like mentioned above, delegating the search for completions to an >> >> existing backend is trivial. These are functions, and as such they are >> >> stateless. Just call (other-backend 'candidates current-prefix). >> > >> > But this results is convoluted and somewhat confusing code. If the >> > selection logic and completion function were separate, you wouldn't >> > need >> > to use such hacks. >> >> Looks straightforward to me. Function delegation is a rather simple >> concept. >> >> We have examples of that in the Company frontends: take a look at >> `company-pseudo-tooltip-unless-just-one-frontend' and >> `company-preview-if-just-one-frontend'. >> >> `company-dabbrev-code' also delegates to `company-dabbrev', but it just >> uses a public function from that backend's package, which is also a >> valid approach. > > Following the code path is unnecessarily convoluted. I can't tell that > company-dabbrev-code internally delegates to the company-dabbrev backend > just from looking at `company-backends'. If it looked more like: > > ((use-dabbrev-code-p . dabbrev-completion) > (use-dabbrev-p . dabbrev-completion)) > > it would be immediately obvious that these are two different ways of > using the same source of completions. > >> > It understand why you're harping on about the limitations of the >> > default hook entries >> >> I just can't see how you find them useful on their own. I'll shut up >> now. :) > > Look at predictive-latex.el, which uses them on their own :) > > But I'll shut up too. It seems clear to me from our discussion that we > should either use CAPF or something closer to Company for the completion > source selection API. > > From a Completion-UI perspective, I don't much care what we > use. Switching between completion sources was never the main focus of > Completion-UI (the UI was the main focus). I should be able to adapt the > code to whatever we settle on. Most of it's abstracted into accessor > functions/macros internally, anyway. > >> > Indeed, though this is starting to look a lot like defining some >> > standard >> > hook functions for auto-completion-source-functions. As you make the >> > company-backends API more flexible and convenient, you'll increasingly >> > find yourself reimplementing equivalent functionality to that of >> > c-a-p-f >> > and the Completion-UI API. If were to start simplifying the >> > Completion-UI >> > API or c-a-p-f, it'll increasingly look more like the Company API. >> > Maybe >> > the sweet spot is in the middle? >> >> Maybe. I can certainly see myself adding a >> auto-completion-source-functions analog in Company, as an advanced >> feature. > > The question for now is, what should go in the generic Emacs API? Vanilla > CAPF, perhaps with some additional standard PROPS properties? If the > generic Emacs API can't replace company-backends in Company, then in my > view the API is no good. > > You're the expert here :) > > >> >> Actually, I was thinking about the former option. Let's define widgets >> >> with an API in the usual sense, so it can be used by both >> >> `completion-at-point' and external packages. We'll need this kind of >> >> API >> >> either way, in order to be able to write new widgets. >> > >> > I'm confused. `completion-at-point' will never use an API for >> > *defining* >> > new completion UI widgets (`completion-ui-register-interface'). It will >> > need to be modified to *invoke* the new UI widgets. >> >> Here I'm thinking in terms of Company approach, I guess. The API for >> defining a backend or frontend and the way it is used are the same: a >> protocol consisting of messages and proper reactions or responses to >> them. >> >> Thus, "widget has to respond to..." would be in the API for defining a >> widget, and "widgets respond to..." would be in the usage API. >> Essentially the same. > > Sounds pretty close to the Completion-UI widget API. A widget has to > provide "function that responds to x", "function that responds to y". > >> If you like to add convenience macros, extra infrastructure, etc, they >> may diverge, but we'll need an API a completion package can use easily >> use anyway, not just a bunch of private functions that can change will >> every release. > > Absolutely. > > The way I imagine it, completion packages like Company would rarely need > to use the API for defining new widgets. (Unless they're defining custom > widgets that are so package-specific they don't belong in core Emacs. But > that seems like a rare case.) > > What they *will* all use is the API for *invoking* the completion > interface, which we began to discuss below. > >> > Copying and adapting some of the code from `complete-in-buffer' into >> > `completion-in-region' would suffice for that. Then >> > `completion-in-region' would remain the generic Emacs API for >> > displaying >> > and selecting completion candidates (only now it would display them in >> > a >> > nicer interface). >> >> Maybe. At a first glance, `completion-in-region' will need access to >> PROPS returned by completion functions, not just COLLECTIONS. > > Agreed. I figured we could add an additional PROPS optional argument, as > this wouldn't break backwards compatibility... > >> >> Yes, but see above. Using 1. from Company would be the current next >> >> step >> >> toward integration, as I see it. >> > >> > Really? Why should we use the Company UI widget API and interfaces in >> > Emacs, when the API is less flexible and less customizable than that >> > the >> > Completion-UI widgetAPI, and it implements fewer widgets? >> >> Sorry for being unclear. I meant the reverse: Company would use the new >> widget API defined here, while still retaining its backends, at least >> initially. > > Thanks, makes sense now. > >> Although while you don't own the popup widget, we do have one. Maybe >> porting that code won't be too hard. > > Great! It would be a good test of the API too, to see how easy it is to > port. > > Sounds like your equivalent of the "dynamic" interface is better, too > (you avoid modifying the buffer text). I'd like to either port the > Company version, or adapt the Completion-UI one to use the Company > mechanism (whichever is simpler). > > >> > Did you really mean that you wanted to throw Completion-UI in the >> > garbage, rewrite Company yourself to be suitable as a generic Emacs >> > completion API, rewrite/port the missing UIs and features, and persuade >> > people to merge it into Emacs core? (If so, great! Let me know when >> > you're done, and then I can stop maintaining Completion-UI and switch >> > Predictive over to the new interface :-) >> >> I'll think about it. :) Not sure about the "persuade people to merge in >> into Emacs core", though. > > Ah, but that's the hard part - the rest is easy ;-) > >> But you could help kick this process off my filing an issue describing >> Company's backend API shortcomings. Is it just the lack of >> `auto-completion-source-functions'? Non-prefix completion? > > Erm, I don't see any major shortcomings in the API. Probably that hasn't > come across in the discussion :-) > > Minor ones that have come here were: > > - No support for non-prefix completion (but it looks easy enough to > extend the API). > > - Difficult (impossible?) to distinguish between "no completions > available" and "backend not applicable". > > >> > Probably I misunderstood what you wrote. >> >> Yep. :) > > Shame! I was hoping you were volunteering to do all the work ;-) > >> > I only really have one strong requirement: if some form of >> > auto-completion mode gets included in the Emacs completion UI, I will >> > argue hard for it to be as powerful as Completion-UI's >> > `auto-completion-mode'. (Take a look at `auto-completion-syntax-alist' >> > and `auto-completion-override-syntax-alist', and at the way all the UI >> > widgets are integrated.) Anything less powerful, and it will be useless >> > to me for Predictive. >> >> I'll take a look, thanks. >> >> >> Customizing hooks is a tricky business. I believe the opaqueness of >> >> c-a-p-f to the user is the main problem with the current API. >> > >> > But above you argued that backends and selection logic are supplied and >> > configured by package authors, not by users!? In which case package >> > authors can simply supply a c-a-p-f function, and users can add the >> > functions they want to c-a-p-f. (I believe Stefan made a similar point >> > earlier.) >> >> Customizing a hook is tricky for user. Try `M-x customize-variable RET >> find-file-hook'. You'll only see the buffer-local value, and not the >> global one. But `run-hooks' uses both if the local value includes `t', >> which it usually does. > > I see the global value in Customize, even when I deliberately add a local > hook function. Showing the global value in Customize seems the correct > thing to do. Is that not what Emacs does? (I wouldn't know - I've never > Customized a hook. I always use `add-hook' in my .emacs for that.) > >> Users can modify the hooks programmatically, of course, but that's a >> step more difficult. And they'll also need to understand the values >> already there, to be able to remove or rearrange them. >> >> We'll also should discourage lambdas there. At the moment, my >> find-file-hook contains this beauty, courtesy of autorevert.el: >> >> #[nil "\302\301!\210\303\304 !8\211 \207" >> [buffer-file-name auto-revert-tail-pos make-local-variable 7 >> file-attributes] >> 3] >> >> We should have less of that. > > :-) > >> > Don't get me wrong. I'm no fan of the complexity of c-a-p-f. But you >> > seem to be arguing both ways. >> >> Of course I'm arguing toward a middle ground comfortable to me. So: >> package authors deciding when their backend is suitable to use -- good; >> users deciding the order of trying backends and their groupings -- also >> good. > > OK, makes sense. If I understand correctly, CAPF is fine for letting > users decide the order of trying backends. It's the groupings part that's > less clear, right? > > >> > Swings and roundabouts. I could always add a >> > `completion-ui-major-mode-source' function that checks an alist, and >> > then >> > it would be: >> >> Yes, well, here you are discarding the "standard, tried-and-tested Emacs >> mechanisms" of major mode hooks and buffer-local values. Which was >> exactly my point. > > Now I'm confused. I thought that was *my* point ;-) > > >> > - foo completion function >> > - bar completion function >> > - foo predicate function >> > - bar predicate function >> > - add two entries to the alist >> >> You forgot adding `completion-ui-major-mode-source' to the relevant list. >> >> > - code 1st bundled completion function and selection logic >> > - code 2nd bundled completion function and selection logic (which >> > duplicates the logic in the 1st one, with different parameters) >> > ... >> > - code 8th bundled completion function and selection logic (which >> > duplicates the logic in the last 7, with different parameters) >> >> Could that be a one-line function call, in each case with different >> arguments? Aside from potential performance problems, it sounds rather >> neat. > > Not really. The selection logic for each source involves a list of > regexps, faces, etc. to test, and these are different for each > source. Passing lists of hard-coded regexps and faces around would be > ugly. > > I guess you could store the regexps etc. in configuration variables (one > variable per source, or a single alist), and just pass an identifier as > an argument. > >> > - add 8 entries to company-backends >> > - arrange for package-specific sources to be added to company-backends >> > dynamically >> > >> > Contrast with Completion-UI's, which involves a *lot* less coding: >> > >> > - register 1st source (no need to code a new completion function, just >> > set one parameter when calling `completion-ui-register-source') >> >> You can likewise delegate to the same completion function in Company >> backends. It's the same amount of code (1 line). > > But in Completion-UI, you don't even have to write the code - the macro > does it for you :) > >> > It would be almost trivial to switch Completion-UI over to >> > c-a-p-f. All I'd need to do to replicate the existing functionality >> > is add a few additional properties to the PROPS return argument. >> >> Great. I'd like to see the patch. :) > > I'll start working on it as soon as we agree on the completion source > API, and as soon as I have some spare time. (Which unfortunately isn't > likely to be before the end of term here in Cambridge, as I have a > lecture course to prepare from scratch. But the feature-freeze makes it > less pressing for now.) > > >> > Presumably I could always set a buffer-local `company-backends' from >> > predictive-mode that only contains relevant backends if necessary. >> >> Yes, of course. >> >> > c-a-p-f is trivially customizable by users, as long as packages / >> > Company >> > supplies suitable c-a-p-f functions for the backends. >> >> When you say "trivially customizable", do you mean via the Customize >> interface? > > I just meant that it's a straight list of functions, so it's easy for > users to understand how to configure it. > > I take your point about configuring hooks via the customization interface > being tricky. But at least configuring hooks is standard Emacs foo. I > suspect most users who want to change the order of functions in CAPF will > know how to set hooks. > > It would be nice to define a Customization interface that lists the > available CAPF functions in a drop-down menu (whilst also allowing > free-form entries, of course). If I find a way of porting the > Completion-UI auto-updating customization interface magic to CAPF, this > would be feasible... > > >> > *Writing* c-a-p-f is non-trivial, but as you said earlier those >> > are/should be supplied by package authors. Or have I misunderstood? >> >> I don't think I said that about c-a-p-f, but yes, they should be. > > Good, we're on the same page. > >> >>> 3. The most popular and useful "list-the-available-completions" UI is >> >>> popup.el. Is there any chance of getting copyright assignment >> >>> for >> >>> this? Or will we be forced to code something equivalent from >> >>> scratch? >> >> >> >> https://github.com/auto-complete/popup-el/issues/50 >> > >> > Fingers crossed...! >> >> I wouldn't hold my breath: IME, getting a response from that crowd on >> non-trivial issues is hard. Collecting copyright assignments could be >> harder still. > > Shame. I guess we'll be recoding it then. At least it won't be from > scratch, since you've already got something similar in Company. > >> P.S. My email client displays each new message from you as a separate >> thread, possibly because your email address is timestamped, or maybe >> because they don't contain a "References" header. Could you do something >> about that? > > Strange. Mutt threads them fine for me. The ones I've checked all contain > an "In-Reply-To" header. I haven't changed the mutt reply header > settings, so presumably mutt doesn't add "References" headers by default? > > Perhaps your email client is ignoring the In-Reply-To header? (I somehow > doubt the date stamps in the tagged addresses are the issue. Threads > always contain messages from different senders, so it threading should > work even if your mail client considers the changing dated addresses to > be different senders.) > > All the best, > Toby > -- > Dr T. S. Cubitt > Royal Society University Research Fellow > and Fellow of Churchill College, Cambridge > Centre for Quantum Information > DAMTP, University of Cambridge > > email: tsc25@cantab.net > web: www.dr-qubit.org > >