Completion

Completion

[Lars Ingebrigtsen]

I've been looking at the completion functions and documentation for
pretty much the first time in my life this morning, and I've got some
questions.  :-/

1)

From

20.6.7 Programmed Completion

     ‘nil’
          This specifies a ‘try-completion’ operation.  The function
          should return ‘t’ if the specified string is a unique and
          exact match; if there is more than one match, it should return
          the common substring of all matches (if the string is an exact
          match for one completion alternative but also matches other
          longer alternatives, the return value is the string); if there
          are no matches, it should return ‘nil’.

Let's say the set is '("foo" "zot") and the string is "fo".  Then
there's one match, and it's unique, of course.  But it's not exact?  So
should it return nil, then?

2) The entire `completing-read' machinery is based on prefix, matches
and nothing else; is that right?  That is, if you want to do completion
on words inside strings (for instance, if you want "foo" to have a match
in a collection string like "zot foo bar", then there is no built-in
machinery in Emacs for this?

3) The completion selection machinery in simple.el is nice, but it would
be nice to be able to pass more information about the strings into it
and get it all back after the user has chosen something.  The obvious
thing is to put a text property on the strings, and that almost, almost
works, except for this:

(defun choose-completion (&optional event)
  "Choose the completion at point.
If EVENT, use EVENT's position to determine the starting position."

[...]

               (setq end (or (next-single-property-change end 'mouse-face)
                             (point-max)))
               (buffer-substring-no-properties beg end)))))

If that had been just `buffer-substring', then the information would
survive back to the caller.

`minibuffer-allow-text-properties' exists.  Would it make sense to add a
new variable, like `completion-allow-text-properties', to handle this
case as well?


Like I said, this entire area is new to me, so I may be misunderstanding
something here...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

RE: Completion

[Drew Adams]

> I've been looking at the completion functions and documentation for
> pretty much the first time in my life this morning, and I've got some
> questions.  :-/

Replying quickly to some of them -

> 2) The entire `completing-read' machinery is based on prefix, matches
> and nothing else; is that right?  

No. See variable `completion-styles'. Literal prefix completion is the `basic' completion style.

In vanilla Emacs there is only one set of completion styles that is ever in effect, defined by option `completion-styles'.  It is a list of different ways to match your input.  Each style in the list is tried, in turn, until one of them successfully completes your input.

All completion candidates you see come from the same style.  You have no control over which style will actually be used for any given input, other than ordering the list ahead of time. And you have no way of knowing which style was actually used to produce a given set of candidates. The relation between your input pattern and the matches is thus sometimes not so clear. There is no way to know, for example, that initial matching failed and partial matching succeeded.

That is, in vanilla Emacs the styles of `completion-styles' can only be used together - all or none; they are never alternatives that you can choose at runtime.

Icicles respects vanilla Emacs `completion-styles' as one of several possible completion "methods" (the method called `vanilla'). Unlike completion styles, Icicles completion methods are alternatives - only one is used at a time to complete your input, and you can switch from one method to another easily.

For method `vanilla', which uses `completion-styles', Icicles gives you ways to change the current set of styles on the fly. Option `icicle-completion-style-sets' is a list of possible `completion-styles' values to choose from. You can flip among these during completion.

Among other things, this means that you can try completing using one style set and, if that doesn't succeed, switch to another.

Any of the sets in `icicle-completion-style-sets' can contain any number of styles, in any order.  In particular, a set can be a singleton, which means that you can selectively try to complete using different individual styles.

> 3) The completion selection machinery in simple.el is nice, but it would
> be nice to be able to pass more information about the strings into it
> and get it all back after the user has chosen something.  The obvious
> thing is to put a text property on the strings, and that almost, almost
> works, except for this:
> 
> (defun choose-completion (&optional event) ...
>    (buffer-substring-no-properties beg end)))))
> 
> If that had been just `buffer-substring', then the information would
> survive back to the caller.

I proposed this for Emacs many years ago. It is what Icicles does. Option `icicle-unpropertize-completion-result-flag' controls this (nil by default).

   Non-nil means strip text properties from the completion result.
   Set or bind this option to non-nil only if you need to ensure, for
   some other library, that the string returned by `completing-read' and
   `read-file-name' has no text properties.

   Typically, you will not use a non-nil value.  Internal text properties
   added by Icicles are always removed anyway.  A non-nil value lets you
   also remove properties such as `face'.

And yes, with Icicles you often put the complete alist-entry (or other value), as a text property, on the string that is displayed as a candidate, and then you retrieve that full value. This can be done automatically.

    (defun icicle-put-whole-cand-prop (cand)
     "Put cdr of CAND on its car, as text property `icicle-whole-candidate'.
   This has no side effects.
   Returns a new propertized string corresponding to (car CAND)."
     (let ((text-cand  (copy-sequence (car cand))))
       (put-text-property 0 (length text-cand) 
                          'icicle-whole-candidate (cdr cand) text-cand)
       (setcar cand text-cand)
       text-cand))

It can also be useful sometimes to allow multiple full candidates that have the same display-candidate string (apart from the full-candidate text property). 

For example, command `icicle-search' uses completion to navigate among search hits.  Duplicate search hits are retained.  Although some search-hit candidates might have the same text, they are located at different buffer positions (and perhaps in different buffers). The position is available in the full-candidate info.

There are many other `completing-read' enhancements that Icicles uses, each of which I've proposed for Emacs in the past.

> `minibuffer-allow-text-properties' exists. 

Icicles binds it to t for `completing-read' (but not for `read-from-minibuffer'). Icicles uses `icicle-unpropertize-completion-result-flag' instead to control this for `completing-read'.

Re: Completion

[Stefan Monnier]

> Let's say the set is '("foo" "zot") and the string is "fo".  Then
> there's one match, and it's unique, of course.  But it's not exact?  So
> should it return nil, then?

It should return "foo".

> 2) The entire `completing-read' machinery is based on prefix, matches
> and nothing else; is that right?

No.  The *completion tables* (i.e. the "objects" that define what
completions are available) only offer methods that work for prefixes.
But `completing-read` uses completion commands which do a lot more than
just let the completion tables do the completion for them.

> That is, if you want to do completion on words inside strings (for
> instance, if you want "foo" to have a match in a collection string
> like "zot foo bar", then there is no built-in machinery in Emacs
> for this?

There's completion-try-completion and completion-all-completions, both
of which offer completions according to completion-styles.

> 3) The completion selection machinery in simple.el is nice, but it would
> be nice to be able to pass more information about the strings into it
> and get it all back after the user has chosen something.  The obvious
> thing is to put a text property on the strings, and that almost, almost
> works, except for this:
>
> (defun choose-completion (&optional event)
>   "Choose the completion at point.
> If EVENT, use EVENT's position to determine the starting position."
>
> [...]
>
>                (setq end (or (next-single-property-change end 'mouse-face)
>                              (point-max)))
>                (buffer-substring-no-properties beg end)))))
>
> If that had been just `buffer-substring', then the information would
> survive back to the caller.
>
> `minibuffer-allow-text-properties' exists.  Would it make sense to add a
> new variable, like `completion-allow-text-properties', to handle this
> case as well?

We could, I guess.  What's your use case?


        Stefan

Re: Completion

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Let's say the set is '("foo" "zot") and the string is "fo".  Then
>> there's one match, and it's unique, of course.  But it's not exact?  So
>> should it return nil, then?
>
> It should return "foo".

Then I guess the documentation should say so...  Does the following look
correct? 

diff --git a/doc/lispref/minibuf.texi b/doc/lispref/minibuf.texi
index d091787a68..4cd2210e2f 100644
--- a/doc/lispref/minibuf.texi
+++ b/doc/lispref/minibuf.texi
@@ -1777,11 +1777,12 @@ Programmed Completion
 @item nil
 This specifies a @code{try-completion} operation.  The function should
 return @code{t} if the specified string is a unique and exact match;
-if there is more than one match, it should return the common substring
-of all matches (if the string is an exact match for one completion
-alternative but also matches other longer alternatives, the return
-value is the string); if there are no matches, it should return
-@code{nil}.
+if there is one match, but it's not an exact match, the function
+should return the match; if there is more than one match, it should
+return the common substring of all matches (if the string is an exact
+match for one completion alternative but also matches other longer
+alternatives, the return value is the string); if there are no
+matches, it should return @code{nil}.
 
 @item t
 This specifies an @code{all-completions} operation.  The function

>> 2) The entire `completing-read' machinery is based on prefix, matches
>> and nothing else; is that right?
>
> No.  The *completion tables* (i.e. the "objects" that define what
> completions are available) only offer methods that work for prefixes.
> But `completing-read` uses completion commands which do a lot more than
> just let the completion tables do the completion for them.

Hm, I see..

>> That is, if you want to do completion on words inside strings (for
>> instance, if you want "foo" to have a match in a collection string
>> like "zot foo bar", then there is no built-in machinery in Emacs
>> for this?
>
> There's completion-try-completion and completion-all-completions, both
> of which offer completions according to completion-styles.

Ah, right, the choices from `completion-styles-alist'.  I thought I had
checked that, but I must have missed that bit.

Thanks.

>> `minibuffer-allow-text-properties' exists.  Would it make sense to add a
>> new variable, like `completion-allow-text-properties', to handle this
>> case as well?
>
> We could, I guess.  What's your use case?

I'm completing over people and films from the IMDB database, and they're
uniquely identified by IDs.  I'd like to complete over the names (or
perhaps also adorn the names with more...  stuff, like years for the
films (but not complete over that bit))), but get the IMDB IDs back
after choosing a name.

So I just propertize the strings before handing them over
`choose-completion', and it almost works except that it strips the text
properties.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Completion

[Lars Ingebrigtsen]

Drew Adams <drew.adams@oracle.com> writes:

> All completion candidates you see come from the same style.  You have
> no control over which style will actually be used for any given input,
> other than ordering the list ahead of time. And you have no way of
> knowing which style was actually used to produce a given set of
> candidates.

Ah, I see.  That explains the somewhat unpredictable behaviour I'm
sometimes seeing when doing completions that I've never figured out.  I
mean, it feels like DWIM 95% of the time, but...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Completion

[Stefan Monnier]

> +if there is one match, but it's not an exact match, the function
> +should return the match;

Actualy, when there is one match, "the common substring of all matches"
is that match, so I think we can make it simpler, e.g.:

    This specifies a @code{try-completion} operation.  The function should
    return @code{t} if the specified string is a unique and exact match;
    It should return @code{nil} if there are no matches; and it should
    return the longest common prefix of all matches otherwise.

>>> `minibuffer-allow-text-properties' exists.  Would it make sense to add a
>>> new variable, like `completion-allow-text-properties', to handle this
>>> case as well?
>>
>> We could, I guess.  What's your use case?
>
> I'm completing over people and films from the IMDB database, and they're
> uniquely identified by IDs.  I'd like to complete over the names (or
> perhaps also adorn the names with more...  stuff, like years for the
> films (but not complete over that bit))), but get the IMDB IDs back
> after choosing a name.

What if the user writes the whole name by hand: how do you get the ID in
this case?  In my experience, the need to handle that "manual case" most
of the time ends up covering just as well the case where the user
selected an entry from the *Completions*.


        Stefan

Re: Completion

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> +if there is one match, but it's not an exact match, the function
>> +should return the match;
>
> Actualy, when there is one match, "the common substring of all matches"
> is that match, so I think we can make it simpler, e.g.:
>
>     This specifies a @code{try-completion} operation.  The function should
>     return @code{t} if the specified string is a unique and exact match;
>     It should return @code{nil} if there are no matches; and it should
>     return the longest common prefix of all matches otherwise.

Yes, that seems clearer.

> What if the user writes the whole name by hand: how do you get the ID in
> this case?  In my experience, the need to handle that "manual case" most
> of the time ends up covering just as well the case where the user
> selected an entry from the *Completions*.

Oh, I didn't even consider that the user may type the name.  Yeah,
you're right; returning the text properties wouldn't really be generally
useful here...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Completion

[Stephen Leake]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> +if there is one match, but it's not an exact match, the function
>>> +should return the match;
>>
>> Actualy, when there is one match, "the common substring of all matches"
>> is that match, so I think we can make it simpler, e.g.:
>>
>>     This specifies a @code{try-completion} operation.  The function should
>>     return @code{t} if the specified string is a unique and exact match;
>>     It should return @code{nil} if there are no matches; and it should
>>     return the longest common prefix of all matches otherwise.
>
> Yes, that seems clearer.
>
>> What if the user writes the whole name by hand: how do you get the ID in
>> this case?  In my experience, the need to handle that "manual case" most
>> of the time ends up covering just as well the case where the user
>> selected an entry from the *Completions*.
>
> Oh, I didn't even consider that the user may type the name.  Yeah,
> you're right; returning the text properties wouldn't really be generally
> useful here...

I had a similar problem completing on names in an Ada buffer; names can
be overloaded, so just the name is not unique.

So I included the line number with the name in the completion table;
each entry looks like:

foo<1>
bar<10>
foo<20>

The user sees those strings, and can complete on the line number to pick
the instance of an overloaded name.

Then the code can retrieve the line number from the completed string.

If the user types the name without completing, you're out of luck (my
code throws an unhelpful error). But that never happens in practice,
since completion works so nicely.

I use icomplete-mode, with completion options set so I have to type C-j
to get out of a completing read with non-matching text.

This code is in a development version of ada-mode; available via email
on request.

For IMDB, you could use | to separate fields, and complete on
filmname|year|actor1|...

Whether the user will ever or often enter text without completing
depends on the use case; finding function names in source code is a much
smaller problem than finding films in IMDB.

-- 
-- Stephe