bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Lennart Borgman]

I get this backtrace below in certain cases when doing C-x C-f and entering

  c:/ema TAB

in the minibuffer. I have no good idea of how to debug it.

It happens only after I have loaded a multi major mode file. However
it happens not after just loading mumamo.el.

If I enter

  c:/emacs/something TAB

it does not happen either, i.e. it happens only before the second "/".

Tramp is loaded after the first TAB above but that is loaded in both
the cases where I get an error and those where I do not get it. (I
have no idea why tramp is loaded though.)

I do not know if I can track this down to something more simple so I
ask for suggestions and qualified guesses instead. Anyone has some?

This is with a checkout from 2010-05-22, my patched version.


Debugger entered--Lisp error: (args-out-of-range "c:" 0 3)
  signal(args-out-of-range ("c:" 0 3))
  completion--some(#[(style) " 	\236A@\n \f
$\207" [style completion-styles-alist string table pred point] 5]
(basic partial-completion emacs22))
  completion-try-completion("c:/ema" read-file-name-internal file-exists-p 6)
  completion--do-completion()
  minibuffer-complete()
  call-interactively(minibuffer-complete nil nil)
  completing-read("Find file: " read-file-name-internal file-exists-p
nil "c:/emacs/p/100522/EmacsW32/nxhtml/util/" file-name-history
"c:/emacs/p/100522/EmacsW32/nxhtml/util/")
  ad-Orig-read-file-name("Find file: " nil
"c:/emacs/p/100522/EmacsW32/nxhtml/util/" nil nil nil)
  (setq ad-return-value (ad-Orig-read-file-name prompt dir
default-filename mustmatch initial predicate))
  (let ((viper-minibuffer-exit-hook ...)) (setq ad-return-value
(ad-Orig-read-file-name prompt dir default-filename mustmatch initial
predicate)))
  (let (ad-return-value) (let (...) (setq ad-return-value ...)) ad-return-value)
  read-file-name("Find file: " nil "c:/emacs/p/100522/EmacsW32/nxhtml/util/")
  (cons (read-file-name "Find file: " nil default-directory) (cond
(... ...) (... ...)))
  call-interactively(find-file nil nil)

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Eli Zaretskii]

> From: Lennart Borgman <lennart.borgman@gmail.com>
> Date: Sat, 29 May 2010 03:35:00 +0200
> Cc: 
> 
> I get this backtrace below in certain cases when doing C-x C-f and entering
> 
>   c:/ema TAB
> 
> in the minibuffer. I have no good idea of how to debug it.
> 
> It happens only after I have loaded a multi major mode file. However
> it happens not after just loading mumamo.el.
> 
> If I enter
> 
>   c:/emacs/something TAB
> 
> it does not happen either, i.e. it happens only before the second "/".
> 
> Tramp is loaded after the first TAB above but that is loaded in both
> the cases where I get an error and those where I do not get it. (I
> have no idea why tramp is loaded though.)
> 
> I do not know if I can track this down to something more simple so I
> ask for suggestions and qualified guesses instead. Anyone has some?
> 
> This is with a checkout from 2010-05-22, my patched version.
> 
> 
> Debugger entered--Lisp error: (args-out-of-range "c:" 0 3)
>   signal(args-out-of-range ("c:" 0 3))
>   completion--some(#[(style) " 	\236A@\n \f

I would suggest to step through completion-emacs22-try-completion in
Edebug.

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Stefan Monnier]

> Debugger entered--Lisp error: (args-out-of-range "c:" 0 3)
>   signal(args-out-of-range ("c:" 0 3))
>   completion--some(#[(style) " 	\236A@\n \f
> $\207" [style completion-styles-alist string table pred point] 5]
> (basic partial-completion emacs22))

First thing: edit completion--some o as to replace condition-case with
condition-case-no-debug.  This way the error won't be delayed and you'll
get a more precise backtrace.


        Stefan

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Lennart Borgman]

On Sat, May 29, 2010 at 11:18 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> Debugger entered--Lisp error: (args-out-of-range "c:" 0 3)
>>   signal(args-out-of-range ("c:" 0 3))
>>   completion--some(#[(style) "        \236A@\n \f
>> $\207" [style completion-styles-alist string table pred point] 5]
>> (basic partial-completion emacs22))
>
> First thing: edit completion--some o as to replace condition-case with
> condition-case-no-debug.  This way the error won't be delayed and you'll
> get a more precise backtrace.

OK:

Debugger entered--Lisp error: (args-out-of-range "c:" 0 3)
  completion-pcm--find-all-completions("c:" read-file-name-internal
file-exists-p 2 completion-pcm--filename-try-filter)
  completion-pcm--find-all-completions("c:/em" read-file-name-internal
file-exists-p 5 completion-pcm--filename-try-filter)
  completion-pcm-try-completion("c:/em" read-file-name-internal file-exists-p 5)
  #[(style) " 	\236A@\n \f
$\207" [style completion-styles-alist string table pred point]
5](partial-completion)
  funcall(#[(style) " 	\236A@\n \f
$\207" [style completion-styles-alist string table pred point] 5]
partial-completion)
  (setq res (funcall fun arg))
  (progn (setq arg (pop xs)) (setq res (funcall fun arg)))
  (lambda nil (progn (setq arg ...) (setq res ...)))()
  funcall((lambda nil (progn (setq arg ...) (setq res ...))))
  (if debug-on-error (funcall body) (condition-case err (funcall body)
(error ... nil)))
  (let ((body ...)) (if debug-on-error (funcall body) (condition-case
err ... ...)))
  (condition-case-no-debug err (progn (setq arg ...) (setq res ...))
(error (unless firsterror ... ...) nil))
  (while (and (not res) xs) (condition-case-no-debug err (progn ...
...) (error ... nil)))
  (let ((firsterror nil) arg res) (while (and ... xs)
(condition-case-no-debug err ... ...)) (or res (if firsterror ...)))
  completion--some(#[(style) " 	\236A@\n \f
$\207" [style completion-styles-alist string table pred point] 5]
(basic partial-completion emacs22))
  completion-try-completion("c:/em" read-file-name-internal file-exists-p 5)
  completion--do-completion()
  minibuffer-complete()
  call-interactively(minibuffer-complete nil nil)
  completing-read("Find file: " read-file-name-internal file-exists-p
nil "c:/emacs/p/100522/emacs/lisp/" file-name-history
"c:/emacs/p/100522/emacs/lisp/")
  ad-Orig-read-file-name("Find file: " nil
"c:/emacs/p/100522/emacs/lisp/" nil nil nil)
  (setq ad-return-value (ad-Orig-read-file-name prompt dir
default-filename mustmatch initial predicate))
  (let ((viper-minibuffer-exit-hook ...)) (setq ad-return-value
(ad-Orig-read-file-name prompt dir default-filename mustmatch initial
predicate)))
  (let (ad-return-value) (let (...) (setq ad-return-value ...)) ad-return-value)
  read-file-name("Find file: " nil "c:/emacs/p/100522/emacs/lisp/")
  (cons (read-file-name "Find file: " nil default-directory) (cond
(... ...) (... ...)))
  call-interactively(find-file nil nil)

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Lennart Borgman]

On Sat, May 29, 2010 at 11:24 PM, Lennart Borgman
<lennart.borgman@gmail.com> wrote:
> On Sat, May 29, 2010 at 11:18 PM, Stefan Monnier
> <monnier@iro.umontreal.ca> wrote:
>>> Debugger entered--Lisp error: (args-out-of-range "c:" 0 3)
>>>   signal(args-out-of-range ("c:" 0 3))
>>>   completion--some(#[(style) "        \236A@\n \f
>>> $\207" [style completion-styles-alist string table pred point] 5]
>>> (basic partial-completion emacs22))
>>
>> First thing: edit completion--some o as to replace condition-case with
>> condition-case-no-debug.  This way the error won't be delayed and you'll
>> get a more precise backtrace.

eval-buffer in minibuffer.el made it more understandable. Here it is
(but I do not know the cause of it):

* I added a little trace here

(defun completion-pcm--find-all-completions (string table pred point
                                                    &optional filter)
  "Find all completions for STRING at POINT in TABLE, satisfying PRED.
POINT is a position inside STRING.
FILTER is a function applied to the return value, that can be used, e.g. to
filter out additional entries (because TABLE migth not obey PRED)."
  (unless filter (setq filter 'identity))
  (message "FIND: filter=%S" filter)
  (let* ((beforepoint (substring string 0 point))
         (afterpoint (substring string point))
         (bounds (completion-boundaries beforepoint table pred afterpoint))
         (dummy (message "FIND: string=%S bounds=%S, beforepoint=%S
afterpoint=%S" string bounds beforepoint afterpoint))
         (prefix (substring beforepoint 0 (car bounds)))
         (suffix (substring afterpoint (cdr bounds)))
         firsterror)

and in

(defun completion-boundaries (string table pred suffix)
  "Return the boundaries of the completions returned by TABLE for STRING.
STRING is the string on which completion will be performed.
SUFFIX is the string after point.
The result is of the form (START . END) where START is the position
in STRING of the beginning of the completion field and END is the position
in SUFFIX of the end of the completion field.
E.g. for simple completion tables, the result is always (0 . (length SUFFIX))
and for file names the result is the positions delimited by
the closest directory separators."
  (message "BOUNDARIES string=%S table=%s pred=%s suffix=%S" string
table pred suffix)
  (let ((boundaries (if (functionp table)
                        (funcall table string pred (cons 'boundaries suffix)))))
    (message "BOUNDARIES boundaries=%S" boundaries)
    (if (not (eq (car-safe boundaries) 'boundaries))
        (setq boundaries nil))
    (cons (or (cadr boundaries) 0)
          (or (cddr boundaries) (length suffix)))))

and got

BOUNDARIES string="c:/ema" table=read-file-name-internal
pred=file-exists-p suffix=""
BOUNDARIES boundaries=(boundaries 3)
FIND: filter=completion-pcm--filename-try-filter
BOUNDARIES string="c:/ema" table=read-file-name-internal
pred=file-exists-p suffix=""
BOUNDARIES boundaries=(boundaries 3)
FIND: string="c:/ema" bounds=(3 . 0), beforepoint="c:/ema" afterpoint=""
FIND: prefix="c:/"
FIND: substring="c:"
FIND: filter=completion-pcm--filename-try-filter
BOUNDARIES string="c:" table=read-file-name-internal
pred=file-exists-p suffix=""
BOUNDARIES boundaries=(boundaries 3)
FIND: string="c:" bounds=(3 . 0), beforepoint="c:" afterpoint=""
Entering debugger...

It looks like perhaps read-file-name-internal gives the wrong information.


PS: There is an advice on read-file-name is

  Around-advice `viper-suffix-advice':
  Tell `exit-minibuffer' to run `viper-file-add-suffix' as a hook.

which does not seem to be involved (I unadvised read-file-name).

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Stefan Monnier]

> BOUNDARIES string="c:" table=read-file-name-internal pred=file-exists-p suffix=""
> BOUNDARIES boundaries=(boundaries 3)

This appears to be the problem.

  (read-file-name-internal "c:" nil '(boundaries . ""))

seems to return (boundaries 3 . 0).
Since the code that does that is most likely the one in
completion--file-name-table which does:

    (let ((start (length (file-name-directory string)))
          (end (string-match-p "/" (cdr action))))
      (list* 'boundaries start end)))

my guess is that (file-name-directory "c:") return "c:/".
I guess we should just use (or (string-match "[^/]*\\'" string) 0)
instead of (length (file-name-directory string)) but it has N^2
complexity :-(


        Stefan

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Lennart Borgman]

On Sun, May 30, 2010 at 1:29 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> BOUNDARIES string="c:" table=read-file-name-internal pred=file-exists-p suffix=""
>> BOUNDARIES boundaries=(boundaries 3)
>
> This appears to be the problem.
>
>  (read-file-name-internal "c:" nil '(boundaries . ""))
>
> seems to return (boundaries 3 . 0).
> Since the code that does that is most likely the one in
> completion--file-name-table which does:
>
>    (let ((start (length (file-name-directory string)))
>          (end (string-match-p "/" (cdr action))))
>      (list* 'boundaries start end)))
>
> my guess is that (file-name-directory "c:") return "c:/".

Yes, that is the case, it returns "c:/".

But I think that is can't return something. However in my opinion it
should return an error because "c:" is not a file path. It is just a
device. It it signaled an error code that tries something like this
would be more easily found and cured.

So I think the problem is that (file-name-directory "c:") is called.
There must be some bad assumption somewhere in the code behind that.


Another suggestion that might help cleaning the code logic in
different places: Add a function directory-root-p

w32: (directory-root-p "c:/") => t
*nix: (directory-root-p "/") => t

w32: (directory-root-p "/") => nil (or signal error, but that would
fit badly with file-directory-p)
*nix: (directory-root-p "c;/") => nil -"-


> I guess we should just use (or (string-match "[^/]*\\'" string) 0)
> instead of (length (file-name-directory string)) but it has N^2
> complexity :-(
>
>
>        Stefan
>

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Stefan Monnier]

>> my guess is that (file-name-directory "c:") return "c:/".
> Yes, that is the case, it returns "c:/".

What about (file-name-nondirectory "c:")?

And what about (file-name-all-completions
                (file-name-nondirectory "c:")
                (file-name-directory "c:")) ?


-- Stefan

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Juanma Barranquero]

On Sun, May 30, 2010 at 05:34, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> What about (file-name-nondirectory "c:")?

ELISP> (file-name-nondirectory "c:")
""

> And what about (file-name-all-completions
>                (file-name-nondirectory "c:")
>                (file-name-directory "c:")) ?

Returns all files and dirs in C:/

    Juanma

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Lennart Borgman]

On Sun, May 30, 2010 at 6:27 AM, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Sun, May 30, 2010 at 05:34, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
>> What about (file-name-nondirectory "c:")?
>
> ELISP> (file-name-nondirectory "c:")
> ""

Same here.

>> And what about (file-name-all-completions
>>                (file-name-nondirectory "c:")
>>                (file-name-directory "c:")) ?
>
> Returns all files and dirs in C:/

The same + tramp's symbols (fcp:, psftp:, plinkx: etc) and tramp's
usernames ("some.one@" etc).

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Lennart Borgman]

On Sun, May 30, 2010 at 2:10 AM, Lennart Borgman
<lennart.borgman@gmail.com> wrote:
> On Sun, May 30, 2010 at 1:29 AM, Stefan Monnier
> <monnier@iro.umontreal.ca> wrote:
>>> BOUNDARIES string="c:" table=read-file-name-internal pred=file-exists-p suffix=""
>>> BOUNDARIES boundaries=(boundaries 3)
>>
>> This appears to be the problem.
>>
>>  (read-file-name-internal "c:" nil '(boundaries . ""))
>>
>> seems to return (boundaries 3 . 0).
>> Since the code that does that is most likely the one in
>> completion--file-name-table which does:
>>
>>    (let ((start (length (file-name-directory string)))
>>          (end (string-match-p "/" (cdr action))))
>>      (list* 'boundaries start end)))
>>
>> my guess is that (file-name-directory "c:") return "c:/".
>
> Yes, that is the case, it returns "c:/".
>
> But I think that is can't return something. However in my opinion it
> should return an error because "c:" is not a file path. It is just a
> device. It it signaled an error code that tries something like this
> would be more easily found and cured.
>
> So I think the problem is that (file-name-directory "c:") is called.
> There must be some bad assumption somewhere in the code behind that.
>
>
> Another suggestion that might help cleaning the code logic in
> different places: Add a function directory-root-p
>
> w32: (directory-root-p "c:/") => t
> *nix: (directory-root-p "/") => t
>
> w32: (directory-root-p "/") => nil (or signal error, but that would
> fit badly with file-directory-p)
> *nix: (directory-root-p "c;/") => nil -"-


I am looking around at the code a bit. I do not understand the comment
in completion--file-name-table:

  (cond
   ((eq (car-safe action) 'boundaries)
    ;; For the boundaries, we can't really delegate to
    ;; completion-file-name-table and then fix them up, because it
    ;; would require us to track the relationship between `str' and
    ;; `string', which is difficult.  And in any case, if
    ;; substitute-in-file-name turns "fo-$TO-ba" into "fo-o/b-ba", there's
    ;; no way for us to return proper boundaries info, because the
    ;; boundary is not (yet) in `string'.

AFAICS it is doing exactly the same as completion-file-name-table
here. What am I missing?

BTW the doc string for action in completing read etc should be fixed.


>> I guess we should just use (or (string-match "[^/]*\\'" string) 0)
>> instead of (length (file-name-directory string)) but it has N^2
>> complexity :-(
>>
>>
>>        Stefan
>>
>

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Stefan Monnier]

> I am looking around at the code a bit. I do not understand the comment
> in completion--file-name-table:

>   (cond
>    ((eq (car-safe action) 'boundaries)
>     ;; For the boundaries, we can't really delegate to
>     ;; completion-file-name-table and then fix them up, because it
>     ;; would require us to track the relationship between `str' and
>     ;; `string', which is difficult.  And in any case, if
>     ;; substitute-in-file-name turns "fo-$TO-ba" into "fo-o/b-ba", there's
>     ;; no way for us to return proper boundaries info, because the
>     ;; boundary is not (yet) in `string'.

> AFAICS it is doing exactly the same as completion-file-name-table
> here. What am I missing?

You're missing that "delegate to completion-file-name-table" means
calling substitute-in-file-name first and then
completion-file-name-table, so it wouldn't do the same.  But yes, we
could call completion-file-name-table directly without going through
substitute-in-file-name.  The reason why I don't do that is mostly
historical (I first started to do something more clever which didn't
work).

> BTW the doc string for action in completing read etc should be fixed.

I don't know what you mean here.


        Stefan

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Stefan Monnier]

>>> What about (file-name-nondirectory "c:")?
ELISP> (file-name-nondirectory "c:")
>> ""

Good thanks.

>>> And what about (file-name-all-completions
>>>                (file-name-nondirectory "c:")
>>>                (file-name-directory "c:")) ?
>> Returns all files and dirs in C:/

Again, thank you.

> The same + tramp's symbols (fcp:, psftp:, plinkx: etc) and tramp's
> usernames ("some.one@" etc).

Not sure if that's correct, but it's a separate problem.
I've installed the patch below which should hopefully fix the original problem.
Please confirm.


        Stefan


=== modified file 'lisp/minibuffer.el'
--- lisp/minibuffer.el	2010-05-12 00:39:46 +0000
+++ lisp/minibuffer.el	2010-05-30 14:13:36 +0000
@@ -1342,7 +1342,14 @@
    ((eq (car-safe action) 'boundaries)
     (let ((start (length (file-name-directory string)))
           (end (string-match-p "/" (cdr action))))
-      (list* 'boundaries start end)))
+      (list* 'boundaries
+             ;; if `string' is "C:" in w32, (file-name-directory string)
+             ;; returns "C:/", so `start' is 3 rather than 2.
+             ;; Not quite sure what is The Right Fix, but clipping it
+             ;; back to 2 will work for this particular case.  We'll
+             ;; see if we can come up with a better fix when we bump
+             ;; into more such problematic cases.
+             (min start (length string)) end)))
 
      ((eq action 'lambda)
       (if (zerop (length string))
@@ -1395,17 +1402,18 @@
   (cond
    ((eq (car-safe action) 'boundaries)
     ;; For the boundaries, we can't really delegate to
-    ;; completion-file-name-table and then fix them up, because it
-    ;; would require us to track the relationship between `str' and
+    ;; substitute-in-file-name+completion-file-name-table and then fix
+    ;; them up (as we do for the other actions), because it would
+    ;; require us to track the relationship between `str' and
     ;; `string', which is difficult.  And in any case, if
-    ;; substitute-in-file-name turns "fo-$TO-ba" into "fo-o/b-ba", there's
-    ;; no way for us to return proper boundaries info, because the
-    ;; boundary is not (yet) in `string'.
-    ;; FIXME: Actually there is a way to return correct boundaries info,
-    ;; at the condition of modifying the all-completions return accordingly.
-    (let ((start (length (file-name-directory string)))
-          (end (string-match-p "/" (cdr action))))
-      (list* 'boundaries start end)))
+    ;; substitute-in-file-name turns "fo-$TO-ba" into "fo-o/b-ba",
+    ;; there's no way for us to return proper boundaries info, because
+    ;; the boundary is not (yet) in `string'.
+    ;;
+    ;; FIXME: Actually there is a way to return correct boundaries
+    ;; info, at the condition of modifying the all-completions
+    ;; return accordingly. But for now, let's not bother.
+    (completion-file-name-table string pred action))
 
        (t
     (let* ((default-directory

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Lennart Borgman]

On Sun, May 30, 2010 at 4:05 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>
> You're missing that "delegate to completion-file-name-table" means

No, but I saw in the patch that you understood what I mean. Excuse my
bad explanation, I was just convinced that you should see it.

>> BTW the doc string for action in completing read etc should be fixed.
>
> I don't know what you mean here.

I meant the argument ACTION to some function, but what exactly was I reading??

Note that my current checkout is from 2010-05-22. Here are some doc
strings which makes me confused:

(defun completion-table-with-predicate (table pred1 strict string pred2 action)
  "Make a completion table equivalent to TABLE but filtered through PRED1.
PRED1 is a function of one argument which returns non-nil if and only if the
argument is an element of TABLE which should be considered for completion.
STRING, PRED2, and ACTION are the usual arguments to completion tables,
as described in `try-completion', `all-completions', and `test-completion'.
If STRICT is t, the predicate always applies; if nil it only applies if
it does not reduce the set of possible completions to nothing.
Note: TABLE needs to be a proper completion table which obeys predicates."

There is no ACTION arg to try-completion etc.

(defun complete-with-action (action table string pred)
  "Perform completion ACTION.
STRING is the string to complete.
TABLE is the completion table, which should not be a function.
PRED is a completion predicate.
ACTION can be one of nil, t or `lambda'."

It looks like ACTION is tested against n and t but not against lambda.
And I think it can have other values, since the code seems to expect
something like '(boundaries ...) ...

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Lennart Borgman]

On Sun, May 30, 2010 at 4:18 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>
>> The same + tramp's symbols (fcp:, psftp:, plinkx: etc) and tramp's
>> usernames ("some.one@" etc).
>
> Not sure if that's correct, but it's a separate problem.

Yes, it seems to be separate.

> I've installed the patch below which should hopefully fix the original problem.
> Please confirm.

Thanks, it works.

bug#6299: In read-file-name: (args-out-of-range "c:" 0 3)

[Stefan Monnier]

>> I've installed the patch below which should hopefully fix the
>> original problem.  Please confirm.
> Thanks, it works.

Good thing done,

> I meant the argument ACTION to some function, but what exactly was
> I reading??
[...]
> There is no ACTION arg to try-completion etc.

The ACTION is not passed to try-completion but from try-completion to
the completion table (when it's a function).  I've just installed the
patch below to try and improve the documentation of those issues.


        Stefan


=== modified file 'doc/lispref/minibuf.texi'
--- doc/lispref/minibuf.texi	2010-05-03 22:01:23 +0000
+++ doc/lispref/minibuf.texi	2010-05-30 21:10:01 +0000
@@ -812,6 +812,24 @@
 If @var{collection} is a function, it is called with three arguments,
 the values @var{string}, @var{predicate} and @code{lambda}; whatever
 it returns, @code{test-completion} returns in turn.
+
+@defun completion-boundaries string collection predicate suffix
+This function returns the boundaries of the field on which @var{collection}
+will operate, assuming that @var{string} holds the text before point
+and @var{suffix} holds the text after point.
+
+Normally completion operates on the whole string, so for all normal
+collections, this will always return @code{(0 . (length
+@var{suffix}))}.  But more complex completion such as completion on
+files is done one field at a time.  For example, completion of
+@code{"/usr/sh"} will include @code{"/usr/share/"} but not
+@code{"/usr/share/doc"} even if @code{"/usr/share/doc"} exists.
+Also @code{all-completions} on @code{"/usr/sh"} will not include
+@code{"/usr/share/"} but only @code{"share/"}.  So if @var{string} is
+@code{"/usr/sh"} and @var{suffix} is @code{"e/doc"},
+@code{completion-boundaries} will return @code{(5 . 1)} which tells us
+that the @var{collection} will only return completion information that
+pertains to the area after @code{"/usr/"} and before @code{"/doc"}.
 @end defun
 
 If you store a completion alist in a variable, you should mark the
@@ -1618,13 +1636,14 @@
 can supply your own function to compute the completion of a given
 string.  This is called @dfn{programmed completion}.  Emacs uses
 programmed completion when completing file names (@pxref{File Name
-Completion}).
+Completion}), among many other cases.
 
-  To use this feature, pass a symbol with a function definition as the
-@var{collection} argument to @code{completing-read}.  The function
+  To use this feature, pass a function as the @var{collection}
+argument to @code{completing-read}.  The function
 @code{completing-read} arranges to pass your completion function along
-to @code{try-completion} and @code{all-completions}, which will then let
-your function do all the work.
+to @code{try-completion}, @code{all-completions}, and other basic
+completion functions, which will then let your function do all
+the work.
 
   The completion function should accept three arguments:
 
@@ -1638,10 +1657,14 @@
 and ignore the possible match if the predicate returns @code{nil}.
 
 @item
-A flag specifying the type of operation.
+A flag specifying the type of operation.  The best way to think about
+it is that the function stands for an object (in the
+``object-oriented'' sense of the word), and this third argument
+specifies which method to run.
 @end itemize
 
-  There are three flag values for three operations:
+  There are currently four methods, i.e. four flag values, one for
+  each of the four different basic operations:
 
 @itemize @bullet
 @item
@@ -1663,6 +1686,13 @@
 @code{lambda} specifies @code{test-completion}.  The completion
 function should return @code{t} if the specified string is an exact
 match for some possibility; @code{nil} otherwise.
+
+@item
+@code{(boundaries . SUFFIX)} specifies @code{completion-boundaries}.
+The function should return a value of the form @code{(boundaries
+START . END)} where START is the position of the beginning boundary in
+in the string to complete, and END is the position of the end boundary
+in SUFFIX.
 @end itemize
 
   It would be consistent and clean for completion functions to allow