bug#52169: 28.0.60; "Args out of range" when completing shell command

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Kévin Le Gouguec]

[-- Attachment #1: Type: text/plain, Size: 783 bytes --]

Hello,

This is sort of a garbage-in, garbage-out situation, so I'll just let
the reproducer speak for itself:

$ emacs -Q
M-! ./nosuchcommand C-b TAB

- In Emacs 27.1, this simply says "[No match]".
- In Emacs 28.0.60, this errors out:

> completion--some: Args out of range: "./nosuchcomma", 0, 14

I'm attaching the backtrace obtained with M-x toggle-debug-on-error.

I'm not on the most ideal setup for compiling ATM, so I apologize for
not bisecting this (and for not trying with 27.2; Debian stable only has
27.1); I'll get to it eventually if no-one beats me.

AFAICT it's not a very impactful issue; however, since Emacs now throws
an error where it didn't use to, I figure there's a somewhat subtle bug
here with consequences I can't quite evaluate.

Thanks for your time.


[-- Attachment #2: args-out-of-range.bt --]
[-- Type: text/plain, Size: 5844 bytes --]

Debugger entered--Lisp error: (args-out-of-range "./nosuchcomma" 0 14)
  completion-pcm--find-all-completions("./nosuchcomma" #f(compiled-function (string pred action) #<bytecode 0x1e249474c53047ad>) nil 13 identity)
  completion-pcm--find-all-completions("./nosuchcomman" #f(compiled-function (string pred action) #<bytecode 0x1e249474c53047ad>) nil 14 nil)
  completion-pcm-try-completion("./nosuchcomman" #f(compiled-function (string pred action) #<bytecode 0x1e249474c53047ad>) nil 14)
  #f(compiled-function (style) #<bytecode 0x5c87d8d40fdf7fb>)(partial-completion)
  completion--some(#f(compiled-function (style) #<bytecode 0x5c87d8d40fdf7fb>) (basic partial-completion emacs22))
  completion--nth-completion(1 "./nosuchcomman" #f(compiled-function (string pred action) #<bytecode -0x54f4538c251c479>) nil 14 (metadata (cycle-sort-function . #f(compiled-function (comps) #<bytecode -0xecaa4cb658defe7>)) (category . file) (completion--unquote-requote . t)))
  completion-try-completion("./nosuchcomman" #f(compiled-function (string pred action) #<bytecode -0x54f4538c251c479>) nil 14 (metadata (cycle-sort-function . #f(compiled-function (comps) #<bytecode -0xecaa4cb658defe7>)) (category . file) (completion--unquote-requote . t)))
  completion--do-completion(#<marker at 16 in  *Minibuf-1*> 30)
  completion--in-region-1(#<marker at 16 in  *Minibuf-1*> 30)
  #f(compiled-function (start end collection predicate) #<bytecode 0x14e6876b72a3496e>)(#<marker at 16 in  *Minibuf-1*> 30 #f(compiled-function (string pred action) #<bytecode -0x54f4538c251c479>) nil)
  apply(#f(compiled-function (start end collection predicate) #<bytecode 0x14e6876b72a3496e>) (#<marker at 16 in  *Minibuf-1*> 30 #f(compiled-function (string pred action) #<bytecode -0x54f4538c251c479>) nil))
  #f(compiled-function (funs global args) #<bytecode -0xf9287ef545e722>)(nil nil (#<marker at 16 in  *Minibuf-1*> 30 #f(compiled-function (string pred action) #<bytecode -0x54f4538c251c479>) nil))
  completion--in-region(#<marker at 16 in  *Minibuf-1*> 30 #f(compiled-function (string pred action) #<bytecode -0x54f4538c251c479>) nil)
  completion-in-region(#<marker at 16 in  *Minibuf-1*> 30 #f(compiled-function (string pred action) #<bytecode -0x54f4538c251c479>) nil)
  completion-at-point()
  funcall-interactively(completion-at-point)
  call-interactively(completion-at-point nil nil)
  command-execute(completion-at-point)
  read-from-minibuffer("Shell command: " nil (keymap (9 . completion-at-point) keymap (menu-bar keymap (minibuf "Minibuf" keymap (previous menu-item "Previous History Item" previous-history-element :help "Put previous minibuffer history element in the min...") (next menu-item "Next History Item" next-history-element :help "Put next minibuffer history element in the minibuf...") (isearch-backward menu-item "Isearch History Backward" isearch-backward :help "Incrementally search minibuffer history backward") (isearch-forward menu-item "Isearch History Forward" isearch-forward :help "Incrementally search minibuffer history forward") (return menu-item "Enter" exit-minibuffer :key-sequence "\15" :help "Terminate input and exit minibuffer") (quit menu-item "Quit" abort-recursive-edit :help "Abort input and exit minibuffer") "Minibuf")) (13 . exit-minibuffer) (10 . exit-minibuffer) (7 . abort-minibuffers) (C-tab . file-cache-minibuffer-complete) (9 . self-insert-command) (XF86Back . previous-history-element) (up . previous-line-or-history-element) (prior . previous-history-element) (XF86Forward . next-history-element) (down . next-line-or-history-element) (next . next-history-element) (27 keymap (60 . minibuffer-beginning-of-buffer) (114 . previous-matching-history-element) (115 . next-matching-history-element) (112 . previous-history-element) (110 . next-history-element))) nil shell-command-history nil)
  apply(read-from-minibuffer "Shell command: " nil (keymap (9 . completion-at-point) keymap (menu-bar keymap (minibuf "Minibuf" keymap (previous menu-item "Previous History Item" previous-history-element :help "Put previous minibuffer history element in the min...") (next menu-item "Next History Item" next-history-element :help "Put next minibuffer history element in the minibuf...") (isearch-backward menu-item "Isearch History Backward" isearch-backward :help "Incrementally search minibuffer history backward") (isearch-forward menu-item "Isearch History Forward" isearch-forward :help "Incrementally search minibuffer history forward") (return menu-item "Enter" exit-minibuffer :key-sequence "\15" :help "Terminate input and exit minibuffer") (quit menu-item "Quit" abort-recursive-edit :help "Abort input and exit minibuffer") "Minibuf")) (13 . exit-minibuffer) (10 . exit-minibuffer) (7 . abort-minibuffers) (C-tab . file-cache-minibuffer-complete) (9 . self-insert-command) (XF86Back . previous-history-element) (up . previous-line-or-history-element) (prior . previous-history-element) (XF86Forward . next-history-element) (down . next-line-or-history-element) (next . next-history-element) (27 keymap (60 . minibuffer-beginning-of-buffer) (114 . previous-matching-history-element) (115 . next-matching-history-element) (112 . previous-history-element) (110 . next-history-element))) nil shell-command-history nil)
  read-shell-command("Shell command: " nil nil nil)
  byte-code("\306\10\203\16\0\307\310\311\11!\"\202\17\0\312\313\211\n\206\37\0\13\314=\205\37\0\315\313\316\"\211\205&\0\317\1!\262\1$\f\15E\207" [shell-command-prompt-show-cwd default-directory buffer-file-name major-mode current-prefix-arg shell-command-default-error-buffer read-shell-command format-message "Shell command in `%s': " abbreviate-file-name "Shell command: " nil dired-mode dired-get-filename t file-relative-name] 7)
  call-interactively(shell-command nil nil)
  command-execute(shell-command)

[-- Attachment #3: Type: text/plain, Size: 810 bytes --]




In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo version 1.16.0)
 of 2021-11-26 built on hirondell
Repository revision: 11860f89a593a8cfe7efb94e86370bbbe4318fba
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure --with-xwidgets --with-cairo --with-gconf --with-xinput2'

Configured features:
ACL CAIRO DBUS FREETYPE GCONF GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ
JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM XWIDGETS GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Lars Ingebrigtsen]

Kévin Le Gouguec <kevin.legouguec@gmail.com> writes:

> $ emacs -Q
> M-! ./nosuchcommand C-b TAB
>
> - In Emacs 27.1, this simply says "[No match]".
> - In Emacs 28.0.60, this errors out:
>
>> completion--some: Args out of range: "./nosuchcomma", 0, 14

As far as I can tell, this comes from:

(defun completion-basic-try-completion (string table pred point)

[...]

      (let* ((suffix (substring afterpoint (cdr bounds)))
             (prefix (substring beforepoint 0 (car bounds)))

But the length of beforepoint (i.e., "./nosuchcomma") is 13, while (car
bounds) is 14.

I'm not sure how this all connects up, so I've added Stefan to the CCs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I'm not sure how this all connects up, so I've added Stefan to the CCs.

I'll try and look at it ASAP,


        Stefan

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> $ emacs -Q
>> M-! ./nosuchcommand C-b TAB
>>
>> - In Emacs 27.1, this simply says "[No match]".
>> - In Emacs 28.0.60, this errors out:
>>
>> > completion--some: Args out of range: "./nosuchcomma", 0, 14

Yuck!

This is one of those cases where the mismatch between the assumptions of
pcomplete and the featureset of minibuffer.el clash: `pcomplete-stub`
tells us we're completing "./nosuchcommand" but without telling us where
that string comes from in the buffer, so we fail to find
"./nosuchcommand" just before point (because the "d" is *after* point)
and thus fall back on the worst-case handler which doesn't behave well
at all here.

I pushed to `master` a fix for the immediate cause of the `Args out of
range` error, which seems to brings us back to Emacs-27's poor behavior.

Maybe it's good enough for `emacs-28` as well, but I have to think about
this problem a bit more.


        Stefan

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Kévin Le Gouguec]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I pushed to `master` a fix for the immediate cause of the `Args out of
> range` error, which seems to brings us back to Emacs-27's poor behavior.

Can confirm, thanks!

> Maybe it's good enough for `emacs-28` as well, but I have to think about
> this problem a bit more.

OK; leaving this open for the time being then.

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Kévin Le Gouguec [2021-12-04 18:16:50] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I pushed to `master` a fix for the immediate cause of the `Args out of
>> range` error, which seems to brings us back to Emacs-27's poor behavior.
> Can confirm, thanks!

Thanks.

>> Maybe it's good enough for `emacs-28` as well, but I have to think about
>> this problem a bit more.
> OK; leaving this open for the time being then.

Hmm, I can't think of a good way to make it work correctly without
significant changes to `pcomplete.el` and its API.

For `emacs-28`, I propose the patch below, which is a slightly improved
version of the patch currently on `master`.  As before, it won't fix the
real problem, but will avoid "Args out of range" and revert to the
behavior we had in Emacs-27 (which was/is broken, but tends
to be milder for the user).

Any objection?


        Stefan


diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index ca82b4a9e60..21d610fdf44 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -283,8 +283,9 @@ completion-table-subvert
          ((eq (car-safe action) 'boundaries)
           (let ((beg (or (and (eq (car-safe res) 'boundaries) (cadr res)) 0)))
             `(boundaries
-              ,(max (length s1)
-                    (+ beg (- (length s1) (length s2))))
+              ,(min (length string)
+                    (max (length s1)
+                         (+ beg (- (length s1) (length s2)))))
               . ,(and (eq (car-safe res) 'boundaries) (cddr res)))))
          ((stringp res)
           (if (string-prefix-p s2 res completion-ignore-case)

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> For `emacs-28`, I propose the patch below, which is a slightly improved
> version of the patch currently on `master`.  As before, it won't fix the
> real problem, but will avoid "Args out of range" and revert to the
> behavior we had in Emacs-27 (which was/is broken, but tends
> to be milder for the user).

Sounds good to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Eli Zaretskii]

> Cc: 52169@debbugs.gnu.org,
>  Kévin Le Gouguec <kevin.legouguec@gmail.com>
> Date: Sun, 05 Dec 2021 23:24:13 -0500
> From:  Stefan Monnier via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> For `emacs-28`, I propose the patch below, which is a slightly improved
> version of the patch currently on `master`.  As before, it won't fix the
> real problem, but will avoid "Args out of range" and revert to the
> behavior we had in Emacs-27 (which was/is broken, but tends
> to be milder for the user).
> 
> Any objection?

Not from me, thanks.

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Any objection?
> Not from me, thanks.

Pushed, thanks,


        Stefan

bug#52169: 28.0.60; "Args out of range" when completing shell command

[Kévin Le Gouguec]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

Kévin Le Gouguec <kevin.legouguec@gmail.com> writes:

> Hello,
>
> This is sort of a garbage-in, garbage-out situation, so I'll just let
> the reproducer speak for itself:
>
> $ emacs -Q
> M-! ./nosuchcommand C-b TAB
>
> - In Emacs 27.1, this simply says "[No match]".
> - In Emacs 28.0.60, this errors out:
>
>> completion--some: Args out of range: "./nosuchcomma", 0, 14
>
> I'm attaching the backtrace obtained with M-x toggle-debug-on-error.
>
> I'm not on the most ideal setup for compiling ATM, so I apologize for
> not bisecting this (and for not trying with 27.2; Debian stable only has
> 27.1); I'll get to it eventually if no-one beats me.

Alright, if I didn't botch the bisection, then we're looking at
2021-04-23 "Fix small bug in 'completion-table-subvert'." (3806d2168b).
Bisection session attached, FWIW.


[-- Attachment #2: Type: text/plain, Size: 1333 bytes --]

Bisect Log (11)
git bisect start '6dabbddb5f' 'b702225619'
6dabbddb5f bad Merge from origin/emacs-27
b702225619 good Merge from origin/emacs-27

git bisect good 3c051db646b34995c144327ed462a92ffbb41f86
3c051db646 good * lisp/files-x.el: Use lexical-binding

git bisect good c7555f037ae7194fc331fb65c34c280680028b80
c7555f037a good Fix :scale's affect on :width and :height (bug#47819)

git bisect good aaf6b6bf80805d18180e1c16350e9473716f5915
aaf6b6bf80 good Ensure that argument to 'verify' is a constant expression.

git bisect bad 21b3ceea3b0321559e9cbdcb034d9c66d3264d89
21b3ceea3b bad ; * etc/NEWS: Fix last change.

git bisect bad 3995f0515ab0b0e359f9976d589eed45f38eaa6d
3995f0515a bad Expand documentation on nontrivial completion boundaries.

git bisect good 522c34f0e80f60969861b0dc34bc7105249f9994
522c34f0e8 good Remove the binding for xref-quit-and-goto-xref

git bisect good 344f769491a84b6d47ee3722054b214167572219
344f769491 good Add support for using a TLS client certificate with 'erc-tls' (bug#47788)

git bisect good 804f5acda4bd947f058c6843e373797202314467
804f5acda4 good Revert "Remove the binding for xref-quit-and-goto-xref"

git bisect bad 3806d2168bdc36796d2dfac5f743582f0ce71c72
3806d2168b bad Fix small bug in 'completion-table-subvert'.

3806d2168bdc36796d2dfac5f743582f0ce71c72 is the first bad commit