bug#42101: icomplete-fido-ret doesn't always use minibuffer-default when input is empty

bug#42101: icomplete-fido-ret doesn't always use minibuffer-default when input is empty

[Andrew Schwartzmeyer]

Hi,

Funny bug, haven’t figured out the cause. It may be hard to repro.

If you have some amount of minibuffer history, you can cause icomplete-fido-ret to not accept the minibuffer-default. An example is having history such that 'C-h v' when point is on ‘completion-styles’ causes ‘completion-styles-alist’ to be the first history element. Since the minibuffer-default shows ‘completion-styles’ (being that it’s under point, and I’ve not entered any text into the minibuffer), I’d expect RET to choose ‘completion-styles’, but instead it choses ‘completion-styles-alist’ (the head of the presented candidates, ‘completion-styles’ is instead the second candidate).

This repros in fido-mode, but not icomplete-mode. I took a look at the relevant functions called on RET for these modes but am having trouble finding the issue.

I think the root cause is that if minibuffer-default matches a candidate exactly, that candidate should always become the first history element, but this isn’t happening. I’m not sure why; I played around with different completion-styles but that had no effect. Regardless, if there is no input but there is minibuffer-default, various code in icomplete.el suggests that 'minibuffer-complete-and-exit’ should just be called, I don’t know why it’s not happening.

Any ideas?

Thanks,

Andy

bug#42101:

[Andy Schwartzmeyer]

[-- Attachment #1: Type: text/plain, Size: 5027 bytes --]

Okay, with some message style debugging I think I narrowed it down:

(defun icomplete-force-complete-and-exit ()
  "Complete the minibuffer with the longest possible match and exit.
Use the first of the matches if there are any displayed, and use
the default otherwise."
  (interactive)
  ;; This function is tricky.  The mandate is to "force", meaning we
  ;; should take the first possible valid completion for the input.
  ;; However, if there is no input and we can prove that that
  ;; coincides with the default, it is much faster to just call
  ;; `minibuffer-complete-and-exit'.  Otherwise, we have to call
  ;; `minibuffer-force-complete-and-exit', which needs the full
  ;; completion set and is potentially slow and blocking.  Do the
  ;; latter if:
  (if (or
       ;; there's some input, meaning the default in off the table by
       ;; definition; OR
       (> (icomplete--field-end) (icomplete--field-beg))
       ;; there's no input, but there's also no minibuffer default
       ;; (and the user really wants to see completions on no input,
       ;; meaning he expects a "force" to be at least attempted); OR
       (and (not minibuffer-default)
            icomplete-show-matches-on-no-input)
       ;; there's no input but the full completion set has been
       ;; calculated, This causes the first cached completion to
       ;; be taken (i.e. the one that the user sees highlighted)
       completion-all-sorted-completions)
      (progn
        (message (if completion-all-sorted-completions (message "completions t") (message "completions nil")))
        (message (if minibuffer-default "default t" "default nil"))
        (message "FORCING")
        (minibuffer-force-complete-and-exit))
    ;; Otherwise take the faster route...
    (minibuffer-complete-and-exit)))

With fido-mode, the final test in the or clause, namely 'completion-all-sorted-completions’, is t, whereas in icomplete-mode it is nil. This is causing the first cached completion to be taken (i.e. the ‘completion-styles-alist’ which probably shouldn’t be the first completion anyway). Moreover, it means that this function’s existence is being overriden: it’s never shortcutting to 'minibuffer-complete-and-exit’ for fido-mode users.

Now to find out two more things:

1. Why is the completion list not bubbling minibuffer-default to the top? (I’m using helpful-variable, but this also happens with describe-variable, and I’ve noticed it with other functions that will use a default from point.)
2. Why is 'completion-all-sorted-completions’ always t when using fido-mode?

At least for #1:

In this bit of code the minibuffer-default is compared with equal and with string-prefix-p:

(defun icomplete--sorted-completions ()
 (or completion-all-sorted-completions
     (cl-loop
      with beg = (icomplete--field-beg)
      with end = (icomplete--field-end)
      with all = (completion-all-sorted-completions beg end)
      for fn in (cond ((and minibuffer-default
                            (stringp minibuffer-default) ; bug#38992
                            (= (icomplete--field-end) (icomplete--field-beg)))
                       ;; When we have a non-nil string default and
                       ;; no input whatsoever: we want to make sure
                       ;; that default is bubbled to the top so that
                       ;; `icomplete-force-complete-and-exit' will
                       ;; select it (do that even if the match
                       ;; doesn't match the completion perfectly.
                       `(,(lambda (comp)
                            (equal minibuffer-default comp))
                         ,(lambda (comp)
                            (string-prefix-p minibuffer-default comp))))
                      ((and fido-mode
                            (not minibuffer-default)
                            (eq (icomplete--category) 'file))
                       ;; `fido-mode' has some extra file-sorting
                       ;; semantics even if there isn't a default,
                       ;; which is to bubble "./" to the top if it
                       ;; exists.  This makes M-x dired RET RET go to
                       ;; the directory of current file, which is
                       ;; what vanilla Emacs and `ido-mode' both do.
                       `(,(lambda (comp)
                            (string= "./" comp)))))
      thereis (cl-loop
               for l on all
               while (consp (cdr l))
               for comp = (cadr l)
               when (funcall fn comp)
               do (setf (cdr l) (cddr l))
               and return
               (completion--cache-all-sorted-completions beg end (cons comp all)))
      finally return all)))

I’m not sure how to get it to do so, but when there are candidates satisfying both ‘equals’ and ‘string-prefix-p’, we need to get this to prefer the former over the latter.

Cheers,

Andy

[-- Attachment #2: Type: text/html, Size: 12290 bytes --]

bug#42101: Acknowledgement (icomplete-fido-ret doesn't always use minibuffer-default when input is empty)

[Andrew Schwartzmeyer]

Hi João, here’s that CC to get into the bug report. I’m sorry I don’t know how debbugs works! I had to actually disable my email provider’s TLS sending guarantee to send this email…maybe one day we’ll see GNU Emacs on GitHub or something :)

Thanks,

Andy

> On Jun 27, 2020, at 11:00 PM, GNU bug Tracking System <help-debbugs@gnu.org> wrote:
> 
> Thank you for filing a new bug report with debbugs.gnu.org.
> 
> This is an automatically generated reply to let you know your message
> has been received.
> 
> Your message is being forwarded to the package maintainers and other
> interested parties for their attention; they will reply in due course.
> 
> Your message has been sent to the package maintainer(s):
> bug-gnu-emacs@gnu.org
> 
> If you wish to submit further information on this problem, please
> send it to 42101@debbugs.gnu.org.
> 
> Please do not send mail to help-debbugs@gnu.org unless you wish
> to report a problem with the Bug-tracking system.
> 
> -- 
> 42101: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=42101
> GNU Bug Tracking System
> Contact help-debbugs@gnu.org with problems

bug#42101: Acknowledgement (icomplete-fido-ret doesn't always use minibuffer-default when input is empty)

[João Távora]

Andrew Schwartzmeyer <andrew@schwartzmeyer.com> writes:

> Hi João, here’s that CC to get into the bug report. I’m sorry I don’t
> know how debbugs works! I had to actually disable my email provider’s
> TLS sending guarantee to send this email…

This is very odd.  it should be as simple as sending an email message to
to 42101@debbugs.gnu.org, in this case.  No idea if that violate any
"TLS sending guarantees".

> maybe one day we’ll see GNU Emacs on GitHub or something :)

I lean towards the "or something".

As to the actual bug, I'll try to look at it soon.

João

bug#42101: icomplete-fido-ret doesn't always use minibuffer-default when input is empty

[João Távora]

Andrew Schwartzmeyer <andrew@schwartzmeyer.com> writes:

> Hi,
>
> Funny bug, haven’t figured out the cause. It may be hard to repro.
>
> If you have some amount of minibuffer history, you can cause
> icomplete-fido-ret to not accept the minibuffer-default. An example is
> having history such that 'C-h v' when point is on ‘completion-styles’
> causes ‘completion-styles-alist’ to be the first history
> element. Since the minibuffer-default shows ‘completion-styles’ (being
> that it’s under point, and I’ve not entered any text into the
> minibuffer), I’d expect RET to choose ‘completion-styles’, but instead
> it choses ‘completion-styles-alist’ (the head of the presented
> candidates, ‘completion-styles’ is instead the second candidate).
>
> This repros in fido-mode, but not icomplete-mode. I took a look at the
> relevant functions called on RET for these modes but am having trouble
> finding the issue.

I can't reproduce this.  I think the first step is for you to try to
identify which sequence of actions as performed from an Emacs
session started with -Q lead to the problem.  For the record, I tried

Emacs -Q (master of around two weeks ago)

  C-h v completion-styles RET
  C-h v completion-styles-alist RET

type "completion-styles" in *scratch* buffer.  With cursor on word.

  C-h v RET (quickly, before the completions appear)
  C-h v RET (slowly, letting the completions appear)

Both approaches lead to the variable `completion-styles` being
described.

João

bug#42101: Acknowledgement (icomplete-fido-ret doesn't always use minibuffer-default when input is empty)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > maybe one day we’ll see GNU Emacs on GitHub or something :)

  > I lean towards the "or something".

Here's why it can't be Github:

https://www.gnu.org/software/repo-criteria-evaluation.html.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#42101: icomplete-fido-ret doesn't always use minibuffer-default when input is empty

[Andrew Schwartzmeyer]

[-- Attachment #1: Type: text/plain, Size: 3386 bytes --]

I’m sorry so João, I had sent you a repro last week, but it again didn’t go through.

It appears I have to leave TLS-sending guarantees disabled because the debbugs mail server is insecurely configured (for others who may be reading but offering no useful input, this is a security issue that does not exist with GitHub or other similar software). Perhaps we should move to emacs-devel which at least accepts email sent over TLS.

Anyway, I can repro this in emacs -Q with:

(defun fido-mode+ ()
  (setq-local completion-styles '(basic)))
(add-hook 'icomplete-minibuffer-setup-hook #'fido-mode+)
(fido-mode)

Basically (heh) this happens if the “basic” completion-style is used (or “partial-completion” or “substring” or any combination including one of those styles).

It only doesn’t repro if “flex” is used by itself (the default for fido-mode, but should be able to be overridden).

This also repros with icomplete-mode if “icomplete-show-matches-on-no-input” is t, as in:

(defun fido-mode+ ()
  (setq-local completion-styles '(basic)
              icomplete-show-matches-on-no-input t))
(add-hook 'icomplete-minibuffer-setup-hook #'fido-mode+)
(icomplete-mode)

Which explains why it’s readily apparent in fido-mode, where that’s set to t already. Unfortunately, I still have no clue how to fix it. I would appreciate your help! Being able to use the default styles in addition to flex makes the fido-mode experience smoother, because they return defaults for no input much faster, I find them to be more predictable, and it’s when they fail that flex tends to shine most.

Thanks,

Andy

> On Jun 29, 2020, at 7:00 AM, João Távora <joaotavora@gmail.com> wrote:
> 
> Andrew Schwartzmeyer <andrew@schwartzmeyer.com <mailto:andrew@schwartzmeyer.com>> writes:
> 
>> Hi,
>> 
>> Funny bug, haven’t figured out the cause. It may be hard to repro.
>> 
>> If you have some amount of minibuffer history, you can cause
>> icomplete-fido-ret to not accept the minibuffer-default. An example is
>> having history such that 'C-h v' when point is on ‘completion-styles’
>> causes ‘completion-styles-alist’ to be the first history
>> element. Since the minibuffer-default shows ‘completion-styles’ (being
>> that it’s under point, and I’ve not entered any text into the
>> minibuffer), I’d expect RET to choose ‘completion-styles’, but instead
>> it choses ‘completion-styles-alist’ (the head of the presented
>> candidates, ‘completion-styles’ is instead the second candidate).
>> 
>> This repros in fido-mode, but not icomplete-mode. I took a look at the
>> relevant functions called on RET for these modes but am having trouble
>> finding the issue.
> 
> I can't reproduce this.  I think the first step is for you to try to
> identify which sequence of actions as performed from an Emacs
> session started with -Q lead to the problem.  For the record, I tried
> 
> Emacs -Q (master of around two weeks ago)
> 
>  C-h v completion-styles RET
>  C-h v completion-styles-alist RET
> 
> type "completion-styles" in *scratch* buffer.  With cursor on word.
> 
>  C-h v RET (quickly, before the completions appear)
>  C-h v RET (slowly, letting the completions appear)
> 
> Both approaches lead to the variable `completion-styles` being
> described.
> 
> João


[-- Attachment #2: Type: text/html, Size: 17433 bytes --]

bug#42101: icomplete-fido-ret doesn't always use minibuffer-default when input is empty

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 272 bytes --]

Schwartzmeyer <andrew@schwartzmeyer.com> wrote

> Any clue?

I'm sorry Andrew haven't had time to look into it yet, but your analyses
help.  Just note that Fido-mode is indeed geared primarily to working with
flex. But it should be customizable of course.

João

[-- Attachment #2: Type: text/html, Size: 548 bytes --]

bug#42101: icomplete-fido-ret doesn't always use minibuffer-default when input is empty

[Andrew Schwartzmeyer]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

This bug can be fixed by deleting the expression:

  ,(lambda (comp)
     (string-prefix-p minibuffer-default comp))

from icomplete--sorted-completions.

But it’s obviously there for some reason, so idk.

> On Jul 4, 2020, at 2:35 AM, João Távora <joaotavora@gmail.com> wrote:
> 
> Schwartzmeyer <andrew@schwartzmeyer.com <mailto:andrew@schwartzmeyer.com>> wrote
> 
> > Any clue?
> 
> I'm sorry Andrew haven't had time to look into it yet, but your analyses help.  Just note that Fido-mode is indeed geared primarily to working with flex. But it should be customizable of course.
> 
> João


[-- Attachment #2: Type: text/html, Size: 1589 bytes --]