bug#38024: 27.0.50; icomplete sometimes fails to show completions after backward-killing words

["Kévin Le Gouguec" <kevin.legouguec@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2697 bytes --]

Hello,

tl;dr unless I'm mistaken, sometimes `while-no-input' returns t despite
no input having arrived; this prevents `icomplete-exhibit' from
displaying completion candidates.

In more details:

This is a bit of a heisenbug; hopefully someone out there will know what
knobs to tweak to find more information about this.

From emacs -Q:

    M-x icomplete-mode RET
    ;; Find a way to set `default-directory' to some long-ish path,
    ;; e.g. with:
    M-x find-library RET icomplete RET
    C-x C-f
    ;; icomplete should show a bunch of completions.
    M-DEL
    M-DEL
    …

After each `backward-kill-word', icomplete usually displays new
completion candidates with no further user input; sometimes however, no
candidates show up until the user does something (e.g. press TAB or
start typing).

Commit 5860fd3 (Make icomplete-exhibit actually work when navigating up
directories) tried to fix this, but AFAICT it merely reduces the odds of
the bug happening: I can still reproduce it, although less often.  Also,
this new call to `redisplay' causes some noticeable flickering when
typing characters: the completions blink in and out as I fill in the
prompt.

I've tinkered with icomplete.el (commenting out the call to `redisplay'
since it makes the bug harder to trigger); the closest I got to
pin-pointing the cause for this issue was while goofing around in
`icomplete-exhibit', specifically with the `while-no-input' part:

          (let* (…
                 (text (while-no-input
                         (icomplete-completions
                          …)))
                 …)
            ;; Do nothing if while-no-input was aborted.
            (when (stringp text)
              update overlay…))

I replaced (when (stringp text) …) with (if (stringp text) (progn …)
(message "text is %s" text)): it seems that text is t when icomplete
fails to show updated candidates, which according to `while-no-input'
means that some input arrived…

However, when I tried to debug `while-no-input', I got stumped: I tried
to do some more printf-debugging[1], but all I observe is that
sometimes, the code seems to be interrupted between

	 (let ((throw-on-input ',catch-sym)
               val)
and
           (setq val (or (input-pending-p)
	                 (progn ,@body)))

I tried to find where exactly the code gets interrupted, but I could not
get a consistent answer; sometimes it seems to be before executing body,
sometimes it seems to be somewhere inside `icomplete-completions'…

What would be the best way to investigate this further?

Thank you for your time.


[1] With this patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: icomplete-debug.patch --]
[-- Type: text/x-diff, Size: 3410 bytes --]

diff --git a/lisp/icomplete.el b/lisp/icomplete.el
index 02eae55a19..31023d5644 100644
--- a/lisp/icomplete.el
+++ b/lisp/icomplete.el
@@ -399,8 +399,6 @@ icomplete-exhibit
 See `icomplete-mode' and `minibuffer-setup-hook'."
   (when (and icomplete-mode
              (icomplete-simple-completing-p)) ;Shouldn't be necessary.
-    (redisplay)     ; FIXME: why is this sometimes needed when moving
-                    ; up dirs in a file-finding table?
     (save-excursion
       (goto-char (point-max))
                                         ; Insert the match-status information:
@@ -430,13 +428,15 @@ icomplete-exhibit
                  (buffer-undo-list t)
                  deactivate-mark)
 	    ;; Do nothing if while-no-input was aborted.
-            (when (stringp text)
-              (move-overlay icomplete-overlay (point) (point) (current-buffer))
-              ;; The current C cursor code doesn't know to use the overlay's
-              ;; marker's stickiness to figure out whether to place the cursor
-              ;; before or after the string, so let's spoon-feed it the pos.
-              (put-text-property 0 1 'cursor t text)
-              (overlay-put icomplete-overlay 'after-string text)))))))
+            (if (stringp text)
+                (progn
+                  (move-overlay icomplete-overlay (point) (point) (current-buffer))
+                  ;; The current C cursor code doesn't know to use the overlay's
+                  ;; marker's stickiness to figure out whether to place the cursor
+                  ;; before or after the string, so let's spoon-feed it the pos.
+                  (put-text-property 0 1 'cursor t text)
+                  (overlay-put icomplete-overlay 'after-string text))
+              (message "text is %s" text)))))))
 
 ;;;_ > icomplete-completions (name candidates predicate require-match)
 (defun icomplete-completions (name candidates predicate require-match)
diff --git a/lisp/subr.el b/lisp/subr.el
index b408ef0931..3610fe30f2 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -3727,11 +3727,14 @@ while-no-input
   (declare (debug t) (indent 0))
   (let ((catch-sym (make-symbol "input")))
     `(with-local-quit
+       (message "after with-local-quit")
        (catch ',catch-sym
 	 (let ((throw-on-input ',catch-sym)
                val)
+           (message "after catch and let")
            (setq val (or (input-pending-p)
 	                 (progn ,@body)))
+           (message "after setq val")
            (cond
             ;; When input arrives while throw-on-input is non-nil,
             ;; kbd_buffer_store_buffered_event sets quit-flag to the
@@ -3746,13 +3749,17 @@ while-no-input
             ;; such as discarding input etc.  We return t in that case
             ;; because input did arrive during execution of BODY.
             ((eq quit-flag throw-on-input)
+             (message "cond: got input")
              (setq quit-flag nil)
              t)
             ;; This is for when the user actually QUITs during
             ;; execution of BODY.
             (quit-flag
+             (message "cond: got quit-flag %s" quit-flag)
              nil)
-            (t val)))))))
+            (t
+             (message "cond: return val")
+             val)))))))
 
 (defmacro condition-case-unless-debug (var bodyform &rest handlers)
   "Like `condition-case' except that it does not prevent debugging.

[-- Attachment #3: Type: text/plain, Size: 1056 bytes --]


When the bug happens, I get the following messages:

    after with-local-quit
    after catch and let
    text is t


PS: It just occured to me that I should search debbugs for
    "while-no-input" in addition to "icomplete".  I just did that;
    bug#15042 looks like it could maybe be related?


In GNU Emacs 27.0.50 (build 6, i686-pc-linux-gnu, GTK+ Version 3.24.5, cairo version 1.16.0)
 of 2019-10-21 built on little-buster
Repository revision: 61fb5214816ef3d57e676d900e499ffcd079a1f9
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux 10 (buster)

Configured using:
 'configure --with-xwidgets --with-cairo'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS
LIBSYSTEMD JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix