bug#21747: 25.0.50; while-no-input breaks kbd event handling when called from post-command-hook

[Tassilo Horn <tsdh@gnu.org>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tassilo Horn <tsdh@gnu.org>
>> Cc: bruce.connor.am@gmail.com, monnier@iro.umontreal.ca, storm@cua.dk,
>> 21747@debbugs.gnu.org
>> Date: Sat, 24 Oct 2015 15:30:11 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> My recipe for reproduction is
>> >> 
>> >>   1. emacs -Q
>> >>   2. M-x package-initialize
>> >>   3. M-x global-aggressive-indent-mode
>> >>   4. C-x C-f ~/test.sh
>> >>   5. type the keyword if where the i is displayed immediately
>> >>      and the display of f is delayed
>> >> 
>> >> This does only occur for sh-mode keywords like if, while, etc. where the
>> >> delaying starts with after entering the last char of the keyword.  Also,
>> >> I can reproduce that problem only if test.sh doesn't exist.
>> >
>> > I don't understand: are you saying that the post-command-hook finished
>> > its job, and yet redisplay is not entered?
>> 
>> Yes, or rather the function aggressive-indent--indent-if-changed which
>> is in post-command-hook finished.
>> 
>> >> Or well, I just tried what happens when I replace the `while-no-input'
>> >> with a `progn'.  Then Emacs goes into some infloop.  Attaching with gdb
>> >> shows:
>> >
>> > There's a procedure in etc/DEBUG to determine which call-stack frame
>> > infloops, please use it and tell what you found.
>> 
>> Ok, so with the aggressive-indent--indent-if-changed where
>> while-no-input is replaced with progn, I perform my recipe until emacs
>> infloops.  Then do "kill -TSTP <PID>" and repeatedly "finish" at the gdb
>> prompt.  But the last frame being displayed in GDB before finish doesn't
>> return anymore is not always the same.  Most of the time it is poll ()
>> from /usr/lib/libc.so.6.  Here are two other results.
>
> No, that's bogus (GC cannot infloop, you just didn't wait long enough
> for that "finish" to return).

Oh, sorry.

> The function that infloops is re-search-backward, because it is called
> with LIMIT set to zero.  The real problem is here:
>
>   (defun sh-smie--keyword-p ()
>     "Non-nil if we're at a keyword position.
>   A keyword position is one where if we're looking at something that looks
>   like a keyword, then it is a keyword."
>     (let ((prev (funcall smie-backward-token-function)))
>       (if (zerop (length prev))
> 	  (looking-back "\\`\\|\\s(" (1- (point)))  <<<<<<<<<<<<<<<<<<
> 	(assoc prev smie-grammar))))
>
> What do you expect looking-back to do here when point is at BOB?

I think when point is a BOB, then the \\` part of the regex would match
for sure so replacing looking-back with (or (bobp) (looking-back ...))
would have the same semantics, right?  So I've tried this patch:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index fbb4a90..2e708f7 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1774,7 +1774,8 @@ sh-smie--keyword-p
 like a keyword, then it is a keyword."
   (let ((prev (funcall smie-backward-token-function)))
     (if (zerop (length prev))
-        (looking-back "\\`\\|\\s(" (1- (point)))
+        (or (bobp)
+            (looking-back "\\`\\|\\s(" (1- (point))))
       (assoc prev smie-grammar))))
 
 (defun sh-smie--newline-semi-p (&optional tok)
@@ -1788,7 +1789,8 @@ sh-smie--newline-semi-p
     (unless tok
       (setq tok (funcall smie-backward-token-function)))
     (if (and (zerop (length tok))
-             (looking-back "\\s(" (1- (point))))
+             (or (bobp)
+                 (looking-back "\\s(" (1- (point)))))
         nil
       (not (numberp (nth 2 (assoc tok smie-grammar)))))))
 
@@ -2172,7 +2174,7 @@ sh-smie--rc-after-special-arg-p
   "Check if we're after the first arg of an if/while/for/... construct.
 Returns the construct's token and moves point before it, if so."
   (forward-comment (- (point)))
-  (when (looking-back ")\\|\\_<not" (- (point) 3))
+  (when (looking-back ")\\|\\_<not" (max (point-min) (- (point) 3)))
     (ignore-errors
       (let ((forward-sexp-function nil))
         (forward-sexp -1)
--8<---------------cut here---------------end--------------->8---

But that still loops.  When tracing all aggressive-indent and sh-smie
functions, that's the problematic call stack, i.e., the case where
(sh-smie-sh-forward-token) returns ";".

--8<---------------cut here---------------start------------->8---
1 -> (aggressive-indent--indent-if-changed)
| 2 -> (aggressive-indent--run-user-hooks)
| 2 <- aggressive-indent--run-user-hooks: nil
| 2 -> (aggressive-indent--softly-indent-region-and-on 2 3)
| | 3 -> (aggressive-indent-indent-region-and-on 2 3)
| | | 4 -> (sh-smie--indent-continuation)
| | | | 5 -> (sh-smie--looking-back-at-continuation-p)
| | | | 5 <- sh-smie--looking-back-at-continuation-p: nil
| | | 4 <- sh-smie--indent-continuation: nil
| | | 4 -> (sh-smie-sh-forward-token)
| | | | 5 -> (sh-smie--default-forward-token)
| | | | 5 <- sh-smie--default-forward-token: "if"
| | | | 5 -> (sh-smie--sh-keyword-p "if")
| | | | | 6 -> (sh-smie--keyword-p)
| | | | | | 7 -> (sh-smie-sh-backward-token)
| | | | | | | 8 -> (sh-smie--default-backward-token)
| | | | | | | 8 <- sh-smie--default-backward-token: ""
| | | | | | 7 <- sh-smie-sh-backward-token: ""
| | | | | 6 <- sh-smie--keyword-p: t
| | | | 5 <- sh-smie--sh-keyword-p: t
| | | 4 <- sh-smie-sh-forward-token: "if"
| | | 4 -> (sh-smie-sh-forward-token)
| | | | 5 -> (sh-smie--newline-semi-p)
| | | | | 6 -> (sh-smie-sh-backward-token)
| | | | | | 7 -> (sh-smie--default-backward-token)
| | | | | | 7 <- sh-smie--default-backward-token: "if"
| | | | | | 7 -> (sh-smie--sh-keyword-p "if")
| | | | | | | 8 -> (sh-smie--keyword-p)
| | | | | | | | 9 -> (sh-smie-sh-backward-token)
| | | | | | | | | 10 -> (sh-smie--default-backward-token)
| | | | | | | | | 10 <- sh-smie--default-backward-token: ""
| | | | | | | | 9 <- sh-smie-sh-backward-token: ""
| | | | | | | 8 <- sh-smie--keyword-p: t
| | | | | | 7 <- sh-smie--sh-keyword-p: t
| | | | | 6 <- sh-smie-sh-backward-token: "if"
| | | | 5 <- sh-smie--newline-semi-p: nil
| | | 4 <- sh-smie-sh-forward-token: ";"
| | | 4 -> (sh-smie-sh-forward-token)
| | | | 5 -> (sh-smie--newline-semi-p)
| | | | | 6 -> (sh-smie-sh-backward-token)
| | | | | | 7 -> (sh-smie--default-backward-token)
| | | | | | 7 <- sh-smie--default-backward-token: "if"
| | | | | | 7 -> (sh-smie--sh-keyword-p "if")
| | | | | | | 8 -> (sh-smie--keyword-p)
| | | | | | | | 9 -> (sh-smie-sh-backward-token)
| | | | | | | | | 10 -> (sh-smie--default-backward-token)
| | | | | | | | | 10 <- sh-smie--default-backward-token: ""
| | | | | | | | 9 <- sh-smie-sh-backward-token: ""
| | | | | | | 8 <- sh-smie--keyword-p: t
| | | | | | 7 <- sh-smie--sh-keyword-p: t
| | | | | 6 <- sh-smie-sh-backward-token: "if"
| | | | 5 <- sh-smie--newline-semi-p: nil
| | | 4 <- sh-smie-sh-forward-token: ";"

[... gazillion of times...]

| | | 4 -> (sh-smie-sh-forward-token)
| | | | 5 -> (sh-smie--newline-semi-p)
| | | | | 6 -> (sh-smie-sh-backward-token)
| | | | | | 7 -> (sh-smie--default-backward-token)
| | | | | | 7 <- sh-smie--default-backward-token: "if"
| | | | | | 7 -> (sh-smie--sh-keyword-p "if")
| | | | | | | 8 -> (sh-smie--keyword-p)
| | | | | | | | 9 -> (sh-smie-sh-backward-token)
| | | | | | | | | 10 -> (sh-smie--default-backward-token)
| | | | | | | | 9 <- sh-smie-sh-backward-token: !non-local\ exit!
| | | | | | | 8 <- sh-smie--keyword-p: !non-local\ exit!
| | | | | | 7 <- sh-smie--sh-keyword-p: !non-local\ exit!
| | | | | 6 <- sh-smie-sh-backward-token: !non-local\ exit!
| | | | 5 <- sh-smie--newline-semi-p: !non-local\ exit!
| | | 4 <- sh-smie-sh-forward-token: !non-local\ exit!
| | 3 <- aggressive-indent-indent-region-and-on: !non-local\ exit!
| 2 <- aggressive-indent--softly-indent-region-and-on: !non-local\ exit!
1 <- aggressive-indent--indent-if-changed: t
--8<---------------cut here---------------end--------------->8---

Bye,
Tassilo