bug#38560: 27.0.50; Emacs ignores input events during startup

bug#38560: 27.0.50; Emacs ignores input events during startup

[Dmitry Gutov]

I have a configuration that takes 1-2 seconds to boot up.

In previous versions (including the latest Emacs 26) it was rarely a
problem because I could always start typing while the configuration
finishes loading (e.g. C-x C-f first-chars-of-file-name).

Emacs 27, however, ignores most of the keystrokes, so when I do that, by
habit, I simply end up in the Scratch buffer with some of the last
characters. E.g. "ile-name" in this example.

In GNU Emacs 27.0.50 (build 29, x86_64-pc-linux-gnu, GTK+ Version 3.24.8)
  of 2019-12-10 built on potemkin
Repository revision: bb8f46d55bb369589ae475e47ad948da88c0d31d
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Ubuntu 19.04

Recent messages:
Loading /home/dgutov/.custom.el (source)...
Loading autorevert...done
Loading help-at-pt...done
Loading /home/dgutov/.custom.el (source)...done
Loading quail/cyrillic...done
Loading /home/dgutov/.emacs.d/site-lisp/.emacs-loaddefs.el (source)...done
Switching to window configuration: nil
Highlight tail mode enabled
For information about GNU Emacs and the GNU system, type C-h C-a.
Package cl is deprecated

Configured using:
  'configure --without-makeinfo'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER LCMS2 GMP

bug#38560: 27.0.50; Emacs ignores input events during startup

[Dmitry Gutov]

Please don't use the email I originally reported this from when 
replying. Use this one.

On 10.12.2019 16:11, Dmitry Gutov wrote:
> I have a configuration that takes 1-2 seconds to boot up.
> 
> In previous versions (including the latest Emacs 26) it was rarely a
> problem because I could always start typing while the configuration
> finishes loading (e.g. C-x C-f first-chars-of-file-name).
> 
> Emacs 27, however, ignores most of the keystrokes, so when I do that, by
> habit, I simply end up in the Scratch buffer with some of the last
> characters. E.g. "ile-name" in this example.

bug#38560: 27.0.50; Emacs ignores input events during startup

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 10 Dec 2019 16:14:19 +0200
> 
> > I have a configuration that takes 1-2 seconds to boot up.
> > 
> > In previous versions (including the latest Emacs 26) it was rarely a
> > problem because I could always start typing while the configuration
> > finishes loading (e.g. C-x C-f first-chars-of-file-name).
> > 
> > Emacs 27, however, ignores most of the keystrokes, so when I do that, by
> > habit, I simply end up in the Scratch buffer with some of the last
> > characters. E.g. "ile-name" in this example.

There was some recent change related to discard-input, no?  Here:

  commit 54c792ece6c20297571aa68c613075c8a8152bcc
  Author:     Juri Linkov <juri@linkov.net>
  AuthorDate: Sat Nov 30 23:33:41 2019 +0200
  Commit:     Juri Linkov <juri@linkov.net>
  CommitDate: Sat Nov 30 23:33:41 2019 +0200

      * lisp/subr.el: Add discard-input to more functions.

      * lisp/subr.el (read-char-from-minibuffer-insert-other)
      (y-or-n-p-insert-other): Use discard-input in case of user mistake.
      (do-after-load-evaluation): Add discard-input before calling 'message',
      so in case of the active minibuffer, calling minibuffer-message will allow
      sit-for to wait the complete timeout, because discard-input will discard
      all initial events that prevent sit-for from waiting during startup.
      https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00581.html

Or maybe this one:

  commit 4e4e5508582b1b5a2e738f4490400f2d78a5f343
  Author:     Juri Linkov <juri@linkov.net>
  AuthorDate: Sun Nov 17 23:51:55 2019 +0200
  Commit:     Juri Linkov <juri@linkov.net>
  CommitDate: Sun Nov 17 23:51:55 2019 +0200

      * lisp/subr.el (read-char-from-minibuffer, y-or-n-p): Remove discard-input.

      (do-after-load-evaluation): Replace run-with-timer with run-with-idle-timer
      to give a chance for the minibuffer to handle initial events before sit-for.
      https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00581.html

If these aren't the cause, can you bisect?

bug#38560: 27.0.50; Emacs ignores input events during startup

[Dmitry Gutov]

On 10.12.2019 18:05, Eli Zaretskii wrote:
>> From: Dmitry Gutov<dgutov@yandex.ru>
>> Date: Tue, 10 Dec 2019 16:14:19 +0200
>>
>>> I have a configuration that takes 1-2 seconds to boot up.
>>>
>>> In previous versions (including the latest Emacs 26) it was rarely a
>>> problem because I could always start typing while the configuration
>>> finishes loading (e.g. C-x C-f first-chars-of-file-name).
>>>
>>> Emacs 27, however, ignores most of the keystrokes, so when I do that, by
>>> habit, I simply end up in the Scratch buffer with some of the last
>>> characters. E.g. "ile-name" in this example.
> There was some recent change related to discard-input, no?  Here:
> 
>    commit 54c792ece6c20297571aa68c613075c8a8152bcc
>    Author:     Juri Linkov<juri@linkov.net>
>    AuthorDate: Sat Nov 30 23:33:41 2019 +0200
>    Commit:     Juri Linkov<juri@linkov.net>
>    CommitDate: Sat Nov 30 23:33:41 2019 +0200
> 
>        * lisp/subr.el: Add discard-input to more functions.
> 
>        * lisp/subr.el (read-char-from-minibuffer-insert-other)
>        (y-or-n-p-insert-other): Use discard-input in case of user mistake.
>        (do-after-load-evaluation): Add discard-input before calling 'message',
>        so in case of the active minibuffer, calling minibuffer-message will allow
>        sit-for to wait the complete timeout, because discard-input will discard
>        all initial events that prevent sit-for from waiting during startup.
>        https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00581.html

This one is indeed the cuplrit.

And I think it's the change to do-after-load-evaluation that created 
this problem.

/Cc Juri.

bug#38560: 27.0.50; Emacs ignores input events during startup

[Juri Linkov]

>> > I have a configuration that takes 1-2 seconds to boot up.
>> >
>> > In previous versions (including the latest Emacs 26) it was rarely a
>> > problem because I could always start typing while the configuration
>> > finishes loading (e.g. C-x C-f first-chars-of-file-name).
>> >
>> > Emacs 27, however, ignores most of the keystrokes, so when I do that, by
>> > habit, I simply end up in the Scratch buffer with some of the last
>> > characters. E.g. "ile-name" in this example.
>
> There was some recent change related to discard-input, no?  Here:
>
>   commit 54c792ece6c20297571aa68c613075c8a8152bcc
>   Author:     Juri Linkov <juri@linkov.net>
>   AuthorDate: Sat Nov 30 23:33:41 2019 +0200
>   Commit:     Juri Linkov <juri@linkov.net>
>   CommitDate: Sat Nov 30 23:33:41 2019 +0200
>
>       * lisp/subr.el: Add discard-input to more functions.
>
>       * lisp/subr.el (read-char-from-minibuffer-insert-other)
>       (y-or-n-p-insert-other): Use discard-input in case of user mistake.
>       (do-after-load-evaluation): Add discard-input before calling 'message',
>       so in case of the active minibuffer, calling minibuffer-message will allow
>       sit-for to wait the complete timeout, because discard-input will discard
>       all initial events that prevent sit-for from waiting during startup.
>       https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00581.html
>
> Or maybe this one:
>
>   commit 4e4e5508582b1b5a2e738f4490400f2d78a5f343
>   Author:     Juri Linkov <juri@linkov.net>
>   AuthorDate: Sun Nov 17 23:51:55 2019 +0200
>   Commit:     Juri Linkov <juri@linkov.net>
>   CommitDate: Sun Nov 17 23:51:55 2019 +0200
>
>       * lisp/subr.el (read-char-from-minibuffer, y-or-n-p): Remove discard-input.
>
>       (do-after-load-evaluation): Replace run-with-timer with run-with-idle-timer
>       to give a chance for the minibuffer to handle initial events before sit-for.
>       https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00581.html

I already created a fix for this a week ago in bug#38457,
but you didn't allow me to install the fix.  That's unfair
to blame me for problems that you don't allow to fix.

bug#38560: 27.0.50; Emacs ignores input events during startup

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  38560@debbugs.gnu.org
> Date: Wed, 11 Dec 2019 01:47:29 +0200
> 
> I already created a fix for this a week ago in bug#38457,
> but you didn't allow me to install the fix.  That's unfair
> to blame me for problems that you don't allow to fix.

I didn't blame anyone, I tried to help Dmitry find the changes
responsible for the issue he was reporting.

What fix was supposed to correct this?  Please show the URL.

bug#38560: 27.0.50; Emacs ignores input events during startup

[Eli Zaretskii]

> Date: Wed, 11 Dec 2019 05:35:04 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 38560@debbugs.gnu.org, dgutov@yandex.ru
> 
> > From: Juri Linkov <juri@linkov.net>
> > Cc: Dmitry Gutov <dgutov@yandex.ru>,  38560@debbugs.gnu.org
> > Date: Wed, 11 Dec 2019 01:47:29 +0200
> > 
> > I already created a fix for this a week ago in bug#38457,
> > but you didn't allow me to install the fix.  That's unfair
> > to blame me for problems that you don't allow to fix.
> 
> I didn't blame anyone, I tried to help Dmitry find the changes
> responsible for the issue he was reporting.
> 
> What fix was supposed to correct this?  Please show the URL.

I've now looked at bug#38457, and the only fix I see there involves a
timer to remove the text displayed by minibuffer-message.  Whereas
Dmitry's report is about a commit which added calls to discard-input
to various places.  How can the proposed timer get back input that was
discarded?  What am I missing?

bug#38560: 27.0.50; Emacs ignores input events during startup

[Juri Linkov]

>> I already created a fix for this a week ago in bug#38457,
>> but you didn't allow me to install the fix.  That's unfair
>> to blame me for problems that you don't allow to fix.
>
> I didn't blame anyone, I tried to help Dmitry find the changes
> responsible for the issue he was reporting.
>
> What fix was supposed to correct this?  Please show the URL.

Below is the output of 'C-1 C-x v L 245957c196 RET' that shows
that this fix was committed 2019-12-02.

Here's the reconstruction of events leading to this situation.
As your previous message shows:

>   commit 54c792ece6c20297571aa68c613075c8a8152bcc
>   Author:     Juri Linkov <juri@linkov.net>
>   AuthorDate: Sat Nov 30 23:33:41 2019 +0200
>   Commit:     Juri Linkov <juri@linkov.net>
>   CommitDate: Sat Nov 30 23:33:41 2019 +0200
>
>       * lisp/subr.el: Add discard-input to more functions.

That's right, I committed this change at Sat Nov 30.
Then it took me less than 2 days to realize there are
problems with this in do-after-load-evaluation and also to
notice other problems with dabbrev, Tramp, icomplete.
Then I immediately created a patch to finish implementation
of this feature that fixed all related problems with the patch
below committed to the local repository.  Then I posted
the first part of the patch in hope to push the remaining
part immediately afterwards.

commit 245957c196e3f982893733b7f025c8d55b28f23b
Author: Juri Linkov <juri@linkov.net>
Date:   2019-12-02 19:59:04 +0200

    * lisp/minibuffer.el: Add minibuffer-message-timer, minibuffer-message-overlay

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index a7bdde478f..3febdeb174 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -702,6 +702,9 @@ minibuffer
 (defvar minibuffer-message-properties nil
   "Text properties added to the text shown by `minibuffer-message'.")
 
+(defvar minibuffer-message-timer nil)
+(defvar minibuffer-message-overlay nil)
+
 (defun minibuffer-message (message &rest args)
   "Temporarily display MESSAGE at the end of the minibuffer.
 The text is displayed for `minibuffer-message-timeout' seconds,
@@ -732,24 +735,23 @@ minibuffer-message
                 ;; Don't overwrite the face properties the caller has set
                 (text-properties-at 0 message))
       (setq message (apply #'propertize message minibuffer-message-properties)))
-    (let ((ol (make-overlay (point-max) (point-max) nil t t))
-          ;; A quit during sit-for normally only interrupts the sit-for,
-          ;; but since minibuffer-message is used at the end of a command,
-          ;; at a time when the command has virtually finished already, a C-g
-          ;; should really cause an abort-recursive-edit instead (i.e. as if
-          ;; the C-g had been typed at top-level).  Binding inhibit-quit here
-          ;; is an attempt to get that behavior.
-          (inhibit-quit t))
-      (unwind-protect
-          (progn
-            (unless (zerop (length message))
-              ;; The current C cursor code doesn't know to use the overlay's
-              ;; marker's stickiness to figure out whether to place the cursor
-              ;; before or after the string, so let's spoon-feed it the pos.
-              (put-text-property 0 1 'cursor t message))
-            (overlay-put ol 'after-string message)
-            (sit-for (or minibuffer-message-timeout 1000000)))
-        (delete-overlay ol)))))
+
+    (when (timerp minibuffer-message-timer)
+      (cancel-timer minibuffer-message-timer))
+    (when (overlayp minibuffer-message-overlay)
+      (delete-overlay minibuffer-message-overlay))
+    (setq minibuffer-message-overlay
+          (make-overlay (point-max) (point-max) nil t t))
+    (setq minibuffer-message-timer
+          (run-with-timer (or minibuffer-message-timeout 1) nil
+                          (lambda () (when (overlayp minibuffer-message-overlay)
+                                       (delete-overlay minibuffer-message-overlay)))))
+    (unless (zerop (length message))
+      ;; The current C cursor code doesn't know to use the overlay's
+      ;; marker's stickiness to figure out whether to place the cursor
+      ;; before or after the string, so let's spoon-feed it the pos.
+      (put-text-property 0 1 'cursor t message))
+    (overlay-put minibuffer-message-overlay 'after-string message)))
 
 (defun minibuffer-completion-contents ()
   "Return the user input in a minibuffer before point as a string.
diff --git a/lisp/subr.el b/lisp/subr.el
index de7d919abf..a9d4443810 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -4609,7 +4609,7 @@ do-after-load-evaluation
 			               (string-match "\\.elc?\\>" file))
                             obarray))
 	   (msg (format "Package %s is deprecated" package))
-	   (fun (lambda (msg) (discard-input) (message "%s" msg))))
+	   (fun (lambda (msg) (message "%s" msg))))
       ;; Cribbed from cl--compiling-file.
       (when (or (not (fboundp 'byte-compile-warning-enabled-p))
                 (byte-compile-warning-enabled-p 'obsolete package))

bug#38560: 27.0.50; Emacs ignores input events during startup

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: dgutov@yandex.ru,  38560@debbugs.gnu.org
> Date: Thu, 12 Dec 2019 01:07:53 +0200
> 
> >> I already created a fix for this a week ago in bug#38457,
> >> but you didn't allow me to install the fix.  That's unfair
> >> to blame me for problems that you don't allow to fix.
> >
> > I didn't blame anyone, I tried to help Dmitry find the changes
> > responsible for the issue he was reporting.
> >
> > What fix was supposed to correct this?  Please show the URL.
> 
> Below is the output of 'C-1 C-x v L 245957c196 RET' that shows
> that this fix was committed 2019-12-02.
> 
> Here's the reconstruction of events leading to this situation.
> As your previous message shows:
> 
> >   commit 54c792ece6c20297571aa68c613075c8a8152bcc
> >   Author:     Juri Linkov <juri@linkov.net>
> >   AuthorDate: Sat Nov 30 23:33:41 2019 +0200
> >   Commit:     Juri Linkov <juri@linkov.net>
> >   CommitDate: Sat Nov 30 23:33:41 2019 +0200
> >
> >       * lisp/subr.el: Add discard-input to more functions.
> 
> That's right, I committed this change at Sat Nov 30.
> Then it took me less than 2 days to realize there are
> problems with this in do-after-load-evaluation and also to
> notice other problems with dabbrev, Tramp, icomplete.
> Then I immediately created a patch to finish implementation
> of this feature that fixed all related problems with the patch
> below committed to the local repository.  Then I posted
> the first part of the patch in hope to push the remaining
> part immediately afterwards.

Are you saying that you meant to revert the changes which added
discard-inputs later on?

bug#38560: 27.0.50; Emacs ignores input events during startup

[Juri Linkov]

> Are you saying that you meant to revert the changes which added
> discard-inputs later on?

Exactly.