bug#31692: Emacs sometimes drops key events

bug#31692: Emacs sometimes drops key events

[Radon Rosborough]

[-- Attachment #1: Type: text/plain, Size: 2376 bytes --]

For the past year or two, my Emacs has occasionally had fits where it
drops some of my key events. The symptom is that I type something like
"radian" and I get "rdian".

This happens more often when things are laggy in general, but also
happens at other times. Usually, my Emacs will get into a state where
it'll do it a lot, at least for a little while, but I can't reliably
reproduce that state.

I noticed that whenever a key is dropped, the text "Quit" is logged in
the *Messages* buffer. However, `keyboard-quit' doesn't get called; I
checked by adding an advice. Something in the C code causes a quit
event to be signaled and also aborts the processing of my key event,
apparently.

I have confirmed that the problem still happens when none of Flycheck,
Company, ElDoc are active. So it's not any of those. I haven't managed
to reproduce the problem when Aggressive Indent is disabled, but that
doesn't necessarily mean that Aggressive Indent is the problem. It
could just be that Aggressive Indent makes my Emacs much slower, and
so the bug is more likely to be observed.

Typically, the dropped key does not even show up in the lossage (C-h
l). However, one time I did notice this curious phenomenon:

     r [self-insert-command]
     a d [self-insert-command]
     i [self-insert-command]
     a [self-insert-command]
     n [self-insert-command]

Where I typed "radian" but what was inserted was "rdian". The
corresponding excerpt of the vector returned by `recent-keys' was:

     114
     (nil . self-insert-command)
     97 100
     (nil . self-insert-command)
     105
     (nil . self-insert-command)
     97
     (nil . self-insert-command)
     110
     (nil . self-insert-command)

In the terminal, I've noticed things like the letters A-D getting
inserted when I use the arrow keys. I presume that this is because one
of the events in the escape sequence is eaten by this bug, and the
rest of it gets misinterpreted.

I have no idea how to go about debugging this, and unfortunately I
don't know how to reliably reproduce the problem. But it's been with
me a long time and it doesn't look like it's going away on its own.
Can someone give me pointers on how to work towards a solution?

Environment information:
  Emacs - 26.1, Homebrew, --with-cocoa --with-modules
  OS - macOS, 10.11.6
  Configuration - https://github.com/raxod502/radian

Thanks,
Radon

[-- Attachment #2: Type: text/html, Size: 3281 bytes --]

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Radon Rosborough <radon.neon@gmail.com>
> Date: Sat, 2 Jun 2018 20:21:32 -0600
> 
> For the past year or two, my Emacs has occasionally had fits where it
> drops some of my key events. The symptom is that I type something like
> "radian" and I get "rdian".

Is this in a text-mode frame or a GUI frame?

bug#31692: Emacs sometimes drops key events

[Radon Rosborough]

[-- Attachment #1: Type: text/plain, Size: 73 bytes --]

> Is this in a text-mode frame or a GUI frame?

It has happened in both.

[-- Attachment #2: Type: text/html, Size: 921 bytes --]

bug#31692: Emacs sometimes drops key events

[Noam Postavsky]

Radon Rosborough <radon.neon@gmail.com> writes:

> I noticed that whenever a key is dropped, the text "Quit" is logged in
> the *Messages* buffer. However, `keyboard-quit' doesn't get called; I
> checked by adding an advice. Something in the C code causes a quit
> event to be signaled and also aborts the processing of my key event,
> apparently.

Maybe you could run under a debugger and make it print a backtrace when
a quit happens?

break quit
commands
  silent
  bt
  continue
end

set height 0 # don't page

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Radon Rosborough <radon.neon@gmail.com>
> Date: Sat, 2 Jun 2018 22:54:11 -0600
> Cc: 31692@debbugs.gnu.org
> 
> > Is this in a text-mode frame or a GUI frame?
> 
> It has happened in both.

Then it's probably something specific to NS event handling.

Does this happen in "emacs -Q" as well?

bug#31692: Emacs sometimes drops key events

[Alan Third]

On Sun, Jun 03, 2018 at 06:05:36PM +0300, Eli Zaretskii wrote:
> > From: Radon Rosborough <radon.neon@gmail.com>
> > Date: Sat, 2 Jun 2018 22:54:11 -0600
> > Cc: 31692@debbugs.gnu.org
> > 
> > > Is this in a text-mode frame or a GUI frame?
> > 
> > It has happened in both.
> 
> Then it's probably something specific to NS event handling.

It could be worth redefining NS_KEYLOG to 1 in nsterm.m. This prints a
whole bunch of stuff to the terminal and should let us find out where
the key presses are being lost if it is in the NS code.
-- 
Alan Third

bug#31692: Emacs sometimes drops key events

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 972 bytes --]

I'd just like to add a data point here.

I'm on Debian GNU/Linux and have experienced the same thing (though not
very often).

I lose a key, and there's a "Quit" in my *Messages*, too.

More importantly perhaps, I'm also using aggressive-indent-mode.

João

On Sun, Jun 3, 2018 at 8:23 PM, Alan Third <alan@idiocy.org> wrote:

> On Sun, Jun 03, 2018 at 06:05:36PM +0300, Eli Zaretskii wrote:
> > > From: Radon Rosborough <radon.neon@gmail.com>
> > > Date: Sat, 2 Jun 2018 22:54:11 -0600
> > > Cc: 31692@debbugs.gnu.org
> > >
> > > > Is this in a text-mode frame or a GUI frame?
> > >
> > > It has happened in both.
> >
> > Then it's probably something specific to NS event handling.
>
> It could be worth redefining NS_KEYLOG to 1 in nsterm.m. This prints a
> whole bunch of stuff to the terminal and should let us find out where
> the key presses are being lost if it is in the NS code.
> --
> Alan Third
>
>
>
>


-- 
João Távora

[-- Attachment #2: Type: text/html, Size: 1740 bytes --]

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> Is this in a text-mode frame or a GUI frame?

`aggressive-indent-mode' has been mentioned multiple times (defined in
Gnu Elpa package "aggressive-indent" btw).

I see a combination of `while-no-input' and `sit-for' in
`aggressive-indent--indent-if-changed':

#+begin_src emacs-lisp
(defun aggressive-indent--indent-if-changed ()
  "Indent any region that changed in the last command loop."
  (when aggressive-indent--changed-list
    (save-excursion
      (save-selected-window
        (unless (or (run-hook-wrapped 'aggressive-indent--internal-dont-indent-if #'eval)
                    (aggressive-indent--run-user-hooks))
          (while-no-input
            (sit-for aggressive-indent-sit-for-time t)
            (redisplay)
            (aggressive-indent--proccess-changed-list-and-indent)))))))
#+end_src

This test function behaves as described if you call it and hit a key
before sit-for has terminated:

(defun test ()
  (interactive)
  (while-no-input
    (sit-for 3))
  (message "Done"))

I then just get a "Quit" - the hitten key, and the (message "Done") call
are skipped.


Michael.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: Radon Rosborough <radon.neon@gmail.com>,  31692@debbugs.gnu.org
> Date: Tue, 05 Jun 2018 02:59:30 +0200
> 
> This test function behaves as described if you call it and hit a key
> before sit-for has terminated:
> 
> (defun test ()
>   (interactive)
>   (while-no-input
>     (sit-for 3))
>   (message "Done"))
> 
> I then just get a "Quit" - the hitten key, and the (message "Done") call
> are skipped.

If you are saying that aggressive-indent-mode is the culprit, then
this bug should be reported to its developer(s).

Thanks.

bug#31692: Emacs sometimes drops key events

[Radon Rosborough]

> If you are saying that aggressive-indent-mode is the culprit, then
> this bug should be reported to its developer(s).

Are you saying that the output of the test snippet is as expected, and
not a bug? It seems odd to me that `while-no-input' causes the key
event to be dropped.

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> If you are saying that aggressive-indent-mode is the culprit, then
> this bug should be reported to its developer(s).

I can't estimate if what they do in their code is bad or should behave
different (sane) and this is a bug in Emacs - please enlighten us!


Thanks,

Michael.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

On June 5, 2018 5:41:02 AM GMT+03:00, Radon Rosborough <radon.neon@gmail.com> wrote:
> > If you are saying that aggressive-indent-mode is the culprit, then
> > this bug should be reported to its developer(s).
> 
> Are you saying that the output of the test snippet is as expected, and
> not a bug? It seems odd to me that `while-no-input' causes the key
> event to be dropped.

My reading of the documentation of while-no-input is that yes, this
is the expected behavior.  The description in the ELisp manual is
especially explicit about "quit-like" effect of early arrival of
input.  And the implementation clearly means what we observe.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

On June 5, 2018 5:48:45 AM GMT+03:00, Michael Heerdegen <michael_heerdegen@web.de> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > If you are saying that aggressive-indent-mode is the culprit, then
> > this bug should be reported to its developer(s).
> 
> I can't estimate if what they do in their code is bad or should behave
> different (sane) and this is a bug in Emacs - please enlighten us!
> 
> 
> Thanks,
> 
> Michael.

All I meant to say is that issues specific to a package should be
reported to the respective developers.  It's a procedural issue, IOW.

bug#31692: Emacs sometimes drops key events

[Noam Postavsky]

retitle 31692 aggressive-indent-mode causes Emacs to sometimes drop key events
quit

Eli Zaretskii <eliz@gnu.org> writes:

> All I meant to say is that issues specific to a package should be
> reported to the respective developers.  It's a procedural issue, IOW.

Seems to have been reported already, with no response:

https://github.com/Malabarba/aggressive-indent-mode/issues/111

Although since it's in GNU ELPA, I guess reporting it here is also
correct.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Cc: Michael Heerdegen <michael_heerdegen@web.de>,  radon.neon@gmail.com,  31692@debbugs.gnu.org
> Date: Tue, 05 Jun 2018 07:57:55 -0400
> 
> Although since it's in GNU ELPA, I guess reporting it here is also
> correct.

Reporting here is OK, but I think then the developer should be CC'ed
(done).

bug#31692: Emacs sometimes drops key events

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

Thanks for the cc.
I'll admit that issue flew under my radar.

I'm not sure how to fix it, though. Is there a way to ensure that the input
that interrupts `while-no-input' still gets properly handled by emacs?


On Tue, Jun 5, 2018, 11:31 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Noam Postavsky <npostavs@gmail.com>
> > Cc: Michael Heerdegen <michael_heerdegen@web.de>,  radon.neon@gmail.com,
> 31692@debbugs.gnu.org
> > Date: Tue, 05 Jun 2018 07:57:55 -0400
> >
> > Although since it's in GNU ELPA, I guess reporting it here is also
> > correct.
>
> Reporting here is OK, but I think then the developer should be CC'ed
> (done).
>

[-- Attachment #2: Type: text/html, Size: 1483 bytes --]

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> My reading of the documentation of while-no-input is that yes, this
> is the expected behavior.  The description in the ELisp manual is
> especially explicit about "quit-like" effect of early arrival of
> input.

AFAIU, this quit-like effect should be limited to `while-no-input' and
not be propagated to top-level.  I also read the documentation of the
low-level tools with which this is implemented - `throw-input' and
`quit-flag', and that also gives me the impression that generating a
"quit" signal is not intended (that would also be not very useful).  We
only want a quit to come through when the user really quits (C-g) -
that's while `while-no-input' (which uses `with-local-quit' that sets
`quit-flag') explicitly quit - but only when a quit had been
intercepted.

I made some further experiments with my test function

#+begin_src emacs-lisp
(defun test ()
  (interactive)
  (while-no-input
    (sit-for 3))
  (message "Done"))
#+end_src

What I found was that when I load the source file subr.el, or only the
definition of `sit-for', calling `test' and hitting a key does not
produce a quit any more.  Something here seems not kosher.


Michael.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: Radon Rosborough <radon.neon@gmail.com>,  31692@debbugs.gnu.org
> Date: Tue, 05 Jun 2018 23:40:55 +0200
> 
> What I found was that when I load the source file subr.el, or only the
> definition of `sit-for', calling `test' and hitting a key does not
> produce a quit any more.

Does it produce the character you type, though?  Because if it
doesn't, then whether or not you see "Quit" doesn't matter much, IMO,
at least not for the main issue at hand.

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> Does it produce the character you type, though?  Because if it
> doesn't, then whether or not you see "Quit" doesn't matter much, IMO,
> at least not for the main issue at hand.

Hmm, right, it doesn't.

But I do not only see the issue in the aggressive-indent-mode, but also
an issue in Emacs...you don't?

For `aggressive-indent-mode' - I think they should use an idle timer:
Add something to `post-command-hook' that activates the timer, or resets
the idle time when the timer is already there.  When the timer fires,
you don't need `sit-for' - and the problem only occurs when
`while-no-input' and `sit-for' are combined.  The timer just calls the
aggressive indent function wrapped in `while-no-input'.


Michael.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Date: Tue, 5 Jun 2018 18:39:25 -0300
> Cc: Noam Postavsky <npostavs@gmail.com>, Michael Heerdegen <michael_heerdegen@web.de>, radon.neon@gmail.com, 
> 	31692@debbugs.gnu.org
> 
> Thanks for the cc.
> I'll admit that issue flew under my radar.
> 
> I'm not sure how to fix it, though. Is there a way to ensure that the input that interrupts `while-no-input' still gets
> properly handled by emacs? 

Can you explain why you needed while-no-input at all?  Why not just
sit-for?

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: Radon Rosborough <radon.neon@gmail.com>,  31692@debbugs.gnu.org
> Date: Tue, 05 Jun 2018 23:40:55 +0200
> 
> AFAIU, this quit-like effect should be limited to `while-no-input' and
> not be propagated to top-level.

I don't think I understand what you mean by "not propagated".  The
documentation says that if input arrives, the forms in the body of
while-no-input are _aborted_ as if by interrupt.

> I also read the documentation of the
> low-level tools with which this is implemented - `throw-input' and
> `quit-flag', and that also gives me the impression that generating a
> "quit" signal is not intended (that would also be not very useful).  We
> only want a quit to come through when the user really quits (C-g) -
> that's while `while-no-input' (which uses `with-local-quit' that sets
> `quit-flag') explicitly quit - but only when a quit had been
> intercepted.

But in your case while-no-input was the top-level form, no?  So where
else can a quit throw?

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: radon.neon@gmail.com,  31692@debbugs.gnu.org
> Date: Wed, 06 Jun 2018 04:58:31 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Does it produce the character you type, though?  Because if it
> > doesn't, then whether or not you see "Quit" doesn't matter much, IMO,
> > at least not for the main issue at hand.
> 
> Hmm, right, it doesn't.
> 
> But I do not only see the issue in the aggressive-indent-mode, but also
> an issue in Emacs...you don't?

Not necessarily.  The forms of the body are aborted, as documented,
and that includes discarding all input, so the character that
interrupted while-no-input gets discarded as well.

> For `aggressive-indent-mode' - I think they should use an idle timer:
> Add something to `post-command-hook' that activates the timer, or resets
> the idle time when the timer is already there.  When the timer fires,
> you don't need `sit-for' - and the problem only occurs when
> `while-no-input' and `sit-for' are combined.  The timer just calls the
> aggressive indent function wrapped in `while-no-input'.

I actually don't understand why not use just sit-for.  Why is
while-no-input needed on top of that?

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't think I understand what you mean by "not propagated".  The
> documentation says that if input arrives, the forms in the body of
> while-no-input are _aborted_ as if by interrupt.

Yes, those in the _body: of while-no-input.  But not those in the
outside scope, which is what is happening here also.

> > I also read the documentation of the
> > low-level tools with which this is implemented - `throw-input' and
> > `quit-flag', and that also gives me the impression that generating a
> > "quit" signal is not intended (that would also be not very useful).  We
> > only want a quit to come through when the user really quits (C-g) -
> > that's while `while-no-input' (which uses `with-local-quit' that sets
> > `quit-flag') explicitly quit - but only when a quit had been
> > intercepted.
>
> But in your case while-no-input was the top-level form, no?  So where
> else can a quit throw?

There should be no quit at all.  `throw-on-input' uses a special value
`quit-flag' internally to implement what it does, but AFAIU signaling a
real quit is not intended.

In my example, `test' was the top-level form, and I would expect that
`while-no-input' completes normally, which it doesn't in this case.
This case is the only one I know of where `while-no-input' is left with
an nonlocal exit.  AFAIU `while-no-input' is also not meant to consume
user input, and `sit-for' also should not do this.  Only the combination
of the two shows this behavior.


Michael.

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> Not necessarily.  The forms of the body are aborted, as documented,
> and that includes discarding all input, so the character that
> interrupted while-no-input gets discarded as well.

I don't think discarding input is intended.  If you replace `sit-for' by
something else (a loop causing a delay, or a sleep for), it doesn't
discard the input.

> > For `aggressive-indent-mode' - I think they should use an idle timer:
> > Add something to `post-command-hook' that activates the timer, or
> > resets
> > the idle time when the timer is already there.  When the timer fires,
> > you don't need `sit-for' - and the problem only occurs when
> > `while-no-input' and `sit-for' are combined.  The timer just calls the
> > aggressive indent function wrapped in `while-no-input'.
>
> I actually don't understand why not use just sit-for.  Why is
> while-no-input needed on top of that?

Seems you never used `while-no-input', or at least not for the same
things as I did.

AFAIU the purpose of `while-no-input' is to make frequently occurring
"background" operations that last long enough to possibly interfere with
user interaction interruptable: hitting a key aborts the operation and
the editor is corresponding without delay.  And the command
corresponding to the hitten key is executed as normal.  Of course you
need to ensure that the aborted code always leaves thing in a consistent
state.

An automatic indent mode is a good example: you potentially want
re-indenting after any editing operation, but it is potentially slow and
would cause delays.  Using `while-no-input' let's you interrupt the
indentation operation and continue typing.  If you later make a long
enough typing pause, indentation will finish.  The `sit-for' is there to
avoid that indenting fires immediately after hitting a key, so that
interrupts are less common.

That's what I think is the purpose of `while-no-input'.  I use it here
and there, and never saw the behavior we see here.  It seems this only
happens if a `sit-for' is aborted inside the scope of a
`while-no-input'.


Michael.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: radon.neon@gmail.com,  31692@debbugs.gnu.org
> Date: Thu, 07 Jun 2018 00:50:00 +0200
> 
> > But in your case while-no-input was the top-level form, no?  So where
> > else can a quit throw?
> 
> There should be no quit at all.  `throw-on-input' uses a special value
> `quit-flag' internally to implement what it does, but AFAIU signaling a
> real quit is not intended.

It is not intended, but it can (and did) happen.  See below.

> In my example, `test' was the top-level form, and I would expect that
> `while-no-input' completes normally, which it doesn't in this case.

That's not what I see.  while-no-input does "complete normally" (for
some value of "normally"), but then we immediately quit to top-level
after while-no-input returns.

> This case is the only one I know of where `while-no-input' is left with
> an nonlocal exit.

Then I guess you just were lucky.  After reading the details below,
I'm sure you will be able to concoct any number of use cases where
similar things happen.

> AFAIU `while-no-input' is also not meant to consume user input, and
> `sit-for' also should not do this.  Only the combination of the two
> shows this behavior.

It's not while-no-input that swallows the input.  It's the quit to
top-level executed after while-no-input returns that discards it.

How do we end up quitting to top-level?  Well, that's because sit-for
exits due to keyboard input, and while-no-input then also exits, since
its BODY finished.  However, quit-flag, which was set when keyboard
input processing noticed that we are inside while-no-input, is still
set to a non-nil value.  It is true that quit-flag's value is set to
the symbol created by while-no-input, but the let-binding inside
while-no-input which was supposed to catch that special kind of
"quitting" is already gone, since we are outside of while-no-input.
And a non-nil value of quit-flag without any valid catcher always
quits.  So that is what we get, and discarding pending input is just
one side effect of doing that.

Once upon a time, when while-no-input was written, Emacs tested
quit-flag in the signal handler which processed input, and we then
would throw to the while-no-input's catcher right out of the signal
handler.  But nowadays we no longer jump out of the signal handler, we
only set a flag there, which is tested "when it's safe", i.e. when
Emacs calls maybe_quit.  And the code run by sit-for doesn't call
maybe_quit, so the flag is never tested, until after sit-for and
while-no-input return, and we are about to call 'message'.  sit-for
stashes input it read in unread-command-events, but when we quit, we
discard any events in unread-command-events.

IOW, any BODY of while-no-input that never calls maybe_quit will
produce the same effect of quitting to top-level.  And that is indeed
not expected by callers of while-no-input.

So with that in mind, I propose the following patch to while-no-input
(and CC Stefan who messed with the related code and with sit-for much
more than I did):

--- lisp/subr.el~0	2018-03-14 06:40:04.000000000 +0200
+++ lisp/subr.el	2018-06-07 17:59:40.229348300 +0300
@@ -3511,9 +3511,25 @@
   (let ((catch-sym (make-symbol "input")))
     `(with-local-quit
        (catch ',catch-sym
-	 (let ((throw-on-input ',catch-sym))
-	   (or (input-pending-p)
-	       (progn ,@body)))))))
+	 (let ((throw-on-input ',catch-sym)
+               val)
+           (setq val
+	         (or (input-pending-p)
+	             (progn ,@body)))
+           (cond
+            ;; If quit-flag is equal to throw-on-input, it means BODY
+            ;; didn't test quit-flag, and therefore ran to completion
+            ;; even though input arrived before it finished.  In that
+            ;; case, we must throw manually, because otherwise
+            ;; quit-flag will remain set, and we get Quit to
+            ;; top-level, which has undesirable consequences, such as
+            ;; discarding input etc.
+            ((eq quit-flag throw-on-input)
+             (throw 'throw-on-input t))
+            ;; This is in case the user actually quits while BODY runs.
+            (quit-flag
+             nil)
+            (t val)))))))
 
 (defmacro condition-case-unless-debug (var bodyform &rest handlers)
   "Like `condition-case' except that it does not prevent debugging.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: radon.neon@gmail.com,  31692@debbugs.gnu.org
> Date: Thu, 07 Jun 2018 01:12:54 +0200
> 
> An automatic indent mode is a good example: you potentially want
> re-indenting after any editing operation, but it is potentially slow and
> would cause delays.  Using `while-no-input' let's you interrupt the
> indentation operation and continue typing.  If you later make a long
> enough typing pause, indentation will finish.  The `sit-for' is there to
> avoid that indenting fires immediately after hitting a key, so that
> interrupts are less common.

If this is the problem to solve, then why not something like below?

  (if (sit-for 3)
      (while-no-input
        (do-something-longish)))

IOW, what I don't understand is why put sit-for, which returns as soon
as there's input, inside while-no-input, which does something similar,
but using a radically different method?

bug#31692: Emacs sometimes drops key events

[Stefan Monnier]

> @@ -3511,9 +3511,25 @@
>    (let ((catch-sym (make-symbol "input")))
>      `(with-local-quit
>         (catch ',catch-sym
> -	 (let ((throw-on-input ',catch-sym))
> -	   (or (input-pending-p)
> -	       (progn ,@body)))))))
> +	 (let ((throw-on-input ',catch-sym)
> +               val)
> +           (setq val
> +	         (or (input-pending-p)
> +	             (progn ,@body)))
> +           (cond
> +            ;; If quit-flag is equal to throw-on-input, it means BODY
> +            ;; didn't test quit-flag, and therefore ran to completion
> +            ;; even though input arrived before it finished.  In that
> +            ;; case, we must throw manually, because otherwise
> +            ;; quit-flag will remain set, and we get Quit to
> +            ;; top-level, which has undesirable consequences, such as
> +            ;; discarding input etc.
> +            ((eq quit-flag throw-on-input)
> +             (throw 'throw-on-input t))
> +            ;; This is in case the user actually quits while BODY runs.
> +            (quit-flag
> +             nil)
> +            (t val)))))))

I don't understand the (throw 'throw-on-input t)
(throw throw-on-input t) would make more sense, but in any case we're
just about to exit that `catch` so throwing won't have any real effect.
I think instead of (throw 'throw-on-input t) what is needed here is

    (setq quit-flag nil)

which will also make the body return nil (rather than t like (throw
throw-on-input t) would), which I believe is also the right thing.


        Stefan

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Michael Heerdegen <michael_heerdegen@web.de>, radon.neon@gmail.com,
>         31692@debbugs.gnu.org
> Date: Thu, 07 Jun 2018 11:30:58 -0400
> 
> I don't understand the (throw 'throw-on-input t)
> (throw throw-on-input t) would make more sense, but in any case we're
> just about to exit that `catch` so throwing won't have any real effect.

I hoped the comment explained that, but evidently not.

> I think instead of (throw 'throw-on-input t) what is needed here is
> 
>     (setq quit-flag nil)
> 
> which will also make the body return nil (rather than t like (throw
> throw-on-input t) would), which I believe is also the right thing.

I believe we need to return t here, because input did arrive.  We
could indeed just return t instead of throwing, but that's not the
main issue for which I wanted your opinion.

bug#31692: Emacs sometimes drops key events

[Stefan Monnier]

>> I don't understand the (throw 'throw-on-input t)
>> (throw throw-on-input t) would make more sense, but in any case we're
>> just about to exit that `catch` so throwing won't have any real effect.
> I hoped the comment explained that, but evidently not.

I think I understood the comment (and the email message before that ;-),
and your analysis makes a lot of sense, but AFAICT the "normal" way this
special quit-flag value is handled in the C code (see process_quit_flag
in src/eval.c) is to

   - read quit-flag
   - notice it's `eq`ual to the value of throw-on-input
   - set quit-flag to nil
   - throw to the flag's value

so the throw is not done to the constant symbol `throw-on-input` but to
the value of this variable, and the magic doesn't happen in response to
this throw (i.e. it's not done within the implementation of the
throw/catch code), but instead it happens before the throw.

As a consequence, instead of

    (throw 'throw-on-input t)

we need to throw like

    (throw throw-on-input t)

and we additionally need to (setq quit-flag nil) before that.

>> I think instead of (throw 'throw-on-input t) what is needed here is
>> 
>>     (setq quit-flag nil)
>> 
>> which will also make the body return nil (rather than t like (throw
>> throw-on-input t) would), which I believe is also the right thing.
>
> I believe we need to return t here, because input did arrive.

Indeed, I had missed that

So I suggest

    (progn (setq quit-flag nil) t)

We could also use (throw throw-on-input t) after the setq, but I don't
see the point since it seems it would give exactly the same result just
in a more costly way.


        Stefan

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> As a consequence, instead of
>
>     (throw 'throw-on-input t)
>
> we need to throw like
>
>     (throw throw-on-input t)
>
> and we additionally need to (setq quit-flag nil) before that.

I tried to adapt Eli's patch accordingly.  To avoid confusion, I simply
use the concrete value - the CATCH-SYM - and just return t instead of
throwing:

diff --git a/lisp/subr.el b/lisp/subr.el
index 914112ccef..7734006d87 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -3520,9 +3520,25 @@ while-no-input
   (let ((catch-sym (make-symbol "input")))
     `(with-local-quit
        (catch ',catch-sym
-	 (let ((throw-on-input ',catch-sym))
-	   (or (input-pending-p)
-	       (progn ,@body)))))))
+	 (let ((throw-on-input ',catch-sym)
+               val)
+           (setq val
+	         (or (input-pending-p)
+	             (progn ,@body)))
+           (cond
+            ;; If quit-flag is bound to the CATCH-SYM, it means BODY
+            ;; didn't test quit-flag, and therefore ran to completion
+            ;; even though input arrived before it finished.  In that
+            ;; case, we must reset quit-flag (and return t), because
+            ;; otherwise quit-flag will remain set, and we get Quit to
+            ;; top-level, which has undesirable consequences, such as
+            ;; discarding input etc.
+            ((eq quit-flag ',catch-sym)
+             (setq quit-flag nil)
+             t)
+            ;; This is in case the user actually quits while BODY runs.
+            (quit-flag nil)
+            (t val)))))))
 
 (defmacro condition-case-unless-debug (var bodyform &rest handlers)
   "Like `condition-case' except that it does not prevent debugging.



Michael.

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Hello,

is this ok to install (Eli)?

> I tried to adapt Eli's patch accordingly.  To avoid confusion, I simply
> use the concrete value - the CATCH-SYM - and just return t instead of
> throwing:
>
> diff --git a/lisp/subr.el b/lisp/subr.el
> index 914112ccef..7734006d87 100644
> --- a/lisp/subr.el
> +++ b/lisp/subr.el
> @@ -3520,9 +3520,25 @@ while-no-input
>    (let ((catch-sym (make-symbol "input")))
>      `(with-local-quit
>         (catch ',catch-sym
> -	 (let ((throw-on-input ',catch-sym))
> -	   (or (input-pending-p)
> -	       (progn ,@body)))))))
> +	 (let ((throw-on-input ',catch-sym)
> +               val)
> +           (setq val
> +	         (or (input-pending-p)
> +	             (progn ,@body)))
> +           (cond
> +            ;; If quit-flag is bound to the CATCH-SYM, it means BODY
> +            ;; didn't test quit-flag, and therefore ran to completion
> +            ;; even though input arrived before it finished.  In that
> +            ;; case, we must reset quit-flag (and return t), because
> +            ;; otherwise quit-flag will remain set, and we get Quit to
> +            ;; top-level, which has undesirable consequences, such as
> +            ;; discarding input etc.
> +            ((eq quit-flag ',catch-sym)
> +             (setq quit-flag nil)
> +             t)
> +            ;; This is in case the user actually quits while BODY runs.
> +            (quit-flag nil)
> +            (t val)))))))
>  
>  (defmacro condition-case-unless-debug (var bodyform &rest handlers)
>    "Like `condition-case' except that it does not prevent debugging.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Date: Mon, 11 Jun 2018 23:08:06 +0200
> Cc: radon.neon@gmail.com, 31692@debbugs.gnu.org
> 
> is this ok to install (Eli)?

This issue is on my queue.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> Date: Tue, 12 Jun 2018 05:27:22 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: radon.neon@gmail.com, 31692@debbugs.gnu.org, monnier@IRO.UMontreal.CA
> 
> > From: Michael Heerdegen <michael_heerdegen@web.de>
> > Date: Mon, 11 Jun 2018 23:08:06 +0200
> > Cc: radon.neon@gmail.com, 31692@debbugs.gnu.org
> > 
> > is this ok to install (Eli)?
> 
> This issue is on my queue.

Should be fixed on master now.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: Eli Zaretskii <eliz@gnu.org>,  radon.neon@gmail.com,  31692@debbugs.gnu.org
> Date: Sat, 09 Jun 2018 00:17:47 +0200
> 
> I tried to adapt Eli's patch accordingly.  To avoid confusion, I simply
> use the concrete value - the CATCH-SYM

I actually prefer my original code in this case, because it's closer
to what process_quit_flag does when quit-flag is tested while BODY
runs.

Thanks.

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> I actually prefer my original code in this case, because it's closer
> to what process_quit_flag does when quit-flag is tested while BODY
> runs.

Looks good - thanks.


Michael.

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> I actually prefer my original code in this case, because it's closer
> to what process_quit_flag does when quit-flag is tested while BODY
> runs.

I found a case where this doesn't work as I had expected.

I want to use `while-no-input' to calculate a helpful string for
`minibuffer-message' when reading certain input from the minibuffer, and
also use `sit-for' (inside the `while-no-input') to add delays
between/after minibuffer messages.  When I hit C-g when this runs, I get
a normal quit instead of what C-g is bound to in the minibuffer.  My
current fix is to catch the quit (once again) and call the command that
is bound to C-g.


Thanks,

Michael.

bug#31692: Emacs sometimes drops key events

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: radon.neon@gmail.com,  31692@debbugs.gnu.org,  monnier@IRO.UMontreal.CA
> Date: Thu, 05 Jul 2018 21:07:08 +0200
> 
> I want to use `while-no-input' to calculate a helpful string for
> `minibuffer-message' when reading certain input from the minibuffer, and
> also use `sit-for' (inside the `while-no-input') to add delays
> between/after minibuffer messages.  When I hit C-g when this runs, I get
> a normal quit instead of what C-g is bound to in the minibuffer.  My
> current fix is to catch the quit (once again) and call the command that
> is bound to C-g.

I don't think you can reliably rebind C-g.  For starters, it produces
SIGINT on TTY frames, so I think rebinding will have no effect, at
least in that use case.

Anyway, can you show some Lisp to reproduce the issue?

Thanks.

bug#31692: Emacs sometimes drops key events

[Michael Heerdegen]

[-- Attachment #1: Type: text/plain, Size: 999 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't think you can reliably rebind C-g.  For starters, it produces
> SIGINT on TTY frames, so I think rebinding will have no effect, at
> least in that use case.

I think you misunderstood.  My fix was not to rebind C-g.

With our fix, hitting C-g while `sit-for' waits inside a
`while-no-input' makes the latter throw a 'quit' signal.  I can use
`condition-case' to catch that quit, and call the actual binding of C-g
(or just a hardcoded `abort-recursive-edit').

> Anyway, can you show some Lisp to reproduce the issue?

Here is a template.  To try it, call `read-test-input'.  To see that C-g
is not always running as expected, hit C-g while a sit-for is running
(i.g. when the hint is shown).

`read-test-input' reads in input and tries to do something: in this
example counts words in the current buffer (which is interruptable) and
uses an overlay to show a message in the minibuffer, which is
automatically removed after a `sit-for' has finished.


[-- Attachment #2: testcase.el --]
[-- Type: application/emacs-lisp, Size: 1701 bytes --]

[-- Attachment #3: Type: text/plain, Size: 11 bytes --]



Michael.