bug#5923: 23.1.95; minibuffer-message discards input events

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

Sorry, I don't have the time to try to track this down further.
 
Suffice it to say that starting with Emacs 23 `minibuffer-message'
discards input events during its `sit-for', that is while it displays
its message.

Dunno if this is a general `sit-for' bug or a `minibuffer-message'
bug. In effect, the `sit-for' is not interrupted by an input event
- it acts like `sleep-for'. Starting with Emacs 23,
`minibuffer-message' is coded differently (in Lisp, not C); dunno
about `sit-for'.
 
In my application, I have a key, `C-RET', bound in the minibuffer
completion maps. It performs an action, and the behavior of that
action can change if you give it a prefix arg: `C-u C-RET'. When
you give a prefix arg in this context, I call `minibuffer-message'
to echo `[prefix (4)]' (or whatever current-prefix-arg is).
 
Prior to Emacs 23, a user can hit `C-RET' after `C-u' and while
`[prefix (4)]' is displayed, and the `sit-for' is interrupted and
the action is executed immediately. Starting with Emacs 23, the
`C-RET' is ignored. A `C-RET' doesn't take effect until the
`sit-for' timeout is finished (as if it were `sleep-for').
 
I hope this is enough info for you to find and fix the bug.
I don't have time to try to track this down further.
 

In GNU Emacs 23.1.95.1 (i386-mingw-nt5.1.2600)
 of 2010-04-03 on G41R2F1
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4) --no-opt --cflags
-Ic:/imagesupport/include'
 

bug#5923: 23.1.95; minibuffer-message discards input events

[Stefan Monnier]

> Prior to Emacs 23, a user can hit `C-RET' after `C-u' and while
> `[prefix (4)]' is displayed, and the `sit-for' is interrupted and
> the action is executed immediately. Starting with Emacs 23, the
> `C-RET' is ignored. A `C-RET' doesn't take effect until the
> `sit-for' timeout is finished (as if it were `sleep-for').

I can't reproduce exactly your test case because I don't know what code
is run by your C-u (and because I'm on GNU/Linux, ...), but at least
when I start "emacs23 -Q" and do C-x C-f TAB TAB, the first tab outputs
a minibuffer-message but the second TAB is executed immediately
(interrupts the minibuffer-message).  So I don't see that problematic
behavior you're seeing.

Can you check whether my test case works for you as well?


        Stefan

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]

> > Prior to Emacs 23, a user can hit `C-RET' after `C-u' and while
> > `[prefix (4)]' is displayed, and the `sit-for' is interrupted and
> > the action is executed immediately. Starting with Emacs 23, the
> > `C-RET' is ignored. A `C-RET' doesn't take effect until the
> > `sit-for' timeout is finished (as if it were `sleep-for').
> 
> I can't reproduce exactly your test case because I don't know 
> what code is run by your C-u (and because I'm on GNU/Linux,
> ...), but at least when I start "emacs23 -Q" and do C-x C-f
> TAB TAB, the first tab outputs a minibuffer-message but the
> second TAB is executed immediately (interrupts the
> minibuffer-message).  So I don't see that problematic
> behavior you're seeing.
> 
> Can you check whether my test case works for you as well?

I confirm that your test case works for me also.
The second tab has its effect - it is not lost.

[However, the minibuffer message remains displayed for the full timeout period.
That seems wrong - why not stop displaying the msg as soon as the tab event
arrives? Unless `Complete but not unique' is perhaps redisplayed by another call
or something?

IOW, after the second tab, *Completions* is shown, indicating that the tab did
take effect, and the message `Complete but not unique' is briefly removed -
replaced by the message `Making completion list...'. But the `Complete but not
unique' message then reappears for the duration of the `minibuffer-message'
timeout. Is this the behavior you see also?

Anyway, this is not the problematic behavior I get with my code, and which this
bug report is about.]


Attached is the code I use for `C-u' in the minibuffer. I hope it helps.

I should have mentioned that the same problem occurs when I use `C-u' in the
minibuffer at any time, not just in the scenario where I follow it by `C-RET'.
IOW, it has nothing to do with the particular user event that follows.

And as I said, in Emacs 22 and before there is no such problem: any user event
immediately interrupts the message display and its timeout, and no such event is
lost (so you don't need to hit the key multiple times).

[-- Attachment #2: bug-5923-emacs.el --]
[-- Type: application/octet-stream, Size: 5004 bytes --]

(defvar icicle-universal-argument-map
  (let ((map  (make-sparse-keymap)))
    (define-key map [t] 'icicle-universal-argument-other-key)
    (define-key map (vector meta-prefix-char t) 'icicle-universal-argument-other-key)
    (define-key map [switch-frame] nil)
    (define-key map [?\C-u] 'icicle-universal-argument-more)
    (define-key map [?-] 'icicle-universal-argument-minus)
    (define-key map [?0] 'icicle-digit-argument)
    (define-key map [?1] 'icicle-digit-argument)
    (define-key map [?2] 'icicle-digit-argument)
    (define-key map [?3] 'icicle-digit-argument)
    (define-key map [?4] 'icicle-digit-argument)
    (define-key map [?5] 'icicle-digit-argument)
    (define-key map [?6] 'icicle-digit-argument)
    (define-key map [?7] 'icicle-digit-argument)
    (define-key map [?8] 'icicle-digit-argument)
    (define-key map [?9] 'icicle-digit-argument)
    (define-key map [kp-0] 'icicle-digit-argument)
    (define-key map [kp-1] 'icicle-digit-argument)
    (define-key map [kp-2] 'icicle-digit-argument)
    (define-key map [kp-3] 'icicle-digit-argument)
    (define-key map [kp-4] 'icicle-digit-argument)
    (define-key map [kp-5] 'icicle-digit-argument)
    (define-key map [kp-6] 'icicle-digit-argument)
    (define-key map [kp-7] 'icicle-digit-argument)
    (define-key map [kp-8] 'icicle-digit-argument)
    (define-key map [kp-9] 'icicle-digit-argument)
    (define-key map [kp-subtract] 'icicle-universal-argument-minus)
    map)
  "Keymap used while processing `C-u' during Icicles completion.")

(defun icicle-universal-argument ()    ; Bound to `C-u' in minibuffer.
  "`universal-argument', but also echo the prefix."
  (interactive)
  (setq prefix-arg                     (list 4)
        universal-argument-num-events  (length (this-command-keys)))
  (icicle-ensure-overriding-map-is-bound)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

;; Bound to `C-<0-9>', `M-<0-9>', `C-M-<0-9>' in minibuffer.
(defun icicle-digit-argument (arg)
  "`digit-argument', but also echo the prefix."
  (interactive "P")
  (let* ((char   (if (integerp last-command-char)
                     last-command-char
                   (get last-command-char 'ascii-character)))
         (digit  (- (logand char ?\177) ?0)))
    (cond ((integerp arg)
           (setq prefix-arg  (+ (* arg 10) (if (< arg 0) (- digit) digit))))
          ((eq arg '-)
           ;; Treat -0 as just -, so that -01 will work.
           (setq prefix-arg  (if (zerop digit) '- (- digit))))
          (t
           (setq prefix-arg  digit))))
  (setq universal-argument-num-events  (length (this-command-keys)))
  (icicle-ensure-overriding-map-is-bound)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-negative-argument (arg) ; Bound to `M--', `C-M--' in minibuffer.
  "`negative-argument', but also echo the prefix."
  (interactive "P")
  (cond ((integerp arg) (setq prefix-arg  (- arg)))
        ((eq arg '-) (setq prefix-arg  nil))
        (t (setq prefix-arg  '-)))
  (setq universal-argument-num-events  (length (this-command-keys)))
  (icicle-ensure-overriding-map-is-bound)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-more (arg)
  "`universal-argument-more', but also echo the prefix."
  (interactive "P")
  (universal-argument-more arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-other-key (arg)
  "`universal-argument-other-key', but also echo the prefix."
  (interactive "P")
  (universal-argument-other-key arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-minus (arg)
  "`universal-argument-minus', but also echo the prefix."
  (interactive "P")
  (universal-argument-minus arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-ensure-overriding-map-is-bound ()
  "Set `overriding-terminal-local-map' to `icicle-universal-argument-map'."
  (if (not (boundp 'overriding-map-is-bound)) ; Emacs 20, 21.
      (setq overriding-terminal-local-map  icicle-universal-argument-map)
    (unless overriding-map-is-bound     ; Emacs 22+.
      (setq saved-overriding-map           overriding-terminal-local-map
            overriding-terminal-local-map  icicle-universal-argument-map
            overriding-map-is-bound        t))))

(defun icicle-msg-maybe-in-minibuffer (format-string &rest args)
  "Display FORMAT-STRING as a message.
If called with the minibuffer inactive, use `message'.
Otherwise:
 If `icicle-minibuffer-message-ok-p', then use `minibuffer-message'.
 Else do nothing (no message display)."
  (if (active-minibuffer-window)
      (when icicle-minibuffer-message-ok-p
        (save-selected-window
          (select-window (minibuffer-window))
          (minibuffer-message (apply #'format (concat "  [" format-string "]") args))))
    (apply #'message format-string args)))

bug#5923: 23.1.95; minibuffer-message discards input events

[Stefan Monnier]

> Attached is the code I use for `C-u' in the minibuffer. I hope it helps.

It does.

But to tell you the truth the handling of this-command-keys and
universal-argument prefix is much too delicate for me to understand it,
so I'm not surprised it doesn't work quite right if you mess with it in
any non-trivial way.


        Stefan

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> > Attached is the code I use for `C-u' in the minibuffer. I 
> > hope it helps.
> 
> It does.
> 
> But to tell you the truth the handling of this-command-keys and
> universal-argument prefix is much too delicate for me to 
> understand it, so I'm not surprised it doesn't work quite right
> if you mess with it in any non-trivial way.

Compare Emacs 22 (and earlier), where there is no bug.

The code was changed (C to Lisp, at least), and that introduced the regression.

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> > > Attached is the code I use for `C-u' in the minibuffer. I 
> > > hope it helps.
> > 
> > It does.
> > 
> > But to tell you the truth the handling of this-command-keys and
> > universal-argument prefix is much too delicate for me to 
> > understand it, so I'm not surprised it doesn't work quite right
> > if you mess with it in any non-trivial way.
> 
> Compare Emacs 22 (and earlier), where there is no bug.
> 
> The code was changed (C to Lisp, at least), and that 
> introduced the regression.

Also, the code I use is _identical_ to the original code in simple.el (Emacs 22
version), except that it also calls `minibuffer-message'. Is that really a
"non-trivial" change?

(And in Emacs 23, `last-command-event' is used instead of `last-command-char' -
that is the only difference I can see from the Emacs 22 code.)

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> > > > Attached is the code I use for `C-u' in the minibuffer. I 
> > > > hope it helps.
> > > 
> > > It does.
> > > 
> > > But to tell you the truth the handling of this-command-keys and
> > > universal-argument prefix is much too delicate for me to 
> > > understand it, so I'm not surprised it doesn't work quite right
> > > if you mess with it in any non-trivial way.
> > 
> > Compare Emacs 22 (and earlier), where there is no bug.
> > 
> > The code was changed (C to Lisp, at least), and that 
> > introduced the regression.
> 
> Also, the code I use is _identical_ to the original code in 
> simple.el (Emacs 22
> version), except that it also calls `minibuffer-message'. Is 
> that really a
> "non-trivial" change?
> 
> (And in Emacs 23, `last-command-event' is used instead of 
> `last-command-char' - that is the only difference I can see
> from the Emacs 22 code.)

And changing `last-command-char' in the code to `last-command-event', for Emacs
23, does not fix the problem. It seems that the problem was introduced in the
translation from C to Lisp for the vanilla Emacs code for `minibuffer-message'.

Here is enough of the rest of the code I use to let you execute it and reproduce
the bug without doing anything extra:

(defun icicle-remap (old new map &optional oldmap)
  (define-key map (vector 'remap old) new))

(icicle-remap 'universal-argument 'icicle-universal-argument ; `C-u'
              minibuffer-local-completion-map (current-global-map))
(icicle-remap 'negative-argument  'icicle-negative-argument  ; `M--'
              minibuffer-local-completion-map (current-global-map))
(icicle-remap 'digit-argument     'icicle-digit-argument     ; `C-9'
              minibuffer-local-completion-map (current-global-map))

(defvar icicle-minibuffer-message-ok-p t)

Together with the code I sent before, this provides a full test case. It works
in Emacs 22 and doesn't work in Emacs 23.

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

[-- Attachment #1: Type: text/plain, Size: 2582 bytes --]

The problem appears to be with `sit-for', not with `minibuffer-message'. More
specifically, the call to `input-pending-p' in `sit-for' does not act as it
should. Dunno if `input-pending-p' is the problem generally, or just in this
context.

Attached is a small, simple, self-contained file that reproduces the problem.
Just load it, then `M-x C-u sssss'. The `s' keystrokes are ignored while the
`sit-for' delay is waited for.

In all cases except `icicle-universal-argument', the original command is called,
followed by a call to `message' and then `sit-for'.

`icicle-universal-argument' is identical to `universal-argument' except:

* It too calls `message' followed by `sit-for'.
* It uses `icicle-ensure-overriding-map-is-bound', not
`ensure-overriding-map-is-bound'. The difference is that the former sets
`overriding-terminal-local-map' to `icicle-universal-argument-map', not
`universal-argument-map'.

The difference between those two maps is that the `icicle-*' commands are used
in place of the originals (and each `icicle-*' command calls `message' followed
by `sit-for').

So all of the changes from the vanilla Emacs code amount to the same trivial
change: Add a call to `message' and `sit-for' after the vanilla command behavior
is finished. There are no other changes.

`sit-for' seems to be the problem, but there is no change in the code for
`sit-for' from Emacs 22 and Emacs 23 (where the bug first appears). Perhaps
there is a relevant change elsewhere that has to do with these keymaps or with
input events? 

Debugging `sit-for' a bit indicates that `input-pending-p' does indeed return
nil when it should return non-nil in this context (after user input). The final
`cond' branch is taken in the `sit-for' code, instead of the `input-pending-p'
branch.

There is no change in the C source code for `input-pending-p' itself, between
Emacs 22 and 23. But it tests several global vars in order to do its thing, so
perhaps the bug was introduced by changing the value of one of those vars. The
vars are `unread-command-events', `unread-command-char',
`unread-post-input-method-events', and `unread-input-method-events'. That seems
likely.

Note this in the doc string of `input-pending-p': "Actually, the value is nil
only if we can be ***sure*** that no input is available; if there is a doubt,
the value is t."

Obviously, we are by no means respecting that declared conservative behavior. It
is returning nil (in this case) even when we should not be sure that no input is
available - even when input is in fact available.

I hope this bug will be fixed. Thx.

[-- Attachment #2: bug-5923-emacs-4.el --]
[-- Type: application/octet-stream, Size: 3264 bytes --]

(defvar icicle-universal-argument-map
  (let ((map  (make-sparse-keymap)))
    (define-key map [t] 'icicle-universal-argument-other-key)
    (define-key map (vector meta-prefix-char t) 'icicle-universal-argument-other-key)
    (define-key map [switch-frame] nil)
    (define-key map [?\C-u] 'icicle-universal-argument-more)
    (define-key map [?-] 'icicle-universal-argument-minus)
    (define-key map [?0] 'icicle-digit-argument)
    (define-key map [?1] 'icicle-digit-argument)
    (define-key map [?2] 'icicle-digit-argument)
    (define-key map [?3] 'icicle-digit-argument)
    (define-key map [?4] 'icicle-digit-argument)
    (define-key map [?5] 'icicle-digit-argument)
    (define-key map [?6] 'icicle-digit-argument)
    (define-key map [?7] 'icicle-digit-argument)
    (define-key map [?8] 'icicle-digit-argument)
    (define-key map [?9] 'icicle-digit-argument)
    (define-key map [kp-0] 'icicle-digit-argument)
    (define-key map [kp-1] 'icicle-digit-argument)
    (define-key map [kp-2] 'icicle-digit-argument)
    (define-key map [kp-3] 'icicle-digit-argument)
    (define-key map [kp-4] 'icicle-digit-argument)
    (define-key map [kp-5] 'icicle-digit-argument)
    (define-key map [kp-6] 'icicle-digit-argument)
    (define-key map [kp-7] 'icicle-digit-argument)
    (define-key map [kp-8] 'icicle-digit-argument)
    (define-key map [kp-9] 'icicle-digit-argument)
    (define-key map [kp-subtract] 'icicle-universal-argument-minus)
    map)
  "...")

(defun icicle-universal-argument ()
  (interactive)
  (setq prefix-arg                     (list 4)
        universal-argument-num-events  (length (this-command-keys)))
  (icicle-ensure-overriding-map-is-bound)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-digit-argument (arg)
  (interactive "P")
  (digit-argument)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-negative-argument (arg)
  (interactive "P")
  (negative-argument arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-more (arg)
  (interactive "P")
  (universal-argument-more arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-other-key (arg)
  (interactive "P")
  (universal-argument-other-key arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-minus (arg)
  (interactive "P")
  (universal-argument-minus arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-ensure-overriding-map-is-bound ()
  (unless overriding-map-is-bound
    (setq saved-overriding-map           overriding-terminal-local-map
          overriding-terminal-local-map  icicle-universal-argument-map
          overriding-map-is-bound        t)))

(defun icicle-msg-maybe-in-minibuffer (format-string &rest args)
  (apply #'message format-string args)
  (sit-for 1))

(define-key minibuffer-local-completion-map [remap universal-argument]
  'icicle-universal-argument)
(define-key minibuffer-local-completion-map [remap negative-argument]
  'icicle-negative-argument)
(define-key minibuffer-local-completion-map [remap digit-argument]
  'icicle-digit-argument)

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> Debugging `sit-for' a bit indicates that `input-pending-p' 
> does indeed return nil when it should return non-nil in this
> context (after user input). The final `cond' branch is taken
> in the `sit-for' code, instead of the `input-pending-p' branch.

Actually, the problem is here in `sit-for':

(let ((read (read-event nil nil seconds)))
      (or (null read)
	  (progn
	    ;; If last command was a prefix arg, e.g. C-u, push this event onto
	    ;; unread-command-events as (t . EVENT) so it will be added to
	    ;; this-command-keys by read-key-sequence.
	    (if (eq overriding-terminal-local-map universal-argument-map)
		(setq read (cons t read)))
	    (push read unread-command-events)
	    nil))))))

Since the value of `overriding-terminal-local-map' is not
`universal-argument-map' in my case, it fails to treat any input received
properly.

Since this needs to span several commands (digit-argument etc.), I can't just
bind `overriding-terminal-local-map' instead of setting it. I guess my options
are to either set it to my map and later unset it or just redefine `sit-for' so
that it uses a test like this instead:

(if (memq overriding-terminal-local-map
          '(universal-argument-map icicle-universal-argument-map))
    (setq read (cons t read)))

Or redefine `universal-argument-map' to use commands that act differently
depending on the mode etc.

Better suggestions are welcome. None that I've thought of so far are appealing.

How about using a different kind of test in the vanilla code, one that would
give users some flexibility here, instead of hard-coding an eq test against a
specific keymap?

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> Actually, the problem is here in `sit-for':
>
> (let ((read (read-event nil nil seconds)))
>       (or (null read)
> 	  (progn
> 	    (if (eq overriding-terminal-local-map 
>                 universal-argument-map)
> 		(setq read (cons t read)))
> 	    (push read unread-command-events)
> 	    nil))))))
> 
> Since the value of `overriding-terminal-local-map' is not
> `universal-argument-map' in my case, it fails to treat any 
> input received properly.

However, something else must be going on also, because the sit-for code is
identical for Emacs 22, and I don't see the bug in Emacs 22.

bug#5923: 23.1.95; minibuffer-message discards input events

[Stefan Monnier]

>> Actually, the problem is here in `sit-for':
>> 
>> (let ((read (read-event nil nil seconds)))
>> (or (null read)
>> (progn
>> (if (eq overriding-terminal-local-map 
>> universal-argument-map)
>> (setq read (cons t read)))
>> (push read unread-command-events)
>> nil))))))
>> 
>> Since the value of `overriding-terminal-local-map' is not
>> `universal-argument-map' in my case, it fails to treat any 
>> input received properly.

> However, something else must be going on also, because the sit-for code is
> identical for Emacs 22, and I don't see the bug in Emacs 22.

Thank you for your efforts digging into this bug.  I must say I know
even less than you do about those parts of the code.  It's clearly too
intricate for its own good, but I don't know how to streamline it.


        Stefan

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> >> Actually, the problem is here in `sit-for':
> >> (let ((read (read-event nil nil seconds)))
> >> (or (null read)
> >> (progn
> >> (if (eq overriding-terminal-local-map 
> >> universal-argument-map)
> >> (setq read (cons t read)))
> >> (push read unread-command-events)
> >> nil))))))
> >> 
> >> Since the value of `overriding-terminal-local-map' is not
> >> `universal-argument-map' in my case, it fails to treat any 
> >> input received properly.
> 
> > However, something else must be going on also, because the 
> > sit-for code is identical for Emacs 22, and I don't see the
> > bug in Emacs 22.
> 
> Thank you for your efforts digging into this bug.  I must say I know
> even less than you do about those parts of the code.  It's clearly too
> intricate for its own good, but I don't know how to streamline it.

Bummer; I'm sorry to hear that, as I had hoped for a fix.

Who wrote the code?  Richard perhaps?
Can't we get someone to understand it and DTRT?

This is after all a regression from Emacs 22.
Can't someone investigate to find out what change
really introduced the regression?

It doesn't seem right that development can break things and then just say that
they can't be fixed because the code is too hard to understand.  I understand
that development of new features and fixing of bugs can sometimes introduce
bugs, including regressions.  But I was hoping this could be fixed (restored).

C-u is a pretty basic part of Emacs.  Seems like this should be fixed before we
worry about adding more bells and whistles to Emacs.

bug#5923: 23.1.95; minibuffer-message discards input events

[npostavs]

"Drew Adams" <drew.adams@oracle.com> writes:

>> Actually, the problem is here in `sit-for':
>>
>> (let ((read (read-event nil nil seconds)))
>>       (or (null read)
>> 	  (progn
>> 	    (if (eq overriding-terminal-local-map 
>>                 universal-argument-map)
>> 		(setq read (cons t read)))
>> 	    (push read unread-command-events)
>> 	    nil))))))
>> 
>> Since the value of `overriding-terminal-local-map' is not
>> `universal-argument-map' in my case, it fails to treat any 
>> input received properly.
>
> However, something else must be going on also, because the sit-for code is
> identical for Emacs 22, and I don't see the bug in Emacs 22.

Is this bug fixed?  The special casing of this in `sit-for' has been
removed meanwhile, but also the overriding-map-is-bound and
universal-argument-other-key have been removed, so it's hard to run the
code examples in current (version 25) Emacs.

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> >> Actually, the problem is here in `sit-for':
> >>
> >> (let ((read (read-event nil nil seconds)))
> >>       (or (null read)
> >> 	  (progn
> >> 	    (if (eq overriding-terminal-local-map
> >>                 universal-argument-map)
> >> 		(setq read (cons t read)))
> >> 	    (push read unread-command-events)
> >> 	    nil))))))
> >>
> >> Since the value of `overriding-terminal-local-map' is not
> >> `universal-argument-map' in my case, it fails to treat any
> >> input received properly.
> >
> > However, something else must be going on also, because the sit-for code is
> > identical for Emacs 22, and I don't see the bug in Emacs 22.
> 
> Is this bug fixed?  The special casing of this in `sit-for' has been
> removed meanwhile, but also the overriding-map-is-bound and
> universal-argument-other-key have been removed, so it's hard to run the
> code examples in current (version 25) Emacs.

I don't know if it is fixed, and I don't have the time to dig into
this again.  (What you quoted was by no means the last part of this
bug thread, BTW.)  I would _not_ assume that this bug has been fixed.
I think someone would need to dig into this, to debug it.

I provided a self-contained repro recipe based on the code at
the time, but it is no longer sufficient because variable
`overriding-map-is-bound' no longer exists, and the underlying
code has changed.

You can close the bug, but I'm not confident it has been fixed.

bug#5923: 23.1.95; minibuffer-message discards input events

[Noam Postavsky]

On Tue, Jul 5, 2016 at 10:19 AM, Drew Adams <drew.adams@oracle.com> wrote:
> I don't know if it is fixed, and I don't have the time to dig into
> this again.  (What you quoted was by no means the last part of this
> bug thread, BTW.)  I would _not_ assume that this bug has been fixed.
> I think someone would need to dig into this, to debug it.
>
> I provided a self-contained repro recipe based on the code at
> the time, but it is no longer sufficient because variable
> `overriding-map-is-bound' no longer exists, and the underlying
> code has changed.

It looks like the examples are reductions from icicles code. Do you
still have problems with this in icicles? A repro recipe along the
lines of "load icicles-foo.el and icicles-bar.el then M-x C-u <etc
etc>..." would be useful. Otherwise I would propose to close the bug
as unreproducible.

PS Removing bug #3938 from cc list, as it seems to cause duplicated
emails (I hadn't noticed that I had sent my previous message to both
merged bugs, seems to be a misfeature of gnus' ephemeral bug groups)

bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> > I don't know if it is fixed, and I don't have the time to dig into
> > this again.  (What you quoted was by no means the last part of this
> > bug thread, BTW.)  I would _not_ assume that this bug has been fixed.
> > I think someone would need to dig into this, to debug it.
> >
> > I provided a self-contained repro recipe based on the code at
> > the time, but it is no longer sufficient because variable
> > `overriding-map-is-bound' no longer exists, and the underlying
> > code has changed.
> 
> It looks like the examples are reductions from icicles code. Do you
> still have problems with this in icicles?  A repro recipe along the
> lines of "load icicles-foo.el and icicles-bar.el then M-x C-u <etc
> etc>..." would be useful. Otherwise I would propose to close the bug
> as unreproducible.

The repro example I gave was self-contained.  It had nothing to
do with Icicles, in spite of some of the function names.

And as I said:

> > I don't know if it is fixed, and I don't have the time to dig
> > into this again.

As I said:

> > I would _not_ assume that this bug has been fixed.  I think
> > someone would need to dig into this, to debug it.
> > You can close the bug, but I'm not confident it has been fixed.

bug#5923: 23.1.95; minibuffer-message discards input events

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]

tag 5923 + confirmed
found 5923 25.0.95
quit

On Wed, Jul 6, 2016 at 9:59 AM, Drew Adams <drew.adams@oracle.com> wrote:
>
> The repro example I gave was self-contained.  It had nothing to
> do with Icicles, in spite of some of the function names.

Ok, had another look, it's actually not so hard to update it for newer
Emacs. Seems the bug is still existing.

emacs -Q -l bug-5923-emacs-25.el
M-x C-u C-f
See that "prefix (4)" is displayed for 2 seconds until
`describe-function' finally gets executed.

[-- Attachment #2: bug-5923-emacs-25.el --]
[-- Type: text/x-emacs-lisp, Size: 4089 bytes --]

(defvar icicle-universal-argument-map
  (let ((map  (make-sparse-keymap)))
    (when (fboundp 'universal-argument-other-key) ; up to 24.3
      (define-key map [t] 'icicle-universal-argument-other-key)
      (define-key map (vector meta-prefix-char t) 'icicle-universal-argument-other-key))
    (define-key map [switch-frame] nil)
    (define-key map [?\C-u] 'icicle-universal-argument-more)
    (define-key map [?-] 'icicle-universal-argument-minus)
    (define-key map [?0] 'icicle-digit-argument)
    (define-key map [?1] 'icicle-digit-argument)
    (define-key map [?2] 'icicle-digit-argument)
    (define-key map [?3] 'icicle-digit-argument)
    (define-key map [?4] 'icicle-digit-argument)
    (define-key map [?5] 'icicle-digit-argument)
    (define-key map [?6] 'icicle-digit-argument)
    (define-key map [?7] 'icicle-digit-argument)
    (define-key map [?8] 'icicle-digit-argument)
    (define-key map [?9] 'icicle-digit-argument)
    (define-key map [kp-0] 'icicle-digit-argument)
    (define-key map [kp-1] 'icicle-digit-argument)
    (define-key map [kp-2] 'icicle-digit-argument)
    (define-key map [kp-3] 'icicle-digit-argument)
    (define-key map [kp-4] 'icicle-digit-argument)
    (define-key map [kp-5] 'icicle-digit-argument)
    (define-key map [kp-6] 'icicle-digit-argument)
    (define-key map [kp-7] 'icicle-digit-argument)
    (define-key map [kp-8] 'icicle-digit-argument)
    (define-key map [kp-9] 'icicle-digit-argument)
    (define-key map [kp-subtract] 'icicle-universal-argument-minus)
    map)
  "...")

(defun icicle-universal-argument ()
  (interactive)
  (setq prefix-arg                     (list 4)
        universal-argument-num-events  (length (this-command-keys)))
  (icicle-ensure-overriding-map-is-bound)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-digit-argument (arg)
  (interactive "P")
  (digit-argument)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-negative-argument (arg)
  (interactive "P")
  (negative-argument arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-more (arg)
  (interactive "P")
  (universal-argument-more arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-other-key (arg)
  (interactive "P")
  (universal-argument-other-key arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

(defun icicle-universal-argument-minus (arg)
  (interactive "P")
  (universal-argument-minus arg)
  (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))

;; (defun icicle-ensure-overriding-map-is-bound ()
;;   (unless overriding-map-is-bound
;;     (setq saved-overriding-map           overriding-terminal-local-map
;;           overriding-terminal-local-map  icicle-universal-argument-map
;;           overriding-map-is-bound        t)))

(defun icicle-ensure-overriding-map-is-bound ()
  "Set `overriding-terminal-local-map' to `icicle-universal-argument-map'."
  (cond
   ((boundp 'universal-argument-map) ; Emacs 24+
    (set-transient-map icicle-universal-argument-map nil
                       (lambda () (icicle-msg-maybe-in-minibuffer "prefix %S" prefix-arg))))
   ((not (boundp 'overriding-map-is-bound)) ; Emacs 20, 21.
    (setq overriding-terminal-local-map  icicle-universal-argument-map))
   ((not overriding-terminal-local-map) ; Emacs 22+.
    (setq saved-overriding-map           overriding-terminal-local-map
          overriding-terminal-local-map  icicle-universal-argument-map
          overriding-map-is-bound        t))))

(defun icicle-msg-maybe-in-minibuffer (format-string &rest args)
  (apply #'message format-string args)
  (sit-for 1))

(define-key minibuffer-local-completion-map [remap universal-argument]
  'icicle-universal-argument)
(define-key minibuffer-local-completion-map [remap negative-argument]
  'icicle-negative-argument)
(define-key minibuffer-local-completion-map [remap digit-argument]
  'icicle-digit-argument)

(defun test2 () (interactive) (describe-function '+))
 
(define-key minibuffer-local-completion-map "\C-f" 'test2)

bug#5923: 23.1.95; minibuffer-message discards input events

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Wed, 6 Jul 2016 10:55:38 -0400
> Cc: 5923@debbugs.gnu.org
> 
> Ok, had another look, it's actually not so hard to update it for newer
> Emacs. Seems the bug is still existing.
> 
> emacs -Q -l bug-5923-emacs-25.el
> M-x C-u C-f
> See that "prefix (4)" is displayed for 2 seconds until
> `describe-function' finally gets executed.

And I arrived at the exactly opposite conclusion, see my other
message.

bug#5923: 23.1.95; minibuffer-message discards input events

[Noam Postavsky]

tag 5923 - confirmed
notfound 5932 25.0.95
fixed 5932
quit

On Wed, Jul 6, 2016 at 11:05 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Noam Postavsky <npostavs@users.sourceforge.net>
>> Date: Wed, 6 Jul 2016 10:55:38 -0400
>> Cc: 5923@debbugs.gnu.org
>>
>> Ok, had another look, it's actually not so hard to update it for newer
>> Emacs. Seems the bug is still existing.
>>
>> emacs -Q -l bug-5923-emacs-25.el
>> M-x C-u C-f
>> See that "prefix (4)" is displayed for 2 seconds until
>> `describe-function' finally gets executed.
>
> And I arrived at the exactly opposite conclusion, see my other
> message.

Yes, your version works. Probably I made a mistake in my updates,
tentatively marking bug as fixed.

bug#5923: 23.1.95; minibuffer-message discards input events

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Wed, 6 Jul 2016 11:30:16 -0400
> Cc: Drew Adams <drew.adams@oracle.com>, 5923@debbugs.gnu.org
> 
> >> emacs -Q -l bug-5923-emacs-25.el
> >> M-x C-u C-f
> >> See that "prefix (4)" is displayed for 2 seconds until
> >> `describe-function' finally gets executed.
> >
> > And I arrived at the exactly opposite conclusion, see my other
> > message.
> 
> Yes, your version works. Probably I made a mistake in my updates,
> tentatively marking bug as fixed.

Thanks.  I think, given Drew's response, we can close this bug.

bug#3938: bug#5923: 23.1.95; minibuffer-message discards input events

[Eli Zaretskii]

[-- Attachment #1: Type: text/plain, Size: 1872 bytes --]

> Date: Tue, 5 Jul 2016 14:19:28 +0000 (UTC)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 5923@debbugs.gnu.org, 3938@debbugs.gnu.org
> 
> > >> Actually, the problem is here in `sit-for':
> > >>
> > >> (let ((read (read-event nil nil seconds)))
> > >>       (or (null read)
> > >> 	  (progn
> > >> 	    (if (eq overriding-terminal-local-map
> > >>                 universal-argument-map)
> > >> 		(setq read (cons t read)))
> > >> 	    (push read unread-command-events)
> > >> 	    nil))))))
> > >>
> > >> Since the value of `overriding-terminal-local-map' is not
> > >> `universal-argument-map' in my case, it fails to treat any
> > >> input received properly.
> > >
> > > However, something else must be going on also, because the sit-for code is
> > > identical for Emacs 22, and I don't see the bug in Emacs 22.
> > 
> > Is this bug fixed?  The special casing of this in `sit-for' has been
> > removed meanwhile, but also the overriding-map-is-bound and
> > universal-argument-other-key have been removed, so it's hard to run the
> > code examples in current (version 25) Emacs.
> 
> I don't know if it is fixed, and I don't have the time to dig into
> this again.  (What you quoted was by no means the last part of this
> bug thread, BTW.)  I would _not_ assume that this bug has been fixed.
> I think someone would need to dig into this, to debug it.

One could hope for a more cooperative response, I think.  It took me
all of 5 minutes, without knowing anything about icicles and very
little about the new implementation of universal-argument, to convert
your test file to the current sources (see the attached).  With it, I
convinced myself that the bug is indeed fixed: typing the first 's'
interrupts the wait immediately, and displays "ssss" in the echo area.

Please verify that my changes to the test file are valid and the bug
is indeed fixed.

Thanks.


[-- Attachment #2: bug-5923-emacs-25.el --]
[-- Type: application/emacs-lisp, Size: 3152 bytes --]

bug#5923: bug#3938: bug#5923: 23.1.95; minibuffer-message discards input events

[Drew Adams]

> > > Is this bug fixed?  The special casing of this in `sit-for' has been
> > > removed meanwhile, but also the overriding-map-is-bound and
> > > universal-argument-other-key have been removed, so it's hard to run the
> > > code examples in current (version 25) Emacs.
> >
> > I don't know if it is fixed, and I don't have the time to dig into
> > this again.  (What you quoted was by no means the last part of this
> > bug thread, BTW.)  I would _not_ assume that this bug has been fixed.
> > I think someone would need to dig into this, to debug it.
> 
> One could hope for a more cooperative response, I think. 

Ditto.  Time passes...

> It took me all of 5 minutes, without knowing anything about icicles and very
> little about the new implementation of universal-argument, to convert
> your test file to the current sources (see the attached).  With it, I
> convinced myself that the bug is indeed fixed: typing the first 's'
> interrupts the wait immediately, and displays "ssss" in the echo area.

I had already done similarly, FWIW.  To me it does not constitute a proof
that the bug is fixed.  If it convinces you, fine; close the bug, as I said.

> Please verify that my changes to the test file are valid and the bug
> is indeed fixed.

Sorry, I don't have the time to do that.  And as I said, IMO this
alone doesn't convince me that the bug is fixed.  It might be fixed;
I don't know.  I hope it is.

If you can convince yourself in 5 minutes that it is fixed, great.

The bug (#3938) was filed almost exactly 7 years ago, with a very short
recipe to reproduce, starting from `emacs -Q'.  Likewise, for the merged
bug (#5923) - 6 years ago, with other emacs -Q recipes
(http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5923#23, http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5923#26).

Responses from Stefan at the time:

 But to tell you the truth the handling of this-command-keys and
 universal-argument prefix is much too delicate for me to understand it...

 Thank you for your efforts digging into this bug.  I must say I know
 even less than you do about those parts of the code.  It's clearly too
 intricate for its own good, but I don't know how to streamline it.

That was it.  It died on the vine in 2010.  If it somehow got fixed
accidentally in the interim, great.

"One could _hope_ for a more cooperative response, I think."
But all's well that ends well.  (Assuming it is ended, and well.)

bug#5923: bug#3938: bug#5923: 23.1.95; minibuffer-message discards input events

[Eli Zaretskii]

> Date: Wed, 6 Jul 2016 15:10:35 +0000 (UTC)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: npostavs@users.sourceforge.net, 5923@debbugs.gnu.org, 3938@debbugs.gnu.org
> 
> > It took me all of 5 minutes, without knowing anything about icicles and very
> > little about the new implementation of universal-argument, to convert
> > your test file to the current sources (see the attached).  With it, I
> > convinced myself that the bug is indeed fixed: typing the first 's'
> > interrupts the wait immediately, and displays "ssss" in the echo area.
> 
> I had already done similarly, FWIW.  To me it does not constitute a proof
> that the bug is fixed.

??? Are you saying that the recipe doesn't exhibit the bug?  If it
does, and now the recipe behaves correctly, what else could be there
that prevents us from concluding the bug is fixed?

> If you can convince yourself in 5 minutes that it is fixed, great.

The behavior which was incorrect when you reported it, is correct
now.  What else do we need to examine?

I will close the bug, unless Noam convinces me that my testing is
incorrect, while his is.