Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Alan Mackenzie]

Hello, Emacs.

In do-after-load-evaluation, we have the following, near the end:

          (run-with-timer 0 nil
                          (lambda (msg)
                            (message "%s" msg))
                          msg)

.  run-with-timer is being used to run message once, immediately.  Why
not just call message?

The reason I ask is that whilst loading my desktop, the prompt

        Please type y, n, or !, or C-v to scroll:

, asking me whether I want dangerous local variables to be loaded, is
getting obscured by the less important

        Package cl is deprecated

, and I have to know that I'm expected to respond to this obscured
prompt.  This has been happening to me only for a short time, at least a
week, but probably less than a month.

This last message about cl being deprecated is being output by the
strange run-with-timer.  If I replace the run-with-timer form with a
straight message call, I see the prompt from hack-local-variables.

What is going on, here?  Is the run-with-timer mechanism being used
deliberately to make the "deprecated" message prevail over other
messages?  If so, perhaps it shouldn't be.

<A bit later>

I've searched the git log, and found that cl was moved into lisp/obsolete
on 2019-08-06.  At the same time, some change was made to output the
"deprecated" message in do-after-load-evaluation.

The current situation seems unsatisfactory; the prompt from
hack-local-variables is more important than the deprecation message, and
shouldn't be obscured by it.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Stefan Monnier]

>           (run-with-timer 0 nil
>                           (lambda (msg)
>                             (message "%s" msg))
>                           msg)
>
> run-with-timer is being used to run message once, immediately.
> Why not just call message?

Good question.  It's been that way since

    commit 5766c380eec20a19844253cbb511922b6c70fc0b
    Author: Stefan Monnier <monnier@iro.umontreal.ca>
    Date:   Sat Sep 12 03:35:40 2009 +0000
    
        * lread.c (Fload): Don't output a message after loading an obsolete
        package any more (done in Lisp now).
        * subr.el (do-after-load-evaluation): Warn the user after loading an
        obsolete package.

but I can't see any trace of an explanation nor can I find it in my memory.

> What is going on, here?  Is the run-with-timer mechanism being used
> deliberately to make the "deprecated" message prevail over other
> messages?

That's my best guess, yes.

> The current situation seems unsatisfactory; the prompt from
> hack-local-variables is more important than the deprecation message,
> and shouldn't be obscured by it.

It's a kind of general problem with messages, indeed.


        Stefan

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The current situation seems unsatisfactory; the prompt from
>> hack-local-variables is more important than the deprecation message,
>> and shouldn't be obscured by it.
>
> It's a kind of general problem with messages, indeed.

Yup.  There are several bug reports on this (general) issue of
asynchronous messages hiding queries.  One suggestion has been to extend
the echo area in these instances, showing the message (perhaps) below
the prompt.  Another is to have the prompt re-assert itself after a
timeout, making the message disappear.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Alan Mackenzie]

Hello, Stefan.

On Sat, Oct 26, 2019 at 08:41:27 -0400, Stefan Monnier wrote:
> >           (run-with-timer 0 nil
> >                           (lambda (msg)
> >                             (message "%s" msg))
> >                           msg)

> > run-with-timer is being used to run message once, immediately.
> > Why not just call message?

> Good question.  It's been that way since

>     commit 5766c380eec20a19844253cbb511922b6c70fc0b
>     Author: Stefan Monnier <monnier@iro.umontreal.ca>
>     Date:   Sat Sep 12 03:35:40 2009 +0000
    
>         * lread.c (Fload): Don't output a message after loading an obsolete
>         package any more (done in Lisp now).
>         * subr.el (do-after-load-evaluation): Warn the user after loading an
>         obsolete package.

> but I can't see any trace of an explanation nor can I find it in my memory.

> > What is going on, here?  Is the run-with-timer mechanism being used
> > deliberately to make the "deprecated" message prevail over other
> > messages?

> That's my best guess, yes.

I think I can see what's happening, now.  The (run-with-timer 0 nil ...)
mechanism doesn't trigger its function until Emacs is waiting for input.
This will be in the minibuffer routines, just after the y-or-n-p has
displayed its prompt string and tries to read a character.  So the
(run-with-timer ...) message will overwrite any prompt string.

I have counted 11 occurrences of run-with-timer or run-at-time with a
zero delay in them.

So, the (run-with-timer ...) seems indeed to be being used to raise the
"priority" of its message.  Maybe a simple message call should be used
here instead.

> > The current situation seems unsatisfactory; the prompt from
> > hack-local-variables is more important than the deprecation message,
> > and shouldn't be obscured by it.

> It's a kind of general problem with messages, indeed.

Yes.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]

On Sat, Oct 26, 2019 at 2:42 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:
> > What is going on, here?  Is the run-with-timer mechanism being used
> > deliberately to make the "deprecated" message prevail over other
> > messages?
>
> That's my best guess, yes.

Wouldn't that kind of message be a perfect candidate for a warning instead?
Warnings are underused in Emacs now, but they have advantages, like the
ability to filter them by level/priority.

The warnings are shown in its own buffer (and window), but by using
fit-to-buffer and the new window machinery we could perhaps make them less
jarring and still hard to miss.

[-- Attachment #2: Type: text/html, Size: 794 bytes --]

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Stefan Monnier]

>> It's a kind of general problem with messages, indeed.
> Yup.  There are several bug reports on this (general) issue of
> asynchronous messages hiding queries.

It goes further than that: it's also messages hiding each other, whether
asynchronous or not.


        Stefan

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[HaiJun Zhang]

[-- Attachment #1: Type: text/plain, Size: 239 bytes --]

There is a variable `minibuffer-message-timeout’ which seems to be related with this issue. From my understanding, if it is set to 2, the message will disappears after 2s and the prompt restores. But it doesn’t work. See bug#34614.


[-- Attachment #2: Type: text/html, Size: 532 bytes --]

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juri Linkov]

> The reason I ask is that whilst loading my desktop, the prompt
>
>         Please type y, n, or !, or C-v to scroll:
>
> , asking me whether I want dangerous local variables to be loaded, is
> getting obscured by the less important
>
>         Package cl is deprecated

For two months I'm seeing the same problem with the message
"Package cl is deprecated" obscuring the y-or-n-p prompt:

  Warning: desktop file appears to be in use.  Use it anyway?

then I type 'y' blindly without seeing the obscured prompt only because
I expect this prompt to appear, but this can't be a general solution.

Maybe it's possible to delay warnings until desktop is loaded?

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juri Linkov]

>>> The current situation seems unsatisfactory; the prompt from
>>> hack-local-variables is more important than the deprecation message,
>>> and shouldn't be obscured by it.
>>
>> It's a kind of general problem with messages, indeed.
>
> Yup.  There are several bug reports on this (general) issue of
> asynchronous messages hiding queries.  One suggestion has been to extend
> the echo area in these instances, showing the message (perhaps) below
> the prompt.  Another is to have the prompt re-assert itself after a
> timeout, making the message disappear.

These bug reports are fixed now when messages are displayed in the
minibuffer, but the problem still exists for the y-or-n-p prompt.

Why y-or-n-p doesn't use normal minibuffer functions?

It can finely use read-from-minibuffer with a keymap where
'y' and 'n' keys are bound to minibuffer-exiting commands.

Then history commands will be available for free, so you don't need
to re-implement minibuffer history commands with `read-char-with-history'.

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juri Linkov]

> Why y-or-n-p doesn't use normal minibuffer functions?
>
> It can finely use read-from-minibuffer with a keymap where
> 'y' and 'n' keys are bound to minibuffer-exiting commands.
>
> Then history commands will be available for free, so you don't need
> to re-implement minibuffer history commands with `read-char-with-history'.

I meant something like

(defvar y-n-p-map
  (let ((map (make-sparse-keymap)))
    (set-keymap-parent map minibuffer-local-map)
    (define-key map "y" (lambda ()
                          (interactive)
                          (delete-minibuffer-contents)
                          (insert "y")
                          (exit-minibuffer)))
    (define-key map "n" (lambda ()
                          (interactive)
                          (delete-minibuffer-contents)
                          (insert "n")
                          (exit-minibuffer)))
    map))

(defun y-n-p (prompt)
  (read-from-minibuffer prompt nil y-n-p-map nil 'y-n-p-history))

(y-n-p "Please type y or n: ")

Then all history commands are available, e.g. 'M-p' or the up arrow.

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Stefan Monnier]

> Why y-or-n-p doesn't use normal minibuffer functions?

Fine by me.


        Stefan

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Sun, 27 Oct 2019 23:51:51 +0200
> Cc: emacs-devel@gnu.org
> 
> Maybe it's possible to delay warnings until desktop is loaded?

We already do, perhaps something there is not working?

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[martin rudalics]

 > I meant something like

It would certainly remove the present confusion wrt usage of echo area
and minibuffer window when asking questions.  Hence I'm all for such a
change.

martin

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Lars Ingebrigtsen]

Juri Linkov <juri@linkov.net> writes:

>> Yup.  There are several bug reports on this (general) issue of
>> asynchronous messages hiding queries.  One suggestion has been to extend
>> the echo area in these instances, showing the message (perhaps) below
>> the prompt.  Another is to have the prompt re-assert itself after a
>> timeout, making the message disappear.
>
> These bug reports are fixed now when messages are displayed in the
> minibuffer, but the problem still exists for the y-or-n-p prompt.

So read-from-minibuffer reasserts the query if an asynchronous message
hides it?  Then that sounds like a good idea.  On the other hand,
y-or-n-p could just call read-char/event with a timeout and do the same
thing. 

> Why y-or-n-p doesn't use normal minibuffer functions?
>
> It can finely use read-from-minibuffer with a keymap where
> 'y' and 'n' keys are bound to minibuffer-exiting commands.
>
> Then history commands will be available for free, so you don't need
> to re-implement minibuffer history commands with `read-char-with-history'.

I think having a history for y-or-n-p doesn't sound very useful?
Hitting `M-p' doesn't to get to the previous answer just sounds
confusing to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juri Linkov]

>> These bug reports are fixed now when messages are displayed in the
>> minibuffer, but the problem still exists for the y-or-n-p prompt.
>
> So read-from-minibuffer reasserts the query if an asynchronous message
> hides it?

An asynchronous message doesn't hide the minibuffer.  It's displayed
at the end of the minibuffer text.

>> Why y-or-n-p doesn't use normal minibuffer functions?
>>
>> It can finely use read-from-minibuffer with a keymap where
>> 'y' and 'n' keys are bound to minibuffer-exiting commands.
>>
>> Then history commands will be available for free, so you don't need
>> to re-implement minibuffer history commands with `read-char-with-history'.
>
> I think having a history for y-or-n-p doesn't sound very useful?
> Hitting `M-p' doesn't to get to the previous answer just sounds
> confusing to me.

Please try the example I sent earlier.  It feels quite naturally
typing 'M-p RET' to repeat a previous y/n answer.

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Stefan Kangas]

Juri Linkov <juri@linkov.net> writes:

> > I think having a history for y-or-n-p doesn't sound very useful?
> > Hitting `M-p' doesn't to get to the previous answer just sounds
> > confusing to me.
>
> Please try the example I sent earlier.  It feels quite naturally
> typing 'M-p RET' to repeat a previous y/n answer.

I think this is a misfeature.  Imagine a user answering "Do you want
to save important file X before closing?" and is used to always having
"y" in her history.  This time, "n" was the first item in history
because of some previous but now forgotten invocation the night
before.  But the user hits "M-p RET" by habit, expecting that to mean
"y", and ends up discarding important work.

Using this puts a cognitive load on the user if she doesn't want to
make mistakes.

Best regards,
Stefan Kangas

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Lars Ingebrigtsen]

Juri Linkov <juri@linkov.net> writes:

>> So read-from-minibuffer reasserts the query if an asynchronous message
>> hides it?
>
> An asynchronous message doesn't hide the minibuffer.  It's displayed
> at the end of the minibuffer text.

The original bug report was about:

(progn
  (run-at-time 2 nil (lambda () (message "foo")))
  (y-or-n-p "Yes? "))

But this seems to have the same problem?  

(progn
  (run-at-time 2 nil (lambda () (message "foo")))
  (read-from-minibuffer "Yes? "))

When I eval that, "foo" completely hides the prompt -- it's not appended
or reasserted.

>> I think having a history for y-or-n-p doesn't sound very useful?
>> Hitting `M-p' doesn't to get to the previous answer just sounds
>> confusing to me.
>
> Please try the example I sent earlier.  It feels quite naturally
> typing 'M-p RET' to repeat a previous y/n answer.

I played with it a bit, and I'm not very enthusiastic about that.  Like
Stefan K says, it seems error-prone, and I think it would just create
frustration.  And it's a bigger change in the interface than it first
sounds like -- people may be used to hitting <up> as a way to reassert
the prompt, for instance.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juri Linkov]

>> Please try the example I sent earlier.  It feels quite naturally
>> typing 'M-p RET' to repeat a previous y/n answer.
>
> I think this is a misfeature.  Imagine a user answering "Do you want
> to save important file X before closing?" and is used to always having
> "y" in her history.  This time, "n" was the first item in history
> because of some previous but now forgotten invocation the night
> before.  But the user hits "M-p RET" by habit, expecting that to mean
> "y", and ends up discarding important work.
>
> Using this puts a cognitive load on the user if she doesn't want to
> make mistakes.

There are many things in Emacs that might go wrong when used carelessly.

Sometimes I run such dangerous shell commands:

  M-! rm * RET

then later I forget about it in the shell-command history and type:

  M-! M-p RET

The only solution is to be more careful.

With more freedom comes more responsibility.

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juri Linkov]

>> An asynchronous message doesn't hide the minibuffer.  It's displayed
>> at the end of the minibuffer text.
>
> The original bug report was about:
>
> (progn
>   (run-at-time 2 nil (lambda () (message "foo")))
>   (y-or-n-p "Yes? "))
>
> But this seems to have the same problem?
>
> (progn
>   (run-at-time 2 nil (lambda () (message "foo")))
>   (read-from-minibuffer "Yes? "))
>
> When I eval that, "foo" completely hides the prompt -- it's not appended
> or reasserted.

'message' can be easily replaced with 'minibuffer-message',
then it doesn't obscure the minibuffer:

(progn
  (run-at-time 2 nil (lambda () (minibuffer-message "foo")))
  (read-from-minibuffer "Yes? "))

Whereas it obscures the prompt in y-or-n-p:

(progn
  (run-at-time 2 nil (lambda () (minibuffer-message "foo")))
  (y-or-n-p "Yes? "))

>>> I think having a history for y-or-n-p doesn't sound very useful?
>>> Hitting `M-p' doesn't to get to the previous answer just sounds
>>> confusing to me.
>>
>> Please try the example I sent earlier.  It feels quite naturally
>> typing 'M-p RET' to repeat a previous y/n answer.
>
> I played with it a bit, and I'm not very enthusiastic about that.  Like
> Stefan K says, it seems error-prone, and I think it would just create
> frustration.  And it's a bigger change in the interface than it first
> sounds like -- people may be used to hitting <up> as a way to reassert
> the prompt, for instance.

In case of doubt, we could add a new function read-y-or-n-p,
not to replace the implementation of the existing y-or-n-p.

Then like many users already put in init files:

  (fset 'yes-or-no-p 'y-or-n-p)

it will be possible to put:

  (fset 'yes-or-no-p 'read-y-or-n-p)
  (fset 'y-or-n-p 'read-y-or-n-p)

And when it will prove to work better than the current y-or-n-p,
maybe after the next release the implementation of y-or-n-p
could be switched to read-y-or-n-p later.

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Lars Ingebrigtsen]

Juri Linkov <juri@linkov.net> writes:

> 'message' can be easily replaced with 'minibuffer-message',
> then it doesn't obscure the minibuffer:
>
> (progn
>   (run-at-time 2 nil (lambda () (minibuffer-message "foo")))
>   (read-from-minibuffer "Yes? "))

Oh, nice.  I guess whenever we're displaying things from an async
context, we should be using minibuffer-message?

> Whereas it obscures the prompt in y-or-n-p:
>
> (progn
>   (run-at-time 2 nil (lambda () (minibuffer-message "foo")))
>   (y-or-n-p "Yes? "))

Then I'm all for your suggestion to rewrite y-or-n-p with
read-from-minibuffer...  but without a history.  (And yes-or-no-p too,
somehow, I guess.)  (Or fix both of those functions to work the same
with respect to minibuffer-message.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[martin rudalics]

 > Then I'm all for your suggestion to rewrite y-or-n-p with
 > read-from-minibuffer...  but without a history.  (And yes-or-no-p too,
 > somehow, I guess.)

Strongly supported for Emacs 27.

martin

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juri Linkov]

>> 'message' can be easily replaced with 'minibuffer-message',
>> then it doesn't obscure the minibuffer:
>>
>> (progn
>>   (run-at-time 2 nil (lambda () (minibuffer-message "foo")))
>>   (read-from-minibuffer "Yes? "))
>
> Oh, nice.  I guess whenever we're displaying things from an async
> context, we should be using minibuffer-message?

'minibuffer-message' just calls the 'message' when used outside of
the minibuffer.  Maybe the dependency should be reversed so 'message'
could call 'minibuffer-message' when used in the minibuffer.

>> Whereas it obscures the prompt in y-or-n-p:
>>
>> (progn
>>   (run-at-time 2 nil (lambda () (minibuffer-message "foo")))
>>   (y-or-n-p "Yes? "))
>
> Then I'm all for your suggestion to rewrite y-or-n-p with
> read-from-minibuffer...  but without a history.  (And yes-or-no-p too,
> somehow, I guess.)  (Or fix both of those functions to work the same
> with respect to minibuffer-message.)

Without a history is fine.  There is another problem with
backward-compatibility: currently 'y-or-n-p' uses 'query-replace-map'
to translate input characters to y/n answers via act/skip symbols.

Users customize 'query-replace-map' to map more characters to act/skip
symbols, so we need to use 'query-replace-map', but not directly like

  (let ((prompt "Confirm? "))
    (read-from-minibuffer prompt nil query-replace-map))

that obviously fails with

  call-interactively: (wrong-type-argument commandp act)

query-replace-map needs to be translated to another keymap
where the same characters from 'query-replace-map'
run real commands, not intermediate symbols.

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Stefan Monnier]

> query-replace-map needs to be translated to another keymap
> where the same characters from 'query-replace-map'
> run real commands, not intermediate symbols.

E.g.

    (defvar foo-remapping-map
      (let ((map (make-sparse-keymap)))
        (define-key map [remap ask] '...)
        ...
        map))

and then

      ... (make-composed-keymap query-replace-map foo-remapping-map) ..


-- Stefan

Re: Strange use of (run-with-timer 0 nil #'foo args) in do-after-load-evaluation

[Juri Linkov]

>> query-replace-map needs to be translated to another keymap
>> where the same characters from 'query-replace-map'
>> run real commands, not intermediate symbols.
>
> E.g.
>
>     (defvar foo-remapping-map
>       (let ((map (make-sparse-keymap)))
>         (define-key map [remap ask] '...)
>         ...
>         map))
>
> and then
>
>       ... (make-composed-keymap query-replace-map foo-remapping-map) ..

Thanks, I tried this, and everything works fine.
I'll post the patch after more testing.