bug#54371: 29.0.50; read-char does not reset idle timer in some cases

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Ignacio Casso]

Hello,

If I evaluate the following snippet, after 3 seconds Emacs prompts me
for a character and prints nil.

  (defun my-test ()
    (read-char "> " nil nil)
    (message "%s" (current-idle-time)))

  (run-with-timer 3 nil 'my-test)

However, if I evaluate the following snippet instead, where the last
argument of read-char is changed to the number of seconds the prompt
should wait at most for user input, it prints (0 K _ _), where K is 3
seconds plus whatever time in seconds you took to answer the
prompt. This means that right after reading a character from user input
Emacs still thinks it has been idle for a while.

  (defun my-test ()
    (read-char "> " nil 10)
    (message "%s" (current-idle-time)))

  (run-with-timer 3 nil 'my-test)

This is the reason behind a bug that occurs when resolving the running
org clock after some idle time, for which I have seen reports more than
10 years old but not as much discussion as I would expect. This makes me
think that Emacs does not behave like this for most systems and the
problem is particular to my setup. My setup should not be particular in
any sense however, just an Ubuntu 20.04 with default and standard
packages. Emacs' is the only configuration I have ever tweaked in my
machine, and this happens also with "emacs -Q", so it's not that.

Can you please evaluate the second snippet to try to reproduce this in
your machine? If you can't, I will be happy to provide any other
information of my machine that you think may be needed to reproduce
this, or to follow whichever steps you suggest to try to debug this
myself (I would have tried already but all functions involved are
written in C or too low-level).

Regards,

--Ignacio

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Eli Zaretskii]

> From: Ignacio Casso <ignaciocasso@hotmail.com>
> Date: Sun, 13 Mar 2022 00:27:59 +0100
> 
> If I evaluate the following snippet, after 3 seconds Emacs prompts me
> for a character and prints nil.
> 
>   (defun my-test ()
>     (read-char "> " nil nil)
>     (message "%s" (current-idle-time)))
> 
>   (run-with-timer 3 nil 'my-test)
> 
> However, if I evaluate the following snippet instead, where the last
> argument of read-char is changed to the number of seconds the prompt
> should wait at most for user input, it prints (0 K _ _), where K is 3
> seconds plus whatever time in seconds you took to answer the
> prompt. This means that right after reading a character from user input
> Emacs still thinks it has been idle for a while.
> 
>   (defun my-test ()
>     (read-char "> " nil 10)
>     (message "%s" (current-idle-time)))
> 
>   (run-with-timer 3 nil 'my-test)
> 
> This is the reason behind a bug that occurs when resolving the running
> org clock after some idle time, for which I have seen reports more than
> 10 years old but not as much discussion as I would expect. This makes me
> think that Emacs does not behave like this for most systems and the
> problem is particular to my setup. My setup should not be particular in
> any sense however, just an Ubuntu 20.04 with default and standard
> packages. Emacs' is the only configuration I have ever tweaked in my
> machine, and this happens also with "emacs -Q", so it's not that.
> 
> Can you please evaluate the second snippet to try to reproduce this in
> your machine?

I can reproduce this.

However, it sounds like we do this on purpose, to avoid problems with
idle timers that call sit-for.  See the discussion that started here:

  https://lists.gnu.org/archive/html/emacs-devel/2006-08/msg00395.html

The change installed at that time made read_char avoid restarting idle
timers when it is called with a non-nil END_TIME argument.

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Ignacio Casso]

> However, it sounds like we do this on purpose, to avoid problems with
> idle timers that call sit-for.  See the discussion that started here:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2006-08/msg00395.html
>
> The change installed at that time made read_char avoid restarting idle
> timers when it is called with a non-nil END_TIME argument.

I see. I'll try to get this fixed on the code invoking read-char then,
replacing

  (read-char PROMPT nil N)

with

  (let ((char (read-char PROMPT nil N)))
    (when char
      <<reset-idle-timer-function>>
      )
    char)

However I can't find the proper function to use in place of
<<reset-idle-timer-function>>. Do you know of a way to reset the idle
timer programmatically, without actual user input?

Regards,

Ignacio

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Lars Ingebrigtsen]

Ignacio Casso <ignaciocasso@hotmail.com> writes:

> However I can't find the proper function to use in place of
> <<reset-idle-timer-function>>. Do you know of a way to reset the idle
> timer programmatically, without actual user input?

Doesn't seem to be anything reachable from Lisp land.  Adding a subr
that just calls timer_start_idle would be trivial, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  54371@debbugs.gnu.org
> Date: Sun, 13 Mar 2022 16:02:23 +0100
> 
> Ignacio Casso <ignaciocasso@hotmail.com> writes:
> 
> > However I can't find the proper function to use in place of
> > <<reset-idle-timer-function>>. Do you know of a way to reset the idle
> > timer programmatically, without actual user input?
> 
> Doesn't seem to be anything reachable from Lisp land.  Adding a subr
> that just calls timer_start_idle would be trivial, though.

I think we should first understand the use case better.  For starters,
I don't think I understand why an idle timer function should want to
call read-char with a time-out.  It is a strange thing to do, IMO.

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Ignacio Casso]

> I think we should first understand the use case better.  For starters,
> I don't think I understand why an idle timer function should want to
> call read-char with a time-out.  It is a strange thing to do, IMO.

The function in question is org-resolve-clocks-if-idle. It runs with a
normal timer (not idle) every 60 seconds, and resolves the running org
clock if (current-idle-time) is greater than the time specified by the
variable org-clock-idle-time. To do so, it prompts the user for a
character that indicates which action to take, with a prompt text that
indicates the current idle time: "Clocked in & idle for X mins
[jkKtTgGSscCiq]? ". To provide a relatively up-to-date current idle time
in the prompt text, it reads the char with a timeout in the following
loop:

  (while (or (null char-pressed)
	     (and (not (memq char-pressed
			     '(?k ?K ?g ?G ?s ?S ?C
				  ?j ?J ?i ?q ?t ?T)))
		  (or (ding) t)))
    (setq char-pressed
	  (read-char (concat (funcall prompt-fn clock)
			     " [jkKtTgGSscCiq]? ")
		     nil 45)))


Since the idle time is not reset after the character is read, this
produces the bug I explained in
https://lists.gnu.org/archive/html/emacs-orgmode/2022-03/msg00127.html,
but otherwise I find it to be reasonable code.

> Doesn't seem to be anything reachable from Lisp land.  Adding a subr
> that just calls timer_start_idle would be trivial, though.

I think that particular bug with org-clock can be fixed with other
workarounds that do not involve resetting the idle timer. No need to
expose that code only for that if it isn't already exposed.

> However, it sounds like we do this on purpose, to avoid problems with
> idle timers that call sit-for.  See the discussion that started here:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2006-08/msg00395.html
>
> The change installed at that time made read_char avoid restarting idle
> timers when it is called with a non-nil END_TIME argument.

However, I still think that if not a bug, this is at least inconsistent
and probably deserving a footnote in the relevant section of the Emacs
Lisp manual
(https://www.gnu.org/software/emacs/manual/html_node/elisp/Idle-Timers.html).

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Eli Zaretskii]

> From: Ignacio Casso <ignaciocasso@hotmail.com>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>, 54371@debbugs.gnu.org
> Date: Sun, 13 Mar 2022 18:45:33 +0100
> 
> However, I still think that if not a bug, this is at least inconsistent
> and probably deserving a footnote in the relevant section of the Emacs
> Lisp manual
> (https://www.gnu.org/software/emacs/manual/html_node/elisp/Idle-Timers.html).

I think accessing current-idle-time from an idle time is inherently
problematic: when the idle timer runs and receives input, whether or
not Emacs is idle is ambiguous in principle.  IOW, code which does
that is splitting hair.

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Ignacio Casso]

> IOW, code which does that is splitting hair.

I agree. And I have already proposed a patch for the org-clock bug that
changes the logic of org-resolve-clocks-if-idle to avoid this issue to
begin with instead of trying to work around it.

>> However, I still think that if not a bug, this is at least inconsistent
>> and probably deserving a footnote in the relevant section of the Emacs
>> Lisp manual
>> (https://www.gnu.org/software/emacs/manual/html_node/elisp/Idle-Timers.html).
>
> I think accessing current-idle-time from an idle time is inherently
> problematic: when the idle timer runs and receives input, whether or
> not Emacs is idle is ambiguous in principle.

However, I still think this should be documented somewhere. The only
thing I found was a comment in keyboard.c, and that was only after you
pointed me to the right email thread.

And this is not only about the use of read-event inside timer
functions. Currently the Idle Timers section of the manual would make
any reader think that these two forms would produce the output "A\nB" if
the user takes a few seconds to input a character, but only the first
one would.

(progn
  (run-with-idle-timer 1 nil (lambda () (message "A")))
  (read-char "Please wait for 2 seconds" nil nil)
  (message "B"))

(progn
  (run-with-idle-timer 1 nil (lambda () (message "A")))
  (read-char "Please wait for 2 seconds" nil 20)
  (message "B"))

So as I see it, the manual is incorrect right now, or at best incomplete
or ambiguous, and a footnote could not hurt. What do you think?

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Ignacio Casso]

Hi again,

Since my other bug report has already been closed, I've picked up again
the next one in my pending list, which was this one.

> However, I still think this should be documented somewhere. The only
> thing I found was a comment in keyboard.c, and that was only after you
> pointed me to the right email thread.

After searching again, I've seen that this behavior is indeed documented
in the section of the Emacs Lisp manual describing `read-event'.

>
> And this is not only about the use of read-event inside timer
> functions. Currently the Idle Timers section of the manual would make
> any reader think that these two forms would produce the output "A\nB" if
> the user takes a few seconds to input a character, but only the first
> one would.
>
> (progn
>   (run-with-idle-timer 1 nil (lambda () (message "A")))
>   (read-char "Please wait for 2 seconds" nil nil)
>   (message "B"))
>
> (progn
>   (run-with-idle-timer 1 nil (lambda () (message "A")))
>   (read-char "Please wait for 2 seconds" nil 20)
>   (message "B"))
>
> So as I see it, the manual is incorrect right now, or at best incomplete
> or ambiguous, and a footnote could not hurt. What do you think?

So unless you see a problem with this, you can close this bug report. Or
if you want, I can send a patch to update that section of the manual
with a footnote, and maybe some other relevant docstrings.

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Eli Zaretskii]

> From: Ignacio Casso <ignaciocasso@hotmail.com>
> Cc: larsi@gnus.org, 54371@debbugs.gnu.org
> Date: Fri, 01 Apr 2022 20:22:00 +0200
> 
> 
> Hi again,
> 
> Since my other bug report has already been closed, I've picked up again
> the next one in my pending list, which was this one.
> 
> > However, I still think this should be documented somewhere. The only
> > thing I found was a comment in keyboard.c, and that was only after you
> > pointed me to the right email thread.
> 
> After searching again, I've seen that this behavior is indeed documented
> in the section of the Emacs Lisp manual describing `read-event'.
> 
> >
> > And this is not only about the use of read-event inside timer
> > functions. Currently the Idle Timers section of the manual would make
> > any reader think that these two forms would produce the output "A\nB" if
> > the user takes a few seconds to input a character, but only the first
> > one would.
> >
> > (progn
> >   (run-with-idle-timer 1 nil (lambda () (message "A")))
> >   (read-char "Please wait for 2 seconds" nil nil)
> >   (message "B"))
> >
> > (progn
> >   (run-with-idle-timer 1 nil (lambda () (message "A")))
> >   (read-char "Please wait for 2 seconds" nil 20)
> >   (message "B"))
> >
> > So as I see it, the manual is incorrect right now, or at best incomplete
> > or ambiguous, and a footnote could not hurt. What do you think?
> 
> So unless you see a problem with this, you can close this bug report. Or
> if you want, I can send a patch to update that section of the manual
> with a footnote, and maybe some other relevant docstrings.

Please send the patch, and let's take it from there.

Thanks.

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Ignacio Casso]

[-- Attachment #1: Type: text/plain, Size: 1056 bytes --]


>> So unless you see a problem with this, you can close this bug report. Or
>> if you want, I can send a patch to update that section of the manual
>> with a footnote, and maybe some other relevant docstrings.
>
> Please send the patch, and let's take it from there.
>
> Thanks.

Here is a minor patch that adds a footnote to the "Idle Timers" section
of the manual. I have not found any docstrings that need to be updated,
since none of them mention that Emacs becomes idle when it is waiting
for user input. So as far as I know the only place where a user can find
that information is either in the "Idle Timers" section or the "Reading
One Event" section, and the later already mentioned that Emacs becomes idle
only when waiting for input without a timeout.

I have also one question regarding patches that modify comments,
docstrings, or any other text. What is the best practice, to do M-q
after updating the text, or to try to minimize the number of lines
changed in the patch, as I did now, even if that results in some shorter
or longer lines?



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Minor patch updating Emacs Manual --]
[-- Type: text/x-diff, Size: 933 bytes --]

From d22bb5a9058667913a48cf8bbb4ff5782b3b6d7c Mon Sep 17 00:00:00 2001
From: Ignacio <ignaciocasso@hotmail.com>
Date: Tue, 5 Apr 2022 19:21:00 +0200
Subject: [PATCH] * doc/lispref/os.texi: minor footnote

---
 doc/lispref/os.texi | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/doc/lispref/os.texi b/doc/lispref/os.texi
index 9cb9bc75d0..7c41dd90c7 100644
--- a/doc/lispref/os.texi
+++ b/doc/lispref/os.texi
@@ -2294,7 +2294,8 @@ Idle Timers
 @end deffn
 
 @cindex idleness
-  Emacs becomes @dfn{idle} when it starts waiting for user input, and
+  Emacs becomes @dfn{idle} when it starts waiting for user input
+@footnote{Unless it waits with a timeout (@pxref{Reading One Event})}, and
 it remains idle until the user provides some input.  If a timer is set
 for five seconds of idleness, it runs approximately five seconds after
 Emacs first becomes idle.  Even if @var{repeat} is non-@code{nil},
-- 
2.25.1

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Eli Zaretskii]

> From: Ignacio Casso <ignaciocasso@hotmail.com>
> Cc: larsi@gnus.org, 54371@debbugs.gnu.org
> Date: Thu, 07 Apr 2022 13:42:19 +0200
> 
> > Please send the patch, and let's take it from there.
> >
> > Thanks.
> 
> Here is a minor patch that adds a footnote to the "Idle Timers" section
> of the manual. I have not found any docstrings that need to be updated,
> since none of them mention that Emacs becomes idle when it is waiting
> for user input. So as far as I know the only place where a user can find
> that information is either in the "Idle Timers" section or the "Reading
> One Event" section, and the later already mentioned that Emacs becomes idle
> only when waiting for input without a timeout.

Thanks, I installed this with a small change (@footnote is for stuff
that is really minor or off-topic).

> I have also one question regarding patches that modify comments,
> docstrings, or any other text. What is the best practice, to do M-q
> after updating the text, or to try to minimize the number of lines
> changed in the patch, as I did now, even if that results in some shorter
> or longer lines?

It is IMO best not to M-q, because that makes it easier to review the
patch.  The person who actually installs the patch can then decide
whether to M-q before committing.

Thanks.

P.S. Can we close this bug now?

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Ignacio Casso]

> Thanks, I installed this with a small change (@footnote is for stuff
> that is really minor or off-topic).
>
> P.S. Can we close this bug now?

Yes, I think we can.

bug#54371: 29.0.50; read-char does not reset idle timer in some cases

[Eli Zaretskii]

> From: Ignacio Casso <ignaciocasso@hotmail.com>
> Cc: larsi@gnus.org, 54371@debbugs.gnu.org
> Date: Mon, 11 Apr 2022 08:42:50 +0200
> 
> 
> > Thanks, I installed this with a small change (@footnote is for stuff
> > that is really minor or off-topic).
> >
> > P.S. Can we close this bug now?
> 
> Yes, I think we can.

Thanks, done.