From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Michael Albinus Newsgroups: gmane.emacs.bugs Subject: bug#53432: [PATCH] Avoid losing keyboard input when inotify is too busy [and 1 more messages] Date: Tue, 25 Jan 2022 15:50:29 +0100 Message-ID: <87czkfrhsa.fsf@gmx.de> References: <25067.17249.604070.872185@chiark.greenend.org.uk> <838rv8nua8.fsf@gnu.org> <87r190wqo1.fsf@yahoo.com> <25068.23625.512978.147194@chiark.greenend.org.uk> <87a6fntgfh.fsf@yahoo.com> <25069.23134.887206.241281@chiark.greenend.org.uk> <877daqtnkw.fsf@gmx.de> <25069.40682.688423.883151@chiark.greenend.org.uk> <87lez5ry92.fsf@gmx.de> <25070.51632.699234.228230@chiark.greenend.org.uk> <87bl01rrmu.fsf@gmx.de> <25070.63232.560314.823060@chiark.greenend.org.uk> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2943"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: Po Lu , 53432@debbugs.gnu.org To: Ian Jackson Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Jan 25 16:04:07 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nCNMc-0000UT-TJ for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 25 Jan 2022 16:04:07 +0100 Original-Received: from localhost ([::1]:53880 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCNMb-0005OV-Fu for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 25 Jan 2022 10:04:05 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:42266) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCN9z-0001JN-Q0 for bug-gnu-emacs@gnu.org; Tue, 25 Jan 2022 09:51:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:54752) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCN9y-0002mY-7e for bug-gnu-emacs@gnu.org; Tue, 25 Jan 2022 09:51:03 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCN9x-0005ma-Vc for bug-gnu-emacs@gnu.org; Tue, 25 Jan 2022 09:51:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Michael Albinus Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 25 Jan 2022 14:51:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53432 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 53432-submit@debbugs.gnu.org id=B53432.164312224122199 (code B ref 53432); Tue, 25 Jan 2022 14:51:01 +0000 Original-Received: (at 53432) by debbugs.gnu.org; 25 Jan 2022 14:50:41 +0000 Original-Received: from localhost ([127.0.0.1]:47655 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCN9d-0005ly-7Y for submit@debbugs.gnu.org; Tue, 25 Jan 2022 09:50:41 -0500 Original-Received: from mout.gmx.net ([212.227.17.22]:58811) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCN9a-0005li-AQ for 53432@debbugs.gnu.org; Tue, 25 Jan 2022 09:50:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1643122230; bh=xNZK+b+uDW2ZODZqNB6QTicfhwFInrwrTSlycHKAi10=; h=X-UI-Sender-Class:From:To:Cc:Subject:References:Date:In-Reply-To; b=EprEImHNZtElgfX7PFI8oeDkjPIQW/f6/TXos2UZblRv+/eleouGFVEZIyOe31iLv Bn3RT9AY9s91dQEw077WVfZ/hVXsqSjTbK3qYq4tJRkGtc/SAiA9uYUaQC4p5zSeQe lQ27pi/ibWlt5F8Ys2cTWpvE+QlYhvvgetZQbtzs= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Original-Received: from gandalf.gmx.de ([79.140.118.216]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MysW2-1mHwqC0qKg-00vxIu; Tue, 25 Jan 2022 15:50:30 +0100 In-Reply-To: <25070.63232.560314.823060@chiark.greenend.org.uk> (Ian Jackson's message of "Mon, 24 Jan 2022 18:59:12 +0000") X-Provags-ID: V03:K1:5Se+J5uqpFJ12kkOTvIsIdFgQPxTkFbmamq55mRhSwnNVhSYZs5 MIAs5q8SdZHsps5mWgolxwwCMU43ZfiRwbAPFKXohmQTvrw0ZmCfodXpoXp7s4pCC7PAIsa mJerfvCVl0UqRhiy46/CXwr/mUBZVJOdpEN9EQRSE+7GMviIztQ/Hvuj40JYlLngFqZ14GN huQVmzycavH11sAvcNZ/A== X-UI-Out-Filterresults: notjunk:1;V03:K0:CB65oikmDRQ=:lv547Vnkf96oo04sj4DJk1 4KIjEvy3XDwmkjKeO15GNyI1icBazLcfRO3ol0PpcIw8WISZu2y/eBy0aXFwyTiRtR1t/5O1e qyybno1HkfER0hC96kMoIwl5qYxaZjZpT6UlicE091vu/RKxyVszsixhkZ4VtPf3MLrGiTCLE 1V8O11FVzt5YAgAD0X+nnIK4UxUQyTKRrjiDxeduzLKSr/81TkEERr02XX+3nRh68KSm+Wvb7 peN5ua8+Q1Jw1NeSPv3z4J9dV5gSEGEk4tSwWuB++WVfSK2RcZ04RRqCLam8yHtn1j4UiIv62 hlK32TfVlHOuV8Gm4qkCK28umsFTf23nVdudDNZVgGhklWwcwwtsdWUHmzBeoG3FFK3WuS4oW qE6LkTdp+xetCZMplC/ihl/cknxJER9PEeUOEJFI0uPw6rQFhAY6WA8kk5pXrloFxe4rif720 V9jc0/k9pUJR9rK4UleFO5vQVC9xPA7M8ohGF4nRCjE33j8dFIzp0gNXd9C/Qqu2+8AduiuRd S7GOXTbm4GffBgM97NdyQXmNA4G8vLB9XOhkfWysuCzAtIy/N1urmxzVDFshiJCwqBQ1Z+V4J 1lCpf67WeJfqZnQNHz5XmDzeQz1/n0Sg3Ka2+7miGpXRnlJmUrQ9OTjWE+LYNGfiZ486spUW6 I/XbVhdZ0wQt4hKJMcgKlOJOUDm83V/vHbasSh2jEoSgE2amOlH3P1oMG5x7x6xNLVManA/NC 6nCRD88fGKEc2t2+KoU0uZBL9mS3qX9PYGGGBMvaH591+UaMvzpl16hzMu9ps72F2YjQ1/lY X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:225186 Archived-At: Ian Jackson writes: Hi Ian, > Let me consider the semantics of these notifications you are > suggesting in a bit more detail. I hope you will forgive the > resulting screed: > > As I understand your proposal, the system starts in "normal" state. > In "normal" state, the file notification system should promise that if > 1. a file notification is set up > 2. subsequently, that file is modified > then the file notification system will make one of these callbacks > A. file notification event > B. "file notifications not working" callback > (And, I gyuess, that this will be done "reasonably promptly" in some > sense; I'm not sure if we need to define this.) Thinking about, we don't need another callback. We can define a special file notification event which tells us. Something like '(file-notify overflow) where is the result of a respective `file-notify-add-watch' call, and `overflow' is a new action. `file-notify-handle-event' calls the registered application specific handler with this event; that handler is responsible for whatever. I see two cases where we could inject such an event: the keyboard buffer size exceeds a given high water mark, like 80% or 90%. Or (in the inotify case) we receive an IN_Q_OVERFLOW event. Other backends might know further situations when such an event is needed. > If B happens then the system is now in "non-working" state. In > non-working state, all bets are off. Users like autorevert must take > alternative measures. Yep. Application specific handlers, like `auto-revert-notify-handler', must react. > At some point, the file notification system will make a "file > notifications working again now" callback. I think this has to > promise that, if: > 1a. a file notification was set up > 1b. the "notifications working" callback was made more recently > than any "notifications not working" callback > (in either order) and then *subsequently* > 2. a relevant file is notified > then a notification will occur, as above. Obviously it can't sensibly > make guarantees about what happened before the "working" notifcation. This might be signalled by a another artifical event like '(file-notify normal) which is treated like the other event. It could happen if the keyboard buffer size falls below a given low water mark, say 40% or 50%. Or (in the inotify case) we receive events w/o IN_Q_OVERFLOW bit mask. (Action names like `overflow' and `normal' are random, any better proposal is appreciated). > How must a user of this API respond ? Consider autorevert. Suppose > the notifications were declared "non-working" and autorevert resorted > to polling. > > When autorevert is told things are working again, it wants to go back > to not polling. But it doesn't know whether notifications were lost > between its last poll, and the file notifications becoming normal > again. There might be a few leftover lost events. > > So autorevert must poll every file again, once. I think it makes most > sense to do this immediately, rather than wait for the next poll > interval. That allows autorevert to avoid a confusing special-case > transitional state where it is not polling but still has one poll to > do - and it means the user gets the buffers reverted as soon as > possible rather than after a timeout. I believe it would be sufficient to signal "after next poll we're going back to file notifications". This is transitional, but not very hard to implement. And we will wait 5 seconds max, which doesn't sound too terrible. > Now, consider the behaviour of the file notification system. > > In practice, inotify.c will probably want to declare itself working as > soon as the event buffer it is writing to becomes empty. Doing so > sooner merely imposes more load on a busy emacs; doing so later > degrades the user experience by having the emacs unnecessarily wait > before catching up with the work that needs to be done. > > What this means is that the file notification system is only in the > "non-working" state when emacs is too busy handling important events > to catch its breath and handle file notifications. Ie, only when > emacs is not idle. > > In this state, we do not actually want to poll for file notifications! > It doesn't make sense to make this fire off a timer. Well, we need to read the incoming events, but we won't push them to the keyboard buffer. Until the low watermark has been reached, which means Emacs isn't too busy anymore. > So autorevert does not actually need to do anything with the "not > working" notification. It should wait for the "working again" > notification, and respond by polling all the files exactly once each. Why not do default polling, every 5 seconds? As said, the "not working" state doesn't mean always that Emacs is too busy. It could also be the case, that too many events are fired by the kernel, events are lost, and we are informed about by IN_Q_OVERFLOW. > I think a similar situation will arise for other kinds of event > generation. So we don't actually need two separate notifications. > We only need one, which means: "err, sorry, it turns out things > weren't working, but I think they're OK now - and I'll let you know > again if not". > > The result of my approach as I describe above will be an emacs which, > when overloaded, suspends file event handling in order to handle user > input, but which promptly catches up with deferred work when it can. Well, injecting the two new events gives the decision what to do to the application handlers. They can just sit and wait, or they can try to receive needed information otherwise, like autorevert does with polling. And such a strategy could be customized. Best regards, Michael.