From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ian Jackson Newsgroups: gmane.emacs.bugs Subject: bug#53432: [PATCH] Avoid losing keyboard input when inotify is too busy [and 1 more messages] Date: Mon, 24 Jan 2022 15:45:52 +0000 Message-ID: <25070.51632.699234.228230@chiark.greenend.org.uk> References: <25067.17249.604070.872185@chiark.greenend.org.uk> <838rv8nua8.fsf@gnu.org> <87r190wqo1.fsf@yahoo.com> <25068.23625.512978.147194@chiark.greenend.org.uk> <87a6fntgfh.fsf@yahoo.com> <25069.23134.887206.241281@chiark.greenend.org.uk> <877daqtnkw.fsf@gmx.de> <25069.40682.688423.883151@chiark.greenend.org.uk> <87lez5ry92.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="22942"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Po Lu , 53432@debbugs.gnu.org To: Michael Albinus Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Jan 24 16:47:02 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nC1Yb-0005kG-9N for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 24 Jan 2022 16:47:01 +0100 Original-Received: from localhost ([::1]:49344 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nC1Ya-0008H4-57 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 24 Jan 2022 10:47:00 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:34692) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nC1Xu-00080h-F6 for bug-gnu-emacs@gnu.org; Mon, 24 Jan 2022 10:46:18 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:52496) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nC1Xd-0002At-UL for bug-gnu-emacs@gnu.org; Mon, 24 Jan 2022 10:46:16 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nC1Xd-0002j6-J1 for bug-gnu-emacs@gnu.org; Mon, 24 Jan 2022 10:46:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Ian Jackson Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 24 Jan 2022 15:46:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53432 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 53432-submit@debbugs.gnu.org id=B53432.164303915810469 (code B ref 53432); Mon, 24 Jan 2022 15:46:01 +0000 Original-Received: (at 53432) by debbugs.gnu.org; 24 Jan 2022 15:45:58 +0000 Original-Received: from localhost ([127.0.0.1]:45399 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nC1Xa-0002im-6h for submit@debbugs.gnu.org; Mon, 24 Jan 2022 10:45:58 -0500 Original-Received: from permutation-city.chiark.greenend.org.uk ([212.13.197.230]:36910 helo=chiark.greenend.org.uk ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nC1XY-0002id-Dy for 53432@debbugs.gnu.org; Mon, 24 Jan 2022 10:45:57 -0500 Original-Received: by chiark.greenend.org.uk (Debian Exim 4.89 #1) with local (return-path ijackson@chiark.greenend.org.uk) id 1nC1XV-0000CZ-FN; Mon, 24 Jan 2022 15:45:53 +0000 In-Reply-To: <87lez5ry92.fsf@gmx.de> X-Mailer: VM 8.2.0b under 24.4.1 (i586-pc-linux-gnu) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:225065 Archived-At: Michael Albinus writes ("Re: bug#53432: [PATCH] Avoid losing keyboard input when inotify is too busy [and 1 more messages]"): > Ian Jackson writes: > > I guess we probably want a "file notifications might have been missed" > > callback. Having it be a callback like a normal file event callback > > will probably make it easiest to deal with. > > Yep. > > I don't think separating out the input buffers would remove that > > requirement. Ie, the ability to cope correctly with a flood of events > > (of whatever kind) will remain necessary. > > Yes. But if we divide the keyboard buffer (also good for mouse events > and other events which do not harm) from the D-Bus and file notification > events, such a check is needed at fewer places. I don't follow. Any buffer that we have can fill up. So each thing that generates events needs to be able to handle the buffer (that it is writing to) becomeing full; and to be able to handle the buffer becoming no-longer-full, so that it can restart. So each event-generator has a "stop" hook and a "start" hook, each of which has one call site, in the code that handles the buffer for that event generator. The number of these calls is the same either way. The difference is just whether there are two buffers, which call disjoint sets of start/stop hooks, or one buffer, which calls all the start/stop hooks. [reordered slightly:] > The advantage of splitting into "keyboard" and "other things" buffers > would be, that the keyboard buffer doesn't overrun, whatever burst of > D-Bus or file notification events arrives. The two sets of code would seem very similar. It would probably be sensible to actually use the very same code and data formats and everything. The advantage is precisely that we can prioritise them differently: the keyboard buffer could keep working, even while the other buffer is full. In practice I don't think this is likely to be particularly important since we don't usually expect emacs to be overloaded in this way. And when it is, there is an argument that processing events in order is likely to be better in the sense that it is less confusing to the user. > > I think every one of these other event sources ought to be able to > > partake of this scheme (adding their own hooks like I proposed for > > inotify) without undue difficulty. > > Yes. But I believe we don't need to handle lost events. If we have a > mean to inform the upper libraries, filenotify.el and dbus.el, about > this case, it would be sufficient. A simple check whether the event > buffer is full. I think we are just having a semantic difference here. To my mind "handling" lost events *consists of* notifying the next layer up that "some events may have been lost". Eventually this informaton will get to somewhere that can do something sensible with it. > > The result would be that if any of the buffers overflowed, we would > > stop processing inotify events altogether until emacs has caught up > > with the user's keyboard input, and then (via the file notification > > "maybe lost" callback, do one check on every file which we might want > > to revert). > > I don't believe that the native backends, like inotify.c, deserve too > much intelligence. They shall do stupid event receiving and reporting, > with a callback invoked in case of problems. This callback must be > clever then. If the buffer overflowed, it is quite easy to discard now-superfluous events at the lower layer (thus coalescing events into the single "some may have been lost" callback). At higher levels the state machine needed to do this becomes more complicated. And discarding events at a lower layer helps performance because it means they don't need to be transformed/analysed/etc. The performance of event processing is particularly important in the case when things are stressed and overloaded. But I don't think this optimisation, at this level, is inherently *necessary* for correct operation. > > Would it be OK to postpone splitting this out ? I don't think > > splitting this up is necessary to fix these lost notifications. > > Not to fix lost events. But it helps to fix lost keyboard strokes. It will help to fix lost keystrokes only if we don't plan to fix the uncontrolled spew from the other event sources. But it is easier to fix the lost keystrokes: I suggest introducing a version of kbd_buffer_store_event which is to be called by anyone that doesn't do flow control properly. It would stop putting things in the buffer when the buffer is (say) 3/4 full. The plan would be to replace all calls to this function eventually. > > I guess the next thing I should do is go and read the file > > notification lisp code to see how a "please do a one-off poll" > > callback could be implemented. > > Hmm. We are still in different camps about the approach ... I am confused. The file notification code must have a "check all files we are polling" facility, for when file events are not available. Perhaps this is buried in a polling loop, but it could be made separately callable. I think that the correct response to discovering that file events have been lost, is to do a one-off poll of every file. Perhaps we would want to rate limit this, but that should be done by *deferring* a poll-all, not by *skipping* one, as skipping would introduce a race that might lose a buffer revert. But I don't think this is actually necessary. In any case no keyboard events will be lost. An alternative would be to turn off inotify entirely and just switch back to polling. But we wouldn't want to do that permanently; when things have calmed down we want emacs to back to quiescently waiting for file events. So there would want to be some kind of timer there, and the system would have two modes. It seems simpler to me to have only one mode. Writing the code to switch modes and coordinate everything seems error-prone to me. Ian. -- Ian Jackson These opinions are my own. Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.