From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Michael Albinus Newsgroups: gmane.emacs.bugs Subject: bug#53432: [PATCH] Avoid losing keyboard input when inotify is too busy [and 1 more messages] Date: Sun, 23 Jan 2022 17:37:51 +0100 Message-ID: <877daqtnkw.fsf@gmx.de> References: <25067.17249.604070.872185@chiark.greenend.org.uk> <838rv8nua8.fsf@gnu.org> <87r190wqo1.fsf@yahoo.com> <25068.23625.512978.147194@chiark.greenend.org.uk> <87a6fntgfh.fsf@yahoo.com> <25069.23134.887206.241281@chiark.greenend.org.uk> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1362"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: Po Lu , 53432@debbugs.gnu.org To: Ian Jackson Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Jan 23 17:39:14 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nBftY-000Acg-AX for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 23 Jan 2022 17:39:12 +0100 Original-Received: from localhost ([::1]:52184 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nBftX-0007E9-2o for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 23 Jan 2022 11:39:11 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:51998) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nBftP-0007Bm-Tn for bug-gnu-emacs@gnu.org; Sun, 23 Jan 2022 11:39:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:48108) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nBftN-000050-NV for bug-gnu-emacs@gnu.org; Sun, 23 Jan 2022 11:39:03 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nBftN-0004n9-LD for bug-gnu-emacs@gnu.org; Sun, 23 Jan 2022 11:39:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Michael Albinus Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 23 Jan 2022 16:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53432 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 53432-submit@debbugs.gnu.org id=B53432.164295588318249 (code B ref 53432); Sun, 23 Jan 2022 16:39:01 +0000 Original-Received: (at 53432) by debbugs.gnu.org; 23 Jan 2022 16:38:03 +0000 Original-Received: from localhost ([127.0.0.1]:41005 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nBfsR-0004kG-8T for submit@debbugs.gnu.org; Sun, 23 Jan 2022 11:38:03 -0500 Original-Received: from mout.gmx.net ([212.227.17.20]:50749) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nBfsO-0004jR-G6 for 53432@debbugs.gnu.org; Sun, 23 Jan 2022 11:38:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1642955872; bh=163Qmn4TwPVeFJ5sDDoxpd+Rn6myajKCDduDRR10Y2w=; h=X-UI-Sender-Class:From:To:Cc:Subject:References:Date:In-Reply-To; b=M5ga4UOSODV/SJ3MZjeZqXdZxOttJHchAFD7E7Iee3AajjFCAowjeHudhJi7MnzMq /bYallbKcLmvFjY/o3lfrv/1hMQvc/SxbK7PAczgrVdcUXerj9YmZbo+qaRXEVVD07 jD1VzG/GApb8EmsD0et0ka0Qf0Z7pjTTKeKO74yg= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Original-Received: from gandalf.gmx.de ([213.220.156.174]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MKsj7-1mqZgr1rQr-00LH2H; Sun, 23 Jan 2022 17:37:52 +0100 In-Reply-To: <25069.23134.887206.241281@chiark.greenend.org.uk> (Ian Jackson's message of "Sun, 23 Jan 2022 13:38:38 +0000") X-Provags-ID: V03:K1:hRS3NP0jhIEm/rp+W3LUK09+rfRa7p8jCdN+3yGUQZFtvndHGAJ l1qeRKl7aSIUA6VniEUNeFm95YAlu9wb+EivEm2McGtCnPEC4yWywjmaItoLPkOsT0o2tzZ WfC2F+K2irUeqkuSoGzE9c1DMB+Rml6Xdn3zAfb1rsuIXfKbn8hUQhoZjkfYS7xW4shm4n+ hm4Y8l9ZK/f7zOZdGkqNw== X-UI-Out-Filterresults: notjunk:1;V03:K0:AJ5OMTJlezQ=:RkDq5y2ge/xMG2RV4bhY5V pw57839A4Z5DXYkJDAh1/QTc1y26Clj0V7cnnWMXvpKKDN7JrT+ahxiIiEa15h93eE3gSNzg6 x81qnIdgvxa/aa5hTbddllcWBzHGb2bBKnJ+/wgaZItA6Gfe6u74Jil8an272nm4h5Ek34xwm /MZ7lVnPw7hUU1PZAQC/xjvEcBTikwu0IcDAe4nl8U5y98TryBomFhIHg1Rn1fuEhwWHlJwJg fXaySbbBW2WAiMvVGK1/x0nXD/8sLKxkWeLHTDl+nDx6+lN0yMNGbBX6K+MEehgYxlbQtLSOo pQD8NVlAgySpDQ3TzFAXR00aWhALi98DdeUheabvP+BmXxvk3OKv5gh3bFFy7o7S6n3hYmHsK rG7pg4UAtICsVJV3b459K+ROwGHnLUGL9vXCCmeqWD++DDt9oq1jhsJm/bEZQ97HKEqOxZbXf pk/9k3XSUaTT2ETEPGitIeE+lOBcTPJfSz/aX8ilb2FEJNBbao1tCrKspyLUUl5gVTHEo7pLn 4xNGE/HY61SuXi+drr0+TzgbY176zx9ZezuVBeyq9hW2+lnpAl4NBv0qk/WkvR+hDd4/wrqs8 ul2qoePhs4/NBT8djMILNUc8DeIbULI9ZBVng9UA7CMlaKUAteYFwdEsMOJmm1lVUw2WJiF2q RKD2VD8dDfRrR+hAHinb5oHof3jTwSBxJ91xRt2+AkmHYWxAI5bNv4xWHe3I5gtXQYk0VyIoM YcwfpWM2WRVEuj1ml7Zr0bbxQOOEiq+3TM/OKHnbujiCnds5vnvgjOzOvrMH0yqsniEUj1Dk X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:224942 Archived-At: Ian Jackson writes: Hi, > I suggest the following approach for stable branches: > > 1. Have the inotify code drop notifications if kbd_on_hold_p(). > This will mean that keystrokes will never be lost. I agree. But then, inotify (and the other file notification backends) shall inform that this happened. Either by an error signal, or by calling a handler. > 2. At least double the size of the keyboard buffer from 4k to 8k. > I think this is necessary because in a stable branch we don't want > to make anyone's experience worse. Doing (1) will cause emacs to > start to lose file notifications when the buffer is half full, > rather than full. Doubling the buffer (at trivial memory cost) is > an easy way to fix this. > > 3. Consider a significantly larger bump to the keyboard buffer to > further reduce the incidence of this unreliability. It's a single > static buffer; we could easily change it from 4k to 40k (say) with > negligible cost. For this, I have no opinion. However, I remember vaguely that Stefan said once that we shall not (mis-)use the keyboard buffer for incoming events, like from file notification or from D-Bus. Another mechanism is needed. This idea hasn't been implemented. > If desired, the increased keyboard buffer size could be made conditional on > inotify support so that systems which don't use the keyboard buffer > for file notifications don't pay a price. The following packages inject events via the keyboard buffer: dbusbind.c, gfilenotify.c, inotify.c, kqueue.c, w32notify.c. All of them could see a burst of incoming events. It isn't just an inotify problem. > For master, I still think we need to make global-auto-revert-mode > reliable. We don't want to stop using file notifications for it > (where they are available), because we don't want emacs to be polling > - that is poor for battery life on laptops. Yep. When the file notification backend reports a loss of event, the autorevert package might fall back to polling. Perhaps we could even raise another signal that things are back to normal, and packages like autorevert could switch back to file notification. > It is true that the kernel's inotify system may sometimes drop events > because the buffer in the kernel filled up - but when that happens, it > sends a IN_Q_OVERFLOW event, with -1 for the associated fd. > I.e. rather than completely losing events, it collapses them into a > single "you missed some stuff" event. Yep, this another case (beside of keyboard buffer full) where a "file notification broken" signal is useful. > If we handled that notification, we could continue to use inotify and > fall back to a single scan through all the files, when we're told that > inotify was overloaded. I haven't looked at the auto-revert code, > but, is there a way we could invoke a "one-off" poll ? Don't know off-hand, but it shall be possible. However, if autorevert falls back to polling anyway, we get this for free. > IF we did this then the inotify code could use the same mechanism to > handle the case where it dropped events due to the kbd_on_hold_p, and > everything would work correctly. Complicated buffer management in the > inotify code wouldn't be needed. Agreed. > We would still probably want to have a significantly larger keyboard > buffer, at leat when inotify is enabled. This is because we can > reasonably expect a much higher rate of file notifications than > keyboard events. Even if we have made emacs reliable when the buffer > fills up, we still don't *want* the buffer to fill up because the > non-file-notification based auto revert is a lot less efficient, > especially in a large emacs visiting many files: it will often be the > case that only a handful of files have changed, perhaps very many > times. > > I definitely think we want to get (back) to the point where choosing > the keyboard buffer size can be done purely with respect to > performance considerations rather than worrying about lossage. As said above: perhaps it isn't the best idea to handle such events via the keyboard buffer. >> All file notification systems are "best effort". At the very least, >> dropping events when the keyboard buffer is full would make the behavior >> consistent with GLib file notifications, which drops notifications >> whenever its own internal buffer fills up. > > I'm not familiar with glib. Presuambly glib either exposes something > with the semantics of IN_Q_OVERFLOW, or internally converts that to a > notification to every watcher. Otherwise programs using glib's API > would have to poll, which seems unlikely - I think that the glib > authors will be well aware of the desire to avoid polling for battery > life reasons. glib uses several native backends, like inotify or kqueue. It has also a backend which polls. However, I don't know whether it switches the backend during runtime. > Thanks, > Ian. Best regards, Michael.