From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: [PATCH] Added inotify support. Date: Wed, 03 Oct 2012 20:34:01 +0200 Message-ID: <83wqz78kx2.fsf@gnu.org> References: <2181827.T3JxG88qQt@descartes> <83lifoa7hr.fsf@gnu.org> <87obkkcu92.fsf@wanadoo.es> <83ipas9pmz.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1349289308 4472 80.91.229.3 (3 Oct 2012 18:35:08 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 3 Oct 2012 18:35:08 +0000 (UTC) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Oct 03 20:35:13 2012 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1TJTmi-0007jV-1o for ged-emacs-devel@m.gmane.org; Wed, 03 Oct 2012 20:35:04 +0200 Original-Received: from localhost ([::1]:57887 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TJTmc-0008HA-84 for ged-emacs-devel@m.gmane.org; Wed, 03 Oct 2012 14:34:58 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:35870) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TJTmY-0008Dp-TB for emacs-devel@gnu.org; Wed, 03 Oct 2012 14:34:56 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TJTmX-0007Xy-E9 for emacs-devel@gnu.org; Wed, 03 Oct 2012 14:34:54 -0400 Original-Received: from mtaout20.012.net.il ([80.179.55.166]:35626) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TJTmX-0007Xq-5m for emacs-devel@gnu.org; Wed, 03 Oct 2012 14:34:53 -0400 Original-Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MBB00C00XWKMI00@a-mtaout20.012.net.il> for emacs-devel@gnu.org; Wed, 03 Oct 2012 20:33:54 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MBB00CWRY8DME10@a-mtaout20.012.net.il>; Wed, 03 Oct 2012 20:33:49 +0200 (IST) In-reply-to: X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-Received-From: 80.179.55.166 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:154027 Archived-At: > From: Stefan Monnier > Cc: ofv@wanadoo.es, emacs-devel@gnu.org > Date: Wed, 03 Oct 2012 08:46:39 -0400 > > >> >> Btw, what are Emacs use cases for using this kind of feature? > >> > Apart from those mentioned on the original message (dired or magit's (or > >> > vc-dir) status view) I'll like to mention auto-revert-mode. > >> Yes, for my own use auto-revert-mode is the main one. > > AFAICS, all these use cases don't need support for every possible > > notification inotify lets you define. How about a simpler interface? > > The patch he suggests is "as simple as it gets" because it just exposes > the C-level API. And yes, we'll want a simpler interface on top. > Both so as to make it easier to use but also so it can work with other > mechanisms than inotify. If providing a higher-level interface is the plan, then I think inotify.el should be renamed to something like file-notifications.el, or some other neutral name, because before long it will be home to other back ends and to those higher-level APIs. That said, when will this simpler interface be added? before or after Emacs 24.3 is locked for changes? If before, then you can skip everything I write below (or save it for a future discussion ;-). That's assuming the inotify support _will_ be in Emacs 24.3. But if Emacs 24.3 is supposed to be released without this simpler interface, then I think we will do a disservice to Lisp programmers. E.g., consider the Dired use case. This wants to know about any and all changes to the files in the directory, because it will need to refresh the listing, right? But as things are, whoever will want to code this will have to carefully read the inotify(7) man page and figure out which of the IN_* flags she wants, because there's no 'just-what-dired-needs' flag in the API that is being proposed. Or consider the autorevert use case. Which flag(s) to use for that? My first guess was IN_ATTRIB, but the man page says IN_ATTRIB Metadata changed, e.g., permissions, timestamps, extended attributes, link count (since Linux 2.6.25), UID, GID, etc. Hmmm... does this include file size? I don't know. If it doesn't, then do I need to catch the series of IN_OPEN, IN_WRITE, IN_CLOSE? Are you confused yet? Because I am. Another concern is how to design the Lisp code that gets run by notifications. Since we are converting notifications into Lisp events, the most natural way of using them would be to bind a function to the 'inotify-event' event. (Btw, I'd drop the "event" part from the symbol name, it's redundant.) But since meaningful file-level events are likely to produce multiple notifications (see below), such a function will need to implement a non-trivial state machine to DTRT. For example, if you type at the shell prompt "echo foo > bar" with 'bar' an existing non-empty file, how many notifications do you see? On Windows, I get 2 or 3, depending on the details: 2 for size change (first one when the file is truncated, second one when "foo" is written into it), and sometimes also an attribute or "security" change (e.g., if the original file was owned by someone else). Unless inotify magically merges all these into a single meaningful notification, the Lisp code that receives this series of notifications will have hard time doing TRT with them, even if the exact expected series are known in advance, especially if other notifications are interspersed with these. As yet another example, many applications update a file by first writing a temporary file, then deleting the old file and renaming the new one. That's another series of notifications someone will have to figure out in Lisp, when the function bound to 'inotify-event' will get called several times, because from the user POV, the file got updated, not overwritten by moving another file over it. This is why I think we _must_ have a higher-level API in place for this feature to be useful in practice, even if just inotify back end is supported. And we should talk _today_ about that API, because some of the processing needed to produce higher-level abstractions of events are much easier done in C than in Lisp.