Re: [PATCH] Added inotify support.

[Eli Zaretskii <eliz@gnu.org>]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: ofv@wanadoo.es,  emacs-devel@gnu.org
> Date: Wed, 03 Oct 2012 08:46:39 -0400
> 
> >> >> Btw, what are Emacs use cases for using this kind of feature?
> >> > Apart from those mentioned on the original message (dired or magit's (or
> >> > vc-dir) status view) I'll like to mention auto-revert-mode.
> >> Yes, for my own use auto-revert-mode is the main one.
> > AFAICS, all these use cases don't need support for every possible
> > notification inotify lets you define.  How about a simpler interface?
> 
> The patch he suggests is "as simple as it gets" because it just exposes
> the C-level API.  And yes, we'll want a simpler interface on top.
> Both so as to make it easier to use but also so it can work with other
> mechanisms than inotify.

If providing a higher-level interface is the plan, then I think
inotify.el should be renamed to something like file-notifications.el, or
some other neutral name, because before long it will be home to other
back ends and to those higher-level APIs.

That said, when will this simpler interface be added? before or after
Emacs 24.3 is locked for changes?  If before, then you can skip
everything I write below (or save it for a future discussion ;-).
That's assuming the inotify support _will_ be in Emacs 24.3.

But if Emacs 24.3 is supposed to be released without this simpler
interface, then I think we will do a disservice to Lisp programmers.

E.g., consider the Dired use case.  This wants to know about any and
all changes to the files in the directory, because it will need to
refresh the listing, right?  But as things are, whoever will want to
code this will have to carefully read the inotify(7) man page and
figure out which of the IN_* flags she wants, because there's no
'just-what-dired-needs' flag in the API that is being proposed.

Or consider the autorevert use case.  Which flag(s) to use for that?
My first guess was IN_ATTRIB, but the man page says

  IN_ATTRIB         Metadata changed, e.g., permissions, timestamps,
                    extended attributes, link count (since Linux 2.6.25),
                    UID, GID, etc.

Hmmm... does this include file size? I don't know.  If it doesn't,
then do I need to catch the series of IN_OPEN, IN_WRITE, IN_CLOSE?

Are you confused yet?  Because I am.

Another concern is how to design the Lisp code that gets run by
notifications.  Since we are converting notifications into Lisp
events, the most natural way of using them would be to bind a function
to the 'inotify-event' event.  (Btw, I'd drop the "event" part from
the symbol name, it's redundant.)  But since meaningful file-level
events are likely to produce multiple notifications (see below), such
a function will need to implement a non-trivial state machine to DTRT.

For example, if you type at the shell prompt "echo foo > bar" with
'bar' an existing non-empty file, how many notifications do you see?
On Windows, I get 2 or 3, depending on the details: 2 for size change
(first one when the file is truncated, second one when "foo" is
written into it), and sometimes also an attribute or "security" change
(e.g., if the original file was owned by someone else).  Unless
inotify magically merges all these into a single meaningful
notification, the Lisp code that receives this series of notifications
will have hard time doing TRT with them, even if the exact expected
series are known in advance, especially if other notifications are
interspersed with these.

As yet another example, many applications update a file by first
writing a temporary file, then deleting the old file and renaming the
new one.  That's another series of notifications someone will have to
figure out in Lisp, when the function bound to 'inotify-event' will
get called several times, because from the user POV, the file got
updated, not overwritten by moving another file over it.

This is why I think we _must_ have a higher-level API in place for
this feature to be useful in practice, even if just inotify back end
is supported.  And we should talk _today_ about that API, because some
of the processing needed to produce higher-level abstractions of
events are much easier done in C than in Lisp.