[PATCH] Support for filesystem watching (inotify)

[PATCH] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

Hello,
I wrote a patch to support watching for file system events. It currently only works with inotify (Linux) but it could be extended to support kqueue 
(*BSD, OS X) or Win32. But I wanted to get some feedback first. Watching for filesystem events would be very useful for, e.g., dired or magit's status 
view.

Here is a quick example (expects a directory foo containing a file bar):

(file-watch "foo" #'(lambda (path events) (message "FS-Event: %s %s") path events)) :all)
(delete-file "foo/bar")
(file-unwatch "foo")

So please tell me what you think of this.

Regards,
Rüdiger


[PATCH] Added basic file system watching support.

It currently only works with inotify/Linux. It adds file-watch and file-unwatch functions.
---
 configure.in    |   14 +++
 src/Makefile.in |    2 +-
 src/emacs.c     |    4 +
 src/filewatch.c |  242 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/lisp.h      |    5 +
 5 files changed, 266 insertions(+), 1 deletions(-)
 create mode 100644 src/filewatch.c

diff --git a/configure.in b/configure.in
index 77deef8..3263876 100644
--- a/configure.in
+++ b/configure.in
@@ -169,6 +169,7 @@ OPTION_DEFAULT_ON([dbus],[don't compile with D-Bus support])
 OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
 OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
 OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
+OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) support])
 
 ## For the times when you want to build Emacs but don't have
 ## a suitable makeinfo, and can live without the manuals.
@@ -1978,6 +1979,19 @@ fi
 AC_SUBST(LIBGNUTLS_LIBS)
 AC_SUBST(LIBGNUTLS_CFLAGS)
 
+dnl inotify is only available on GNU/Linux.
+HAVE_INOTIFY=no
+if test "${with_inotify}" = "yes"; then
+   AC_CHECK_HEADERS(sys/inotify.h)
+   if test "$ac_cv_header_sys_inotify_h" = yes ; then
+     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, HAVE_INOTIFY=yes)
+   fi
+fi
+if test "${HAVE_INOTIFY}" = "yes"; then
+   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
+   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
+fi
+
 dnl Do not put whitespace before the #include statements below.
 dnl Older compilers (eg sunos4 cc) choke on it.
 HAVE_XAW3D=no
diff --git a/src/Makefile.in b/src/Makefile.in
index e119596..0cd6d6e 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -354,7 +354,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o $(XMENU_OBJ) window.o \
 	syntax.o $(UNEXEC_OBJ) bytecode.o \
 	process.o gnutls.o callproc.o \
 	region-cache.o sound.o atimer.o \
-	doprnt.o intervals.o textprop.o composite.o xml.o \
+	doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
 	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ)
 obj = $(base_obj) $(NS_OBJC_OBJ)
 
diff --git a/src/emacs.c b/src/emacs.c
index 6bdd255..5ca5cda 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -1550,6 +1550,10 @@ main (int argc, char **argv)
       syms_of_gnutls ();
 #endif
 
+#ifdef HAVE_FILEWATCH
+      syms_of_filewatch ();
+#endif /* HAVE_FILEWATCH */
+
 #ifdef HAVE_DBUS
       syms_of_dbusbind ();
 #endif /* HAVE_DBUS */
diff --git a/src/filewatch.c b/src/filewatch.c
new file mode 100644
index 0000000..5b7c130
--- /dev/null
+++ b/src/filewatch.c
@@ -0,0 +1,242 @@
+/* Watching file system changes.
+
+Copyright (C) 2011
+  Free Software Foundation, Inc.
+
+This file is part of GNU Emacs.
+
+GNU Emacs is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+GNU Emacs is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <config.h>
+
+#ifdef HAVE_FILEWATCH
+
+#include <setjmp.h>
+
+#include "lisp.h"
+#include "process.h"
+
+static Lisp_Object QCmodify, QCmove, QCattrib, QCdelete, QCfrom, QCto, QCall;
+
+#ifdef HAVE_INOTIFY
+#include <sys/inotify.h>
+#include <sys/ioctl.h>
+
+enum { uninitialized = -100 };
+static int inotifyfd = uninitialized;
+
+// Assoc list of files being watched.
+static Lisp_Object data;
+
+static void 
+inotify_callback(int fd, void *_, int for_read) {
+  eassert(for_read);
+  eassert(data);
+
+  int to_read = 0;
+  if(ioctl(fd, FIONREAD,  &to_read) == -1)
+    report_file_error("ioctl(2) on inotify", Qnil);
+  char *const buffer = xmalloc(to_read);
+  ssize_t const n = read(fd, buffer, to_read);
+  if(n < 0)
+    report_file_error("read from inotify", Qnil);
+  else if(n < to_read)
+    {
+      // TODO
+      message1("n < to_read");
+    }
+
+  size_t i = 0;
+  while(i < (size_t)n)
+    {
+      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
+
+      Lisp_Object callback = Fassoc(make_number(ev->wd), data);
+      if(!NILP(callback))
+        {
+          Lisp_Object call[3];
+          call[0] = Fcar(Fcdr(Fcdr(callback))); /* callback */
+          call[1] = Fcar(Fcdr(callback));       /* path */
+
+          /* TODO check how to handle UTF-8 in file names:
+             make_string_from_bytes NCHARS vs. NBYTES */
+          /* ev->len seems to be an arbitrary large number
+             and not the exact length of ev->name */
+          size_t const len = strlen(ev->name);
+          /* if a directory is watched name contains the name
+             of the file that was changed */
+          Lisp_Object name = make_string_from_bytes (ev->name, len, len);
+
+          Lisp_Object events = Qnil;
+          if(ev->mask & (IN_MODIFY|IN_CREATE) )
+            events = Fcons(Fcons(QCmodify, name), events);
+          if(ev->mask & IN_MOVE_SELF)
+            events = Fcons(Fcons(QCmove, name), events);
+          if(ev->mask & IN_MOVED_FROM)
+            events = Fcons(Fcons(QCmove, Fcons(QCfrom, Fcons(name, make_number(ev->cookie)))), events);
+          if(ev->mask & IN_MOVED_TO)
+            events = Fcons(Fcons(QCmove, Fcons(QCto, Fcons(name, make_number(ev->cookie)))), events);
+          if(ev->mask & IN_ATTRIB)
+            events = Fcons(Fcons(QCattrib, name), events);
+          if(ev->mask & (IN_DELETE|IN_DELETE_SELF) )
+            events = Fcons(Fcons(QCdelete, name), events);
+      
+          if(!NILP(events))
+            {
+              call[2] = events;
+              Ffuncall(3, call);
+            }
+      
+          if(ev->mask & IN_IGNORED)
+            {
+              /* Event was removed automatically: Drop it from data list. */
+              message("File-watch: \"%s\" will be ignored", SSDATA(call[1]));
+              data = Fdelete(callback, data);
+            }
+          if(ev->mask & IN_Q_OVERFLOW)
+            message1("File watch: Inotify Queue Overflow!");
+        }
+
+      i += sizeof(*ev) + ev->len;
+    }
+
+  free(buffer);
+}
+
+DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, MANY, 0,
+       doc: /* Watch a file or directory.
+
+file-watch watches the file or directory given in PATH. If a change occurs
+CALLBACK is called with PATH as first argument and a list of changes as second
+argument. FLAGS can be
+
+:modify -- notify when a file is modified or created.
+
+:move -- notify when a file/directory is moved.
+
+:attrib -- notify when attributes change.
+
+:delete -- notify when a file/directory is deleted.
+
+:all -- notify for all of the above.
+
+Watching a directory is not recursive. CALLBACK receives the events as a list
+with each list element being a list containing information about an event. The
+first element is a flag symbol. If a directory is watched the second element is
+the name of the file that changed. If a file is moved from or to the directory
+the second element is either :from or :to and the third element is the file
+name. A fourth element contains a numeric identifier (cookie) that can be used
+to identify matching move operations if a file is moved inside the directory.
+
+Use `file-unwatch' to stop watching.
+
+usage: (file-watch PATH CALLBACK &rest FLAGS) */)
+  (size_t nargs, Lisp_Object *args)
+{
+  if(nargs < 3)
+    return Qnil;
+
+  CHECK_STRING(args[0]);
+
+  if(inotifyfd == uninitialized)
+    {
+      inotifyfd = inotify_init1(IN_NONBLOCK|IN_CLOEXEC);
+      if(inotifyfd == -1)
+        report_file_error("Initializing file watching", Qnil);
+      data = Qnil;
+      add_read_fd(inotifyfd, &inotify_callback, &data);
+    }
+  uint32_t mask = 0;
+  int i;
+  for(i = 2; i < nargs; ++i)
+    {
+      if(EQ(args[i], QCmodify))
+        mask |= IN_MODIFY | IN_CREATE;
+      else if(EQ(args[i], QCmove))
+        mask |= IN_MOVE_SELF | IN_MOVE;
+      else if(EQ(args[i], QCattrib))
+        mask |= IN_ATTRIB;
+      else if(EQ(args[i], QCdelete))
+        mask |= IN_DELETE_SELF | IN_DELETE;
+      else if(EQ(args[i], QCall))
+        mask |= IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
+          | IN_DELETE_SELF | IN_DELETE;
+      else /* TODO: should this be an error? */
+        message("Unkown parameter %s (ignored)", SSDATA(Fprin1_to_string(args[i], Qnil)));
+    }
+  int watchdesc = inotify_add_watch(inotifyfd, SSDATA(args[0]), mask);
+  if(watchdesc == -1) {
+    report_file_error("Watching file", Qnil);
+  }
+  data = Fcons(Fcons(make_number(watchdesc), Flist(2, args)), data);
+  return Qt;
+}
+
+DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
+       doc: /* Stop watching a file or directory. */)
+  (Lisp_Object path)
+{
+  if(inotifyfd == uninitialized)
+    return Qnil;
+  CHECK_STRING(path);
+
+  Lisp_Object x = data;
+  Lisp_Object cell, path_data;
+  while(!NILP(x))
+    {
+      cell = Fcar(x);
+      x = Fcdr(x);
+      path_data = Fcdr(cell);
+      if(!NILP(path_data) && STRINGP(Fcar(path_data))
+         && Fstring_equal(Fcar(path_data), path)
+         && NUMBERP(Fcar(cell)))
+        {
+          int const magicno = XINT(Fcar(cell));
+          data = Fdelete(cell, data);
+          if(inotify_rm_watch(inotifyfd, magicno) == -1)
+            report_file_error("Unwatch path", Qnil);
+          return Qt;
+        }
+    }
+  return Qnil;
+}
+
+#else /* HAVE_INOTIFY */
+#error "Filewatch defined but no watch mechanism (inotify) available"
+#endif /* HAVE_INOTIFY */
+
+void
+syms_of_filewatch (void)
+{
+  QCmodify = intern_c_string (":modify");
+  staticpro (&QCmodify);
+  QCmove = intern_c_string (":move");
+  staticpro (&QCmove);
+  QCattrib = intern_c_string (":attrib");
+  staticpro (&QCattrib);
+  QCdelete = intern_c_string (":delete");
+  staticpro (&QCdelete);
+  QCfrom = intern_c_string (":from");
+  staticpro (&QCfrom);
+  QCto = intern_c_string (":to");
+  staticpro (&QCto);
+  QCall = intern_c_string (":all");
+  staticpro (&QCall);
+
+  defsubr (&Sfile_watch);
+  defsubr (&Sfile_unwatch);
+}
+
+
+#endif /* HAVE_FILEWATCH */
diff --git a/src/lisp.h b/src/lisp.h
index 85838d1..3ed5281 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3405,6 +3405,11 @@ EXFUN (Fxw_display_color_p, 1);
 EXFUN (Fx_focus_frame, 1);
 #endif
 
+/* Defined in filewatch.c */
+#ifdef HAVE_FILEWATCH
+extern void syms_of_filewatch (void);
+#endif
+
 /* Defined in xfaces.c */
 extern Lisp_Object Qdefault, Qtool_bar, Qregion, Qfringe;
 extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor, Qborder, Qmouse, Qmenu;
-- 
1.7.5.2

Re: [PATCH] Support for filesystem watching (inotify)

[joakim]

Rüdiger Sonderfeld <ruediger@c-plusplus.de> writes:

> Hello,
> I wrote a patch to support watching for file system events. It currently only works with inotify (Linux) but it could be extended to support kqueue 
> (*BSD, OS X) or Win32. But I wanted to get some feedback first. Watching for filesystem events would be very useful for, e.g., dired or magit's status 
> view.

Very interesting! Have you signed copyright papers and so on?

>
> Here is a quick example (expects a directory foo containing a file bar):
>
> (file-watch "foo" #'(lambda (path events) (message "FS-Event: %s %s") path events)) :all)
> (delete-file "foo/bar")
> (file-unwatch "foo")
>
> So please tell me what you think of this.
>
> Regards,
> Rüdiger
>
>
> [PATCH] Added basic file system watching support.
>
> It currently only works with inotify/Linux. It adds file-watch and file-unwatch functions.
> ---
>  configure.in    |   14 +++
>  src/Makefile.in |    2 +-
>  src/emacs.c     |    4 +
>  src/filewatch.c |  242 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  src/lisp.h      |    5 +
>  5 files changed, 266 insertions(+), 1 deletions(-)
>  create mode 100644 src/filewatch.c
>
> diff --git a/configure.in b/configure.in
> index 77deef8..3263876 100644
> --- a/configure.in
> +++ b/configure.in
> @@ -169,6 +169,7 @@ OPTION_DEFAULT_ON([dbus],[don't compile with D-Bus support])
>  OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
>  OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
>  OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
> +OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) support])
>  
>  ## For the times when you want to build Emacs but don't have
>  ## a suitable makeinfo, and can live without the manuals.
> @@ -1978,6 +1979,19 @@ fi
>  AC_SUBST(LIBGNUTLS_LIBS)
>  AC_SUBST(LIBGNUTLS_CFLAGS)
>  
> +dnl inotify is only available on GNU/Linux.
> +HAVE_INOTIFY=no
> +if test "${with_inotify}" = "yes"; then
> +   AC_CHECK_HEADERS(sys/inotify.h)
> +   if test "$ac_cv_header_sys_inotify_h" = yes ; then
> +     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, HAVE_INOTIFY=yes)
> +   fi
> +fi
> +if test "${HAVE_INOTIFY}" = "yes"; then
> +   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
> +   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
> +fi
> +
>  dnl Do not put whitespace before the #include statements below.
>  dnl Older compilers (eg sunos4 cc) choke on it.
>  HAVE_XAW3D=no
> diff --git a/src/Makefile.in b/src/Makefile.in
> index e119596..0cd6d6e 100644
> --- a/src/Makefile.in
> +++ b/src/Makefile.in
> @@ -354,7 +354,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o $(XMENU_OBJ) window.o \
>  	syntax.o $(UNEXEC_OBJ) bytecode.o \
>  	process.o gnutls.o callproc.o \
>  	region-cache.o sound.o atimer.o \
> -	doprnt.o intervals.o textprop.o composite.o xml.o \
> +	doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
>  	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ)
>  obj = $(base_obj) $(NS_OBJC_OBJ)
>  
> diff --git a/src/emacs.c b/src/emacs.c
> index 6bdd255..5ca5cda 100644
> --- a/src/emacs.c
> +++ b/src/emacs.c
> @@ -1550,6 +1550,10 @@ main (int argc, char **argv)
>        syms_of_gnutls ();
>  #endif
>  
> +#ifdef HAVE_FILEWATCH
> +      syms_of_filewatch ();
> +#endif /* HAVE_FILEWATCH */
> +
>  #ifdef HAVE_DBUS
>        syms_of_dbusbind ();
>  #endif /* HAVE_DBUS */
> diff --git a/src/filewatch.c b/src/filewatch.c
> new file mode 100644
> index 0000000..5b7c130
> --- /dev/null
> +++ b/src/filewatch.c
> @@ -0,0 +1,242 @@
> +/* Watching file system changes.
> +
> +Copyright (C) 2011
> +  Free Software Foundation, Inc.
> +
> +This file is part of GNU Emacs.
> +
> +GNU Emacs is free software: you can redistribute it and/or modify
> +it under the terms of the GNU General Public License as published by
> +the Free Software Foundation, either version 3 of the License, or
> +(at your option) any later version.
> +
> +GNU Emacs is distributed in the hope that it will be useful,
> +but WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +GNU General Public License for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
> +
> +#include <config.h>
> +
> +#ifdef HAVE_FILEWATCH
> +
> +#include <setjmp.h>
> +
> +#include "lisp.h"
> +#include "process.h"
> +
> +static Lisp_Object QCmodify, QCmove, QCattrib, QCdelete, QCfrom, QCto, QCall;
> +
> +#ifdef HAVE_INOTIFY
> +#include <sys/inotify.h>
> +#include <sys/ioctl.h>
> +
> +enum { uninitialized = -100 };
> +static int inotifyfd = uninitialized;
> +
> +// Assoc list of files being watched.
> +static Lisp_Object data;
> +
> +static void 
> +inotify_callback(int fd, void *_, int for_read) {
> +  eassert(for_read);
> +  eassert(data);
> +
> +  int to_read = 0;
> +  if(ioctl(fd, FIONREAD,  &to_read) == -1)
> +    report_file_error("ioctl(2) on inotify", Qnil);
> +  char *const buffer = xmalloc(to_read);
> +  ssize_t const n = read(fd, buffer, to_read);
> +  if(n < 0)
> +    report_file_error("read from inotify", Qnil);
> +  else if(n < to_read)
> +    {
> +      // TODO
> +      message1("n < to_read");
> +    }
> +
> +  size_t i = 0;
> +  while(i < (size_t)n)
> +    {
> +      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
> +
> +      Lisp_Object callback = Fassoc(make_number(ev->wd), data);
> +      if(!NILP(callback))
> +        {
> +          Lisp_Object call[3];
> +          call[0] = Fcar(Fcdr(Fcdr(callback))); /* callback */
> +          call[1] = Fcar(Fcdr(callback));       /* path */
> +
> +          /* TODO check how to handle UTF-8 in file names:
> +             make_string_from_bytes NCHARS vs. NBYTES */
> +          /* ev->len seems to be an arbitrary large number
> +             and not the exact length of ev->name */
> +          size_t const len = strlen(ev->name);
> +          /* if a directory is watched name contains the name
> +             of the file that was changed */
> +          Lisp_Object name = make_string_from_bytes (ev->name, len, len);
> +
> +          Lisp_Object events = Qnil;
> +          if(ev->mask & (IN_MODIFY|IN_CREATE) )
> +            events = Fcons(Fcons(QCmodify, name), events);
> +          if(ev->mask & IN_MOVE_SELF)
> +            events = Fcons(Fcons(QCmove, name), events);
> +          if(ev->mask & IN_MOVED_FROM)
> +            events = Fcons(Fcons(QCmove, Fcons(QCfrom, Fcons(name, make_number(ev->cookie)))), events);
> +          if(ev->mask & IN_MOVED_TO)
> +            events = Fcons(Fcons(QCmove, Fcons(QCto, Fcons(name, make_number(ev->cookie)))), events);
> +          if(ev->mask & IN_ATTRIB)
> +            events = Fcons(Fcons(QCattrib, name), events);
> +          if(ev->mask & (IN_DELETE|IN_DELETE_SELF) )
> +            events = Fcons(Fcons(QCdelete, name), events);
> +      
> +          if(!NILP(events))
> +            {
> +              call[2] = events;
> +              Ffuncall(3, call);
> +            }
> +      
> +          if(ev->mask & IN_IGNORED)
> +            {
> +              /* Event was removed automatically: Drop it from data list. */
> +              message("File-watch: \"%s\" will be ignored", SSDATA(call[1]));
> +              data = Fdelete(callback, data);
> +            }
> +          if(ev->mask & IN_Q_OVERFLOW)
> +            message1("File watch: Inotify Queue Overflow!");
> +        }
> +
> +      i += sizeof(*ev) + ev->len;
> +    }
> +
> +  free(buffer);
> +}
> +
> +DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, MANY, 0,
> +       doc: /* Watch a file or directory.
> +
> +file-watch watches the file or directory given in PATH. If a change occurs
> +CALLBACK is called with PATH as first argument and a list of changes as second
> +argument. FLAGS can be
> +
> +:modify -- notify when a file is modified or created.
> +
> +:move -- notify when a file/directory is moved.
> +
> +:attrib -- notify when attributes change.
> +
> +:delete -- notify when a file/directory is deleted.
> +
> +:all -- notify for all of the above.
> +
> +Watching a directory is not recursive. CALLBACK receives the events as a list
> +with each list element being a list containing information about an event. The
> +first element is a flag symbol. If a directory is watched the second element is
> +the name of the file that changed. If a file is moved from or to the directory
> +the second element is either :from or :to and the third element is the file
> +name. A fourth element contains a numeric identifier (cookie) that can be used
> +to identify matching move operations if a file is moved inside the directory.
> +
> +Use `file-unwatch' to stop watching.
> +
> +usage: (file-watch PATH CALLBACK &rest FLAGS) */)
> +  (size_t nargs, Lisp_Object *args)
> +{
> +  if(nargs < 3)
> +    return Qnil;
> +
> +  CHECK_STRING(args[0]);
> +
> +  if(inotifyfd == uninitialized)
> +    {
> +      inotifyfd = inotify_init1(IN_NONBLOCK|IN_CLOEXEC);
> +      if(inotifyfd == -1)
> +        report_file_error("Initializing file watching", Qnil);
> +      data = Qnil;
> +      add_read_fd(inotifyfd, &inotify_callback, &data);
> +    }
> +  uint32_t mask = 0;
> +  int i;
> +  for(i = 2; i < nargs; ++i)
> +    {
> +      if(EQ(args[i], QCmodify))
> +        mask |= IN_MODIFY | IN_CREATE;
> +      else if(EQ(args[i], QCmove))
> +        mask |= IN_MOVE_SELF | IN_MOVE;
> +      else if(EQ(args[i], QCattrib))
> +        mask |= IN_ATTRIB;
> +      else if(EQ(args[i], QCdelete))
> +        mask |= IN_DELETE_SELF | IN_DELETE;
> +      else if(EQ(args[i], QCall))
> +        mask |= IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
> +          | IN_DELETE_SELF | IN_DELETE;
> +      else /* TODO: should this be an error? */
> +        message("Unkown parameter %s (ignored)", SSDATA(Fprin1_to_string(args[i], Qnil)));
> +    }
> +  int watchdesc = inotify_add_watch(inotifyfd, SSDATA(args[0]), mask);
> +  if(watchdesc == -1) {
> +    report_file_error("Watching file", Qnil);
> +  }
> +  data = Fcons(Fcons(make_number(watchdesc), Flist(2, args)), data);
> +  return Qt;
> +}
> +
> +DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
> +       doc: /* Stop watching a file or directory. */)
> +  (Lisp_Object path)
> +{
> +  if(inotifyfd == uninitialized)
> +    return Qnil;
> +  CHECK_STRING(path);
> +
> +  Lisp_Object x = data;
> +  Lisp_Object cell, path_data;
> +  while(!NILP(x))
> +    {
> +      cell = Fcar(x);
> +      x = Fcdr(x);
> +      path_data = Fcdr(cell);
> +      if(!NILP(path_data) && STRINGP(Fcar(path_data))
> +         && Fstring_equal(Fcar(path_data), path)
> +         && NUMBERP(Fcar(cell)))
> +        {
> +          int const magicno = XINT(Fcar(cell));
> +          data = Fdelete(cell, data);
> +          if(inotify_rm_watch(inotifyfd, magicno) == -1)
> +            report_file_error("Unwatch path", Qnil);
> +          return Qt;
> +        }
> +    }
> +  return Qnil;
> +}
> +
> +#else /* HAVE_INOTIFY */
> +#error "Filewatch defined but no watch mechanism (inotify) available"
> +#endif /* HAVE_INOTIFY */
> +
> +void
> +syms_of_filewatch (void)
> +{
> +  QCmodify = intern_c_string (":modify");
> +  staticpro (&QCmodify);
> +  QCmove = intern_c_string (":move");
> +  staticpro (&QCmove);
> +  QCattrib = intern_c_string (":attrib");
> +  staticpro (&QCattrib);
> +  QCdelete = intern_c_string (":delete");
> +  staticpro (&QCdelete);
> +  QCfrom = intern_c_string (":from");
> +  staticpro (&QCfrom);
> +  QCto = intern_c_string (":to");
> +  staticpro (&QCto);
> +  QCall = intern_c_string (":all");
> +  staticpro (&QCall);
> +
> +  defsubr (&Sfile_watch);
> +  defsubr (&Sfile_unwatch);
> +}
> +
> +
> +#endif /* HAVE_FILEWATCH */
> diff --git a/src/lisp.h b/src/lisp.h
> index 85838d1..3ed5281 100644
> --- a/src/lisp.h
> +++ b/src/lisp.h
> @@ -3405,6 +3405,11 @@ EXFUN (Fxw_display_color_p, 1);
>  EXFUN (Fx_focus_frame, 1);
>  #endif
>  
> +/* Defined in filewatch.c */
> +#ifdef HAVE_FILEWATCH
> +extern void syms_of_filewatch (void);
> +#endif
> +
>  /* Defined in xfaces.c */
>  extern Lisp_Object Qdefault, Qtool_bar, Qregion, Qfringe;
>  extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor, Qborder, Qmouse, Qmenu;

-- 
Joakim Verona

Re: [PATCH] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

On Saturday 04 June 2011 10:52:07 joakim@verona.se wrote:
> Rüdiger Sonderfeld <ruediger@c-plusplus.de> writes:
> > Hello,
> > I wrote a patch to support watching for file system events. It currently
> > only works with inotify (Linux) but it could be extended to support
> > kqueue (*BSD, OS X) or Win32. But I wanted to get some feedback first.
> > Watching for filesystem events would be very useful for, e.g., dired or
> > magit's status view.
> 
> Very interesting! Have you signed copyright papers and so on?

Yes. Since 25 May 2011.

Re: [PATCH] Support for filesystem watching (inotify)

[Jan Djärv]

Hi. 
In general, you should not call Lisp from the read fd callback. Recursive calls into Lisp may fail. Instead post a Lisp event and handle it in keyboard.c and some Lisp code.

    Jan D.

4 jun 2011 kl. 00:34 skrev Rüdiger Sonderfeld <ruediger@c-plusplus.de>:

> Hello,
> I wrote a patch to support watching for file system events. It currently only works with inotify (Linux) but it could be extended to support kqueue 
> (*BSD, OS X) or Win32. But I wanted to get some feedback first. Watching for filesystem events would be very useful for, e.g., dired or magit's status 
> view.
> 
> Here is a quick example (expects a directory foo containing a file bar):
> 
> (file-watch "foo" #'(lambda (path events) (message "FS-Event: %s %s") path events)) :all)
> (delete-file "foo/bar")
> (file-unwatch "foo")
> 
> So please tell me what you think of this.
> 
> Regards,
> Rüdiger
> 
> 
> [PATCH] Added basic file system watching support.
> 
> It currently only works with inotify/Linux. It adds file-watch and file-unwatch functions.
> ---
> configure.in    |   14 +++
> src/Makefile.in |    2 +-
> src/emacs.c     |    4 +
> src/filewatch.c |  242 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
> src/lisp.h      |    5 +
> 5 files changed, 266 insertions(+), 1 deletions(-)
> create mode 100644 src/filewatch.c
> 
> diff --git a/configure.in b/configure.in
> index 77deef8..3263876 100644
> --- a/configure.in
> +++ b/configure.in
> @@ -169,6 +169,7 @@ OPTION_DEFAULT_ON([dbus],[don't compile with D-Bus support])
> OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
> OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
> OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
> +OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) support])
> 
> ## For the times when you want to build Emacs but don't have
> ## a suitable makeinfo, and can live without the manuals.
> @@ -1978,6 +1979,19 @@ fi
> AC_SUBST(LIBGNUTLS_LIBS)
> AC_SUBST(LIBGNUTLS_CFLAGS)
> 
> +dnl inotify is only available on GNU/Linux.
> +HAVE_INOTIFY=no
> +if test "${with_inotify}" = "yes"; then
> +   AC_CHECK_HEADERS(sys/inotify.h)
> +   if test "$ac_cv_header_sys_inotify_h" = yes ; then
> +     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, HAVE_INOTIFY=yes)
> +   fi
> +fi
> +if test "${HAVE_INOTIFY}" = "yes"; then
> +   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
> +   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
> +fi
> +
> dnl Do not put whitespace before the #include statements below.
> dnl Older compilers (eg sunos4 cc) choke on it.
> HAVE_XAW3D=no
> diff --git a/src/Makefile.in b/src/Makefile.in
> index e119596..0cd6d6e 100644
> --- a/src/Makefile.in
> +++ b/src/Makefile.in
> @@ -354,7 +354,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o $(XMENU_OBJ) window.o \
>    syntax.o $(UNEXEC_OBJ) bytecode.o \
>    process.o gnutls.o callproc.o \
>    region-cache.o sound.o atimer.o \
> -    doprnt.o intervals.o textprop.o composite.o xml.o \
> +    doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
>    $(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ)
> obj = $(base_obj) $(NS_OBJC_OBJ)
> 
> diff --git a/src/emacs.c b/src/emacs.c
> index 6bdd255..5ca5cda 100644
> --- a/src/emacs.c
> +++ b/src/emacs.c
> @@ -1550,6 +1550,10 @@ main (int argc, char **argv)
>       syms_of_gnutls ();
> #endif
> 
> +#ifdef HAVE_FILEWATCH
> +      syms_of_filewatch ();
> +#endif /* HAVE_FILEWATCH */
> +
> #ifdef HAVE_DBUS
>       syms_of_dbusbind ();
> #endif /* HAVE_DBUS */
> diff --git a/src/filewatch.c b/src/filewatch.c
> new file mode 100644
> index 0000000..5b7c130
> --- /dev/null
> +++ b/src/filewatch.c
> @@ -0,0 +1,242 @@
> +/* Watching file system changes.
> +
> +Copyright (C) 2011
> +  Free Software Foundation, Inc.
> +
> +This file is part of GNU Emacs.
> +
> +GNU Emacs is free software: you can redistribute it and/or modify
> +it under the terms of the GNU General Public License as published by
> +the Free Software Foundation, either version 3 of the License, or
> +(at your option) any later version.
> +
> +GNU Emacs is distributed in the hope that it will be useful,
> +but WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +GNU General Public License for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
> +
> +#include <config.h>
> +
> +#ifdef HAVE_FILEWATCH
> +
> +#include <setjmp.h>
> +
> +#include "lisp.h"
> +#include "process.h"
> +
> +static Lisp_Object QCmodify, QCmove, QCattrib, QCdelete, QCfrom, QCto, QCall;
> +
> +#ifdef HAVE_INOTIFY
> +#include <sys/inotify.h>
> +#include <sys/ioctl.h>
> +
> +enum { uninitialized = -100 };
> +static int inotifyfd = uninitialized;
> +
> +// Assoc list of files being watched.
> +static Lisp_Object data;
> +
> +static void 
> +inotify_callback(int fd, void *_, int for_read) {
> +  eassert(for_read);
> +  eassert(data);
> +
> +  int to_read = 0;
> +  if(ioctl(fd, FIONREAD,  &to_read) == -1)
> +    report_file_error("ioctl(2) on inotify", Qnil);
> +  char *const buffer = xmalloc(to_read);
> +  ssize_t const n = read(fd, buffer, to_read);
> +  if(n < 0)
> +    report_file_error("read from inotify", Qnil);
> +  else if(n < to_read)
> +    {
> +      // TODO
> +      message1("n < to_read");
> +    }
> +
> +  size_t i = 0;
> +  while(i < (size_t)n)
> +    {
> +      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
> +
> +      Lisp_Object callback = Fassoc(make_number(ev->wd), data);
> +      if(!NILP(callback))
> +        {
> +          Lisp_Object call[3];
> +          call[0] = Fcar(Fcdr(Fcdr(callback))); /* callback */
> +          call[1] = Fcar(Fcdr(callback));       /* path */
> +
> +          /* TODO check how to handle UTF-8 in file names:
> +             make_string_from_bytes NCHARS vs. NBYTES */
> +          /* ev->len seems to be an arbitrary large number
> +             and not the exact length of ev->name */
> +          size_t const len = strlen(ev->name);
> +          /* if a directory is watched name contains the name
> +             of the file that was changed */
> +          Lisp_Object name = make_string_from_bytes (ev->name, len, len);
> +
> +          Lisp_Object events = Qnil;
> +          if(ev->mask & (IN_MODIFY|IN_CREATE) )
> +            events = Fcons(Fcons(QCmodify, name), events);
> +          if(ev->mask & IN_MOVE_SELF)
> +            events = Fcons(Fcons(QCmove, name), events);
> +          if(ev->mask & IN_MOVED_FROM)
> +            events = Fcons(Fcons(QCmove, Fcons(QCfrom, Fcons(name, make_number(ev->cookie)))), events);
> +          if(ev->mask & IN_MOVED_TO)
> +            events = Fcons(Fcons(QCmove, Fcons(QCto, Fcons(name, make_number(ev->cookie)))), events);
> +          if(ev->mask & IN_ATTRIB)
> +            events = Fcons(Fcons(QCattrib, name), events);
> +          if(ev->mask & (IN_DELETE|IN_DELETE_SELF) )
> +            events = Fcons(Fcons(QCdelete, name), events);
> +      
> +          if(!NILP(events))
> +            {
> +              call[2] = events;
> +              Ffuncall(3, call);
> +            }
> +      
> +          if(ev->mask & IN_IGNORED)
> +            {
> +              /* Event was removed automatically: Drop it from data list. */
> +              message("File-watch: \"%s\" will be ignored", SSDATA(call[1]));
> +              data = Fdelete(callback, data);
> +            }
> +          if(ev->mask & IN_Q_OVERFLOW)
> +            message1("File watch: Inotify Queue Overflow!");
> +        }
> +
> +      i += sizeof(*ev) + ev->len;
> +    }
> +
> +  free(buffer);
> +}
> +
> +DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, MANY, 0,
> +       doc: /* Watch a file or directory.
> +
> +file-watch watches the file or directory given in PATH. If a change occurs
> +CALLBACK is called with PATH as first argument and a list of changes as second
> +argument. FLAGS can be
> +
> +:modify -- notify when a file is modified or created.
> +
> +:move -- notify when a file/directory is moved.
> +
> +:attrib -- notify when attributes change.
> +
> +:delete -- notify when a file/directory is deleted.
> +
> +:all -- notify for all of the above.
> +
> +Watching a directory is not recursive. CALLBACK receives the events as a list
> +with each list element being a list containing information about an event. The
> +first element is a flag symbol. If a directory is watched the second element is
> +the name of the file that changed. If a file is moved from or to the directory
> +the second element is either :from or :to and the third element is the file
> +name. A fourth element contains a numeric identifier (cookie) that can be used
> +to identify matching move operations if a file is moved inside the directory.
> +
> +Use `file-unwatch' to stop watching.
> +
> +usage: (file-watch PATH CALLBACK &rest FLAGS) */)
> +  (size_t nargs, Lisp_Object *args)
> +{
> +  if(nargs < 3)
> +    return Qnil;
> +
> +  CHECK_STRING(args[0]);
> +
> +  if(inotifyfd == uninitialized)
> +    {
> +      inotifyfd = inotify_init1(IN_NONBLOCK|IN_CLOEXEC);
> +      if(inotifyfd == -1)
> +        report_file_error("Initializing file watching", Qnil);
> +      data = Qnil;
> +      add_read_fd(inotifyfd, &inotify_callback, &data);
> +    }
> +  uint32_t mask = 0;
> +  int i;
> +  for(i = 2; i < nargs; ++i)
> +    {
> +      if(EQ(args[i], QCmodify))
> +        mask |= IN_MODIFY | IN_CREATE;
> +      else if(EQ(args[i], QCmove))
> +        mask |= IN_MOVE_SELF | IN_MOVE;
> +      else if(EQ(args[i], QCattrib))
> +        mask |= IN_ATTRIB;
> +      else if(EQ(args[i], QCdelete))
> +        mask |= IN_DELETE_SELF | IN_DELETE;
> +      else if(EQ(args[i], QCall))
> +        mask |= IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
> +          | IN_DELETE_SELF | IN_DELETE;
> +      else /* TODO: should this be an error? */
> +        message("Unkown parameter %s (ignored)", SSDATA(Fprin1_to_string(args[i], Qnil)));
> +    }
> +  int watchdesc = inotify_add_watch(inotifyfd, SSDATA(args[0]), mask);
> +  if(watchdesc == -1) {
> +    report_file_error("Watching file", Qnil);
> +  }
> +  data = Fcons(Fcons(make_number(watchdesc), Flist(2, args)), data);
> +  return Qt;
> +}
> +
> +DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
> +       doc: /* Stop watching a file or directory. */)
> +  (Lisp_Object path)
> +{
> +  if(inotifyfd == uninitialized)
> +    return Qnil;
> +  CHECK_STRING(path);
> +
> +  Lisp_Object x = data;
> +  Lisp_Object cell, path_data;
> +  while(!NILP(x))
> +    {
> +      cell = Fcar(x);
> +      x = Fcdr(x);
> +      path_data = Fcdr(cell);
> +      if(!NILP(path_data) && STRINGP(Fcar(path_data))
> +         && Fstring_equal(Fcar(path_data), path)
> +         && NUMBERP(Fcar(cell)))
> +        {
> +          int const magicno = XINT(Fcar(cell));
> +          data = Fdelete(cell, data);
> +          if(inotify_rm_watch(inotifyfd, magicno) == -1)
> +            report_file_error("Unwatch path", Qnil);
> +          return Qt;
> +        }
> +    }
> +  return Qnil;
> +}
> +
> +#else /* HAVE_INOTIFY */
> +#error "Filewatch defined but no watch mechanism (inotify) available"
> +#endif /* HAVE_INOTIFY */
> +
> +void
> +syms_of_filewatch (void)
> +{
> +  QCmodify = intern_c_string (":modify");
> +  staticpro (&QCmodify);
> +  QCmove = intern_c_string (":move");
> +  staticpro (&QCmove);
> +  QCattrib = intern_c_string (":attrib");
> +  staticpro (&QCattrib);
> +  QCdelete = intern_c_string (":delete");
> +  staticpro (&QCdelete);
> +  QCfrom = intern_c_string (":from");
> +  staticpro (&QCfrom);
> +  QCto = intern_c_string (":to");
> +  staticpro (&QCto);
> +  QCall = intern_c_string (":all");
> +  staticpro (&QCall);
> +
> +  defsubr (&Sfile_watch);
> +  defsubr (&Sfile_unwatch);
> +}
> +
> +
> +#endif /* HAVE_FILEWATCH */
> diff --git a/src/lisp.h b/src/lisp.h
> index 85838d1..3ed5281 100644
> --- a/src/lisp.h
> +++ b/src/lisp.h
> @@ -3405,6 +3405,11 @@ EXFUN (Fxw_display_color_p, 1);
> EXFUN (Fx_focus_frame, 1);
> #endif
> 
> +/* Defined in filewatch.c */
> +#ifdef HAVE_FILEWATCH
> +extern void syms_of_filewatch (void);
> +#endif
> +
> /* Defined in xfaces.c */
> extern Lisp_Object Qdefault, Qtool_bar, Qregion, Qfringe;
> extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor, Qborder, Qmouse, Qmenu;
> -- 
> 1.7.5.2
> 

Re: [PATCH update2] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

Hi,

On Saturday 04 June 2011 12:43:37 you wrote:
> In general, you should not call Lisp from the read fd callback. Recursive
> calls into Lisp may fail. Instead post a Lisp event and handle it in
> keyboard.c and some Lisp code.

Thanks for your advise. I changed the code to use events. I imitated the
behaviour of the dbus subsystem. I hope this is what you had in mind.

Subject: [PATCH] Added basic file system watching support.

It currently only works with inotify/Linux. It adds file-watch and file-
unwatch functions.
---
 configure.in    |   14 +++
 src/Makefile.in |    2 +-
 src/emacs.c     |    4 +
 src/filewatch.c |  268 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/lisp.h      |    5 +
 5 files changed, 292 insertions(+), 1 deletions(-)
 create mode 100644 src/filewatch.c

diff --git a/configure.in b/configure.in
index 06880ea..5696fa2 100644
--- a/configure.in
+++ b/configure.in
@@ -169,6 +169,7 @@ OPTION_DEFAULT_ON([dbus],[don't compile with D-Bus 
support])
 OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
 OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
 OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
+OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) 
support])
 
 ## For the times when you want to build Emacs but don't have
 ## a suitable makeinfo, and can live without the manuals.
@@ -1981,6 +1982,19 @@ fi
 AC_SUBST(LIBGNUTLS_LIBS)
 AC_SUBST(LIBGNUTLS_CFLAGS)
 
+dnl inotify is only available on GNU/Linux.
+HAVE_INOTIFY=no
+if test "${with_inotify}" = "yes"; then
+   AC_CHECK_HEADERS(sys/inotify.h)
+   if test "$ac_cv_header_sys_inotify_h" = yes ; then
+     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, 
HAVE_INOTIFY=yes)
+   fi
+fi
+if test "${HAVE_INOTIFY}" = "yes"; then
+   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
+   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
+fi
+
 dnl Do not put whitespace before the #include statements below.
 dnl Older compilers (eg sunos4 cc) choke on it.
 HAVE_XAW3D=no
diff --git a/src/Makefile.in b/src/Makefile.in
index c4250b9..9135e7d 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -334,7 +334,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o 
$(XMENU_OBJ) window.o \
 	syntax.o $(UNEXEC_OBJ) bytecode.o \
 	process.o gnutls.o callproc.o \
 	region-cache.o sound.o atimer.o \
-	doprnt.o intervals.o textprop.o composite.o xml.o \
+	doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
 	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ)
 obj = $(base_obj) $(NS_OBJC_OBJ)
 
diff --git a/src/emacs.c b/src/emacs.c
index 3a7c5c0..fb734e5 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -1558,6 +1558,10 @@ main (int argc, char **argv)
       syms_of_gnutls ();
 #endif
 
+#ifdef HAVE_FILEWATCH
+      syms_of_filewatch ();
+#endif /* HAVE_FILEWATCH */
+
 #ifdef HAVE_DBUS
       syms_of_dbusbind ();
 #endif /* HAVE_DBUS */
diff --git a/src/filewatch.c b/src/filewatch.c
new file mode 100644
index 0000000..55da25f
--- /dev/null
+++ b/src/filewatch.c
@@ -0,0 +1,268 @@
+/* Watching file system changes.
+
+Copyright (C) 2011
+  Free Software Foundation, Inc.
+
+This file is part of GNU Emacs.
+
+GNU Emacs is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+GNU Emacs is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <config.h>
+
+#ifdef HAVE_FILEWATCH
+#include <setjmp.h>
+#include "lisp.h"
+#include "coding.h"
+#include "process.h"
+
+static Lisp_Object QCmodify, QCmove, QCattrib, QCdelete, QCfrom, QCto, QCall;
+
+#ifdef HAVE_INOTIFY
+#include <sys/inotify.h>
+#include <sys/ioctl.h>
+
+enum { uninitialized = -100 };
+static int inotifyfd = uninitialized;
+
+/* Assoc list of files being watched.  */
+static Lisp_Object watch_list;
+
+static void
+inotify_callback(int fd, void *_, int for_read)
+{
+  int to_read;
+  char *buffer;
+  ssize_t n;
+  size_t i;
+
+  if(!for_read)
+    return;
+
+  to_read = 0;
+  if(ioctl(fd, FIONREAD,  &to_read) == -1)
+    report_file_error("Error while trying to retrieve file system events",
+                      Qnil);
+  buffer = xmalloc(to_read);
+  n = read(fd, buffer, to_read);
+  if(n < 0)
+    report_file_error("Error while trying to read file system events",
+                      Qnil);
+
+  i = 0;
+  while(i < (size_t)n)
+    {
+      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
+
+      Lisp_Object callback = Fassoc(make_number(ev->wd), watch_list);
+      if(!NILP(callback))
+        {
+          size_t len;
+          Lisp_Object name, events;
+          Lisp_Object call[3];
+          call[0] = Fcar(Fcdr(Fcdr(callback))); /* callback */
+          call[1] = Fcar(Fcdr(callback));       /* file name */
+
+          /* If a directory is watched name contains the name
+             of the file that was changed.  */
+          len = strlen(ev->name);
+          name = make_unibyte_string (ev->name, len);
+          name = DECODE_FILE(name);
+
+          events = Qnil;
+          if(ev->mask & (IN_MODIFY|IN_CREATE) )
+            events = Fcons(Fcons(QCmodify, name), events);
+          if(ev->mask & IN_MOVE_SELF)
+            events = Fcons(Fcons(QCmove, name), events);
+          if(ev->mask & IN_MOVED_FROM)
+            events = Fcons(Fcons(QCmove,
+                                 Fcons(QCfrom,
+                                       Fcons(name, make_number(ev-
>cookie)))),
+                           events);
+          if(ev->mask & IN_MOVED_TO)
+            events = Fcons(Fcons(QCmove,
+                                 Fcons(QCto,
+                                       Fcons(name, make_number(ev-
>cookie)))),
+                           events);
+          if(ev->mask & IN_ATTRIB)
+            events = Fcons(Fcons(QCattrib, name), events);
+          if(ev->mask & (IN_DELETE|IN_DELETE_SELF) )
+            events = Fcons(Fcons(QCdelete, name), events);
+
+          if(!NILP(events))
+            {
+              call[2] = events;
+              Ffuncall(3, call);
+            }
+
+          if(ev->mask & IN_IGNORED)
+            {
+              /* Event was removed automatically: Drop it from data list.  */
+              add_to_log("File-watch: \"%s\" will be ignored", call[1], 
Qnil);
+              watch_list = Fdelete(callback, watch_list);
+            }
+          if(ev->mask & IN_Q_OVERFLOW)
+            add_to_log("File watch: Inotify Queue Overflow!", Qnil, Qnil);
+        }
+
+      i += sizeof(*ev) + ev->len;
+    }
+
+  xfree(buffer);
+}
+
+DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, MANY, 0,
+       doc: /* Watch a file or directory.
+
+file-watch watches the file or directory given in FILENAME.  If a change 
occurs
+CALLBACK is called with FILENAME as first argument and a list of changes as 
second
+argument.  FLAGS can be
+
+:modify -- notify when a file is modified or created.
+
+:move -- notify when a file/directory is moved.
+
+:attrib -- notify when attributes change.
+
+:delete -- notify when a file/directory is deleted.
+
+:all -- notify for all of the above.
+
+Watching a directory is not recursive.  CALLBACK receives the events as a 
list
+with each list element being a list containing information about an event.  
The
+first element is a flag symbol.  If a directory is watched the second element 
is
+the name of the file that changed.  If a file is moved from or to the 
directory
+the second element is either :from or :to and the third element is the file
+name.  A fourth element contains a numeric identifier (cookie) that can be 
used
+to identify matching move operations if a file is moved inside the directory.
+
+Example:
+(file-watch "foo" #'(lambda (file event) (message "%s %s" file event)) :all)
+
+Use `file-unwatch' to stop watching.
+
+usage: (file-watch FILENAME CALLBACK &rest FLAGS) */)
+  (size_t nargs, Lisp_Object *args)
+{
+  uint32_t mask;
+  size_t i;
+  int watchdesc;
+  Lisp_Object decoded_file_name;
+
+  if(nargs < 3)
+    return Qnil;
+
+  CHECK_STRING(args[0]);
+
+  if(inotifyfd == uninitialized)
+    {
+      inotifyfd = inotify_init1(IN_NONBLOCK|IN_CLOEXEC);
+      if(inotifyfd == -1)
+        report_file_error("File watching feature (inotify) is not available",
+                          Qnil);
+      watch_list = Qnil;
+      add_read_fd(inotifyfd, &inotify_callback, &watch_list);
+    }
+  mask = 0;
+  for(i = 2; i < nargs; ++i)
+    {
+      if(EQ(args[i], QCmodify))
+        mask |= IN_MODIFY | IN_CREATE;
+      else if(EQ(args[i], QCmove))
+        mask |= IN_MOVE_SELF | IN_MOVE;
+      else if(EQ(args[i], QCattrib))
+        mask |= IN_ATTRIB;
+      else if(EQ(args[i], QCdelete))
+        mask |= IN_DELETE_SELF | IN_DELETE;
+      else if(EQ(args[i], QCall))
+        mask |= IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
+          | IN_DELETE_SELF | IN_DELETE;
+      else /* TODO: should this be an error? */
+        message("Unkown parameter %s (ignored)",
+                SSDATA(Fprin1_to_string(args[i], Qnil)));
+    }
+  decoded_file_name = ENCODE_FILE(args[0]);
+  watchdesc = inotify_add_watch(inotifyfd, SSDATA(decoded_file_name), mask);
+  if(watchdesc == -1) {
+    report_file_error("Could not watch file", Fcons(args[0], Qnil));
+  }
+  /* TODO: check if file is already in the watch_list.  */
+  watch_list = Fcons(Fcons(make_number(watchdesc), Flist(2, args)), 
watch_list);
+  return Qt;
+}
+
+DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
+       doc: /* Stop watching a file or directory.  */)
+  (Lisp_Object file_name)
+{
+  Lisp_Object cell, file_name_data;
+  Lisp_Object x = watch_list;
+
+  if(inotifyfd == uninitialized)
+    return Qnil;
+  CHECK_STRING(file_name);
+
+  while(!NILP(x))
+    {
+      cell = Fcar(x);
+      x = Fcdr(x);
+      file_name_data = Fcdr(cell);
+      if(!NILP(file_name_data) && STRINGP(Fcar(file_name_data))
+         && Fstring_equal(Fcar(file_name_data), file_name)
+         && NUMBERP(Fcar(cell)))
+        {
+          int const magicno = XINT(Fcar(cell));
+          watch_list = Fdelete(cell, watch_list);
+          if(inotify_rm_watch(inotifyfd, magicno) == -1)
+            report_file_error("Could not unwatch file", Fcons(file_name, 
Qnil));
+          /* Cleanup if watch_list is empty.  */
+          if(NILP(watch_list))
+            {
+              close(inotifyfd);
+              delete_read_fd(inotifyfd);
+              inotifyfd = uninitialized;
+            }
+          return Qt;
+        }
+    }
+  return Qnil;
+}
+
+#else /* HAVE_INOTIFY */
+#error "Filewatch defined but no watch mechanism (inotify) available"
+#endif /* HAVE_INOTIFY */
+
+void
+syms_of_filewatch (void)
+{
+  QCmodify = intern_c_string (":modify");
+  staticpro (&QCmodify);
+  QCmove = intern_c_string (":move");
+  staticpro (&QCmove);
+  QCattrib = intern_c_string (":attrib");
+  staticpro (&QCattrib);
+  QCdelete = intern_c_string (":delete");
+  staticpro (&QCdelete);
+  QCfrom = intern_c_string (":from");
+  staticpro (&QCfrom);
+  QCto = intern_c_string (":to");
+  staticpro (&QCto);
+  QCall = intern_c_string (":all");
+  staticpro (&QCall);
+
+  defsubr (&Sfile_watch);
+  defsubr (&Sfile_unwatch);
+}
+
+
+#endif /* HAVE_FILEWATCH */
diff --git a/src/lisp.h b/src/lisp.h
index 8a504e8..8b21e0e 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3430,6 +3430,11 @@ EXFUN (Fxw_display_color_p, 1);
 EXFUN (Fx_focus_frame, 1);
 #endif
 
+/* Defined in filewatch.c */
+#ifdef HAVE_FILEWATCH
+extern void syms_of_filewatch (void);
+#endif
+
 /* Defined in xfaces.c */
 extern Lisp_Object Qdefault, Qtool_bar, Qfringe;
 extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor;
-- 
1.7.5.2

Re: [PATCH update2] Support for filesystem watching (inotify)

[Eli Zaretskii]

> From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> Date: Sun, 5 Jun 2011 01:36:35 +0200
> 
> > In general, you should not call Lisp from the read fd callback. Recursive
> > calls into Lisp may fail. Instead post a Lisp event and handle it in
> > keyboard.c and some Lisp code.
> 
> Thanks for your advise. I changed the code to use events.

You did?  Did you send the correct patch?

Re: [PATCH update3] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

On Sunday 05 June 2011 07:45:58 Eli Zaretskii wrote:
> > From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> > Date: Sun, 5 Jun 2011 01:36:35 +0200
> > 
> > > In general, you should not call Lisp from the read fd callback.
> > > Recursive calls into Lisp may fail. Instead post a Lisp event and
> > > handle it in keyboard.c and some Lisp code.
> > 
> > Thanks for your advise. I changed the code to use events.
> 
> You did?  Did you send the correct patch?

Oh. This should be the correct patch.

Subject: [PATCH] Added basic file system watching support.

It currently only works with inotify/Linux. It adds file-watch and file-unwatch functions.
---
 configure.in      |   14 +++
 lisp/filewatch.el |   54 ++++++++++
 src/Makefile.in   |    2 +-
 src/emacs.c       |    4 +
 src/filewatch.c   |  305 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/keyboard.c    |   28 +++++
 src/lisp.h        |    5 +
 src/termhooks.h   |    5 +
 8 files changed, 416 insertions(+), 1 deletions(-)
 create mode 100644 lisp/filewatch.el
 create mode 100644 src/filewatch.c

diff --git a/configure.in b/configure.in
index 06880ea..5696fa2 100644
--- a/configure.in
+++ b/configure.in
@@ -169,6 +169,7 @@ OPTION_DEFAULT_ON([dbus],[don't compile with D-Bus support])
 OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
 OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
 OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
+OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) support])
 
 ## For the times when you want to build Emacs but don't have
 ## a suitable makeinfo, and can live without the manuals.
@@ -1981,6 +1982,19 @@ fi
 AC_SUBST(LIBGNUTLS_LIBS)
 AC_SUBST(LIBGNUTLS_CFLAGS)
 
+dnl inotify is only available on GNU/Linux.
+HAVE_INOTIFY=no
+if test "${with_inotify}" = "yes"; then
+   AC_CHECK_HEADERS(sys/inotify.h)
+   if test "$ac_cv_header_sys_inotify_h" = yes ; then
+     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, HAVE_INOTIFY=yes)
+   fi
+fi
+if test "${HAVE_INOTIFY}" = "yes"; then
+   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
+   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
+fi
+
 dnl Do not put whitespace before the #include statements below.
 dnl Older compilers (eg sunos4 cc) choke on it.
 HAVE_XAW3D=no
diff --git a/lisp/filewatch.el b/lisp/filewatch.el
new file mode 100644
index 0000000..2c97589
--- /dev/null
+++ b/lisp/filewatch.el
@@ -0,0 +1,54 @@
+;;; filewatch.el --- Elisp support for watching filesystem events.
+
+;; Copyright (C) 2011 Free Software Foundation, Inc.
+
+;; Author: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
+;; Keywords: files
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; This file contains the elisp part of the filewatch API.
+
+;;; Code:
+
+(eval-when-compile
+  (require 'cl))
+
+(defun filewatch-event-p (event)
+  "Check if EVENT is a filewatch event."
+  (and (listp event)
+       (eq (car event) 'filewatch-event)))
+
+;;;###autoload
+(defun filewatch-handle-event (event)
+  "Handle file system monitoring event.
+If EVENT is a filewatch event then the callback is called.  If EVENT is
+not a filewatch event then a `filewatch-error' is signaled."
+  (interactive "e")
+
+  (unless (filewatch-event-p event)
+    (signal 'filewatch-error (cons "Not a valid filewatch event" event)))
+
+  (loop for ev in (cdr event)
+     unless (and (listp ev) (>= (length ev) 3))
+     do (signal 'filewatch-error (cons "Not a valid filewatch event" event))
+     do (funcall (cadr ev) (car ev) (caddr ev))))
+
+(provide 'filewatch)
+
+;;; filewatch.el ends here
diff --git a/src/Makefile.in b/src/Makefile.in
index c4250b9..9135e7d 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -334,7 +334,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o $(XMENU_OBJ) window.o \
 	syntax.o $(UNEXEC_OBJ) bytecode.o \
 	process.o gnutls.o callproc.o \
 	region-cache.o sound.o atimer.o \
-	doprnt.o intervals.o textprop.o composite.o xml.o \
+	doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
 	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ)
 obj = $(base_obj) $(NS_OBJC_OBJ)
 
diff --git a/src/emacs.c b/src/emacs.c
index 3a7c5c0..fb734e5 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -1558,6 +1558,10 @@ main (int argc, char **argv)
       syms_of_gnutls ();
 #endif
 
+#ifdef HAVE_FILEWATCH
+      syms_of_filewatch ();
+#endif /* HAVE_FILEWATCH */
+
 #ifdef HAVE_DBUS
       syms_of_dbusbind ();
 #endif /* HAVE_DBUS */
diff --git a/src/filewatch.c b/src/filewatch.c
new file mode 100644
index 0000000..56b79b7
--- /dev/null
+++ b/src/filewatch.c
@@ -0,0 +1,305 @@
+/* Watching file system changes.
+
+Copyright (C) 2011
+  Free Software Foundation, Inc.
+
+This file is part of GNU Emacs.
+
+GNU Emacs is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+GNU Emacs is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <config.h>
+
+#ifdef HAVE_FILEWATCH
+#include <setjmp.h> /* Required for lisp.h.  */
+#include "lisp.h"
+#include "coding.h"
+#include "process.h"
+#include "keyboard.h"
+#include "character.h"
+#include "frame.h" /* Required for termhooks.h.  */
+#include "termhooks.h"
+
+static Lisp_Object Qmodify, Qmove, Qattrib, Qdelete, Qfrom, Qto, Qunkown_aspect;
+
+#ifdef HAVE_INOTIFY
+#include <sys/inotify.h>
+#include <sys/ioctl.h>
+
+enum { uninitialized = -100 };
+/* File handle for inotify.  */
+static int inotifyfd = uninitialized;
+
+/* Assoc list of files being watched.  */
+static Lisp_Object watch_list;
+
+/* This callback is called when the FD is available FOR_READ.  The inotify
+   events are read from FD and converted into input_events.  */
+
+static void
+inotify_callback (int fd, void *_, int for_read)
+{
+  struct input_event event;
+  int to_read;
+  char *buffer;
+  ssize_t n;
+  size_t i;
+
+  if (!for_read)
+    return;
+
+  to_read = 0;
+  if (ioctl (fd, FIONREAD,  &to_read) == -1)
+    report_file_error ("Error while trying to retrieve file system events",
+                       Qnil);
+  buffer = xmalloc (to_read);
+  n = read (fd, buffer, to_read);
+  if (n < 0)
+    {
+      xfree (buffer);
+      report_file_error ("Error while trying to read file system events",
+                         Qnil);
+    }
+
+  EVENT_INIT (event);
+  event.kind = FILEWATCH_EVENT;
+  event.arg = Qnil;
+
+  i = 0;
+  while (i < (size_t)n)
+    {
+      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
+
+      Lisp_Object callback = Fassoc (make_number (ev->wd), watch_list);
+      if (!NILP (callback))
+        {
+          size_t len;
+          Lisp_Object name, events, event_info;
+
+          /* If a directory is watched name contains the name
+             of the file that was changed.  */
+          len = strlen (ev->name);
+          name = make_unibyte_string (ev->name, len);
+          name = DECODE_FILE (name);
+
+          events = Qnil;
+          if (ev->mask & (IN_MODIFY|IN_CREATE) )
+            events = Fcons (Fcons (Qmodify, name), events);
+          if (ev->mask & IN_MOVE_SELF)
+            events = Fcons (Fcons (Qmove, name), events);
+          if (ev->mask & IN_MOVED_FROM)
+            events = Fcons (Fcons (Qmove,
+                                   Fcons (Qfrom,
+                                          Fcons (name,
+                                                 make_number (ev->cookie)))),
+                           events);
+          if (ev->mask & IN_MOVED_TO)
+            events = Fcons (Fcons (Qmove,
+                                   Fcons (Qto,
+                                          Fcons (name,
+                                                 make_number (ev->cookie)))),
+                           events);
+          if (ev->mask & IN_ATTRIB)
+            events = Fcons (Fcons (Qattrib, name), events);
+          if (ev->mask & (IN_DELETE|IN_DELETE_SELF) )
+            events = Fcons (Fcons (Qdelete, name), events);
+
+          if (!NILP (events))
+            {
+              Lisp_Object args[2], event_info;
+              args[0] = Fcdr (callback);
+              args[1] = Fcons (events, Qnil);
+              event_info = Fcons (Fappend (2, args), Qnil);
+              if (NILP (event.arg))
+                event.arg = event_info;
+              else
+                {
+                  args[0] = event.arg;
+                  args[1] = event_info;
+                  event.arg = Fappend (2, args);
+                }
+            }
+
+          if (ev->mask & IN_IGNORED)
+            {
+              /* Event was removed automatically: Drop it from data list.  */
+              add_to_log ("File-watch: \"%s\" will be ignored", name, Qnil);
+              watch_list = Fdelete (callback, watch_list);
+            }
+          if (ev->mask & IN_Q_OVERFLOW)
+            add_to_log ("File watch: Inotify Queue Overflow!", Qnil, Qnil);
+        }
+
+      i += sizeof (*ev) + ev->len;
+    }
+
+  if (!NILP (event.arg))
+    kbd_buffer_store_event (&event);
+
+  xfree (buffer);
+}
+
+static uint32_t
+symbol_to_inotifymask (Lisp_Object symb)
+{
+  if (EQ (symb, Qmodify))
+    return IN_MODIFY | IN_CREATE;
+  else if (EQ (symb, Qmove))
+    return IN_MOVE_SELF | IN_MOVE;
+  else if (EQ (symb, Qattrib))
+    return IN_ATTRIB;
+  else if (EQ (symb, Qdelete))
+    return IN_DELETE_SELF | IN_DELETE;
+  else if (EQ (symb, Qt))
+    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
+      | IN_DELETE_SELF | IN_DELETE;
+  else
+    Fsignal (Qunkown_aspect, Fcons (symb, Qnil));
+}
+
+DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, 3, 0,
+       doc: /* Arrange to call FUNC if ASPECT of FILENAME changes.
+
+ASPECT might be one of the following symbols or a list of those symbols:
+
+modify -- notify when a file is modified or created.
+
+move -- notify when a file/directory is moved.
+
+attrib -- notify when attributes change.
+
+delete -- notify when a file/directory is deleted.
+
+t -- notify for all of the above.
+
+Watching a directory is not recursive.  CALLBACK receives the events as a list
+with each list element being a list containing information about an event.  The
+first element is a flag symbol.  If a directory is watched the second element is
+the name of the file that changed.  If a file is moved from or to the directory
+the second element is either 'from or 'to and the third element is the file
+name.  A fourth element contains a numeric identifier (cookie) that can be used
+to identify matching move operations if a file is moved inside the directory.
+
+Example:
+(file-watch "foo" t #'(lambda (file event) (message "%s %s" file event)))
+
+Use `file-unwatch' to stop watching.
+
+usage: (file-watch FILENAME ASPECT CALLBACK) */)
+  (Lisp_Object file_name, Lisp_Object aspect, Lisp_Object callback)
+{
+  uint32_t mask;
+  size_t i;
+  int watchdesc;
+  Lisp_Object decoded_file_name;
+
+  CHECK_STRING (file_name);
+
+  if (inotifyfd == uninitialized)
+    {
+      inotifyfd = inotify_init1 (IN_NONBLOCK|IN_CLOEXEC);
+      if (inotifyfd == -1)
+        {
+          inotifyfd = uninitialized;
+          report_file_error ("File watching feature (inotify) is not available",
+                             Qnil);
+        }
+      watch_list = Qnil;
+      add_read_fd (inotifyfd, &inotify_callback, &watch_list);
+    }
+
+  /* Convert ASPECT into the inotify event mask.  */
+  if (CONSP (aspect))
+    {
+      Lisp_Object x = aspect;
+      mask = 0;
+      while (!NILP(x))
+        {
+          mask |= symbol_to_inotifymask (Fcar (x));
+          x = Fcdr(x);
+        }
+    }
+  else
+    mask = symbol_to_inotifymask(aspect);
+
+  decoded_file_name = ENCODE_FILE (file_name);
+  watchdesc = inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), mask);
+  if (watchdesc == -1)
+    report_file_error ("Could not watch file", Fcons (file_name, Qnil));
+  /* TODO: check if file is already in the watch_list.  */
+  watch_list = Fcons (Fcons (make_number (watchdesc),
+                             Fcons (file_name, Fcons (callback, Qnil))),
+                      watch_list);
+  return Qt;
+}
+
+DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
+       doc: /* Stop watching a file or directory.  */)
+  (Lisp_Object file_name)
+{
+  Lisp_Object cell, file_name_data;
+  Lisp_Object x = watch_list;
+
+  if (inotifyfd == uninitialized)
+    return Qnil;
+  CHECK_STRING (file_name);
+
+  while (!NILP (x))
+    {
+      cell = Fcar (x);
+      x = Fcdr (x);
+      file_name_data = Fcdr (cell);
+      if (!NILP (file_name_data) && STRINGP (Fcar (file_name_data))
+         && Fstring_equal (Fcar (file_name_data), file_name)
+         && NUMBERP (Fcar (cell)))
+        {
+          int const magicno = XINT (Fcar (cell));
+          watch_list = Fdelete (cell, watch_list);
+          if (inotify_rm_watch (inotifyfd, magicno) == -1)
+            report_file_error ("Could not unwatch file",
+                               Fcons (file_name, Qnil));
+
+          /* Cleanup if watch_list is empty.  */
+          if (NILP (watch_list))
+            {
+              close (inotifyfd);
+              delete_read_fd (inotifyfd);
+              inotifyfd = uninitialized;
+            }
+          return Qt;
+        }
+    }
+  return Qnil;
+}
+
+#else /* HAVE_INOTIFY */
+#error "Filewatch defined but no watch mechanism (inotify) available"
+#endif /* HAVE_INOTIFY */
+
+void
+syms_of_filewatch (void)
+{
+  DEFSYM (Qmodify, "modify");
+  DEFSYM (Qmove, "move");
+  DEFSYM (Qattrib, "attrib");
+  DEFSYM (Qdelete, "delete");
+  DEFSYM (Qfrom, "from");
+  DEFSYM (Qto, "to");
+
+  DEFSYM (Qunkown_aspect, "unkown-aspect");
+
+  defsubr (&Sfile_watch);
+  defsubr (&Sfile_unwatch);
+}
+
+#endif /* HAVE_FILEWATCH */
diff --git a/src/keyboard.c b/src/keyboard.c
index 7bc406a..ccf25f7 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -329,6 +329,9 @@ static Lisp_Object Qsave_session;
 #ifdef HAVE_DBUS
 static Lisp_Object Qdbus_event;
 #endif
+#ifdef HAVE_FILEWATCH
+static Lisp_Object Qfilewatch_event;
+#endif /* HAVE_FILEWATCH */
 static Lisp_Object Qconfig_changed_event;
 
 /* Lisp_Object Qmouse_movement; - also an event header */
@@ -4017,6 +4020,13 @@ kbd_buffer_get_event (KBOARD **kbp,
 	  kbd_fetch_ptr = event + 1;
 	}
 #endif
+#ifdef HAVE_FILEWATCH
+      else if (event->kind == FILEWATCH_EVENT)
+        {
+          obj = make_lispy_event (event);
+          kbd_fetch_ptr = event + 1;
+        }
+#endif
       else if (event->kind == CONFIG_CHANGED_EVENT)
 	{
 	  obj = make_lispy_event (event);
@@ -5913,6 +5923,13 @@ make_lispy_event (struct input_event *event)
       }
 #endif /* HAVE_DBUS */
 
+#ifdef HAVE_FILEWATCH
+    case FILEWATCH_EVENT:
+      {
+        return Fcons (Qfilewatch_event, event->arg);
+      }
+#endif /* HAVE_FILEWATCH */
+
     case CONFIG_CHANGED_EVENT:
 	return Fcons (Qconfig_changed_event,
                       Fcons (event->arg,
@@ -11524,6 +11541,10 @@ syms_of_keyboard (void)
   DEFSYM (Qdbus_event, "dbus-event");
 #endif
 
+#ifdef HAVE_FILEWATCH
+  DEFSYM (Qfilewatch_event, "filewatch-event");
+#endif /* HAVE_FILEWATCH */
+
   DEFSYM (QCenable, ":enable");
   DEFSYM (QCvisible, ":visible");
   DEFSYM (QChelp, ":help");
@@ -12274,6 +12295,13 @@ keys_of_keyboard (void)
 			    "dbus-handle-event");
 #endif
 
+#ifdef HAVE_FILEWATCH
+  /* Define a special event which is raised for filewatch callback
+     functions.  */
+  initial_define_lispy_key (Vspecial_event_map, "filewatch-event",
+                            "filewatch-handle-event");
+#endif /* HAVE_FILEWATCH */
+
   initial_define_lispy_key (Vspecial_event_map, "config-changed-event",
 			    "ignore");
 }
diff --git a/src/lisp.h b/src/lisp.h
index 8a504e8..8b21e0e 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3430,6 +3430,11 @@ EXFUN (Fxw_display_color_p, 1);
 EXFUN (Fx_focus_frame, 1);
 #endif
 
+/* Defined in filewatch.c */
+#ifdef HAVE_FILEWATCH
+extern void syms_of_filewatch (void);
+#endif
+
 /* Defined in xfaces.c */
 extern Lisp_Object Qdefault, Qtool_bar, Qfringe;
 extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor;
diff --git a/src/termhooks.h b/src/termhooks.h
index 6a58517..9db2019 100644
--- a/src/termhooks.h
+++ b/src/termhooks.h
@@ -206,6 +206,11 @@ enum event_kind
   , NS_NONKEY_EVENT
 #endif
 
+#ifdef HAVE_FILEWATCH
+  /* File or directory was changed.  */
+  , FILEWATCH_EVENT
+#endif
+
 };
 
 /* If a struct input_event has a kind which is SELECTION_REQUEST_EVENT
-- 
1.7.5.2

Re: [PATCH update2] Support for filesystem watching (inotify)

[Jan Djärv]

Rüdiger Sonderfeld skrev 2011-06-05 01.36:
> Hi,
>
> On Saturday 04 June 2011 12:43:37 you wrote:
>> In general, you should not call Lisp from the read fd callback. Recursive
>> calls into Lisp may fail. Instead post a Lisp event and handle it in
>> keyboard.c and some Lisp code.
>
> Thanks for your advise. I changed the code to use events. I imitated the
> behaviour of the dbus subsystem. I hope this is what you had in mind.

As Eli already said, it doesn't appear in the patch you sent.

	Jan D.

Re: [PATCH update2] Support for filesystem watching (inotify)

[Andreas Schwab]

What happens if inotify is not available and you call file-watch and
then file-unwatch?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [PATCH update2] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

On Sunday 05 June 2011 09:54:20 Andreas Schwab wrote:
> What happens if inotify is not available and you call file-watch and
> then file-unwatch?
> 
> Andreas.

Oh you are right. I should reset inotifyfd to uninitialized in case of error. 
I changed this in Version 3 of the patch. Thanks!

Re: [PATCH update2] Support for filesystem watching (inotify)

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 252 bytes --]

Is there a good reason not to provide access to the full functionality of
the inotify API?

- Why conflate IN_MODIFY and IN_CREATE?
- Why conflate IN_DELETE and IN_DELETE_SELF?
- Why omit IN_OPEN, IN_ACCESS, IN_CLOSE_WRITE and IN_CLOSE_NOWRITE?

/john

[-- Attachment #2: Type: text/html, Size: 1384 bytes --]

Re: [PATCH update2] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

Hi,

On Sunday 05 June 2011 17:59:03 John Yates wrote:
> Is there a good reason not to provide access to the full functionality of
> the inotify API?

Yes, the reason is portability. I want the same API to be available on Linux, 
*BSD, OS X and Windows. inotify is the most advanced API and therefore I can 
not support every inotify feature if I want to stay compatible.

Porting to kqueue (*BSD, OS X) will be especially troublesome because they 
don't seem to tell you which file actually changed if you watch a directory.

Regards,
Rüdiger
 

Re: [PATCH update2] Support for filesystem watching (inotify)

[Eli Zaretskii]

> From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> Date: Sun, 5 Jun 2011 18:14:33 +0200
> Cc: emacs-devel@gnu.org
> 
> On Sunday 05 June 2011 17:59:03 John Yates wrote:
> > Is there a good reason not to provide access to the full functionality of
> > the inotify API?
> 
> Yes, the reason is portability. I want the same API to be available on Linux, 
> *BSD, OS X and Windows. inotify is the most advanced API and therefore I can 
> not support every inotify feature if I want to stay compatible.

Being compatible does not mean providing only the least common
denominator.  We have already several features that provide different
levels of support depending on the underlying facilities.  One example
is process-attributes (which is the main primitive on which Proced is
based).  All you need is provide a superset of all the supported
attributes, and document which ones are supported on which platform.

Re: [PATCH update2] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

On Sunday 05 June 2011 18:58:07 Eli Zaretskii wrote:
> > From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> > Date: Sun, 5 Jun 2011 18:14:33 +0200
> > Cc: emacs-devel@gnu.org
> > 
> > On Sunday 05 June 2011 17:59:03 John Yates wrote:
> > > Is there a good reason not to provide access to the full functionality
> > > of the inotify API?
> > 
> > Yes, the reason is portability. I want the same API to be available on
> > Linux, *BSD, OS X and Windows. inotify is the most advanced API and
> > therefore I can not support every inotify feature if I want to stay
> > compatible.
> 
> Being compatible does not mean providing only the least common
> denominator.  We have already several features that provide different
> levels of support depending on the underlying facilities.  One example
> is process-attributes (which is the main primitive on which Proced is
> based).  All you need is provide a superset of all the supported
> attributes, and document which ones are supported on which platform.

I could add the IN_OPEN, IN_ACCESS, IN_CLOSE_WRITE and IN_CLOSE_NOWRITE 
feature. But having separate IN_MODIFY, IN_CREATE and IN_DELETE, 
IN_DELETE_SELF would be problematic because they can't be separated for 
kqueue. Maybe I should just export the basic inotify and kqueue API and 
implement a portable layer on top of it in elisp.

Regards,
Rüdiger

Re: [PATCH update2] Support for filesystem watching (inotify)

[Eli Zaretskii]

> From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> Date: Mon, 6 Jun 2011 20:56:10 +0200
> Cc: john@yates-sheets.org,
>  emacs-devel@gnu.org
> 
> I could add the IN_OPEN, IN_ACCESS, IN_CLOSE_WRITE and IN_CLOSE_NOWRITE 
> feature. But having separate IN_MODIFY, IN_CREATE and IN_DELETE, 
> IN_DELETE_SELF would be problematic because they can't be separated for 
> kqueue. Maybe I should just export the basic inotify and kqueue API and 
> implement a portable layer on top of it in elisp.

I trust you to make the right decision.  I just wanted you to know you
shouldn't confine yourself to a subset for portability's sake, in this
case.

Re: [PATCH] Support for filesystem watching (inotify)

[Eli Zaretskii]

> From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> Date: Sat, 4 Jun 2011 00:34:15 +0200
> 
> I wrote a patch to support watching for file system events. It currently only works with inotify (Linux) but it could be extended to support kqueue 
> (*BSD, OS X) or Win32. But I wanted to get some feedback first. Watching for filesystem events would be very useful for, e.g., dired or magit's status 
> view.

Thanks.  A few comments below.

> +#include <setjmp.h>

Why do you need this header?

> +// Assoc list of files being watched.

We don't use C++-style comments, because we don't require C9x compiler
to build Emacs.

> +static Lisp_Object data;

The name "data" is too generic.  Use something more specific and
descriptive, like inotify_watch_alist.

> +static void 
> +inotify_callback(int fd, void *_, int for_read) {

What's that _ in the argument list?

Also, the style of braces is not according to GNU coding standards.

> +  if(ioctl(fd, FIONREAD,  &to_read) == -1)
> +    report_file_error("ioctl(2) on inotify", Qnil);

Error messages that are shown to the user should not be cryptic.
Please use some text that would make sense to a user who is not an
expert on inotify.  I would think something like this would be
appropriate:

  File watching feature (inotify) is not available

> +  char *const buffer = xmalloc(to_read);
> +  ssize_t const n = read(fd, buffer, to_read);

Can't declare locals in the middle of code: this isn't C++.

> +  if(n < 0)
> +    report_file_error("read from inotify", Qnil);

The text of this message also needs work, to make it understandable by
mere mortals.

> +  size_t i = 0;

C++/C9x again.

> +  while(i < (size_t)n)

I think this cast, in addition to being ugly, is also unneeded,
because you already verified above that n is positive.  So `i' can be
ssize_t as well, and the need for the cast is gone.

> +          call[1] = Fcar(Fcdr(callback));       /* path */

GNU coding standards frown on using "path" for anything except
PATH-style lists of directories separated by colons.  Use "file name"
instead.

> +          /* TODO check how to handle UTF-8 in file names:
> +             make_string_from_bytes NCHARS vs. NBYTES */

Not make_string_from_bytes, but DECODE_FILE.

> +          size_t const len = strlen(ev->name);

C9x again.

> +          /* if a directory is watched name contains the name
> +             of the file that was changed */

Comments should begin with a capital letter and end with a period and
2 blanks.

> +          if(ev->mask & IN_IGNORED)
> +            {
> +              /* Event was removed automatically: Drop it from data list. */
> +              message("File-watch: \"%s\" will be ignored", SSDATA(call[1]));
> +              data = Fdelete(callback, data);
> +            }

Do we really need this message?  Consider outputting it only to the
*Messages* buffer.

> +          if(ev->mask & IN_Q_OVERFLOW)
> +            message1("File watch: Inotify Queue Overflow!");

Same here.

> +  free(buffer);

xfree, not free.

> +Watching a directory is not recursive. CALLBACK receives the events as a list
> +with each list element being a list containing information about an event. The
> +first element is a flag symbol. If a directory is watched the second element is
> +the name of the file that changed. If a file is moved from or to the directory
> +the second element is either :from or :to and the third element is the file
> +name.

Please leave two blanks after a period that ends a sentence.  Also, I
think it's best to show the forms of the non-primitive arguments to
CALLBACK, rather than trying to describe them: a picture is worth a
thousand words.  Finally, the lines are too long.

>     A fourth element contains a numeric identifier (cookie) that can be used
> +to identify matching move operations if a file is moved inside the directory.

This part is extremely unclear.  How about an example?

> +      add_read_fd(inotifyfd, &inotify_callback, &data);

I don't see any code that does the corresponding delete_read_fd.

> +  uint32_t mask = 0;

Why not `unsigned'?  uint32_t is not part of C90, so it's less
portable.

> +  int watchdesc = inotify_add_watch(inotifyfd, SSDATA(args[0]), mask);

Declaration in the middle of code.

More importantly, you cannot pass to inotify_add_watch a string in its
internal Emacs representation.  You need to run it through ENCODE_FILE
first.

> +  if(watchdesc == -1) {
> +    report_file_error("Watching file", Qnil);

Again, this error message text is not helpful.  It should also mention
the file's name.

> +  (Lisp_Object path)
> +{
> +  if(inotifyfd == uninitialized)
> +    return Qnil;
> +  CHECK_STRING(path);
> +
> +  Lisp_Object x = data;
> +  Lisp_Object cell, path_data;

`path' and `path_data' again go against the GNU coding standards wrt
using "path".

> +  while(!NILP(x))
> +    {
> +      cell = Fcar(x);
> +      x = Fcdr(x);
> +      path_data = Fcdr(cell);

??? Isn't `data' an alist?  If so, why not use Fassq etc.?

> +      if(!NILP(path_data) && STRINGP(Fcar(path_data))
> +         && Fstring_equal(Fcar(path_data), path)
> +         && NUMBERP(Fcar(cell)))
> +        {
> +          int const magicno = XINT(Fcar(cell));
> +          data = Fdelete(cell, data);
> +          if(inotify_rm_watch(inotifyfd, magicno) == -1)
> +            report_file_error("Unwatch path", Qnil);
> +          return Qt;
> +        }
> +    }

I think this should call delete_read_fd when `data' becomes empty.

Re: [PATCH updated] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

On Saturday 04 June 2011 13:30:42 Eli Zaretskii wrote:

> Thanks.  A few comments below.

Thank you for your comments! I updated the patch and I hope I fixed everything
except the things noted below.

> > +#include <setjmp.h>
> 
> Why do you need this header?

"lisp.h" requires it. (lisp.h:2034:3: error: expected specifier-qualifier-list
before ‘jmp_buf’)

> > +static void
> > +inotify_callback(int fd, void *_, int for_read) {
> 
> What's that _ in the argument list?

I ignore this argument.

> > +  while(i < (size_t)n)
> 
> I think this cast, in addition to being ugly, is also unneeded,
> because you already verified above that n is positive.  So `i' can be
> ssize_t as well, and the need for the cast is gone.

I think changing i to ssize_t is the wrong way around, because i should never
be negative.

> > +          if(ev->mask & IN_IGNORED)
> > +            {
> > +              /* Event was removed automatically: Drop it from data
> > list. */ +              message("File-watch: \"%s\" will be ignored",
> > SSDATA(call[1])); +              data = Fdelete(callback, data);
> > +            }
> 
> Do we really need this message?  Consider outputting it only to the
> *Messages* buffer.
> > +          if(ev->mask & IN_Q_OVERFLOW)
> > +            message1("File watch: Inotify Queue Overflow!");
> 
> Same here.

I changes it to use add_to_log. I'm not sure if there is a better way to do
it.

> >     A fourth element contains a numeric identifier (cookie) that can be
> >     used
> > 
> > +to identify matching move operations if a file is moved inside the
> > directory.
> 
> This part is extremely unclear.  How about an example?


> > +  uint32_t mask = 0;
> 
> Why not `unsigned'?  uint32_t is not part of C90, so it's less
> portable.

It's the type used in the inotify structure.

> > +  while(!NILP(x))
> > +    {
> > +      cell = Fcar(x);
> > +      x = Fcdr(x);
> > +      path_data = Fcdr(cell);
> 
> ??? Isn't `data' an alist?  If so, why not use Fassq etc.?

Because I have to query for the file name and the structure of data is
((magicno . (filename callback)) ...)



Subject: [PATCH] Added basic file system watching support.

It currently only works with inotify/Linux. It adds file-watch and file-unwatch functions.
---
 configure.in    |   14 +++
 src/Makefile.in |    2 +-
 src/emacs.c     |    4 +
 src/filewatch.c |  268 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/lisp.h      |    5 +
 5 files changed, 292 insertions(+), 1 deletions(-)
 create mode 100644 src/filewatch.c

diff --git a/configure.in b/configure.in
index 06880ea..5696fa2 100644
--- a/configure.in
+++ b/configure.in
@@ -169,6 +169,7 @@ OPTION_DEFAULT_ON([dbus],[don't compile with D-Bus support])
 OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
 OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
 OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
+OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) support])
 
 ## For the times when you want to build Emacs but don't have
 ## a suitable makeinfo, and can live without the manuals.
@@ -1981,6 +1982,19 @@ fi
 AC_SUBST(LIBGNUTLS_LIBS)
 AC_SUBST(LIBGNUTLS_CFLAGS)
 
+dnl inotify is only available on GNU/Linux.
+HAVE_INOTIFY=no
+if test "${with_inotify}" = "yes"; then
+   AC_CHECK_HEADERS(sys/inotify.h)
+   if test "$ac_cv_header_sys_inotify_h" = yes ; then
+     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, HAVE_INOTIFY=yes)
+   fi
+fi
+if test "${HAVE_INOTIFY}" = "yes"; then
+   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
+   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
+fi
+
 dnl Do not put whitespace before the #include statements below.
 dnl Older compilers (eg sunos4 cc) choke on it.
 HAVE_XAW3D=no
diff --git a/src/Makefile.in b/src/Makefile.in
index c4250b9..9135e7d 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -334,7 +334,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o $(XMENU_OBJ) window.o \
 	syntax.o $(UNEXEC_OBJ) bytecode.o \
 	process.o gnutls.o callproc.o \
 	region-cache.o sound.o atimer.o \
-	doprnt.o intervals.o textprop.o composite.o xml.o \
+	doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
 	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ)
 obj = $(base_obj) $(NS_OBJC_OBJ)
 
diff --git a/src/emacs.c b/src/emacs.c
index 3a7c5c0..fb734e5 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -1558,6 +1558,10 @@ main (int argc, char **argv)
       syms_of_gnutls ();
 #endif
 
+#ifdef HAVE_FILEWATCH
+      syms_of_filewatch ();
+#endif /* HAVE_FILEWATCH */
+
 #ifdef HAVE_DBUS
       syms_of_dbusbind ();
 #endif /* HAVE_DBUS */
diff --git a/src/filewatch.c b/src/filewatch.c
new file mode 100644
index 0000000..55da25f
--- /dev/null
+++ b/src/filewatch.c
@@ -0,0 +1,268 @@
+/* Watching file system changes.
+
+Copyright (C) 2011
+  Free Software Foundation, Inc.
+
+This file is part of GNU Emacs.
+
+GNU Emacs is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+GNU Emacs is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <config.h>
+
+#ifdef HAVE_FILEWATCH
+#include <setjmp.h>
+#include "lisp.h"
+#include "coding.h"
+#include "process.h"
+
+static Lisp_Object QCmodify, QCmove, QCattrib, QCdelete, QCfrom, QCto, QCall;
+
+#ifdef HAVE_INOTIFY
+#include <sys/inotify.h>
+#include <sys/ioctl.h>
+
+enum { uninitialized = -100 };
+static int inotifyfd = uninitialized;
+
+/* Assoc list of files being watched.  */
+static Lisp_Object watch_list;
+
+static void
+inotify_callback(int fd, void *_, int for_read)
+{
+  int to_read;
+  char *buffer;
+  ssize_t n;
+  size_t i;
+
+  if(!for_read)
+    return;
+
+  to_read = 0;
+  if(ioctl(fd, FIONREAD,  &to_read) == -1)
+    report_file_error("Error while trying to retrieve file system events",
+                      Qnil);
+  buffer = xmalloc(to_read);
+  n = read(fd, buffer, to_read);
+  if(n < 0)
+    report_file_error("Error while trying to read file system events",
+                      Qnil);
+
+  i = 0;
+  while(i < (size_t)n)
+    {
+      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
+
+      Lisp_Object callback = Fassoc(make_number(ev->wd), watch_list);
+      if(!NILP(callback))
+        {
+          size_t len;
+          Lisp_Object name, events;
+          Lisp_Object call[3];
+          call[0] = Fcar(Fcdr(Fcdr(callback))); /* callback */
+          call[1] = Fcar(Fcdr(callback));       /* file name */
+
+          /* If a directory is watched name contains the name
+             of the file that was changed.  */
+          len = strlen(ev->name);
+          name = make_unibyte_string (ev->name, len);
+          name = DECODE_FILE(name);
+
+          events = Qnil;
+          if(ev->mask & (IN_MODIFY|IN_CREATE) )
+            events = Fcons(Fcons(QCmodify, name), events);
+          if(ev->mask & IN_MOVE_SELF)
+            events = Fcons(Fcons(QCmove, name), events);
+          if(ev->mask & IN_MOVED_FROM)
+            events = Fcons(Fcons(QCmove,
+                                 Fcons(QCfrom,
+                                       Fcons(name, make_number(ev->cookie)))),
+                           events);
+          if(ev->mask & IN_MOVED_TO)
+            events = Fcons(Fcons(QCmove,
+                                 Fcons(QCto,
+                                       Fcons(name, make_number(ev->cookie)))),
+                           events);
+          if(ev->mask & IN_ATTRIB)
+            events = Fcons(Fcons(QCattrib, name), events);
+          if(ev->mask & (IN_DELETE|IN_DELETE_SELF) )
+            events = Fcons(Fcons(QCdelete, name), events);
+
+          if(!NILP(events))
+            {
+              call[2] = events;
+              Ffuncall(3, call);
+            }
+
+          if(ev->mask & IN_IGNORED)
+            {
+              /* Event was removed automatically: Drop it from data list.  */
+              add_to_log("File-watch: \"%s\" will be ignored", call[1], Qnil);
+              watch_list = Fdelete(callback, watch_list);
+            }
+          if(ev->mask & IN_Q_OVERFLOW)
+            add_to_log("File watch: Inotify Queue Overflow!", Qnil, Qnil);
+        }
+
+      i += sizeof(*ev) + ev->len;
+    }
+
+  xfree(buffer);
+}
+
+DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, MANY, 0,
+       doc: /* Watch a file or directory.
+
+file-watch watches the file or directory given in FILENAME.  If a change occurs
+CALLBACK is called with FILENAME as first argument and a list of changes as second
+argument.  FLAGS can be
+
+:modify -- notify when a file is modified or created.
+
+:move -- notify when a file/directory is moved.
+
+:attrib -- notify when attributes change.
+
+:delete -- notify when a file/directory is deleted.
+
+:all -- notify for all of the above.
+
+Watching a directory is not recursive.  CALLBACK receives the events as a list
+with each list element being a list containing information about an event.  The
+first element is a flag symbol.  If a directory is watched the second element is
+the name of the file that changed.  If a file is moved from or to the directory
+the second element is either :from or :to and the third element is the file
+name.  A fourth element contains a numeric identifier (cookie) that can be used
+to identify matching move operations if a file is moved inside the directory.
+
+Example:
+(file-watch "foo" #'(lambda (file event) (message "%s %s" file event)) :all)
+
+Use `file-unwatch' to stop watching.
+
+usage: (file-watch FILENAME CALLBACK &rest FLAGS) */)
+  (size_t nargs, Lisp_Object *args)
+{
+  uint32_t mask;
+  size_t i;
+  int watchdesc;
+  Lisp_Object decoded_file_name;
+
+  if(nargs < 3)
+    return Qnil;
+
+  CHECK_STRING(args[0]);
+
+  if(inotifyfd == uninitialized)
+    {
+      inotifyfd = inotify_init1(IN_NONBLOCK|IN_CLOEXEC);
+      if(inotifyfd == -1)
+        report_file_error("File watching feature (inotify) is not available",
+                          Qnil);
+      watch_list = Qnil;
+      add_read_fd(inotifyfd, &inotify_callback, &watch_list);
+    }
+  mask = 0;
+  for(i = 2; i < nargs; ++i)
+    {
+      if(EQ(args[i], QCmodify))
+        mask |= IN_MODIFY | IN_CREATE;
+      else if(EQ(args[i], QCmove))
+        mask |= IN_MOVE_SELF | IN_MOVE;
+      else if(EQ(args[i], QCattrib))
+        mask |= IN_ATTRIB;
+      else if(EQ(args[i], QCdelete))
+        mask |= IN_DELETE_SELF | IN_DELETE;
+      else if(EQ(args[i], QCall))
+        mask |= IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
+          | IN_DELETE_SELF | IN_DELETE;
+      else /* TODO: should this be an error? */
+        message("Unkown parameter %s (ignored)",
+                SSDATA(Fprin1_to_string(args[i], Qnil)));
+    }
+  decoded_file_name = ENCODE_FILE(args[0]);
+  watchdesc = inotify_add_watch(inotifyfd, SSDATA(decoded_file_name), mask);
+  if(watchdesc == -1) {
+    report_file_error("Could not watch file", Fcons(args[0], Qnil));
+  }
+  /* TODO: check if file is already in the watch_list.  */
+  watch_list = Fcons(Fcons(make_number(watchdesc), Flist(2, args)), watch_list);
+  return Qt;
+}
+
+DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
+       doc: /* Stop watching a file or directory.  */)
+  (Lisp_Object file_name)
+{
+  Lisp_Object cell, file_name_data;
+  Lisp_Object x = watch_list;
+
+  if(inotifyfd == uninitialized)
+    return Qnil;
+  CHECK_STRING(file_name);
+
+  while(!NILP(x))
+    {
+      cell = Fcar(x);
+      x = Fcdr(x);
+      file_name_data = Fcdr(cell);
+      if(!NILP(file_name_data) && STRINGP(Fcar(file_name_data))
+         && Fstring_equal(Fcar(file_name_data), file_name)
+         && NUMBERP(Fcar(cell)))
+        {
+          int const magicno = XINT(Fcar(cell));
+          watch_list = Fdelete(cell, watch_list);
+          if(inotify_rm_watch(inotifyfd, magicno) == -1)
+            report_file_error("Could not unwatch file", Fcons(file_name, Qnil));
+          /* Cleanup if watch_list is empty.  */
+          if(NILP(watch_list))
+            {
+              close(inotifyfd);
+              delete_read_fd(inotifyfd);
+              inotifyfd = uninitialized;
+            }
+          return Qt;
+        }
+    }
+  return Qnil;
+}
+
+#else /* HAVE_INOTIFY */
+#error "Filewatch defined but no watch mechanism (inotify) available"
+#endif /* HAVE_INOTIFY */
+
+void
+syms_of_filewatch (void)
+{
+  QCmodify = intern_c_string (":modify");
+  staticpro (&QCmodify);
+  QCmove = intern_c_string (":move");
+  staticpro (&QCmove);
+  QCattrib = intern_c_string (":attrib");
+  staticpro (&QCattrib);
+  QCdelete = intern_c_string (":delete");
+  staticpro (&QCdelete);
+  QCfrom = intern_c_string (":from");
+  staticpro (&QCfrom);
+  QCto = intern_c_string (":to");
+  staticpro (&QCto);
+  QCall = intern_c_string (":all");
+  staticpro (&QCall);
+
+  defsubr (&Sfile_watch);
+  defsubr (&Sfile_unwatch);
+}
+
+
+#endif /* HAVE_FILEWATCH */
diff --git a/src/lisp.h b/src/lisp.h
index 8a504e8..8b21e0e 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3430,6 +3430,11 @@ EXFUN (Fxw_display_color_p, 1);
 EXFUN (Fx_focus_frame, 1);
 #endif
 
+/* Defined in filewatch.c */
+#ifdef HAVE_FILEWATCH
+extern void syms_of_filewatch (void);
+#endif
+
 /* Defined in xfaces.c */
 extern Lisp_Object Qdefault, Qtool_bar, Qfringe;
 extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor;
-- 
1.7.5.2

Re: [PATCH updated] Support for filesystem watching (inotify)

[Eli Zaretskii]

> From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> Date: Sat, 4 Jun 2011 19:13:08 +0200
> Cc: emacs-devel@gnu.org
> 
> Thank you for your comments! I updated the patch and I hope I fixed everything
> except the things noted below.

Style of braces is still incorrect, at least in this place:

> +  if(watchdesc == -1) {
> +    report_file_error("Could not watch file", Fcons(args[0], Qnil));
> +  }

The doc string still uses lines that are too long.

See also Jan's important comment about how to expose these events to
the rest of Emacs.

Re: [PATCH updated] Support for filesystem watching (inotify)

[Thien-Thi Nguyen]

() Rüdiger Sonderfeld <ruediger@c-plusplus.de>
() Sat, 4 Jun 2011 19:13:08 +0200

   +DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, MANY, 0,
   +       doc: /* Watch a file or directory.
   +
   +file-watch watches the file or directory given in FILENAME.  If a change occurs
   +CALLBACK is called with FILENAME as first argument and a list of changes as second
   +argument.  FLAGS can be
   +
   +:modify -- notify when a file is modified or created.
   +
   +:move -- notify when a file/directory is moved.
   +
   +:attrib -- notify when attributes change.
   +
   +:delete -- notify when a file/directory is deleted.
   +
   +:all -- notify for all of the above.

I think FLAGS should be symbols, not keywords.  Furthermore, "flags"
is too generic; maybe something like "aspect" or even "what" would be
(slightly) better.  Instead of ‘:all’ (or ‘all’), consider using ‘t’.

Also, it's cool if the first line names all the args.  How about:
"Arrange to call FUNC if ASPECT of FILENAME changes."
?

   +Watching a directory is not recursive.  CALLBACK receives the events as a list
   +with each list element being a list containing information about an event.  The
   +first element is a flag symbol.  If a directory is watched the second element is
   +the name of the file that changed.  If a file is moved from or to the directory
   +the second element is either :from or :to and the third element is the file
   +name.  A fourth element contains a numeric identifier (cookie) that can be used
   +to identify matching move operations if a file is moved inside the directory.
   +
   +Example:
   +(file-watch "foo" #'(lambda (file event) (message "%s %s" file event)) :all)
   +
   +Use `file-unwatch' to stop watching.
   +
   +usage: (file-watch FILENAME CALLBACK &rest FLAGS) */)
   +  (size_t nargs, Lisp_Object *args)

For upward-compatability, it is better to not use rest-args for these
aspects.  You should specify one arg ASPECT, document that it can be
"either a symbol, one of: ..., or a list of of these symbols".

Style preference: I'd prefer CALLBACK to be last so that long lambda
expressions need not be (visually) scanned to determine the aspects watched.
Consider:

(file-watch "foo" t (lambda (name event)
                      ;; LONG
                      ;; COMPLICATED
                      ;; DRAWN-OUT
                      ;; COMPUTATION
                      ))

vs

(file-watch "foo" (lambda (name event)
                    ;; LONG
                    ;; COMPLICATED
                    ;; DRAWN-OUT
                    ;; COMPUTATION
                    )
            t)

No big deal, just sympathy for the lone t.

   +  CHECK_STRING(args[0]);

Please insert a space between a function (or macro) and its arg list.
Generally, (info "(standards) Formatting").

   +      else /* TODO: should this be an error? */

Yes.

   +  /* TODO: check if file is already in the watch_list.  */

Moreover, you need to decide what to do given, for example:
 (progn (file-watch "foo" 'modify 'notify-modify)
        (file-watch "foo" 'move 'notify-move))
Is this OK, is this an error, is this "close to" an error?
I think a simple "file is already in" check is insufficient.

   +  watch_list = Fcons(Fcons(make_number(watchdesc), Flist(2, args)), watch_list);

You can use ‘acons’: (acons K V ALIST) ≡ (cons (cons K V) ALIST).

Re: [PATCH updated] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

Hi,
thank your for your comments. I changed the arguments and changed an unkown 
aspect into an error. See my reply to Jan Djärv for the version of the patch 
containing the changes.

On Saturday 04 June 2011 22:10:45 Thien-Thi Nguyen wrote:

>    +  /* TODO: check if file is already in the watch_list.  */
> 
> Moreover, you need to decide what to do given, for example:
>  (progn (file-watch "foo" 'modify 'notify-modify)
>         (file-watch "foo" 'move 'notify-move))
> Is this OK, is this an error, is this "close to" an error?
> I think a simple "file is already in" check is insufficient.

Yes, this is a bit complicated. The inotify API itself does not create a 
separate event number if you try to watch the same file. But this could of 
course be done in the implementation by checking which callback corresponds to 
which event. But I'm not sure if this is the right way to handle it.
 
>    +  watch_list = Fcons(Fcons(make_number(watchdesc), Flist(2, args)),
> watch_list);
> 
> You can use ‘acons’: (acons K V ALIST) ≡ (cons (cons K V) ALIST).

Sadly acons is not available from the C API.

Re: [PATCH updated] Support for filesystem watching (inotify)

[Thien-Thi Nguyen]

() Rüdiger Sonderfeld <ruediger@c-plusplus.de>
() Sun, 5 Jun 2011 01:43:16 +0200

   Yes, this is a bit complicated. The inotify API itself does not create a 
   separate event number if you try to watch the same file. But this could of 
   course be done in the implementation by checking which callback corresponds to 
   which event. But I'm not sure if this is the right way to handle it.

   [...]

   Sadly acons is not available from the C API.

Perhaps the book-keeping parts should be moved to Lisp.

Re: [PATCH updated] Support for filesystem watching (inotify)

[Štěpán Němec]

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> () Sun, 5 Jun 2011 01:43:16 +0200

[...]

>    Sadly acons is not available from the C API.
>
> Perhaps the book-keeping parts should be moved to Lisp.

It's not really available from Lisp either. It's a cl.el function, i.e.
still a forbidden territory for GNU Emacs core code.

That alone doesn't necessarily affect validity of your Lisp-moving
suggestion, of course.

  Štěpán

Re: [PATCH updated] Support for filesystem watching (inotify)

[Johan Bockgård]

Štěpán Němec <stepnem@gmail.com> writes:

> Thien-Thi Nguyen <ttn@gnuvola.org> writes:
>
>> () Rüdiger Sonderfeld <ruediger@c-plusplus.de>
>> () Sun, 5 Jun 2011 01:43:16 +0200
>
> [...]
>
>>    Sadly acons is not available from the C API.
>>
>> Perhaps the book-keeping parts should be moved to Lisp.
>
> It's not really available from Lisp either. It's a cl.el function, i.e.
> still a forbidden territory for GNU Emacs core code.

FWIW, cl arranges for acons to be inlined at compile time, so it's
enough to

    (eval-when-compile (require 'cl))

Inlined cl functions -- how to learn about them

[Štěpán Němec]

Johan Bockgård <bojohan@gnu.org> writes:

> FWIW, cl arranges for acons to be inlined at compile time, so it's
> enough to
>
>     (eval-when-compile (require 'cl))

Interesting, thank you!

Is there a list of such exceptions somewhere? Inspecting the symbol's
plist or trying to byte-compile such functions and see if the warning
pops up or not hardly seems practical even for true GNU-fearing folk.

  Štěpán

Re: [PATCH updated] Support for filesystem watching (inotify)

[Stefan Monnier]

Thanks for your review, I generally agree with your comments.

> Moreover, you need to decide what to do given, for example:
>  (progn (file-watch "foo" 'modify 'notify-modify)
>         (file-watch "foo" 'move 'notify-move))
> Is this OK, is this an error, is this "close to" an error?

I think dired shouldn't need to know if some other package decided to
watch the same directory, so having several watchers for the same file
should be accepted and work correctly, i.e. both callbacks should be run
when needed.

> I think a simple "file is already in" check is insufficient.
>    +  watch_list = Fcons(Fcons(make_number(watchdesc), Flist(2, args)), watch_list);
> You can use ‘acons’: (acons K V ALIST) ≡ (cons (cons K V) ALIST).

No, Fcons is the right thing to use there.
We could #define ACONS(a,b,c) Fcons(Fcons(a,b),c)
but that's orthogonal to this patch.


        Stefan

Re: [PATCH updated] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

On Monday 06 June 2011 17:21:35 Stefan Monnier wrote:
> Thanks for your review, I generally agree with your comments.
> 
> > Moreover, you need to decide what to do given, for example:
> >  (progn (file-watch "foo" 'modify 'notify-modify)
> >  
> >         (file-watch "foo" 'move 'notify-move))
> > 
> > Is this OK, is this an error, is this "close to" an error?
> 
> I think dired shouldn't need to know if some other package decided to
> watch the same directory, so having several watchers for the same file
> should be accepted and work correctly, i.e. both callbacks should be run
> when needed.

The problem is: How to implement the file-unwatch then? I need a way to 
identify each separate file-watch request. What would be the best way to do 
that?

Regards,
Rüdiger

Re: [PATCH updated] Support for filesystem watching (inotify)

[Stefan Monnier]

> The problem is: How to implement the file-unwatch then? I need a way
> to identify each separate file-watch request.  What would be the best
> way to do that?

How about letting file-watch return a "file-watcher" which you then need
to pass to file-unwatch?  This "file-watcher" could be any kind of Elisp
data you find convenient for this.  You may decide to provide no other
operation than file-unwatch, but you could also decide to provided
additional operations such as changing the callback (I'm not saying
that would necessarily be a good idea, tho, but maybe other operations
would be handy).


        Stefan

Re: [PATCH updated] Support for filesystem watching (inotify)

[Ted Zlatanov]

On Mon, 06 Jun 2011 14:11:12 -0300 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> The problem is: How to implement the file-unwatch then? I need a way
>> to identify each separate file-watch request.  What would be the best
>> way to do that?

SM> How about letting file-watch return a "file-watcher" which you then need
SM> to pass to file-unwatch?  This "file-watcher" could be any kind of Elisp
SM> data you find convenient for this.  You may decide to provide no other
SM> operation than file-unwatch, but you could also decide to provided
SM> additional operations such as changing the callback (I'm not saying
SM> that would necessarily be a good idea, tho, but maybe other operations
SM> would be handy).

(background: url-future.el is really a generic futures library, but the
only expected use for it currently is in url-*.el)

The "file-watcher" could derive from url-future, with the nice
error-catching and accessor functions that come with it, plus it's
protected from double-invocation.

Ted

Re: [PATCH updated] Support for filesystem watching (inotify)

[Stefan Monnier]

>>> The problem is: How to implement the file-unwatch then? I need a way
>>> to identify each separate file-watch request.  What would be the best
>>> way to do that?

SM> How about letting file-watch return a "file-watcher" which you then need
SM> to pass to file-unwatch?  This "file-watcher" could be any kind of Elisp
SM> data you find convenient for this.  You may decide to provide no other
SM> operation than file-unwatch, but you could also decide to provided
SM> additional operations such as changing the callback (I'm not saying
SM> that would necessarily be a good idea, tho, but maybe other operations
SM> would be handy).

> (background: url-future.el is really a generic futures library, but the
> only expected use for it currently is in url-*.el)

> The "file-watcher" could derive from url-future, with the nice
> error-catching and accessor functions that come with it, plus it's
> protected from double-invocation.

That doesn't sound right to me.  At least I'm having trouble figuring
out how to bend my mind such that this file-watcher matches (even
coarsely) the concept of a "future".


        Stefan

Re: [PATCH updated] Support for filesystem watching (inotify)

[Ted Zlatanov]

On Tue, 07 Jun 2011 11:42:40 -0300 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>>>> The problem is: How to implement the file-unwatch then? I need a way
>>>> to identify each separate file-watch request.  What would be the best
>>>> way to do that?

SM> How about letting file-watch return a "file-watcher" which you then need
SM> to pass to file-unwatch?  This "file-watcher" could be any kind of Elisp
SM> data you find convenient for this.  You may decide to provide no other
SM> operation than file-unwatch, but you could also decide to provided
SM> additional operations such as changing the callback (I'm not saying
SM> that would necessarily be a good idea, tho, but maybe other operations
SM> would be handy).

>> (background: url-future.el is really a generic futures library, but the
>> only expected use for it currently is in url-*.el)

>> The "file-watcher" could derive from url-future, with the nice
>> error-catching and accessor functions that come with it, plus it's
>> protected from double-invocation.

SM> That doesn't sound right to me.  At least I'm having trouble figuring
SM> out how to bend my mind such that this file-watcher matches (even
SM> coarsely) the concept of a "future".

In this case it's just a place to stash the callback, with protection
against double invocation and error catching.  Each instance holds a
separate `file-watch' request.

`file-unwatch' simply invokes the lambda to get the value to deregister.

So `file-watch' would return

(make-url-future :value (lambda () (unwatch file here))
                 :errorback  (lambda (&rest d) (handle unwatch errors))
                 :callback (lambda (f) (arbitrary callback)))

Which, in turn, can be used (say it's saved in `f'):

(url-future-call f) ; funcall :value
(url-future-value good) ; whatever (unwatch file here) returned
(url-future-done-p f) ; true
(url-future-completed-p f) ; true if no errors, plus :callback was called
(url-future-errored-p f) ; true if errors, plus :errorback was called

(url-future-call f) ; throws an error

I think that's more convenient than a new special "file-watcher" type,
especially since the "file-watcher" can be derived from "url-future" and
inherit all the functions above under its own namespace, plus of course
any fields it cares to define on top.  The built-in support for
callbacks and error callbacks is pretty nice too.

Ted

Re: [PATCH updated] Support for filesystem watching (inotify)

[Stefan Monnier]

SM> That doesn't sound right to me.  At least I'm having trouble figuring
SM> out how to bend my mind such that this file-watcher matches (even
SM> coarsely) the concept of a "future".
[...]
> So `file-watch' would return
> (make-url-future :value (lambda () (unwatch file here))
>                  :errorback  (lambda (&rest d) (handle unwatch errors))
>                  :callback (lambda (f) (arbitrary callback)))

I'm not saying you can't twist the code so that it works, but it makes
no sense at the conceptual level.  The file-watcher callback may be
called several times; unwatching a file has nothing to do with "get the
resulting value"; ...
It's just a completely wrong analogy.


        Stefan

Re: [PATCH updated] Support for filesystem watching (inotify)

[Ted Zlatanov]

On Tue, 07 Jun 2011 15:06:51 -0300 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> That doesn't sound right to me.  At least I'm having trouble figuring
SM> out how to bend my mind such that this file-watcher matches (even
SM> coarsely) the concept of a "future".
SM> [...]
>> So `file-watch' would return
>> (make-url-future :value (lambda () (unwatch file here))
>> :errorback  (lambda (&rest d) (handle unwatch errors))
>> :callback (lambda (f) (arbitrary callback)))

SM> I'm not saying you can't twist the code so that it works, but it
SM> makes no sense at the conceptual level [...] It's just a completely
SM> wrong analogy.

So you're cautiously in favor, then? ;)

Ted

Re: [PATCH updated] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

On Monday 06 June 2011 19:11:12 Stefan Monnier wrote:
> > The problem is: How to implement the file-unwatch then? I need a way
> > to identify each separate file-watch request.  What would be the best
> > way to do that?
> 
> How about letting file-watch return a "file-watcher" which you then need
> to pass to file-unwatch?  This "file-watcher" could be any kind of Elisp
> data you find convenient for this.  You may decide to provide no other
> operation than file-unwatch, but you could also decide to provided
> additional operations such as changing the callback (I'm not saying
> that would necessarily be a good idea, tho, but maybe other operations
> would be handy).

I updated the patch to return a "file-watcher". file-watch can now be called
several times for the same file with different flags and callbacks. This
however required large changes to the patch. So it would be great if someone
could take a look at it again. I added an automated test. But it doesn't test the
actually file-watching yet. This is a bit tricky to get right in a unit test.
But I'll add it. The code requires some heavy testing.

Regards,
Rüdiger

From: =?UTF-8?q?R=C3=BCdiger=20Sonderfeld?= <ruediger@c-plusplus.de>
Date: Fri, 3 Jun 2011 23:58:12 +0200
Subject: [PATCH] Added basic file system watching support.

It currently only works with inotify/Linux. It adds file-watch and file-unwatch functions.
---
 configure.in                      |   14 +
 lisp/filewatch.el                 |   54 ++++
 src/Makefile.in                   |    2 +-
 src/emacs.c                       |    4 +
 src/filewatch.c                   |  485 +++++++++++++++++++++++++++++++++++++
 src/keyboard.c                    |   28 +++
 src/lisp.h                        |    5 +
 src/termhooks.h                   |    5 +
 test/automated/filewatch-tests.el |   50 ++++
 9 files changed, 646 insertions(+), 1 deletions(-)
 create mode 100644 lisp/filewatch.el
 create mode 100644 src/filewatch.c
 create mode 100644 test/automated/filewatch-tests.el

diff --git a/configure.in b/configure.in
index 0135b9f..4a816ec 100644
--- a/configure.in
+++ b/configure.in
@@ -174,6 +174,7 @@ OPTION_DEFAULT_ON([dbus],[don't compile with D-Bus support])
 OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
 OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
 OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
+OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) support])
 
 ## For the times when you want to build Emacs but don't have
 ## a suitable makeinfo, and can live without the manuals.
@@ -1999,6 +2000,19 @@ fi
 AC_SUBST(LIBGNUTLS_LIBS)
 AC_SUBST(LIBGNUTLS_CFLAGS)
 
+dnl inotify is only available on GNU/Linux.
+HAVE_INOTIFY=no
+if test "${with_inotify}" = "yes"; then
+   AC_CHECK_HEADERS(sys/inotify.h)
+   if test "$ac_cv_header_sys_inotify_h" = yes ; then
+     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, HAVE_INOTIFY=yes)
+   fi
+fi
+if test "${HAVE_INOTIFY}" = "yes"; then
+   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
+   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
+fi
+
 dnl Do not put whitespace before the #include statements below.
 dnl Older compilers (eg sunos4 cc) choke on it.
 HAVE_XAW3D=no
diff --git a/lisp/filewatch.el b/lisp/filewatch.el
new file mode 100644
index 0000000..2c97589
--- /dev/null
+++ b/lisp/filewatch.el
@@ -0,0 +1,54 @@
+;;; filewatch.el --- Elisp support for watching filesystem events.
+
+;; Copyright (C) 2011 Free Software Foundation, Inc.
+
+;; Author: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
+;; Keywords: files
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; This file contains the elisp part of the filewatch API.
+
+;;; Code:
+
+(eval-when-compile
+  (require 'cl))
+
+(defun filewatch-event-p (event)
+  "Check if EVENT is a filewatch event."
+  (and (listp event)
+       (eq (car event) 'filewatch-event)))
+
+;;;###autoload
+(defun filewatch-handle-event (event)
+  "Handle file system monitoring event.
+If EVENT is a filewatch event then the callback is called.  If EVENT is
+not a filewatch event then a `filewatch-error' is signaled."
+  (interactive "e")
+
+  (unless (filewatch-event-p event)
+    (signal 'filewatch-error (cons "Not a valid filewatch event" event)))
+
+  (loop for ev in (cdr event)
+     unless (and (listp ev) (>= (length ev) 3))
+     do (signal 'filewatch-error (cons "Not a valid filewatch event" event))
+     do (funcall (cadr ev) (car ev) (caddr ev))))
+
+(provide 'filewatch)
+
+;;; filewatch.el ends here
diff --git a/src/Makefile.in b/src/Makefile.in
index c4250b9..9135e7d 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -334,7 +334,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o $(XMENU_OBJ) window.o \
 	syntax.o $(UNEXEC_OBJ) bytecode.o \
 	process.o gnutls.o callproc.o \
 	region-cache.o sound.o atimer.o \
-	doprnt.o intervals.o textprop.o composite.o xml.o \
+	doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
 	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ)
 obj = $(base_obj) $(NS_OBJC_OBJ)
 
diff --git a/src/emacs.c b/src/emacs.c
index d14acd6..db897d5 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -1563,6 +1563,10 @@ Using an Emacs configured with --with-x-toolkit=lucid does not have this problem
       syms_of_gnutls ();
 #endif
 
+#ifdef HAVE_FILEWATCH
+      syms_of_filewatch ();
+#endif /* HAVE_FILEWATCH */
+
 #ifdef HAVE_DBUS
       syms_of_dbusbind ();
 #endif /* HAVE_DBUS */
diff --git a/src/filewatch.c b/src/filewatch.c
new file mode 100644
index 0000000..2833544
--- /dev/null
+++ b/src/filewatch.c
@@ -0,0 +1,485 @@
+/* Watching file system changes.
+
+Copyright (C) 2011
+  Free Software Foundation, Inc.
+
+This file is part of GNU Emacs.
+
+GNU Emacs is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+GNU Emacs is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <config.h>
+
+#ifdef HAVE_FILEWATCH
+#include <setjmp.h> /* Required for lisp.h.  */
+#include "lisp.h"
+#include "coding.h"
+#include "process.h"
+#include "keyboard.h"
+#include "character.h"
+#include "frame.h" /* Required for termhooks.h.  */
+#include "termhooks.h"
+
+static Lisp_Object Qmodify, Qmove, Qattrib, Qdelete, Qfrom, Qto, Qall_modify;
+static Lisp_Object Qunknown_aspect, Qfile_watch;
+static Lisp_Object Qaccess, Qclose_write, Qclose_nowrite, Qopen, Qall;
+
+#ifdef HAVE_INOTIFY
+#include <sys/inotify.h>
+#include <sys/ioctl.h>
+
+enum { uninitialized = -100 };
+/* File handle for inotify.  */
+static int inotifyfd = uninitialized;
+
+/* Assoc list of files being watched.
+ Format:
+ (id . (filename ((subid callback flags) (subid callbacks flags) ...))) */
+static Lisp_Object watch_list;
+
+#define CADDR(x) Fcar (Fcdr (Fcdr (x)))
+#define CADR(x) Fcar (Fcdr (x))
+
+static Lisp_Object
+append2 (Lisp_Object s1, Lisp_Object s2)
+{
+  Lisp_Object args[2];
+  args[0] = s1;
+  args[1] = s2;
+  return Fappend (2, args);
+}
+
+/* Returns a list with all the elements from EVENTS the `watch_list' CELL
+   is interested in.  */
+static Lisp_Object
+match_aspects (Lisp_Object cell, Lisp_Object events)
+{
+  Lisp_Object ret = Qnil;
+  Lisp_Object aspects = CADDR (cell);
+  Lisp_Object i;
+  if (EQ (aspects, Qt) || EQ (aspects, Qall))
+    return events;
+  else if (EQ (aspects, Qall_modify))
+    aspects = list4 (Qmodify, Qmove, Qattrib, Qdelete);
+
+  i = Fcar (aspects);
+  while (!NILP (i))
+    {
+      Lisp_Object e = Fcar (events);
+      while (!NILP (e))
+        {
+          if (EQ (Fcar (e), i))
+            ret = Fcons (e, ret);
+          events = Fcdr (events);
+          e = Fcar (events);
+        }
+      aspects = Fcdr (aspects);
+      i = Fcar (aspects);
+    }
+  return ret;
+}
+
+static Lisp_Object
+inotifyevent_to_aspects (Lisp_Object name, struct inotify_event *ev)
+{
+  Lisp_Object events = Qnil;
+  if (ev->mask & (IN_MODIFY|IN_CREATE) )
+    events = Fcons (Fcons (Qmodify, name), events);
+  if (ev->mask & IN_MOVE_SELF)
+    events = Fcons (Fcons (Qmove, name), events);
+  if (ev->mask & IN_MOVED_FROM)
+    events = Fcons (Fcons (Qmove,
+                           Fcons (Qfrom,
+                                  Fcons (name,
+                                         make_number (ev->cookie)))),
+                    events);
+  if (ev->mask & IN_MOVED_TO)
+    events = Fcons (Fcons (Qmove,
+                           Fcons (Qto,
+                                  Fcons (name,
+                                         make_number (ev->cookie)))),
+                    events);
+  if (ev->mask & IN_ATTRIB)
+    events = Fcons (Fcons (Qattrib, name), events);
+  if (ev->mask & (IN_DELETE|IN_DELETE_SELF) )
+    events = Fcons (Fcons (Qdelete, name), events);
+  if (ev->mask & IN_ACCESS)
+    events = Fcons (Fcons (Qaccess, name), events);
+  if (ev->mask & IN_CLOSE_WRITE)
+    events = Fcons (Fcons (Qclose_write, name), events);
+  if (ev->mask & IN_CLOSE_NOWRITE)
+    events = Fcons (Fcons (Qclose_nowrite, name), events);
+  if (ev->mask & IN_OPEN)
+    events = Fcons (Fcons (Qopen, name), events);
+  return events;
+}
+
+/* This callback is called when the FD is available FOR_READ.  The inotify
+   events are read from FD and converted into input_events.  */
+static void
+inotify_callback (int fd, void *_, int for_read)
+{
+  struct input_event event;
+  int to_read;
+  char *buffer;
+  ssize_t n;
+  size_t i;
+
+  if (!for_read)
+    return;
+
+  to_read = 0;
+  if (ioctl (fd, FIONREAD,  &to_read) == -1)
+    report_file_error ("Error while trying to retrieve file system events",
+                       Qnil);
+  buffer = xmalloc (to_read);
+  n = read (fd, buffer, to_read);
+  if (n < 0)
+    {
+      xfree (buffer);
+      report_file_error ("Error while trying to read file system events",
+                         Qnil);
+    }
+
+  EVENT_INIT (event);
+  event.kind = FILEWATCH_EVENT;
+  event.arg = Qnil;
+
+  i = 0;
+  while (i < (size_t)n)
+    {
+      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
+
+      Lisp_Object watch_data = Fassoc (make_number (ev->wd), watch_list);
+      if (!NILP (watch_data))
+        {
+          Lisp_Object name, events;
+
+          /* If a directory is watched name contains the name
+             of the file that was changed.  */
+          if (ev->len > 0)
+            {
+              size_t const len = strlen (ev->name);
+              name = make_unibyte_string (ev->name, min (len, ev->len));
+              name = DECODE_FILE (name);
+            }
+          else
+            {
+              name = CADR (watch_data);
+            }
+
+          events = inotifyevent_to_aspects (name, ev);
+
+          if (!NILP (events))
+            {
+              Lisp_Object x = CADDR (watch_data);
+              while (!NILP (x))
+                {
+                  Lisp_Object cell = Fcar (x);
+                  Lisp_Object aspects = match_aspects (cell, events);
+                  if (!NILP (aspects))
+                    {
+                      Lisp_Object id = list3 (Qfile_watch, make_number (ev->wd),
+                                              Fcar (cell));
+                      Lisp_Object event_info = list1 (list3 (id, CADR (cell),
+                                                             aspects));
+
+                      if (NILP (event.arg))
+                        event.arg = event_info;
+                      else
+                        event.arg = append2 (event.arg, event_info);
+                    }
+                  x = Fcdr (x);
+                }
+            }
+
+          if (ev->mask & IN_IGNORED)
+            {
+              /* Event was removed automatically: Drop it from data list.  */
+              add_to_log ("File-watch: \"%s\" will be ignored", name, Qnil);
+              watch_list = Fdelete (watch_data, watch_list);
+            }
+          if (ev->mask & IN_Q_OVERFLOW)
+            add_to_log ("File watch: Inotify Queue Overflow!", Qnil, Qnil);
+        }
+
+      i += sizeof (*ev) + ev->len;
+    }
+
+  if (!NILP (event.arg))
+    kbd_buffer_store_event (&event);
+
+  xfree (buffer);
+}
+
+static uint32_t
+symbol_to_inotifymask (Lisp_Object symb, int in_list)
+{
+  if (EQ (symb, Qmodify))
+    return IN_MODIFY | IN_CREATE;
+  else if (EQ (symb, Qmove))
+    return IN_MOVE_SELF | IN_MOVE;
+  else if (EQ (symb, Qattrib))
+    return IN_ATTRIB;
+  else if (EQ (symb, Qdelete))
+    return IN_DELETE_SELF | IN_DELETE;
+  else if (!in_list && EQ (symb, Qall_modify))
+    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
+      | IN_DELETE_SELF | IN_DELETE;
+  else if (EQ (symb, Qaccess))
+    return IN_ACCESS;
+  else if (EQ (symb, Qclose_write))
+    return IN_CLOSE_WRITE;
+  else if (EQ (symb, Qclose_nowrite))
+    return IN_CLOSE_NOWRITE;
+  else if (EQ (symb, Qopen))
+    return IN_OPEN;
+  else if (!in_list && (EQ (symb, Qt) || EQ (symb, Qall)))
+    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
+      | IN_DELETE_SELF | IN_DELETE | IN_ACCESS | IN_CLOSE_WRITE
+      | IN_CLOSE_NOWRITE | IN_OPEN;
+  else
+    Fsignal (Qunknown_aspect, Fcons (symb, Qnil));
+}
+
+static uint32_t
+aspect_to_inotifymask (Lisp_Object aspect)
+{
+  if (CONSP (aspect))
+    {
+      Lisp_Object x = aspect;
+      uint32_t mask = 0;
+      while (!NILP (x))
+        {
+          mask |= symbol_to_inotifymask (Fcar (x), 1);
+          x = Fcdr (x);
+        }
+      return mask;
+    }
+  else
+    return symbol_to_inotifymask (aspect, 0);
+}
+
+static Lisp_Object
+get_watch_data_by_file_name (Lisp_Object file_name)
+{
+  Lisp_Object cell, file_name_data;
+  Lisp_Object x = watch_list;
+  CHECK_STRING (file_name);
+
+  while (!NILP (x))
+    {
+      cell = Fcar (x);
+      x = Fcdr (x);
+      file_name_data = Fcdr (cell);
+      if (!NILP (file_name_data) && STRINGP (Fcar (file_name_data))
+          && !NILP (Fstring_equal (Fcar (file_name_data), file_name))
+          && NUMBERP (Fcar (cell)))
+        return cell;
+    }
+  return Qnil;
+}
+
+DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, 3, 0,
+       doc: /* Arrange to call FUNC if ASPECT of FILENAME changes.
+
+ASPECT might be one of the following symbols or a list of those symbols (except
+for all-modify and t):
+
+modify -- notify when a file is modified or created.
+
+move -- notify when a file/directory is moved.
+
+attrib -- notify when attributes change.
+
+delete -- notify when a file/directory is deleted.
+
+all-modify -- notify for all of the above (all modifying changes).
+
+access -- notify when file was accessed/read.
+
+close-write -- notify when a file opened for writing was closed.
+
+close-nowrite -- notify when a file not opened for writing was closed.
+
+open -- notify when a file was opened.
+
+t -- notify for all of the above.
+
+Watching a directory is not recursive.  CALLBACK receives the events as a list
+with each list element being a list containing information about an event.  The
+first element is a flag symbol.  If a directory is watched the second element is
+the name of the file that changed.  If a file is moved from or to the directory
+the second element is either 'from or 'to and the third element is the file
+name.  A fourth element contains a numeric identifier (cookie) that can be used
+to identify matching move operations if a file is moved inside the directory.
+
+Example:
+(file-watch "foo" t #'(lambda (id event) (message "%s %s" id event)))
+
+Use `file-unwatch' to stop watching.
+
+usage: (file-watch FILENAME ASPECT CALLBACK) */)
+  (Lisp_Object file_name, Lisp_Object aspect, Lisp_Object callback)
+{
+  static int intern_counter = 0; /* assign local ids */
+  uint32_t mask;
+  int watchdesc;
+  Lisp_Object decoded_file_name;
+  Lisp_Object data;
+  Lisp_Object info;
+
+  CHECK_STRING (file_name);
+
+  if (inotifyfd == uninitialized)
+    {
+      inotifyfd = inotify_init1 (IN_NONBLOCK|IN_CLOEXEC);
+      if (inotifyfd == -1)
+        {
+          inotifyfd = uninitialized;
+          report_file_error ("File watching feature (inotify) is not available",
+                             Qnil);
+        }
+      watch_list = Qnil;
+      add_read_fd (inotifyfd, &inotify_callback, &watch_list);
+    }
+
+  mask = aspect_to_inotifymask (aspect);
+  data = get_watch_data_by_file_name (file_name);
+  if (!NILP (data))
+    mask |= IN_MASK_ADD;
+
+  decoded_file_name = ENCODE_FILE (file_name);
+  watchdesc = inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), mask);
+  if (watchdesc == -1)
+    report_file_error ("Could not watch file", Fcons (file_name, Qnil));
+
+  info = list3 (make_number (intern_counter), callback, aspect);
+  ++intern_counter;
+
+  if (!NILP (data))
+    Fsetcdr (Fcdr (data), Fcons (append2 (CADDR (data), Fcons (info, Qnil)),
+                                 Qnil));
+  else
+    watch_list = Fcons (list3 (make_number (watchdesc),
+                               file_name, Fcons (info, Qnil)),
+                        watch_list);
+
+  return list3 (Qfile_watch, make_number (watchdesc), Fcar (info));
+}
+
+static int
+file_watch_objectp (Lisp_Object obj)
+{
+  return CONSP (obj) && XINT (Flength (obj)) == 3
+    && EQ (Fcar (obj), Qfile_watch);
+}
+
+DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
+       doc: /* Stop watching a file or directory.  */)
+  (Lisp_Object watch_object)
+{
+  Lisp_Object watch_data;
+  Lisp_Object info;
+
+  if (!file_watch_objectp (watch_object))
+    wrong_type_argument (Qfile_watch, watch_object);
+
+  if (inotifyfd == uninitialized)
+    return Qnil;
+
+  watch_data = Fassoc ( CADR (watch_object), watch_list );
+  if (NILP (watch_data))
+    return Qnil;
+
+  info = Fassoc (CADDR (watch_object), CADDR (watch_data));
+  if (NILP (info))
+    return Qnil;
+
+  Fsetcdr (Fcdr (watch_data), Fcons (Fdelete (info, CADDR (watch_data)), Qnil));
+
+  if (NILP (CADDR (watch_data)))
+    {
+      int const magicno = XINT (CADR (watch_object));
+      watch_list = Fdelete (watch_data, watch_list);
+
+      if (inotify_rm_watch (inotifyfd, magicno) == -1)
+        report_file_error ("Could not unwatch file",
+                           Fcons (Fcar (watch_data), Qnil));
+
+      /* Cleanup if watch_list is empty.  */
+      if (NILP (watch_list))
+        {
+          close (inotifyfd);
+          delete_read_fd (inotifyfd);
+          inotifyfd = uninitialized;
+        }
+    }
+  else
+    {
+      Lisp_Object decoded_file_name = ENCODE_FILE (CADR (watch_data));
+      Lisp_Object x = CADDR (watch_data);
+      uint32_t mask = 0;
+      while (!NILP (x))
+        {
+          Lisp_Object cell = Fcar (x);
+          mask |= aspect_to_inotifymask (CADDR (cell));
+          x = Fcdr (x);
+        }
+      /* Reset watch mask */
+      if (inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), mask) == -1)
+        report_file_error ("Could not unwatch file",
+                           Fcons (Fcar (watch_data), Qnil));
+    }
+
+  return Qt;
+}
+
+#else /* HAVE_INOTIFY */
+#error "Filewatch defined but no watch mechanism (inotify) available"
+#endif /* HAVE_INOTIFY */
+
+void
+syms_of_filewatch (void)
+{
+  DEFSYM (Qmodify, "modify");
+  DEFSYM (Qmove, "move");
+  DEFSYM (Qattrib, "attrib");
+  DEFSYM (Qdelete, "delete");
+  DEFSYM (Qfrom, "from");
+  DEFSYM (Qto, "to");
+  DEFSYM (Qall_modify, "all-modify");
+
+  DEFSYM (Qaccess, "access");
+  DEFSYM (Qclose_write, "close-write");
+  DEFSYM (Qclose_nowrite, "close-nowrite");
+  DEFSYM (Qopen, "open");
+  DEFSYM (Qall, "all");
+
+  DEFSYM (Qunknown_aspect, "unknown-aspect");
+  DEFSYM (Qfile_watch, "file-watch");
+
+  Fput (Qunknown_aspect, Qerror_conditions,
+        list2 (Qunknown_aspect, Qerror));
+  Fput (Qunknown_aspect, Qerror_message,
+        make_pure_c_string ("Unknown Aspect"));
+
+  defsubr (&Sfile_watch);
+  defsubr (&Sfile_unwatch);
+
+  staticpro (&watch_list);
+
+  Fprovide (intern_c_string ("filewatch"), Qnil);
+}
+
+#endif /* HAVE_FILEWATCH */
diff --git a/src/keyboard.c b/src/keyboard.c
index e7a0598..673cb6e 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -329,6 +329,9 @@ static Lisp_Object Qsave_session;
 #ifdef HAVE_DBUS
 static Lisp_Object Qdbus_event;
 #endif
+#ifdef HAVE_FILEWATCH
+static Lisp_Object Qfilewatch_event;
+#endif /* HAVE_FILEWATCH */
 static Lisp_Object Qconfig_changed_event;
 
 /* Lisp_Object Qmouse_movement; - also an event header */
@@ -4017,6 +4020,13 @@ kbd_buffer_get_event (KBOARD **kbp,
 	  kbd_fetch_ptr = event + 1;
 	}
 #endif
+#ifdef HAVE_FILEWATCH
+      else if (event->kind == FILEWATCH_EVENT)
+        {
+          obj = make_lispy_event (event);
+          kbd_fetch_ptr = event + 1;
+        }
+#endif
       else if (event->kind == CONFIG_CHANGED_EVENT)
 	{
 	  obj = make_lispy_event (event);
@@ -5913,6 +5923,13 @@ make_lispy_event (struct input_event *event)
       }
 #endif /* HAVE_DBUS */
 
+#ifdef HAVE_FILEWATCH
+    case FILEWATCH_EVENT:
+      {
+        return Fcons (Qfilewatch_event, event->arg);
+      }
+#endif /* HAVE_FILEWATCH */
+
     case CONFIG_CHANGED_EVENT:
 	return Fcons (Qconfig_changed_event,
                       Fcons (event->arg,
@@ -11524,6 +11541,10 @@ syms_of_keyboard (void)
   DEFSYM (Qdbus_event, "dbus-event");
 #endif
 
+#ifdef HAVE_FILEWATCH
+  DEFSYM (Qfilewatch_event, "filewatch-event");
+#endif /* HAVE_FILEWATCH */
+
   DEFSYM (QCenable, ":enable");
   DEFSYM (QCvisible, ":visible");
   DEFSYM (QChelp, ":help");
@@ -12274,6 +12295,13 @@ keys_of_keyboard (void)
 			    "dbus-handle-event");
 #endif
 
+#ifdef HAVE_FILEWATCH
+  /* Define a special event which is raised for filewatch callback
+     functions.  */
+  initial_define_lispy_key (Vspecial_event_map, "filewatch-event",
+                            "filewatch-handle-event");
+#endif /* HAVE_FILEWATCH */
+
   initial_define_lispy_key (Vspecial_event_map, "config-changed-event",
 			    "ignore");
 }
diff --git a/src/lisp.h b/src/lisp.h
index 1e30363..a8419d1 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3494,6 +3494,11 @@ EXFUN (Fxw_display_color_p, 1);
 EXFUN (Fx_focus_frame, 1);
 #endif
 
+/* Defined in filewatch.c */
+#ifdef HAVE_FILEWATCH
+extern void syms_of_filewatch (void);
+#endif
+
 /* Defined in xfaces.c */
 extern Lisp_Object Qdefault, Qtool_bar, Qfringe;
 extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor;
diff --git a/src/termhooks.h b/src/termhooks.h
index 6a58517..9db2019 100644
--- a/src/termhooks.h
+++ b/src/termhooks.h
@@ -206,6 +206,11 @@ enum event_kind
   , NS_NONKEY_EVENT
 #endif
 
+#ifdef HAVE_FILEWATCH
+  /* File or directory was changed.  */
+  , FILEWATCH_EVENT
+#endif
+
 };
 
 /* If a struct input_event has a kind which is SELECTION_REQUEST_EVENT
diff --git a/test/automated/filewatch-tests.el b/test/automated/filewatch-tests.el
new file mode 100644
index 0000000..494d704
--- /dev/null
+++ b/test/automated/filewatch-tests.el
@@ -0,0 +1,50 @@
+;;; filewatch-tests.el --- Test suite for filewatch.
+
+;; Copyright (C) 2011 Free Software Foundation, Inc.
+
+;; Author: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
+;; Keywords:       internal
+;; Human-Keywords: internal
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Code:
+
+(require 'ert)
+
+(when (featurep 'filewatch)
+
+  (ert-deftest filewatch-file-unwatch-type-check ()
+    "Test whether `file-unwatch' does proper type checking."
+    (should-error (file-unwatch "path") :type 'wrong-type-argument)
+    (should-error (file-unwatch '(file 1 2)) :type 'wrong-type-argument))
+
+  (ert-deftest filewatch-file-watch-aspects-check ()
+    "Test whether `file-watch' properly checks the aspects."
+    (let ((temp-file (make-temp-file "filewatch-aspects")))
+      (should (stringp temp-file))
+      (should-error (file-watch temp-file 'wrong nil)
+                    :type 'unknown-aspect)
+      (should-error (file-watch temp-file '(modify t) nil)
+                    :type 'unknown-aspect)
+      (should-error (file-watch temp-file '(modify Qall-modify) nil)
+                    :type 'unknown-aspect)
+      (should-error (file-watch temp-file '(access wrong modify) nil)
+                    :type 'unknown-aspect)))
+)
+
+(provide 'filewatch-tests)
+;;; filewatch-tests.el ends here.
-- 
1.7.5.4

Re: [PATCH updated] Support for filesystem watching (inotify)

[Ted Zlatanov]

On Fri, 24 Jun 2011 02:50:00 +0200 Rüdiger Sonderfeld <ruediger@c-plusplus.de> wrote: 

RS> +static Lisp_Object Qmodify, Qmove, Qattrib, Qdelete, Qfrom, Qto, Qall_modify;
...
RS> +  if (ev->mask & (IN_MODIFY|IN_CREATE) )
RS> +    events = Fcons (Fcons (Qmodify, name), events);
...
RS> +  if (EQ (symb, Qmodify))
RS> +    return IN_MODIFY | IN_CREATE;

You have several symbols like Qmodify, and I wanted to suggest that you
could use a name prefix like for example gnutls.c:

#+begin_src c
  Qgnutls_bootprop_verify_hostname_error = intern_c_string (":verify-error");
  staticpro (&Qgnutls_bootprop_verify_error);
#+end_src

and that you assign a numeric value directly to the symbol when it's
initialized instead of statically doing case statements in your code.
That way your code will be shorter and simpler because you'll work with
the integer values directly.  This could be a problem if the IN_*
constants overflow the Emacs integer size, but I don't think they will,
after checking inotify.h.  Only the special flags are large (2147483648
= IN_ONESHOT = 0x80000000) and you won't use those.

Ted

Re: [PATCH updated] Support for filesystem watching (inotify)

[Ted Zlatanov]

On Fri, 24 Jun 2011 05:19:20 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> You have several symbols like Qmodify, and I wanted to suggest that you
TZ> could use a name prefix like for example gnutls.c:
TZ> and that you assign a numeric value directly to the symbol when it's
TZ> initialized instead of statically doing case statements in your
TZ> code.

I noticed you were using the DEFSYM macro, I didn't know about it.  I
put it into gnutls.c.

#+begin_src c
  DEFSYM(Qgnutls_e_interrupted, "gnutls-e-interrupted");
  Fput (Qgnutls_e_interrupted, Qgnutls_code,
        make_number (GNUTLS_E_INTERRUPTED));
#+end_src

...and I wanted to mention how I put a numeric value in the symbol's
property list instead of setting it as the value.  HTH.

Ted

Re: [PATCH updated] Support for filesystem watching (inotify)

[Stefan Monnier]

Do you guys think this patch is good enough to make it into Emacs-24.1?

My preference would be to wait for 24.2 since we're in feature freeze,
but it's the kind of feature which benefits from being available early
so other packages can start using it.  WDYT?


        Stefan


>> > The problem is: How to implement the file-unwatch then? I need a way
>> > to identify each separate file-watch request.  What would be the best
>> > way to do that?
>> 
>> How about letting file-watch return a "file-watcher" which you then need
>> to pass to file-unwatch?  This "file-watcher" could be any kind of Elisp
>> data you find convenient for this.  You may decide to provide no other
>> operation than file-unwatch, but you could also decide to provided
>> additional operations such as changing the callback (I'm not saying
>> that would necessarily be a good idea, tho, but maybe other operations
>> would be handy).

> I updated the patch to return a "file-watcher". file-watch can now be called
> several times for the same file with different flags and callbacks. This
> however required large changes to the patch. So it would be great if someone
> could take a look at it again. I added an automated test. But it doesn't test the
> actually file-watching yet. This is a bit tricky to get right in a unit test.
> But I'll add it. The code requires some heavy testing.

> Regards,
> Rüdiger

> From: =?UTF-8?q?R=C3=BCdiger=20Sonderfeld?= <ruediger@c-plusplus.de>
> Date: Fri, 3 Jun 2011 23:58:12 +0200
> Subject: [PATCH] Added basic file system watching support.

> It currently only works with inotify/Linux. It adds file-watch and file-unwatch functions.
> ---
>  configure.in                      |   14 +
>  lisp/filewatch.el                 |   54 ++++
>  src/Makefile.in                   |    2 +-
>  src/emacs.c                       |    4 +
>  src/filewatch.c                   |  485 +++++++++++++++++++++++++++++++++++++
>  src/keyboard.c                    |   28 +++
>  src/lisp.h                        |    5 +
>  src/termhooks.h                   |    5 +
>  test/automated/filewatch-tests.el |   50 ++++
>  9 files changed, 646 insertions(+), 1 deletions(-)
>  create mode 100644 lisp/filewatch.el
>  create mode 100644 src/filewatch.c
>  create mode 100644 test/automated/filewatch-tests.el

> diff --git a/configure.in b/configure.in
> index 0135b9f..4a816ec 100644
> --- a/configure.in
> +++ b/configure.in
> @@ -174,6 +174,7 @@ OPTION_DEFAULT_ON([dbus],[don't compile with D-Bus support])
>  OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
>  OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
>  OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
> +OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) support])
 
>  ## For the times when you want to build Emacs but don't have
>  ## a suitable makeinfo, and can live without the manuals.
> @@ -1999,6 +2000,19 @@ fi
>  AC_SUBST(LIBGNUTLS_LIBS)
>  AC_SUBST(LIBGNUTLS_CFLAGS)
 
> +dnl inotify is only available on GNU/Linux.
> +HAVE_INOTIFY=no
> +if test "${with_inotify}" = "yes"; then
> +   AC_CHECK_HEADERS(sys/inotify.h)
> +   if test "$ac_cv_header_sys_inotify_h" = yes ; then
> +     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, HAVE_INOTIFY=yes)
> +   fi
> +fi
> +if test "${HAVE_INOTIFY}" = "yes"; then
> +   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
> +   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
> +fi
> +
>  dnl Do not put whitespace before the #include statements below.
>  dnl Older compilers (eg sunos4 cc) choke on it.
>  HAVE_XAW3D=no
> diff --git a/lisp/filewatch.el b/lisp/filewatch.el
> new file mode 100644
> index 0000000..2c97589
> --- /dev/null
> +++ b/lisp/filewatch.el
> @@ -0,0 +1,54 @@
> +;;; filewatch.el --- Elisp support for watching filesystem events.
> +
> +;; Copyright (C) 2011 Free Software Foundation, Inc.
> +
> +;; Author: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> +;; Keywords: files
> +
> +;; This file is part of GNU Emacs.
> +
> +;; GNU Emacs is free software: you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation, either version 3 of the License, or
> +;; (at your option) any later version.
> +
> +;; GNU Emacs is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
> +
> +;;; Commentary:
> +
> +;; This file contains the elisp part of the filewatch API.
> +
> +;;; Code:
> +
> +(eval-when-compile
> +  (require 'cl))
> +
> +(defun filewatch-event-p (event)
> +  "Check if EVENT is a filewatch event."
> +  (and (listp event)
> +       (eq (car event) 'filewatch-event)))
> +
> +;;;###autoload
> +(defun filewatch-handle-event (event)
> +  "Handle file system monitoring event.
> +If EVENT is a filewatch event then the callback is called.  If EVENT is
> +not a filewatch event then a `filewatch-error' is signaled."
> +  (interactive "e")
> +
> +  (unless (filewatch-event-p event)
> +    (signal 'filewatch-error (cons "Not a valid filewatch event" event)))
> +
> +  (loop for ev in (cdr event)
> +     unless (and (listp ev) (>= (length ev) 3))
> +     do (signal 'filewatch-error (cons "Not a valid filewatch event" event))
> +     do (funcall (cadr ev) (car ev) (caddr ev))))
> +
> +(provide 'filewatch)
> +
> +;;; filewatch.el ends here
> diff --git a/src/Makefile.in b/src/Makefile.in
> index c4250b9..9135e7d 100644
> --- a/src/Makefile.in
> +++ b/src/Makefile.in
> @@ -334,7 +334,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o $(XMENU_OBJ) window.o \
>  	syntax.o $(UNEXEC_OBJ) bytecode.o \
>  	process.o gnutls.o callproc.o \
>  	region-cache.o sound.o atimer.o \
> -	doprnt.o intervals.o textprop.o composite.o xml.o \
> +	doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
>  	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ)
>  obj = $(base_obj) $(NS_OBJC_OBJ)
 
> diff --git a/src/emacs.c b/src/emacs.c
> index d14acd6..db897d5 100644
> --- a/src/emacs.c
> +++ b/src/emacs.c
> @@ -1563,6 +1563,10 @@ Using an Emacs configured with --with-x-toolkit=lucid does not have this problem
>        syms_of_gnutls ();
>  #endif
 
> +#ifdef HAVE_FILEWATCH
> +      syms_of_filewatch ();
> +#endif /* HAVE_FILEWATCH */
> +
>  #ifdef HAVE_DBUS
>        syms_of_dbusbind ();
>  #endif /* HAVE_DBUS */
> diff --git a/src/filewatch.c b/src/filewatch.c
> new file mode 100644
> index 0000000..2833544
> --- /dev/null
> +++ b/src/filewatch.c
> @@ -0,0 +1,485 @@
> +/* Watching file system changes.
> +
> +Copyright (C) 2011
> +  Free Software Foundation, Inc.
> +
> +This file is part of GNU Emacs.
> +
> +GNU Emacs is free software: you can redistribute it and/or modify
> +it under the terms of the GNU General Public License as published by
> +the Free Software Foundation, either version 3 of the License, or
> +(at your option) any later version.
> +
> +GNU Emacs is distributed in the hope that it will be useful,
> +but WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +GNU General Public License for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
> +
> +#include <config.h>
> +
> +#ifdef HAVE_FILEWATCH
> +#include <setjmp.h> /* Required for lisp.h.  */
> +#include "lisp.h"
> +#include "coding.h"
> +#include "process.h"
> +#include "keyboard.h"
> +#include "character.h"
> +#include "frame.h" /* Required for termhooks.h.  */
> +#include "termhooks.h"
> +
> +static Lisp_Object Qmodify, Qmove, Qattrib, Qdelete, Qfrom, Qto, Qall_modify;
> +static Lisp_Object Qunknown_aspect, Qfile_watch;
> +static Lisp_Object Qaccess, Qclose_write, Qclose_nowrite, Qopen, Qall;
> +
> +#ifdef HAVE_INOTIFY
> +#include <sys/inotify.h>
> +#include <sys/ioctl.h>
> +
> +enum { uninitialized = -100 };
> +/* File handle for inotify.  */
> +static int inotifyfd = uninitialized;
> +
> +/* Assoc list of files being watched.
> + Format:
> + (id . (filename ((subid callback flags) (subid callbacks flags) ...))) */
> +static Lisp_Object watch_list;
> +
> +#define CADDR(x) Fcar (Fcdr (Fcdr (x)))
> +#define CADR(x) Fcar (Fcdr (x))
> +
> +static Lisp_Object
> +append2 (Lisp_Object s1, Lisp_Object s2)
> +{
> +  Lisp_Object args[2];
> +  args[0] = s1;
> +  args[1] = s2;
> +  return Fappend (2, args);
> +}
> +
> +/* Returns a list with all the elements from EVENTS the `watch_list' CELL
> +   is interested in.  */
> +static Lisp_Object
> +match_aspects (Lisp_Object cell, Lisp_Object events)
> +{
> +  Lisp_Object ret = Qnil;
> +  Lisp_Object aspects = CADDR (cell);
> +  Lisp_Object i;
> +  if (EQ (aspects, Qt) || EQ (aspects, Qall))
> +    return events;
> +  else if (EQ (aspects, Qall_modify))
> +    aspects = list4 (Qmodify, Qmove, Qattrib, Qdelete);
> +
> +  i = Fcar (aspects);
> +  while (!NILP (i))
> +    {
> +      Lisp_Object e = Fcar (events);
> +      while (!NILP (e))
> +        {
> +          if (EQ (Fcar (e), i))
> +            ret = Fcons (e, ret);
> +          events = Fcdr (events);
> +          e = Fcar (events);
> +        }
> +      aspects = Fcdr (aspects);
> +      i = Fcar (aspects);
> +    }
> +  return ret;
> +}
> +
> +static Lisp_Object
> +inotifyevent_to_aspects (Lisp_Object name, struct inotify_event *ev)
> +{
> +  Lisp_Object events = Qnil;
> +  if (ev->mask & (IN_MODIFY|IN_CREATE) )
> +    events = Fcons (Fcons (Qmodify, name), events);
> +  if (ev->mask & IN_MOVE_SELF)
> +    events = Fcons (Fcons (Qmove, name), events);
> +  if (ev->mask & IN_MOVED_FROM)
> +    events = Fcons (Fcons (Qmove,
> +                           Fcons (Qfrom,
> +                                  Fcons (name,
> +                                         make_number (ev->cookie)))),
> +                    events);
> +  if (ev->mask & IN_MOVED_TO)
> +    events = Fcons (Fcons (Qmove,
> +                           Fcons (Qto,
> +                                  Fcons (name,
> +                                         make_number (ev->cookie)))),
> +                    events);
> +  if (ev->mask & IN_ATTRIB)
> +    events = Fcons (Fcons (Qattrib, name), events);
> +  if (ev->mask & (IN_DELETE|IN_DELETE_SELF) )
> +    events = Fcons (Fcons (Qdelete, name), events);
> +  if (ev->mask & IN_ACCESS)
> +    events = Fcons (Fcons (Qaccess, name), events);
> +  if (ev->mask & IN_CLOSE_WRITE)
> +    events = Fcons (Fcons (Qclose_write, name), events);
> +  if (ev->mask & IN_CLOSE_NOWRITE)
> +    events = Fcons (Fcons (Qclose_nowrite, name), events);
> +  if (ev->mask & IN_OPEN)
> +    events = Fcons (Fcons (Qopen, name), events);
> +  return events;
> +}
> +
> +/* This callback is called when the FD is available FOR_READ.  The inotify
> +   events are read from FD and converted into input_events.  */
> +static void
> +inotify_callback (int fd, void *_, int for_read)
> +{
> +  struct input_event event;
> +  int to_read;
> +  char *buffer;
> +  ssize_t n;
> +  size_t i;
> +
> +  if (!for_read)
> +    return;
> +
> +  to_read = 0;
> +  if (ioctl (fd, FIONREAD,  &to_read) == -1)
> +    report_file_error ("Error while trying to retrieve file system events",
> +                       Qnil);
> +  buffer = xmalloc (to_read);
> +  n = read (fd, buffer, to_read);
> +  if (n < 0)
> +    {
> +      xfree (buffer);
> +      report_file_error ("Error while trying to read file system events",
> +                         Qnil);
> +    }
> +
> +  EVENT_INIT (event);
> +  event.kind = FILEWATCH_EVENT;
> +  event.arg = Qnil;
> +
> +  i = 0;
> +  while (i < (size_t)n)
> +    {
> +      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
> +
> +      Lisp_Object watch_data = Fassoc (make_number (ev->wd), watch_list);
> +      if (!NILP (watch_data))
> +        {
> +          Lisp_Object name, events;
> +
> +          /* If a directory is watched name contains the name
> +             of the file that was changed.  */
> +          if (ev->len > 0)
> +            {
> +              size_t const len = strlen (ev->name);
> +              name = make_unibyte_string (ev->name, min (len, ev->len));
> +              name = DECODE_FILE (name);
> +            }
> +          else
> +            {
> +              name = CADR (watch_data);
> +            }
> +
> +          events = inotifyevent_to_aspects (name, ev);
> +
> +          if (!NILP (events))
> +            {
> +              Lisp_Object x = CADDR (watch_data);
> +              while (!NILP (x))
> +                {
> +                  Lisp_Object cell = Fcar (x);
> +                  Lisp_Object aspects = match_aspects (cell, events);
> +                  if (!NILP (aspects))
> +                    {
> +                      Lisp_Object id = list3 (Qfile_watch, make_number (ev->wd),
> +                                              Fcar (cell));
> +                      Lisp_Object event_info = list1 (list3 (id, CADR (cell),
> +                                                             aspects));
> +
> +                      if (NILP (event.arg))
> +                        event.arg = event_info;
> +                      else
> +                        event.arg = append2 (event.arg, event_info);
> +                    }
> +                  x = Fcdr (x);
> +                }
> +            }
> +
> +          if (ev->mask & IN_IGNORED)
> +            {
> +              /* Event was removed automatically: Drop it from data list.  */
> +              add_to_log ("File-watch: \"%s\" will be ignored", name, Qnil);
> +              watch_list = Fdelete (watch_data, watch_list);
> +            }
> +          if (ev->mask & IN_Q_OVERFLOW)
> +            add_to_log ("File watch: Inotify Queue Overflow!", Qnil, Qnil);
> +        }
> +
> +      i += sizeof (*ev) + ev->len;
> +    }
> +
> +  if (!NILP (event.arg))
> +    kbd_buffer_store_event (&event);
> +
> +  xfree (buffer);
> +}
> +
> +static uint32_t
> +symbol_to_inotifymask (Lisp_Object symb, int in_list)
> +{
> +  if (EQ (symb, Qmodify))
> +    return IN_MODIFY | IN_CREATE;
> +  else if (EQ (symb, Qmove))
> +    return IN_MOVE_SELF | IN_MOVE;
> +  else if (EQ (symb, Qattrib))
> +    return IN_ATTRIB;
> +  else if (EQ (symb, Qdelete))
> +    return IN_DELETE_SELF | IN_DELETE;
> +  else if (!in_list && EQ (symb, Qall_modify))
> +    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
> +      | IN_DELETE_SELF | IN_DELETE;
> +  else if (EQ (symb, Qaccess))
> +    return IN_ACCESS;
> +  else if (EQ (symb, Qclose_write))
> +    return IN_CLOSE_WRITE;
> +  else if (EQ (symb, Qclose_nowrite))
> +    return IN_CLOSE_NOWRITE;
> +  else if (EQ (symb, Qopen))
> +    return IN_OPEN;
> +  else if (!in_list && (EQ (symb, Qt) || EQ (symb, Qall)))
> +    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
> +      | IN_DELETE_SELF | IN_DELETE | IN_ACCESS | IN_CLOSE_WRITE
> +      | IN_CLOSE_NOWRITE | IN_OPEN;
> +  else
> +    Fsignal (Qunknown_aspect, Fcons (symb, Qnil));
> +}
> +
> +static uint32_t
> +aspect_to_inotifymask (Lisp_Object aspect)
> +{
> +  if (CONSP (aspect))
> +    {
> +      Lisp_Object x = aspect;
> +      uint32_t mask = 0;
> +      while (!NILP (x))
> +        {
> +          mask |= symbol_to_inotifymask (Fcar (x), 1);
> +          x = Fcdr (x);
> +        }
> +      return mask;
> +    }
> +  else
> +    return symbol_to_inotifymask (aspect, 0);
> +}
> +
> +static Lisp_Object
> +get_watch_data_by_file_name (Lisp_Object file_name)
> +{
> +  Lisp_Object cell, file_name_data;
> +  Lisp_Object x = watch_list;
> +  CHECK_STRING (file_name);
> +
> +  while (!NILP (x))
> +    {
> +      cell = Fcar (x);
> +      x = Fcdr (x);
> +      file_name_data = Fcdr (cell);
> +      if (!NILP (file_name_data) && STRINGP (Fcar (file_name_data))
> +          && !NILP (Fstring_equal (Fcar (file_name_data), file_name))
> +          && NUMBERP (Fcar (cell)))
> +        return cell;
> +    }
> +  return Qnil;
> +}
> +
> +DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, 3, 0,
> +       doc: /* Arrange to call FUNC if ASPECT of FILENAME changes.
> +
> +ASPECT might be one of the following symbols or a list of those symbols (except
> +for all-modify and t):
> +
> +modify -- notify when a file is modified or created.
> +
> +move -- notify when a file/directory is moved.
> +
> +attrib -- notify when attributes change.
> +
> +delete -- notify when a file/directory is deleted.
> +
> +all-modify -- notify for all of the above (all modifying changes).
> +
> +access -- notify when file was accessed/read.
> +
> +close-write -- notify when a file opened for writing was closed.
> +
> +close-nowrite -- notify when a file not opened for writing was closed.
> +
> +open -- notify when a file was opened.
> +
> +t -- notify for all of the above.
> +
> +Watching a directory is not recursive.  CALLBACK receives the events as a list
> +with each list element being a list containing information about an event.  The
> +first element is a flag symbol.  If a directory is watched the second element is
> +the name of the file that changed.  If a file is moved from or to the directory
> +the second element is either 'from or 'to and the third element is the file
> +name.  A fourth element contains a numeric identifier (cookie) that can be used
> +to identify matching move operations if a file is moved inside the directory.
> +
> +Example:
> +(file-watch "foo" t #'(lambda (id event) (message "%s %s" id event)))
> +
> +Use `file-unwatch' to stop watching.
> +
> +usage: (file-watch FILENAME ASPECT CALLBACK) */)
> +  (Lisp_Object file_name, Lisp_Object aspect, Lisp_Object callback)
> +{
> +  static int intern_counter = 0; /* assign local ids */
> +  uint32_t mask;
> +  int watchdesc;
> +  Lisp_Object decoded_file_name;
> +  Lisp_Object data;
> +  Lisp_Object info;
> +
> +  CHECK_STRING (file_name);
> +
> +  if (inotifyfd == uninitialized)
> +    {
> +      inotifyfd = inotify_init1 (IN_NONBLOCK|IN_CLOEXEC);
> +      if (inotifyfd == -1)
> +        {
> +          inotifyfd = uninitialized;
> +          report_file_error ("File watching feature (inotify) is not available",
> +                             Qnil);
> +        }
> +      watch_list = Qnil;
> +      add_read_fd (inotifyfd, &inotify_callback, &watch_list);
> +    }
> +
> +  mask = aspect_to_inotifymask (aspect);
> +  data = get_watch_data_by_file_name (file_name);
> +  if (!NILP (data))
> +    mask |= IN_MASK_ADD;
> +
> +  decoded_file_name = ENCODE_FILE (file_name);
> +  watchdesc = inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), mask);
> +  if (watchdesc == -1)
> +    report_file_error ("Could not watch file", Fcons (file_name, Qnil));
> +
> +  info = list3 (make_number (intern_counter), callback, aspect);
> +  ++intern_counter;
> +
> +  if (!NILP (data))
> +    Fsetcdr (Fcdr (data), Fcons (append2 (CADDR (data), Fcons (info, Qnil)),
> +                                 Qnil));
> +  else
> +    watch_list = Fcons (list3 (make_number (watchdesc),
> +                               file_name, Fcons (info, Qnil)),
> +                        watch_list);
> +
> +  return list3 (Qfile_watch, make_number (watchdesc), Fcar (info));
> +}
> +
> +static int
> +file_watch_objectp (Lisp_Object obj)
> +{
> +  return CONSP (obj) && XINT (Flength (obj)) == 3
> +    && EQ (Fcar (obj), Qfile_watch);
> +}
> +
> +DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
> +       doc: /* Stop watching a file or directory.  */)
> +  (Lisp_Object watch_object)
> +{
> +  Lisp_Object watch_data;
> +  Lisp_Object info;
> +
> +  if (!file_watch_objectp (watch_object))
> +    wrong_type_argument (Qfile_watch, watch_object);
> +
> +  if (inotifyfd == uninitialized)
> +    return Qnil;
> +
> +  watch_data = Fassoc ( CADR (watch_object), watch_list );
> +  if (NILP (watch_data))
> +    return Qnil;
> +
> +  info = Fassoc (CADDR (watch_object), CADDR (watch_data));
> +  if (NILP (info))
> +    return Qnil;
> +
> +  Fsetcdr (Fcdr (watch_data), Fcons (Fdelete (info, CADDR (watch_data)), Qnil));
> +
> +  if (NILP (CADDR (watch_data)))
> +    {
> +      int const magicno = XINT (CADR (watch_object));
> +      watch_list = Fdelete (watch_data, watch_list);
> +
> +      if (inotify_rm_watch (inotifyfd, magicno) == -1)
> +        report_file_error ("Could not unwatch file",
> +                           Fcons (Fcar (watch_data), Qnil));
> +
> +      /* Cleanup if watch_list is empty.  */
> +      if (NILP (watch_list))
> +        {
> +          close (inotifyfd);
> +          delete_read_fd (inotifyfd);
> +          inotifyfd = uninitialized;
> +        }
> +    }
> +  else
> +    {
> +      Lisp_Object decoded_file_name = ENCODE_FILE (CADR (watch_data));
> +      Lisp_Object x = CADDR (watch_data);
> +      uint32_t mask = 0;
> +      while (!NILP (x))
> +        {
> +          Lisp_Object cell = Fcar (x);
> +          mask |= aspect_to_inotifymask (CADDR (cell));
> +          x = Fcdr (x);
> +        }
> +      /* Reset watch mask */
> +      if (inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), mask) == -1)
> +        report_file_error ("Could not unwatch file",
> +                           Fcons (Fcar (watch_data), Qnil));
> +    }
> +
> +  return Qt;
> +}
> +
> +#else /* HAVE_INOTIFY */
> +#error "Filewatch defined but no watch mechanism (inotify) available"
> +#endif /* HAVE_INOTIFY */
> +
> +void
> +syms_of_filewatch (void)
> +{
> +  DEFSYM (Qmodify, "modify");
> +  DEFSYM (Qmove, "move");
> +  DEFSYM (Qattrib, "attrib");
> +  DEFSYM (Qdelete, "delete");
> +  DEFSYM (Qfrom, "from");
> +  DEFSYM (Qto, "to");
> +  DEFSYM (Qall_modify, "all-modify");
> +
> +  DEFSYM (Qaccess, "access");
> +  DEFSYM (Qclose_write, "close-write");
> +  DEFSYM (Qclose_nowrite, "close-nowrite");
> +  DEFSYM (Qopen, "open");
> +  DEFSYM (Qall, "all");
> +
> +  DEFSYM (Qunknown_aspect, "unknown-aspect");
> +  DEFSYM (Qfile_watch, "file-watch");
> +
> +  Fput (Qunknown_aspect, Qerror_conditions,
> +        list2 (Qunknown_aspect, Qerror));
> +  Fput (Qunknown_aspect, Qerror_message,
> +        make_pure_c_string ("Unknown Aspect"));
> +
> +  defsubr (&Sfile_watch);
> +  defsubr (&Sfile_unwatch);
> +
> +  staticpro (&watch_list);
> +
> +  Fprovide (intern_c_string ("filewatch"), Qnil);
> +}
> +
> +#endif /* HAVE_FILEWATCH */
> diff --git a/src/keyboard.c b/src/keyboard.c
> index e7a0598..673cb6e 100644
> --- a/src/keyboard.c
> +++ b/src/keyboard.c
> @@ -329,6 +329,9 @@ static Lisp_Object Qsave_session;
>  #ifdef HAVE_DBUS
>  static Lisp_Object Qdbus_event;
>  #endif
> +#ifdef HAVE_FILEWATCH
> +static Lisp_Object Qfilewatch_event;
> +#endif /* HAVE_FILEWATCH */
>  static Lisp_Object Qconfig_changed_event;
 
>  /* Lisp_Object Qmouse_movement; - also an event header */
> @@ -4017,6 +4020,13 @@ kbd_buffer_get_event (KBOARD **kbp,
>  	  kbd_fetch_ptr = event + 1;
>  	}
>  #endif
> +#ifdef HAVE_FILEWATCH
> +      else if (event->kind == FILEWATCH_EVENT)
> +        {
> +          obj = make_lispy_event (event);
> +          kbd_fetch_ptr = event + 1;
> +        }
> +#endif
>        else if (event->kind == CONFIG_CHANGED_EVENT)
>  	{
>  	  obj = make_lispy_event (event);
> @@ -5913,6 +5923,13 @@ make_lispy_event (struct input_event *event)
>        }
>  #endif /* HAVE_DBUS */
 
> +#ifdef HAVE_FILEWATCH
> +    case FILEWATCH_EVENT:
> +      {
> +        return Fcons (Qfilewatch_event, event->arg);
> +      }
> +#endif /* HAVE_FILEWATCH */
> +
>      case CONFIG_CHANGED_EVENT:
>  	return Fcons (Qconfig_changed_event,
>                        Fcons (event->arg,
> @@ -11524,6 +11541,10 @@ syms_of_keyboard (void)
>    DEFSYM (Qdbus_event, "dbus-event");
>  #endif
 
> +#ifdef HAVE_FILEWATCH
> +  DEFSYM (Qfilewatch_event, "filewatch-event");
> +#endif /* HAVE_FILEWATCH */
> +
>    DEFSYM (QCenable, ":enable");
>    DEFSYM (QCvisible, ":visible");
>    DEFSYM (QChelp, ":help");
> @@ -12274,6 +12295,13 @@ keys_of_keyboard (void)
>  			    "dbus-handle-event");
>  #endif
 
> +#ifdef HAVE_FILEWATCH
> +  /* Define a special event which is raised for filewatch callback
> +     functions.  */
> +  initial_define_lispy_key (Vspecial_event_map, "filewatch-event",
> +                            "filewatch-handle-event");
> +#endif /* HAVE_FILEWATCH */
> +
>    initial_define_lispy_key (Vspecial_event_map, "config-changed-event",
>  			    "ignore");
>  }
> diff --git a/src/lisp.h b/src/lisp.h
> index 1e30363..a8419d1 100644
> --- a/src/lisp.h
> +++ b/src/lisp.h
> @@ -3494,6 +3494,11 @@ EXFUN (Fxw_display_color_p, 1);
>  EXFUN (Fx_focus_frame, 1);
>  #endif
 
> +/* Defined in filewatch.c */
> +#ifdef HAVE_FILEWATCH
> +extern void syms_of_filewatch (void);
> +#endif
> +
>  /* Defined in xfaces.c */
>  extern Lisp_Object Qdefault, Qtool_bar, Qfringe;
>  extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor;
> diff --git a/src/termhooks.h b/src/termhooks.h
> index 6a58517..9db2019 100644
> --- a/src/termhooks.h
> +++ b/src/termhooks.h
> @@ -206,6 +206,11 @@ enum event_kind
>    , NS_NONKEY_EVENT
>  #endif
 
> +#ifdef HAVE_FILEWATCH
> +  /* File or directory was changed.  */
> +  , FILEWATCH_EVENT
> +#endif
> +
>  };
 
>  /* If a struct input_event has a kind which is SELECTION_REQUEST_EVENT
> diff --git a/test/automated/filewatch-tests.el b/test/automated/filewatch-tests.el
> new file mode 100644
> index 0000000..494d704
> --- /dev/null
> +++ b/test/automated/filewatch-tests.el
> @@ -0,0 +1,50 @@
> +;;; filewatch-tests.el --- Test suite for filewatch.
> +
> +;; Copyright (C) 2011 Free Software Foundation, Inc.
> +
> +;; Author: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> +;; Keywords:       internal
> +;; Human-Keywords: internal
> +
> +;; This file is part of GNU Emacs.
> +
> +;; GNU Emacs is free software: you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation, either version 3 of the License, or
> +;; (at your option) any later version.
> +
> +;; GNU Emacs is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
> +
> +;;; Code:
> +
> +(require 'ert)
> +
> +(when (featurep 'filewatch)
> +
> +  (ert-deftest filewatch-file-unwatch-type-check ()
> +    "Test whether `file-unwatch' does proper type checking."
> +    (should-error (file-unwatch "path") :type 'wrong-type-argument)
> +    (should-error (file-unwatch '(file 1 2)) :type 'wrong-type-argument))
> +
> +  (ert-deftest filewatch-file-watch-aspects-check ()
> +    "Test whether `file-watch' properly checks the aspects."
> +    (let ((temp-file (make-temp-file "filewatch-aspects")))
> +      (should (stringp temp-file))
> +      (should-error (file-watch temp-file 'wrong nil)
> +                    :type 'unknown-aspect)
> +      (should-error (file-watch temp-file '(modify t) nil)
> +                    :type 'unknown-aspect)
> +      (should-error (file-watch temp-file '(modify Qall-modify) nil)
> +                    :type 'unknown-aspect)
> +      (should-error (file-watch temp-file '(access wrong modify) nil)
> +                    :type 'unknown-aspect)))
> +)
> +
> +(provide 'filewatch-tests)
> +;;; filewatch-tests.el ends here.
> -- 
> 1.7.5.4

Re: [PATCH updated] Support for filesystem watching (inotify)

[Paul Eggert]

On 07/06/11 06:36, Stefan Monnier wrote:

> My preference would be to wait for 24.2 since we're in feature freeze,
> but it's the kind of feature which benefits from being available early
> so other packages can start using it.  WDYT?

I briefly looked at it for integer-overflow issues, and found one: it
assumes that inotify cookies fit into an Emacs fixnum.  This assumption
isn't true on 32-bit hosts, unless Emacs is configured with
--with-wide-int.  As you know I'm a fan of wide integers, and my preferred
solution would be to make --with-wide-int the default, which would solve
the problem.  As that is also being considered for 24.2, perhaps the
inotify feature should wait for 24.2 as well.

Re: [PATCH updated] Support for filesystem watching (inotify)

[Stefan Monnier]

>> My preference would be to wait for 24.2 since we're in feature freeze,
>> but it's the kind of feature which benefits from being available early
>> so other packages can start using it.  WDYT?
> I briefly looked at it for integer-overflow issues, and found one: it
> assumes that inotify cookies fit into an Emacs fixnum.

Thanks.  This needs to be fixed, indeed.

> This assumption isn't true on 32-bit hosts, unless Emacs is configured
> with --with-wide-int.  As you know I'm a fan of wide integers, and my
> preferred solution would be to make --with-wide-int the default, which
> would solve the problem.

No, that would only solve the problem if the --with-wide-int option is
removed on 32bit systems and imposed as the only choice.  Even if we may
change the default for 24.2 we're definitely not going to drop support
for "narrow int" operation so soon.


        Stefan

Re: [PATCH updated] Support for filesystem watching (inotify)

[Paul Eggert]

On 07/06/11 11:30, Stefan Monnier wrote:
> that would only solve the problem if the --with-wide-int option is
> removed on 32bit systems and imposed as the only choice.

Another possibility would be to enable inotify only if
--with-wide-int is also in effect.  Every platform
that has inotify also has wide ints, so in 24.2 this would
work except for small number of people who configure
--without-wide-int.

Come to think of it, we can do that now.  I.e., we can put
in the inotify stuff now, but have it available only for
the small number of people who configure --with-wide-int.
This might help early adopters try out inotify without it being
"mainstream" in 24.

wide-int crash [was Re: [PATCH updated] Support for filesystem watching (inotify)]

[Glenn Morris]

Paul Eggert wrote:

> As you know I'm a fan of wide integers, and my preferred solution
> would be to make --with-wide-int the default, which would solve the
> problem.

Could you look at this report sometime?

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8884

Re: wide-int crash [was Re: [PATCH updated] Support for filesystem watching (inotify)]

[Paul Eggert]

On 07/06/11 12:14, Glenn Morris wrote:

> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8884

Thanks for the heads-up.  Although the nearby code has a portability bug
regardless of whether --with-wide-int is used, --with-wide-int
is more likely to tickle the bug.  The bug could well explain the
symptoms Peter observed.

I committed the following patch into the trunk.  Peter,
can you please check whether it solves your problem?  If not,
can you please compile with -g and without -O, and use GDB to report
the following values at the point of the crash: buffer_local_flags,
idx, offset, sizeof (struct buffer).  Thanks.

=== modified file 'src/ChangeLog'
--- src/ChangeLog	2011-07-05 09:51:56 +0000
+++ src/ChangeLog	2011-07-06 22:22:32 +0000
@@ -1,3 +1,15 @@
+2011-07-06  Paul Eggert  <eggert@cs.ucla.edu>
+
+	Remove unportable assumption about struct layout (Bug#8884).
+	* alloc.c (mark_buffer):
+	* buffer.c (reset_buffer_local_variables, Fbuffer_local_variables)
+	(clone_per_buffer_values): Don't assume that
+	sizeof (struct buffer) is a multiple of sizeof (Lisp_Object).
+	This isn't true in general, and it's particularly not true
+	if Emacs is configured with --with-wide-int.
+	* buffer.h (FIRST_FIELD_PER_BUFFER, LAST_FIELD_PER_BUFFER):
+	New macros, used in the buffer.c change.
+
 2011-07-05  Jan Djärv  <jan.h.d@swipnet.se>
 
 	* xsettings.c: Use both GConf and GSettings if both are available.

=== modified file 'src/alloc.c'
--- src/alloc.c	2011-06-24 21:25:22 +0000
+++ src/alloc.c	2011-07-06 22:22:32 +0000
@@ -5619,7 +5619,8 @@
   /* buffer-local Lisp variables start at `undo_list',
      tho only the ones from `name' on are GC'd normally.  */
   for (ptr = &buffer->BUFFER_INTERNAL_FIELD (name);
-       (char *)ptr < (char *)buffer + sizeof (struct buffer);
+       ptr <= &PER_BUFFER_VALUE (buffer,
+				 PER_BUFFER_VAR_OFFSET (LAST_FIELD_PER_BUFFER));
        ptr++)
     mark_object (*ptr);
 

=== modified file 'src/buffer.c'
--- src/buffer.c	2011-07-04 15:32:22 +0000
+++ src/buffer.c	2011-07-06 22:22:32 +0000
@@ -471,8 +471,8 @@
 
   /* buffer-local Lisp variables start at `undo_list',
      tho only the ones from `name' on are GC'd normally.  */
-  for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
-       offset < sizeof *to;
+  for (offset = PER_BUFFER_VAR_OFFSET (FIRST_FIELD_PER_BUFFER);
+       offset <= PER_BUFFER_VAR_OFFSET (LAST_FIELD_PER_BUFFER);
        offset += sizeof (Lisp_Object))
     {
       Lisp_Object obj;
@@ -830,8 +830,8 @@
 
   /* buffer-local Lisp variables start at `undo_list',
      tho only the ones from `name' on are GC'd normally.  */
-  for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
-       offset < sizeof *b;
+  for (offset = PER_BUFFER_VAR_OFFSET (FIRST_FIELD_PER_BUFFER);
+       offset <= PER_BUFFER_VAR_OFFSET (LAST_FIELD_PER_BUFFER);
        offset += sizeof (Lisp_Object))
     {
       int idx = PER_BUFFER_IDX (offset);
@@ -1055,8 +1055,8 @@
 
     /* buffer-local Lisp variables start at `undo_list',
        tho only the ones from `name' on are GC'd normally.  */
-    for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
-	 offset < sizeof (struct buffer);
+    for (offset = PER_BUFFER_VAR_OFFSET (FIRST_FIELD_PER_BUFFER);
+	 offset <= PER_BUFFER_VAR_OFFSET (LAST_FIELD_PER_BUFFER);
 	 /* sizeof EMACS_INT == sizeof Lisp_Object */
 	 offset += (sizeof (EMACS_INT)))
       {

=== modified file 'src/buffer.h'
--- src/buffer.h	2011-06-21 21:32:10 +0000
+++ src/buffer.h	2011-07-06 21:53:56 +0000
@@ -612,6 +612,7 @@
   /* Everything from here down must be a Lisp_Object.  */
   /* buffer-local Lisp variables start at `undo_list',
      tho only the ones from `name' on are GC'd normally.  */
+  #define FIRST_FIELD_PER_BUFFER undo_list
 
   /* Changes in the buffer are recorded here for undo.
      t means don't record anything.
@@ -846,6 +847,9 @@
      t means to use hollow box cursor.
      See `cursor-type' for other values.  */
   Lisp_Object BUFFER_INTERNAL_FIELD (cursor_in_non_selected_windows);
+
+  /* This must be the last field in the above list.  */
+  #define LAST_FIELD_PER_BUFFER cursor_in_non_selected_windows
 };
 
 \f

bug#8884: wide-int crash

[Glenn Morris]

Paul Eggert wrote:

> I committed the following patch into the trunk.  Peter,
> can you please check whether it solves your problem?  If not,
> can you please compile with -g and without -O, and use GDB to report
> the following values at the point of the crash: buffer_local_flags,
> idx, offset, sizeof (struct buffer).

Since no reply was received I am closing this under the presumption that
it was fixed.

bug#8884: 24.0.50; temacs crashes and dumps core when configured --with-wide-int

[Peter Dyballa]

Mac OS X 10.5.8, PPC, PowerPC 7447A (7450), GCC 4.5.2. Configuration  
with:

	CFLAGS="-g -H -pipe -fPIC -fno-common -mcpu=7450 -mtune=7450 - 
maltivec -faltivec -mabi=altivec -Os -mfused-madd -mmultiple -ftree- 
vectorize"
	LDFLAGS="-Wl,-dead_strip_dylibs -Wl,-bind_at_load -Wl,-t"


I'm trying again with MacPorts instead of Fink.

--
Greetings

   Pete

There is no national science just as there is no national  
multiplication table; what is national is no longer science.
				– Anton Checov

bug#8884: 24.0.50; temacs crashes and dumps core when configured --with-wide-int

[Peter Dyballa]

I removed all optimisation CFLAGS ("-g -H -pipe -fPIC -O0" were left),  
even -fPIC, and all LDFLAGS. The result is always the same, crash and  
core dump:

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000001075ad60
Crashed Thread:  0

Thread 0 Crashed:
0   temacs                        	0x00241084  
reset_buffer_local_variables + 2192 (buffer.c:833)
1   temacs                        	0x00249d9c  
Fkill_all_local_variables + 112 (buffer.c:2462)
2   temacs                        	0x00270ad4 get_minibuffer + 876  
(minibuf.c:807)
3   temacs                        	0x00255d60 init_buffer + 1032  
(buffer.c:5137)
4   temacs                        	0x001f33b0 main + 5616 (emacs.c:1444)
5   temacs                        	0x00002a3c start + 64

--
Greetings

   Pete

Sending unsolicited commercial eMail to this account incurs a fee of €  
500 per message and acknowledges the legality of this contract.

bug#8884: closed (Re: bug#8884: wide-int crash)

[Peter Dyballa]

Am 07.10.2011 um 09:31 schrieb GNU bug Tracking System:

> Since no reply was received I am closing this under the presumption that
> it was fixed.

Paul,

I'm sorry, I was too occupied with other things! Now that GNU Emacs 24.0.90 compiles well it also runs well with this wide-int patch. I compiled with -O0 and -Os with GCC 4.5.3, the original compiler. I'll try now with Apple modified GCC version 4.0 and 4.2 and their -fast optimiser switch. For theses tests I'll use the new optimised GNU Emacs 24.0.90.

--
Greetings

  Pete

One person with a belief is a social power equal to ninety-nine who have only interests.
				– John Stuart Mill

bug#8884: closed (Re: bug#8884: wide-int crash)

[Peter Dyballa]

Hello!

I managed to create quite large test files on an external disk:

-rw-r--r--  1 pete pete 11936487768 17. Okt 15:26 Halde.tar.lzo
-rw-r--r--  1 pete pete  5990912000 17. Okt 19:40 SW.tar

On my 64-bit intel-based (Sandy Bridge) system GNU Emacs 24.0.90 quite  
easily loads SW.tar (it allocates some GB of swap space) and also  
started to allocate even more swap space when I tried to view a small  
file in the archive – as expected!

On my 32-bit PowerPC 7447A based system GNU Emacs 24.0.90 (started  
with -Q) fails to open the same file (I created it on this system):

	File SW.tar is large (5713MB), really open? (y or n)  y
	byte-code: Maximum buffer size exceeded
	Auto-saving...

The *Buffers List* buffer has:

	. * *Messages*            2320  Fundamental
	  * *unsent mail to bug-: 4855  Message		  ~/Mail/drafts/ 
*message*-20111017-233119
	 %  Leo                   1025  Dired by name	  /Volumes/Leo/
	    *scratch*              191  Lisp Interaction
	 %  *Bug Help*             279  Help
	    SW.tar                   0  Fundamental

The unsent mail contains:

	In GNU Emacs 24.0.90.1 (powerpc-apple-darwin9.8.0, X toolkit, Xaw3d  
scroll bars)
	 of 2011-10-17 on Latsche.fritz.box
	Windowing system distributor `The X.Org Foundation', version  
11.0.11100000
	configured using `configure  '--without-sound' '--without-dbus' '-- 
without-pop' '--without-gconf' '--without-gpm' '--without-gsettings'  
'--without-gif' '--without-jpeg' '--without-png' '--without-rsvg' '-- 
without-tiff' '--without-xpm' '--with-wide-int' '--with-x- 
toolkit=athena' '--x-libraries=/usr/X11/lib' '--x-includes=/usr/X11/ 
include' '--enable-locallisppath=/Library/Application Support/Emacs/ 
calendar24:/Library/Application Support/Emacs' 'CFLAGS=-g -H -pipe - 
fPIC -fno-common -mcpu=7450 -mtune=7450 -maltivec -faltivec - 
mabi=altivec -Os -mfused-madd -mmultiple -ftree-vectorize -mpowerpc- 
gfxopt' 'LDFLAGS=-Wl,-dead_strip_dylibs -Wl,-bind_at_load -Wl,-t'  
'CC=gcc-4' 'CPP=cpp-4' 'PKG_CONFIG_PATH=/sw/lib/xft2/lib/pkgconfig:/sw/ 
share/pkgconfig:/sw/lib/pkgconfig:/usr/X11/lib/pkgconfig:/usr/X11/ 
share/pkgconfig:/usr/lib/pkgconfig''
	
	Important settings:
	  value of $LC_ALL: nil
	  value of $LC_COLLATE: nil
	  value of $LC_CTYPE: de_DE.UTF-8
	  value of $LC_MESSAGES: nil
	  value of $LC_MONETARY: nil
	  value of $LC_NUMERIC: nil
	  value of $LC_TIME: nil
	  value of $LANG: de_DE.UTF-8
	  value of $XMODIFIERS: nil
	  locale-coding-system: utf-8-unix
	  default enable-multibyte-characters: t
	
	Major mode: Lisp Interaction
	
	Minor modes in effect:
	  tooltip-mode: t
	  mouse-wheel-mode: t
	  tool-bar-mode: t
	  menu-bar-mode: t
	  file-name-shadow-mode: t
	  global-font-lock-mode: t
	  font-lock-mode: t
	  blink-cursor-mode: t
	  auto-composition-mode: t
	  auto-encryption-mode: t
	  auto-compression-mode: t
	  line-number-mode: t
	  transient-mark-mode: t

"gcc-4" is: gcc-4 (GCC) 4.5.3.


--
Greetings

              ~  O
   Pete       ~~_\\_/%
              ~  O  o

bug#8884: closed (Re: bug#8884: wide-int crash)

[Paul Eggert]

On 10/17/11 15:24, Peter Dyballa wrote:
> On my 32-bit PowerPC 7447A based system GNU Emacs 24.0.90 (started with -Q) fails to open the same file (I created it on this system):
> 
>     File SW.tar is large (5713MB), really open? (y or n)  y
>     byte-code: Maximum buffer size exceeded
>     Auto-saving...

Perhaps I'm missing something, but
this is the behavior I expect.
SW.tar is too large to fit into that system's memory.
(A 32-bit machine can't access more than 4 GiB of memory.

bug#8884: closed (Re: bug#8884: wide-int crash)

[Peter Dyballa]

Am 18.10.2011 um 07:42 schrieb Paul Eggert:

> Perhaps I'm missing something, but
> this is the behavior I expect.
> SW.tar is too large to fit into that system's memory.
> (A 32-bit machine can't access more than 4 GiB of memory.

It rather looks that *I* am missing some understanding... I thought that the use of wide ints on a 32-bit architecture was meant to give it the ability to handle larger files. What is the intention of their introduction then? Increasing the size of the binary? The Emacs NEWS/Changelog only tells:

	** There is a new configure option --with-wide-int.
	With it, Emacs integers typically have 62 bits, even on 32-bit machines.

From buffers.texi this text is derived:

	   A buffer's size cannot be larger than some maximum, which is defined
	by the largest buffer position representable by the "Emacs integer"
	data type.  This is because Emacs tracks buffer positions using that
	data type.  For 32-bit machines, the largest buffer size is 256
	megabytes.

4 GiB is 16 times that value, quite a bit more. Maths with large numbers, greater than 2**28, works. 

--
Greetings

  Pete

Increase the size of your bike by at least *five* inches!

bug#8884: closed (Re: bug#8884: wide-int crash)

[Eli Zaretskii]

> From: Peter Dyballa <Peter_Dyballa@Freenet.DE>
> Date: Tue, 18 Oct 2011 11:14:10 +0200
> Cc: 8884@debbugs.gnu.org
> 
> > (A 32-bit machine can't access more than 4 GiB of memory.
> 
> It rather looks that *I* am missing some understanding... I thought that the use of wide ints on a 32-bit architecture was meant to give it the ability to handle larger files.

It was, and it does.  See below.

> What is the intention of their introduction then? Increasing the size of the binary?

:-)


> 	** There is a new configure option --with-wide-int.
> 	With it, Emacs integers typically have 62 bits, even on 32-bit machines.

Perhaps the NEWS entry could be improved.

> From buffers.texi this text is derived:
> 
> 	   A buffer's size cannot be larger than some maximum, which is defined
> 	by the largest buffer position representable by the "Emacs integer"
> 	data type.  This is because Emacs tracks buffer positions using that
> 	data type.  For 32-bit machines, the largest buffer size is 256
> 	megabytes.
> 
> 4 GiB is 16 times that value, quite a bit more. Maths with large numbers, greater than 2**28, works. 

Exactly.  So with this option, you can:

  . visit files or have buffers up to 4GB

  . use integer arithmetics up to 62-bit values

None of these is possible without this option.  The first of these two
advantages is the more important one: Emacs can now access the largest
files supported by the OS, instead of having a more-or-less arbitrary
limit below that.

Sounds better now?

bug#8884: closed (Re: bug#8884: wide-int crash)

[Stefan Monnier]

>> 4 GiB is 16 times that value, quite a bit more. Maths with large
>> numbers, greater than 2**28, works.
> Exactly.  So with this option, you can:
>   . visit files or have buffers up to 4GB

Actually, it will usually be less than 4GB, it depends on the amount of
space the OS gives us (it can be as little as 2GB).  E.g. you definitely
won't be able to open two files of 2.5GB each.

The current buffers.texi is fairly clear:

     A buffer's size cannot be larger than some maximum, which is defined
   by the largest buffer position representable by the @dfn{Emacs
   integer} data type.  This is because Emacs tracks buffer positions
   using that data type.  For typical 64-bit machines, the maximum buffer size
   enforced by the data types is @math{2^61 - 2} bytes, or about 2 EiB.
   For typical 32-bit machines, the maximum is @math{2^29 - 2} bytes, or
   about 512 MiB.  Buffer sizes are also limited by the size of Emacs's
   virtual memory.

So w.r.t accessing large files, --with-wide-int pushes the limit from
512MB to "a bit less than 4GB".  That's one of the reasons why I'm not
convinced it's worth the trouble, tho admittedly, the zone between 512MB
and 4GB might contain a fair number of mbox files, so it can be useful
for Rmail users.

> None of these is possible without this option.  The first of these two
> advantages is the more important one: Emacs can now access the largest
> files supported by the OS, instead of having a more-or-less arbitrary
> limit below that.

Of course, it's actually not "the largest files supported by the OS",
since most OSes nowadays happily handle files larger than 4GB, even on
32bit systems.


        Stefan

bug#8884: closed (Re: bug#8884: wide-int crash)

[Paul Eggert]

On 10/18/11 03:55, Eli Zaretskii wrote:
> Perhaps the NEWS entry could be improved.

I made the following change, to try to do that:

--- etc/NEWS	2011-10-18 06:52:32 +0000
+++ etc/NEWS	2011-10-18 15:34:06 +0000
@@ -51,6 +51,8 @@ This is not a new feature; only the conf
 ---
 ** There is a new configure option --with-wide-int.
 With it, Emacs integers typically have 62 bits, even on 32-bit machines.
+On 32-bit hosts, this raises the limit on buffer sizes from 512 MiB to
+a bit less than 4 GiB, with the exact limit set by the operating system.
 
 ---
 ** New translation of the Emacs Tutorial in Hebrew is available.

bug#8884: closed (Re: bug#8884: wide-int crash)

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> --- etc/NEWS	2011-10-18 06:52:32 +0000
> +++ etc/NEWS	2011-10-18 15:34:06 +0000
> @@ -51,6 +51,8 @@ This is not a new feature; only the conf
>  ---
>  ** There is a new configure option --with-wide-int.
>  With it, Emacs integers typically have 62 bits, even on 32-bit machines.
> +On 32-bit hosts, this raises the limit on buffer sizes from 512 MiB to
> +a bit less than 4 GiB, with the exact limit set by the operating system.

"a bit less than 4 GB" is an exaggeration, "almost 2GB" is much closer
to the truth.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#8884: closed (Re: bug#8884: wide-int crash)

[Paul Eggert]

On 10/18/11 08:45, Andreas Schwab wrote:
> "a bit less than 4 GB" is an exaggeration, "almost 2GB" is much closer
> to the truth.

Whoops!  Thanks for catching that; I forgot about the ptrdiff_t limit.
I installed this instead:

--- etc/NEWS	2011-10-18 06:52:32 +0000
+++ etc/NEWS	2011-10-18 15:55:20 +0000
@@ -51,6 +51,8 @@
 ---
 ** There is a new configure option --with-wide-int.
 With it, Emacs integers typically have 62 bits, even on 32-bit machines.
+On 32-bit hosts, this raises the limit on buffer sizes from about 512 MiB
+to about 2 GiB.
 
 ---
 ** New translation of the Emacs Tutorial in Hebrew is available.

bug#8884: closed (Re: bug#8884: wide-int crash)

[Peter Dyballa]

Am 18.10.2011 um 17:45 schrieb Andreas Schwab:

> "a bit less than 4 GB" is an exaggeration, "almost 2GB" is much closer
> to the truth.

With Mac OS X 10.5.8 and a PowerPC 7447A CPU GNU Emacs 24.0.90 launched with -Q can open (tar) files up to 2,047 MB. Ls reports that 2,147,450,880 is OK and 2,148,024,320 is too much (2,048 MB). 2**31 is 2,147,483,648.

The TAR file was created by an i386 tar (GNU tar) 1.26 on an external FireWire disk I attached either on my fast 64-bit intel based Mac or on my slow 32-bit G4, where I performed the tests. In dired-mode I opened the directory and read the file in with v. The MB numbers where reported by GNU Emacs' routine which asked me to confirm that I really wanted to open such a large file.

While I opened too small files I killed the tar-mode buffer, ejected the disk, created a larger file, attached the disk again, and opened from freshened dired listing (in my PATH only a gls exists which accepts --dired) the tar file with v. This produced failures like these:

	emacs(84226) malloc: *** mmap(size=2113650688) failed (error code=12)
	*** error: can't allocate region
	*** set a breakpoint in malloc_error_break to debug
	
	emacs(84357) malloc: *** mmap(size=2126585856) failed (error code=12)
	*** error: can't allocate region
	*** set a breakpoint in malloc_error_break to debug
	
	emacs(84410) malloc: *** mmap(size=2128408576) failed (error code=12)
	*** error: can't allocate region
	*** set a breakpoint in malloc_error_break to debug
	
	emacs(84517) malloc: *** mmap(size=2136588288) failed (error code=12)
	*** error: can't allocate region
	*** set a breakpoint in malloc_error_break to debug
	
	emacs(84576) malloc: *** mmap(size=2140327936) failed (error code=12)
	*** error: can't allocate region
	*** set a breakpoint in malloc_error_break to debug

Is this expected behaviour? I would have thought GNU Emacs could clear all for the file allocated memory... There were still 8 or 9 GB disk space available for swap/vm to grow, so this could not have caused the failures. (I killed GNU Emacs and launched a new one for each new test.) Or should I send a new bug report?

(I can, some rather rainy time, repeat the tests on Mac OS X 10.4.11...)

--
Greetings

  Pete

Work is the curse of the drinking class.
				– Oscar Wilde

bug#8884: closed (Re: bug#8884: wide-int crash)

[Paul Eggert]

On 10/18/11 14:41, Peter Dyballa wrote:

> With Mac OS X 10.5.8 and a PowerPC 7447A CPU GNU Emacs 24.0.90
> launched with -Q can open (tar) files up to 2,047 MB. Ls reports
> that 2,147,450,880 is OK and 2,148,024,320 is too much (2,048
> MB). 2**31 is 2,147,483,648.

Yes, that sounds about right.  There's a ~2**31 byte buffer size
limit due to PTRDIFF_MAX, and Emacs itself enforces this limit.

> 	emacs(84226) malloc: *** mmap(size=2113650688) failed (error code=12)
> 	*** error: can't allocate region
> 	*** set a breakpoint in malloc_error_break to debug

I'm not familiar with this error format, but I expect it's some
Mac OS X thing.  Possibly Mac OS malloc stops working around 2 GiB.
Or perhaps your applications have a system-imposed limit
(try the shell command "ulimit -a").

> I would have thought GNU Emacs could clear all for the file
> allocated memory... There were still 8 or 9 GB disk space available
> for swap/vm to grow,

A single 32-bit process can't possibly allocate more than 4 GiB of
address space, so as long as there's plenty of swap space (which
there appears to be), it's an address-space limit you're running
into, not a swap-space limit.

bug#8884: closed (Re: bug#8884: wide-int crash)

[Peter Dyballa]

Am 19.10.2011 um 03:47 schrieb Paul Eggert:

> I'm not familiar with this error format, but I expect it's some
> Mac OS X thing.

Likely. Google seems to indicate this.

> Or perhaps your applications have a system-imposed limit
> (try the shell command "ulimit -a").

Not likely:

	core file size          (blocks, -c) unlimited
	data seg size           (kbytes, -d) unlimited
	file size               (blocks, -f) unlimited
	max locked memory       (kbytes, -l) unlimited
	max memory size         (kbytes, -m) unlimited
	open files                      (-n) 256
	pipe size            (512 bytes, -p) 1
	stack size              (kbytes, -s) 8192
	cpu time               (seconds, -t) unlimited
	max user processes              (-u) 266
	virtual memory          (kbytes, -v) unlimited


> A single 32-bit process can't possibly allocate more than 4 GiB of
> address space, so as long as there's plenty of swap space (which
> there appears to be), it's an address-space limit you're running
> into, not a swap-space limit.


It seems the limit starts to be turned on around 3 GiB... From here it  
succeeds or does not succeed to load a file greater than 10 MiB. It  
even plays a role whether I load the 1 GiB or the 2 GiB file first...

--
Greetings

   Pete

A common mistake that people make when trying to design something  
completely foolproof is to underestimate the ingenuity of complete  
fools.
	- Douglas Adams, »Mostly Harmless«

bug#8884: closed (Re: bug#8884: wide-int crash)

[Stefan Monnier]

> While I opened too small files I killed the tar-mode buffer, ejected the
> disk, created a larger file, attached the disk again, and opened from
> freshened dired listing (in my PATH only a gls exists which accepts --dired)
> the tar file with v. This produced failures like these:

> 	emacs(84226) malloc: *** mmap(size=2113650688) failed (error code=12)
> 	*** error: can't allocate region
> 	*** set a breakpoint in malloc_error_break to debug
	
It may simply be due to fragmentation: while there may still be more
than 2GB of unused memory, there may not be a single contiguous unused
chunk of 2GB.


        Stefan

bug#8884: closed (Re: bug#8884: wide-int crash)

[Peter Dyballa]

Am 19.10.2011 um 05:43 schrieb Stefan Monnier:

> It may simply be due to fragmentation: while there may still be more
> than 2GB of unused memory, there may not be a single contiguous unused
> chunk of 2GB.

Not likely. VM is on a dedicated disk volume of almost 12 GB. It  
happened that I used (faulty) CPAN/Perl related tools that allocated 7  
or 8 GB until I could kill them.

--
Greetings

   Pete

One cannot live by television, video games, top ten CDs, and dumb  
movies alone.
				– Amiri Baraka, 1999

bug#8884: closed (Re: bug#8884: wide-int crash)

[Stefan Monnier]

>> It may simply be due to fragmentation: while there may still be more
>> than 2GB of unused memory, there may not be a single contiguous unused
>> chunk of 2GB.
> Not likely.

Why so?

> VM is on a dedicated disk volume of almost 12 GB.

Irrelevant.  The fragmentation is in the process's address space
(limited at most to 4GB).  Think of the following scenario:

1- start Emacs:
   30MB used, + (4096 - 30)MB free.
2- load 2GB file:
   30MB + 2048MB + 2018MB free
3- look at the file, causing some allocation:
   30MB + 2048MB + 1MB + 2017MB free
4- delete the large buffer
   30MB + 2048MB free + 1MB + 2017MB free
5- run a few minor commands, e.g. visit some small files.
   32MB + 2046MB free + 1MB + 2017MB free
6- try to load 2G file again
   Oops there's no hole large enough to put the 2048MB file.


-- Stefan

bug#8884: closed (Re: bug#8884: wide-int crash)

[Peter Dyballa]

Am 19.10.2011 um 16:44 schrieb Stefan Monnier:

> Think of the following scenario:


Why did it work better, without that failure, when I killed previous  
GNU Emacs from last test and launched a new one for new test? Then the  
OS would have allocated VM in a similar manner. But I never hit that  
situation.

I'm trying to find a means to observe whether GNU Emacs really frees  
the once allocated memory...

--
Greetings

   Pete

Bake pizza not war!

bug#8884: closed (Re: bug#8884: wide-int crash)

[Stefan Monnier]

>> Think of the following scenario:
> Why did it work better, without that failure, when I killed previous GNU
> Emacs from last test and launched a new one for new test? Then the OS would
> have allocated VM in a similar manner. But I never hit that situation.

No, again the VM is irrelevant.  When a start a new Emacs process, its
address space is completely clean (and in "30MB used and 4066MB free",
even if you only have 100MB of space left in your swap space).

You really need to read up on how virtual memory and process memory works.


        Stefan

bug#8884: closed (Re: bug#8884: wide-int crash)

[Peter Dyballa]

Am 19.10.2011 um 20:00 schrieb Stefan Monnier:

> You really need to read up on how virtual memory and process memory  
> works.

Those in GNU Emacs? Experiments seem to indicate that its behaviour is  
not so easy to predict...

In Mac OS X the swap files are created in a normal formatted file  
system (HFS+ or HFSX). The first file has 64 MiB, the second as well,  
the third 128 MiB, the fourth one 256 MiB, the fifth one 512 MiB, and  
all the others following have each 1 GiB size. It always succeeds to  
allocate a big chunk of disk space.

--
Greetings

   Pete

Don't force it; get a larger hammer.
				– Anthony's Law of Force

bug#8884: closed (Re: bug#8884: wide-int crash)

[Stefan Monnier]

>> You really need to read up on how virtual memory and process memory works.
> Those in GNU Emacs?

No, the general concepts as they're used in Unixy (and Windows)
operating systems.


        Stefan

bug#8884: wide-int crash [was Re: [PATCH updated] Support for filesystem watching (inotify)]

[Paul Eggert]

On 07/06/11 12:14, Glenn Morris wrote:

> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8884

Thanks for the heads-up.  Although the nearby code has a portability bug
regardless of whether --with-wide-int is used, --with-wide-int
is more likely to tickle the bug.  The bug could well explain the
symptoms Peter observed.

I committed the following patch into the trunk.  Peter,
can you please check whether it solves your problem?  If not,
can you please compile with -g and without -O, and use GDB to report
the following values at the point of the crash: buffer_local_flags,
idx, offset, sizeof (struct buffer).  Thanks.

=== modified file 'src/ChangeLog'
--- src/ChangeLog	2011-07-05 09:51:56 +0000
+++ src/ChangeLog	2011-07-06 22:22:32 +0000
@@ -1,3 +1,15 @@
+2011-07-06  Paul Eggert  <eggert@cs.ucla.edu>
+
+	Remove unportable assumption about struct layout (Bug#8884).
+	* alloc.c (mark_buffer):
+	* buffer.c (reset_buffer_local_variables, Fbuffer_local_variables)
+	(clone_per_buffer_values): Don't assume that
+	sizeof (struct buffer) is a multiple of sizeof (Lisp_Object).
+	This isn't true in general, and it's particularly not true
+	if Emacs is configured with --with-wide-int.
+	* buffer.h (FIRST_FIELD_PER_BUFFER, LAST_FIELD_PER_BUFFER):
+	New macros, used in the buffer.c change.
+
 2011-07-05  Jan Djärv  <jan.h.d@swipnet.se>
 
 	* xsettings.c: Use both GConf and GSettings if both are available.

=== modified file 'src/alloc.c'
--- src/alloc.c	2011-06-24 21:25:22 +0000
+++ src/alloc.c	2011-07-06 22:22:32 +0000
@@ -5619,7 +5619,8 @@
   /* buffer-local Lisp variables start at `undo_list',
      tho only the ones from `name' on are GC'd normally.  */
   for (ptr = &buffer->BUFFER_INTERNAL_FIELD (name);
-       (char *)ptr < (char *)buffer + sizeof (struct buffer);
+       ptr <= &PER_BUFFER_VALUE (buffer,
+				 PER_BUFFER_VAR_OFFSET (LAST_FIELD_PER_BUFFER));
        ptr++)
     mark_object (*ptr);
 

=== modified file 'src/buffer.c'
--- src/buffer.c	2011-07-04 15:32:22 +0000
+++ src/buffer.c	2011-07-06 22:22:32 +0000
@@ -471,8 +471,8 @@
 
   /* buffer-local Lisp variables start at `undo_list',
      tho only the ones from `name' on are GC'd normally.  */
-  for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
-       offset < sizeof *to;
+  for (offset = PER_BUFFER_VAR_OFFSET (FIRST_FIELD_PER_BUFFER);
+       offset <= PER_BUFFER_VAR_OFFSET (LAST_FIELD_PER_BUFFER);
        offset += sizeof (Lisp_Object))
     {
       Lisp_Object obj;
@@ -830,8 +830,8 @@
 
   /* buffer-local Lisp variables start at `undo_list',
      tho only the ones from `name' on are GC'd normally.  */
-  for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
-       offset < sizeof *b;
+  for (offset = PER_BUFFER_VAR_OFFSET (FIRST_FIELD_PER_BUFFER);
+       offset <= PER_BUFFER_VAR_OFFSET (LAST_FIELD_PER_BUFFER);
        offset += sizeof (Lisp_Object))
     {
       int idx = PER_BUFFER_IDX (offset);
@@ -1055,8 +1055,8 @@
 
     /* buffer-local Lisp variables start at `undo_list',
        tho only the ones from `name' on are GC'd normally.  */
-    for (offset = PER_BUFFER_VAR_OFFSET (undo_list);
-	 offset < sizeof (struct buffer);
+    for (offset = PER_BUFFER_VAR_OFFSET (FIRST_FIELD_PER_BUFFER);
+	 offset <= PER_BUFFER_VAR_OFFSET (LAST_FIELD_PER_BUFFER);
 	 /* sizeof EMACS_INT == sizeof Lisp_Object */
 	 offset += (sizeof (EMACS_INT)))
       {

=== modified file 'src/buffer.h'
--- src/buffer.h	2011-06-21 21:32:10 +0000
+++ src/buffer.h	2011-07-06 21:53:56 +0000
@@ -612,6 +612,7 @@
   /* Everything from here down must be a Lisp_Object.  */
   /* buffer-local Lisp variables start at `undo_list',
      tho only the ones from `name' on are GC'd normally.  */
+  #define FIRST_FIELD_PER_BUFFER undo_list
 
   /* Changes in the buffer are recorded here for undo.
      t means don't record anything.
@@ -846,6 +847,9 @@
      t means to use hollow box cursor.
      See `cursor-type' for other values.  */
   Lisp_Object BUFFER_INTERNAL_FIELD (cursor_in_non_selected_windows);
+
+  /* This must be the last field in the above list.  */
+  #define LAST_FIELD_PER_BUFFER cursor_in_non_selected_windows
 };
 
 \f

Re: [PATCH updated] Support for filesystem watching (inotify)

[Stefan Monnier]

> I updated the patch to return a "file-watcher".  file-watch can now be
> called several times for the same file with different flags and
> callbacks.  This however required large changes to the patch.  So it
> would be great if someone could take a look at it again.  I added an
> automated test.  But it doesn't test the actually file-watching
> yet. This is a bit tricky to get right in a unit test.  But I'll add
> it.  The code requires some heavy testing.

Here are some comments, only based on a cursory read.  I do not know
anything about the inotify API and have not tried your code.

> --- /dev/null
> +++ b/lisp/filewatch.el
> @@ -0,0 +1,54 @@
> +;;; filewatch.el --- Elisp support for watching filesystem events.
> +
> +;; Copyright (C) 2011 Free Software Foundation, Inc.
> +
> +;; Author: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> +;; Keywords: files
> +
> +;; This file is part of GNU Emacs.
> +
> +;; GNU Emacs is free software: you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation, either version 3 of the License, or
> +;; (at your option) any later version.
> +
> +;; GNU Emacs is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
> +
> +;;; Commentary:
> +
> +;; This file contains the elisp part of the filewatch API.
> +
> +;;; Code:
> +
> +(eval-when-compile
> +  (require 'cl))
> +
> +(defun filewatch-event-p (event)
> +  "Check if EVENT is a filewatch event."
> +  (and (listp event)
> +       (eq (car event) 'filewatch-event)))
> +
> +;;;###autoload
> +(defun filewatch-handle-event (event)
> +  "Handle file system monitoring event.
> +If EVENT is a filewatch event then the callback is called.  If EVENT is
> +not a filewatch event then a `filewatch-error' is signaled."
> +  (interactive "e")
> +
> +  (unless (filewatch-event-p event)
> +    (signal 'filewatch-error (cons "Not a valid filewatch event" event)))
> +
> +  (loop for ev in (cdr event)
> +     unless (and (listp ev) (>= (length ev) 3))
> +     do (signal 'filewatch-error (cons "Not a valid filewatch event" event))
> +     do (funcall (cadr ev) (car ev) (caddr ev))))
> +
> +(provide 'filewatch)

Unless we expect many more functions in this file, we could move those
functions to subr.el (tho after removing the CL dependency since
subr.el can't use CL for bootstrapping reasons).

> +/* Assoc list of files being watched.
> + Format:
> + (id . (filename ((subid callback flags) (subid callbacks flags) ...))) */
> +static Lisp_Object watch_list;
> +
> +#define CADDR(x) Fcar (Fcdr (Fcdr (x)))
> +#define CADR(x) Fcar (Fcdr (x))

Most C code should use XCAR/XCDR rather than Fcr/Fcdr.

> +  while (!NILP (i))
> +    {
> +      Lisp_Object e = Fcar (events);

E.g. here, better test CONSP rather than !NILP, which lets you then use
XCAR/XCDR.

> +      while (!NILP (e))
> +        {
> +          if (EQ (Fcar (e), i))
> +            ret = Fcons (e, ret);
> +          events = Fcdr (events);
> +          e = Fcar (events);
> +        }
> +      aspects = Fcdr (aspects);
> +      i = Fcar (aspects);
> +    }

I'd have used an outside loop on events so that the inner loop on
aspects can use Fmemq.

> +  return ret;
> +}
> +
> +static Lisp_Object
> +inotifyevent_to_aspects (Lisp_Object name, struct inotify_event *ev)
> +{

It doesn't just return aspects but events (using a Lips_Object
representation), so the name is somewhat misleading.

> +  Lisp_Object events = Qnil;
> +  if (ev->mask & (IN_MODIFY|IN_CREATE) )
> +    events = Fcons (Fcons (Qmodify, name), events);
> +  if (ev->mask & IN_MOVE_SELF)
> +    events = Fcons (Fcons (Qmove, name), events);
> +  if (ev->mask & IN_MOVED_FROM)
> +    events = Fcons (Fcons (Qmove,
> +                           Fcons (Qfrom,
> +                                  Fcons (name,

The structure of those events is a bit odd.

It seems that each event is either of form (SYMBOL . FILENAME) where
FILENAME is the watched object, or of the form (SYMBOL from NAME . COOKIE)
where NAME is the file added/removed from the watched object (a
directory, presumably), but this object's FILENAME is not present.
Any particular reason for this inconsistency?

> +                                         make_number (ev->cookie)))),
> +                    events);
> +  if (ev->mask & IN_MOVED_TO)
> +    events = Fcons (Fcons (Qmove,
> +                           Fcons (Qto,
> +                                  Fcons (name,
> +                                         make_number (ev->cookie)))),

IIUC, these cookies are part of the patch to which Paul objected.
And IIUC their content has no importance, they're only checked for
equality, right?

So they don't have to be represented as numbers, but could be
represented by some other special object, as long as (eq cookie1
cookie2) returns the proper boolean value when comparing those
special objects.

I guess that several `ev' objects can have the same `ev->cookie'
value, right?

> +/* This callback is called when the FD is available FOR_READ.  The inotify
> +   events are read from FD and converted into input_events.  */
> +static void
> +inotify_callback (int fd, void *_, int for_read)
> +{

AFAICT, the void* argument will be a pointer to watch_list.
I think that either you should use it here, or you should pass NULL to
add_read_fd (rather than passing &watch_list).

> +      Lisp_Object watch_data = Fassoc (make_number (ev->wd), watch_list);
> +      if (!NILP (watch_data))
> +        {
> +          Lisp_Object name, events;
> +
> +          /* If a directory is watched name contains the name
> +             of the file that was changed.  */
> +          if (ev->len > 0)
> +            {
> +              size_t const len = strlen (ev->name);
> +              name = make_unibyte_string (ev->name, min (len, ev->len));
> +              name = DECODE_FILE (name);
> +            }
> +          else
> +            {
> +              name = CADR (watch_data);
> +            }
> +
> +          events = inotifyevent_to_aspects (name, ev);
> +
> +          if (!NILP (events))

Wouldn't it be a bug for events to be nil here (that would mean you
receive inotify events for files you do not know you're watching, or
something like that)?

> +            {
> +              Lisp_Object x = CADDR (watch_data);
> +              while (!NILP (x))
> +                {
> +                  Lisp_Object cell = Fcar (x);
> +                  Lisp_Object aspects = match_aspects (cell, events);
> +                  if (!NILP (aspects))
> +                    {
> +                      Lisp_Object id = list3 (Qfile_watch, make_number (ev->wd),
> +                                              Fcar (cell));

I don't think you need ev->wd in your file-watch objects.  You could use
Fcons (Qfile_watch, cell) instead, IIUC.  It would be good, because it
would let you use a SAVE_VALUE object for ev->wd (since it wouldn't
escape to Lisp any more), which would solve the problem of losing a few
bits of data in make_number.

> +
> +static uint32_t
> +symbol_to_inotifymask (Lisp_Object symb, int in_list)
> +{
> +  if (EQ (symb, Qmodify))
> +    return IN_MODIFY | IN_CREATE;
> +  else if (EQ (symb, Qmove))
> +    return IN_MOVE_SELF | IN_MOVE;
> +  else if (EQ (symb, Qattrib))
> +    return IN_ATTRIB;
> +  else if (EQ (symb, Qdelete))
> +    return IN_DELETE_SELF | IN_DELETE;
> +  else if (!in_list && EQ (symb, Qall_modify))
> +    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
> +      | IN_DELETE_SELF | IN_DELETE;
> +  else if (EQ (symb, Qaccess))
> +    return IN_ACCESS;
> +  else if (EQ (symb, Qclose_write))
> +    return IN_CLOSE_WRITE;
> +  else if (EQ (symb, Qclose_nowrite))
> +    return IN_CLOSE_NOWRITE;
> +  else if (EQ (symb, Qopen))
> +    return IN_OPEN;
> +  else if (!in_list && (EQ (symb, Qt) || EQ (symb, Qall)))
> +    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
> +      | IN_DELETE_SELF | IN_DELETE | IN_ACCESS | IN_CLOSE_WRITE
> +      | IN_CLOSE_NOWRITE | IN_OPEN;
> +  else
> +    Fsignal (Qunknown_aspect, Fcons (symb, Qnil));

Just make it an `error'.

> +static Lisp_Object
> +get_watch_data_by_file_name (Lisp_Object file_name)
> +{
> +  Lisp_Object cell, file_name_data;
> +  Lisp_Object x = watch_list;
> +  CHECK_STRING (file_name);
> +
> +  while (!NILP (x))
> +    {
> +      cell = Fcar (x);
> +      x = Fcdr (x);
> +      file_name_data = Fcdr (cell);
> +      if (!NILP (file_name_data) && STRINGP (Fcar (file_name_data))
> +          && !NILP (Fstring_equal (Fcar (file_name_data), file_name))
> +          && NUMBERP (Fcar (cell)))

Why do you need NUMBERP?
Shouldn't it be an "eassert (NUMBERP (...))" instead?

> +Watching a directory is not recursive.  CALLBACK receives the events as a list
> +with each list element being a list containing information about an event.  The
> +first element is a flag symbol.  If a directory is watched the second element is
> +the name of the file that changed.  If a file is moved from or to the directory
> +the second element is either 'from or 'to and the third element is the file
> +name.  A fourth element contains a numeric identifier (cookie) that can be used
> +to identify matching move operations if a file is moved inside the directory.

We typically try to avoid such "third element" style and use forms like:
"the argument takes the form (OP . FILENAME) or (OP DIRECTION NAME COOKIE)
where OP is blabl, etc...".  Also please say how many arguments are
passed to CALLBACK and what means each one of them.

> +  mask = aspect_to_inotifymask (aspect);
> +  data = get_watch_data_by_file_name (file_name);
> +  if (!NILP (data))
> +    mask |= IN_MASK_ADD;

Would it hurt to always add IN_MASK_ADD?

> +  decoded_file_name = ENCODE_FILE (file_name);
> +  watchdesc = inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), mask);

Are we sure that when data is nil, watchdesc is different from all other
watchdesc we currently have, and that when data is non-nil, watchdesc is
equal to the watchdesc stored in data?
If so, let's says so via asserts, and if not, we should check and decide
what to do.

IIUC inotify operates on inodes whereas our watch_list is indexed by
file names, so get_watch_data_by_file_name may think we're changing an
existing watcher but in reality it may operate on a new inode and hence
return a new watchdesc.  Inversely, two different filenames can refer to
the same inode, so I suspect that there are also cases where
get_watch_data_by_file_name thinks this is a new watcher but the
watchdesc returned will be one that already exists.

IOW, I think we should always say IN_MASK_ADD (just in case) and we
should not use get_watch_data_by_file_name before calling
inotify_add_watch, but instead use get_watch_data_by_watchdesc afterwards.

> +  return list3 (Qfile_watch, make_number (watchdesc), Fcar (info));

See comment earlier about eliding watchdesc and including `info' instead.

> +  if (inotifyfd == uninitialized)
> +    return Qnil;

Shouldn't this signal an error instead?


        Stefan

Re: [PATCH] Added basic file system watching support.

[Rüdiger Sonderfeld]

Hello,
this is an updated version of my "file system watching" patch
(full thread: http://thread.gmane.org/gmane.emacs.devel/140158 )

I don't think the patch will be ready for 24.3. It is still not complete and
needs some heavy testing.

I addressed some of the issues mentioned by Stefan Monnier here
http://article.gmane.org/gmane.emacs.devel/141741

> It seems that each event is either of form (SYMBOL . FILENAME) where
> FILENAME is the watched object, or of the form (SYMBOL from NAME . COOKIE)
> where NAME is the file added/removed from the watched object (a
> directory, presumably), but this object's FILENAME is not present.
> Any particular reason for this inconsistency?

FILENAME is not always the watched object.  If a directory is watched then
it is the file name of the file inside the directory.  This should really not
be used as identification for the watched object!  That's why CALLBACK
also gets the ID.

> IIUC, these cookies are part of the patch to which Paul objected.
> And IIUC their content has no importance, they're only checked for
> equality, right?
>
> So they don't have to be represented as numbers, but could be
> represented by some other special object, as long as (eq cookie1
> cookie2) returns the proper boolean value when comparing those
> special objects.
>
> I guess that several `ev' objects can have the same `ev->cookie'
> value, right?

Yes exactly.  The value is unimportant as long as it is unique to an event
and can be compared with eq.  What would you propose as an alternative
object here?

> Wouldn't it be a bug for events to be nil here (that would mean you
> receive inotify events for files you do not know you're watching, or
> something like that)?

The problem is that even after removal of a watch descriptor there can
still be issues in the queue.  Therefore I think it is best to ignore any
events for files we do not watch.  There could also be event types we do not
know about when inotify gets expanded.  That's why I think unknown
event types should be ignored instead of reporting an error.

> I don't think you need ev->wd in your file-watch objects.  You could use
> Fcons (Qfile_watch, cell) instead, IIUC.  It would be good, because it
> would let you use a SAVE_VALUE object for ev->wd (since it wouldn't
> escape to Lisp any more), which would solve the problem of losing a few
> bits of data in make_number.

I don't need to make the watchdescriptor (wd) available to elisp.  But I need
a way to get the descriptor from the watch handle.  How could this be
implemented with SAVE_VALUE?

> Shouldn't this signal an error instead?

This could be the result of calling file-unwatch too often.  Currently
the behaviour is to return nil in that case.  But this could also be
seen as an error of course.

Regards,
Rüdiger

-- >8 --

The current patch only works with inotify (Linux).  It adds two elisp
functions file-watch and file-unwatch.

Example

(let ((fh (file-watch "foo" t #'(lambda (id event) (message "%s %s" id
event)))))
  (delete-file "foo/bar")
  (file-unwatch fh))

(Expects a directory foo containing a file bar)

This watches for any kind of changes in a directory foo and then
deletes a file foo/bar and finally removes the watch.

Signed-off-by: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
---
 configure.ac                      |  14 ++
 lisp/subr.el                      |  23 ++
 src/Makefile.in                   |   2 +-
 src/emacs.c                       |   4 +
 src/filewatch.c                   | 457 ++++++++++++++++++++++++++++++++++++++
 src/keyboard.c                    |  28 +++
 src/lisp.h                        |   5 +
 src/termhooks.h                   |   5 +
 test/automated/filewatch-tests.el |  50 +++++
 9 files changed, 587 insertions(+), 1 deletion(-)
 create mode 100644 src/filewatch.c
 create mode 100644 test/automated/filewatch-tests.el

diff --git a/configure.ac b/configure.ac
index 5a3aea7..8b8054b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -184,6 +184,7 @@ OPTION_DEFAULT_ON([gconf],[don't compile with GConf support])
 OPTION_DEFAULT_ON([gsettings],[don't compile with GSettings support])
 OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
 OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
+OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) support])
 
 ## For the times when you want to build Emacs but don't have
 ## a suitable makeinfo, and can live without the manuals.
@@ -2101,6 +2102,19 @@ fi
 AC_SUBST(LIBGNUTLS_LIBS)
 AC_SUBST(LIBGNUTLS_CFLAGS)
 
+dnl inotify is only available on GNU/Linux.
+HAVE_INOTIFY=no
+if test "${with_inotify}" = "yes"; then
+   AC_CHECK_HEADERS(sys/inotify.h)
+   if test "$ac_cv_header_sys_inotify_h" = yes ; then
+     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, HAVE_INOTIFY=yes)
+   fi
+fi
+if test "${HAVE_INOTIFY}" = "yes"; then
+   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
+   AC_DEFINE(HAVE_FILEWATCH, [1], [Define to 1 to support filewatch])
+fi
+
 dnl Do not put whitespace before the #include statements below.
 dnl Older compilers (eg sunos4 cc) choke on it.
 HAVE_XAW3D=no
diff --git a/lisp/subr.el b/lisp/subr.el
index 8dfe78d..f47d4fb 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -4159,6 +4159,29 @@ convenience wrapper around `make-progress-reporter' and friends.
        nil ,@(cdr (cdr spec)))))
 
 \f
+;;;; Support for watching filesystem events.
+
+(defun filewatch-event-p (event)
+  "Check if EVENT is a filewatch event."
+  (and (listp event)
+       (eq (car event) 'filewatch-event)))
+
+;;;###autoload
+(defun filewatch-handle-event (event)
+  "Handle file system monitoring event.
+If EVENT is a filewatch event then the callback is called.  If EVENT is
+not a filewatch event then a `filewatch-error' is signaled."
+  (interactive "e")
+
+  (unless (filewatch-event-p event)
+    (signal 'filewatch-error (cons "Not a valid filewatch event" event)))
+
+  (dolist (ev (cdr event))
+    (unless (and (listp ev) (>= (length ev) 3))
+      (signal 'filewatch-error (cons "Not a valid filewatch event" event)))
+    (funcall (car (cdr ev)) (car ev) (car (cdr (cdr ev))))))
+
+\f
 ;;;; Comparing version strings.
 
 (defconst version-separator "."
diff --git a/src/Makefile.in b/src/Makefile.in
index e43f83e..1f17830 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -338,7 +338,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o $(XMENU_OBJ) window.o \
 	syntax.o $(UNEXEC_OBJ) bytecode.o \
 	process.o gnutls.o callproc.o \
 	region-cache.o sound.o atimer.o \
-	doprnt.o intervals.o textprop.o composite.o xml.o \
+	doprnt.o intervals.o textprop.o composite.o xml.o filewatch.o \
 	profiler.o \
 	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ) \
 	$(WINDOW_SYSTEM_OBJ)
diff --git a/src/emacs.c b/src/emacs.c
index 05affee..7c306ee 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -1411,6 +1411,10 @@ Using an Emacs configured with --with-x-toolkit=lucid does not have this problem
       syms_of_gnutls ();
 #endif
 
+#ifdef HAVE_FILEWATCH
+      syms_of_filewatch ();
+#endif /* HAVE_FILEWATCH */
+
 #ifdef HAVE_DBUS
       syms_of_dbusbind ();
 #endif /* HAVE_DBUS */
diff --git a/src/filewatch.c b/src/filewatch.c
new file mode 100644
index 0000000..c29e8b1
--- /dev/null
+++ b/src/filewatch.c
@@ -0,0 +1,457 @@
+/* Watching file system changes.
+
+Copyright (C) 2011
+  Free Software Foundation, Inc.
+
+This file is part of GNU Emacs.
+
+GNU Emacs is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+GNU Emacs is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <config.h>
+
+#ifdef HAVE_FILEWATCH
+#include "lisp.h"
+#include "coding.h"
+#include "process.h"
+#include "keyboard.h"
+#include "character.h"
+#include "frame.h" /* Required for termhooks.h.  */
+#include "termhooks.h"
+
+static Lisp_Object Qmodify, Qmove, Qattrib, Qdelete, Qfrom, Qto, Qall_modify;
+static Lisp_Object Qfile_watch;
+static Lisp_Object Qaccess, Qclose_write, Qclose_nowrite, Qopen, Qall;
+
+#ifdef HAVE_INOTIFY
+#include <sys/inotify.h>
+#include <sys/ioctl.h>
+
+enum { uninitialized = -100 };
+/* File handle for inotify.  */
+static int inotifyfd = uninitialized;
+
+/* Assoc list of files being watched.
+ Format:
+ (id . (filename ((subid callback flags) (subid callbacks flags) ...))) */
+static Lisp_Object watch_list;
+
+static Lisp_Object
+get_watch_data_by_watchdesc (int watchdesc)
+{
+  return Fassq (make_number (watchdesc), watch_list);
+}
+
+#define CADDR(x) XCAR (XCDR (XCDR (x)))
+#define CADR(x) XCAR (XCDR (x))
+
+static Lisp_Object
+append2 (Lisp_Object s1, Lisp_Object s2)
+{
+  Lisp_Object args[2];
+  args[0] = s1;
+  args[1] = s2;
+  return Fappend (2, args);
+}
+
+/* Returns a list with all the elements from EVENTS the `watch_list' CELL
+   is interested in.  */
+static Lisp_Object
+match_aspects (Lisp_Object cell, Lisp_Object events)
+{
+  Lisp_Object e;
+  Lisp_Object ret = Qnil;
+  Lisp_Object aspects = CADDR (cell);
+  if (EQ (aspects, Qt) || EQ (aspects, Qall))
+    return events;
+  else if (EQ (aspects, Qall_modify))
+    aspects = list4 (Qmodify, Qmove, Qattrib, Qdelete);
+  else if (! CONSP (aspects))
+    aspects = list1 (aspects);
+
+  while (CONSP (events))
+    {
+      e = XCAR (events);
+      if (! NILP (Fmemq (XCAR (e), aspects)))
+        ret = Fcons (e, ret);
+      events = XCDR (events);
+    }
+  return ret;
+}
+
+static Lisp_Object
+inotifyevent_to_events (Lisp_Object name, struct inotify_event *ev)
+{
+  Lisp_Object events = Qnil;
+  if (ev->mask & (IN_MODIFY|IN_CREATE) )
+    events = Fcons (Fcons (Qmodify, name), events);
+  if (ev->mask & IN_MOVE_SELF)
+    events = Fcons (Fcons (Qmove, name), events);
+  if (ev->mask & IN_MOVED_FROM)
+    events = Fcons (Fcons (Qmove,
+                           Fcons (Qfrom,
+                                  Fcons (name,
+                                         make_number (ev->cookie)))),
+                    events);
+  if (ev->mask & IN_MOVED_TO)
+    events = Fcons (Fcons (Qmove,
+                           Fcons (Qto,
+                                  Fcons (name,
+                                         make_number (ev->cookie)))),
+                    events);
+  if (ev->mask & IN_ATTRIB)
+    events = Fcons (Fcons (Qattrib, name), events);
+  if (ev->mask & (IN_DELETE|IN_DELETE_SELF) )
+    events = Fcons (Fcons (Qdelete, name), events);
+  if (ev->mask & IN_ACCESS)
+    events = Fcons (Fcons (Qaccess, name), events);
+  if (ev->mask & IN_CLOSE_WRITE)
+    events = Fcons (Fcons (Qclose_write, name), events);
+  if (ev->mask & IN_CLOSE_NOWRITE)
+    events = Fcons (Fcons (Qclose_nowrite, name), events);
+  if (ev->mask & IN_OPEN)
+    events = Fcons (Fcons (Qopen, name), events);
+  return events;
+}
+
+/* This callback is called when the FD is available for read.  The inotify
+   events are read from FD and converted into input_events.  */
+static void
+inotify_callback (int fd, void *_)
+{
+  struct input_event event;
+  int to_read;
+  char *buffer;
+  ssize_t n;
+  size_t i;
+
+  to_read = 0;
+  if (ioctl (fd, FIONREAD,  &to_read) == -1)
+    report_file_error ("Error while trying to retrieve file system events",
+                       Qnil);
+  buffer = xmalloc (to_read);
+  n = read (fd, buffer, to_read);
+  if (n < 0)
+    {
+      xfree (buffer);
+      report_file_error ("Error while trying to read file system events",
+                         Qnil);
+    }
+
+  EVENT_INIT (event);
+  event.kind = FILEWATCH_EVENT;
+  event.arg = Qnil;
+
+  i = 0;
+  while (i < (size_t)n)
+    {
+      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
+
+      Lisp_Object watch_data = get_watch_data_by_watchdesc (ev->wd);
+      if (!NILP (watch_data))
+        {
+          Lisp_Object name, events;
+
+          /* If a directory is watched name contains the name
+             of the file that was changed.  */
+          if (ev->len > 0)
+            {
+              size_t const len = strlen (ev->name);
+              name = make_unibyte_string (ev->name, min (len, ev->len));
+              name = DECODE_FILE (name);
+            }
+          else
+            {
+              name = CADR (watch_data);
+            }
+
+          events = inotifyevent_to_events (name, ev);
+
+          if (!NILP (events))
+            {
+              Lisp_Object x = CADDR (watch_data);
+              while (CONSP (x))
+                {
+                  Lisp_Object cell = XCAR (x);
+                  Lisp_Object aspects = match_aspects (cell, events);
+                  if (!NILP (aspects))
+                    {
+                      Lisp_Object id = list3 (Qfile_watch, make_number (ev->wd),
+                                              XCAR (cell));
+                      Lisp_Object event_info = list1 (list3 (id, CADR (cell),
+                                                             aspects));
+
+                      if (NILP (event.arg))
+                        event.arg = event_info;
+                      else
+                        event.arg = append2 (event.arg, event_info);
+                    }
+                  x = XCDR (x);
+                }
+            }
+
+          if (ev->mask & IN_IGNORED)
+            {
+              /* Event was removed automatically: Drop it from data list.  */
+              add_to_log ("File-watch: \"%s\" will be ignored", name, Qnil);
+              watch_list = Fdelete (watch_data, watch_list);
+            }
+          if (ev->mask & IN_Q_OVERFLOW)
+            add_to_log ("File watch: Inotify Queue Overflow!", Qnil, Qnil);
+        }
+
+      i += sizeof (*ev) + ev->len;
+    }
+
+  if (!NILP (event.arg))
+    kbd_buffer_store_event (&event);
+
+  xfree (buffer);
+}
+
+static uint32_t
+symbol_to_inotifymask (Lisp_Object symb, int in_list)
+{
+  if (EQ (symb, Qmodify))
+    return IN_MODIFY | IN_CREATE;
+  else if (EQ (symb, Qmove))
+    return IN_MOVE_SELF | IN_MOVE;
+  else if (EQ (symb, Qattrib))
+    return IN_ATTRIB;
+  else if (EQ (symb, Qdelete))
+    return IN_DELETE_SELF | IN_DELETE;
+  else if (!in_list && EQ (symb, Qall_modify))
+    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
+      | IN_DELETE_SELF | IN_DELETE;
+  else if (EQ (symb, Qaccess))
+    return IN_ACCESS;
+  else if (EQ (symb, Qclose_write))
+    return IN_CLOSE_WRITE;
+  else if (EQ (symb, Qclose_nowrite))
+    return IN_CLOSE_NOWRITE;
+  else if (EQ (symb, Qopen))
+    return IN_OPEN;
+  else if (!in_list && (EQ (symb, Qt) || EQ (symb, Qall)))
+    return IN_MODIFY | IN_CREATE | IN_MOVE_SELF | IN_MOVE | IN_ATTRIB
+      | IN_DELETE_SELF | IN_DELETE | IN_ACCESS | IN_CLOSE_WRITE
+      | IN_CLOSE_NOWRITE | IN_OPEN;
+  else
+    signal_error ("Unknown aspect", symb);
+}
+
+static uint32_t
+aspect_to_inotifymask (Lisp_Object aspect)
+{
+  if (CONSP (aspect))
+    {
+      Lisp_Object x = aspect;
+      uint32_t mask = 0;
+      while (CONSP (x))
+        {
+          mask |= symbol_to_inotifymask (XCAR (x), 1);
+          x = XCDR (x);
+        }
+      return mask;
+    }
+  else
+    return symbol_to_inotifymask (aspect, 0);
+}
+
+DEFUN ("file-watch", Ffile_watch, Sfile_watch, 3, 3, 0,
+       doc: /* Arrange to call FUNC if ASPECT of FILENAME changes.
+
+ASPECT might be one of the following symbols or a list of those symbols (except
+for all-modify and t):
+
+modify -- notify when a file is modified or created.
+
+move -- notify when a file/directory is moved.
+
+attrib -- notify when attributes change.
+
+delete -- notify when a file/directory is deleted.
+
+all-modify -- notify for all of the above (all modifying changes).
+
+access -- notify when file was accessed/read.
+
+close-write -- notify when a file opened for writing was closed.
+
+close-nowrite -- notify when a file not opened for writing was closed.
+
+open -- notify when a file was opened.
+
+t -- notify for all of the above.
+
+Watching a directory is not recursive.  CALLBACK gets passed two
+arguments ID and EVENT.  The ID argument is the id returned by
+file-watch.  The EVENT argument is a list of events taking the form
+(OP . FILENAME) or (OP DIRECTION NAME COOKIE).  OP is the operation
+that caused the event and FILENAME is the name of the file the
+operation applied to.  If the even belongs to a move operation inside
+a directory then the (OP DIRECTION NAME COOKIE) form is used and
+DIRECTION is either 'from or 'to indicating the direction of the move
+operation.  NAME is the corresponding file name.  COOKIE is an
+unspecified object that can be compared using `eq' to identify the
+matching from and to move operation.
+
+Example:
+(file-watch "foo" t #'(lambda (id event) (message "%s %s" id event)))
+
+Use `file-unwatch' to stop watching.
+
+usage: (file-watch FILENAME ASPECT CALLBACK) */)
+  (Lisp_Object file_name, Lisp_Object aspect, Lisp_Object callback)
+{
+  static int intern_counter = 0; /* assign local ids */
+  uint32_t mask;
+  int watchdesc;
+  Lisp_Object decoded_file_name;
+  Lisp_Object data;
+  Lisp_Object info;
+
+  CHECK_STRING (file_name);
+
+  if (inotifyfd == uninitialized)
+    {
+      inotifyfd = inotify_init1 (IN_NONBLOCK|IN_CLOEXEC);
+      if (inotifyfd == -1)
+        {
+          inotifyfd = uninitialized;
+          report_file_error ("File watching feature (inotify) is not available",
+                             Qnil);
+        }
+      watch_list = Qnil;
+      add_read_fd (inotifyfd, &inotify_callback, NULL);
+    }
+
+  mask = aspect_to_inotifymask (aspect) | IN_MASK_ADD;
+  decoded_file_name = ENCODE_FILE (file_name);
+  watchdesc = inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), mask);
+  if (watchdesc == -1)
+    report_file_error ("Could not watch file", Fcons (file_name, Qnil));
+
+  info = list3 (make_number (intern_counter), callback, aspect);
+  ++intern_counter;
+
+  data = get_watch_data_by_watchdesc (watchdesc);
+  if (CONSP (data))
+    XSETCDR (XCDR (data), Fcons (append2 (CADDR (data), Fcons (info, Qnil)),
+                                 Qnil));
+  else
+    watch_list = Fcons (list3 (make_number (watchdesc),
+                               file_name, Fcons (info, Qnil)),
+                        watch_list);
+
+  return list3 (Qfile_watch, make_number (watchdesc), XCAR (info));
+}
+
+static int
+file_watch_objectp (Lisp_Object obj)
+{
+  return CONSP (obj) && XINT (Flength (obj)) == 3
+    && EQ (XCAR (obj), Qfile_watch);
+}
+
+DEFUN ("file-unwatch", Ffile_unwatch, Sfile_unwatch, 1, 1, 0,
+       doc: /* Stop watching a file or directory.  */)
+  (Lisp_Object watch_object)
+{
+  Lisp_Object watch_data;
+  Lisp_Object info;
+
+  if (!file_watch_objectp (watch_object))
+    wrong_type_argument (Qfile_watch, watch_object);
+
+  if (inotifyfd == uninitialized)
+    return Qnil;
+
+  watch_data = Fassoc ( CADR (watch_object), watch_list );
+  if (NILP (watch_data))
+    return Qnil;
+
+  info = Fassoc (CADDR (watch_object), CADDR (watch_data));
+  if (NILP (info))
+    return Qnil;
+
+  XSETCDR (XCDR (watch_data), Fcons (Fdelete (info, CADDR (watch_data)), Qnil));
+
+  if (NILP (CADDR (watch_data)))
+    {
+      int const magicno = XINT (CADR (watch_object));
+      watch_list = Fdelete (watch_data, watch_list);
+
+      if (inotify_rm_watch (inotifyfd, magicno) == -1)
+        report_file_error ("Could not unwatch file",
+                           Fcons (XCAR (watch_data), Qnil));
+
+      /* Cleanup if watch_list is empty.  */
+      if (NILP (watch_list))
+        {
+          close (inotifyfd);
+          delete_read_fd (inotifyfd);
+          inotifyfd = uninitialized;
+        }
+    }
+  else
+    {
+      Lisp_Object decoded_file_name = ENCODE_FILE (CADR (watch_data));
+      Lisp_Object x = CADDR (watch_data);
+      uint32_t mask = 0;
+      while (CONSP (x))
+        {
+          Lisp_Object cell = XCAR (x);
+          mask |= aspect_to_inotifymask (CADDR (cell));
+          x = XCDR (x);
+        }
+      /* Reset watch mask */
+      if (inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), mask) == -1)
+        report_file_error ("Could not unwatch file",
+                           Fcons (XCAR (watch_data), Qnil));
+    }
+
+  return Qt;
+}
+
+#else /* HAVE_INOTIFY */
+#error "Filewatch defined but no watch mechanism (inotify) available"
+#endif /* HAVE_INOTIFY */
+
+void
+syms_of_filewatch (void)
+{
+  DEFSYM (Qmodify, "modify");
+  DEFSYM (Qmove, "move");
+  DEFSYM (Qattrib, "attrib");
+  DEFSYM (Qdelete, "delete");
+  DEFSYM (Qfrom, "from");
+  DEFSYM (Qto, "to");
+  DEFSYM (Qall_modify, "all-modify");
+
+  DEFSYM (Qaccess, "access");
+  DEFSYM (Qclose_write, "close-write");
+  DEFSYM (Qclose_nowrite, "close-nowrite");
+  DEFSYM (Qopen, "open");
+  DEFSYM (Qall, "all");
+
+  DEFSYM (Qfile_watch, "file-watch");
+
+  defsubr (&Sfile_watch);
+  defsubr (&Sfile_unwatch);
+
+  staticpro (&watch_list);
+
+  Fprovide (intern_c_string ("filewatch"), Qnil);
+}
+
+#endif /* HAVE_FILEWATCH */
diff --git a/src/keyboard.c b/src/keyboard.c
index f3d7df5..f694b13 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -321,6 +321,9 @@ static Lisp_Object Qsave_session;
 #ifdef HAVE_DBUS
 static Lisp_Object Qdbus_event;
 #endif
+#ifdef HAVE_FILEWATCH
+static Lisp_Object Qfilewatch_event;
+#endif /* HAVE_FILEWATCH */
 static Lisp_Object Qconfig_changed_event;
 
 /* Lisp_Object Qmouse_movement; - also an event header */
@@ -4021,6 +4024,13 @@ kbd_buffer_get_event (KBOARD **kbp,
 	  kbd_fetch_ptr = event + 1;
 	}
 #endif
+#ifdef HAVE_FILEWATCH
+      else if (event->kind == FILEWATCH_EVENT)
+        {
+          obj = make_lispy_event (event);
+          kbd_fetch_ptr = event + 1;
+        }
+#endif
       else if (event->kind == CONFIG_CHANGED_EVENT)
 	{
 	  obj = make_lispy_event (event);
@@ -5938,6 +5948,13 @@ make_lispy_event (struct input_event *event)
       }
 #endif /* HAVE_DBUS */
 
+#ifdef HAVE_FILEWATCH
+    case FILEWATCH_EVENT:
+      {
+        return Fcons (Qfilewatch_event, event->arg);
+      }
+#endif /* HAVE_FILEWATCH */
+
     case CONFIG_CHANGED_EVENT:
 	return Fcons (Qconfig_changed_event,
                       Fcons (event->arg,
@@ -11408,6 +11425,10 @@ syms_of_keyboard (void)
   DEFSYM (Qdbus_event, "dbus-event");
 #endif
 
+#ifdef HAVE_FILEWATCH
+  DEFSYM (Qfilewatch_event, "filewatch-event");
+#endif /* HAVE_FILEWATCH */
+
   DEFSYM (QCenable, ":enable");
   DEFSYM (QCvisible, ":visible");
   DEFSYM (QChelp, ":help");
@@ -12168,6 +12189,13 @@ keys_of_keyboard (void)
 			    "dbus-handle-event");
 #endif
 
+#ifdef HAVE_FILEWATCH
+  /* Define a special event which is raised for filewatch callback
+     functions.  */
+  initial_define_lispy_key (Vspecial_event_map, "filewatch-event",
+                            "filewatch-handle-event");
+#endif /* HAVE_FILEWATCH */
+
   initial_define_lispy_key (Vspecial_event_map, "config-changed-event",
 			    "ignore");
 #if defined (WINDOWSNT)
diff --git a/src/lisp.h b/src/lisp.h
index 21ac55c..2f3e8b7 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3497,6 +3497,11 @@ extern void syms_of_fontset (void);
 extern Lisp_Object Qfont_param;
 #endif
 
+/* Defined in filewatch.c */
+#ifdef HAVE_FILEWATCH
+extern void syms_of_filewatch (void);
+#endif
+
 /* Defined in xfaces.c.  */
 extern Lisp_Object Qdefault, Qtool_bar, Qfringe;
 extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor;
diff --git a/src/termhooks.h b/src/termhooks.h
index f35bd92..ccb5887 100644
--- a/src/termhooks.h
+++ b/src/termhooks.h
@@ -211,6 +211,11 @@ enum event_kind
   , NS_NONKEY_EVENT
 #endif
 
+#ifdef HAVE_FILEWATCH
+  /* File or directory was changed.  */
+  , FILEWATCH_EVENT
+#endif
+
 };
 
 /* If a struct input_event has a kind which is SELECTION_REQUEST_EVENT
diff --git a/test/automated/filewatch-tests.el b/test/automated/filewatch-tests.el
new file mode 100644
index 0000000..494d704
--- /dev/null
+++ b/test/automated/filewatch-tests.el
@@ -0,0 +1,50 @@
+;;; filewatch-tests.el --- Test suite for filewatch.
+
+;; Copyright (C) 2011 Free Software Foundation, Inc.
+
+;; Author: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
+;; Keywords:       internal
+;; Human-Keywords: internal
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Code:
+
+(require 'ert)
+
+(when (featurep 'filewatch)
+
+  (ert-deftest filewatch-file-unwatch-type-check ()
+    "Test whether `file-unwatch' does proper type checking."
+    (should-error (file-unwatch "path") :type 'wrong-type-argument)
+    (should-error (file-unwatch '(file 1 2)) :type 'wrong-type-argument))
+
+  (ert-deftest filewatch-file-watch-aspects-check ()
+    "Test whether `file-watch' properly checks the aspects."
+    (let ((temp-file (make-temp-file "filewatch-aspects")))
+      (should (stringp temp-file))
+      (should-error (file-watch temp-file 'wrong nil)
+                    :type 'unknown-aspect)
+      (should-error (file-watch temp-file '(modify t) nil)
+                    :type 'unknown-aspect)
+      (should-error (file-watch temp-file '(modify Qall-modify) nil)
+                    :type 'unknown-aspect)
+      (should-error (file-watch temp-file '(access wrong modify) nil)
+                    :type 'unknown-aspect)))
+)
+
+(provide 'filewatch-tests)
+;;; filewatch-tests.el ends here.
-- 
1.7.11.3

Re: [PATCH] Added basic file system watching support.

[Stefan Monnier]

> I don't think the patch will be ready for 24.3.  It is still not complete and
> needs some heavy testing.

I'm not very familiar with inotify and other file-system watching
facilities, so just as a guideline for others's reviews: I'm OK with
installing in 24.3 a non-complete version of the code, and I'm OK to
install before the freeze a code that needs more testing, *but* only if
those missing functionalities and testing won't require a redesign of
the API.

>> It seems that each event is either of form (SYMBOL . FILENAME) where
>> FILENAME is the watched object, or of the form (SYMBOL from NAME . COOKIE)
>> where NAME is the file added/removed from the watched object (a
>> directory, presumably), but this object's FILENAME is not present.
>> Any particular reason for this inconsistency?

> FILENAME is not always the watched object.  If a directory is watched then
> it is the file name of the file inside the directory.  This should really not
> be used as identification for the watched object!  That's why CALLBACK
> also gets the ID.

I understand that NAME refers to the added/removed file, but I'm
wondering why we don't also add FILENAME (the name of the directory) to
the event.  If inotify doesn't provide it, we can provide it
ourselves, right?
Or is it rarely/never needed?
Or is it because that directory may change name without us knowing?

>> I guess that several `ev' objects can have the same `ev->cookie'
>> value, right?
> Yes exactly.  The value is unimportant as long as it is unique to an event
> and can be compared with eq.  What would you propose as an alternative
> object here?

I can't remember enough of the context, but objects that are "unique"
are plentiful.  A cons cell would do, regardless of its value, for
example (now, as to whether it's better than a number, that depends on
what it's use for, e.g. whether you can put the cons's fields to good
use).

> The problem is that even after removal of a watch descriptor there can
> still be issues in the queue.  Therefore I think it is best to ignore any
> events for files we do not watch.  There could also be event types we do not
> know about when inotify gets expanded.  That's why I think unknown
> event types should be ignored instead of reporting an error.

Sounds fair.  Please mention it in the comments.


        Stefan

Re: [PATCH] Added basic file system watching support.

[Rüdiger Sonderfeld]

Thank you for your reply!

On Friday 28 September 2012 10:13:52 you wrote:
> I'm not very familiar with inotify and other file-system watching
> facilities, so just as a guideline for others's reviews: I'm OK with
> installing in 24.3 a non-complete version of the code, and I'm OK to
> install before the freeze a code that needs more testing, *but* only if
> those missing functionalities and testing won't require a redesign of
> the API.

It could require a minor redesign of the API.  At least I want to gather some 
experience using the API first.  And maybe porting it to other systems will 
require API adjustments.  Both BSD's and Windows' filesystem watch APIs are 
not as powerful as inotify.

> I understand that NAME refers to the added/removed file, but I'm
> wondering why we don't also add FILENAME (the name of the directory) to
> the event.  If inotify doesn't provide it, we can provide it
> ourselves, right?
> Or is it rarely/never needed?
> Or is it because that directory may change name without us knowing?

The name might change.  This would require file-watch to register MOVE_SELF 
events for every watched file and then update the name.  This would make the 
code quite complicated and worst of all leave the user wondering why the name 
suddenly changed unless we force him to always accept MOVE_SELF events.  Which 
would then again impose something on the user he might not need.  Therefore I 
think it is better if the user keeps track of any ID to FILENAME mapping 
himself.  Maybe there could be some user defined storage in the ID to handle 
it.

> I can't remember enough of the context, but objects that are "unique"
> are plentiful.  A cons cell would do, regardless of its value, for
> example (now, as to whether it's better than a number, that depends on
> what it's use for, e.g. whether you can put the cons's fields to good
> use).

Currently the only risk is that cookie is bigger than MOST_POSITIVE_FIXNUM and 
then gets mapped to a value that is already taken by another cookie.  This is 
only a problem on 32bit systems without the use of wide-int because cookie is 
uint32_t.

There is a similar problem with the watch descriptor.  It is currently also 
stored in the ID with make_number.  Inotify uses an incremental system for 
watch descriptors and should be run out of descriptors by the time 
MOST_POSITIVE_FIXNUM is reached.  But there is no guarantee for it.  Therefore 
something like SAVE_VALUE should probably be used.

Regards
Rüdiger

Re: [PATCH] Added basic file system watching support.

[Stefan Monnier]

>> I'm not very familiar with inotify and other file-system watching
>> facilities, so just as a guideline for others's reviews: I'm OK with
>> installing in 24.3 a non-complete version of the code, and I'm OK to
>> install before the freeze a code that needs more testing, *but* only if
>> those missing functionalities and testing won't require a redesign of
>> the API.
> It could require a minor redesign of the API.

I know of "minor changes" (where backward compatibility is easy to
preserve) and "redesigns" but I don't know what a "minor redesign" would
look like.

> At least I want to gather some  experience using the API first.
> And maybe porting it to other systems will  require API adjustments.
> Both BSD's and Windows' filesystem watch APIs are  not as powerful
> as inotify.

[ I'm probably rehashing something already discussed at the time.  ]
If the API is sufficiently general that it can be reused for other
systems, that's great.

If there's a good chance this won't work without breaking compatibility,
maybe a better option is to provide a low-level API that maps very
closely to inotify and then an Elisp layer on top which abstracts away
differences between different systems.  In that case we can install the
inotify support right away while we're still experimenting with the
higher-level abstraction.

>> I understand that NAME refers to the added/removed file, but I'm
>> wondering why we don't also add FILENAME (the name of the directory) to
>> the event.  If inotify doesn't provide it, we can provide it
>> ourselves, right?
>> Or is it rarely/never needed?
>> Or is it because that directory may change name without us knowing?
> The name might change.

OK, makes sense.

>> I can't remember enough of the context, but objects that are "unique"
>> are plentiful.  A cons cell would do, regardless of its value, for
>> example (now, as to whether it's better than a number, that depends on
>> what it's use for, e.g. whether you can put the cons's fields to good
>> use).
> Currently the only risk is that cookie is bigger than
> MOST_POSITIVE_FIXNUM and  then gets mapped to a value that is already
> taken by another cookie.  This is only a problem on 32bit systems
> without the use of wide-int because cookie is uint32_t.

Ah, I remember what this is now.  Rather than `cookie', I'd rather name
it `identifier' or `identity'.  So IIUC Emacs's code doesn't never needs
to pass this cookie back to the inotify code, right?  So the only issue
with the current "make_number (ev->cookie)" is that it might incorrectly
lead to matching two events that are really unrelated, in case they have
the same lower 30bits.  I don't know how important those last 2bits are
in practice.  But if they're unlikely to be important in practice, then
I guess the current solution might be acceptable.

OTOH we should stipulate that COOKIES should be compared with `equal',
not `eq', so we could later use something like Fcons (make_number
(ev->cookie / 65536), make_number (ev->cookie % 65536)).

> There is a similar problem with the watch descriptor.

This is slightly different in that here, the descriptor is later passed
back to inotify, so it's indispensable that it be preserved correctly.
Also, descriptors aren't usually compared for equality: it might be OK
to have two watch-descriptors that refer to the same watch object but
which aren't `eq'.

The docstring of file-watch doesn't really make it clear what file-watch
returns (it only says it indirectly with "The ID argument is the id returned by
file-watch") so that should be fixed.
Also this ID should be called WD for "watch descriptor", WH for "watch
handle", or FW for "file-watcher".

> Inotify uses an incremental system for watch descriptors and should
> be run out of descriptors by the time MOST_POSITIVE_FIXNUM is
> reached.  But there is no guarantee for it.  Therefore something like
> SAVE_VALUE should probably be used.

If watch descriptors are documented to be "small integers", like file
descriptors, it might be OK to use fixnums, but it's a bit ugly.

I think the cleaner option is to define a new object type for it.
It could be either a new Lisp_Misc type, so you can make them print as
something like "#<file-watcher NNN>" (take a look at "enum
Lisp_Misc_Type" and "union Lisp_Misc" in src/lisp.h for starters; adding
a new type will require adding corresponding branches to the switch
statements in alloc.c and in print.c).


        Stefan

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 01 Oct 2012 00:38:09 -0400
> Cc: Leo <sdl.web@gmail.com>, emacs-devel@gnu.org
> 
> >> I'm not very familiar with inotify and other file-system watching
> >> facilities, so just as a guideline for others's reviews: I'm OK with
> >> installing in 24.3 a non-complete version of the code, and I'm OK to
> >> install before the freeze a code that needs more testing, *but* only if
> >> those missing functionalities and testing won't require a redesign of
> >> the API.
> > It could require a minor redesign of the API.
> 
> I know of "minor changes" (where backward compatibility is easy to
> preserve) and "redesigns" but I don't know what a "minor redesign" would
> look like.
> 
> > At least I want to gather some  experience using the API first.
> > And maybe porting it to other systems will  require API adjustments.
> > Both BSD's and Windows' filesystem watch APIs are  not as powerful
> > as inotify.
> 
> [ I'm probably rehashing something already discussed at the time.  ]
> If the API is sufficiently general that it can be reused for other
> systems, that's great.

Perhaps the API could be spelled out here, then this question could be
answered for systems without inotify.

[PATCH] Added inotify support.

[Rüdiger Sonderfeld]

On Monday 01 October 2012 00:38:09 Stefan Monnier wrote:
> If there's a good chance this won't work without breaking compatibility,
> maybe a better option is to provide a low-level API that maps very
> closely to inotify and then an Elisp layer on top which abstracts away
> differences between different systems.  In that case we can install the
> inotify support right away while we're still experimenting with the
> higher-level abstraction.

That's probably the best approach here.  I changed the patch to provide a low
level inotify interface.  However I did not provide an inotify_init(2) like 
function and instead initialization and closing of the inotify handle is done
internally.  I don't think that this should be exposed to elisp even in the 
low level interface.

> But if they're unlikely to be important in practice, then
> I guess the current solution might be acceptable.

I think we are safe.  I added that `equal' should be used to compare cookies.  
So we can easily change it without breaking the API.

> I think the cleaner option is to define a new object type for it.
> It could be either a new Lisp_Misc type, so you can make them print as
> something like "#<file-watcher NNN>" (take a look at "enum
> Lisp_Misc_Type" and "union Lisp_Misc" in src/lisp.h for starters; adding
> a new type will require adding corresponding branches to the switch
> statements in alloc.c and in print.c).

That sounds like the best option.  I haven't implemented it yet.  Is it 
possible to make the Lisp_Misc_Type comparable with `equal'? Because the 
watch-descriptor has to be comparable.

Regards,
Rüdiger

-- >8 --

Inotify is a Linux kernel API to monitor file system events.  This
patch adds a basic inotify based API to Emacs:

`inotify-add-watch' is based on inotify_add_watch(2).  But uses
callbacks and accepts lisp objects instead of a bit mask.

`inotify-rm-watch' is based on inotify_rm_watch(2).

The creation and destruction of an inotify fd with inotify_init1(2) is
handled internally.

Example:

    (let ((wd (inotify-add-watch "foo.txt" t
          (lambda (ev) (message "FS Event %s" ev)))))
      ;; ...
      (inotify-rm-watch wd))

Signed-off-by: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
---
 configure.ac                   |  13 ++
 lisp/subr.el                   |  21 ++
 src/Makefile.in                |   2 +-
 src/emacs.c                    |   4 +
 src/inotify.c                  | 431 
+++++++++++++++++++++++++++++++++++++++++
 src/keyboard.c                 |  28 +++
 src/lisp.h                     |   5 +
 src/termhooks.h                |   5 +
 test/automated/inotify-test.el |  60 ++++++
 9 files changed, 568 insertions(+), 1 deletion(-)
 create mode 100644 src/inotify.c
 create mode 100644 test/automated/inotify-test.el

diff --git a/configure.ac b/configure.ac
index 5a3aea7..c1ce23f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -184,6 +184,7 @@ OPTION_DEFAULT_ON([gconf],[don't compile with GConf 
support])
 OPTION_DEFAULT_ON([gsettings],[don't compile with GSettings support])
 OPTION_DEFAULT_ON([selinux],[don't compile with SELinux support])
 OPTION_DEFAULT_ON([gnutls],[don't use -lgnutls for SSL/TLS support])
+OPTION_DEFAULT_ON([inotify],[don't compile with inotify (file-watch) 
support])
 
 ## For the times when you want to build Emacs but don't have
 ## a suitable makeinfo, and can live without the manuals.
@@ -2101,6 +2102,18 @@ fi
 AC_SUBST(LIBGNUTLS_LIBS)
 AC_SUBST(LIBGNUTLS_CFLAGS)
 
+dnl inotify is only available on GNU/Linux.
+HAVE_INOTIFY=no
+if test "${with_inotify}" = "yes"; then
+   AC_CHECK_HEADERS(sys/inotify.h)
+   if test "$ac_cv_header_sys_inotify_h" = yes ; then
+     AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch, 
HAVE_INOTIFY=yes)
+   fi
+fi
+if test "${HAVE_INOTIFY}" = "yes"; then
+   AC_DEFINE(HAVE_INOTIFY, [1], [Define to 1 to use inotify])
+fi
+
 dnl Do not put whitespace before the #include statements below.
 dnl Older compilers (eg sunos4 cc) choke on it.
 HAVE_XAW3D=no
diff --git a/lisp/subr.el b/lisp/subr.el
index 8dfe78d..134a1dc 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -4159,6 +4159,27 @@ convenience wrapper around `make-progress-reporter' and 
friends.
        nil ,@(cdr (cdr spec)))))
 
 \f
+;;;; Support for watching filesystem events.
+
+(defun inotify-event-p (event)
+  "Check if EVENT is an inotify event."
+  (and (listp event)
+       (>= (length event) 3)
+       (eq (car event) 'inotify-event)))
+
+;;;###autoload
+(defun inotify-handle-event (event)
+  "Handle file system monitoring event.
+If EVENT is a filewatch event then the callback is called.  If EVENT is
+not a filewatch event then a `filewatch-error' is signaled."
+  (interactive "e")
+
+  (unless (inotify-event-p event)
+    (signal 'inotify-error (cons "Not a valid inotify event" event)))
+
+  (funcall (nth 2 event) (nth 1 event)))
+
+\f
 ;;;; Comparing version strings.
 
 (defconst version-separator "."
diff --git a/src/Makefile.in b/src/Makefile.in
index f8da009..1492dc6 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -339,7 +339,7 @@ base_obj = dispnew.o frame.o scroll.o xdisp.o menu.o 
$(XMENU_OBJ) window.o \
 	syntax.o $(UNEXEC_OBJ) bytecode.o \
 	process.o gnutls.o callproc.o \
 	region-cache.o sound.o atimer.o \
-	doprnt.o intervals.o textprop.o composite.o xml.o \
+	doprnt.o intervals.o textprop.o composite.o xml.o inotify.o \
 	profiler.o \
 	$(MSDOS_OBJ) $(MSDOS_X_OBJ) $(NS_OBJ) $(CYGWIN_OBJ) $(FONT_OBJ) \
 	$(WINDOW_SYSTEM_OBJ)
diff --git a/src/emacs.c b/src/emacs.c
index 05affee..e43fa0e 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -1411,6 +1411,10 @@ Using an Emacs configured with --with-x-toolkit=lucid 
does not have this problem
       syms_of_gnutls ();
 #endif
 
+#ifdef HAVE_INOTIFY
+      syms_of_inotify ();
+#endif /* HAVE_INOTIFY */
+
 #ifdef HAVE_DBUS
       syms_of_dbusbind ();
 #endif /* HAVE_DBUS */
diff --git a/src/inotify.c b/src/inotify.c
new file mode 100644
index 0000000..49a45cb
--- /dev/null
+++ b/src/inotify.c
@@ -0,0 +1,431 @@
+/* Inotify support for Emacs
+
+Copyright (C) 2012
+  Free Software Foundation, Inc.
+
+This file is part of GNU Emacs.
+
+GNU Emacs is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+GNU Emacs is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <config.h>
+
+#ifdef HAVE_INOTIFY
+
+#include "lisp.h"
+#include "coding.h"
+#include "process.h"
+#include "keyboard.h"
+#include "character.h"
+#include "frame.h" /* Required for termhooks.h.  */
+#include "termhooks.h"
+
+static Lisp_Object Qaccess;        /* IN_ACCESS */
+static Lisp_Object Qattrib;        /* IN_ATTRIB */
+static Lisp_Object Qclose_write;   /* IN_CLOSE_WRITE */
+static Lisp_Object Qclose_nowrite; /* IN_CLOSE_NOWRITE */
+static Lisp_Object Qcreate;        /* IN_CREATE */
+static Lisp_Object Qdelete;        /* IN_DELETE */
+static Lisp_Object Qdelete_self;   /* IN_DELETE_SELF */
+static Lisp_Object Qmodify;        /* IN_MODIFY */
+static Lisp_Object Qmove_self;     /* IN_MOVE_SELF */
+static Lisp_Object Qmoved_from;    /* IN_MOVED_FROM */
+static Lisp_Object Qmoved_to;      /* IN_MOVED_TO */
+static Lisp_Object Qopen;          /* IN_OPEN */
+
+static Lisp_Object Qall_events;    /* IN_ALL_EVENTS */
+static Lisp_Object Qmove;          /* IN_MOVE */
+static Lisp_Object Qclose;         /* IN_CLOSE */
+
+static Lisp_Object Qdont_follow;   /* IN_DONT_FOLLOW */
+static Lisp_Object Qexcl_unlink;   /* IN_EXCL_UNLINK */
+static Lisp_Object Qmask_add;      /* IN_MASK_ADD */
+static Lisp_Object Qoneshot;       /* IN_ONESHOT */
+static Lisp_Object Qonlydir;       /* IN_ONLYDIR */
+
+static Lisp_Object Qignored;       /* IN_IGNORED */
+static Lisp_Object Qisdir;         /* IN_ISDIR */
+static Lisp_Object Qq_overflow;    /* IN_Q_OVERFLOW */
+static Lisp_Object Qunmount;       /* IN_UNMOUNT */
+
+static Lisp_Object Qinotify_event;
+
+#include <sys/inotify.h>
+#include <sys/ioctl.h>
+
+enum { uninitialized = -100 };
+/* File handle for inotify.  */
+static int inotifyfd = uninitialized;
+
+/* Assoc list of files being watched.
+   Format:
+   (watch-descriptor . callback)
+ */
+static Lisp_Object watch_list;
+
+static Lisp_Object
+make_watch_descriptor (int wd)
+{
+  /* TODO replace this with a Misc Object! */
+  return make_number (wd);
+}
+
+static Lisp_Object
+mask_to_aspects (uint32_t mask) {
+  Lisp_Object aspects = Qnil;
+  if (mask & IN_ACCESS)
+    aspects = Fcons (Qaccess, aspects);
+  if (mask & IN_ATTRIB)
+    aspects = Fcons (Qattrib, aspects);
+  if (mask & IN_CLOSE_WRITE)
+    aspects = Fcons (Qclose_write, aspects);
+  if (mask & IN_CLOSE_NOWRITE)
+    aspects = Fcons (Qclose_nowrite, aspects);
+  if (mask & IN_CREATE)
+    aspects = Fcons (Qcreate, aspects);
+  if (mask & IN_DELETE)
+    aspects = Fcons (Qdelete, aspects);
+  if (mask & IN_DELETE_SELF)
+    aspects = Fcons (Qdelete_self, aspects);
+  if (mask & IN_MODIFY)
+    aspects = Fcons (Qmodify, aspects);
+  if (mask & IN_MOVE_SELF)
+    aspects = Fcons (Qmove_self, aspects);
+  if (mask & IN_MOVED_FROM)
+    aspects = Fcons (Qmoved_from, aspects);
+  if (mask & IN_MOVED_TO)
+    aspects = Fcons (Qmoved_to, aspects);
+  if (mask & IN_OPEN)
+    aspects = Fcons (Qopen,  aspects);
+  if (mask & IN_IGNORED)
+    aspects = Fcons (Qignored, aspects);
+  if (mask & IN_ISDIR)
+    aspects = Fcons (Qisdir, aspects);
+  if (mask & IN_Q_OVERFLOW)
+    aspects = Fcons (Qq_overflow, aspects);
+  if (mask & IN_UNMOUNT)
+    aspects = Fcons (Qunmount, aspects);
+  return aspects;
+}
+
+static Lisp_Object
+inotifyevent_to_event (Lisp_Object watch_object, struct inotify_event const 
*ev)
+{
+  Lisp_Object name = Qnil;
+  if (ev->len > 0)
+    {
+      size_t const len = strlen (ev->name);
+      name = make_unibyte_string (ev->name, min (len, ev->len));
+      name = DECODE_FILE (name);
+    }
+
+  return list2 (list4 (make_watch_descriptor (ev->wd),
+                       mask_to_aspects (ev->mask),
+                       make_number (ev->cookie),
+                       name),
+                XCDR (watch_object));
+}
+
+/* This callback is called when the FD is available for read.  The inotify
+   events are read from FD and converted into input_events.  */
+static void
+inotify_callback (int fd, void *_)
+{
+  struct input_event event;
+  Lisp_Object watch_object;
+  int to_read;
+  char *buffer;
+  ssize_t n;
+  size_t i;
+
+  to_read = 0;
+  if (ioctl (fd, FIONREAD, &to_read) == -1)
+    report_file_error ("Error while trying to retrieve file system events",
+                       Qnil);
+  buffer = xmalloc (to_read);
+  n = read (fd, buffer, to_read);
+  if (n < 0)
+    {
+      xfree (buffer);
+      report_file_error ("Error while trying to read file system events",
+                         Qnil);
+    }
+
+  EVENT_INIT (event);
+  event.kind = INOTIFY_EVENT;
+  event.arg = Qnil;
+
+  i = 0;
+  while (i < (size_t)n)
+    {
+      struct inotify_event *ev = (struct inotify_event*)&buffer[i];
+
+      watch_object = Fassoc (make_watch_descriptor (ev->wd), watch_list);
+      if (!NILP (watch_object))
+        {
+          event.arg = inotifyevent_to_event (watch_object, ev);
+
+          /* If event was removed automatically: Drop it from watch list.  */
+          if (ev->mask & IN_IGNORED)
+            watch_list = Fdelete (watch_object, watch_list);
+        }
+
+      i += sizeof (*ev) + ev->len;
+    }
+
+  if (!NILP (event.arg))
+    kbd_buffer_store_event (&event);
+
+  xfree (buffer);
+}
+
+static uint32_t
+symbol_to_inotifymask (Lisp_Object symb)
+{
+  if (EQ (symb, Qaccess))
+    return IN_ACCESS;
+  else if (EQ (symb, Qattrib))
+    return IN_ATTRIB;
+  else if (EQ (symb, Qclose_write))
+    return IN_CLOSE_WRITE;
+  else if (EQ (symb, Qclose_nowrite))
+    return IN_CLOSE_NOWRITE;
+  else if (EQ (symb, Qcreate))
+    return IN_CREATE;
+  else if (EQ (symb, Qdelete))
+    return IN_DELETE;
+  else if (EQ (symb, Qdelete_self))
+    return IN_DELETE_SELF;
+  else if (EQ (symb, Qmodify))
+    return IN_MODIFY;
+  else if (EQ (symb, Qmove_self))
+    return IN_MOVE_SELF;
+  else if (EQ (symb, Qmoved_from))
+    return IN_MOVED_FROM;
+  else if (EQ (symb, Qmoved_to))
+    return IN_MOVED_TO;
+  else if (EQ (symb, Qopen))
+    return IN_OPEN;
+  else if (EQ (symb, Qmove))
+    return IN_MOVE;
+  else if (EQ (symb, Qclose))
+    return IN_CLOSE;
+
+  else if (EQ (symb, Qdont_follow))
+    return IN_DONT_FOLLOW;
+  else if (EQ (symb, Qexcl_unlink))
+    return IN_EXCL_UNLINK;
+  else if (EQ (symb, Qmask_add))
+    return IN_MASK_ADD;
+  else if (EQ (symb, Qoneshot))
+    return IN_ONESHOT;
+  else if (EQ (symb, Qonlydir))
+    return IN_ONLYDIR;
+
+  else if (EQ (symb, Qt) || EQ (symb, Qall_events))
+    return IN_ALL_EVENTS;
+  else
+    signal_error ("Unknown aspect", symb);
+}
+
+static uint32_t
+aspect_to_inotifymask (Lisp_Object aspect)
+{
+  if (CONSP (aspect))
+    {
+      Lisp_Object x = aspect;
+      uint32_t mask = 0;
+      while (CONSP (x))
+        {
+          mask |= symbol_to_inotifymask (XCAR (x));
+          x = XCDR (x);
+        }
+      return mask;
+    }
+  else
+    return symbol_to_inotifymask (aspect);
+}
+
+DEFUN ("inotify-add-watch", Finotify_add_watch, Sinotify_add_watch, 3, 3, 0,
+       doc: /* Add a watch for FILE-NAME to inotify.
+
+A WATCH-DESCRIPTOR is returned on success.  ASPECT might be one of the 
following
+symbols or a list of those symbols:
+
+access
+attrib
+close-write
+close-nowrite
+create
+delete
+delete-self
+modify
+move-self
+moved-from
+moved-to
+open
+
+all-events or t
+move
+close
+
+The following symbols can also be added to a list of aspects
+
+dont-follow
+excl-unlink
+mask-add
+oneshot
+onlydir
+
+Watching a directory is not recursive.  CALLBACK gets called in case of an
+event.  It gets passed a single argument EVENT which contains an event 
structure
+of the format
+
+(WATCH-DESCRIPTOR ASPECTS COOKIE NAME)
+
+WATCH-DESCRIPTOR is the same object that was returned by this function.  It 
can
+be tested for equality using `equal'.  ASPECTS describes the event.  It is a
+list of ASPECT symbols described above and can also contain one of the 
following
+symbols
+
+ignored
+isdir
+q-overflow
+unmount
+
+COOKIE is an object that can be compared using `equal' to identify two 
matching
+renames (moved-from and moved-to).
+
+If a directory is watched then NAME is the name of file that caused the 
event.
+
+See inotify(7) and inotify_add_watch(2) for further information.  The inotify 
fd
+is managed internally and there is no corresponding inotify_init.  Use
+`inotify-rm-watch' to remove a watch.
+             */)
+     (Lisp_Object file_name, Lisp_Object aspect, Lisp_Object callback)
+{
+  uint32_t mask;
+  Lisp_Object watch_object;
+  Lisp_Object decoded_file_name;
+  Lisp_Object watch_descriptor;
+  int watchdesc = -1;
+
+  CHECK_STRING (file_name);
+
+  if (inotifyfd == uninitialized)
+    {
+      inotifyfd = inotify_init1 (IN_NONBLOCK|IN_CLOEXEC);
+      if (inotifyfd == -1)
+        {
+          inotifyfd = uninitialized;
+          report_file_error ("File watching feature (inotify) is not 
available",
+                             Qnil);
+        }
+      watch_list = Qnil;
+      add_read_fd (inotifyfd, &inotify_callback, NULL);
+    }
+
+  mask = aspect_to_inotifymask (aspect);
+  decoded_file_name = ENCODE_FILE (file_name);
+  watchdesc = inotify_add_watch (inotifyfd, SSDATA (decoded_file_name), 
mask);
+  if (watchdesc == -1)
+    report_file_error ("Could not add watch for file", Fcons (file_name, 
Qnil));
+
+  watch_descriptor = make_watch_descriptor (watchdesc);
+
+  /* Delete existing watch object. */
+  watch_object = Fassoc (watch_descriptor, watch_list);
+  if (!NILP (watch_object))
+      watch_list = Fdelete (watch_object, watch_list);
+
+  /* Store watch object in watch list. */
+  watch_object = Fcons (watch_descriptor, callback);
+  watch_list = Fcons (watch_object, watch_list);
+
+  return watch_descriptor;
+}
+
+DEFUN ("inotify-rm-watch", Finotify_rm_watch, Sinotify_rm_watch, 1, 1, 0,
+       doc: /* Remove an existing WATCH-DESCRIPTOR.
+
+WATCH-DESCRIPTOR should be an object returned by `inotify-add-watch'.
+
+See inotify_rm_watch(2) for more information.
+             */)
+     (Lisp_Object watch_descriptor)
+{
+  Lisp_Object watch_object;
+  int wd = XINT (watch_descriptor);
+
+  if (inotify_rm_watch (inotifyfd, wd) == -1)
+    report_file_error ("Could not rm watch", Fcons (watch_descriptor,
+                                                    Qnil));
+
+  /* Remove watch descriptor from watch list. */
+  watch_object = Fassoc (watch_descriptor, watch_list);
+  if (!NILP (watch_object))
+    watch_list = Fdelete (watch_object, watch_list);
+
+  /* Cleanup if no more files are watched. */
+  if (NILP (watch_list))
+    {
+      close (inotifyfd);
+      delete_read_fd (inotifyfd);
+      inotifyfd = uninitialized;
+    }
+
+  return Qt;
+}
+
+void
+syms_of_inotify (void)
+{
+  DEFSYM (Qaccess, "access");
+  DEFSYM (Qattrib, "attrib");
+  DEFSYM (Qclose_write, "close-write");
+  DEFSYM (Qclose_nowrite, "close-nowrite");
+  DEFSYM (Qcreate, "create");
+  DEFSYM (Qdelete, "delete");
+  DEFSYM (Qdelete_self, "delete-self");
+  DEFSYM (Qmodify, "modify");
+  DEFSYM (Qmove_self, "move-self");
+  DEFSYM (Qmoved_from, "moved-from");
+  DEFSYM (Qmoved_to, "moved-to");
+  DEFSYM (Qopen, "open");
+
+  DEFSYM (Qall_events, "all-events");
+  DEFSYM (Qmove, "move");
+  DEFSYM (Qclose, "close");
+
+  DEFSYM (Qdont_follow, "dont-follow");
+  DEFSYM (Qexcl_unlink, "excl-unlink");
+  DEFSYM (Qmask_add, "mask-add");
+  DEFSYM (Qoneshot, "oneshot");
+  DEFSYM (Qonlydir, "onlydir");
+
+  DEFSYM (Qignored, "ignored");
+  DEFSYM (Qisdir, "isdir");
+  DEFSYM (Qq_overflow, "q-overflow");
+  DEFSYM (Qunmount, "unmount");
+
+  DEFSYM (Qinotify_event, "inotify-event");
+
+  defsubr (&Sinotify_add_watch);
+  defsubr (&Sinotify_rm_watch);
+
+  staticpro (&watch_list);
+
+  Fprovide (intern_c_string ("inotify"), Qnil);
+}
+
+#endif /* HAVE_INOTIFY */
diff --git a/src/keyboard.c b/src/keyboard.c
index f3d7df5..02e1473 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -321,6 +321,9 @@ static Lisp_Object Qsave_session;
 #ifdef HAVE_DBUS
 static Lisp_Object Qdbus_event;
 #endif
+#ifdef HAVE_INOTIFY
+static Lisp_Object Qinotify_event;
+#endif /* HAVE_INOTIFY */
 static Lisp_Object Qconfig_changed_event;
 
 /* Lisp_Object Qmouse_movement; - also an event header */
@@ -4021,6 +4024,13 @@ kbd_buffer_get_event (KBOARD **kbp,
 	  kbd_fetch_ptr = event + 1;
 	}
 #endif
+#ifdef HAVE_INOTIFY
+      else if (event->kind == INOTIFY_EVENT)
+        {
+          obj = make_lispy_event (event);
+          kbd_fetch_ptr = event + 1;
+        }
+#endif
       else if (event->kind == CONFIG_CHANGED_EVENT)
 	{
 	  obj = make_lispy_event (event);
@@ -5938,6 +5948,13 @@ make_lispy_event (struct input_event *event)
       }
 #endif /* HAVE_DBUS */
 
+#ifdef HAVE_INOTIFY
+    case INOTIFY_EVENT:
+      {
+        return Fcons (Qinotify_event, event->arg);
+      }
+#endif /* HAVE_INOTIFY */
+
     case CONFIG_CHANGED_EVENT:
 	return Fcons (Qconfig_changed_event,
                       Fcons (event->arg,
@@ -11408,6 +11425,10 @@ syms_of_keyboard (void)
   DEFSYM (Qdbus_event, "dbus-event");
 #endif
 
+#ifdef HAVE_INOTIFY
+  DEFSYM (Qinotify_event, "inotify-event");
+#endif /* HAVE_INOTIFY */
+
   DEFSYM (QCenable, ":enable");
   DEFSYM (QCvisible, ":visible");
   DEFSYM (QChelp, ":help");
@@ -12168,6 +12189,13 @@ keys_of_keyboard (void)
 			    "dbus-handle-event");
 #endif
 
+#ifdef HAVE_INOTIFY
+  /* Define a special event which is raised for inotify callback
+     functions.  */
+  initial_define_lispy_key (Vspecial_event_map, "inotify-event",
+                            "inotify-handle-event");
+#endif /* HAVE_INOTIFY */
+
   initial_define_lispy_key (Vspecial_event_map, "config-changed-event",
 			    "ignore");
 #if defined (WINDOWSNT)
diff --git a/src/lisp.h b/src/lisp.h
index c3cabe0..02bfa08 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3497,6 +3497,11 @@ extern void syms_of_fontset (void);
 extern Lisp_Object Qfont_param;
 #endif
 
+/* Defined in inotify.c */
+#ifdef HAVE_INOTIFY
+extern void syms_of_inotify (void);
+#endif
+
 /* Defined in xfaces.c.  */
 extern Lisp_Object Qdefault, Qtool_bar, Qfringe;
 extern Lisp_Object Qheader_line, Qscroll_bar, Qcursor;
diff --git a/src/termhooks.h b/src/termhooks.h
index f35bd92..5890d10 100644
--- a/src/termhooks.h
+++ b/src/termhooks.h
@@ -211,6 +211,11 @@ enum event_kind
   , NS_NONKEY_EVENT
 #endif
 
+#ifdef HAVE_INOTIFY
+  /* File or directory was changed.  */
+  , INOTIFY_EVENT
+#endif
+
 };
 
 /* If a struct input_event has a kind which is SELECTION_REQUEST_EVENT
diff --git a/test/automated/inotify-test.el b/test/automated/inotify-test.el
new file mode 100644
index 0000000..edda7ef
--- /dev/null
+++ b/test/automated/inotify-test.el
@@ -0,0 +1,60 @@
+;;; inotify-tests.el --- Test suite for inotify. -*- lexical-binding: t -*-
+
+;; Copyright (C) 2012 Free Software Foundation, Inc.
+
+;; Author: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
+;; Keywords:       internal
+;; Human-Keywords: internal
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Code:
+
+(require 'ert)
+
+(when (featurep 'inotify)
+
+  ;; (ert-deftest filewatch-file-watch-aspects-check ()
+  ;;   "Test whether `file-watch' properly checks the aspects."
+  ;;   (let ((temp-file (make-temp-file "filewatch-aspects")))
+  ;;     (should (stringp temp-file))
+  ;;     (should-error (file-watch temp-file 'wrong nil)
+  ;;                   :type 'error)
+  ;;     (should-error (file-watch temp-file '(modify t) nil)
+  ;;                   :type 'error)
+  ;;     (should-error (file-watch temp-file '(modify all-modify) nil)
+  ;;                   :type 'error)
+  ;;     (should-error (file-watch temp-file '(access wrong modify) nil)
+  ;;                   :type 'error)))
+
+  (ert-deftest inotify-file-watch-simple ()
+    "Test if watching a normal file works."
+    (let ((temp-file (make-temp-file "inotify-simple"))
+	   (events 0))
+      (let ((wd
+	     (inotify-add-watch temp-file t (lambda (ev)
+                                              (setq events (1+ events))))))
+	(unwind-protect
+	    (progn
+	      (with-temp-file temp-file
+		(insert "Foo\n"))
+	      (sit-for 5) ;; Hacky. Wait for 5s until events are processed
+	      (should (> events 0)))
+	  (inotify-rm-watch wd)))))
+)
+
+(provide 'inotify-tests)
+;;; inotify-tests.el ends here.
-- 
1.7.11.3

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> I think the cleaner option is to define a new object type for it.
>> It could be either a new Lisp_Misc type, so you can make them print as
>> something like "#<file-watcher NNN>" (take a look at "enum
>> Lisp_Misc_Type" and "union Lisp_Misc" in src/lisp.h for starters; adding
>> a new type will require adding corresponding branches to the switch
>> statements in alloc.c and in print.c).
> That sounds like the best option.  I haven't implemented it yet.  Is it 
> possible to make the Lisp_Misc_Type comparable with `equal'?

It could be done (fairly easily), but I'd rather not.

> Because the  watch-descriptor has to be comparable.

It's comparable with `eq'.  Why would you need two different
#<file-watcher...> objects for the same C-level `wd'?


        Stefan

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> Date: Mon, 01 Oct 2012 16:09:55 +0200
> Cc: Leo <sdl.web@gmail.com>, emacs-devel@gnu.org
> 
> On Monday 01 October 2012 00:38:09 Stefan Monnier wrote:
> > If there's a good chance this won't work without breaking compatibility,
> > maybe a better option is to provide a low-level API that maps very
> > closely to inotify and then an Elisp layer on top which abstracts away
> > differences between different systems.  In that case we can install the
> > inotify support right away while we're still experimenting with the
> > higher-level abstraction.
> 
> That's probably the best approach here.  I changed the patch to provide a low
> level inotify interface.  However I did not provide an inotify_init(2) like 
> function and instead initialization and closing of the inotify handle is done
> internally.  I don't think that this should be exposed to elisp even in the 
> low level interface.

Btw, what are Emacs use cases for using this kind of feature?

Watching a directory or even a single file can easily flood Emacs with
many events, and in some extreme cases (like watching /tmp, especially
using IN_ACCESS) so many that it will probably make Emacs unusable.
(The equivalent Windows APIs give you an even longer rope, in that you
can watch a directory and all its subdirectories, recursively, with a
single watch handle.)  Also, a typical file operation can be presented
as a series of notifications that are not easy to make sense of,
unless you are a filesystem expert and know exactly what to expect.

Maybe we should think a little more, before we expose all the guts of
inotify to the Lisp level.  It's not like anyone will write an OS in
Emacs Lisp any time soon, is it?

Re: [PATCH] Added inotify support.

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> Btw, what are Emacs use cases for using this kind of feature?

Apart from those mentioned on the original message (dired or magit's (or
vc-dir) status view) I'll like to mention auto-revert-mode.

Currently auto-revert is a bit of a hack, in the sense that Emacs
implements a poor man's method for being aware of file changes. Checking
the timestamps every N seconds is cumbersome and has annoying
side-effects compared to the Right Thing.

> Watching a directory or even a single file can easily flood Emacs with
> many events,

There are users that turn on auto-revert-mode on all file buffers. When
you accumulated quite a few buffers, checking all the associated
timestamps can be a bit lengthy and disruptive, so
auto-revert-stop-on-user-input was introduced (another hack). Plus,
waiting 5 seconds for checking for changes is, sometimes, annoyingly
long, and decreasing that value may end on too much workload for Emacs.

> and in some extreme cases (like watching /tmp, especially
> using IN_ACCESS) so many that it will probably make Emacs unusable.
> (The equivalent Windows APIs give you an even longer rope, in that you
> can watch a directory and all its subdirectories, recursively, with a
> single watch handle.)  Also, a typical file operation can be presented
> as a series of notifications that are not easy to make sense of,
> unless you are a filesystem expert and know exactly what to expect.
>
> Maybe we should think a little more, before we expose all the guts of
> inotify to the Lisp level.  It's not like anyone will write an OS in
> Emacs Lisp any time soon, is it?

You are leaving this case to extremes. Emacs already allows you to do
very silly things. Let's not throw out useful features simply because
they might be abused.

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> Btw, what are Emacs use cases for using this kind of feature?
> Apart from those mentioned on the original message (dired or magit's (or
> vc-dir) status view) I'll like to mention auto-revert-mode.

Yes, for my own use auto-revert-mode is the main one.


        Stefan

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 02 Oct 2012 22:10:41 -0400
> Cc: emacs-devel@gnu.org
> 
> >> Btw, what are Emacs use cases for using this kind of feature?
> > Apart from those mentioned on the original message (dired or magit's (or
> > vc-dir) status view) I'll like to mention auto-revert-mode.
> 
> Yes, for my own use auto-revert-mode is the main one.

AFAICS, all these use cases don't need support for every possible
notification inotify lets you define.  How about a simpler interface?

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> >> Btw, what are Emacs use cases for using this kind of feature?
>> > Apart from those mentioned on the original message (dired or magit's (or
>> > vc-dir) status view) I'll like to mention auto-revert-mode.
>> Yes, for my own use auto-revert-mode is the main one.
> AFAICS, all these use cases don't need support for every possible
> notification inotify lets you define.  How about a simpler interface?

The patch he suggests is "as simple as it gets" because it just exposes
the C-level API.  And yes, we'll want a simpler interface on top.
Both so as to make it easier to use but also so it can work with other
mechanisms than inotify.


        Stefan

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: ofv@wanadoo.es,  emacs-devel@gnu.org
> Date: Wed, 03 Oct 2012 08:46:39 -0400
> 
> >> >> Btw, what are Emacs use cases for using this kind of feature?
> >> > Apart from those mentioned on the original message (dired or magit's (or
> >> > vc-dir) status view) I'll like to mention auto-revert-mode.
> >> Yes, for my own use auto-revert-mode is the main one.
> > AFAICS, all these use cases don't need support for every possible
> > notification inotify lets you define.  How about a simpler interface?
> 
> The patch he suggests is "as simple as it gets" because it just exposes
> the C-level API.  And yes, we'll want a simpler interface on top.
> Both so as to make it easier to use but also so it can work with other
> mechanisms than inotify.

If providing a higher-level interface is the plan, then I think
inotify.el should be renamed to something like file-notifications.el, or
some other neutral name, because before long it will be home to other
back ends and to those higher-level APIs.

That said, when will this simpler interface be added? before or after
Emacs 24.3 is locked for changes?  If before, then you can skip
everything I write below (or save it for a future discussion ;-).
That's assuming the inotify support _will_ be in Emacs 24.3.

But if Emacs 24.3 is supposed to be released without this simpler
interface, then I think we will do a disservice to Lisp programmers.

E.g., consider the Dired use case.  This wants to know about any and
all changes to the files in the directory, because it will need to
refresh the listing, right?  But as things are, whoever will want to
code this will have to carefully read the inotify(7) man page and
figure out which of the IN_* flags she wants, because there's no
'just-what-dired-needs' flag in the API that is being proposed.

Or consider the autorevert use case.  Which flag(s) to use for that?
My first guess was IN_ATTRIB, but the man page says

  IN_ATTRIB         Metadata changed, e.g., permissions, timestamps,
                    extended attributes, link count (since Linux 2.6.25),
                    UID, GID, etc.

Hmmm... does this include file size? I don't know.  If it doesn't,
then do I need to catch the series of IN_OPEN, IN_WRITE, IN_CLOSE?

Are you confused yet?  Because I am.

Another concern is how to design the Lisp code that gets run by
notifications.  Since we are converting notifications into Lisp
events, the most natural way of using them would be to bind a function
to the 'inotify-event' event.  (Btw, I'd drop the "event" part from
the symbol name, it's redundant.)  But since meaningful file-level
events are likely to produce multiple notifications (see below), such
a function will need to implement a non-trivial state machine to DTRT.

For example, if you type at the shell prompt "echo foo > bar" with
'bar' an existing non-empty file, how many notifications do you see?
On Windows, I get 2 or 3, depending on the details: 2 for size change
(first one when the file is truncated, second one when "foo" is
written into it), and sometimes also an attribute or "security" change
(e.g., if the original file was owned by someone else).  Unless
inotify magically merges all these into a single meaningful
notification, the Lisp code that receives this series of notifications
will have hard time doing TRT with them, even if the exact expected
series are known in advance, especially if other notifications are
interspersed with these.

As yet another example, many applications update a file by first
writing a temporary file, then deleting the old file and renaming the
new one.  That's another series of notifications someone will have to
figure out in Lisp, when the function bound to 'inotify-event' will
get called several times, because from the user POV, the file got
updated, not overwritten by moving another file over it.

This is why I think we _must_ have a higher-level API in place for
this feature to be useful in practice, even if just inotify back end
is supported.  And we should talk _today_ about that API, because some
of the processing needed to produce higher-level abstractions of
events are much easier done in C than in Lisp.

Re: [PATCH] Added inotify support.

[Stefan Monnier]

> That said, when will this simpler interface be added? before or after
> Emacs 24.3 is locked for changes?

After: there is no time to design such an API before the freeze.

> That's assuming the inotify support _will_ be in Emacs 24.3.

I think it would be good for it to be in 24.3, mostly to get the
ball rolling.

> Hmmm... does this include file size? I don't know.  If it doesn't,
> then do I need to catch the series of IN_OPEN, IN_WRITE, IN_CLOSE?

Yes, someone will have to figure that out.  Leaving inotify outside of
24.3 won't save this work, tho.

> For example, if you type at the shell prompt "echo foo > bar" with
> 'bar' an existing non-empty file, how many notifications do you see?
> On Windows, I get 2 or 3, depending on the details: 2 for size change
> (first one when the file is truncated, second one when "foo" is
> written into it), and sometimes also an attribute or "security" change
> (e.g., if the original file was owned by someone else).  Unless
> inotify magically merges all these into a single meaningful
> notification, the Lisp code that receives this series of notifications
> will have hard time doing TRT with them, even if the exact expected
> series are known in advance, especially if other notifications are
> interspersed with these.

Yes, but I don't see what alternative approach would save us from doing
this work.

> And we should talk _today_ about that API, because some of the
> processing needed to produce higher-level abstractions of events are
> much easier done in C than in Lisp.

Ah, so that's what it's about.  I don't see why that should be the case,
but if it is so, then indeed we might be better off postponing
the merge.  Or to only include it if Emacs is compiled with some
"experimental-inotify" flag.


        Stefan

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 03 Oct 2012 14:47:44 -0400
> Cc: emacs-devel@gnu.org
> 
> Yes, someone will have to figure that out.  Leaving inotify outside of
> 24.3 won't save this work, tho.

I'm not lobbying for leaving it out.

> > And we should talk _today_ about that API, because some of the
> > processing needed to produce higher-level abstractions of events are
> > much easier done in C than in Lisp.
> 
> Ah, so that's what it's about.  I don't see why that should be the case

Let me put it this way: we won't be able to see whether or not it is
the case, unless we discuss how to overcome these difficulties.  When
we find good solution(s), we can then see where they are implemented
best.

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> Ah, so that's what it's about.  I don't see why that should be the case
> Let me put it this way: we won't be able to see whether or not it is
> the case, unless we discuss how to overcome these difficulties.  When
> we find good solution(s), we can then see where they are implemented
> best.

I think we'll only see it with experience.  And (based on our previous
experiences with code on non-trunk branches) as long as there's no
code in trunk for it, we won't see too many people experiment with it.

So I think we should include it in 24.3 but label it as experimental.


        Stefan

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Wed, 03 Oct 2012 16:59:43 -0400
> 
> So I think we should include it in 24.3 but label it as experimental.

Is this still the intention?  If so, why isn't inotify.c and stuff in
the repository yet?

With Óscar's help, I will soon have a w32 implementation for this.
But if the inotify stuff is not going to be committed, it doesn't make
sense to commit the w32 implementation, either.

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> Date: Fri, 12 Oct 2012 15:54:34 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: emacs-devel@gnu.org
> > Date: Wed, 03 Oct 2012 16:59:43 -0400
> > 
> > So I think we should include it in 24.3 but label it as experimental.
> 
> Is this still the intention?  If so, why isn't inotify.c and stuff in
> the repository yet?

Any word on this?  Anything at all?

Re: [PATCH] Added inotify support.

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

> This is why I think we _must_ have a higher-level API in place for
> this feature to be useful in practice, even if just inotify back end
> is supported.

Agreed. For starters, IMHO we need just two events:
file-contents-changed, which comprises write, truncate, delete, and
rename operations and fulfills the requirements of autorevert-like
features, and file-metadata-changed, which is what dired-like features
need.

BTW, I would take care of the MSWindows implementation, if nobody beats
me to it.

[snip]

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 03 Oct 2012 21:33:08 +0200
> 
> BTW, I would take care of the MSWindows implementation, if nobody beats
> me to it.

Thank you.  I started working on that, too, and will probably have
some working code, still outside of Emacs, by tomorrow.  I could give
the code to you then, or we could work together.  There's a design
issue with how to signal these events to the Emacs input channels; if
you got that figured out, maybe you would like to start from that end.

Re: [PATCH] Added inotify support.

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> BTW, I would take care of the MSWindows implementation, if nobody beats
>> me to it.
>
> Thank you.  I started working on that, too, and will probably have
> some working code, still outside of Emacs, by tomorrow.  I could give
> the code to you then, or we could work together.  There's a design
> issue with how to signal these events to the Emacs input channels; if
> you got that figured out, maybe you would like to start from that end.

You can publish your branch somewhere or send a patch to me, explaining
what's missing or broken, and I'll look at it this weekend.

(Note: I'm at home with the MSWindows API, but not even a novice on
Emacs's C<->Elisp interface, so adjust your expectations on accordance
:-)

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Fri, 05 Oct 2012 15:07:19 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> BTW, I would take care of the MSWindows implementation, if nobody beats
> >> me to it.
> >
> > Thank you.  I started working on that, too, and will probably have
> > some working code, still outside of Emacs, by tomorrow.  I could give
> > the code to you then, or we could work together.  There's a design
> > issue with how to signal these events to the Emacs input channels; if
> > you got that figured out, maybe you would like to start from that end.
> 
> You can publish your branch somewhere or send a patch to me, explaining
> what's missing or broken, and I'll look at it this weekend.

I cannot send anything directly to you, because your mail server
rejects my mails, has been doing that for months if not years:

  Your message cannot be delivered to the following recipients:

    Recipient address: ofv@wanadoo.es
    Reason: Rejection greeting returned by server.
    Diagnostic code: smtp;550 Your domain is blacklisted mtaout20.012.net.il [80.179.55.166].
    Remote system: dns;inw.wanadoo.es (TCP|80.179.55.166|62212|62.36.20.20|25) (Your domain is blacklisted mtaout20.012.net.il [80.179.55.166].)

If you can whitelist me, fine; otherwise I will post here.

> (Note: I'm at home with the MSWindows API, but not even a novice on
> Emacs's C<->Elisp interface, so adjust your expectations on accordance
> :-)

In that case, I will stop working on the API part (what I have already
supports a single watched file), and instead work on incorporating
this into Emacs.  You can then extend the API interaction, or even
redesign it, if you think it's not up to speed.

Thanks.

Re: [PATCH] Added inotify support.

[Nix]

On 3 Oct 2012, Stefan Monnier spake thusly:

>>> Btw, what are Emacs use cases for using this kind of feature?
>> Apart from those mentioned on the original message (dired or magit's (or
>> vc-dir) status view) I'll like to mention auto-revert-mode.
>
> Yes, for my own use auto-revert-mode is the main one.

Of course you can't rely on inotify rather than polling, because inotify
simply silently omits all changes that come from other hosts when an fs
is mounted or exported over the network. inotify and friends are only
spying on local VFS traffic, which in my experience makes them less than
useful for most applications.

-- 
NULL && (void)

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Emacs: Lovecraft was an optimist.
> Date: Fri, 05 Oct 2012 17:55:34 +0100
> Cc: Óscar Fuentes <ofv@wanadoo.es>, emacs-devel@gnu.org
> 
> Of course you can't rely on inotify rather than polling, because inotify
> simply silently omits all changes that come from other hosts when an fs
> is mounted or exported over the network. inotify and friends are only
> spying on local VFS traffic

If this is true (I don't see this limitation in the inotify(7) man
page), it is specific to inotify.  The MS-Windows equivalents work for
exported filesystems as well.

Nevertheless, the general observation is correct: these notifications
are not 100% reliable.  Even for local filesystems, if the event queue
overflows, events are thrown away without being reported.

Re: [PATCH] Added inotify support.

[Nix]

On 5 Oct 2012, Eli Zaretskii stated:

>> Of course you can't rely on inotify rather than polling, because inotify
>> simply silently omits all changes that come from other hosts when an fs
>> is mounted or exported over the network. inotify and friends are only
>> spying on local VFS traffic
>
> If this is true (I don't see this limitation in the inotify(7) man
> page), it is specific to inotify.  The MS-Windows equivalents work for
> exported filesystems as well.

Yeah, it's specific to inotify/dnotify and anything built atop the
fsnotify Linux kernel subsystem. It is really annoying for those of us
with $HOME on NFS :( I periodically wish I had the time to write
something better, but it's quite hard.

-- 
NULL && (void)

Re: [PATCH] Added inotify support.

[Stefan Monnier]

> Of course you can't rely on inotify rather than polling, because inotify
> simply silently omits all changes that come from other hosts when an fs
> is mounted or exported over the network.

Of course, it also doesn't help when Emacs is built without inotify.
The main questions would then be:
- how often does it work?  I think this is likely to be "often" since it
  should work for most/all the "home desktop" as well as laptop use cases.
- can we reliably determine whether it will work?
  IIUC inotify works reliably for local file-systems (even if they're
  exported because the other hosts's actions will be locally performed
  by the nfsd) but not for file-systems mounted from a remote host.
  Can inotify inform Emacs that its notices won't be reliable (so Emacs
  can decide to use polling instead)?
  
> inotify and friends are only spying on local VFS traffic, which in my
> experience makes them less than useful for most applications.

Aren't they used by most GUI file managers?


        Stefan

Re: [PATCH] Added inotify support.

[Óscar Fuentes]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> inotify and friends are only spying on local VFS traffic, which in my
>> experience makes them less than useful for most applications.
>
> Aren't they used by most GUI file managers?

For ages, on my Kubuntu machines the usual file managers (Dolphin &
Konqueror) usually fail to notice changes on directories (most
annoyingly, file creation). Dunno if this is a KDE problem or something
about the underlying mechanism.

Re: [PATCH] Added inotify support.

[Nix]

On 5 Oct 2012, Stefan Monnier verbalised:

>> Of course you can't rely on inotify rather than polling, because inotify
>> simply silently omits all changes that come from other hosts when an fs
>> is mounted or exported over the network.
>
> Of course, it also doesn't help when Emacs is built without inotify.
> The main questions would then be:
> - how often does it work?  I think this is likely to be "often" since it
>   should work for most/all the "home desktop" as well as laptop use
>   cases.

Most, but certainly not all: I have a home desktop, but it so happens
that I have my home directory on a separate low-power home server so
that I can shut the desktop down when not in use. If I run Emacs on the
desktop machine, it's not going to see inotify watches on my home
directory -- or, rather, it'll see those performed by the desktop, but
not those performed by the server (e.g. email delivery).

> - can we reliably determine whether it will work?

No :( you get that subset of events that happened on the local machine
only, but even I ran Emacs on the server, it would see a local
filesystem but would *still* miss events -- those happening on the
desktop.

>   IIUC inotify works reliably for local file-systems (even if they're
>   exported because the other hosts's actions will be locally performed
>   by the nfsd)

Those don't always appear :( the nfsd doesn't do all its actions at a
level that inotify can see, alas, and certainly won't necessarily
generate the expected events for them (a touch shows up as an attrib
event locally, but an open if it's something the nfsd has done on behalf
of a remote client (!).)

>                but not for file-systems mounted from a remote host.
>   Can inotify inform Emacs that its notices won't be reliable (so Emacs
>   can decide to use polling instead)?

I don't think so. The answer in any case is 'inotify is never reliable':
even if the FS is not exported across the network and never becomes
exportted across the network, the queue can fill up and lose events and
the like.

I really really wish notify worked over the network :(

>> inotify and friends are only spying on local VFS traffic, which in my
>> experience makes them less than useful for most applications.
>
> Aren't they used by most GUI file managers?

Yes. If you have an NFS-mounted home directory, you get used to hitting
refresh in GUI file managers :(

inotify basically sucks for these reasons and others, but the kernel
people say that they don't care if it doesn't work over the network and
that it can't be made to work anyway, and the desktop people who make
use of inotify say that nobody uses NFS and everyone just has a single
laptop and your use case is out of scope, go away. Meanwhile, Windows
does file notification over the net perfectly well and has for years.

:(

-- 
NULL && (void)

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> - can we reliably determine whether it will work?
> No :( you get that subset of events that happened on the local machine
> only, but even I ran Emacs on the server, it would see a local
> filesystem but would *still* miss events -- those happening on the
> desktop.

That's a misfeature that should be easy to fix (in the kernel): any FS
should be able to indicate whether it will reliably send
inotify notices, so that client code can be told when it requests
inotify events for a particular object whether it will work reliably.

>> IIUC inotify works reliably for local file-systems (even if they're
>> exported because the other hosts's actions will be locally performed
>> by the nfsd)
> Those don't always appear :( the nfsd doesn't do all its actions at a
> level that inotify can see, alas,

If nfsd applies a modification and it's not reflected with some inotify
event, I think that's a bug that the kernel developers would want to fix.

> and certainly won't necessarily generate the expected events for them
> (a touch shows up as an attrib event locally, but an open if it's
> something the nfsd has done on behalf of a remote client (!).)

Slight semantic differences are probably unavoidable, so I think this
would likely be considered as something you need to live with.

>> Can inotify inform Emacs that its notices won't be reliable (so Emacs
>> can decide to use polling instead)?
> I don't think so. The answer in any case is 'inotify is never reliable':
> even if the FS is not exported across the network and never becomes
> exportted across the network, the queue can fill up and lose events and
> the like.

Can't that be fixed by making it possible to "compress" the queue,
e.g. replace a bunch of events on FILE with a single "something happened
to FILE", or in the worst case replace all the events with a single
"queue filled up, all the files might have been modified in arbitrary
ways".
This would let it be reliable even in the face of lost events.

>> Aren't they used by most GUI file managers?
> Yes. If you have an NFS-mounted home directory, you get used to hitting
> refresh in GUI file managers :(

So there's nothing for Emacs to worry about, because Emacs won't fix
those issues.


        Stefan

Re: [PATCH] Added inotify support.

[Nix]

On 6 Oct 2012, Stefan Monnier stated:

>>> - can we reliably determine whether it will work?
>> No :( you get that subset of events that happened on the local machine
>> only, but even I ran Emacs on the server, it would see a local
>> filesystem but would *still* miss events -- those happening on the
>> desktop.
>
> That's a misfeature that should be easy to fix (in the kernel): any FS
> should be able to indicate whether it will reliably send
> inotify notices, so that client code can be told when it requests
> inotify events for a particular object whether it will work reliably.

Unfortunately not, because e.g. a perfectly normal local ext4 filesystem
can be unreliable if it is NFS-exported -- worse, it can be *partially*
unreliable if only a subtree is exported, and it can flip between
reliable and potentially unreliable on the fly as you export or unexport
filesystems.

>>> IIUC inotify works reliably for local file-systems (even if they're
>>> exported because the other hosts's actions will be locally performed
>>> by the nfsd)
>> Those don't always appear :( the nfsd doesn't do all its actions at a
>> level that inotify can see, alas,
>
> If nfsd applies a modification and it's not reflected with some inotify
> event, I think that's a bug that the kernel developers would want to fix.

They don't care. If an event doesn't come through the normal VFS layers,
inotify won't see it, and that's that.

>>> Can inotify inform Emacs that its notices won't be reliable (so Emacs
>>> can decide to use polling instead)?
>> I don't think so. The answer in any case is 'inotify is never reliable':
>> even if the FS is not exported across the network and never becomes
>> exportted across the network, the queue can fill up and lose events and
>> the like.
>
> Can't that be fixed by making it possible to "compress" the queue,
> e.g. replace a bunch of events on FILE with a single "something happened
> to FILE", or in the worst case replace all the events with a single
> "queue filled up, all the files might have been modified in arbitrary
> ways".

Theoretically, yes, only that would require you to look at previously
queued events, which soon gets you into locking horrors. (As a first
approximation, *anything* to do with the filesystem soon gets you into
locking horrors.)

> This would let it be reliable even in the face of lost events.
>
>>> Aren't they used by most GUI file managers?
>> Yes. If you have an NFS-mounted home directory, you get used to hitting
>> refresh in GUI file managers :(
>
> So there's nothing for Emacs to worry about, because Emacs won't fix
> those issues.

Oh yes, indeed. I'm just arguing against ripping out polling and
replacing it with inotify, really: we need at least a customization knob
for people who know their filesystems are NFS-exported or otherwise
network-affected to tweak to say 'poll, please'.

-- 
NULL && (void)

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> If nfsd applies a modification and it's not reflected with some inotify
>> event, I think that's a bug that the kernel developers would want to fix.
> They don't care. If an event doesn't come through the normal VFS layers,
> inotify won't see it, and that's that.

But that's a bug in nfsd, still.

> Oh yes, indeed. I'm just arguing against ripping out polling and
> replacing it with inotify, really:

I have no intention to rip out polling.  Rather I hope that the
higher-level API we come up with can be implemented by any of Windows's
low-level API, inotify, MacOSX's equvalent, or polling.


        Stefan

Re: [PATCH] Added inotify support.

[Nix]

On 6 Oct 2012, Stefan Monnier uttered the following:

>>> If nfsd applies a modification and it's not reflected with some inotify
>>> event, I think that's a bug that the kernel developers would want to fix.
>> They don't care. If an event doesn't come through the normal VFS layers,
>> inotify won't see it, and that's that.
>
> But that's a bug in nfsd, still.

Well, yes, I think so, and you think so, but the Linux kernel hackers do
not think so :(
from userspace via the VFS: other activity might

>> Oh yes, indeed. I'm just arguing against ripping out polling and
>> replacing it with inotify, really:
>
> I have no intention to rip out polling.  Rather I hope that the
> higher-level API we come up with can be implemented by any of Windows's
> low-level API, inotify, MacOSX's equvalent, or polling.

... which is what I hoped to hear. I've seen several projects rip out
polling because inotify can do anything, and end up with something that
worked worse for NFS users than what they had before. (NFSv4 could in
theory implement something like inotify, but for earlier versions,
polling is really all you could hope to do. Well, that or have some
parallel daemon inotifying on the clients and informing the server of
inotify activity, and vice versa...)

-- 
NULL && (void)

Re: [PATCH] Added inotify support.

[Achim Gratz]

Nix writes:
>> But that's a bug in nfsd, still.
>
> Well, yes, I think so, and you think so, but the Linux kernel hackers do
> not think so :(

This behaviour doesn't have anything to do with nfsd (other than the
stateless nature of NFS and the way it implements that) and you can
trigger it quite easily without nfsd involved.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>>> But that's a bug in nfsd, still.
>> Well, yes, I think so, and you think so, but the Linux kernel hackers do
>> not think so :(
> This behaviour doesn't have anything to do with nfsd (other than the
> stateless nature of NFS and the way it implements that) and you can
> trigger it quite easily without nfsd involved.

Which behavior?


        Stefan

Re: [PATCH] Added inotify support.

[Achim Gratz]

Stefan Monnier writes:
>>>> But that's a bug in nfsd, still.
>>> Well, yes, I think so, and you think so, but the Linux kernel hackers do
>>> not think so :(
>> This behaviour doesn't have anything to do with nfsd (other than the
>> stateless nature of NFS and the way it implements that) and you can
>> trigger it quite easily without nfsd involved.
>
> Which behavior?

The nfsd is made stateless by using hidden hardlinks to the files it
uses and then uses their inode numbers to manipulate these files (all
IIRC).  Inotify watches files based on their names and maps that to
inodes using a cache, which can be out of sync or even drop the
association between inode and filename.  Any further operation on that
file will then not be seen by inotify unless you re-create the watch.
It works somewhat more reliably if you monitor individual files rather
than directories, but that isn't foolproof either.


Regards
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: [PATCH] Added inotify support.

[Stephen J. Turnbull]

Nix writes:
 > On 6 Oct 2012, Stefan Monnier uttered the following:

 > > I have no intention to rip out polling.  Rather I hope that the
 > > higher-level API we come up with can be implemented by any of Windows's
 > > low-level API, inotify, MacOSX's equvalent, or polling.
 > 
 > ... which is what I hoped to hear.

Plus combinations of the above, as needed.  Right?

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> > I have no intention to rip out polling.  Rather I hope that the
>> > higher-level API we come up with can be implemented by any of Windows's
>> > low-level API, inotify, MacOSX's equvalent, or polling.
>> ... which is what I hoped to hear.
> Plus combinations of the above, as needed.  Right?

If someone bothers to implement it, why not, tho I'm not sure how
important that would be (unless that patch can automatically figure out
when to use which).


        Stefan

Re: [PATCH] Added inotify support.

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> > I have no intention to rip out polling.  Rather I hope that the
>>> > higher-level API we come up with can be implemented by any of Windows's
>>> > low-level API, inotify, MacOSX's equvalent, or polling.
>>> ... which is what I hoped to hear.
>> Plus combinations of the above, as needed.  Right?
>
> If someone bothers to implement it, why not, tho I'm not sure how
> important that would be (unless that patch can automatically figure out
> when to use which).

In theory, it is possible to automatically figure out when to use
polling and when to use the OS notification features. But if inotify
doesn't work on some filesystems, we need a method for detecting those
filesystems. Does inotify report "I don't work on this" when it is used
on those unsupported filesystems?

Eli and I discussed the high level Lisp API for filesystem
notifications. I think it is compatible with polling, in the sense that
it can transparently fall back to polling, once the issue mentioned on
the previous paragraph is addressed.

Re: [PATCH] Added inotify support.

[Stefan Monnier]

> Eli and I discussed the high level Lisp API for filesystem
> notifications. I think it is compatible with polling, in the sense that
> it can transparently fall back to polling, once the issue mentioned on
> the previous paragraph is addressed.

AFAIK inotify does not provide the needed info, so for now the choice
should be based on a global variable.


        Stefan

Re: [PATCH] Added inotify support.

[Stephen J. Turnbull]

Stefan Monnier writes:
 > >> > I have no intention to rip out polling.  Rather I hope that the
 > >> > higher-level API we come up with can be implemented by any of Windows's
 > >> > low-level API, inotify, MacOSX's equvalent, or polling.
 > >> ... which is what I hoped to hear.
 > > Plus combinations of the above, as needed.  Right?
 > 
 > If someone bothers to implement it, why not, tho I'm not sure how
 > important that would be

According to nix, inotify is unreliable, end of discussion.

If so, it may make sense to back up inotify with polling, although at
a greatly reduced rate.  Also, files may get moved across filesystems
which support different mechanisms, etc, etc.  So allowing different
files to get notifications in different ways, and perhaps even
changing backends on the fly may be useful/necessary for maximum
reliability.

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Mon, 08 Oct 2012 16:07:18 +0900
> Cc: Nix <nix@esperi.org.uk>, Óscar Fuentes <ofv@wanadoo.es>,
> 	emacs-devel@gnu.org
> 
> According to nix, inotify is unreliable, end of discussion.
> 
> If so, it may make sense to back up inotify with polling, although at
> a greatly reduced rate.  Also, files may get moved across filesystems
> which support different mechanisms, etc, etc.  So allowing different
> files to get notifications in different ways, and perhaps even
> changing backends on the fly may be useful/necessary for maximum
> reliability.

If one wants a 100% reliability, polling cannot be done at slower
rates without degrading response times.  Some applications might not
like the long response times.  And of course, you won't know when the
notifications are less reliable than you would like to, so selectively
increasing the polling rate in those problematic cases seems
impossible.

Moreover, both inotify and the equivalent Windows APIs are documented
to lose notifications on a busy filesystem, so even a perfectly local
file/directory cannot be watched with 100% reliability.

And that is even before we think about the effects of the internal
Emacs mechanisms of handling these events.  E.g., if there's a lot of
input from the keyboard and the window system, and if some heavy Lisp
is running, it is quite possible to have the file-notification event
be stuck in limbo for some time, before Emacs examines it and takes
action.

IOW, this will work well "most of the time", but that's about it.

Re: [PATCH] Added inotify support.

[Jim Meyering]

Nix wrote:
...
> Oh yes, indeed. I'm just arguing against ripping out polling and
> replacing it with inotify, really: we need at least a customization knob
> for people who know their filesystems are NFS-exported or otherwise
> network-affected to tweak to say 'poll, please'.

GNU tail's -f support has addressed precisely this problem
by distinguishing between file system types for which inotify
is known to work and those for which it does not work reliably.
It uses inotify only for a file system on which it is known to work,
polls for others, and warns about if the file system type is not known.

To see the list, look at coreutils/src/stat.c's S_MAGIC_* case
statements.  The ones with "local" in the comment work with inotify,
the ones with admittedly poorly named "remote" in the comment
require that tail -f support use polling.

The only problem is that the list is a moving target.
New file system types arise regularly.
To give you an idea of the pace, there have been two additions
since the latest release, which was less than two months ago.
Here's the NEWS entry for those:

  stat and tail know about ZFS, VZFS and VMHGFS.  stat -f --format=%T now
  reports the file system type, and tail -f now uses inotify for files on
  ZFS and VZFS file systems, rather than the default (for unknown file
  system types) of issuing a warning and reverting to polling.  tail -f
  still uses polling for files on VMHGFS file systems.

Re: [PATCH] Added inotify support.

[Stephen J. Turnbull]

Nix writes:

 > [Only capable of] spying on local VFS traffic, which in my
 > experience makes them less than useful for most applications.

These days, though (as I am very tired of hearing but seems to be
true) most people who use Emacs don't mount remote filesystems.

Re: [PATCH] Added inotify support.

[Andreas Schwab]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> These days, though (as I am very tired of hearing but seems to be
> true) most people who use Emacs don't mount remote filesystems.

I think NAS boxes are still common.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [PATCH] Added inotify support.

[Nix]

On 6 Oct 2012, Andreas Schwab verbalised:

> "Stephen J. Turnbull" <stephen@xemacs.org> writes:
>
>> These days, though (as I am very tired of hearing but seems to be
>> true) most people who use Emacs don't mount remote filesystems.
>
> I think NAS boxes are still common.

If anything they are more common than they used to be in the large parts
of the world where the cost of power is shooting up, not going down:
NASes often use quite a lot less power than a desktop with the same
number of disks would, and much less than a headless server with the
same number of disks in addition to the desktop. (Though I actually do
have a headless server with a disk farm, but I know that is pretty
unusual for a home user.)

-- 
NULL && (void)

Re: [PATCH] Added inotify support.

[Stephen J. Turnbull]

Nix writes:

 > If anything they are more common than they used to be in the large
 > parts of the world where the cost of power is shooting up, not
 > going down:

Makes sense.  Unlike the IT and energy behavior of my host country,
unfortunately. :-P  Sorry for spreading disinformation (I got it from
the same people you did, anyway. ;-)

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> If anything they are more common than they used to be in the large
>> parts of the world where the cost of power is shooting up, not
>> going down:
> Makes sense.  Unlike the IT and energy behavior of my host country,
> unfortunately. :-P  Sorry for spreading disinformation (I got it from
> the same people you did, anyway. ;-)

BTW, I do have a low-power home-server with a large disk, so that my
desktop can sleep to save energy.  But I don't really access it via NFS.

Basically, I stopped sharing my home directory via NFS years ago,
replacing it with a VCS with a remote repository.

So other than collections of large files (typically multimedia), the
increase in disk sizes has made NFS-sharing much less important for me.

OTOH, if you know of a good file-system that can provide a model
half-way between NFS and "VCS + remote repository", so that I get the
best of both worlds (i.e. the transparent synchronization of NFS, along
with the reliability and disconnected operation of VCS), I'd love to
know about it.


        Stefan

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 03 Oct 2012 01:46:33 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Btw, what are Emacs use cases for using this kind of feature?
> 
> Apart from those mentioned on the original message (dired or magit's (or
> vc-dir) status view) I'll like to mention auto-revert-mode.

All those need either a single file to be watched or don't need any
details about what exactly changed (dired).  That's a far cry from
what's being proposed.

> You are leaving this case to extremes. Emacs already allows you to do
> very silly things. Let's not throw out useful features simply because
> they might be abused.

I didn't suggest to throw away anything.  I suggested to think about
the API we provide, so that it could be simpler and safer.  Abuse has
nothing to do with naivette.

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> Date: Mon, 01 Oct 2012 16:09:55 +0200
> Cc: Leo <sdl.web@gmail.com>, emacs-devel@gnu.org
> 
> On Monday 01 October 2012 00:38:09 Stefan Monnier wrote:
> > If there's a good chance this won't work without breaking compatibility,
> > maybe a better option is to provide a low-level API that maps very
> > closely to inotify and then an Elisp layer on top which abstracts away
> > differences between different systems.  In that case we can install the
> > inotify support right away while we're still experimenting with the
> > higher-level abstraction.
> 
> That's probably the best approach here.  I changed the patch to provide a low
> level inotify interface.  However I did not provide an inotify_init(2) like 
> function and instead initialization and closing of the inotify handle is done
> internally.  I don't think that this should be exposed to elisp even in the 
> low level interface.
> 
> > But if they're unlikely to be important in practice, then
> > I guess the current solution might be acceptable.
> 
> I think we are safe.  I added that `equal' should be used to compare cookies.  
> So we can easily change it without breaking the API.
> 
> > I think the cleaner option is to define a new object type for it.
> > It could be either a new Lisp_Misc type, so you can make them print as
> > something like "#<file-watcher NNN>" (take a look at "enum
> > Lisp_Misc_Type" and "union Lisp_Misc" in src/lisp.h for starters; adding
> > a new type will require adding corresponding branches to the switch
> > statements in alloc.c and in print.c).
> 
> That sounds like the best option.  I haven't implemented it yet.  Is it 
> possible to make the Lisp_Misc_Type comparable with `equal'? Because the 
> watch-descriptor has to be comparable.

Any news on this?  The corresponding w32 implementation collects dust
in my local branch, waiting for the inotify based implementation to be
committed.

Re: [PATCH] Added inotify support.

[Stefan Monnier]

> Any news on this?  The corresponding w32 implementation collects dust
> in my local branch, waiting for the inotify based implementation to be
> committed.

Please go ahead and install both in trunk,


        Stefan

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Rüdiger Sonderfeld <ruediger@c-plusplus.de>,
>         emacs-devel@gnu.org, sdl.web@gmail.com
> Date: Mon, 03 Dec 2012 12:18:50 -0500
> 
> > Any news on this?  The corresponding w32 implementation collects dust
> > in my local branch, waiting for the inotify based implementation to be
> > committed.
> 
> Please go ahead and install both in trunk,

Done.

Re: [PATCH] Added inotify support.

[Stefan Monnier]

>> > Any news on this?  The corresponding w32 implementation collects dust
>> > in my local branch, waiting for the inotify based implementation to be
>> > committed.
>> Please go ahead and install both in trunk,

Thank you very much,


        Stefan

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Rüdiger Sonderfeld <ruediger@c-plusplus.de>
> Date: Mon, 01 Oct 2012 16:09:55 +0200
> Cc: Leo <sdl.web@gmail.com>, emacs-devel@gnu.org
> 
> On Monday 01 October 2012 00:38:09 Stefan Monnier wrote:
> > If there's a good chance this won't work without breaking compatibility,
> > maybe a better option is to provide a low-level API that maps very
> > closely to inotify and then an Elisp layer on top which abstracts away
> > differences between different systems.  In that case we can install the
> > inotify support right away while we're still experimenting with the
> > higher-level abstraction.
> 
> That's probably the best approach here.  I changed the patch to provide a low
> level inotify interface.  However I did not provide an inotify_init(2) like 
> function and instead initialization and closing of the inotify handle is done
> internally.  I don't think that this should be exposed to elisp even in the 
> low level interface.
> 
> > But if they're unlikely to be important in practice, then
> > I guess the current solution might be acceptable.
> 
> I think we are safe.  I added that `equal' should be used to compare cookies.  
> So we can easily change it without breaking the API.

Committed (with necessary fixes and additions, like ChangeLog entries
and NEWS) as trunk revision 111171.

Thanks.

Re: [PATCH] Added inotify support.

[Rüdiger Sonderfeld]

Hello,

On Monday 10 December 2012 13:52:48 Eli Zaretskii wrote:
> Committed (with necessary fixes and additions, like ChangeLog entries
> and NEWS) as trunk revision 111171.
> 
> Thanks.

Great news! Thanks.

I'm currently very busy.  But I'll try to find some time between Christmas and 
New Year to look into the existing issues.

Regards,
Rüdiger

Re: [PATCH] Added inotify support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> Committed (with necessary fixes and additions, like ChangeLog entries
> and NEWS) as trunk revision 111171.

I've played a little bit with this. Looks, like not all events are
handled by the callback. Testcase:

In a shell, a have applied

$ inotifywait -mq /tmp/123

In another shell, I have started Emacs

$ emacs --eval "(inotify-add-watch \"/tmp/123\" t (lambda (&rest rest) (message \"callback %s\" rest)))"

Then I have modified&saved /tmp/123. In the first shell, I see

/tmp/123 MODIFY 
/tmp/123 OPEN 
/tmp/123 MODIFY 
/tmp/123 CLOSE_WRITE,CLOSE 

But in Emacs' *Messages* buffer, there's only

callback ((1 (modify) 0 nil))
callback ((1 (close-write) 0 nil))

What do I miss?

> Thanks.

Best regards, Michael.

Re: [PATCH] Added inotify support.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Rüdiger Sonderfeld <ruediger@c-plusplus.de>,
>   emacs-devel@gnu.org,  monnier@iro.umontreal.ca,  sdl.web@gmail.com
> Date: Tue, 11 Dec 2012 08:43:35 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Committed (with necessary fixes and additions, like ChangeLog entries
> > and NEWS) as trunk revision 111171.
> 
> I've played a little bit with this. Looks, like not all events are
> handled by the callback. Testcase:
> 
> In a shell, a have applied
> 
> $ inotifywait -mq /tmp/123
> 
> In another shell, I have started Emacs
> 
> $ emacs --eval "(inotify-add-watch \"/tmp/123\" t (lambda (&rest rest) (message \"callback %s\" rest)))"
> 
> Then I have modified&saved /tmp/123. In the first shell, I see
> 
> /tmp/123 MODIFY 
> /tmp/123 OPEN 
> /tmp/123 MODIFY 
> /tmp/123 CLOSE_WRITE,CLOSE 
> 
> But in Emacs' *Messages* buffer, there's only
> 
> callback ((1 (modify) 0 nil))
> callback ((1 (close-write) 0 nil))
> 
> What do I miss?

No idea.  Perhaps Rüdiger could help.  Or step with a debugger through
the code and see what's going on.

Re: [PATCH] Added inotify support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> I've played a little bit with this. Looks, like not all events are
>> handled by the callback. Testcase:
>> 
>> In a shell, a have applied
>> 
>> $ inotifywait -mq /tmp/123
>> 
>> In another shell, I have started Emacs
>> 
>> $ emacs --eval "(inotify-add-watch \"/tmp/123\" t (lambda (&rest
>> rest) (message \"callback %s\" rest)))"
>> 
>> Then I have modified&saved /tmp/123. In the first shell, I see
>> 
>> /tmp/123 MODIFY 
>> /tmp/123 OPEN 
>> /tmp/123 MODIFY 
>> /tmp/123 CLOSE_WRITE,CLOSE 
>> 
>> But in Emacs' *Messages* buffer, there's only
>> 
>> callback ((1 (modify) 0 nil))
>> callback ((1 (close-write) 0 nil))
>> 
>> What do I miss?
>
> No idea.  Perhaps Rüdiger could help.  Or step with a debugger through
> the code and see what's going on.

I've fixed this in the trunk.

There is still something not clear to me: An inotify event

  /tmp/123 CLOSE_WRITE,CLOSE 

is transformed into an Emacs event

  (file-notify (1 (close-write) 0 nil) callback)

Is there a reason, that CLOSE (and also MOVE) are not handled in
mask_to_aspects?

Best regards, Michael.

Re: [PATCH] Added inotify support.

[Rüdiger Sonderfeld]

On Tuesday 11 December 2012 17:36:41 Michael Albinus wrote:
> I've fixed this in the trunk.

Thanks!

> There is still something not clear to me: An inotify event
> 
>   /tmp/123 CLOSE_WRITE,CLOSE
> 
> is transformed into an Emacs event
> 
>   (file-notify (1 (close-write) 0 nil) callback)
> 
> Is there a reason, that CLOSE (and also MOVE) are not handled in
> mask_to_aspects?

IN_CLOSE and IN_MOVE are not separate events but convenience macros.

From the notify(7) manpage

    Two additional convenience macros are IN_MOVE, which equates to 
IN_MOVED_FROM|IN_MOVED_TO, and IN_CLOSE, which equates to IN_CLOSE_WRITE|
IN_CLOSE_NOWRITE.

Regards,
Rüdiger

Re: [PATCH] Added inotify support.

[Michael Albinus]

Rüdiger Sonderfeld <ruediger@c-plusplus.de> writes:

>> Is there a reason, that CLOSE (and also MOVE) are not handled in
>> mask_to_aspects?
>
> IN_CLOSE and IN_MOVE are not separate events but convenience macros.
>
> From the notify(7) manpage
>
>     Two additional convenience macros are IN_MOVE, which equates to
> IN_MOVED_FROM|IN_MOVED_TO, and IN_CLOSE, which equates to IN_CLOSE_WRITE|
> IN_CLOSE_NOWRITE.

I see. I will filter them out in the Trammp handler as well.

Maybe you could precise it in the docstring of inotify-add-watch. Close,
move and all-events are separated from the other aspects, but there is
no reasoning why.

And midterm it might be useful to describe the inotify mecahnism in the
Elisp manual in more detail.

> Regards,
> Rüdiger

Best regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> If there's a good chance this won't work without breaking compatibility,
> maybe a better option is to provide a low-level API that maps very
> closely to inotify and then an Elisp layer on top which abstracts away
> differences between different systems.  In that case we can install the
> inotify support right away while we're still experimenting with the
> higher-level abstraction.

This has inspired me to check, whether we could extend inotify support
for remote files, via Tramp. I know it might have performance issues,
but I'm curious :-)

A first shot to write a file handler for `inotify-add-watch' was quite
easy. It works pretty well, and would need only some polishing before
being installed.

For `inotify-rm-watch' that's not possible right now. It takes as
argument WATCH-DESCRIPTOR, which is not a file name, and which does not
contain a file name.

Could we extend the interface of `inotify-rm-watch' to add a file name?
It would be sufficient already, if WATCH-DESCRIPTOR would contain a file
name as first element of its structure.

>         Stefan

Best regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Tue, 11 Dec 2012 08:54:54 +0100
> Cc: Rüdiger Sonderfeld <ruediger@c-plusplus.de>,
> 	Leo <sdl.web@gmail.com>, emacs-devel@gnu.org
> 
> For `inotify-rm-watch' that's not possible right now. It takes as
> argument WATCH-DESCRIPTOR, which is not a file name, and which does not
> contain a file name.
> 
> Could we extend the interface of `inotify-rm-watch' to add a file name?

Why do you need an extension?  This is Lisp: we could pass the file
name _as_ the descriptor.  Since the remote file handler will take
care of the call anyway, it will assign the correct meaning to the
descriptor.

Am I missing something?

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> For `inotify-rm-watch' that's not possible right now. It takes as
>> argument WATCH-DESCRIPTOR, which is not a file name, and which does not
>> contain a file name.
>> 
>> Could we extend the interface of `inotify-rm-watch' to add a file name?
>
> Why do you need an extension?  This is Lisp: we could pass the file
> name _as_ the descriptor.  Since the remote file handler will take
> care of the call anyway, it will assign the correct meaning to the
> descriptor.

Sure, that would be sufficient for Tramp. I had the impression, that a
more complex structure as descriptor was chosen, because there could be
several `inotify-add-watch' calls for the *same* file, being different
in the aspect or the callback function.

Best regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: monnier@iro.umontreal.ca,  ruediger@c-plusplus.de,  sdl.web@gmail.com,  emacs-devel@gnu.org
> Date: Tue, 11 Dec 2012 09:19:10 +0100
> 
> >> Could we extend the interface of `inotify-rm-watch' to add a file name?
> >
> > Why do you need an extension?  This is Lisp: we could pass the file
> > name _as_ the descriptor.  Since the remote file handler will take
> > care of the call anyway, it will assign the correct meaning to the
> > descriptor.
> 
> Sure, that would be sufficient for Tramp. I had the impression, that a
> more complex structure as descriptor was chosen, because there could be
> several `inotify-add-watch' calls for the *same* file, being different
> in the aspect or the callback function.

Then Tramp could maintain its own data structure for watched files,
and give each watch a unique identifier, like (FILENAME . NUMBER), or
just NUMBER, or whatever.

IOW, the DESCRIPTOR is an opaque data type, which the implementation
back-end, and the back-end alone, can and should interpret.

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> Then Tramp could maintain its own data structure for watched files,
> and give each watch a unique identifier, like (FILENAME . NUMBER), or
> just NUMBER, or whatever.

That I do already in my inotify-add-watch handler. Of course.

> IOW, the DESCRIPTOR is an opaque data type, which the implementation
> back-end, and the back-end alone, can and should interpret.

The point is, that something must trigger Tramp. In inotify-add-watch, I
have added the following code (shortened, there's more):

--8<---------------cut here---------------start------------->8---
  /* If the file name has special constructs in it,
     call the corresponding file handler.  */
  handler = Ffind_file_name_handler (file_name, Qinotify_add_watch);
  if (!NILP (handler))
    {
      return call4 (handler, Qinotify_add_watch, file_name, aspect,
     callback);
    }
--8<---------------cut here---------------end--------------->8---

In inotify-rm-watch I couldn't add similar lines, because file_name is
unknown. My proposal is either to add file_name as first argument of
inotify-rm-watch, or to declare WATCH-DESCRIPTOR as a cons cell, which
car is always the file name.

Best regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: monnier@iro.umontreal.ca,  ruediger@c-plusplus.de,  sdl.web@gmail.com,  emacs-devel@gnu.org
> Date: Tue, 11 Dec 2012 09:44:55 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Then Tramp could maintain its own data structure for watched files,
> > and give each watch a unique identifier, like (FILENAME . NUMBER), or
> > just NUMBER, or whatever.
> 
> That I do already in my inotify-add-watch handler. Of course.
> 
> > IOW, the DESCRIPTOR is an opaque data type, which the implementation
> > back-end, and the back-end alone, can and should interpret.
> 
> The point is, that something must trigger Tramp. In inotify-add-watch, I
> have added the following code (shortened, there's more):

Sorry, I misunderstood the issue.  See below.

> --8<---------------cut here---------------start------------->8---
>   /* If the file name has special constructs in it,
>      call the corresponding file handler.  */
>   handler = Ffind_file_name_handler (file_name, Qinotify_add_watch);
>   if (!NILP (handler))
>     {
>       return call4 (handler, Qinotify_add_watch, file_name, aspect,
>      callback);
>     }
> --8<---------------cut here---------------end--------------->8---
> 
> In inotify-rm-watch I couldn't add similar lines, because file_name is
> unknown. My proposal is either to add file_name as first argument of
> inotify-rm-watch, or to declare WATCH-DESCRIPTOR as a cons cell, which
> car is always the file name.

IMO, this design is wrong.  Tramp is just one more back-end for this
feature, in addition to two others: inotify and w32notify.  So I think
Tramp handlers should be called from a higher-level code, one that
calls whichever back-end is appropriate.  Otherwise, we will need to
implement the Tramp support twice, in 2 different sets of primitives.

Which, of course, goes back to the kind of design discussion I
suggested to have at the time, where we were supposed to consider
various alternatives and eventually agree on some higher-level APIs.
Jumping to coding right away is IMO not the right way.  E.g.,
currently there are subtle but very real differences between the 2
back-ends: w32notify doesn't accept t or a lone symbol as the 2nd
argument (it insists on getting a list); the list of supported watch
types is entirely different; and the w32 back-ends actually watches
the entire directory of the file, not just that file.

IOW, this feature is not really ready for Tramp-ization, or for
user-land in general.  Stefan wanted people to experiment with this
and gather experience, before we know enough to discuss how to make it
user- and Lisp-friendly.

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> Date: Tue, 11 Dec 2012 11:39:15 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: ruediger@c-plusplus.de, emacs-devel@gnu.org, monnier@iro.umontreal.ca,
> 	sdl.web@gmail.com
> 
> > --8<---------------cut here---------------start------------->8---
> >   /* If the file name has special constructs in it,
> >      call the corresponding file handler.  */
> >   handler = Ffind_file_name_handler (file_name, Qinotify_add_watch);
> >   if (!NILP (handler))
> >     {
> >       return call4 (handler, Qinotify_add_watch, file_name, aspect,
> >      callback);
> >     }
> > --8<---------------cut here---------------end--------------->8---
> > 
> > In inotify-rm-watch I couldn't add similar lines, because file_name is
> > unknown. My proposal is either to add file_name as first argument of
> > inotify-rm-watch, or to declare WATCH-DESCRIPTOR as a cons cell, which
> > car is always the file name.
> 
> IMO, this design is wrong.  Tramp is just one more back-end for this
> feature, in addition to two others: inotify and w32notify.  So I think
> Tramp handlers should be called from a higher-level code, one that
> calls whichever back-end is appropriate.  Otherwise, we will need to
> implement the Tramp support twice, in 2 different sets of primitives.

Moreover, Tramp shouldn't use Qinotify_* symbols, but some (currently
non-existent) platform-independent symbols, e.g. Qfile_notify_*.
Otherwise, a Windows user will not be able to use this feature in
conjunction with remote files residing on Posix hosts, because
Qinotify_* are only defined when Emacs is built with local inotify
support, which is impossible on Windows.

Again, this calls for some design discussions.

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> Moreover, Tramp shouldn't use Qinotify_* symbols, but some (currently
> non-existent) platform-independent symbols, e.g. Qfile_notify_*.
> Otherwise, a Windows user will not be able to use this feature in
> conjunction with remote files residing on Posix hosts, because
> Qinotify_* are only defined when Emacs is built with local inotify
> support, which is impossible on Windows.

Sure. It's just a matter of the primitive function, Tramp is called
from.

Best regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> In inotify-rm-watch I couldn't add similar lines, because file_name is
>> unknown. My proposal is either to add file_name as first argument of
>> inotify-rm-watch, or to declare WATCH-DESCRIPTOR as a cons cell, which
>> car is always the file name.
>
> IMO, this design is wrong.  Tramp is just one more back-end for this
> feature, in addition to two others: inotify and w32notify.  So I think
> Tramp handlers should be called from a higher-level code, one that
> calls whichever back-end is appropriate.  Otherwise, we will need to
> implement the Tramp support twice, in 2 different sets of primitives.

Maybe. But this would require a common understanding of the API between
all involved backends.

Honestly, I don't plan Tramp handlers for w32notify-add-watch and
w32notify-rm-watch. I wouldn't know how to implement, neither if the
local host runs MS Windows, nor if the remote host runs MS Windows. At
least this thread does not exist :-)

For me, it would be sufficient to regard Tramp handlers as an extension
to inotify-add-watch and inotify-rm-watch, and not as a third backend in
parallel to two others.

> IOW, this feature is not really ready for Tramp-ization, or for
> user-land in general.  Stefan wanted people to experiment with this
> and gather experience, before we know enough to discuss how to make it
> user- and Lisp-friendly.

It was never planned for Tramp; as I said I'm curious. And we got what
Stefan has asked for: "people to experiment with this and gather
experience".

Best regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Tue, 11 Dec 2012 11:24:38 +0100
> Cc: ruediger@c-plusplus.de, emacs-devel@gnu.org, monnier@iro.umontreal.ca,
> 	sdl.web@gmail.com
> 
> Honestly, I don't plan Tramp handlers for w32notify-add-watch and
> w32notify-rm-watch. I wouldn't know how to implement, neither if the
> local host runs MS Windows, nor if the remote host runs MS Windows. At
> least this thread does not exist :-)

That was not my intent.  The intent is to let Windows users use this
feature for remote files residing on GNU/Linux hosts.

> For me, it would be sufficient to regard Tramp handlers as an extension
> to inotify-add-watch and inotify-rm-watch, and not as a third backend in
> parallel to two others.

But then inotify-*-watch primitives will have to be able to be
compiled on platforms where HAVE_INOTIFY is not defined.  Currently,
this is impossible, so what you regard as sufficient would limit this
feature to GNU/Linux users and remote files on other GNU/Linux
systems.  I don't think such a limitation is desirable.

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> For me, it would be sufficient to regard Tramp handlers as an extension
>> to inotify-add-watch and inotify-rm-watch, and not as a third backend in
>> parallel to two others.
>
> But then inotify-*-watch primitives will have to be able to be
> compiled on platforms where HAVE_INOTIFY is not defined.  Currently,
> this is impossible, so what you regard as sufficient would limit this
> feature to GNU/Linux users and remote files on other GNU/Linux
> systems.  I don't think such a limitation is desirable.

I've implemented a quick shot what's possible just now. If we have a
more general solution, I'm open. Your proposal with a unified API would
do it, but it means we need to unify. Now we have seen in practice first
problems.

Since I'm not an inotify expert, I would let the proposal for such an
API to somebody else. I would participate if it comes to practical
details for Tramp implementation.

If support for remote files is desired.

Bes regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Tue, 11 Dec 2012 14:17:28 +0100
> Cc: ruediger@c-plusplus.de, sdl.web@gmail.com, monnier@iro.umontreal.ca,
> 	emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> For me, it would be sufficient to regard Tramp handlers as an extension
> >> to inotify-add-watch and inotify-rm-watch, and not as a third backend in
> >> parallel to two others.
> >
> > But then inotify-*-watch primitives will have to be able to be
> > compiled on platforms where HAVE_INOTIFY is not defined.  Currently,
> > this is impossible, so what you regard as sufficient would limit this
> > feature to GNU/Linux users and remote files on other GNU/Linux
> > systems.  I don't think such a limitation is desirable.
> 
> I've implemented a quick shot what's possible just now.

I understand, and thank you for doing this.

Re: [PATCH] Added basic file system watching support.

[Leo]

On 2012-12-11 20:51 +0800, Eli Zaretskii wrote:
> That was not my intent.  The intent is to let Windows users use this
> feature for remote files residing on GNU/Linux hosts.

Sorry to jump in here. I seem to remember the original patch made use of
kqueue and worked on BSD systems too. What is happening there? Thanks.

Leo

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> From: Leo <sdl.web@gmail.com>
> Date: Wed, 12 Dec 2012 09:09:26 +0800
> 
> On 2012-12-11 20:51 +0800, Eli Zaretskii wrote:
> > That was not my intent.  The intent is to let Windows users use this
> > feature for remote files residing on GNU/Linux hosts.
> 
> Sorry to jump in here. I seem to remember the original patch made use of
> kqueue and worked on BSD systems too. What is happening there? Thanks.

You are talking about another patch and another contributor, IIRC.

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> IMO, this design is wrong.  Tramp is just one more back-end for this
> feature, in addition to two others: inotify and w32notify.  So I think
> Tramp handlers should be called from a higher-level code, one that
> calls whichever back-end is appropriate.  Otherwise, we will need to
> implement the Tramp support twice, in 2 different sets of primitives.
>
> Which, of course, goes back to the kind of design discussion I
> suggested to have at the time, where we were supposed to consider
> various alternatives and eventually agree on some higher-level APIs.
> Jumping to coding right away is IMO not the right way.  E.g.,
> currently there are subtle but very real differences between the 2
> back-ends: w32notify doesn't accept t or a lone symbol as the 2nd
> argument (it insists on getting a list); the list of supported watch
> types is entirely different; and the w32 back-ends actually watches
> the entire directory of the file, not just that file.
>
> IOW, this feature is not really ready for Tramp-ization, or for
> user-land in general.  Stefan wanted people to experiment with this
> and gather experience, before we know enough to discuss how to make it
> user- and Lisp-friendly.

Is there any progress on this? Any attempt of a unified interface?

Best regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: ruediger@c-plusplus.de,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,  sdl.web@gmail.com
> Date: Mon, 07 Jan 2013 12:33:26 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > IMO, this design is wrong.  Tramp is just one more back-end for this
> > feature, in addition to two others: inotify and w32notify.  So I think
> > Tramp handlers should be called from a higher-level code, one that
> > calls whichever back-end is appropriate.  Otherwise, we will need to
> > implement the Tramp support twice, in 2 different sets of primitives.
> >
> > Which, of course, goes back to the kind of design discussion I
> > suggested to have at the time, where we were supposed to consider
> > various alternatives and eventually agree on some higher-level APIs.
> > Jumping to coding right away is IMO not the right way.  E.g.,
> > currently there are subtle but very real differences between the 2
> > back-ends: w32notify doesn't accept t or a lone symbol as the 2nd
> > argument (it insists on getting a list); the list of supported watch
> > types is entirely different; and the w32 back-ends actually watches
> > the entire directory of the file, not just that file.
> >
> > IOW, this feature is not really ready for Tramp-ization, or for
> > user-land in general.  Stefan wanted people to experiment with this
> > and gather experience, before we know enough to discuss how to make it
> > user- and Lisp-friendly.
> 
> Is there any progress on this? Any attempt of a unified interface?

Not that I'm aware of, no.  OTOH, I haven't seen any attempts to use
the feature in any Lisp package, either.  Maybe we just don't need it ;-)

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> Is there any progress on this? Any attempt of a unified interface?
>
> Not that I'm aware of, no.  OTOH, I haven't seen any attempts to use
> the feature in any Lisp package, either.  Maybe we just don't need it ;-)

It might be too early judging it. And people might wait for a stable API ...

Best regards, Michael.

Re: [PATCH] Added basic file system watching support.

[Stefan Monnier]

>>> Is there any progress on this? Any attempt of a unified interface?
>> Not that I'm aware of, no.  OTOH, I haven't seen any attempts to use
>> the feature in any Lisp package, either.  Maybe we just don't need it ;-)
> It might be too early judging it. And people might wait for a stable API ...

I don't see how we can develop a good API without first seeing the
feature in use.
So someone should try and use the existing code for auto-revert and
dired, to get a good idea of how it plays out.  And then we can abstract
out the differences into a sane API.


        Stefan

Re: [PATCH] Added basic file system watching support.

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>>> Is there any progress on this? Any attempt of a unified interface?
>>> Not that I'm aware of, no.  OTOH, I haven't seen any attempts to use
>>> the feature in any Lisp package, either.  Maybe we just don't need it ;-)
>> It might be too early judging it. And people might wait for a stable API ...
>
> I don't see how we can develop a good API without first seeing the
> feature in use.

Maybe we shall mark this change as experimental in NEWS then? And it
could also be useful to install Tramp's `inotify-add-watch' handler?

> So someone should try and use the existing code for auto-revert and
> dired, to get a good idea of how it plays out.  And then we can abstract
> out the differences into a sane API.

Hmm. If time permits, I'll check what I could do. But likely I'm not
able to implement something for `w32notify-*'; I don't run MS Windows
locally.

>         Stefan

Best regards, Michael.

Re: [PATCH updated] Support for filesystem watching (inotify)

[Stefan Monnier]

> Thank you for your comments! I updated the patch and I hope I fixed
> everything except the things noted below.

Eli's comments took care of most nitpicks, but there's still one I care
about: every open paren should be preceded by a space.

BTW, thanks very much for your contribution.


        Stefan

Re: [PATCH updated] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

On Monday 06 June 2011 17:14:53 Stefan Monnier wrote:
> > Thank you for your comments! I updated the patch and I hope I fixed
> > everything except the things noted below.
> 
> Eli's comments took care of most nitpicks, but there's still one I care
> about: every open paren should be preceded by a space.

This should be fixed in Version 3 of the patch

http://lists.gnu.org/archive/html/emacs-devel/2011-06/msg00178.html

Regards,
Rüdiger

Re: [PATCH] Support for filesystem watching (inotify)

[Leo]

On 2011-06-04 06:34 +0800, Rüdiger Sonderfeld wrote:
> I wrote a patch to support watching for file system events. It currently only works with inotify (Linux) but it could be extended to support kqueue 
> (*BSD, OS X) or Win32. But I wanted to get some feedback first. Watching for filesystem events would be very useful for, e.g., dired or magit's status 
> view.

(whole thread: http://thread.gmane.org/gmane.emacs.devel/140158)

Is this patch ready for 24.3?

Leo

Re: [PATCH] Support for filesystem watching (inotify)

[Rüdiger Sonderfeld]

Hello,
sorry the Patch is not ready, yet. I haven't added the suggestions made by 
Stefan Monnier: <jwv39ii6jmw.fsf-monnier+emacs@gnu.org>

What is the time window for 24.3?

Regards,
Rüdiger

On Tuesday 18 September 2012 19:50:49 Leo wrote:
> On 2011-06-04 06:34 +0800, Rüdiger Sonderfeld wrote:
> > I wrote a patch to support watching for file system events. It currently
> > only works with inotify (Linux) but it could be extended to support
> > kqueue (*BSD, OS X) or Win32. But I wanted to get some feedback first.
> > Watching for filesystem events would be very useful for, e.g., dired or
> > magit's status view.
> 
> (whole thread: http://thread.gmane.org/gmane.emacs.devel/140158)
> 
> Is this patch ready for 24.3?
> 
> Leo

Re: [PATCH] Support for filesystem watching (inotify)

[Stefan Monnier]

> sorry the Patch is not ready, yet.  I haven't added the suggestions
> made by Stefan Monnier: <jwv39ii6jmw.fsf-monnier+emacs@gnu.org>
> What is the time window for 24.3?

The freeze is planned for Oct 1 (i.e. Real Soon Now(tm)).


        Stefan