Emacs inotify support?

Emacs inotify support?

[joakim]

It would be useful with inotify support in Emacs to for example enhance
dired. When a directory being monitored by dired changes, dired could be
notified and refresh the view automatically. W32 has a similar feature.

My personal use-case would be for image dired to automatically update
when new images arrive from a scanner with an automatic document feeder.

WDYT?

-- 
Joakim Verona

Re: Emacs inotify support?

[Lennart Borgman]

On Fri, Sep 11, 2009 at 10:34 PM,  <joakim@verona.se> wrote:
> It would be useful with inotify support in Emacs to for example enhance
> dired. When a directory being monitored by dired changes, dired could be
> notified and refresh the view automatically. W32 has a similar feature.
>
> My personal use-case would be for image dired to automatically update
> when new images arrive from a scanner with an automatic document feeder.
>
> WDYT?

I think it would be a good idea. I asked for it some years ago.

Re: Emacs inotify support?

[Antoine Levitt]

[-- Attachment #1: Type: text/plain, Size: 1461 bytes --]

I also think that would be a good idea. I would even argue that this would
be beneficial for other files too. Current situation, either in dired or in
files, is not very satisfactory : if a user has a buffer open with some file
and that file changes outside emacs, he won't be notified of changes until
he actually tries to modify it. Ideal case : emacs would monitor every file
used by buffers, and notify of changes with the current dialog (continue,
refresh, etc) in visible buffers.

Problems include : what to do if there is no notification system supported
in the OS, remote files, and what would happen on very frequently modified
files (ie would I be able to view a log file without being flooded by
notifications). Maybe the simplest way would be to keep the default, and
just provide a wrapper over inotify, a set of hooks, and let the user decide
what to do.

2009/9/11 Lennart Borgman <lennart.borgman@gmail.com>

> On Fri, Sep 11, 2009 at 10:34 PM,  <joakim@verona.se> wrote:
> > It would be useful with inotify support in Emacs to for example enhance
> > dired. When a directory being monitored by dired changes, dired could be
> > notified and refresh the view automatically. W32 has a similar feature.
> >
> > My personal use-case would be for image dired to automatically update
> > when new images arrive from a scanner with an automatic document feeder.
> >
> > WDYT?
>
> I think it would be a good idea. I asked for it some years ago.
>
>
>

[-- Attachment #2: Type: text/html, Size: 1892 bytes --]

Re: Emacs inotify support?

[Lennart Borgman]

On Sat, Sep 12, 2009 at 12:54 AM, Antoine Levitt
<antoine.levitt@gmail.com> wrote:
> I also think that would be a good idea. I would even argue that this would
> be beneficial for other files too.

Agreed.


> Problems include : what to do if there is no notification system supported
> in the OS, remote files,

Just do as today.


> and what would happen on very frequently modified
> files (ie would I be able to view a log file without being flooded by
> notifications).

Is not this a case when you want notifications?

Re: Emacs inotify support?

[Eli Zaretskii]

> Date: Sat, 12 Sep 2009 00:54:50 +0200
> From: Antoine Levitt <antoine.levitt@gmail.com>
> Cc: joakim@verona.se, Emacs Development <emacs-devel@gnu.org>
> 
> if a user has a buffer open with some file
> and that file changes outside emacs, he won't be notified of changes until
> he actually tries to modify it. Ideal case : emacs would monitor every file
> used by buffers, and notify of changes with the current dialog (continue,
> refresh, etc) in visible buffers.

This use-case does not need any help from the OS to implement the
feature in Emacs.  It should be quite easy to write a function that
runs off the idle timer and checks each file that Emacs visits.

The OP was talking about detecting changes in files that Emacs does
not know about, including in files that are created while Emacs runs.

Re: Emacs inotify support?

[Antoine Levitt]

[-- Attachment #1: Type: text/plain, Size: 1628 bytes --]

2009/9/12 Lennart Borgman <lennart.borgman@gmail.com>
>
> On Sat, Sep 12, 2009 at 12:54 AM, Antoine Levitt
> <antoine.levitt@gmail.com> wrote:
> > and what would happen on very frequently modified
> > files (ie would I be able to view a log file without being flooded by
> > notifications).
>
> Is not this a case when you want notifications?
>
Imagine a log file on a web server, when there is a new line each second.
With a naïve implementation (current dialog each time the file changes), you
wouldn't be able to do anything because notifications would be too frequent.



> > Date: Sat, 12 Sep 2009 00:54:50 +0200
> > From: Antoine Levitt <antoine.levitt@gmail.com>
> > Cc: joakim@verona.se, Emacs Development <emacs-devel@gnu.org>
> >
> > if a user has a buffer open with some file
> > and that file changes outside emacs, he won't be notified of changes
until
> > he actually tries to modify it. Ideal case : emacs would monitor every
file
> > used by buffers, and notify of changes with the current dialog
(continue,
> > refresh, etc) in visible buffers.
>
> This use-case does not need any help from the OS to implement the
> feature in Emacs.  It should be quite easy to write a function that
> runs off the idle timer and checks each file that Emacs visits.
>
> The OP was talking about detecting changes in files that Emacs does
> not know about, including in files that are created while Emacs runs.

Indeed. If possible, polling should be avoided, though. And if inotify
or similar notification systems are implemented, it would be easy to
support both files and directories.

[-- Attachment #2: Type: text/html, Size: 2461 bytes --]

Re: Emacs inotify support?

[Richard Stallman]

    It would be useful with inotify support in Emacs to for example enhance
    dired. When a directory being monitored by dired changes, dired could be
    notified and refresh the view automatically. W32 has a similar feature.

Would this mean that all changes in the file system
would be reflected immediately in Dired buffers?
That could be a very nice feature.
But it could also be a pain in the neck to have these changes occur
while you're moving around in the Dired buffer and making changes.

Re: Emacs inotify support?

[Richard Stallman]

    Ideal case : emacs would monitor every file
    used by buffers, and notify of changes with the current dialog (continue,
    refresh, etc) in visible buffers.

To have the capability to watch a file for changes would be useful.
To do this by default for all visited files could be more of a pain in
the neck than a convenience.

The current system works well for visited files -- you can't
accidentally overwrite someone else's change, and any time you want to
find out whether the file has changed, just save it.

Re: Emacs inotify support?

[Eli Zaretskii]

> Date: Sat, 12 Sep 2009 18:36:47 +0200
> From: Antoine Levitt <antoine.levitt@gmail.com>
> Cc: lennart.borgman@gmail.com, joakim@verona.se, emacs-devel@gnu.org
> 
> If possible, polling should be avoided, though.

Why?

Re: Emacs inotify support?

[Antoine Levitt]

[-- Attachment #1: Type: text/plain, Size: 1620 bytes --]

> > Date: Sat, 12 Sep 2009 18:36:47 +0200
> > From: Antoine Levitt <antoine.levitt@gmail.com>
> > Cc: lennart.borgman@gmail.com, joakim@verona.se, emacs-devel@gnu.org
> >
> > If possible, polling should be avoided, though.
>
> Why?

Well, I don't know much about file systems, but isn't it always better
to be notified than to poll ?  First, having a notification system
means it's instantaneous. Second, I'd imagine querying the state of a
file has a cost (especially if it can't be cached and needs a real
access to the hard drive). Third, it'd avoid a waste of CPU resources
(which may be important for power consumption, since, from what I
understood, the more a program has fixed timers, the more it wakes the
CPU from sleep). Fourth, being notified is more high-level, since the
notification itself can be implemented by polling. One could imagine a
notification API where the backend would use inotify if available,
polling if not.

> But it could also be a pain in the neck to have these changes occur
> while you're moving around in the Dired buffer and making changes.

Agreed. Maybe couple it with a condition like "the user hasn't done
anything to the buffer for n seconds" ?

I also agree that notification for files shouldn't be turned on by
default. When dealing with files that are modified in more than one
place at the same time, the simpler and safer the default is, the
better. It would  be interesting to provide the feature for advanced users
though (ie users who know what they want). In any case, an inotify API could
be a nice basis for customisation/other modes (kinda like the dbus
interface)

[-- Attachment #2: Type: text/html, Size: 1992 bytes --]

Re: Emacs inotify support?

[Eli Zaretskii]

> Date: Sat, 12 Sep 2009 19:26:14 +0200
> From: Antoine Levitt <antoine.levitt@gmail.com>
> Cc: lennart.borgman@gmail.com, joakim@verona.se, emacs-devel@gnu.org
> 
> > > Date: Sat, 12 Sep 2009 18:36:47 +0200
> > > From: Antoine Levitt <antoine.levitt@gmail.com>
> > > Cc: lennart.borgman@gmail.com, joakim@verona.se, emacs-devel@gnu.org
> > >
> > > If possible, polling should be avoided, though.
> >
> > Why?
> 
> Well, I don't know much about file systems, but isn't it always better
> to be notified than to poll ?

But (AFAIU) inotify works by giving the application a file descriptor
that the application needs to pass to `select' or `poll' in order to
get notifications.  This is exactly the kind of ``polling'' Emacs does
with any external event (except for signals).

So we are up for polling anyway.

> First, having a notification system
> means it's instantaneous. Second, I'd imagine querying the state of a
> file has a cost (especially if it can't be cached and needs a real
> access to the hard drive). Third, it'd avoid a waste of CPU resources
> (which may be important for power consumption, since, from what I
> understood, the more a program has fixed timers, the more it wakes the
> CPU from sleep). Fourth, being notified is more high-level, since the
> notification itself can be implemented by polling. One could imagine a
> notification API where the backend would use inotify if available,
> polling if not.

You seem to be thinking of some signal-like mechanism.  But that's not
how these notifications work (nor do I think it's a good idea for them
to work like that in Emacs, where interrupting code at an arbitrary
moment is not a good idea).

As for wasting CPU time, system calls like `select' and `poll' don't
waste them too much.

Re: Emacs inotify support?

[Antoine Levitt]

[-- Attachment #1: Type: text/plain, Size: 2556 bytes --]

I'm afraid I got confused. It seems the confusion stems from the use of the
term poll. I was referring to the use described in
http://en.wikipedia.org/wiki/Polling_%28computer_science%29 , ie something
like : check the status of the files by looking at their timestamps, wait a
fixed timeout, and check again (which is terrible, I hope you agree). "poll"
and "select" system calls look like what the wiki page calls
interrupt-driven IO (which does have the qualities I described in my
previous post).

I was not aware of these (shame on me). They mean inotify has no relevance
to the action of watching for changes in a file, where poll and select do
the trick perfectly well.
2009/9/12 Eli Zaretskii <eliz@gnu.org>

> > Date: Sat, 12 Sep 2009 19:26:14 +0200
> > From: Antoine Levitt <antoine.levitt@gmail.com>
> > Cc: lennart.borgman@gmail.com, joakim@verona.se, emacs-devel@gnu.org
> >
> > > > Date: Sat, 12 Sep 2009 18:36:47 +0200
> > > > From: Antoine Levitt <antoine.levitt@gmail.com>
> > > > Cc: lennart.borgman@gmail.com, joakim@verona.se, emacs-devel@gnu.org
> > > >
> > > > If possible, polling should be avoided, though.
> > >
> > > Why?
> >
> > Well, I don't know much about file systems, but isn't it always better
> > to be notified than to poll ?
>
> But (AFAIU) inotify works by giving the application a file descriptor
> that the application needs to pass to `select' or `poll' in order to
> get notifications.  This is exactly the kind of ``polling'' Emacs does
> with any external event (except for signals).
>
> So we are up for polling anyway.
>
> > First, having a notification system
> > means it's instantaneous. Second, I'd imagine querying the state of a
> > file has a cost (especially if it can't be cached and needs a real
> > access to the hard drive). Third, it'd avoid a waste of CPU resources
> > (which may be important for power consumption, since, from what I
> > understood, the more a program has fixed timers, the more it wakes the
> > CPU from sleep). Fourth, being notified is more high-level, since the
> > notification itself can be implemented by polling. One could imagine a
> > notification API where the backend would use inotify if available,
> > polling if not.
>
> You seem to be thinking of some signal-like mechanism.  But that's not
> how these notifications work (nor do I think it's a good idea for them
> to work like that in Emacs, where interrupting code at an arbitrary
> moment is not a good idea).
>
> As for wasting CPU time, system calls like `select' and `poll' don't
> waste them too much.
>

[-- Attachment #2: Type: text/html, Size: 3602 bytes --]

Re: Emacs inotify support?

[Giuseppe Scrivano]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sat, 12 Sep 2009 19:26:14 +0200
>> From: Antoine Levitt <antoine.levitt@gmail.com>
>> Cc: lennart.borgman@gmail.com, joakim@verona.se, emacs-devel@gnu.org
>> 
>> > > Date: Sat, 12 Sep 2009 18:36:47 +0200
>> > > From: Antoine Levitt <antoine.levitt@gmail.com>
>> > > Cc: lennart.borgman@gmail.com, joakim@verona.se, emacs-devel@gnu.org
>> > >
>> > > If possible, polling should be avoided, though.
>> >
>> > Why?
>> 
>> Well, I don't know much about file systems, but isn't it always better
>> to be notified than to poll ?
>
> But (AFAIU) inotify works by giving the application a file descriptor
> that the application needs to pass to `select' or `poll' in order to
> get notifications.  This is exactly the kind of ``polling'' Emacs does
> with any external event (except for signals).

There are events that can't be catched directly with a `select' or
`poll'.  For example, to check if a file was modified there is need to
`stat' it, using inotify you will see this change immediately and
avoiding unuseful `stat's at fixed intervals.

Inotify events are read from the inotify instance descriptor, that is a
blocking file descriptor.  So you can mix it in a `select' with
sockets/pipes/etc.


Cheers,
Giuseppe

Re: Emacs inotify support?

[Miles Bader]

Richard Stallman <rms@gnu.org> writes:
>     Ideal case : emacs would monitor every file
>     used by buffers, and notify of changes with the current dialog (continue,
>     refresh, etc) in visible buffers.
>
> To have the capability to watch a file for changes would be useful.
> To do this by default for all visited files could be more of a pain in
> the neck than a convenience.

I was thinking such a feature could be used simply to optimize
`auto-revert-mode' and `global-auto-revert-mode'.

Whether it's worth it, I don't know.  Is the cost of the current
implementation of global-auto-revert-mode high enough to be a problem,
or the granularity too coarse?

[I guess it's actually quite system-dependent]

-Miles

-- 
Religion, n. A daughter of Hope and Fear, explaining to Ignorance the nature
of the Unknowable.

Re: Emacs inotify support?

[David Kastrup]

[-- Attachment #1: Type: text/plain, Size: 915 bytes --]

Miles Bader <miles@gnu.org> writes:

> Richard Stallman <rms@gnu.org> writes:
>>     Ideal case : emacs would monitor every file
>>     used by buffers, and notify of changes with the current dialog (continue,
>>     refresh, etc) in visible buffers.
>>
>> To have the capability to watch a file for changes would be useful.
>> To do this by default for all visited files could be more of a pain in
>> the neck than a convenience.
>
> I was thinking such a feature could be used simply to optimize
> `auto-revert-mode' and `global-auto-revert-mode'.
>
> Whether it's worth it, I don't know.  Is the cost of the current
> implementation of global-auto-revert-mode high enough to be a problem,
> or the granularity too coarse?
>
> [I guess it's actually quite system-dependent]

I use the following "executable" for paging output for applications
called from within Emacs, and it _is_ jerkier than really desirable.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: emacs-pager --]
[-- Type: text/x-sh, Size: 264 bytes --]

#!/bin/sh
TMP=`mktemp -t emacs-pager.XXXXXX`
trap "rm $TMP* 2>/dev/null" 0
echo '-*- mode: view; auto-revert-interval: 1; mode: auto-revert-tail; view-exit-action: kill-buffer -*-' >"$TMP"
exec 5<&0 <&-
cat "$@" <&5 >>"$TMP" &
eval "${VISUAL:-${EDITOR}}" '"$TMP"'

[-- Attachment #3: Type: text/plain, Size: 20 bytes --]



-- 
David Kastrup

Re: Emacs inotify support?

[Richard Stallman]

    The OP was talking about detecting changes in files that Emacs does
    not know about, including in files that are created while Emacs runs.

I think he wanted to detect changes in the contents of a directory
that Emacs is looking at.  From another point of view, that can be
described as changes in the files that directory.

Re: Emacs inotify support?

[joakim]

Richard Stallman <rms@gnu.org> writes:

>     The OP was talking about detecting changes in files that Emacs does
>     not know about, including in files that are created while Emacs runs.
>
> I think he wanted to detect changes in the contents of a directory
> that Emacs is looking at.  From another point of view, that can be
> described as changes in the files that directory.

One way to look at it is auto-revert-mode for directories visited with
dired. Something other than Emacs is changing the contents of a
directory and I would like dired to notice. This can be achieved in a
number of ways, but the way I had in mind is an inotify handler in
Emacs, that would insert some kind of event in the event stream, so one
could bind a handler to it. This should be portable across platforms
with different filesystem notification schemes, and consistent with
Emacs behavior in general. 


-- 
Joakim Verona

Re: Emacs inotify support?

[Eli Zaretskii]

> From: joakim@verona.se
> Cc: Eli Zaretskii <eliz@gnu.org>, antoine.levitt@gmail.com,
>         lennart.borgman@gmail.com, emacs-devel@gnu.org
> Date: Sun, 13 Sep 2009 19:28:45 +0200
> 
> One way to look at it is auto-revert-mode for directories visited with
> dired. Something other than Emacs is changing the contents of a
> directory and I would like dired to notice. This can be achieved in a
> number of ways, but the way I had in mind is an inotify handler in
> Emacs, that would insert some kind of event in the event stream, so one
> could bind a handler to it. This should be portable across platforms
> with different filesystem notification schemes, and consistent with
> Emacs behavior in general. 

I can hardly imagine that using inotify is more portable than looking
at the time stamp of the directory (which works even on Windows, if
you have NTFS).

But maybe I didn't understand what you were trying to say.

Re: Emacs inotify support?

[Lennart Borgman]

On Sun, Sep 13, 2009 at 9:42 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: joakim@verona.se
>> Cc: Eli Zaretskii <eliz@gnu.org>, antoine.levitt@gmail.com,
>>         lennart.borgman@gmail.com, emacs-devel@gnu.org
>> Date: Sun, 13 Sep 2009 19:28:45 +0200
>>
>> One way to look at it is auto-revert-mode for directories visited with
>> dired. Something other than Emacs is changing the contents of a
>> directory and I would like dired to notice. This can be achieved in a
>> number of ways, but the way I had in mind is an inotify handler in
>> Emacs, that would insert some kind of event in the event stream, so one
>> could bind a handler to it. This should be portable across platforms
>> with different filesystem notification schemes, and consistent with
>> Emacs behavior in general.
>
> I can hardly imagine that using inotify is more portable than looking
> at the time stamp of the directory (which works even on Windows, if
> you have NTFS).
>
> But maybe I didn't understand what you were trying to say.


I do not know how inotify works, but the corresponding function in w32
will send a notification when a file has changed. It is not more
portable, but it allows changes in the direcotry from outside of Emacs
to be handled rather similar to those from within Emacs.

Re: Emacs inotify support?

[joakim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: joakim@verona.se
>> Cc: Eli Zaretskii <eliz@gnu.org>, antoine.levitt@gmail.com,
>>         lennart.borgman@gmail.com, emacs-devel@gnu.org
>> Date: Sun, 13 Sep 2009 19:28:45 +0200
>> 
>> One way to look at it is auto-revert-mode for directories visited with
>> dired. Something other than Emacs is changing the contents of a
>> directory and I would like dired to notice. This can be achieved in a
>> number of ways, but the way I had in mind is an inotify handler in
>> Emacs, that would insert some kind of event in the event stream, so one
>> could bind a handler to it. This should be portable across platforms
>> with different filesystem notification schemes, and consistent with
>> Emacs behavior in general. 
>
> I can hardly imagine that using inotify is more portable than looking
> at the time stamp of the directory (which works even on Windows, if
> you have NTFS).

Re-reading my message I see it was not very clear.

I meant that a portable emacs interface can be built on the various
plattform specific interfaces, such as inotify. Looking at the directory
timestamp could be one such implementation, but that would require
polling in Emacs right? That would make Emacs slow with a large number
of open dired buffers. 

> But maybe I didn't understand what you were trying to say.
-- 
Joakim Verona

Re: Emacs inotify support?

[Paul R]

On Sat, 12 Sep 2009 12:46:44 -0400, Richard Stallman <rms@gnu.org> said:
> Would this mean that all changes in the file system would be reflected
> immediately in Dired buffers? That could be a very nice feature. But
> it could also be a pain in the neck to have these changes occur while
> you're moving around in the Dired buffer and making changes.

What kind of changes are you thinking of ? My gut feeling is that dired
would have to face two kind of situation :

 - trivialy mergeable divergence (f.e. you mark a file that gets touched
   externally) : AFAIK, dired already handles this kind of situation
   when you press 'g' while working, so it could do so without
   interupting the user work.
 - unmergeable divergence (f.e. you initiate a rename operation but in
   the meantime the file gets removed) : Dired will have to complain
   that source file is missing, the situation would not be worse with
   instant reflexion than without.

On the other hand, having instant reflexion in dired would be a big
usability improvement for most browsing operations. That could even go
as far as tracking displayed directory path changes, i.e. you browse
a directory A, browse a subdirectory A/B, rename A/B to A/C (from inside
dired A, or from outside Emacs) : dired buffer A/B is magically
refreshed from A/C. The same could be used for file buffers.

-- 
  Paul

Re: Emacs inotify support?

[Eli Zaretskii]

> From: joakim@verona.se
> Cc: rms@gnu.org, antoine.levitt@gmail.com, lennart.borgman@gmail.com,
>         emacs-devel@gnu.org
> Date: Mon, 14 Sep 2009 07:47:35 +0200
> 
> I meant that a portable emacs interface can be built on the various
> plattform specific interfaces, such as inotify.

Actually, I suspect that the portability could be only on the Lisp
level; the guts of the interface to the notifications will probably be
quite different for Linux inotify and native Windows calls, either
FindFirst/NextChangeNotification or ReadDirectoryChangesW.

That's not to say that this is not a viable possibility.

> Looking at the directory timestamp could be one such implementation,
> but that would require polling in Emacs right? That would make Emacs
> slow with a large number of open dired buffers.

If we do that from an idle timer, and never test too many directories
before returning to the command loop (similar to what JIT lock does),
I don't think the slowdown will be visible.  At least on Posix
platforms, `stat' is very fast and efficient, and the OS caches its
results.  On non-Posix platforms, `stat' could be expensive, but we
don't need the full `stat' just to check the time stamp of a
directory, we only need a single system call that is much faster than
the full emulation of `stat'.

Re: Emacs inotify support?

[Richard Stallman]

     - trivialy mergeable divergence (f.e. you mark a file that gets touched
       externally) : AFAIK, dired already handles this kind of situation
       when you press 'g' while working, so it could do so without
       interupting the user work.

Mergeable differences can be more than that.  For instance,
if some files appear and others disappear.

g makes these updates, of course.  But you expect them when you type
g.  They may surprise you otherwise.

What happens if the file whose line point is on gets renamed?  That
seems like a mergeable difference, but you might be a bit surprised to
see the line move to some other position in the buffer.

Ultimately the only way to see how good or bad this feature is is to
try it.  But I think that paying a lot of attention to minimizing the
bad aspect is crucial to making the good outweigh the bad.

     - unmergeable divergence (f.e. you initiate a rename operation but in
       the meantime the file gets removed) : Dired will have to complain
       that source file is missing, the situation would not be worse with
       instant reflexion than without.

Well, it might be worse, because the very line you are editing may
disappear!

Re: Emacs inotify support?

[Juri Linkov]

>      - trivialy mergeable divergence (f.e. you mark a file that gets touched
>        externally) : AFAIK, dired already handles this kind of situation
>        when you press 'g' while working, so it could do so without
>        interupting the user work.
>
> Mergeable differences can be more than that.  For instance,
> if some files appear and others disappear.
>
> g makes these updates, of course.  But you expect them when you type
> g.  They may surprise you otherwise.
>
> What happens if the file whose line point is on gets renamed?  That
> seems like a mergeable difference, but you might be a bit surprised to
> see the line move to some other position in the buffer.

Most file managers automatically update changed files and that
doesn't surprise users.

> Ultimately the only way to see how good or bad this feature is is to
> try it.  But I think that paying a lot of attention to minimizing the
> bad aspect is crucial to making the good outweigh the bad.
>
>      - unmergeable divergence (f.e. you initiate a rename operation but in
>        the meantime the file gets removed) : Dired will have to complain
>        that source file is missing, the situation would not be worse with
>        instant reflexion than without.
>
> Well, it might be worse, because the very line you are editing may
> disappear!

`auto-revert-mode' takes precautions against this case with the help
of `dired-buffer-stale-p' that defines that dired buffers shouldn't be
reverted when in editing mode.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Emacs inotify support?

[Richard Stallman]

    > What happens if the file whose line point is on gets renamed?  That
    > seems like a mergeable difference, but you might be a bit surprised to
    > see the line move to some other position in the buffer.

    Most file managers automatically update changed files and that
    doesn't surprise users.

The precise details of display format can have a big effect on whether
the update bothers users, and how much.

Ultimately, the only way to judge whether it bothers users too much
is to try it.  But we should POLL THE USERS before enabling it by default.

Re: Emacs inotify support?

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     > What happens if the file whose line point is on gets renamed?  That
>     > seems like a mergeable difference, but you might be a bit surprised to
>     > see the line move to some other position in the buffer.
>
>     Most file managers automatically update changed files and that
>     doesn't surprise users.
>
> The precise details of display format can have a big effect on whether
> the update bothers users, and how much.
>
> Ultimately, the only way to judge whether it bothers users too much
> is to try it.  But we should POLL THE USERS before enabling it by default.

I think that any updates should try to keep the line that point is in
under point, preferably at the same window position.  In that case
editing would not usually be disrupted, just navigation.  Any unsaved
editing operation in progress can continue as long as the file name
identity is not changed (if the file disappears, that would be reason to
throw an error and abort pending editing changes on that line).

-- 
David Kastrup

Re: Emacs inotify support?

[Stefan Monnier]

[ ...discussion about how to react to inotify in dired... ]

While this discussion is interesting and worthwhile, it has no direct
impact on whether or not Emacs would benefit from having
inotify support.

So to answer the original question: yes, I'd be interested to see
a patch that adds support for inotify (and other similar facilities), as
well as another patch that provides a layer on top that tries to
abstract away the details specific to each alternative (inotify,
dumb-polling, you name it...).


        Stefan

Re: Emacs inotify support?

[joakim]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> [ ...discussion about how to react to inotify in dired... ]
>
> While this discussion is interesting and worthwhile, it has no direct
> impact on whether or not Emacs would benefit from having
> inotify support.
>
> So to answer the original question: yes, I'd be interested to see
> a patch that adds support for inotify (and other similar facilities), as
> well as another patch that provides a layer on top that tries to
> abstract away the details specific to each alternative (inotify,
> dumb-polling, you name it...).

Thank you Stefan.

>
>         Stefan
>
-- 
Joakim Verona

Dired auto-revert (was: Emacs inotify support?)

[Juri Linkov]

There are at least three possible levels of automatic reverting of
a dired buffer:

1. When the user visits a dired buffer with `C-x d' or with
dired navigation commands like `^', etc.

2. Polling at given time intervals like `auto-revert-mode' does.

3. inotify support.

Currently I'd like to focus on the shortcomings of the first method.
Visiting an existing dired buffer with `C-x d' or with dired navigation
keys doesn't automatically revert its old content.  Instead of that,
a message "Directory has changed on disk; type g to update Dired"
is displayed in the echo area.  But the first problem is that it's hard
to notice this message (especially with quick dired navigation).
And the second problem is that even when this message is noticed,
it is still inconvenient to type `g' in every changed dired buffer.

A comment at the beginning of `dired-internal-noselect' says:

  (defun dired-internal-noselect (dir-or-list &optional switches mode)
  ;; If there is an existing dired buffer for DIRNAME, just leave
  ;; buffer as it is (don't even call dired-revert).

Why should the user do the tedious task of manual reverting?

  ;; This saves time especially for deep trees or with ange-ftp.

The function `dired-buffer-stale-p' could be use to detect such special
cases and not to automatically revert remote directories.

  ;; The user can type `g' easily, and it is more consistent with find-file.

It is not easy to type `g' in every changed visited dired buffer.
And it is not consistent with find-file because visiting a changed file
prompts for "File changed on disk.  Reread from disk? (yes or no)"
so it is impossible to avoid this message since it requires a confirmation.

  ;; A pity we can't possibly do "Directory has changed - refresh? "
  ;; like find-file does.

Yes, using the same prompt for dired buffers would be annoying because
untracked changes of a visited directory are much more frequent than
changes of a visited file.

  ;; But if SWITCHES are given they are probably different from the
  ;; buffer's old value, so call dired-sort-other, which does
  ;; revert the buffer.

A new SWITCHES reverts the buffer, thus reverting is not a problem,
so why not revert it automatically.

Or maybe we should add a new option that defines whether revert
automatically or just display a message.  The following patch implements
this option:

Index: lisp/dired.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/dired.el,v
retrieving revision 1.429
diff -c -r1.429 dired.el
*** lisp/dired.el	18 Sep 2009 16:59:36 -0000	1.429
--- lisp/dired.el	21 Sep 2009 21:35:03 -0000
***************
*** 753,758 ****
--- 753,770 ----
  	 buffer-read-only
  	 (dired-directory-changed-p dirname))))
  
+ ;;;###autoload
+ (defcustom dired-auto-revert-buffer nil
+   "Automatically revert changed dired buffer on revisiting.
+ If non-nil, revisiting an existing dired buffer automatically reverts it
+ if directory has changed on disk.  Otherwise, a message offering to revert
+ the changed dired buffer is displayed.
+ Note that this is not the same as `auto-revert-mode' that
+ periodically reverts at specified time intervals."
+   :type 'boolean
+   :version "23.2"
+   :group 'dired)
+ 
  (defun dired-internal-noselect (dir-or-list &optional switches mode)
    ;; If there is an existing dired buffer for DIRNAME, just leave
    ;; buffer as it is (don't even call dired-revert).
***************
*** 780,790 ****
  	       (setq dired-directory dir-or-list)
  	       ;; this calls dired-revert
  	       (dired-sort-other switches))
! 	      ;; If directory has changed on disk, offer to revert.
  	      ((when (dired-directory-changed-p dirname)
! 		 (message "%s"
! 			  (substitute-command-keys
! 			   "Directory has changed on disk; type \\[revert-buffer] to update Dired")))))
        ;; Else a new buffer
        (setq default-directory
  	    ;; We can do this unconditionally
--- 792,805 ----
  	       (setq dired-directory dir-or-list)
  	       ;; this calls dired-revert
  	       (dired-sort-other switches))
! 	      ;; If directory has changed on disk, revert or offer to revert.
  	      ((when (dired-directory-changed-p dirname)
! 		 (if (not dired-auto-revert-buffer)
! 		     (message "%s"
! 			      (substitute-command-keys
! 			       "Directory has changed on disk; type \\[revert-buffer] to update Dired")))
! 		 (revert-buffer)
! 		 (message "%s" "Changed directory automatically updated"))))
        ;; Else a new buffer
        (setq default-directory
  	    ;; We can do this unconditionally

-- 
Juri Linkov
http://www.jurta.org/emacs/

RE: Dired auto-revert (was: Emacs inotify support?)

[Drew Adams]

> Or maybe we should add a new option that defines whether revert
> automatically or just display a message.

Yes, please, if we implement the possibility of auto-reverting, then let's keep
it only a possibility, not an obligation. IOW, let's also make it possible to
choose not to auto-revert.

It might even be useful to give such control for specific directories or classes
of directories: have two options, analogous to `special-display-buffer-names'
and `special-display-regexps' (and `same-window-*').

`dired-auto-revert-names':
List of directories for which Dired auto-reverts.

`dired-auto-revert-regexps':
List of regexps specifying which Dired buffers are auto-reverted.

If both are nil, then there is no auto-reverting.
If `dired-auto-revert-regexps' is `(".")', then all Dired buffers are
auto-reverted.

Just a thought. But please keep the possibility of not auto-reverting.

Re: Dired auto-revert

[Juri Linkov]

>> Or maybe we should add a new option that defines whether revert
>> automatically or just display a message.
>
> Yes, please, if we implement the possibility of auto-reverting, then let's keep
> it only a possibility, not an obligation. IOW, let's also make it possible to
> choose not to auto-revert.
>
> It might even be useful to give such control for specific directories or classes
> of directories: have two options, analogous to `special-display-buffer-names'
> and `special-display-regexps' (and `same-window-*').
>
> `dired-auto-revert-names':
> List of directories for which Dired auto-reverts.
>
> `dired-auto-revert-regexps':
> List of regexps specifying which Dired buffers are auto-reverted.
>
> If both are nil, then there is no auto-reverting.
> If `dired-auto-revert-regexps' is `(".")', then all Dired buffers are
> auto-reverted.

I thought just an option for remote files would be enough.  Maybe it could
be implemented as a regexp but better as a predicate for defcustom that
tests `file-remote-p' and also could test the directory size, subdirectories
presence in the same dired buffer, etc.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Dired auto-revert

[Juri Linkov]

>> Or maybe we should add a new option that defines whether revert
>> automatically or just display a message.
>
> Yes, please, if we implement the possibility of auto-reverting, then let's keep
> it only a possibility, not an obligation. IOW, let's also make it possible to
> choose not to auto-revert.

This patch provides a defcustom that allows specifying a predicate function
for anyone who wants a non-standard auto-reverting.  By default it is nil
preserving the current behavior.

Index: lisp/dired.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/dired.el,v
retrieving revision 1.445
diff -u -r1.445 dired.el
--- lisp/dired.el	25 Nov 2009 17:11:34 -0000	1.445
+++ lisp/dired.el	4 Dec 2009 00:07:45 -0000
@@ -752,6 +752,25 @@
 	 buffer-read-only
 	 (dired-directory-changed-p dirname))))
 
+;;;###autoload
+(defcustom dired-auto-revert-buffer nil
+  "Automatically revert changed dired buffer on revisiting.
+If t, revisiting an existing dired buffer automatically reverts it.
+If its value is a predicate function that checks if directory has
+changed on disk, call this function with the directory name argument
+and revert the buffer if it returns non-nil.
+Otherwise, a message offering to revert the changed dired buffer
+is displayed.
+Note that this is not the same as `auto-revert-mode' that
+periodically reverts at specified time intervals."
+  :type '(choice
+          (const :tag "Don't revert" nil)
+          (const :tag "Always revert visited dired buffer" t)
+          (const :tag "Revert changed dired buffer" dired-directory-changed-p)
+          (function :tag "Predicate function"))
+  :group 'dired
+  :version "23.2")
+
 (defun dired-internal-noselect (dir-or-list &optional switches mode)
   ;; If there is an existing dired buffer for DIRNAME, just leave
   ;; buffer as it is (don't even call dired-revert).
@@ -779,6 +798,14 @@
 	       (setq dired-directory dir-or-list)
 	       ;; this calls dired-revert
 	       (dired-sort-other switches))
+	      ;; Always revert regardless of whether it has changed or not.
+	      ((eq dired-auto-revert-buffer t)
+	       (revert-buffer))
+	      ;; Revert when a predicate returns non-nil.
+	      ((functionp dired-auto-revert-buffer)
+	       (when (funcall dired-auto-revert-buffer dirname)
+		 (revert-buffer)
+		 (message "Changed directory automatically updated")))
 	      ;; If directory has changed on disk, offer to revert.
 	      ((when (dired-directory-changed-p dirname)
 		 (message "%s"

-- 
Juri Linkov
http://www.jurta.org/emacs/

Proced revert (Re: Dired auto-revert)

[Juri Linkov]

After typing `M-x proced RET' I see a list of inexistent processes that
were terminated long ago.  That's because `M-x proced RET' brings up
to display the old *Proced* buffer with obsolete process information.

I think when someone sometimes wants to get the old *Proced* buffer, the
right command to do this is `C-x b *Proced* RET'.  But `M-x proced RET'
should always display up-to-date process information.

BTW, there is also a related problem.  Typing `q' exits from Proced
and hides the *Proced* buffer, but switches to *Marked Processes*
where typing `q' has no effect.

In Dired, the *Marked Files* buffer is always hidden because it has
a space character at the beginning of its name " *Marked Files*".
This patch also adds a space to the *Marked Processes* buffer name.

Index: lisp/proced.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/proced.el,v
retrieving revision 1.42
diff -u -r1.42 proced.el
--- lisp/proced.el	12 Oct 2009 04:59:07 -0000	1.42
+++ lisp/proced.el	4 Dec 2009 00:11:12 -0000
@@ -682,9 +682,9 @@
         (progn
           (display-buffer buffer)
           (with-current-buffer buffer
-            (run-hooks 'proced-post-display-hook)))
+            (proced-update t)))
       (pop-to-buffer buffer)
-      (run-hooks 'proced-post-display-hook)
+      (proced-update t)
       (message
        (substitute-command-keys
         "Type \\<proced-mode-map>\\[quit-window] to quit, \\[proced-help] for help")))))
@@ -1711,7 +1711,7 @@
                          (line-end-position))))))
     (unless signal
       ;; Display marked processes (code taken from `dired-mark-pop-up').
-      (let ((bufname  "*Marked Processes*")
+      (let ((bufname  " *Marked Processes*")
             (header-line (substring-no-properties proced-header-line)))
         (with-current-buffer (get-buffer-create bufname)
           (setq truncate-lines t

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Dired auto-revert

[Stefan Monnier]

> This patch provides a defcustom that allows specifying a predicate function
> for anyone who wants a non-standard auto-reverting.  By default it is nil
> preserving the current behavior.

Looks OK, except for the little docstring nitpick below:

> +If its value is a predicate function that checks if directory has
> +changed on disk, call this function with the directory name argument
> +and revert the buffer if it returns non-nil.

There's no way the code can check whether the value is "a predicate
function that checks if directory has changed on disk" (it's one of
those halting-problem-equivalent).
So really it should just say

   If its value is a function, call this function with the directory
   name as single argument and revert the buffer if it returns non-nil.


-- Stefan

Re: Proced revert (Re: Dired auto-revert)

[Stefan Monnier]

> I think when someone sometimes wants to get the old *Proced* buffer, the
> right command to do this is `C-x b *Proced* RET'.  But `M-x proced RET'
> should always display up-to-date process information.

Agreed.

> BTW, there is also a related problem.  Typing `q' exits from Proced
> and hides the *Proced* buffer, but switches to *Marked Processes*
> where typing `q' has no effect.

> In Dired, the *Marked Files* buffer is always hidden because it has
> a space character at the beginning of its name " *Marked Files*".
> This patch also adds a space to the *Marked Processes* buffer name.

Interesting.  I may be to blame for this name choice.  So please add
a comment explaining why a leading space is used.


        Stefan

Proced revert (Re: Dired auto-revert)

[Roland Winkler]

On Fri Dec 4 2009 Juri Linkov wrote:
> After typing `M-x proced RET' I see a list of inexistent processes that
> were terminated long ago.  That's because `M-x proced RET' brings up
> to display the old *Proced* buffer with obsolete process information.
> 
> I think when someone sometimes wants to get the old *Proced* buffer, the
> right command to do this is `C-x b *Proced* RET'.  But `M-x proced RET'
> should always display up-to-date process information.
> 
> BTW, there is also a related problem.  Typing `q' exits from Proced
> and hides the *Proced* buffer, but switches to *Marked Processes*
> where typing `q' has no effect.
> 
> In Dired, the *Marked Files* buffer is always hidden because it has
> a space character at the beginning of its name " *Marked Files*".
> This patch also adds a space to the *Marked Processes* buffer name.

Both changes are fine with me!

Roland