bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

1. Turn on global-auto-revert-mode.
2. Open a few files that are different between different branches of the
Git repository you're working in (e.g. Emacs). Have at least 2 such files.
3. Switch from one branch to another. Notice that not every buffer has
updated its contents.
4. Switch to one such buffer, press SPC (or any other letter) - see the
"xxx changed on disk; really edit the buffer?" notification.

There's not much difference if I enable auto-revert-mode in each buffer
explicitly, instead of using global-auto-revert-mode.

In GNU Emacs 25.0.50.13 (x86_64-unknown-linux-gnu, GTK+ Version 3.16.7)
 of 2015-11-06
Repository revision: 267e0e80e1ad8c33c10ccaff77169a7aa759c163
Windowing system distributor 'The X.Org Foundation', version 11.0.11702000
System Description:	Ubuntu 15.10

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GSETTINGS NOTIFY
GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
GTK3 X11

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

I should also note:

- 'make check' shows no relevant failures here (just something in 
subr-tests).

- Customizing auto-revert-use-notify to nil seems to work around the 
problem successfully.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

> - 'make check' shows no relevant failures here (just something in
> subr-tests).

I've added auto-revert-test01-auto-revert-several-files to
auto-revert-tests.el. With backend inotify it reproduces your
problem. With gfilenotify, the test passes.

I will work on this.

Best regards, Michael.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

On 11/07/2015 12:07 PM, Michael Albinus wrote:

> I've added auto-revert-test01-auto-revert-several-files to
> auto-revert-tests.el. With backend inotify it reproduces your
> problem. With gfilenotify, the test passes.

Thank you.

But my ./configure output contains this:

checking whether GLib is linked in... yes
...
   Does Emacs use a file notification library?      yes -lglibc (inotify)

Could you suggest how I can make Emacs use gfilenotify in Ubuntu?

I do have libglib2.0-dev installed.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

> But my ./configure output contains this:
>
> checking whether GLib is linked in... yes
> ...
>   Does Emacs use a file notification library?      yes -lglibc (inotify)
>
> Could you suggest how I can make Emacs use gfilenotify in Ubuntu?
>
> I do have libglib2.0-dev installed.

"./configure --help" is always your friend:

# ./configure --with-file-notification=gfile

Don't be nervous: I hope to fix this over the weekend.

Best regards, Michael.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 7 Nov 2015 15:16:26 +0200
> Cc: 21841@debbugs.gnu.org
> 
> On 11/07/2015 12:07 PM, Michael Albinus wrote:
> 
> > I've added auto-revert-test01-auto-revert-several-files to
> > auto-revert-tests.el. With backend inotify it reproduces your
> > problem. With gfilenotify, the test passes.
> 
> Thank you.
> 
> But my ./configure output contains this:
> 
> checking whether GLib is linked in... yes
> ...
>    Does Emacs use a file notification library?      yes -lglibc (inotify)
> 
> Could you suggest how I can make Emacs use gfilenotify in Ubuntu?

You can do that by giving the --with-file-notification=gfile switch to
configure (see "./configure --help").

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

On 11/07/2015 03:41 PM, Michael Albinus wrote:

> "./configure --help" is always your friend:
>
> # ./configure --with-file-notification=gfile

Thank you both.

I thought it looked like a bug, because the beginning of 
lisp/filenotify.el suggests that gfilenotify has priority, and yet the 
configure script chooses inotify over it by default.

> Don't be nervous: I hope to fix this over the weekend.

Thanks in advance.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Eli Zaretskii]

> Cc: 21841@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 7 Nov 2015 15:50:31 +0200
> 
> I thought it looked like a bug, because the beginning of 
> lisp/filenotify.el suggests that gfilenotify has priority, and yet the 
> configure script chooses inotify over it by default.

That was once true, but we changed that due to some annoying problems
with gfilenotify.  I guess the comments were not updated.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> I thought it looked like a bug, because the beginning of
> lisp/filenotify.el suggests that gfilenotify has priority, and yet the
> configure script chooses inotify over it by default.

The order was changed in configure.ac some weeks ago. filenotify.el
doesn't say anything about priorities. But maybe I'll change the order
in the comments and code next time I touch this file.

Best regards, Michael.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

On 11/07/2015 03:53 PM, Eli Zaretskii wrote:

> That was once true, but we changed that due to some annoying problems
> with gfilenotify.  I guess the comments were not updated.

Could you point me to them? I don't see any mention of it in the 
documentation, and the debbugs search for gfilenotify doesn't seem to 
return any relevant bugs that are still open.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Could you point me to them? I don't see any mention of it in the
> documentation, and the debbugs search for gfilenotify doesn't seem to
> return any relevant bugs that are still open.

Bug#21241

Best regards, Michael.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Eli Zaretskii]

> Cc: michael.albinus@gmx.de, 21841@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 7 Nov 2015 16:03:58 +0200
> 
> On 11/07/2015 03:53 PM, Eli Zaretskii wrote:
> 
> > That was once true, but we changed that due to some annoying problems
> > with gfilenotify.  I guess the comments were not updated.
> 
> Could you point me to them? I don't see any mention of it in the 
> documentation, and the debbugs search for gfilenotify doesn't seem to 
> return any relevant bugs that are still open.

Bug #21241 was one of them, #20417 is another.  Yet another problem
was reported in this thread:

  http://lists.gnu.org/archive/html/emacs-devel/2014-10/msg00941.html

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

On 11/07/2015 04:15 PM, Eli Zaretskii wrote:
ugs that are still open.
>
> Bug #21241 was one of them, #20417 is another.  Yet another problem
> was reported in this thread:
>
>    http://lists.gnu.org/archive/html/emacs-devel/2014-10/msg00941.html

Thank you.  Do you think they should be mentioned in etc/PROBLEMS?

And/or in the ./configure output, when the glib file-notification 
backend is enabled? Similarly to how it complains about GTK3 and closing 
open displays.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Eli Zaretskii]

> Cc: michael.albinus@gmx.de, 21841@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 7 Nov 2015 18:15:02 +0200
> 
> > Bug #21241 was one of them, #20417 is another.  Yet another problem
> > was reported in this thread:
> >
> >    http://lists.gnu.org/archive/html/emacs-devel/2014-10/msg00941.html
> 
> Thank you.  Do you think they should be mentioned in etc/PROBLEMS?

I think so, yes.

> And/or in the ./configure output, when the glib file-notification 
> backend is enabled? Similarly to how it complains about GTK3 and closing 
> open displays.

Not sure about configure.  We warn about GTK because there's a
configure-time option to disable that.  By contrast, gfilenotify is
only used as a last resort, or if the user explicitly asked for it, so
the warning will have little effect, I think, and perhaps might even
annoy.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

On 11/07/2015 07:04 PM, Eli Zaretskii wrote:

> Not sure about configure.  We warn about GTK because there's a
> configure-time option to disable that.  By contrast, gfilenotify is
> only used as a last resort,

The "last resort" case is worrying. In that situation, auto-revert-mode 
would switch over to filenotify, but, if I'm reading #21241 right, then 
it might start missing some notifications.

I wonder whether we should use gfilenotify at all, unless explicitly asked.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Eli Zaretskii]

> Cc: michael.albinus@gmx.de, 21841@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 7 Nov 2015 19:14:09 +0200
> 
> On 11/07/2015 07:04 PM, Eli Zaretskii wrote:
> 
> > Not sure about configure.  We warn about GTK because there's a
> > configure-time option to disable that.  By contrast, gfilenotify is
> > only used as a last resort,
> 
> The "last resort" case is worrying. In that situation, auto-revert-mode 
> would switch over to filenotify, but, if I'm reading #21241 right, then 
> it might start missing some notifications.
> 
> I wonder whether we should use gfilenotify at all, unless explicitly asked.

Maybe.  I don't have enough experience using gfilenotify to say how
bad the problem is.  How about if you try using it for a while and see
how much trouble it gives you?

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> The "last resort" case is worrying. In that situation,
> auto-revert-mode would switch over to filenotify, but, if I'm reading
> #21241 right, then it might start missing some notifications.
>
> I wonder whether we should use gfilenotify at all, unless explicitly asked.

There are still platforms which have nothing else. cygwin, *BSD, ...
Maybe we could try to make gfilenotify more robust, and let it
autodetect when it cannot work properly. In this case it shall commit
suicide (send the `stopped' event); auto-revert-mode would fall back to
polling then.

Best regards, Michael.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Dmitry,
>
>> - 'make check' shows no relevant failures here (just something in
>> subr-tests).
>
> I've added auto-revert-test01-auto-revert-several-files to
> auto-revert-tests.el. With backend inotify it reproduces your
> problem. With gfilenotify, the test passes.
>
> I will work on this.

Should be fixed now. Could you, pls, check for the inotify backend?

Best regards, Michael.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

On 11/08/2015 04:24 PM, Michael Albinus wrote:

> Should be fixed now. Could you, pls, check for the inotify backend?

Seems to work now. Thank you!

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Dmitry Gutov]

On 11/07/2015 07:20 PM, Eli Zaretskii wrote:

> Maybe.  I don't have enough experience using gfilenotify to say how
> bad the problem is.  How about if you try using it for a while and see
> how much trouble it gives you?

All right, I'll continue using it. So far it's been fine, but AFAICT 
bug#21241 depends on the number of files in play, and I don't have too 
many file open at the same time, normally. Which might be different for 
a significant number of users.

bug#21841: 25.0.50; auto-revert-mode broken when several files have modifications on disk

[Eli Zaretskii]

> Cc: michael.albinus@gmx.de, 21841@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 9 Nov 2015 01:28:27 +0200
> 
> On 11/07/2015 07:20 PM, Eli Zaretskii wrote:
> 
> > Maybe.  I don't have enough experience using gfilenotify to say how
> > bad the problem is.  How about if you try using it for a while and see
> > how much trouble it gives you?
> 
> All right, I'll continue using it. So far it's been fine, but AFAICT 
> bug#21241 depends on the number of files in play, and I don't have too 
> many file open at the same time, normally. Which might be different for 
> a significant number of users.

If it misbehaves only for a large number of files (e.g.,
global-auto-revert-mode), then it's an important detail for those who
use it.

Thanks.