bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

Hello,

At work, I've just observed a regression in the trunk [1] wrt the
emacs-24 branch [2].

If I visit some file, say "t:\myfile" (where t: is a network unit,
mounted via samba, that points to an hp-ux server), and I enable
auto-revert-mode, the buffer is not reverted when some other program
changes the contents of the file.

The problem doesn't show up when the file is stored in a local unit
(c:\) but fails in the above case.

I've observed this bug in version [1], but not in version [2].

----- Footnotes ------------

[1]
In GNU Emacs 24.3.50.1 (i386-mingw-nt5.2.3790)
 of 2013-01-22 on ODIEONE
Bzr revision: 111584 dmantipov@yandex.ru-20130122114800-5m9xn29kwf5n9nwx
Windowing system distributor `Microsoft Corp.', version 5.2.3790
Configured using:
 `configure --with-gcc (4.7) --no-opt --enable-checking --cflags
 -IC:/Devel/emacs/build/include --ldflags -LC:/Devel/emacs/build/lib'


[2]
In GNU Emacs 24.2.92.1 (i386-mingw-nt5.2.3790)
 of 2013-01-16 on MS-W8-DANI
Bzr revision: 111185 rgm@gnu.org-20130116122101-t5ychlz0gwh9u34x
Windowing system distributor `Microsoft Corp.', version 5.2.3790
Configured using:
 `configure --with-gcc (4.7) --no-opt --enable-checking --cflags
 -Ic:/emacs/libs/libXpm-3.5.10/include -Ic:/emacs/libs/libXpm-3.5.10/src
 -Ic:/emacs/libs/libpng-dev_1.4.3-1_win32/include
 -Ic:/emacs/libs/zlib-dev_1.2.5-2_win32/include
 -Ic:/emacs/libs/giflib-4.1.4-1-lib/include
 -Ic:/emacs/libs/jpeg-6b-4-lib/include
 -Ic:/emacs/libs/tiff-3.8.2-1-lib/include
 -Ic:/emacs/libs/libxml2-2.7.8-w32-bin/include/libxml2
 -Ic:/emacs/libs/gnutls-3.1.5-w32/include
 -Ic:/emacs/libs/libiconv-1.14-2-mingw32-dev/include'



-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Michael Albinus]

Dani Moncayo <dmoncayo@gmail.com> writes:

> Hello,

Hi Dani,

> At work, I've just observed a regression in the trunk [1] wrt the
> emacs-24 branch [2].
>
> If I visit some file, say "t:\myfile" (where t: is a network unit,
> mounted via samba, that points to an hp-ux server), and I enable
> auto-revert-mode, the buffer is not reverted when some other program
> changes the contents of the file.
>
> The problem doesn't show up when the file is stored in a local unit
> (c:\) but fails in the above case.

There are recent changes in autorevert.el. If possible, file
notifications will be used, in order to be triggered by the file system
if a file is modified. For Windows systems, the new function
`w32notify-add-watch' is used.

Likely, it is not possible to enable file notifications for mounted file
systems under Windows. In that case, `w32notify-add-watch' shall return
nil, and the code in auto-revert.el shall fall back to the previous
implementation.

Could you, please, check what

(w32notify-add-watch "t:\myfile" '(size last-write-time) 'ignore)

returns?

Best regards, Michael.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

> Could you, please, check what
>
> (w32notify-add-watch "t:\myfile" '(size last-write-time) 'ignore)
>
> returns?

It returns a number like 14510816, but the number varies each time I
evaluate the expression.

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Michael Albinus]

Dani Moncayo <dmoncayo@gmail.com> writes:

>> Could you, please, check what
>>
>> (w32notify-add-watch "t:\myfile" '(size last-write-time) 'ignore)
>>
>> returns?
>
> It returns a number like 14510816, but the number varies each time I
> evaluate the expression.

That's an indication that a watch has been installed. Eli, could you,
please, check?

Best regards, Michael.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: 13540@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
> Date: Thu, 24 Jan 2013 11:23:54 +0100
> 
> Dani Moncayo <dmoncayo@gmail.com> writes:
> 
> >> Could you, please, check what
> >>
> >> (w32notify-add-watch "t:\myfile" '(size last-write-time) 'ignore)
> >>
> >> returns?
> >
> > It returns a number like 14510816, but the number varies each time I
> > evaluate the expression.
> 
> That's an indication that a watch has been installed. Eli, could you,
> please, check?

I don't have access to Samba-mounted volumes.

Dani, if you attach a debugger to Emacs before evaluating the above
expression, do you see any warning messages printed by GDB?  For
example, this one:

	DebPrint (("watch_worker, abnormal exit: %lu\n", GetLastError ()));

If you see any messages, please post them here.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

> I don't have access to Samba-mounted volumes.
>
> Dani, if you attach a debugger to Emacs before evaluating the above
> expression, do you see any warning messages printed by GDB?  For
> example, this one:
>
>         DebPrint (("watch_worker, abnormal exit: %lu\n", GetLastError ()));
>
> If you see any messages, please post them here.

Mmmm I'm afraid I can't do that, be cause that machine doesn't have
MinGW and I'm not allowed to install it there.

I tried to create a similar environment at home, by creating an ubuntu
virtual machine with virtualbox, and publishing (via samba) some
folder in ubuntu, so that I can't define a network unit from my host
windows OS.  But I didn't get it to work.

Another possibility would be to compile a modified version of Emacs
which dumps any useful information to a file, for example.

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

>> I don't have access to Samba-mounted volumes.
>>
>> Dani, if you attach a debugger to Emacs before evaluating the above
>> expression, do you see any warning messages printed by GDB?  For
>> example, this one:
>>
>>         DebPrint (("watch_worker, abnormal exit: %lu\n", GetLastError ()));
>>
>> If you see any messages, please post them here.
>
> Mmmm I'm afraid I can't do that, be cause that machine doesn't have
> MinGW and I'm not allowed to install it there.
>
> I tried to create a similar environment at home, by creating an ubuntu
> virtual machine with virtualbox, and publishing (via samba) some
> folder in ubuntu, so that I can't define a network unit from my host
                              ^^^^^
... I can ...

> windows OS.  But I didn't get it to work.
>
> Another possibility would be to compile a modified version of Emacs
> which dumps any useful information to a file, for example.


-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Thu, 24 Jan 2013 17:34:46 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Michael Albinus <michael.albinus@gmx.de>, 13540@debbugs.gnu.org
> 
> > I don't have access to Samba-mounted volumes.
> >
> > Dani, if you attach a debugger to Emacs before evaluating the above
> > expression, do you see any warning messages printed by GDB?  For
> > example, this one:
> >
> >         DebPrint (("watch_worker, abnormal exit: %lu\n", GetLastError ()));
> >
> > If you see any messages, please post them here.
> 
> Mmmm I'm afraid I can't do that, be cause that machine doesn't have
> MinGW and I'm not allowed to install it there.

So you are allowed to bring the Emacs executable there, but not the
GDB executable?  gdb.exe and its dependency DLLs is all you need.

The debugging messages are received by any debugger attached to Emacs,
not just GDB.  So if you have there Windows Debugging Tools, like
WinDbg, you can use that to display those messages.

> Another possibility would be to compile a modified version of Emacs
> which dumps any useful information to a file, for example.

You could redefine the DebPrint macro so that it prints stuff to
stderr.  Then invoke emacs.exe from the shell prompt, not via
runemacs.exe or a desktop icon, and this stuff will go to the shell
window.  (Enlarge the "Screeen Buffer Size Height" in the Properties
of the Command Prompt window to a large number, like 5000, to be able
to see many past messages.)

Without knowing what goes wrong with Samba-mounted volumes, the only
way to deal with this is to customize auto-revert-use-notify to nil,
which will fall back to the old methods of auto-reverting.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

>> Mmmm I'm afraid I can't do that, be cause that machine doesn't have
>> MinGW and I'm not allowed to install it there.
>
> So you are allowed to bring the Emacs executable there, but not the
> GDB executable?  gdb.exe and its dependency DLLs is all you need.

I thought I'd need a more complex installation, like installing MinGW.
 Exactly what files do I need and where can I get them?

> The debugging messages are received by any debugger attached to Emacs,
> not just GDB.  So if you have there Windows Debugging Tools, like
> WinDbg, you can use that to display those messages.

I've just tried this: From Visual Studio 2008, "tools" menu -> "Attach
to Process...", select the emacs.exe process and from Emacs eval the
expression sent by Michael (w32notify-add-watch "t:\myfile" '(size
last-write-time) 'ignore).  But I see nothing in the "Output" window
of VS.

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

On Thu, Jan 24, 2013 at 6:23 PM, Dani Moncayo <dmoncayo@gmail.com> wrote:
>>> Mmmm I'm afraid I can't do that, be cause that machine doesn't have
>>> MinGW and I'm not allowed to install it there.
>>
>> So you are allowed to bring the Emacs executable there, but not the
>> GDB executable?  gdb.exe and its dependency DLLs is all you need.
>
> I thought I'd need a more complex installation, like installing MinGW.
>  Exactly what files do I need and where can I get them?

Actually I think I could Install MinGW there.  I'll try it.

(I hope my boss is not subscribed to this list :))

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Thu, 24 Jan 2013 18:23:00 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: michael.albinus@gmx.de, 13540@debbugs.gnu.org
> 
> I thought I'd need a more complex installation, like installing MinGW.
>  Exactly what files do I need and where can I get them?

I think these 4 is all you need:

  http://sourceforge.net/projects/mingw/files/MinGW/Extension/gdb/GDB-7.5/gdb-7.5-1-mingw32-bin.tar.lzma/download
  http://sourceforge.net/projects/mingw/files/MinGW/Extension/expat/expat-2.0.1-1/libexpat-2.0.1-1-mingw32-dll-1.tar.gz/download
  http://sourceforge.net/projects/mingw/files/MinGW/Base/libiconv/libiconv-1.14-2/libiconv-1.14-2-mingw32-dll-2.tar.lzma/download
  http://sourceforge.net/projects/mingw/files/MinGW/Base/gettext/gettext-0.18.1.1-2/libintl-0.18.1.1-2-mingw32-dll-8.tar.lzma/download

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Thu, 24 Jan 2013 18:23:00 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: michael.albinus@gmx.de, 13540@debbugs.gnu.org
> 
> > The debugging messages are received by any debugger attached to Emacs,
> > not just GDB.  So if you have there Windows Debugging Tools, like
> > WinDbg, you can use that to display those messages.
> 
> I've just tried this: From Visual Studio 2008, "tools" menu -> "Attach
> to Process...", select the emacs.exe process and from Emacs eval the
> expression sent by Michael (w32notify-add-watch "t:\myfile" '(size
> last-write-time) 'ignore).  But I see nothing in the "Output" window
> of VS.

I never did this with VS, so I don't know whether something else needs
to be done.  Perhaps the "Output" window is not the right one; do you
have there another window called "Debug" or "Debug output"?  If not,
search the VS documentation for how to display the text sent by the
program using the OutputDebugString API.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

>> >> Could you, please, check what
>> >>
>> >> (w32notify-add-watch "t:\myfile" '(size last-write-time) 'ignore)
>> >>
>> >> returns?
>> >
>> > It returns a number like 14510816, but the number varies each time I
>> > evaluate the expression.
>>
>> That's an indication that a watch has been installed. Eli, could you,
>> please, check?
>
> I don't have access to Samba-mounted volumes.
>
> Dani, if you attach a debugger to Emacs before evaluating the above
> expression, do you see any warning messages printed by GDB?  For
> example, this one:
>
>         DebPrint (("watch_worker, abnormal exit: %lu\n", GetLastError ()));

I don't see any interesting message in the GDB console when I eval the
elisp expression.  This is what I see now, after evaluating that
expression:

  C:\emacs\emacs-24.3.50\bin>gdb --args emacs.exe -Q
   GNU gdb (GDB) 7.5
  Copyright (C) 2012 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
  and "show warranty" for details.
  This GDB was configured as "i686-pc-mingw32".
  For bug reporting instructions, please see:
  <http://www.gnu.org/software/gdb/bugs/>...
  Reading symbols from C:\emacs\emacs-24.3.50\bin\emacs.exe...done.
  (gdb) run
  Starting program: C:\emacs\emacs-24.3.50\bin\emacs.exe -Q
  [New Thread 5892.0xc44]
  [New Thread 5892.0x1418]
  [New Thread 5892.0x16c8]
  [New Thread 5892.0x16a0]
  [New Thread 5892.0x1700]
  [New Thread 5892.0x158c]
  [New Thread 5892.0x171c]


-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

> I don't see any interesting message in the GDB console when I eval the
> elisp expression.  This is what I see now, after evaluating that
> expression:

And FWIW, the elisp expression also returns an integer when I supply a
nonexistent file as first argument, and GDB doesn't show any message
either (besides the "[New Thread ...]").

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Thu, 24 Jan 2013 19:13:31 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Michael Albinus <michael.albinus@gmx.de>, 13540@debbugs.gnu.org
> 
> > I don't see any interesting message in the GDB console when I eval the
> > elisp expression.  This is what I see now, after evaluating that
> > expression:
> 
> And FWIW, the elisp expression also returns an integer when I supply a
> nonexistent file as first argument, and GDB doesn't show any message
> either (besides the "[New Thread ...]").

This is normal.  Windows can only watch an entire directory, so
w32notify ignores the file name and sets the watch on its parent
directory.  It doesn't care whether or not the file exists, only
whether its parent directory exists.  As a nice side effect, you can
watch file creation.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Thu, 24 Jan 2013 18:44:56 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: Michael Albinus <michael.albinus@gmx.de>, 13540@debbugs.gnu.org
> 
>   (gdb) run
>   Starting program: C:\emacs\emacs-24.3.50\bin\emacs.exe -Q
>   [New Thread 5892.0xc44]
>   [New Thread 5892.0x1418]
>   [New Thread 5892.0x16c8]
>   [New Thread 5892.0x16a0]
>   [New Thread 5892.0x1700]
>   [New Thread 5892.0x158c]
>   [New Thread 5892.0x171c]

Each time you evaluate the expression that calls w32notify-add-watch,
you should see another "New Thread" announcement.  This is normal,
since each watch you add starts a new thread where the watching code
runs.

Can you use Process Explorer or "pslist -d emacs" (both tools are from
the Sysinternals package) to see if the threads started when you
invoke w32notify-add-watch keep running, or die/exit?

The lack of any messages suggests that the threads continue to run,
but file notifications never come in, which probably means we cannot
detect this problem automatically, and must rely on the users who know
which volumes can and cannot be supported.

Also, I presume that you tried to modify files on the mounted volumes,
while you had GDB attached to Emacs.  If not, please try modifying
files there, and see if that triggers any debug messages.

Thanks.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

> Each time you evaluate the expression that calls w32notify-add-watch,
> you should see another "New Thread" announcement.

Yes I do.

> This is normal,
> since each watch you add starts a new thread where the watching code
> runs.

Ok.

> Can you use Process Explorer or "pslist -d emacs" (both tools are from
> the Sysinternals package) to see if the threads started when you
> invoke w32notify-add-watch keep running, or die/exit?

I observe (with process explorer) that the new threads keep running.

> The lack of any messages suggests that the threads continue to run,
> but file notifications never come in, which probably means we cannot
> detect this problem automatically, and must rely on the users who know
> which volumes can and cannot be supported.

Ok, if Emacs cannot determine itself whether file notifications work,
I'll set `auto-revert-use-notify' to nil on that machine.

> Also, I presume that you tried to modify files on the mounted volumes,
> while you had GDB attached to Emacs.

Of course.  I started and "emacs -Q" process under GDB, and from that
emacs session I visited the file on the mounted volume and turned on
auto-revert-mode.  Then, if I modify the file with another program,
the emacs buffer isn't reverted and the GDB console doesn't show any
message.

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Fri, 25 Jan 2013 08:46:59 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: michael.albinus@gmx.de, 13540@debbugs.gnu.org
> 
> > Each time you evaluate the expression that calls w32notify-add-watch,
> > you should see another "New Thread" announcement.
> 
> Yes I do.
> 
> > This is normal,
> > since each watch you add starts a new thread where the watching code
> > runs.
> 
> Ok.
> 
> > Can you use Process Explorer or "pslist -d emacs" (both tools are from
> > the Sysinternals package) to see if the threads started when you
> > invoke w32notify-add-watch keep running, or die/exit?
> 
> I observe (with process explorer) that the new threads keep running.
> 
> > The lack of any messages suggests that the threads continue to run,
> > but file notifications never come in, which probably means we cannot
> > detect this problem automatically, and must rely on the users who know
> > which volumes can and cannot be supported.
> 
> Ok, if Emacs cannot determine itself whether file notifications work,
> I'll set `auto-revert-use-notify' to nil on that machine.
> 
> > Also, I presume that you tried to modify files on the mounted volumes,
> > while you had GDB attached to Emacs.
> 
> Of course.  I started and "emacs -Q" process under GDB, and from that
> emacs session I visited the file on the mounted volume and turned on
> auto-revert-mode.  Then, if I modify the file with another program,
> the emacs buffer isn't reverted and the GDB console doesn't show any
> message.

Thanks.

I guess autorevert should allow users to specify whether notifications
should be used on a per filesystem basis, rather than just globally
for the entire session.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> I guess autorevert should allow users to specify whether notifications
> should be used on a per filesystem basis, rather than just globally
> for the entire session.

Makes sense to me. With this mechanism, a user could also decide,
whether remote files shall be reverted based on file notifications, or
not. I'll prepare a patch for autorevert.el.

Maybe we could also enhance the docstring of w32notify-add-watch, that
even a returned descriptor does not guarantee that the files are
watched. Whether we need it for inotify-add-watch as well I don't
know. I've tested shortly on an NFS mounted filesystem, which is
exported from my NAS, and there were no problems (notifications did
arrive). But the more interesting test might be SMB mounts, which I
haven't available locally. I should ask my wife to share something to me
from her laptop :-)

Best regards, Michael.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Dani Moncayo <dmoncayo@gmail.com>,  13540@debbugs.gnu.org
> Date: Fri, 25 Jan 2013 09:13:55 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I guess autorevert should allow users to specify whether notifications
> > should be used on a per filesystem basis, rather than just globally
> > for the entire session.
> 
> Makes sense to me. With this mechanism, a user could also decide,
> whether remote files shall be reverted based on file notifications, or
> not. I'll prepare a patch for autorevert.el.

Thanks.

> Maybe we could also enhance the docstring of w32notify-add-watch, that
> even a returned descriptor does not guarantee that the files are
> watched.

Done in trunk revision 111601.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Fri, 25 Jan 2013 09:57:17 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: michael.albinus@gmx.de, 13540@debbugs.gnu.org
> 
> > > The lack of any messages suggests that the threads continue to run,
> > > but file notifications never come in, which probably means we cannot
> > > detect this problem automatically, and must rely on the users who know
> > > which volumes can and cannot be supported.
> > 
> > Ok, if Emacs cannot determine itself whether file notifications work,
> > I'll set `auto-revert-use-notify' to nil on that machine.
> > 
> > > Also, I presume that you tried to modify files on the mounted volumes,
> > > while you had GDB attached to Emacs.
> > 
> > Of course.  I started and "emacs -Q" process under GDB, and from that
> > emacs session I visited the file on the mounted volume and turned on
> > auto-revert-mode.  Then, if I modify the file with another program,
> > the emacs buffer isn't reverted and the GDB console doesn't show any
> > message.
> 
> Thanks.
> 
> I guess autorevert should allow users to specify whether notifications
> should be used on a per filesystem basis, rather than just globally
> for the entire session.

Actually, there's one more thing I'd ask you to check.  Can you put a
breakpoint inside the watch_completion function, say on this line:

  if (!io_info) <<<<<<<<<<<<<<<<<<<<<
    return;

and see if it ever breaks when you modify files in the directory that
is being watched?

The issue here is: do any notifications, even invalid ones, come in,
or not?  If we get invalid notifications, we could perhaps signal to
the main thread that the watch is not working, and autorevert could
then fall back on the old methods automatically.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

> Actually, there's one more thing I'd ask you to check.  Can you put a
> breakpoint inside the watch_completion function, say on this line:
>
>   if (!io_info) <<<<<<<<<<<<<<<<<<<<<
>     return;
>
> and see if it ever breaks when you modify files in the directory that
> is being watched?

Can I do that with a binary distribution of Emacs (if so, how?), or do
I need a full distribution with sourcecode?  (sorry I know very little
about GDB)

Here at work I have only binary distributions of Emacs.

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Fri, 25 Jan 2013 09:29:12 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: michael.albinus@gmx.de, 13540@debbugs.gnu.org
> 
> > Actually, there's one more thing I'd ask you to check.  Can you put a
> > breakpoint inside the watch_completion function, say on this line:
> >
> >   if (!io_info) <<<<<<<<<<<<<<<<<<<<<
> >     return;
> >
> > and see if it ever breaks when you modify files in the directory that
> > is being watched?
> 
> Can I do that with a binary distribution of Emacs (if so, how?), or do
> I need a full distribution with sourcecode?

Line number information is recorded in the binary as well.  So:

 > gdb emacs.exe
 (gdb) break w32notify.c:206
 (gdb) r

should set you up.  If the breakpoint breaks, GDB will complain that
sources are not available, so it cannot display the source.  But it
doesn't matter, since all I'm interested in is whether it breaks and
if so, what do we get in io_info->hEvent object.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

>  > gdb emacs.exe
>  (gdb) break w32notify.c:206
>  (gdb) r
>
> should set you up.  If the breakpoint breaks, GDB will complain that
> sources are not available, so it cannot display the source.  But it
> doesn't matter, since all I'm interested in is whether it breaks and
> if so, what do we get in io_info->hEvent object.

If the file is in the network volume, it doesn't break.

If the file is stored locally, it breaks two times when the file being
watched is updated (I have to type `c' two times).

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Fri, 25 Jan 2013 10:34:45 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: michael.albinus@gmx.de, 13540@debbugs.gnu.org
> 
> >  > gdb emacs.exe
> >  (gdb) break w32notify.c:206
> >  (gdb) r
> >
> > should set you up.  If the breakpoint breaks, GDB will complain that
> > sources are not available, so it cannot display the source.  But it
> > doesn't matter, since all I'm interested in is whether it breaks and
> > if so, what do we get in io_info->hEvent object.
> 
> If the file is in the network volume, it doesn't break.

Thanks.  So I guess user customizations is the only way to deal with
this.

> If the file is stored locally, it breaks two times when the file being
> watched is updated (I have to type `c' two times).

How did you "update the file", exactly?  I just type something like

  echo "foo" >> FILE

This results in only one notification, not two.  Maybe your update
method causes the file to be modified twice in a row.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Dani Moncayo]

>> If the file is stored locally, it breaks two times when the file being
>> watched is updated (I have to type `c' two times).
>
> How did you "update the file", exactly?  I just type something like
>
>   echo "foo" >> FILE
>
> This results in only one notification, not two.  Maybe your update
> method causes the file to be modified twice in a row.

If I update the file that way (echo "foo" >> FILE), I get a single
break like you, but I get two breaks in these cases:
* Doing [echo "foo" > FILE] (i.e. replacing the file entirely).
* Updating the file using notepad.
* Updating the file using another instance of Emacs.

-- 
Dani Moncayo

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> Date: Fri, 25 Jan 2013 12:19:20 +0100
> From: Dani Moncayo <dmoncayo@gmail.com>
> Cc: michael.albinus@gmx.de, 13540@debbugs.gnu.org
> 
> >> If the file is stored locally, it breaks two times when the file being
> >> watched is updated (I have to type `c' two times).
> >
> > How did you "update the file", exactly?  I just type something like
> >
> >   echo "foo" >> FILE
> >
> > This results in only one notification, not two.  Maybe your update
> > method causes the file to be modified twice in a row.
> 
> If I update the file that way (echo "foo" >> FILE), I get a single
> break like you, but I get two breaks in these cases:
> * Doing [echo "foo" > FILE] (i.e. replacing the file entirely).
> * Updating the file using notepad.
> * Updating the file using another instance of Emacs.

That's expected: you get one notification when the file is shrunk to
zero size (this happens when the application opens the file for
writing), and then again when the new contents is written to it.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> If the file is in the network volume, it doesn't break.
>
> Thanks.  So I guess user customizations is the only way to deal with
> this.

Well, I have committed this to trunk in revision #111603. There is now
an option `auto-revert-notify-exclude-dir-regexp', which could be used
to exclude directories from file notifications, and let files
autorevert via the file checks as done until now. Dani, in your case it
might be sufficient to apply

(setq auto-revert-notify-exclude-dir-regexp (regexp-quote "t:\myfile"))

I gave that option an initial value to exclude mounted files and remote
files. This value might be tuned in further tests.

With this patch I have also committed two other changes which I have
tested last days:

- File notifications are established for directories instead of single
  files. w32notify does it by default already, and for the inotify setup
  this is achieved by watching default-directory in autorevert.

- There is a new option `auto-revert-remote-files', which allows you to
  revert remote files. Until now, remote files were excluded. The
  default value of this option is nil.

I've started also performance tests, in order to see the difference
between file notifications and explicit file checks. This is not
finished yet.

Best regards, Michael.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> If the file is in the network volume, it doesn't break.
>>
>> Thanks.  So I guess user customizations is the only way to deal with
>> this.
>
> Well, I have committed this to trunk in revision #111603. There is now
> an option `auto-revert-notify-exclude-dir-regexp', which could be used
> to exclude directories from file notifications, and let files
> autorevert via the file checks as done until now. Dani, in your case it
> might be sufficient to apply
>
> (setq auto-revert-notify-exclude-dir-regexp (regexp-quote "t:\myfile"))
>
> I gave that option an initial value to exclude mounted files and remote
> files. This value might be tuned in further tests.

bug#13540: 24.3.50; Regression in trunk: auto-revert-mode fails on files accessed via samba

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Thu, 24 Jan 2013 11:13:52 +0100
> Cc: 13540@debbugs.gnu.org
> 
> Likely, it is not possible to enable file notifications for mounted file
> systems under Windows. In that case, `w32notify-add-watch' shall return
> nil, and the code in auto-revert.el shall fall back to the previous
> implementation.

Actually, both w32notify-add-watch and inotify-add-watch signal an
error in that case.  It's auto-revert that catches the error with
ignore-errors and converts that to nil.  I guess we need to update the
doc strings.

Anyway, w32notify does support mounted filesystems, as long as the
server that does the mounting supports the ReadDirectoryChangesW API
which w32notify uses.  Googling suggests that Samba has problems in
this area, see, for example this page:

  http://qualapps.blogspot.co.il/2010/05/understanding-readdirectorychangesw_19.html

(which btw was a great source of information for me while I worked on
w32notify.c).  It says:

   Network Drives

   ReadDirectoryChangesW works with network drives, but only if the
   remote server supports the functionality. Drives shared from other
   Windows-based computers will correctly generate notifications. Samba
   servers may or may not generate notifications, depending on whether
   the underlying operating system supports the functionality. Network
   Attached Storage (NAS) devices usually run Linux, so won't support
   notifications. High-end SANs are anybody's guess.

It could be that the API we use does not fail, but the notifications
simply don't come in from that volume.  I asked Dani to report if
there's any sign of trouble when the watch is installed.  Depending on
the findings, we may or may not be able to detect the problem
automatically and fall back to the old method.

Failing that, I guess we will have to introduce more elaborate
customizations that would allow users to specify filesystems that
should or should not use file notifications.  (Doesn't inotify have
similar issues with NFS-mounted volumes?)