bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Bertrand Brelier]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

Hello everybody,

I am using emacs on a Ubuntu server (14.04) to edit files.

I am editing files that are located in an encrypted directory with encfs
located on a Microsoft server (accessed with cifs).

When I edit the file and save it, and I continue to edit the file (without
closing it), I have the following issue where emacs thinks that the file
has changed on disk :

changed on disk; really edit the buffer? (y, n, r or C-h)
has changed since visited or saved.  Save anyway? (yes or no)

This problem is reproducible (it actually happens every-time) but only
happens with Emacs 24.4.1 and not with Emacs 24.3.1.

Also, I do not have the problem when I use an encrypted directory locally
(not on an external server) or when I use an non-encrypted directory on the
Microsoft server.

I have also checked that the file is not used or modified by another user
or program and since I do not have any issue with the version 24.3.1 on the
same files, I am wondering if the issue could come from the version 24.4.1

Please let me know if you need further information to investigate the issue.

Thank you for your help, I really appreciate working with emacs.

Regards,

Bertrand

[-- Attachment #2: Type: text/html, Size: 1377 bytes --]

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

If memory serves, Emacs 24.4 is a bit more careful to detect race conditions 
when a file changes while you're editing it, and most likely this is running 
afoul of some aspect of encfs.  I suggest running 'strace -o emacs.tr emacs -Q', 
reproducing the problem, and looking at the resulting trace file.  You can do 
the same thing with both old and new Emacs and see how the traces differ.  You 
can email the traces to 19607@debbugs.gnu.org.  We don't need the full traces, 
just the parts relevant to accessing the files in question, notably the file 
status and time stamps.

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Bertrand Brelier]

[-- Attachment #1.1: Type: text/plain, Size: 1101 bytes --]

Dear Paul,

Thank you for your help.

I reproduced the problem and saved the trace files in the attached tar
file.
I edited two files with the 2 emacs versions with the commands :

strace -o emacs24.4.1.tr emacs-24.4 -Q Test24.4.1.txt
strace -o emacs24.3.1.tr emacs24 -Q Test24.3.1.txt

I was not sure how to just select the file access, so I provided the full
trace files.

Let me know if you need anything-else.

Regards,

Bertrand

On Tue, Jan 20, 2015 at 4:39 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:

> If memory serves, Emacs 24.4 is a bit more careful to detect race
> conditions when a file changes while you're editing it, and most likely
> this is running afoul of some aspect of encfs.  I suggest running 'strace
> -o emacs.tr emacs -Q', reproducing the problem, and looking at the
> resulting trace file.  You can do the same thing with both old and new
> Emacs and see how the traces differ.  You can email the traces to
> 19607@debbugs.gnu.org.  We don't need the full traces, just the parts
> relevant to accessing the files in question, notably the file status and
> time stamps.
>

[-- Attachment #1.2: Type: text/html, Size: 1684 bytes --]

[-- Attachment #2: TraceFiles.tar.gz --]
[-- Type: application/x-gzip, Size: 133250 bytes --]

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

On 01/20/2015 05:19 AM, Bertrand Brelier wrote:
>
> Let me know if you need anything-else.

Thanks, those traces lack time stamps which are at the heart of the 
problem.  Can you please rerun both tests with time stamp details in the 
stat-related system calls?  Something like the following shell command 
should do it:

strace -o /tmp/tr -e abbrev=\!stat,fstat,lstat emacs -Q filename

Depending on your shell, you may need to omit the backslash.

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Bertrand Brelier]

[-- Attachment #1.1: Type: text/plain, Size: 666 bytes --]

Hello Paul,

Thanks for your help,

Here are the files with the time stamps included in the traces.

Cheers,

Bertrand

On Tue, Jan 20, 2015 at 7:21 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

> On 01/20/2015 05:19 AM, Bertrand Brelier wrote:
>
>>
>> Let me know if you need anything-else.
>>
>
> Thanks, those traces lack time stamps which are at the heart of the
> problem.  Can you please rerun both tests with time stamp details in the
> stat-related system calls?  Something like the following shell command
> should do it:
>
> strace -o /tmp/tr -e abbrev=\!stat,fstat,lstat emacs -Q filename
>
> Depending on your shell, you may need to omit the backslash.
>

[-- Attachment #1.2: Type: text/html, Size: 1179 bytes --]

[-- Attachment #2: TraceFiles.tar.gz --]
[-- Type: application/x-gzip, Size: 149058 bytes --]

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

Sorry, I'm still having trouble seeing what the bug is. strace doesn't output 
full time stamps; it tells us their values only to the nearest second.
Can you run Emacs under a debugger?  Here's a recipe, if you have the time:

$ wget ftp://ftp.gnu.org/gnu/emacs/emacs-24.4.tar.xz
$ tar xf emacs-24.4.tar.xz
$ cd emacs-24.4
$ ./configure CFLAGS='-g3 -O0'
$ make
$ cd src
$ gdb emacs
(gdb) source .gdbinit
(gdb) b fileio.c:5338
(gdb) r
[ Now edit the file in Emacs, until the breakpoint is hit. ]
(gdb) p mtime
(gdb) p b->modtime

I just now tried all that and got:

(gdb) p mtime
$5 = {
   tv_sec = 1421893112,
   tv_nsec = 705816459
}
(gdb) p b->modtime
$6 = {
   tv_sec = 1421893112,
   tv_nsec = 705816459
}

which is what I'd expect: the last-modified time of the file (mtime) equals the 
buffer's opinion of it (b->modtime). What do you get?

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Bertrand Brelier]

[-- Attachment #1: Type: text/plain, Size: 4498 bytes --]

Hello Paul,

I followed the steps but as soon as I try to edit the file when using gdb,
the emacs windows freezes and I cannot do anything (I tried to open a local
file, not on the network, with no encryption).

Here is what I did :
pwd
/home/bertrand/Emacs/emacs-24.4/src

gdb ./emacs
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./emacs...done.
warning: File "/home/bertrand/Emacs/emacs-24.4/src/.gdbinit" auto-loading
has been declined by your `auto-load safe-path' set to
"$debugdir:$datadir/auto-load".
To enable execution of this file add
        add-auto-load-safe-path /home/bertrand/Emacs/emacs-24.4/src/.gdbinit
line to your configuration file "/home/bertrand/.gdbinit".
To completely disable this security protection add
        set auto-load safe-path /
line to your configuration file "/home/bertrand/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the
shell:
        info "(gdb)Auto-loading safe path"
(gdb) source .gdbinit
SIGINT is used by the debugger.
Are you sure you want to change it? (y or n) [answered Y; input not from
terminal]
DISPLAY = localhost:10.0
TERM = xterm
Breakpoint 1 at 0x5432ba: file emacs.c, line 351.
Temporary breakpoint 2 at 0x56507f: file sysdep.c, line 850.
(gdb) b fileio.c:5338
Breakpoint 3 at 0x58e027: file fileio.c, line 5338.
(gdb) r
Starting program: /home/bertrand/Emacs/emacs-24.4/src/emacs
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffea11c700 (LWP 2263)]

** (emacs:2259): WARNING **: Couldn't connect to accessibility bus: Failed
to connect to socket /tmp/dbus-eho0AgRKqo: Connection refused
[New Thread 0x7fffe928d700 (LWP 2265)]
[New Thread 0x7fffe8a8c700 (LWP 2267)]
[New Thread 0x7fffd80a7700 (LWP 2268)]
[New Thread 0x7fffd78a6700 (LWP 2269)]
[Thread 0x7fffd78a6700 (LWP 2269) exited]
[New Thread 0x7fffd78a6700 (LWP 2270)]
[Thread 0x7fffd80a7700 (LWP 2268) exited]
[New Thread 0x7fffd80a7700 (LWP 2272)]
[Thread 0x7fffd78a6700 (LWP 2270) exited]

Breakpoint 3, Fverify_visited_file_modtime (buf=29001941) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb)
(gdb) quit
A debugging session is active.

        Inferior 1 [process 2259] will be killed.

Quit anyway? (y or n) y


I had to quit the gdb session as the emacs windows froze (repeated the
procedure 3 times and it froze each time).

I thought the problem could come from the bus warning but without gdb I
have the same warning but no issue to edit the same file :
./emacs

** (emacs:2275): WARNING **: Couldn't connect to accessibility bus: Failed
to connect to socket /tmp/dbus-eho0AgRKqo: Connection refused

Any suggestions ?

Thank you for your help,

Cheers,

Bertrand


On Wed, Jan 21, 2015 at 9:20 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

> Sorry, I'm still having trouble seeing what the bug is. strace doesn't
> output full time stamps; it tells us their values only to the nearest
> second.
> Can you run Emacs under a debugger?  Here's a recipe, if you have the time:
>
> $ wget ftp://ftp.gnu.org/gnu/emacs/emacs-24.4.tar.xz
> $ tar xf emacs-24.4.tar.xz
> $ cd emacs-24.4
> $ ./configure CFLAGS='-g3 -O0'
> $ make
> $ cd src
> $ gdb emacs
> (gdb) source .gdbinit
> (gdb) b fileio.c:5338
> (gdb) r
> [ Now edit the file in Emacs, until the breakpoint is hit. ]
> (gdb) p mtime
> (gdb) p b->modtime
>
> I just now tried all that and got:
>
> (gdb) p mtime
> $5 = {
>   tv_sec = 1421893112,
>   tv_nsec = 705816459
> }
> (gdb) p b->modtime
> $6 = {
>   tv_sec = 1421893112,
>   tv_nsec = 705816459
> }
>
> which is what I'd expect: the last-modified time of the file (mtime)
> equals the buffer's opinion of it (b->modtime). What do you get?
>

[-- Attachment #2: Type: text/html, Size: 5733 bytes --]

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

On 01/22/2015 06:16 AM, Bertrand Brelier wrote:
> Hello Paul,
>
> I followed the steps but as soon as I try to edit the file when using 
> gdb, the emacs windows freezes and I cannot do anything (I tried to 
> open a local file, not on the network, with no encryption).
>

That's interesting, as it worked for me.  We should try to see what differs.

Perhaps it's something involved with your startup.  Can you try running 
Emacs with the -Q option instead?  That is, instead of this:

(gdb) r

do this:

(gdb) r -Q

Sorry if this all seems slow and painstaking, but that's life with 
remote debugging.

For what it's worth, I tried to reproduce the problem with encfs on my 
machine (Fedora 21 x86-64) with a local directory, and could not 
reproduce it.

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Bertrand Brelier]

[-- Attachment #1: Type: text/plain, Size: 1445 bytes --]

Hello Paul,

Thanks for your help and patience.
I tried with r- Q but same issue, emacs freezes when I start editing the
file (I can open it but not modify its content), For this test, the file is
stored locally (no remote cifs access, no encryption),

I do not have the problem with encfs on a local directory, only when the
encrypted directory is stored on another server accessed with cifs. I do
not have any problem with a remote directory accessed with cifs it if is
not encrypted (I only have the issue when I use an encrypted directory with
cifs).

Cheers,

Bertrand



On Thu, Jan 22, 2015 at 3:55 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

> On 01/22/2015 06:16 AM, Bertrand Brelier wrote:
>
>> Hello Paul,
>>
>> I followed the steps but as soon as I try to edit the file when using
>> gdb, the emacs windows freezes and I cannot do anything (I tried to open a
>> local file, not on the network, with no encryption).
>>
>>
> That's interesting, as it worked for me.  We should try to see what
> differs.
>
> Perhaps it's something involved with your startup.  Can you try running
> Emacs with the -Q option instead?  That is, instead of this:
>
> (gdb) r
>
> do this:
>
> (gdb) r -Q
>
> Sorry if this all seems slow and painstaking, but that's life with remote
> debugging.
>
> For what it's worth, I tried to reproduce the problem with encfs on my
> machine (Fedora 21 x86-64) with a local directory, and could not reproduce
> it.
>

[-- Attachment #2: Type: text/html, Size: 2020 bytes --]

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

On 01/22/2015 01:26 PM, Bertrand Brelier wrote:
> I tried with r- Q but same issue, emacs freezes when I start editing 
> the file (I can open it but not modify its content)
Ah, sorry, I wasn't explicit enough.  How about this.  When Emacs 
freezes, print the requested values in the debugger and then continue 
with the "c" command.  I just now did that, with the following results 
for me:

$ gdb ./emacs
(gdb) source .gdbinit
(gdb) b fileio.c:5338
(gdb) r -Q
[In emacs, type C-x C-f abcdef RET, then type "x".]
Breakpoint 3, Fverify_visited_file_modtime (buf=27596677) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$1 = {
   tv_sec = 1421966150,
   tv_nsec = 791881570
}
(gdb) p b->modtime
$2 = {
   tv_sec = 1421966150,
   tv_nsec = 791881570
}
(gdb) c
Continuing.
[In emacs, type C-x C-s.]

Breakpoint 3, Fverify_visited_file_modtime (buf=27596677) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$3 = {
   tv_sec = 1421966150,
   tv_nsec = 791881570
}
(gdb) p b->modtime
$4 = {
   tv_sec = 1421966150,
   tv_nsec = 791881570
}
(gdb) c
Continuing.

Breakpoint 3, Fverify_visited_file_modtime (buf=27596677) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$5 = {
   tv_sec = 0,
   tv_nsec = -1
}
(gdb) p b->modtime
$6 = {
   tv_sec = 1421966150,
   tv_nsec = 791881570
}
(gdb) c
Continuing.
[At this point, Emacs says "Wrote /home/eggert/decrypted/abcdef.]

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Bertrand Brelier]

[-- Attachment #1.1: Type: text/plain, Size: 2046 bytes --]

Hello Paul,

Sorry, I have not used gdb for a while a forgot about asking the debugger
to continue.

I followed your request and you can see the log file attached to this email.

I modified the file 2 times : the first time, no issue I can save the file
but when I try to modify it a second time, emacs tells me :changed on disk;
really edit the buffer .....

Thanks for your help,

Cheers,

Bertrand

On Thu, Jan 22, 2015 at 5:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

> On 01/22/2015 01:26 PM, Bertrand Brelier wrote:
>
>> I tried with r- Q but same issue, emacs freezes when I start editing the
>> file (I can open it but not modify its content)
>>
> Ah, sorry, I wasn't explicit enough.  How about this.  When Emacs freezes,
> print the requested values in the debugger and then continue with the "c"
> command.  I just now did that, with the following results for me:
>
> $ gdb ./emacs
> (gdb) source .gdbinit
> (gdb) b fileio.c:5338
> (gdb) r -Q
> [In emacs, type C-x C-f abcdef RET, then type "x".]
> Breakpoint 3, Fverify_visited_file_modtime (buf=27596677) at fileio.c:5338
> 5338      if (timespec_cmp (mtime, b->modtime) == 0
> (gdb) p mtime
> $1 = {
>   tv_sec = 1421966150,
>   tv_nsec = 791881570
> }
> (gdb) p b->modtime
> $2 = {
>   tv_sec = 1421966150,
>   tv_nsec = 791881570
> }
> (gdb) c
> Continuing.
> [In emacs, type C-x C-s.]
>
> Breakpoint 3, Fverify_visited_file_modtime (buf=27596677) at fileio.c:5338
> 5338      if (timespec_cmp (mtime, b->modtime) == 0
> (gdb) p mtime
> $3 = {
>   tv_sec = 1421966150,
>   tv_nsec = 791881570
> }
> (gdb) p b->modtime
> $4 = {
>   tv_sec = 1421966150,
>   tv_nsec = 791881570
> }
> (gdb) c
> Continuing.
>
> Breakpoint 3, Fverify_visited_file_modtime (buf=27596677) at fileio.c:5338
> 5338      if (timespec_cmp (mtime, b->modtime) == 0
> (gdb) p mtime
> $5 = {
>   tv_sec = 0,
>   tv_nsec = -1
> }
> (gdb) p b->modtime
> $6 = {
>   tv_sec = 1421966150,
>   tv_nsec = 791881570
> }
> (gdb) c
> Continuing.
> [At this point, Emacs says "Wrote /home/eggert/decrypted/abcdef.]
>

[-- Attachment #1.2: Type: text/html, Size: 2938 bytes --]

[-- Attachment #2: log --]
[-- Type: application/octet-stream, Size: 4351 bytes --]

gdb ./emacs
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./emacs...done.
warning: File "/home/bertrand/Emacs/emacs-24.4/src/.gdbinit" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file add
        add-auto-load-safe-path /home/bertrand/Emacs/emacs-24.4/src/.gdbinit
line to your configuration file "/home/bertrand/.gdbinit".
To completely disable this security protection add
        set auto-load safe-path /
line to your configuration file "/home/bertrand/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
        info "(gdb)Auto-loading safe path"
(gdb) source .gdbinit
SIGINT is used by the debugger.
Are you sure you want to change it? (y or n) [answered Y; input not from terminal]
DISPLAY = localhost:10.0
TERM = xterm
Breakpoint 1 at 0x5432ba: file emacs.c, line 351.
Temporary breakpoint 2 at 0x56507f: file sysdep.c, line 850.
(gdb) b fileio.c:5338
Breakpoint 3 at 0x58e027: file fileio.c, line 5338.
(gdb) r -Q
Starting program: /home/bertrand/Emacs/emacs-24.4/src/emacs -Q
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffea11c700 (LWP 10714)]
[New Thread 0x7fffe928d700 (LWP 10716)]
[New Thread 0x7fffe8a8c700 (LWP 10718)]
[New Thread 0x7fffd8230700 (LWP 10719)]
[New Thread 0x7fffd7a2f700 (LWP 10720)]
[Thread 0x7fffd8230700 (LWP 10719) exited]
[New Thread 0x7fffd8230700 (LWP 10721)]
[Thread 0x7fffd7a2f700 (LWP 10720) exited]
[New Thread 0x7fffd7a2f700 (LWP 10722)]
[Thread 0x7fffd8230700 (LWP 10721) exited]

Breakpoint 3, Fverify_visited_file_modtime (buf=27021317) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$1 = {
  tv_sec = 1421845549, 
  tv_nsec = 345981000
}
(gdb) p b->modtime
$2 = {
  tv_sec = 1421845549, 
  tv_nsec = 345981000
}
(gdb) c
Continuing.
[Thread 0x7fffd7a2f700 (LWP 10722) exited]

Breakpoint 3, Fverify_visited_file_modtime (buf=27021317) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$3 = {
  tv_sec = 1421845549, 
  tv_nsec = 345981000
}
(gdb) p b->modtime
$4 = {
  tv_sec = 1421845549, 
  tv_nsec = 345981000
}
(gdb) c
Continuing.

Breakpoint 3, Fverify_visited_file_modtime (buf=27021317) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$5 = {
  tv_sec = 0, 
  tv_nsec = -1
}
(gdb) p b->modtime
$6 = {
  tv_sec = 1421845549, 
  tv_nsec = 345981000
}
(gdb) c
Continuing.

Breakpoint 3, Fverify_visited_file_modtime (buf=27021317) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$7 = {
  tv_sec = 1422018451, 
  tv_nsec = 132061000
}
(gdb) p b->modtime
$8 = {
  tv_sec = 1422018451, 
  tv_nsec = 40059000
}
(gdb) c
Continuing.

Breakpoint 3, Fverify_visited_file_modtime (buf=27021317) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$9 = {
  tv_sec = 1422018451, 
  tv_nsec = 132061000
}
(gdb) p b->modtime
$10 = {
  tv_sec = 1422018451, 
  tv_nsec = 40059000
}
(gdb) c
Continuing.

Breakpoint 3, Fverify_visited_file_modtime (buf=27021317) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$11 = {
  tv_sec = 0, 
  tv_nsec = -1
}
(gdb) p b->modtime
$12 = {
  tv_sec = 1422018451, 
  tv_nsec = 40059000
}
(gdb) c
Continuing.
[Thread 0x7fffe8a8c700 (LWP 10718) exited]
[Thread 0x7fffe928d700 (LWP 10716) exited]
[Thread 0x7fffea11c700 (LWP 10714) exited]
[Inferior 1 (process 10710) exited normally]
(gdb) quit

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

Thanks, the key problem is the fourth time the breakpoint is hit, where 
the output in your log looks like this:

Breakpoint 3, Fverify_visited_file_modtime (buf=27021317) at fileio.c:5338
5338      if (timespec_cmp (mtime, b->modtime) == 0
(gdb) p mtime
$7 = {
   tv_sec = 1422018451,
   tv_nsec = 132061000
}
(gdb) p b->modtime
$8 = {
   tv_sec = 1422018451,
   tv_nsec = 40059000
}

The two time stamps should be the same, but the nanoseconds component 
differ (the tv_sec components are the same, which is why we don't 
observe any bugs in the strace output).

I suspect a bug in the file system.  Can you please try running the 
attached program, with the current directory being that file system?  It 
should take about 2 seconds.  You might try running it twice (it may 
depend on whether the file already exists).

[-- Attachment #2: test.c --]
[-- Type: text/plain, Size: 1004 bytes --]

#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>

int
main (void)
{
  char const *filename = "test.txt";
  int fd = open(filename, O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666);
  if (fd < 0)
    return perror ("open"), 1;
  static char const message[] = "This is a test.\n";
  int messagelen = sizeof message - 1;
  if (write (fd, message, messagelen) != messagelen)
    return perror ("write"), 1;
  if (fsync (fd) != 0)
    return perror ("fsync"), 1;
  struct stat st1, st2;
  if (fstat (fd, &st1) != 0)
    return perror ("fstat"), 1;
  if (close (fd) != 0)
    return perror ("close"), 1;
  sleep (2);
  if (stat (filename, &st2) != 0)
    return perror ("stat"), 1;
  if (! (st1.st_mtim.tv_sec == st2.st_mtim.tv_sec
	 && st1.st_mtim.tv_nsec == st2.st_mtim.tv_nsec))
    {
      printf ("last-modified times do not conform to:\n"
	      "http://pubs.opengroup.org/onlinepubs/"
	      "9699919799/basedefs/V1_chap04.html#tag_04_08\n");
      return 1;
    }
  return 0;
}

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Bertrand Brelier]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

Hello Paul,

I ran your code (twice to make sure) and in both cases it returned :

last-modified times do not conform to:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_08

Cheers,

Bertrand

On Fri, Jan 23, 2015 at 5:27 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

> Thanks, the key problem is the fourth time the breakpoint is hit, where
> the output in your log looks like this:
>
> Breakpoint 3, Fverify_visited_file_modtime (buf=27021317) at fileio.c:5338
> 5338      if (timespec_cmp (mtime, b->modtime) == 0
> (gdb) p mtime
> $7 = {
>   tv_sec = 1422018451,
>   tv_nsec = 132061000
> }
> (gdb) p b->modtime
> $8 = {
>   tv_sec = 1422018451,
>   tv_nsec = 40059000
> }
>
> The two time stamps should be the same, but the nanoseconds component
> differ (the tv_sec components are the same, which is why we don't observe
> any bugs in the strace output).
>
> I suspect a bug in the file system.  Can you please try running the
> attached program, with the current directory being that file system?  It
> should take about 2 seconds.  You might try running it twice (it may depend
> on whether the file already exists).
>

[-- Attachment #2: Type: text/html, Size: 1695 bytes --]

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

OK, thanks, then this is definitely not a GNU Emacs bug, since the 
problem can be reproduced by the small C test program without involving 
Emacs.  I have filed a bug report for encfs here:

https://github.com/vgough/encfs/issues/52

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

I see from Jakob's email that the bug can be reproduced on CIFS without 
EncFS.  However, there's still something wrong here, as Emacs has code 
to work around the CIFS bug; see the comment about CIFS in 
emacs/src/fileio.c starting at line 4897, here:

http://git.savannah.gnu.org/cgit/emacs.git/tree/src/fileio.c?id=a56eab8259568ea1389e972623e46359e73c0233#n4897

Unfortunately, this workaround for the CIFS bug does not appear to 
suffice for EncFS over CIFS.

Bertrand, can you please try the attached program, both on plain CIFS 
and on EncFS over CIFS?  It attempts to mimic Emacs's actions more 
precisely than the previous test program, which may help us to narrow 
down the problem.  Thanks.

[-- Attachment #2: test.c --]
[-- Type: text/plain, Size: 1311 bytes --]

#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>

int
main (void)
{
  char const *filename = "test.txt";
  int fd = open(filename, O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666);
  if (fd < 0)
    return perror ("open"), 1;
  static char const message[] = "This is a test.\n";
  int messagelen = sizeof message - 1;
  if (write (fd, message, messagelen) != messagelen)
    return perror ("write"), 1;
  if (fsync (fd) != 0)
    return perror ("fsync"), 1;
  struct stat st1, st2, st3;
  if (fstat (fd, &st1) != 0)
    return perror ("fstat"), 1;
  if (close (fd) != 0)
    return perror ("close"), 1;
  int fd2 = open(filename, O_WRONLY|O_CLOEXEC);
  if (fd2 < 0)
    return perror ("open 2"), 1;
  if (fstat (fd2, &st2) != 0)
    return perror ("fstat 2"), 1;
  if (close (fd2) != 0)
    return perror ("close 2"), 1;
  if (! (st1.st_mtim.tv_sec == st2.st_mtim.tv_sec
	 && st1.st_mtim.tv_nsec == st2.st_mtim.tv_nsec))
    printf ("Emacs should work around this POSIX-conformance bug.\n");
  sleep (2);
  if (stat (filename, &st3) != 0)
    return perror ("stat"), 1;
  if (! (st2.st_mtim.tv_sec == st3.st_mtim.tv_sec
	 && st2.st_mtim.tv_nsec == st3.st_mtim.tv_nsec))
    {
      printf ("Emacs does not work around this POSIX-conformance bug.\n");
      return 1;
    }
  return 0;
}

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Jakob Unterwurzacher]

Looks like that's it. I have added printfs so I understand what is 
going on:

    CIFS:

        Emacs should work around this POSIX-conformance bug.
        stat #1: 1422398418.561215369
        stat #2: 1422398418.561215300
        stat #3: 1422398418.561215300


    CIFS + encfs:

        Emacs does not work around this POSIX-conformance bug.
        stat #1: 1422398433.735274107
        stat #2: 1422398433.735274107
        stat #3: 1422398433.735274100


    CIFS + "encfs -o attr_timeout=0":

        Emacs should work around this POSIX-conformance bug.
        stat #1: 1422398527.607547169
        stat #2: 1422398527.607547100
        stat #3: 1422398527.607547100


The kernel caches FUSE stat() values for 1 second by default, this 
defeats emacs' workaround.
Passing "-o attr_timeout=0" to encfs disables that cache.

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

Jakob Unterwurzacher wrote:
> The kernel caches FUSE stat() values for 1 second by default, this defeats
> emacs' workaround.
> Passing "-o attr_timeout=0" to encfs disables that cache.

Looking at the fuse man page, it appears that if one uses EncFS atop a network 
file system, the mount should not use the kernel_cache option, and should use 
the auto_cache option or perhaps ac_attr_timeout=0 (the man page is unclear 
about all this, unfortunately).  Perhaps EncFS should use appropriate settings 
for these options by default if it determines that the underlying file system is 
network-based.

Anyway, assuming "-o attr_timeout=0" (or something similar) solves the problem, 
we do have a workaround now.

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Glenn Morris]

So does Emacs need to do anything about this, or should this be closed?

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Paul Eggert]

Glenn Morris wrote:
>
> So does Emacs need to do anything about this, or should this be closed?
>

I'll close it.  It's an open bug in encfs:

https://github.com/vgough/encfs/issues/52

and the bug needs to be fixed there regardless of what Emacs does.  I don't see 
an easy way for Emacs to work around the bug.

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Bertrand Brelier]

[-- Attachment #1: Type: text/plain, Size: 1026 bytes --]

Hello Paul,

I tested the new code :

plain CIFS :
Emacs should work around this POSIX-conformance bug.

on EncFS over CIFS :
Emacs does not work around this POSIX-conformance bug.

Thanks for your help,

Cheers,

Bertrand





On Tue, Jan 27, 2015 at 5:29 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:

> I see from Jakob's email that the bug can be reproduced on CIFS without
> EncFS.  However, there's still something wrong here, as Emacs has code to
> work around the CIFS bug; see the comment about CIFS in emacs/src/fileio.c
> starting at line 4897, here:
>
> http://git.savannah.gnu.org/cgit/emacs.git/tree/src/fileio.c?id=
> a56eab8259568ea1389e972623e46359e73c0233#n4897
>
> Unfortunately, this workaround for the CIFS bug does not appear to suffice
> for EncFS over CIFS.
>
> Bertrand, can you please try the attached program, both on plain CIFS and
> on EncFS over CIFS?  It attempts to mimic Emacs's actions more precisely
> than the previous test program, which may help us to narrow down the
> problem.  Thanks.
>

[-- Attachment #2: Type: text/html, Size: 1861 bytes --]

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk; really edit the buffer

[Jakob Unterwurzacher]

Encfs contributor here.

Interesting problem.
Does the test program run on the plain cifs mount (without encfs in 
between)
without complaining?

bug#19607: issue with Emacs 24.4.1 but not with 24.3.1 : changed on disk;

[Jakob Unterwurzacher]

This can also be reproduced on CIFS (without EncFS).
Forwarded to https://bugzilla.samba.org/show_bug.cgi?id=11078