Making fsync() optional

Making fsync() optional

[Romain Francoise]

Would people be conceptually opposed to making the fsync() call optional
in Fwrite_region?  Calling it after each write defeats the kernel's
ability to commit consecutive buffers to disk in one go, which can be
beneficial in a lot of cases (like nnml files in an nnml directory).  It
also means that the user has to wait for the file to hit the disk when
saving files interactively (under heavy system load, this can take a
while).  It also forces the disk to spin up on laptops (even with laptop
mode and friends).

Of course calling fsync() is safer if the system goes down right after
saving the file, but for systems with uninterruptible power, it doesn't
matter.

Tentative patch:

Index: src/fileio.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/fileio.c,v
retrieving revision 1.553
diff -c -r1.553 fileio.c
*** src/fileio.c	12 Sep 2005 13:42:49 -0000	1.553
--- src/fileio.c	12 Sep 2005 18:49:20 -0000
***************
*** 225,230 ****
--- 225,236 ----
     expanding file names.  This can be bound to / or \. */
  Lisp_Object Vdirectory_sep_char;
  
+ #ifdef HAVE_FSYNC
+ /* Nonzero means avoid calling fsync() after each write in
+    Fwrite-region.  */
+ int inhibit_fsync;
+ #endif
+ 
  extern Lisp_Object Vuser_login_name;
  
  #ifdef WINDOWSNT
***************
*** 5298,5304 ****
       Disk full in NFS may be reported here.  */
    /* mib says that closing the file will try to write as fast as NFS can do
       it, and that means the fsync here is not crucial for autosave files.  */
!   if (!auto_saving && fsync (desc) < 0)
      {
        /* If fsync fails with EINTR, don't treat that as serious.  */
        if (errno != EINTR)
--- 5304,5310 ----
       Disk full in NFS may be reported here.  */
    /* mib says that closing the file will try to write as fast as NFS can do
       it, and that means the fsync here is not crucial for autosave files.  */
!   if (!auto_saving && !inhibit_fsync && fsync (desc) < 0)
      {
        /* If fsync fails with EINTR, don't treat that as serious.  */
        if (errno != EINTR)
***************
*** 6743,6748 ****
--- 6749,6760 ----
  shortly after Emacs reads your `.emacs' file, if you have not yet given it
  a non-nil value.  */);
    Vauto_save_list_file_name = Qnil;
+ 
+ #ifdef HAVE_FSYNC
+   DEFVAR_BOOL ("inhibit-fsync", &inhibit_fsync,
+ 	       doc: /* Non-nil means avoid calling fsync() after each save.  */);
+   inhibit_fsync = 0;
+ #endif
  
    defsubr (&Sfind_file_name_handler);
    defsubr (&Sfile_name_directory);

-- 
Romain Francoise <romain@orebokech.com> | I've become someone else's
it's a miracle -- http://orebokech.com/ | nightmare...

Re: Making fsync() optional

[Richard M. Stallman]

    Would people be conceptually opposed to making the fsync() call optional
    in Fwrite_region?  Calling it after each write

It is not called after each write, just once at the end of writing the
whole file.

This is very important for safety.  Without this, you can type C-x
C-s, and see the command finish, and see the message that the file has
been written, but you can still lose it all if the system crashes
after that.

However, I would not mind adding a flag to turn it off, as long as the
doc string of that flag warns that this is dangerous.

  It also forces the disk to spin up on laptops (even with laptop
    mode and friends).

It has to do that anyway.  I don't want it to wait a minute before it
writes the file!

    Of course calling fsync() is safer if the system goes down right after
    saving the file, but for systems with uninterruptible power, it doesn't
    matter.

A power failure is not the only cause of crashes.

Re: Making fsync() optional

[Romain Francoise]

"Richard M. Stallman" <rms@gnu.org> writes:

> It is not called after each write, just once at the end of writing the
> whole file.

Yes, that's what I meant, sorry for the confusion.

> This is very important for safety.  Without this, you can type C-x
> C-s, and see the command finish, and see the message that the file has
> been written, but you can still lose it all if the system crashes
> after that.

In theory, yes.  In practice, IDE drives use write caching and lie to
the kernel about the status of the data: even if fsync() returns the
data may not be on the platter.  The drives do that to write data to
disk out of order, and to be able to delay writing blocks as long as
needed under heavy seek load.  The only way to ensure immediate data
consistency is to disable write caching (on GNU/Linux, using the -W
option to hdparm) or to use SCSI disks.

> However, I would not mind adding a flag to turn it off, as long as the
> doc string of that flag warns that this is dangerous.

OK.  I'm proposing a revised patch below.

I'm not sure if I should document the variable in the manual: it is
rather specialized so people who need it will know what it does and
won't need the manual to find it... and it is self-explanatory.  (And as
it's potentially dangerous, we don't want inexperienced users to enable
it.)

>> It also forces the disk to spin up on laptops (even with laptop mode
>> and friends).

> It has to do that anyway.  I don't want it to wait a minute before it
> writes the file!

There are specialized modes in the Linux kernel that cache all data for
some time and spin up the drive periodically only to commit all the
data, then spin down the drive, etc.  This allows for maximum power
saving on laptops--provided no fsync() calls are issued.
(Of course, not everyone wants this.)

Proposed patch:

Index: src/fileio.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/fileio.c,v
retrieving revision 1.553
diff -c -r1.553 fileio.c
*** src/fileio.c	12 Sep 2005 13:42:49 -0000	1.553
--- src/fileio.c	13 Sep 2005 17:54:16 -0000
***************
*** 225,230 ****
--- 225,235 ----
     expanding file names.  This can be bound to / or \. */
  Lisp_Object Vdirectory_sep_char;
  
+ #ifdef HAVE_FSYNC
+ /* Nonzero means skip the call to fsync() in Fwrite-region.  */
+ int inhibit_fsync;
+ #endif
+ 
  extern Lisp_Object Vuser_login_name;
  
  #ifdef WINDOWSNT
***************
*** 5298,5304 ****
       Disk full in NFS may be reported here.  */
    /* mib says that closing the file will try to write as fast as NFS can do
       it, and that means the fsync here is not crucial for autosave files.  */
!   if (!auto_saving && fsync (desc) < 0)
      {
        /* If fsync fails with EINTR, don't treat that as serious.  */
        if (errno != EINTR)
--- 5303,5309 ----
       Disk full in NFS may be reported here.  */
    /* mib says that closing the file will try to write as fast as NFS can do
       it, and that means the fsync here is not crucial for autosave files.  */
!   if (!auto_saving && !inhibit_fsync && fsync (desc) < 0)
      {
        /* If fsync fails with EINTR, don't treat that as serious.  */
        if (errno != EINTR)
***************
*** 6743,6748 ****
--- 6748,6760 ----
  shortly after Emacs reads your `.emacs' file, if you have not yet given it
  a non-nil value.  */);
    Vauto_save_list_file_name = Qnil;
+ 
+ #ifdef HAVE_FSYNC
+   DEFVAR_BOOL ("inhibit-fsync", &inhibit_fsync,
+ 	       doc: /* Non-nil means don't call fsync() after saving files.
+ Enabling this variable may result in data loss!  */);
+   inhibit_fsync = 0;
+ #endif
  
    defsubr (&Sfind_file_name_handler);
    defsubr (&Sfile_name_directory);

-- 
Romain Francoise <romain@orebokech.com> | I've become someone else's
it's a miracle -- http://orebokech.com/ | nightmare...

Re: Making fsync() optional

[Richard M. Stallman]

    In theory, yes.  In practice, IDE drives use write caching and lie to
    the kernel about the status of the data: even if fsync() returns the
    data may not be on the platter.  The drives do that to write data to
    disk out of order, and to be able to delay writing blocks as long as
    needed under heavy seek load.  The only way to ensure immediate data
    consistency is to disable write caching (on GNU/Linux, using the -W
    option to hdparm) or to use SCSI disks.

Will the drive finish writing the blocks even if the computer crashes?
If so, this isn't a serious problem, because only a sudden power failure
would stop it.  That simply does not happen on a laptop.  It could
be a real problem on a desktop machine without UPS.  It has been many
years since I had a desktop machine; are IDE disks commonly used on them?
It seems really dumb if there is no way for the CPU to tell the disk,
"Write these blocks now, and tell me when you're done."

Re: Making fsync() optional

[Romain Francoise]

"Richard M. Stallman" <rms@gnu.org> writes:

> Will the drive finish writing the blocks even if the computer crashes?

If it's a software crash, probably.  The write caching feature is very
manufacturer dependent so it's hard to tell, there isn't much
documentation about it.

> It has been many years since I had a desktop machine; are IDE disks
> commonly used on them?

Yes, almost all desktop machines use IDE/SATA disks nowadays.

> It seems really dumb if there is no way for the CPU to tell the disk,
> "Write these blocks now, and tell me when you're done."

There is a way, but when the drive reports that it has written the
blocks it may just have stored them in its write cache.

-- 
Romain Francoise <romain@orebokech.com> | The world is a fine place,
it's a miracle -- http://orebokech.com/ | and worth fighting for.
                                        | --Ernest Hemingway

Re: Making fsync() optional

[Richard M. Stallman]

    > It seems really dumb if there is no way for the CPU to tell the disk,
    > "Write these blocks now, and tell me when you're done."

    There is a way, but when the drive reports that it has written the
    blocks it may just have stored them in its write cache.

How stupid, to make a feature like that not work!