bug#14273: `write-region-inhibit-fsync' should default to t

bug#14273: `write-region-inhibit-fsync' should default to t

[Paul Eggert]

Tags: patch

These days it's better to default write-region-inhibit-fsync to t.
Here's a proposed patch, which I hope is self-explanatory.
This change speeds up "make compile-always" by 15% on my platform.

=== modified file 'doc/emacs/ChangeLog'
--- doc/emacs/ChangeLog	2013-04-04 03:46:25 +0000
+++ doc/emacs/ChangeLog	2013-04-26 06:35:33 +0000
@@ -1,3 +1,20 @@
+2013-04-26  Paul Eggert  <eggert@cs.ucla.edu>
+
+	`write-region-inhibit-fsync' now defaults to t.
+	* files.texi (Customize Save): Document this.
+	fsync can be a major performance hit.  Often it doesn't suffice to
+	make the file-save operation survive a crash anyway, so its
+	utility is questionable.
+	For more on why fsync often fails to work on today's hardware, see:
+	Zheng M, Tucek J, Qin F, Lillibridge M. Understanding the
+	robustness of SSDs under power fault. 11th USENIX Conference on
+	File and Storage Technologies, 2013 (FAST '13), 271-84
+	http://www.usenix.org/system/files/conference/fast13/fast13-final80.pdf
+	For more on why fsync does not suffice even if it works properly, see:
+	Roche X. Necessary step(s) to synchronize filename operations on disk.
+	Austin Group Defect 672, 2013-03-19
+	http://austingroupbugs.net/view.php?id=672
+
 2013-03-17  Paul Eggert  <eggert@cs.ucla.edu>
 
 	doc: convert some TeX accents to UTF-8

=== modified file 'doc/emacs/files.texi'
--- doc/emacs/files.texi	2013-02-25 17:36:03 +0000
+++ doc/emacs/files.texi	2013-04-26 06:35:33 +0000
@@ -705,13 +705,20 @@
 final newlines.
 
 @vindex write-region-inhibit-fsync
-  When Emacs saves a file, it invokes the @code{fsync} system call to
-force the data immediately out to disk.  This is important for safety
-if the system crashes or in case of power outage.  However, it can be
-disruptive on laptops using power saving, as it may force a disk
-spin-up each time you save a file.  If you accept an increased risk of
-data loss, you can set @code{write-region-inhibit-fsync} to a
-non-@code{nil} value to disable the synchronization.
+  Normally, when programs like Emacs write a file, the operating
+system briefly caches the file's data in main memory before committing
+the data to disk.  This can greatly improve performance; for example,
+when running on laptops, it can avoid a disk spin-up each time Emacs
+saves a file.  However, it risks data loss if the operating system
+crashes before committing the cache to disk.
+
+  To lessen this risk, set @code{write-region-inhibit-fsync} to
+@code{nil}: this causes Emacs to invoke the @code{fsync} system call
+after saving a file.  Using @code{fsync} does not eliminate the risk
+of data loss, partly because many systems do not implement
+@code{fsync} properly, and partly because Emacs's file-saving
+procedure typically relies also on directory updates that might not
+survive a crash even if @code{fsync} works properly.
 
 @node Interlocking
 @subsection Protection against Simultaneous Editing

=== modified file 'etc/ChangeLog'
--- etc/ChangeLog	2013-04-24 16:50:14 +0000
+++ etc/ChangeLog	2013-04-26 06:35:33 +0000
@@ -1,3 +1,7 @@
+2013-04-26  Paul Eggert  <eggert@cs.ucla.edu>
+
+	* NEWS: `write-region-inhibit-fsync' now defaults to t.
+
 2013-04-24  Tassilo Horn  <tsdh@gnu.org>
 
 	* themes/tsdh-dark-theme.el (tsdh-dark): Add ido faces and remove

=== modified file 'etc/NEWS'
--- etc/NEWS	2013-04-25 17:39:44 +0000
+++ etc/NEWS	2013-04-26 06:35:33 +0000
@@ -61,6 +61,8 @@
 ** `initial-buffer-choice' can now specify a function to set up the
 initial buffer.
 
+** `write-region-inhibit-fsync' now defaults to t instead of to nil.
+
 ** ACL support has been added.
 +++
 *** Emacs preserves the ACL entries of files when backing up.

=== modified file 'src/ChangeLog'
--- src/ChangeLog	2013-04-24 15:09:49 +0000
+++ src/ChangeLog	2013-04-26 06:35:33 +0000
@@ -1,3 +1,8 @@
+2013-04-26  Paul Eggert  <eggert@cs.ucla.edu>
+
+	* fileio.c (syms_of_fileio):
+	`write-region-inhibit-fsync' now defaults to t.
+
 2013-04-24  Kenichi Handa  <handa@gnu.org>
 
 	* coding.c (decode_coding_iso_2022): When an invalid escape

=== modified file 'src/fileio.c'
--- src/fileio.c	2013-04-07 16:18:41 +0000
+++ src/fileio.c	2013-04-26 06:35:33 +0000
@@ -6072,8 +6072,9 @@
   DEFVAR_BOOL ("write-region-inhibit-fsync", write_region_inhibit_fsync,
 	       doc: /* Non-nil means don't call fsync in `write-region'.
 This variable affects calls to `write-region' as well as save commands.
-A non-nil value may result in data loss!  */);
-  write_region_inhibit_fsync = 0;
+Setting this to nil may avoid data loss if the system loses power or
+the operating system crashes.  */);
+  write_region_inhibit_fsync = 1;
 
   DEFVAR_BOOL ("delete-by-moving-to-trash", delete_by_moving_to_trash,
                doc: /* Specifies whether to use the system's trash can.

bug#14273: `write-region-inhibit-fsync' should default to t

[Stefan Monnier]

> These days it's better to default write-region-inhibit-fsync to t.

Agreed,


        Stefan

bug#14273: `write-region-inhibit-fsync' should default to t

[Romain Francoise]

Paul Eggert <eggert@cs.ucla.edu> writes:

> These days it's better to default write-region-inhibit-fsync to t.

FWIW, I disagree.

Yes, fsync hurts performance, and yes it's not always effective anyway.
But there are still many situations where it does help get the data to
disk in time to prevent data loss, and as far as default values are
concerned Emacs should err on the side of safety.

(You'll note that I was the one who added this option back in 2005.)

> This change speeds up "make compile-always" by 15% on my platform.

If build time is an issue, you can use `eatmydata' (or any of the
existing safety-disabling wrappers) to get the same benefit.

bug#14273: `write-region-inhibit-fsync' should default to t

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 730 bytes --]

On 04/26/2013 06:50 AM, Romain Francoise wrote:
> there are still many situations where it does help get the data to
> disk in time to prevent data loss

I'm sympathetic to this comment.  Still, Emacs uses fsync
far too often, and this often significantly hurts performance
for no particularly good reason.

How about the attached patch instead?  It causes Emacs to use fsync by
default only when operating interactively.  When running in batch
mode, fsync is disabled by default.  This would fix the performance
problem I observed (Emacs batch byte-compiles) without affecting
the warm fuzzy feeling one gets when one types C-x C-s.

I suppose Stefan may prefer the original patch, as it's simpler.
I'm OK either way, I guess.



[-- Attachment #2: chefsync.txt --]
[-- Type: text/plain, Size: 6805 bytes --]

=== modified file 'doc/emacs/ChangeLog'
--- doc/emacs/ChangeLog	2013-04-04 03:46:25 +0000
+++ doc/emacs/ChangeLog	2013-04-28 07:34:00 +0000
@@ -1,3 +1,23 @@
+2013-04-28  Paul Eggert  <eggert@cs.ucla.edu>
+
+	`write-region-inhibit-fsync' defaults to noninteractive (Bug#14273).
+	fsync can be a significant performance hit.  Often it doesn't
+	suffice to make the file-save operation survive a crash.  For
+	batch scripts, which are typically part of larger shell commands
+	that don't fsync other files, its effect on performance can be
+	significant so its utility is particularly questionable.
+	For more on why fsync often fails to work on today's hardware, see:
+	Zheng M, Tucek J, Qin F, Lillibridge M. Understanding the
+	robustness of SSDs under power fault. 11th USENIX Conference on
+	File and Storage Technologies, 2013 (FAST '13), 271-84
+	http://www.usenix.org/system/files/conference/fast13/fast13-final80.pdf
+	For more on why fsync does not suffice even if it works properly, see:
+	Roche X. Necessary step(s) to synchronize filename operations on disk.
+	Austin Group Defect 672, 2013-03-19
+	http://austingroupbugs.net/view.php?id=672
+	* cmdargs.texi (Initial Options):
+	* files.texi (Customize Save): Document this.
+
 2013-03-17  Paul Eggert  <eggert@cs.ucla.edu>
 
 	doc: convert some TeX accents to UTF-8

=== modified file 'doc/emacs/cmdargs.texi'
--- doc/emacs/cmdargs.texi	2013-02-13 00:42:56 +0000
+++ doc/emacs/cmdargs.texi	2013-04-28 07:34:00 +0000
@@ -251,7 +251,8 @@
 but @file{site-start.el} is loaded nonetheless.  It also causes Emacs
 to exit after processing all the command options.  In addition, it
 disables auto-saving except in buffers for which auto-saving is
-explicitly requested.
+explicitly requested, and when saving files it omits the @code{fsync}
+system call unless otherwise requested.
 
 @item --script @var{file}
 @opindex --script

=== modified file 'doc/emacs/files.texi'
--- doc/emacs/files.texi	2013-02-25 17:36:03 +0000
+++ doc/emacs/files.texi	2013-04-28 07:34:00 +0000
@@ -705,13 +705,27 @@
 final newlines.
 
 @vindex write-region-inhibit-fsync
-  When Emacs saves a file, it invokes the @code{fsync} system call to
-force the data immediately out to disk.  This is important for safety
-if the system crashes or in case of power outage.  However, it can be
-disruptive on laptops using power saving, as it may force a disk
-spin-up each time you save a file.  If you accept an increased risk of
-data loss, you can set @code{write-region-inhibit-fsync} to a
-non-@code{nil} value to disable the synchronization.
+  Normally, when a program writes a file, the operating system briefly
+caches the file's data in main memory before committing the data to
+disk.  This can greatly improve performance; for example, when running
+on laptops, it can avoid a disk spin-up each time a file is written.
+However, it risks data loss if the operating system crashes before
+committing the cache to disk.
+
+  To lessen this risk, Emacs can invoke the @code{fsync} system call
+after saving a file.  Using @code{fsync} does not eliminate the risk
+of data loss, partly because many systems do not implement
+@code{fsync} properly, and partly because Emacs's file-saving
+procedure typically relies also on directory updates that might not
+survive a crash even if @code{fsync} works properly.
+
+  The @code{write-region-inhibit-fsync} variable controls whether
+Emacs invokes @code{fsync} after saving a file.  The variable's
+default value is @code{nil} when Emacs is interactive, and @code{t}
+when Emacs runs in batch mode.
+
+  Emacs never uses @code{fsync} when writing auto-save files, as these
+files might lose data anyway.
 
 @node Interlocking
 @subsection Protection against Simultaneous Editing

=== modified file 'etc/ChangeLog'
--- etc/ChangeLog	2013-04-24 16:50:14 +0000
+++ etc/ChangeLog	2013-04-28 07:34:00 +0000
@@ -1,3 +1,8 @@
+2013-04-28  Paul Eggert  <eggert@cs.ucla.edu>
+
+	`write-region-inhibit-fsync' defaults to noninteractive (Bug#14273).
+	* NEWS: Document this.
+
 2013-04-24  Tassilo Horn  <tsdh@gnu.org>
 
 	* themes/tsdh-dark-theme.el (tsdh-dark): Add ido faces and remove

=== modified file 'etc/NEWS'
--- etc/NEWS	2013-04-27 20:55:00 +0000
+++ etc/NEWS	2013-04-28 07:34:00 +0000
@@ -61,6 +61,8 @@
 ** `initial-buffer-choice' can now specify a function to set up the
 initial buffer.
 
+** `write-region-inhibit-fsync' now defaults to t in batch mode.
+
 ** ACL support has been added.
 +++
 *** Emacs preserves the ACL entries of files when backing up.

=== modified file 'src/ChangeLog'
--- src/ChangeLog	2013-04-27 21:12:17 +0000
+++ src/ChangeLog	2013-04-28 07:34:00 +0000
@@ -1,3 +1,8 @@
+2013-04-28  Paul Eggert  <eggert@cs.ucla.edu>
+
+	`write-region-inhibit-fsync' defaults to noninteractive (Bug#14273).
+	* fileio.c (syms_of_fileio): Implement this.
+
 2013-04-27  Juri Linkov  <juri@jurta.org>
 
 	* callint.c (Fcall_interactively): Call `Qread_number' for
@@ -14,6 +19,7 @@
 	size.
 
 2013-04-26  Paul Eggert  <eggert@cs.ucla.edu>
+
 	Port better to AIX (Bug#14258).
 	* lisp.h (ENUM_BF) [__IBMC__]: Make it 'unsigned int' here, too,
 	to pacify AIX xlc.

=== modified file 'src/fileio.c'
--- src/fileio.c	2013-04-07 16:18:41 +0000
+++ src/fileio.c	2013-04-28 07:34:00 +0000
@@ -4979,15 +4979,14 @@
 
   immediate_quit = 0;
 
-  /* fsync appears to change the modtime on BSD4.2.
-     Disk full in NFS may be reported here.  */
-  /* mib says that closing the file will try to write as fast as NFS can do
-     it, and that means the fsync here is not crucial for autosave files.  */
+  /* fsync is not crucial for auto-save files, since they might lose
+     some work anyway.  */
   if (!auto_saving && !write_region_inhibit_fsync)
     {
-      /* Transfer data and metadata to disk, retrying if interrupted.  Also,
-	 ignore EINVAL which happens when fsync is not supported on this
-	 file.  */
+      /* Transfer data and metadata to disk, retrying if interrupted.
+	 fsync can report a write failure here, e.g., due to disk full
+	 under NFS.  But ignore EINVAL, which means fsync is not
+	 supported on this file.  */
       while (fsync (desc) != 0)
 	if (errno != EINTR)
 	  {
@@ -6072,8 +6071,9 @@
   DEFVAR_BOOL ("write-region-inhibit-fsync", write_region_inhibit_fsync,
 	       doc: /* Non-nil means don't call fsync in `write-region'.
 This variable affects calls to `write-region' as well as save commands.
-A non-nil value may result in data loss!  */);
-  write_region_inhibit_fsync = 0;
+Setting this to nil may avoid data loss if the system loses power or
+the operating system crashes.  */);
+  write_region_inhibit_fsync = noninteractive;
 
   DEFVAR_BOOL ("delete-by-moving-to-trash", delete_by_moving_to_trash,
                doc: /* Specifies whether to use the system's trash can.

bug#14273: `write-region-inhibit-fsync' should default to t

[Romain Francoise]

Paul Eggert <eggert@cs.ucla.edu> writes:

> How about the attached patch instead?  It causes Emacs to use fsync by
> default only when operating interactively.  When running in batch
> mode, fsync is disabled by default.  This would fix the performance
> problem I observed (Emacs batch byte-compiles) without affecting
> the warm fuzzy feeling one gets when one types C-x C-s.

Looks good to me. Thanks.

bug#14273: `write-region-inhibit-fsync' should default to t

[Stefan Monnier]

> How about the attached patch instead?  It causes Emacs to use fsync by
> default only when operating interactively.

That looks OK for now.

> +	fsync can be a significant performance hit.  Often it doesn't
> +	suffice to make the file-save operation survive a crash.  For
> +	batch scripts, which are typically part of larger shell commands
> +	that don't fsync other files, its effect on performance can be
> +	significant so its utility is particularly questionable.
> +	For more on why fsync often fails to work on today's hardware, see:
> +	Zheng M, Tucek J, Qin F, Lillibridge M. Understanding the
> +	robustness of SSDs under power fault. 11th USENIX Conference on
> +	File and Storage Technologies, 2013 (FAST '13), 271-84
> +	http://www.usenix.org/system/files/conference/fast13/fast13-final80.pdf
> +	For more on why fsync does not suffice even if it works properly, see:
> +	Roche X. Necessary step(s) to synchronize filename operations on disk.
> +	Austin Group Defect 672, 2013-03-19
> +	http://austingroupbugs.net/view.php?id=672
> +	* cmdargs.texi (Initial Options):
> +	* files.texi (Customize Save): Document this.

But this longish text does not belong in the commitlog/changelog but in
the code.


        Stefan

bug#14273: `write-region-inhibit-fsync' should default to t

[Paul Eggert]

On 05/04/2013 12:53 PM, Stefan Monnier wrote:

> But this longish text does not belong in the commitlog/changelog but in
> the code.

OK, thanks, I moved it to the code and installed it.

I noticed another fsync that wasn't needed, when creating lock
files on hosts that don't support symlinks, and deleted that
fsync too.

I'm marking this bug as done.