Corrupt .drv files

[ludo@gnu.org (Ludovic Courtès)]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

(Moving to guix-devel.)

ludo@gnu.org (Ludovic Courtès) skribis:

> Andreas Enge <andreas@enge.fr> skribis:
>
>> guix archive: error: build failed: error parsing derivation `/gnu/store/k49lwfwgs8wcamys5qzn8c5n2zk0prc1-tcl8.6.4-src.tar.xz.drv': expected string `Derive(['
>
> It looks like the store on this machine is corrupt.

Indeed, the daemon doesn’t attempt to atomically write files coming from
an add-to-store RPC, which includes .drv files.

So I think that if you pull the plug before the .drv has been flushed to
disk but after the .drv has been marked as valid in the SQLite database
(which is likely to happen in a timely fashion because SQLite does the
‘fdatasync’ dance appropriately), then you end up with a truncated .drv
file.

(This is acknowledged by the comment in
‘LocalStore::registerValidPaths’, which can call ‘sync’, a
sledgehammer.)

The attached patch should fix it.  I think the performance overhead of
the extra ‘fdatasync’ should be OK but I haven’t made any measurements.

Thanks,
Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1848 bytes --]

diff --git a/nix/libstore/local-store.cc b/nix/libstore/local-store.cc
index 347e8a7..80566ea 100644
--- a/nix/libstore/local-store.cc
+++ b/nix/libstore/local-store.cc
@@ -1391,7 +1391,7 @@ Path LocalStore::addToStoreFromDump(const string & dump, const string & name,
                 StringSource source(dump);
                 restorePath(dstPath, source);
             } else
-                writeFile(dstPath, dump);
+                writeFileAtomically(dstPath, dump);
 
             canonicalisePathMetaData(dstPath, -1);
 
diff --git a/nix/libutil/util.cc b/nix/libutil/util.cc
index c077544..e33824b 100644
--- a/nix/libutil/util.cc
+++ b/nix/libutil/util.cc
@@ -272,6 +272,22 @@ void writeFile(const Path & path, const string & s)
     writeFull(fd, s);
 }
 
+void writeFileAtomically(const Path & path, const string & s)
+{
+    auto tmpPath = path + ".tmp";
+    AutoCloseFD fd = open(tmpPath.c_str(), O_WRONLY | O_TRUNC | O_CREAT, 0666);
+    if (fd == -1)
+        throw SysError(format("opening file '%1%'") % tmpPath);
+    writeFull(fd, s);
+#if _POSIX_SYNCHRONIZED_IO > 0
+    if (fdatasync(fd) != 0)
+        throw SysError(format("flushing file '%1%'") % tmpPath);
+#endif
+    fd.close();
+    if (rename(tmpPath.c_str(), path.c_str()) != 0)
+        throw SysError(format("renaming '%1%' to '%2'") % tmpPath % path);
+}
+
 
 string readLine(int fd)
 {
diff --git a/nix/libutil/util.hh b/nix/libutil/util.hh
index e84d64d..829f90b 100644
--- a/nix/libutil/util.hh
+++ b/nix/libutil/util.hh
@@ -86,6 +86,9 @@ string readFile(const Path & path, bool drain = false);
 /* Write a string to a file. */
 void writeFile(const Path & path, const string & s);
 
+/* Same, but do it atomically.  */
+void writeFileAtomically(const Path & path, const string & s);
+
 /* Read a line from a file descriptor. */
 string readLine(int fd);