unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Corrupt .drv files
       [not found] ` <87k2gsjooh.fsf@gnu.org>
@ 2016-07-11 21:49   ` Ludovic Courtès
  2016-07-11 22:04     ` Andreas Enge
  0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2016-07-11 21:49 UTC (permalink / raw)
  To: Andreas Enge; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

(Moving to guix-devel.)

ludo@gnu.org (Ludovic Courtès) skribis:

> Andreas Enge <andreas@enge.fr> skribis:
>
>> guix archive: error: build failed: error parsing derivation `/gnu/store/k49lwfwgs8wcamys5qzn8c5n2zk0prc1-tcl8.6.4-src.tar.xz.drv': expected string `Derive(['
>
> It looks like the store on this machine is corrupt.

Indeed, the daemon doesn’t attempt to atomically write files coming from
an add-to-store RPC, which includes .drv files.

So I think that if you pull the plug before the .drv has been flushed to
disk but after the .drv has been marked as valid in the SQLite database
(which is likely to happen in a timely fashion because SQLite does the
‘fdatasync’ dance appropriately), then you end up with a truncated .drv
file.

(This is acknowledged by the comment in
‘LocalStore::registerValidPaths’, which can call ‘sync’, a
sledgehammer.)

The attached patch should fix it.  I think the performance overhead of
the extra ‘fdatasync’ should be OK but I haven’t made any measurements.

Thanks,
Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1848 bytes --]

diff --git a/nix/libstore/local-store.cc b/nix/libstore/local-store.cc
index 347e8a7..80566ea 100644
--- a/nix/libstore/local-store.cc
+++ b/nix/libstore/local-store.cc
@@ -1391,7 +1391,7 @@ Path LocalStore::addToStoreFromDump(const string & dump, const string & name,
                 StringSource source(dump);
                 restorePath(dstPath, source);
             } else
-                writeFile(dstPath, dump);
+                writeFileAtomically(dstPath, dump);
 
             canonicalisePathMetaData(dstPath, -1);
 
diff --git a/nix/libutil/util.cc b/nix/libutil/util.cc
index c077544..e33824b 100644
--- a/nix/libutil/util.cc
+++ b/nix/libutil/util.cc
@@ -272,6 +272,22 @@ void writeFile(const Path & path, const string & s)
     writeFull(fd, s);
 }
 
+void writeFileAtomically(const Path & path, const string & s)
+{
+    auto tmpPath = path + ".tmp";
+    AutoCloseFD fd = open(tmpPath.c_str(), O_WRONLY | O_TRUNC | O_CREAT, 0666);
+    if (fd == -1)
+        throw SysError(format("opening file '%1%'") % tmpPath);
+    writeFull(fd, s);
+#if _POSIX_SYNCHRONIZED_IO > 0
+    if (fdatasync(fd) != 0)
+        throw SysError(format("flushing file '%1%'") % tmpPath);
+#endif
+    fd.close();
+    if (rename(tmpPath.c_str(), path.c_str()) != 0)
+        throw SysError(format("renaming '%1%' to '%2'") % tmpPath % path);
+}
+
 
 string readLine(int fd)
 {
diff --git a/nix/libutil/util.hh b/nix/libutil/util.hh
index e84d64d..829f90b 100644
--- a/nix/libutil/util.hh
+++ b/nix/libutil/util.hh
@@ -86,6 +86,9 @@ string readFile(const Path & path, bool drain = false);
 /* Write a string to a file. */
 void writeFile(const Path & path, const string & s);
 
+/* Same, but do it atomically.  */
+void writeFileAtomically(const Path & path, const string & s);
+
 /* Read a line from a file descriptor. */
 string readLine(int fd);
 

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: Corrupt .drv files
  2016-07-11 21:49   ` Corrupt .drv files Ludovic Courtès
@ 2016-07-11 22:04     ` Andreas Enge
  2016-07-12  8:59       ` Ludovic Courtès
  0 siblings, 1 reply; 6+ messages in thread
From: Andreas Enge @ 2016-07-11 22:04 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

On Mon, Jul 11, 2016 at 11:49:08PM +0200, Ludovic Courtès wrote:
> > Andreas Enge <andreas@enge.fr> skribis:
> >
> >> guix archive: error: build failed: error parsing derivation `/gnu/store/k49lwfwgs8wcamys5qzn8c5n2zk0prc1-tcl8.6.4-src.tar.xz.drv': expected string `Derive(['
> >
> > It looks like the store on this machine is corrupt.
> 
> Indeed, the daemon doesn’t attempt to atomically write files coming from
> an add-to-store RPC, which includes .drv files.
> 
> So I think that if you pull the plug before the .drv has been flushed to
> disk but after the .drv has been marked as valid in the SQLite database
> (which is likely to happen in a timely fashion because SQLite does the
> ‘fdatasync’ dance appropriately), then you end up with a truncated .drv
> file.

I do not think that this was the problem. I opened the .drv files with vim,
and they did not contain ASCII characters. Also, the file command marked
them as binary data instead of text files. "guix gc --verify" passed, however.

There were quite a few corrupted .drv files, and I ended up doing a complete
"guix gc" instead of "guix gc -d ..." on every file, and this solved the
problem.

Thanks for your help with this,

Andreas

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Corrupt .drv files
  2016-07-11 22:04     ` Andreas Enge
@ 2016-07-12  8:59       ` Ludovic Courtès
  2016-07-12  9:03         ` Andreas Enge
  0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2016-07-12  8:59 UTC (permalink / raw)
  To: Andreas Enge; +Cc: guix-devel

Andreas Enge <andreas@enge.fr> skribis:

> On Mon, Jul 11, 2016 at 11:49:08PM +0200, Ludovic Courtès wrote:
>> > Andreas Enge <andreas@enge.fr> skribis:
>> >
>> >> guix archive: error: build failed: error parsing derivation `/gnu/store/k49lwfwgs8wcamys5qzn8c5n2zk0prc1-tcl8.6.4-src.tar.xz.drv': expected string `Derive(['
>> >
>> > It looks like the store on this machine is corrupt.
>> 
>> Indeed, the daemon doesn’t attempt to atomically write files coming from
>> an add-to-store RPC, which includes .drv files.
>> 
>> So I think that if you pull the plug before the .drv has been flushed to
>> disk but after the .drv has been marked as valid in the SQLite database
>> (which is likely to happen in a timely fashion because SQLite does the
>> ‘fdatasync’ dance appropriately), then you end up with a truncated .drv
>> file.
>
> I do not think that this was the problem. I opened the .drv files with vim,
> and they did not contain ASCII characters. Also, the file command marked
> them as binary data instead of text files.

So did they contain random bytes?

That could be a hard disk problem or something equally fishy.

Nevertheless I think the problem I described above really exists and is
worth mitigating.

Thanks,
Ludo’.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Corrupt .drv files
  2016-07-12  8:59       ` Ludovic Courtès
@ 2016-07-12  9:03         ` Andreas Enge
  2016-07-12 17:30           ` Kei Kebreau
  0 siblings, 1 reply; 6+ messages in thread
From: Andreas Enge @ 2016-07-12  9:03 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

On Tue, Jul 12, 2016 at 10:59:04AM +0200, Ludovic Courtès wrote:
> So did they contain random bytes?

I do not know, since I did not look at the files with a hex editor
(we should package hexedit...).

Andreas

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Corrupt .drv files
  2016-07-12  9:03         ` Andreas Enge
@ 2016-07-12 17:30           ` Kei Kebreau
  2016-07-14 15:17             ` Andreas Enge
  0 siblings, 1 reply; 6+ messages in thread
From: Kei Kebreau @ 2016-07-12 17:30 UTC (permalink / raw)
  To: Andreas Enge; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 505 bytes --]

Andreas Enge <andreas@enge.fr> writes:

> On Tue, Jul 12, 2016 at 10:59:04AM +0200, Ludovic Courtès wrote:
>> So did they contain random bytes?
>
> I do not know, since I did not look at the files with a hex editor
> (we should package hexedit...).
>
> Andreas
>
>

Just popping in...
I was considering packaging hexedit! Now I know there is a demand for it
In the meantime, you can try using "M-x hexl-mode" in Emacs if you
haven't already.

-- 
Kei (GPG Key: 4096R/E6A5EE3C19467A0D)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Corrupt .drv files
  2016-07-12 17:30           ` Kei Kebreau
@ 2016-07-14 15:17             ` Andreas Enge
  0 siblings, 0 replies; 6+ messages in thread
From: Andreas Enge @ 2016-07-14 15:17 UTC (permalink / raw)
  To: Kei Kebreau; +Cc: guix-devel

On Tue, Jul 12, 2016 at 01:30:43PM -0400, Kei Kebreau wrote:
> I was considering packaging hexedit! Now I know there is a demand for it

Definitely, it is a nice little tool.

> In the meantime, you can try using "M-x hexl-mode" in Emacs if you
> haven't already.

Thanks for the tip! No, I had not...

Andreas

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2016-07-14 15:18 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <20160711081921.GA1651@solar>
     [not found] ` <87k2gsjooh.fsf@gnu.org>
2016-07-11 21:49   ` Corrupt .drv files Ludovic Courtès
2016-07-11 22:04     ` Andreas Enge
2016-07-12  8:59       ` Ludovic Courtès
2016-07-12  9:03         ` Andreas Enge
2016-07-12 17:30           ` Kei Kebreau
2016-07-14 15:17             ` Andreas Enge

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).