From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Invalid nar signature Date: Thu, 15 Jan 2015 10:51:16 +0100 Message-ID: <87zj9k74ob.fsf@gnu.org> References: <87zjapejo3.fsf@izanagi.i-did-not-set--mail-host-address--so-tickle-me> <87ppbkompf.fsf@gnu.org> <87ppbkfw4w.fsf@gnu.org> <87ppbkedf2.fsf@izanagi.i-did-not-set--mail-host-address--so-tickle-me> <871tnz8qwx.fsf@gnu.org> <87mw5kkbue.fsf@izanagi.i-did-not-set--mail-host-address--so-tickle-me> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:49539) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YBh5J-0001SV-Fv for guix-devel@gnu.org; Thu, 15 Jan 2015 04:51:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YBh5D-0001ci-CW for guix-devel@gnu.org; Thu, 15 Jan 2015 04:51:25 -0500 Received: from hera.aquilenet.fr ([2a01:474::1]:40771) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YBh5C-0001c3-TW for guix-devel@gnu.org; Thu, 15 Jan 2015 04:51:19 -0500 In-Reply-To: <87mw5kkbue.fsf@izanagi.i-did-not-set--mail-host-address--so-tickle-me> (David Thompson's message of "Wed, 14 Jan 2015 21:38:01 -0500") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: David Thompson Cc: guix-devel --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable David Thompson skribis: > Ludovic Court=C3=A8s writes: [...] >> 2. strace the substituter and/or =E2=80=98guix publish=E2=80=99 to see= exactly what >> happens on the wire. Is the end-of-file marker string sent? Is it >> received? etc. > > Here's a snippet of the strace output: > > http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore= -0.1-6a9fbe0 94.1% of 127.5 KiB) =3D 104 > read(10, "vector tox-max-status-message-le"..., 7728) =3D 117 > read(10, " "..., 7611) =3D 1448 > read(10, "t-last-online (unwrap-tox tox) f"..., 6163) =3D 1448 > read(10, "tox tox) nospam))\n\n(define/unwra"..., 4715) =3D 1448 > read(10, " friend-number group-number)))\n\n"..., 3267) =3D 1448 > read(10, ")))\n (if (negative? result)\n "..., 1819) =3D 1819 > http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore= -0.1-6a9fbe0 100.0% of 127.5 KiB) =3D 104 > http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore= -0.1-6a9fbe0 100.0% of 127.5 KiB) =3D 104 > brk(0x41875000) =3D 0x41875000 > mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,= 0) =3D 0x7f3d2aaaf000 > mremap(0x7f3d2aaaf000, 135168, 266240, MREMAP_MAYMOVE) =3D 0x7f3d2aa6e0= 00 > mremap(0x7f3d2aa6e000, 266240, 528384, MREMAP_MAYMOVE) =3D 0x7f3d2a9ed0= 00 > mremap(0x7f3d2a9ed000, 528384, 430080, MREMAP_MAYMOVE) =3D 0x7f3d2a9ed0= 00 > munmap(0x7f3d2a9ed000, 430080) =3D 0 > http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore= -0.1-6a9fbe0 100.0% of 127.5 KiB) =3D 104 > open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/shar= e/locale/en_US.UTF-8/LC_MESSAGES/guix.mo", O_RDONLY) =3D -1 ENOENT (No such= file or directory) > open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/shar= e/locale/en_US.utf8/LC_MESSAGES/guix.mo", O_RDONLY) =3D -1 ENOENT (No such = file or directory) > open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/shar= e/locale/en_US/LC_MESSAGES/guix.mo", O_RDONLY) =3D -1 ENOENT (No such file = or directory) > open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/shar= e/locale/en.UTF-8/LC_MESSAGES/guix.mo", O_RDONLY) =3D -1 ENOENT (No such fi= le or directory) > open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/shar= e/locale/en.utf8/LC_MESSAGES/guix.mo", O_RDONLY) =3D -1 ENOENT (No such fil= e or directory) > open("/gnu/store/72qm7kc9phvsiw6j7xgn1ii0f6s9mx8i-guix-0.8.3b09332/shar= e/locale/en/LC_MESSAGES/guix.mo", O_RDONLY) =3D -1 ENOENT (No such file or = directory) > write(2, "guix substitute-binary: error: i"..., 62guix substitute-binar= y: error: invalid nar end-of-file marker > ) =3D 62 > exit_group(1) =3D ? > +++ exited with 1 +++ Normally, when the error happens, the substituter has already created at least one file in the target directory, /gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0. Could you apply the patch below, and then run: rm -rf foo ./pre-inst-env guix substitute-binary --substitute \ /gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0 \ $PWD/foo > stdout ls -lRa foo > ls-R and then send =E2=80=98stdout=E2=80=99 and =E2=80=98ls-R=E2=80=99? Could you also check if the files in =E2=80=98foo=E2=80=99 look corrupt or = anything? --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/guix/serialization.scm b/guix/serialization.scm index 64eacf9..d3bdea3 100644 --- a/guix/serialization.scm +++ b/guix/serialization.scm @@ -292,19 +292,21 @@ Restore it as FILE." (define (read-eof-marker) (match (read-string port) (")" #t) - (x (raise + (x + (pk 'bad-eof x) + (raise (condition (&message (message "invalid nar end-of-file marker")) (&nar-read-error (port port) (file file) (token x))))))) (match (list (read-string port) (read-string port) (read-string port)) (("(" "type" "regular") - (call-with-output-file file (cut read-contents port <>)) + (call-with-output-file (pk 'reg file) (cut read-contents port <>)) (read-eof-marker)) (("(" "type" "symlink") (match (list (read-string port) (read-string port)) (("target" target) - (symlink target file) + (symlink target (pk 'symlink file)) (read-eof-marker)) (x (raise (condition @@ -312,7 +314,7 @@ Restore it as FILE." (&nar-read-error (port port) (file file) (token x))))))) (("(" "type" "directory") (let ((dir file)) - (mkdir dir) + (mkdir (pk 'dir dir)) (let loop ((prefix (read-string port))) (match prefix ("entry" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > Despite that, I tried to compress the nar with bzip2 just for fun, but I > ran into another problem: > > warning: call to primitive-fork while multiple threads are running; > further behavior unspecified. See "Processes" in the > manual, for more information. > > I'm running a REPL server in addition to the web server, but I imagine > the web server also spawns additional threads to handle requests, so > either way 'filtered-output-port' won't work here. (web server http) does not use threads so it should be OK. But we=E2=80=99= ll see that afterwards. :-) Thanks, Ludo=E2=80=99. --=-=-=--