From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Subject: bug#28659: v0.13: guix pull fails; libgit2-0.26.0 and 0.25.1 content hashes fail Date: Thu, 14 Dec 2017 17:53:37 +0100 Message-ID: <874lot9rou.fsf@gnu.org> References: <877ewf18d4.fsf@gnu.org> <87o9ppoabw.fsf@gnu.org> <20171002182208.GB10773@jasmine.lan> <878tgt721q.fsf@gnu.org> <20171020211700.GA32355@jasmine.lan> <87d1421qek.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45747) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ePWlr-0001C2-HI for bug-guix@gnu.org; Thu, 14 Dec 2017 11:54:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ePWlm-0000XQ-IE for bug-guix@gnu.org; Thu, 14 Dec 2017 11:54:07 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:53515) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ePWlm-0000XA-FE for bug-guix@gnu.org; Thu, 14 Dec 2017 11:54:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ePWlm-0000Tb-3G for bug-guix@gnu.org; Thu, 14 Dec 2017 11:54:02 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87d1421qek.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Tue, 28 Nov 2017 14:30:59 +0100") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Leo Famulari Cc: 28659@debbugs.gnu.org ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > Thinking more about it, why not simply always enable substitutes for > fixed-output derivations, like this: > > diff --git a/nix/libstore/build.cc b/nix/libstore/build.cc > index d68e8b2bc..03a8f5080 100644 > --- a/nix/libstore/build.cc > +++ b/nix/libstore/build.cc > @@ -1034,8 +1034,10 @@ void DerivationGoal::haveDerivation() >=20=20 > /* We are first going to try to create the invalid output paths > through substitutes. If that doesn't work, we'll build > - them. */ > - if (settings.useSubstitutes && substitutesAllowed(drv)) > + them. Always enable substitutes for fixed-output derivations to > + protect against disappearing files and in-place modifications on > + upstream sites. */ > + if ((fixedOutput || settings.useSubstitutes) && substitutesAllowed(d= rv)) > foreach (PathSet::iterator, i, invalidOutputs) > addWaitee(worker.makeSubstitutionGoal(*i, buildMode =3D=3D b= mRepair)); [...] > The downside is that it still requires one to authorize the server=E2=80= =99s > key, although it=E2=80=99s in theory unnecessary since it=E2=80=99s conte= nt addressed. > I=E2=80=99m not sure how to solve that because =E2=80=98guix substitute= =E2=80=99 doesn=E2=80=99t know > that it=E2=80=99s substituting a fixed-output derivation. I suppose we= =E2=80=99d need > to modify the =E2=80=9Cprotocol=E2=80=9D between guix-daemon and =E2=80= =98guix substitute=E2=80=99. I looked at how to address this by having =E2=80=98guix substitute=E2=80=99 automatically determine whether it=E2=80=99s being asked for a content-addr= essed item or not. The guts of it is this procedure: (define* (content-addressed-item? item hash #:key (hash-algo 'sha256)) "Return true if ITEM, a store file name, is definitely a content-addres= sed item (result of a fixed-output derivation) with the given HASH of type HASH-ALGO, false otherwise. Note: This procedure is useful when the deriver of ITEM is unknown. In o= ther cases, the recommended approach is to check 'fixed-output-derivation?' on= the deriver." ;; XXX: This returns #f for "text" items produced by 'add-text-to-store= '. ;; There's not much we can do because the file name for these is a func= tion ;; of their content. (let ((name (store-path-package-name item))) (or (string=3D? item (fixed-output-path name hash #:recursive? #f #:hash-algo hash-algo)) (string=3D? item (fixed-output-path name hash #:recursive? #t #:hash-algo hash-algo))))) It works as expected for the result of =E2=80=9Crecursive fixed-output derivations=E2=80=9D=E2=80=94i.e., fixed-output derivations that produce a = directory, such as VCS checkouts. However it doesn=E2=80=99t work for fixed-output derivations that produce a= flat file, such as origins with the =E2=80=98url-fetch=E2=80=99 method. The rea= son is because in the case of non-recursive derivations, the store file name is computed as a function of the file hash, not as a function of the nar hash, whereas narinfos only contains the nar hash (the thing that =E2=80=98= guix hash -r=E2=80=99 computes.) So I think we have to communicate more info from the daemon to =E2=80=98guix substitute=E2=80=99. Ludo=E2=80=99.