From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Graft hooks Date: Wed, 22 Aug 2018 16:21:48 +0200 Message-ID: <874lfmh477.fsf@gnu.org> References: <87k1ovbc0t.fsf@ngyro.com> <87y3d1h05e.fsf@gnu.org> <87k1ojafqa.fsf@ngyro.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:50272) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsU1A-000093-IY for guix-devel@gnu.org; Wed, 22 Aug 2018 10:21:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsU18-0003Uu-Ul for guix-devel@gnu.org; Wed, 22 Aug 2018 10:21:52 -0400 In-Reply-To: <87k1ojafqa.fsf@ngyro.com> (Timothy Sample's message of "Tue, 21 Aug 2018 11:42:21 -0400") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Timothy Sample Cc: guix-devel@gnu.org Hello Timothy, Timothy Sample skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> Hello Timothy, >> >> Timothy Sample skribis: >> >>> The basic idea would be to add a field (or use a property) to the >>> package record. Let=E2=80=99s call it =E2=80=9Cgraft-hook=E2=80=9D. I= t would be Scheme code >>> that gets run after grafting takes place, giving us a chance to patch >>> special things like checksums. The hook would be passed the list of >>> files that were been modified during grafting. Then, in the Racket >>> package for example, I could write a graft-hook that updates the SHA-1 >>> hash of each of the modified source files. >>> >>> Since grafting is done at the derivation level, the hook code would have >>> to be propagated down from the package level. I haven=E2=80=99t looked= at all >>> the details yet, because maybe this is a bad idea and I shouldn=E2=80= =99t waste >>> my time! :) My first impression is that it is not too tricky. >>> >>> Are these problems too specialized to deserve a general mechanism like >>> this? Let me know what you think! >> >> I agree that this would be the right thing to do! (I=E2=80=99d really l= ike to >> do it for GDB as discussed in .) >> >> Package properties would be the right way to make it extensible, but >> there are complications (notably we=E2=80=99d need to use gexps, but bui= ld >> systems don=E2=80=99t use gexps yet.) > > But soon, right? ;) Well, it=E2=80=99s complicated. :-) Also, I realized that some things, like the .gnu_debuglink and build-id hooks, don=E2=80=99t really fit in any package; they=E2=80=99re transverse. > Here=E2=80=99s a draft patch (it=E2=80=99s mercifully small). I have a f= ew questions > about it, but if it looks like the right approach, I will clean it up > and submit it. > > Basically, it checks if we are grafting Racket, and then adds some code > to the build expression to run the hook. OK. In theory, should it be just for Racket, or should it also be for Racket libraries (we don=E2=80=99t have any currently AFAIK)? > Also, is there a preference for patching the files using Guile or using > an external tool? This patch uses Racket=E2=80=99s =E2=80=9Craco setup= =E2=80=9D command to > recompile the files and fix the checksums. Unfortunately, it also > updates timestamps. I=E2=80=99m pretty sure our Racket package is not > reproducible at the moment, so I didn=E2=80=99t worry about it too much. = The > timestamps could be patched out, though. The reason I shied away from > writing my own code is that Racket also hashes all the dependencies for > a bytecode file. This means that the custom code would have to traverse > the Racket dependency graph to get the checksums right. It is not too > hard to do so, but it would be a couple hundred lines of code (compared > to the five or so it took to invoke =E2=80=9Craco setup=E2=80=9D). Regarding whether or not to write our own code: let=E2=80=99s do whichever = is more convenient. In this case, using =E2=80=98raco setup=E2=80=99 looks li= ke the right thing to do, given that raco is available in the build environment anyway (see below); for .gnu_debuglink, I found it nicer (and more fun :-)) to write a Guile module. Regarding timestamps: I guess there=E2=80=99s no problem since timestamps a= re reset in the store. Some comments: > diff --git a/guix/grafts.scm b/guix/grafts.scm > index d6b0e93e8..88a99312d 100644 > --- a/guix/grafts.scm > +++ b/guix/grafts.scm > @@ -75,6 +75,36 @@ > (($ (? string? item)) > item))) >=20=20 > +(define (fix-racket-checksums store drv system) > + (define racket-drv > + (let ((package-derivation (module-ref (resolve-interface '(guix pack= ages)) > + 'package-derivation)) > + (racket (module-ref (resolve-interface '(gnu packages scheme)) > + 'racket))) > + (package-derivation store racket system #:graft? #f))) > + > + (define hook-exp > + `(lambda (input output mapping) > + (let ((raco (string-append output "/bin/raco"))) > + ;; Setting PLT_COMPILED_FILE_CHECK to "exists" tells Racket to > + ;; ignore timestamps when checking if a compiled file is valid. > + ;; Without it, Racket attempts a complete rebuild of > + ;; everything. > + (setenv "PLT_COMPILED_FILE_CHECK" "exists") > + ;; All of the --no-* flags below keep Racket from making > + ;; unecessary and unhelpful changes (like rewriting scripts and > + ;; reverting their shebangs in the process). > + (invoke raco "setup" "--no-launcher" "--no-install" > + "--no-post-install" "--no-info-domain" "--no-docs")))) Since this is used when grafting Racket, I would suggest moving this graft to the =E2=80=9Cbuild side=E2=80=9D entirely, similar to what I did in . Probably you=E2=80=99d just add a single procedure to (guix build graft) and add it = to %graft-hooks. That procedure could be the same as what you have above, except that it=E2=80=99d run OUT/bin/raco, if it exists, and do nothing if OUT/bin/raco= does not exist. WDYT? Thanks, Ludo=E2=80=99.