From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?G=C3=A1bor_Boskovits?= Subject: Re: Graft hooks Date: Mon, 13 Aug 2018 09:19:47 +0200 Message-ID: References: <87k1ovbc0t.fsf@ngyro.com> <87ftzjf6un.fsf@dustycloud.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000bbf9b105734beb0f" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:37756) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fp78y-0008Lo-IK for guix-devel@gnu.org; Mon, 13 Aug 2018 03:20:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fp78x-0002Rk-4q for guix-devel@gnu.org; Mon, 13 Aug 2018 03:20:00 -0400 Received: from mail-io0-x22d.google.com ([2607:f8b0:4001:c06::22d]:38103) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fp78w-0002RV-TO for guix-devel@gnu.org; Mon, 13 Aug 2018 03:19:59 -0400 Received: by mail-io0-x22d.google.com with SMTP id v26-v6so13794613iog.5 for ; Mon, 13 Aug 2018 00:19:58 -0700 (PDT) In-Reply-To: <87ftzjf6un.fsf@dustycloud.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Christopher Lemmer Webber Cc: Guix-devel --000000000000bbf9b105734beb0f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Christopher Lemmer Webber ezt =C3=ADrta (id=C5=91p= ont: 2018. aug. 13., H, 2:28): > Timothy Sample writes: > > > Hi Guix, > > > > I just submitted a patch for , but now I=E2= =80=99m > > wondering if there isn=E2=80=99t a more general way to solve the proble= m. > > > > The bug has to do with grafting and checksums. I know three bugs that > > follow this theme: the one above (Racket), > > (GDB), and (Go). The basic problem is tha= t > > these packages store checksums of files during build time. If they get > > updated due to grafting, the files change, but the checksums do not. > > The checksums become invalid, which causes other problems like trying t= o > > update files in the store or asserting that debugging information is > > invalid. > > > > The patch I submitted makes Racket assume that files in the store are > > good. It patches Racket to skip checksum validation if it is checking = a > > file in the store. A similar approach could be taken for GDB and Go. > > > > It occurs to me that if we had some way to run package-specific code > > during grafting we could solve problems like this easily and without > > patching software that is not broken. > > > > The basic idea would be to add a field (or use a property) to the > > package record. Let=E2=80=99s call it =E2=80=9Cgraft-hook=E2=80=9D. I= t would be Scheme code > > that gets run after grafting takes place, giving us a chance to patch > > special things like checksums. The hook would be passed the list of > > files that were been modified during grafting. Then, in the Racket > > package for example, I could write a graft-hook that updates the SHA-1 > > hash of each of the modified source files. > > +1 I think this would be a good design choice. We also gain back the security that the original check provides. > This seems like a really good approach to me and seems also much nicer / > safer in the long run than the solution in #30680 since it wouldn't just > patch out the package in question's checks, it would correct them. That > seems very good indeed to me. > > > Since grafting is done at the derivation level, the hook code would hav= e > > to be propagated down from the package level. I haven=E2=80=99t looked= at all > > the details yet, because maybe this is a bad idea and I shouldn=E2=80= =99t waste > > my time! :) My first impression is that it is not too tricky. > > > > Are these problems too specialized to deserve a general mechanism like > > this? Let me know what you think! > > > > > > -- Tim > > As said, it seems good to me. But I would be interested in what Mark > would think, since he is mostly responsible for the grafts design. > > --000000000000bbf9b105734beb0f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


Christ= opher Lemmer Webber <cwebber@d= ustycloud.org> ezt =C3=ADrta (id=C5=91pont: 2018. aug. 13., H, 2:28)= :
Timothy Sample writes:

> Hi Guix,
>
> I just submitted a patch for <https://bugs.gnu.org/30680>, = but now I=E2=80=99m
> wondering if there isn=E2=80=99t a more general way to solve the probl= em.
>
> The bug has to do with grafting and checksums.=C2=A0 I know three bugs= that
> follow this theme: the one above (Racket), <https://bugs.gnu.org/1= 9973>
> (GDB), and <https://bugs.gnu.org/25752> (Go).=C2=A0 The bas= ic problem is that
> these packages store checksums of files during build time.=C2=A0 If th= ey get
> updated due to grafting, the files change, but the checksums do not. > The checksums become invalid, which causes other problems like trying = to
> update files in the store or asserting that debugging information is > invalid.
>
> The patch I submitted makes Racket assume that files in the store are<= br> > good.=C2=A0 It patches Racket to skip checksum validation if it is che= cking a
> file in the store.=C2=A0 A similar approach could be taken for GDB and= Go.
>
> It occurs to me that if we had some way to run package-specific code > during grafting we could solve problems like this easily and without > patching software that is not broken.
>
> The basic idea would be to add a field (or use a property) to the
> package record.=C2=A0 Let=E2=80=99s call it =E2=80=9Cgraft-hook=E2=80= =9D.=C2=A0 It would be Scheme code
> that gets run after grafting takes place, giving us a chance to patch<= br> > special things like checksums.=C2=A0 The hook would be passed the list= of
> files that were been modified during grafting.=C2=A0 Then, in the Rack= et
> package for example, I could write a graft-hook that updates the SHA-1=
> hash of each of the modified source files.


+1
I think this would be= a good design choice.
We also gain back the security that the or= iginal check provides.
=C2=A0
This seems like a really good approach to me and seems also much nicer / safer in the long run than the solution in #30680 since it wouldn't jus= t
patch out the package in question's checks, it would correct them.=C2= =A0 That
seems very good indeed to me.

> Since grafting is done at the derivation level, the hook code would ha= ve
> to be propagated down from the package level.=C2=A0 I haven=E2=80=99t = looked at all
> the details yet, because maybe this is a bad idea and I shouldn=E2=80= =99t waste
> my time!=C2=A0 :)=C2=A0 My first impression is that it is not too tric= ky.
>
> Are these problems too specialized to deserve a general mechanism like=
> this?=C2=A0 Let me know what you think!
>
>
> -- Tim

As said, it seems good to me.=C2=A0 But I would be interested in what Mark<= br> would think, since he is mostly responsible for the grafts design.

--000000000000bbf9b105734beb0f--