From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Storing serialised graph along with packages Date: Mon, 24 Jul 2017 14:33:09 +0200 Message-ID: <87vamim2uy.fsf@gnu.org> References: <87a83waerc.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45955) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dZcY0-0005iA-OM for guix-devel@gnu.org; Mon, 24 Jul 2017 08:33:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dZcXw-0003DR-Qa for guix-devel@gnu.org; Mon, 24 Jul 2017 08:33:16 -0400 In-Reply-To: <87a83waerc.fsf@elephly.net> (Ricardo Wurmus's message of "Sat, 22 Jul 2017 13:32:23 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus Cc: guix-devel Hello! Ricardo Wurmus skribis: > it always bothered me that after building a package we lose all of the > beautiful features that Guix as a Guile library gives us. We always > need to keep track of the Guix version at the time of building the > package and only then can we hope to rebuild the same thing again at > some point in the future. > > What do you think about storing the serialised subset of the package > graph in a separate output of the package? Currently, the only place > where we store anything meta is the database. Wouldn=E2=80=99t it be gre= at if > we could =E2=80=9Cdump an image=E2=80=9D of the state of Guile when it ha= s evaluated the > section of the package graph that is needed to build it? > > Then we could just load the serialised state into Guile at a later point > and inspect the package graph as if we had Guix checked out at the given > version. I suppose we could also store this kind of information in the > database. > > I=E2=80=99d really like the graph to stay alive even after Guix has moved= on to > later versions. It also sounds like a really lispy thing to do. I sympathize with the goal, and I like the parallel with Lisp. However I=E2=80=99m skeptical about our ability to do something that is rob= ust enough. The package =E2=86=92 bag =E2=86=92 derivation compilation process= is =E2=80=9Clossy=E2=80=9D in the sense that at each layer we lose a bit of context from the higher layers. Each arrow potentially involves all the code and package definitions of Guix, as opposed to just a subset of the package definitions. We could certainly serialize package objects to sexps, but that would not capture the implementation of build systems, =E2=80=98package-derivation=E2=80=99, or even lower-level primitives. So t= his would be a rough approximation, at best. The safe way to achieve what you want would be to store the whole Guix tree (+ GUIX_PACKAGE_PATH), or a pointer to that (a Git commit). There=E2=80=99s a also the problem of bit-for-bit reproducibility: there=E2= =80=99s an infinite set of source trees that can lead to a given store item. If we stored along with, say, Emacs, the Guix source tree/commit that led to it, then we=E2=80=99d effectively remove that equivalence (whitespace would become significant, for instance[*].) Tough issue! Ludo=E2=80=99. [*] For gexps we have a similar problem, where we=E2=80=99d like to have so= urce code mapping, but that how that mapping is done should not affect the .drv itself.