From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Developing libraries for the GNU system with Guix Date: Fri, 14 Oct 2016 15:01:30 +0200 Message-ID: <87bmynaxud.fsf@gnu.org> References: <87h98fkdjb.fsf@catern.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:46871) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bv27L-0004CE-7n for guix-devel@gnu.org; Fri, 14 Oct 2016 09:01:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bv27F-0003zK-6h for guix-devel@gnu.org; Fri, 14 Oct 2016 09:01:42 -0400 Received: from hera.aquilenet.fr ([2a01:474::1]:51631) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bv27F-0003yw-0G for guix-devel@gnu.org; Fri, 14 Oct 2016 09:01:37 -0400 In-Reply-To: <87h98fkdjb.fsf@catern.com> (sbaugh@catern.com's message of "Thu, 13 Oct 2016 19:57:44 -0400") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: sbaugh@catern.com Cc: guix-devel@gnu.org Hello! sbaugh@catern.com skribis: > When I am hacking on some library Z, I continuously want to test the > effects that my changes to Z have on packages A/B/C which depend on > Z. The same applies, in general, when hacking on any package Z which > other packages A/B/C depend on: While developing, I want to be able to > rapidly mutate Z and see how this affects A/B/C. A very common use case. Others have asked about it. > I am not sure how to best achieve this. Here are some solutions: > > - When you change Z, rebuild A/B/C. > This is much too slow at present. Right, but note that it=E2=80=99s the only way to be confident that the cha= nge in Z doesn=E2=80=99t break A/B/C. That said, I do understand that sometime= s, you want an =E2=80=9CI know what I=E2=80=99m doing=E2=80=9D (i.e., the ABI = of Z hasn=E2=80=99t changed) option to bypass that and test the run-time behavior of A/B/C. For cases where you do want to rebuild anyway, I was thinking of making the =E2=80=98--with-source=E2=80=99 option =E2=80=9Crecursive=E2=80=9D, suc= h that: guix build --with-source=3D./guile-2.0.14rc1.tar.gz guile-json would replace the source of Guile and build Guile-JSON against that. > - Use grafts to update A/B/C through binary patching. > This is also too slow, and AFAIK it can't really be sped up. A/B/C have to be really big for this to be too slow! I think grafting processes several MiB/s on my SSD laptop. Again to make this more convenient, I thought we could have a --with-graft option, which would work like --with-input except that it would graft the new Z onto A/B/C instead of rebuilding them. > - Use LD_LIBRARY_PATH to cause A/B/C to search for Z in a mutable place. > This works for C libraries, but not generically; there are equivalent > variables for some other languages but it's not a full solution. Yeah, not great. > - Before starting to hack on Z, build a new version of Z which includes > a random hash and which A/B/C depend on; then bind-mount a mutable > directory on top of that. (suggested by mark_weaver on IRC) > This is the most palatable hack, but still a hack. The inclusion of a > random hash prevents collision with other packages and the use of > bind-mounting means we aren't actually mutating the store. This > unfortunately also requires privileges: it's not usable by > unprivileged users. It is usable by unprivileged users when user namespaces are available. Under these conditions, you could use =E2=80=98call-with-container=E2=80=99= and bind-mount anything anywhere. Doesn=E2=80=99t sound too nice to me though. > Here are some not currently available possibilities: > > - Create some kind of GUIX_LIBRARY_PATH in which packages are looked up > first before looking at the compiled-in hash. > This would be an attempt to make a generic equivalent of > LD_LIBRARY_PATH. This could theoretically be implemented with variant > symlinks, if they were available: https://lwn.net/Articles/680705/ At first sight this sounds very hacky, very much against the whole idea of functional package management. > - Currently every dependency is located at a well known globally unique > and globally meaningful path; add some kind of "variant package" > construct which specifies a package which is "passed in" to the > environment (maybe by bind-mounting it in a filesystem namespace; > implementation specifics aren't important). To put this in a > programming language sense, one of these packages being present would > turn a Guix distribution from a value into a function. Not sure I understand. What do you mean by =E2=80=9Cpassed in to the environment=E2=80=9D? > - Massively speed up rebuilding A/B/C by performing incremental > builds. Not sure how exactly this could work, it's just a thought. No idea how this could work either. Thanks for raising the issue! Ludo=E2=80=99.