From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Workflow management with GNU Guix Date: Sat, 29 Oct 2016 22:56:12 +0200 Message-ID: <87h97uvpqr.fsf@gnu.org> References: <87wpmzhdk2.fsf@gnu.org> <87wpmvyocy.fsf@gnu.org> <87wpgwimn3.fsf@gnu.org> <87wpgvs2nz.fsf@gnu.org> <87twbz9qi3.fsf@gnu.org> <87zilobonw.fsf@gnu.org> <874m3wh6yu.fsf@gnu.org> <87k2cs8pfe.fsf@gnu.org> <877f8swflw.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:60456) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c0afq-0001L0-HX for guix-devel@gnu.org; Sat, 29 Oct 2016 16:56:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c0afn-00075I-Dp for guix-devel@gnu.org; Sat, 29 Oct 2016 16:56:18 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56338) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c0afn-00075E-AH for guix-devel@gnu.org; Sat, 29 Oct 2016 16:56:15 -0400 In-Reply-To: <877f8swflw.fsf@gnu.org> (Roel Janssen's message of "Fri, 28 Oct 2016 19:25:15 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Roel Janssen Cc: guix-devel@gnu.org Roel Janssen skribis: > Ludovic Court=C3=A8s writes: > >> Roel Janssen skribis: >> >>> Ludovic Court=C3=A8s writes: >> >> [...] >> >>>> So I guess that=E2=80=99s an argument in favor of the approach you cho= se. >>> >>> Can't a derivation write its output to some other place than the store? >>> Maybe by running it "by hand"? >> >> Yes, if you run it =E2=80=9Cby hand=E2=80=9D, then you can tweak things = as you see fit. >> >>>> Fundamentally, a derivation just describes a command, its arguments, i= ts >>>> dependencies, its outputs, and its environment variables. >>>> >>>> So you=E2=80=99re right: you can very much run a derivation =E2=80=9Cb= y hand=E2=80=9D instead of >>>> letting the daemon do it on your behalf. The only difference is that >>>> you won=E2=80=99t have write access to the store. >>> >>> And that's fine, because we don't want to write the output to the store= :). >>> >>> So, the workflow language should create a derivation, but then >>> guix-daemon should not execute the derivation, but instead, the workflow >>> execution engine can do it so it can avoid writing the output to the >>> store.. right? >> >> Right. In addition to the snippet I gave, you=E2=80=99d need to set the >> environment variables that are specified in the derivation. >> >> For each output of the derivation, one environment variable is defined >> that points to its /gnu/store/=E2=80=A6 file name. So for instance, you= =E2=80=99d also >> need to do: >> >> (setenv "out" "/home/roel/something") >> >> if you want to =E2=80=9Credirect=E2=80=9D the =E2=80=9Cout=E2=80=9D outp= ut to a place that=E2=80=99s not its >> normal place in the store. >> >> With user namespaces, you could simply bind mount /home/roel/something >> to /gnu/store/=E2=80=A6 in the process that runs the derivation builder,= instead >> using of the =E2=80=98setenv=E2=80=99 hack above. > > Ideally, we would do the equivalent of @code{guix environment > --container --ad-hoc --pure } and execute the programs inside > the environment. Unfortunately, that requires super user privileges as > well (for good reasons!). =E2=80=A6 if user namespaces are disabled, but yeah. > It would be great to build this in though.. just for those who want to > do things properly and have the luxury of doing so... > > I'll try to implement this in the upcoming week(s) so we have something > to try out. Cool! Check out =E2=80=98call-with-container=E2=80=99. Essentially you ne= ed to tell it to bind-mount all of (derivation-inputs drv), where drv is the derivation you want to build. Ludo=E2=80=99.