From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Latest guile-daemon changes and bewilderment Date: Tue, 25 Jul 2017 10:44:58 +0200 Message-ID: <87a83snbw5.fsf@gnu.org> References: <8760eicf4m.fsf@cune.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:56752) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dZvSo-0001gp-OW for guix-devel@gnu.org; Tue, 25 Jul 2017 04:45:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dZvSn-0007cA-IM for guix-devel@gnu.org; Tue, 25 Jul 2017 04:45:10 -0400 In-Reply-To: <8760eicf4m.fsf@cune.org> (Caleb Ristvedt's message of "Mon, 24 Jul 2017 05:18:17 -0500") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Caleb Ristvedt Cc: guix-devel@gnu.org Hi reepca! Caleb Ristvedt skribis: > After a pileup of changes and testing, I finally got around to pushing > my changes to the guile-daemon to the branch on savannah. The big > addition is half-working derivation-building (just one at a time, for > now, and only chroot builds for now) in the form of > guix/store/build-derivations.scm. More on the not-working half later. > > This includes reference-scanning, which I think pretty much works, Woow! > but which I regret to say I spent too much time thinking > about. Premature optimization and all that. It takes a rather > different approach to the way the C++ code does it - it uses a trie to > recognize references out of a list of possibilities (namely, anything > thrown in the build environment's store). The idea is that additional > pre-calculation can be performed on each node and it can become a sort > of branching skip table of the sort used in that one string-matching > algorithm whose name I can't remember. At least, that's the way it > works in my head - the parts to make it fast haven't been implemented > yet. I should probably split the scanner out into a separate module. I haven=E2=80=99t put as much thought into the scanning algorithm as you di= d, so I=E2=80=99ll defer to you. Great that you already have a good prototype of that! And yes, I agree it=E2=80=99s probably better to have the scanner in its own module. > The build environment currently is isolated by creating new pid, mount, > ipc, and uts namespaces, remounting everything as MS_PRIVATE, using > pivot-root to make a generated directory the new root, and unmounting > the old root. Sounds good. > So, does it work? Well... sort of. Which brings me to a really bizarre > problem to debug. *deep breath* > > The way I've been going about testing this has been to first delete > test-tmp completely, then run (as root, since I'm testing chroot stuff): > > ./test-env guix build --dry-run hello > ./test-env guile > scheme@(guile-user)> (use-modules (ice-9 readline)) > scheme@(guile-user)> (activate-readline) > scheme@(guile-user)> ,m (guix store build-derivations) > scheme@(guile-user)> (build-derivation > (read-derivation-from-file > "path/to/guix/test-tmp/store/...hello-2.10.drv")) > > This eventually fails trying to build the bootstrap make. This is the > only error message I get: http://paste.lisp.org/display/351519. So it > seems that some low-level guile thing is failing. I know from > print-statement debugging that the exact place it fails is when the > gnu-build-system calls apply to start the set-paths phase, but have no > indication of why this is. It doesn't make it to the set-paths > procedure, though. Perhaps someone more familiar with guile's internals > knows more? > > Builder: http://paste.lisp.org/display/351521. > Environment variables: http://paste.lisp.org/display/351523. Is there a line above or below the backtrace mentioning the uncaught exception? Could you =E2=80=98strace -f=E2=80=99 the daemon process? Another test would be to manually run the very command that appears in gnu-make-boot0.drv =E2=80=9Cby hand=E2=80=9D, first outside of a chroot, th= en within a chroot with /gnu/store bind-mounted in it (or you could use PRoot for that.) BTW, I see the code uses =E2=80=98clone=E2=80=99 directly. It would be saf= er to use =E2=80=98call-with-container=E2=80=99, which already handles bind mounts, n= on-local exits, and so on. Would it be an option? Regarding scanning, (guix build grafts) contains a special-purpose reference scanner that Mark carefully optimized; it might be worth looking at. > You'll notice that among the environment variables is > GUILE_AUTO_COMPILE=3D0. This is actually something I added myself because > for some reason the bootstrap guile wasn't honoring the > --no-auto-compile flag, but does honor the environment > variable. Strange. Yeah, we shouldn=E2=80=99t add this environment variable to derivations bec= ause that would influence everything that goes in there. Could it be that =E2=80=98argv=E2=80=99 lacked the executable name as argv[= 0], and thus the argument list was shifted to the left? Let=E2=80=99s maybe try to further debug this interactively on IRC. Thanks for the update! Ludo=E2=80=99.