From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id kCPlL8FOYWKQQAEAbAwnHQ (envelope-from ) for ; Thu, 21 Apr 2022 14:32:01 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id cNvjL8FOYWJbGgAA9RJhRA (envelope-from ) for ; Thu, 21 Apr 2022 14:32:01 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 68C48C739 for ; Thu, 21 Apr 2022 14:32:01 +0200 (CEST) Received: from localhost ([::1]:55298 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nhVya-0005el-GI for larch@yhetil.org; Thu, 21 Apr 2022 08:32:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43440) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nhVlp-0000rm-9F for guix-devel@gnu.org; Thu, 21 Apr 2022 08:18:54 -0400 Received: from coleridge.kublai.com ([166.84.7.167]:55898 helo=mail.spork.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nhVlm-0005BT-9C for guix-devel@gnu.org; Thu, 21 Apr 2022 08:18:48 -0400 Received: from ditto (unknown [IPv6:2001:470:1f07:1b9:8650:a942:ec5e:856b]) by mail.spork.org (Postfix) with ESMTPSA id D295B4648; Thu, 21 Apr 2022 08:17:38 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=spork.org; s=dkim; t=1650543474; bh=Y6wooGpESMtLitfeUCBtAZi2/HUdyzBHRxm7fNYg1ZM=; h=References:From:To:Cc:Subject:Date:In-reply-to; b=UbnxM0SYr2lk3fGVwDh4c+DsIDfUIIU1ZTdPRtTonvX2Ihvr3/dFeSH5jsEwZZQXC T7dJlqc2I0Mc9uHEZCXPYPgqmOkDGVyWKjWlxptpLKlPbP6k4Y53WUG3m7GExyijjk GC+23ae3FqbqlA1AA1GwQRBWGVHDCt1VnCnai3nQ= References: <87ee1rn1ph.fsf@ditto.jhoto.spork.org> <40e59c1a7b3cd1bb4333b912739688a5146336cb.camel@telenet.be> <87v8v2g9tm.fsf@jpoiret.xyz> User-agent: mu4e 1.6.10; emacs 28.1 From: Brian Cully To: Josselin Poiret Subject: Re: Viewing derivation output in the store Date: Thu, 21 Apr 2022 07:50:13 -0400 In-reply-to: <87v8v2g9tm.fsf@jpoiret.xyz> Message-ID: <87sfq6pr4w.fsf@ditto.jhoto.spork.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=166.84.7.167; envelope-from=bjc@spork.org; helo=mail.spork.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1650544321; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=e96SmBMPcpTEQK8OdF1fJ2zBflq3brxy863SVQ+KI5w=; b=Vz26MfydpQh1HzZNEAGar1Wh02mctAMYqQz4HJtIG6nt3U6xbkB++S7HNiaNSaP3halMpJ sJTU3B+YlmW0aqP1KW10KR1q2bp02pBpsPVKz695xLKrMEr4vG0hj4ZsTx3wMkhGgZ2AMl YecZc+WC+VoImzCirSjrqgQ8V4HLsMjRc/UhGx8bi1rMGMiKhkthok9ArsNP2KJyvK5GTP LROt98C9uqvSFSnC/Fd3Tl9FB772G7u1pWo9q2aSd4OU/czbY8Tl+0E7wSriYlWz54bH5d a29lwACAXild2dKIz2IqaV42NhTM2wdH2ZObSy9XBYNEEZHQHCtMSxXd5NtwXg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1650544321; a=rsa-sha256; cv=none; b=S9BF8izHCT08bBLiQCAK4NzH/jBq4e8E2BQnOnqxflfsdcVNxrBVES/J+UATpldEKUuUiL tkZ6U2j7oeSWZW8w2wdbqwcW8S3/b+uYTS58OQOysXLChWUdK0ccE3mRRJ3GrhBNUGcfF2 bFNI6y6iGRmuoI7MKrJ21cwkd1wWoI53lvZYo2Zj2NKWCzAE9xfBluEOY4Ueb5Ocv9hACi hfrQDgAfVptJoSfOKc9xNHRHQb6L3HGhWJfxeFW2gfPjN2IGz1HwthWvYmM3tfiVEUFFyW a4BZoGb7+aaSawFsg/t3yrASeeOiSOUw375gYe00NYiVRiFl+cPWHLLnbR4cow== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=spork.org header.s=dkim header.b=UbnxM0SY; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 0.36 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=spork.org header.s=dkim header.b=UbnxM0SY; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 68C48C739 X-Spam-Score: 0.36 X-Migadu-Scanner: scn0.migadu.com X-TUID: N2Otu0mcscC9 Josselin Poiret writes: > Just for completeness' sake, here's my take on it from yesterday while > we were talking about it with Brian: Thank you for that, it was very enlightening. > (define-module (test) > #:use-module (guix gexp) > #:use-module (guix monads) > #:use-module (guix derivations) > #:use-module (guix store)) > > (define test-gexp > #~(begin > (copy-file #$(plain-file "helloes.txt" "contents") #$output) > (format #t "Helloes~%"))) > > (with-store store > (run-with-store store > (mlet* %store-monad > ((drv (gexp->derivation "myderivation" test-gexp)) > (output -> (derivation->output-path drv))) > (mbegin %store-monad > (built-derivations (list drv)) > (return (format #t "~a~%" output)))))) I tried playing with this for a while and ran into an issue when actually executing it in a REPL: if the derivation hasn=E2=80=99t yet been built, then during execution of of the =E2=80=98built-derivations=E2=80=99,= an error is thrown: ---[snip]--- ice-9/boot-9.scm:1685:16: In procedure raise-exception: In procedure display: Wrong type argument in position 2: # Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. scheme@(zgrub) [1]> ,bt In ice-9/boot-9.scm: 1752:10 9 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) In /home/bjc/guix/guix/store.scm: 658:37 8 (thunk) 2129:24 7 (run-with-store # _ #:gu= ile-for-build _ #:system _ #:target _) In current input: 6373:8 6 (_ _) In /home/bjc/guix/guix/store.scm: 2001:38 5 (_ #) 1421:15 4 (_ # _ _) 759:13 3 (process-stderr _ _) In unknown file: 2 (display "building path(s) `/gnu/store/5jdlbc8vf5d0hf4inzj9k7g= q6r9k6xsy-grub.cfg'\n" #) In ice-9/boot-9.scm: 1685:16 1 (raise-exception _ #:continuable? _) 1685:16 0 (raise-exception _ #:continuable? _) ---[snip]--- I=E2=80=99m way too new to all this to say for sure, but it seems to me that the build log output is being sent to a string, which, at least in the REPL, is closed prematurely (if it=E2=80=99s ever even opened). If I wait a bit for the derivation to build, executing the above code works fine and produces output in the output path. Is this something that=E2=80=99s worthy of a bug report, or am I doing things I=E2=80=99m not supposed to be doing by building derivations in a RE= PL? > I tried to use the most monadic approach here to demonstrate its > usefulness, however, it seems to me that derivation->output-path is not > documented, along with built-derivations (which is just (store-lift > build-derivations)). Yes, thankfully the code was either easy to infer from the name (derivation->output-path) or easy to read (built-derivations). > Lesson learned: don't shadow gexp (hence `test-gexp`)! > > As an aside: is there anything preventing us from having do notation =C3= =A0 > la Haskell? This could help bridge mlet and mbegin with >>=3D, which in > its current form is impractical. Here's what it could look like: Because of the above error, and general cluelessness, I tried removing the =E2=80=98mbegin=E2=80=99 and found that the error went away, b= ut the derivation wasn=E2=80=99t built. My (wrong) assumption was that the =E2=80= =98mbegin=E2=80=99 was redundant with the =E2=80=98mlet=E2=80=99 since I figured the body of the m= let was in the monad context. The documentation also describes mbegin as akin to mlet, although at least the argument list implies a difference with careful reading: mlet has =E2=80=98body=E2=80=99 while mbegin uses =E2=80= =98mexp=E2=80=99. > (mdo %store-monad > (drv <- (gexp-derivation "myderivation" test-gexp)) > (output <- (return (derivation->output-path drv))) > (built-derivations (list drv)) > (return (format #t "~a~%" output))) > > We could even have some more sugar for (x <- (return y)), Haskell has > `let x =3D y`, but we could instead have something like `(x :=3D y)`? This would definitely be more intelligible to me, at least, and I know precious little Haskell. One final note that=E2=80=99s worth covering: while I was digging through code I noticed that file-like objects have an embedded gexp derivation. In the documentation they are described as the declarative counterpart of =E2=80=98gexp->derivation=E2=80=99, but since there is no do= cumented way of turning them directly into a derivation, I was at a loss for what to do with them until someone in IRC suggested using the #~(copy-file =E2=80= =A6) gexp above. It turns out that there are =E2=80=98gexp-compiler=E2=80=99 instructions f= or these things, which will turn them back into derivations. However, there doesn=E2=80=99t seem to be a way to call them directly, and the only way I = could find to invoke them was by using =E2=80=98lower-object=E2=80=99 (again foun= d by grep-find through the codebase). This makes an intuitive sense, but the naming implies I=E2=80=99m missing some larger architectural picture here w= here =E2=80=98raising=E2=80=99 and =E2=80=98lowering=E2=80=99 are more precisely= defined. So the copy-file gexp can be substituted by =E2=80=98(lower-object test-gexp)=E2=80=99: ---[snip]--- (mlet* %store-monad ((drv (lower-object (plain-file =E2=80=A6)))) =E2=80=A6) ---[snip]--- I don=E2=80=99t know how useful these observations are, but hopefully at least a little. I know I=E2=80=99m not alone in trying to understand how to= make the leap from file-likes to actual files; it seems to come up on IRC with some regularity. At a minimum, this email will get indexed by search engines and hopefully answer some future questions from there. -bjc