From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id SOhkLsFkiF9/NQAA0tVLHw (envelope-from ) for ; Thu, 15 Oct 2020 15:03:29 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 6ChgKsFkiF+ebgAA1q6Kng (envelope-from ) for ; Thu, 15 Oct 2020 15:03:29 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1C7B1940466 for ; Thu, 15 Oct 2020 15:03:29 +0000 (UTC) Received: from localhost ([::1]:41562 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kT4mt-0003iU-UT for larch@yhetil.org; Thu, 15 Oct 2020 11:03:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kT4iu-0008Ui-Ul for help-guix@gnu.org; Thu, 15 Oct 2020 10:59:21 -0400 Received: from mailout.easymail.ca ([64.68.200.34]:39642) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kT4is-0002sP-FF for help-guix@gnu.org; Thu, 15 Oct 2020 10:59:20 -0400 Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id CBA6321990 for ; Thu, 15 Oct 2020 14:59:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at emo06-pco.easydns.vpn Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (emo06-pco.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kDE6IHPYtJ9G for ; Thu, 15 Oct 2020 14:59:11 +0000 (UTC) Received: from laptop (unknown [108.162.141.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 7DC942173C for ; Thu, 15 Oct 2020 14:59:11 +0000 (UTC) From: Simon South To: help-guix@gnu.org Subject: Re: Specifying dependencies among package outputs? References: <87eem0l9qc.fsf@simonsouth.net> <87y2k8mihx.fsf@nckx> Date: Thu, 15 Oct 2020 10:54:49 -0400 In-Reply-To: <87y2k8mihx.fsf@nckx> (Tobias Geerinckx-Rice's message of "Thu, 15 Oct 2020 02:37:46 +0200") Message-ID: <87o8l3h746.fsf@simonsouth.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=64.68.200.34; envelope-from=simon@simonsouth.net; helo=mailout.easymail.ca X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/15 10:59:16 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Spam-Score: -0.01 X-TUID: bm8GEpYGJ0A9 Thank you Brett, Julien and Tobias for your responses. They've been super helpful and have cleared things up for me quite a bit. Julien Lepiller writes: > For tracking runtime dependencies, guix uses another mechanism: when a > package holds a reference to another package (by embedding the store > path of that other package), it depends on it, anl guix will pull them > both. This was the key piece of information I was missing: Guix packages don't really deal with "dependencies" per se. Instead a package has inputs and outputs, plus a build process that transforms one set to the other; and the references among these items that result naturally from the transformation process are what Guix uses to determine which packages should be installed alongside which. So a packager's focus ought to be on specifying the package's inputs, outputs and transformation correctly, after which Guix can generally be relied on to do the right thing automatically with regard to "dependencies". This makes sense, and even seems painfully obvious now that I've written it out (hello, functional programming). But it wasn't obvious before. This also clears up for me a bit of remaining confusion around the distinction between "inputs" and "propagated inputs": I wondered why, if a package's inputs are its dependencies, the "propagated-inputs" field is needed at all since surely a package's inputs would always be installed alongside it. The explanation is that a package's inputs are _not_ its dependencies; they are merely inputs to its build process, and whether one becomes a "dependency" depends entirely on whether a reference to it remains in any of the package's outputs. The "propagated-input" field is used to ensure this association is made, even when there is no apparent reference (that Guix can find) in the outputs. Tobias Geerinckx-Rice writes: > If you apply the patch below you'll see (e.g., with =E2=80=98guix size=E2= =80=99) that > installing only knot:tools will pull in knot{:out,:lib} without any > human-made hints to that effect. Thank you for this! This is amazing, and exactly the sort of thing I had in mind. (Though I wonder if the "tools" output would better be called "utils", to match the isc-bind package?) Are you planning on committing these changes? I think they're great. > A saving of 13.9 MiB or <10% is not considered impressive. It's not that bad either, and anyway it seems logical to me for the package to be split up this way considering each of the outputs serves a distinct purpose. >> However, Knot's daemon and utilities have the same dependency on its >> own libraries, so pulling those into a separate "lib" output would be >> liable to break everything else. > > Why? Assuming you didn't mean this rhetorically: My assumption was that without an explicitly stated dependency between the "out" and "lib" packages, Guix wouldn't know to install the two together. So a user running "guix install knot" would get only a binary that aborts at startup with a complaint about missing libraries. Not being aware of "references" in this context is what had me confused. > Which other examples or documentation have you read? I actually did a fair bit of searching in multiple places but because I was looking for "dependencies" and not "references", none of what I found seemed very helpful. The manual could certainly do more to highlight this distinction; in its "package" reference for instance it says quite plainly regarding the "input" fields, "These fields list dependencies of the package."[0] That said, looking at the manual again this morning I quickly found this[1]: The outputs of derivations=E2=80=94i.e., the build results=E2=80=94h= ave a set of =E2=80=9Creferences=E2=80=9D, as reported by the =E2=80=98references=E2= =80=99 RPC or the =E2=80=98guix gc --references=E2=80=99 command (*note Invoking guix gc::). References a= re the set of run-time dependencies of the build results. References are a subset of the inputs of the derivation; this subset is automatically computed by the build daemon by scanning all the files in the outputs. That more-or-less lays it out, but I wouldn't have found it since - It's in the page on "Derivations", which I'm accustomed to thinking of as a low-level detail that can safely be ignored; - A quick scan of the paragraph gives the impression it has more to do with the "guix gc" command than with writing package definitions; and - The paragraph makes no mention of "dependencies" (nor how they and "references" relate to one another), so I wouldn't have landed on it with a plain-text search. To be fair, that paragraph _is_ linked to in the concept index under "dependencies, run-time", so I perhaps should have found it a bit sooner than I did. Even so I think it might be good to add a brief section to the manual that addresses directly how "dependencies" are managed by Guix, since I doubt I'll be the last person who arrives with a traditional package-management mindset and is confused about how Guix handles things. > How would you consider Knot more complex or problematic? Knot seemed to be unique in that its distribution contains the source code for both its executables and the libraries on which they depend, all of which are meant to be built together in a single run. That didn't strike me as unusual at first but searching through gnu/packages, I found plenty of packages whose associated libraries were distributed in a completely separate source distribution and thus were naturally built as a separate package, which made clear the dependency between them. Not finding any obvious examples of packages dependent on their own outputs made me start to think this was perhaps actually very uncommon and not supported by Guix for this reason---though now I understand I didn't find anything because stating this kind of dependency is generally unnecessary, so I was basically searching for something that doesn't exist. [0] https://guix.gnu.org/manual/en/html_node/package-Reference.html#package= -Reference [1] https://guix.gnu.org/manual/en/html_node/Derivations.html#Derivations --=20 Simon South simon@simonsouth.net