Re: Specifying dependencies among package outputs?

[Tobias Geerinckx-Rice <me@tobias.gr>]

[-- Attachment #1.1: Type: text/plain, Size: 3816 bytes --]

Simon,

Simon South 写道：
> Am I right in thinking there is no way to specify dependencies 
> among the
> outputs of a single package?

Well, yes, but probably not in the way you mean: they aren't 
specified at all.  Oh dear, nckx's responding to a question about 
‘dependencies’.  Apologies to those who know what's coming.

I avoid the word dependency when discussing Guix and urge you to 
do the same.  It's too ambiguous to be useful.  Some distributions 
use terms like ‘run-time dependency’ or ‘build dependency’ as if 
they're similar, but to me Guix/Nix prove that they're unrelated 
by treating them very differently.  We have inputs, which are 
specified by humans (or implicitly by the build system), and 
references -- which are not[0].

Inputs are built or downloaded when building a package, but only 
references -- the actual occurence of the string 
/gnu/store/<hash>-<package>-<version> in another 
/gnu/store/<subdirectory> -- determine which of those inputs are 
actually relevant after the build has finished.  Only referenced 
store items will be kept around during the next GC or downloaded 
when installing pre-built substitutes.

> To specify that a package's "out" output depends on its "lib" 
> output, for instance.

So it's the build process itself that specifies this: when it's 
complete, /gnu/store/<...>-knot-3.0.1 (:out) may or may not retain 
a reference to /gnu/store/<...>-knot-3.0.1-lib (:lib), and that's 
what determines whether or not :out depends on :lib.

I suspect the majority of packages with a :lib output do so.

If you apply the patch below you'll see (e.g., with ‘guix size’) 
that installing only knot:tools will pull in knot{:out,:lib} 
without any human-made hints to that effect.

> I ask because the Knot package (in gnu/package/dns.scm) builds a 
> number
> of logically distinct targets---daemon, libraries, 
> administrative
> utilities, general-purpose utilities, and documentation---and it 
> would
> be nice to separate at least some of these into individual 
> outputs

Separating tools into administrative vs. general-purpose goes too 
far.

Attached patch:

  $ guix size /gnu/store/...-knot-3.0.1-doc
  total: 0.2 MiB (no references)
  $ guix size /gnu/store/...-knot-3.0.1-lib
  total: 145.0 MiB (self: 2.4 MiB)
  $ guix size /gnu/store/...-knot-3.0.1
  total: 171.1 MiB (self: 5.2 MiB; refers to :lib)
  $ guix size /gnu/store/...-knot-3.0.1-tools
  total: 164.9 MiB (self: 0.4 MiB; refers to :lib)

Old monolithic knot:

  $ guix size /gnu/store/...-knot-3.0.1
  total: 171.5 MiB (self: 8.0 MiB)

> [only the libraries end up a dependency of] Knot Resolver.

Correct.  Building knot-resolver with only ("knot:lib" ,knot 
"lib"):

  $ guix size /gnu/store/...-knot-resolver-5.1.3
  total: 169.1 MiB

Old monolithic knot:

  $ guix size /gnu/store/...-knot-resolver-5.1.3
  total: 183.0 MiB

A saving of 13.9 MiB or <10% is not considered impressive.  Some 
might say it's not worth the complexity although I'm OK with it.

> However, Knot's daemon and utilities have the same dependency on 
> its own
> libraries, so pulling those into a separate "lib" output would 
> be liable
> to break everything else.

Why?

> I've searched and can't find an example of this being done, nor 
> can I
> find any mention of it in the documentation. So I assume it's 
> simply not
> possible, and you would need to define an entirely separate 
> package that
> builds from the same source code---right?

Which other examples or documentation have you read?  How would 
you consider Knot more complex or problematic?

Kind regards,

T G-R

[0]: I'm ignoring propagated-inputs, both to simplify things and 
because it makes me happy.


[-- Attachment #1.2: 0001-gnu-knot-Build-separate-outputs.patch --]
[-- Type: text/x-patch, Size: 3718 bytes --]

From 38ae89365b9bff6676d771c74589af391e53283b Mon Sep 17 00:00:00 2001
From: Tobias Geerinckx-Rice <me@tobias.gr>
Date: Thu, 15 Oct 2020 02:36:02 +0200
Subject: [PATCH] gnu: knot: Build separate outputs.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* gnu/packages/dns.scm (knot)[outputs]: New field adding :doc, :lib,
and :tools outputs.
[arguments]: Add #:configure-flags to install into :doc and :lib.
Add a new ‘split-:tools’ phase to install into :tools.
Add a new ‘break-circular-:lib->:out-reference’ phase to do just that.
---
 gnu/packages/dns.scm | 38 +++++++++++++++++++++++++++++++++-----
 1 file changed, 33 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/dns.scm b/gnu/packages/dns.scm
index 1775660162..c566e7260e 100644
--- a/gnu/packages/dns.scm
+++ b/gnu/packages/dns.scm
@@ -827,13 +827,21 @@ Extensions} (DNSSEC).")
            (delete-file-recursively "src/contrib/libbpf")
            #t))))
     (build-system gnu-build-system)
+    (outputs (list "out" "doc" "lib" "tools"))
     (arguments
      `(#:configure-flags
-       (list "--sysconfdir=/etc"
+       (list (string-append "--docdir=" (assoc-ref %outputs "doc")
+                            "/share/" ,name "-" ,version)
+             (string-append "--mandir=" (assoc-ref %outputs "doc")
+                            "/share/man")
+             (string-append "--infodir=" (assoc-ref %outputs "doc")
+                            "/share/info")
+             (string-append "--libdir=" (assoc-ref %outputs "lib") "/lib")
+             "--sysconfdir=/etc"
              "--localstatedir=/var"
-             "--enable-dnstap"          ; let tools read/write capture files
-             "--enable-fastparser"      ; disabled by default when .git/ exists
-             "--enable-xdp=auto"        ; XXX [=yes] currently means =embedded
+             "--enable-dnstap"         ; let tools read/write capture files
+             "--enable-fastparser"     ; disabled by default when .git/ exists
+             "--enable-xdp=auto"       ; XXX [=yes] currently means =embedded
              "--with-module-dnstap=yes") ; detailed query capturing & logging
        #:phases
        (modify-phases %standard-phases
@@ -868,7 +876,27 @@ Extensions} (DNSSEC).")
                        "install"))))
          (add-after 'install 'install-info
            (lambda _
-             (invoke "make" "install-info"))))))
+             (invoke "make" "install-info")))
+         (add-after 'install 'break-circular-:lib->:out-reference
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let ((lib (assoc-ref outputs "lib")))
+               (for-each (lambda (file)
+                           (substitute* file
+                             (("(prefix=).*" _ assign)
+                              (string-append assign lib "\n"))))
+                         (find-files lib "\\.pc$"))
+               #t)))
+         (add-after 'install 'split-:tools
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out   (assoc-ref outputs "out"))
+                    (tools (assoc-ref outputs "tools")))
+               (for-each (lambda (command)
+                           (mkdir-p (string-append tools (dirname command)))
+                           (rename-file (string-append out command)
+                                        (string-append tools command)))
+                         (list "/bin/kdig"
+                               "/bin/khost"))
+               #t))))))
     (native-inputs
      `(("autoconf" ,autoconf)
        ("automake" ,automake)
-- 
2.28.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]