bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos <maximedevos@telenet.be>]

Christopher Rodriguez schreef op di 21-12-2021 om 16:38 [-0500]:
> Ah, I see. That makes sense.
> 
> However, I don't think we need to necessarily use all of 'beets' inputs 
> as inputs for 'beets-bandcamp', because it will build fine with just the 
> inputs listed. I know it isn't DRY, but it seems like the most efficient 
> way to define the package might be to simply define the packages it is 
> expecting to see, and only those packages:

I don't have a strong opinion here.

>  That way, should someone 
> install 'beets' and then 'beets-bandcamp' at a later time, they don't 
> need to download unused inputs (like, for instance, 'python-rarfile').

This doesn't hold: if someone installs beets and beets-bandcamp, then
they need to download python-rarfile in any case, because beets has
python-rarfile as a reference (see later).

> That said, I suppose at least 'beets' needs to be a propagated-input for 
> 'beets-bandcamp', because IIUC the main difference between the 
> propagated-inputs and inputs is that inputs are used only at build time 
> (like 'BuildRequires' in RPM), whereas propagated-inputs are pulled in 
> as installed dependencies (like 'Requires' in RPM). 'beets' would need 
> to be a propagated-input because 'beets-bandcamp' is a plugin for 
> 'beets', and requires 'beets' to function as expected. Is that correct?

This is vaguely correct in some ways but also incorrect in other ways.

At guix core, the only runtime dependency mechanism is references,
and there's no concept of ‘build-time only’.

Basically, if the store item /gnu/store/xxxx has a file that contains
the string /gnu/store/yyyy, then /gnu/store/xxxx is said to have a
reference to /gnu/store/yyyy. Also, whenever a store item
/gnu/store/xxxx is substituted, its reference /gnu/store/yyyy is
substituted as well.  And having /gnu/store/xxxx in the store prevents
/gnu/store/yyyy from being garbage collected.

This works very well for C and C++ things, where we have nice things
like RUNPATH for libraries and other binaries. But for python, guile,
etc. things, there's a snag: the ‘compiled code’ is (almost) exactly
the source code shuffled around in some directory (possibly with
bytecode compilation, but the bytecode typically doesn't have an
equivalent to RUNPATH), so Guix doesn't find a reference between
/gnu/store/xxxx and /gnu/store/yyyy.

To work around this, there's propagated-inputs: if 'xxx' has 'yyy' in
its propagated-inputs, then whenever 'xxx' is installed in a profile,
'yyy' is installed in the profile as well.

(This is very different from ‘classical’ distros!)

About letting 'beets-bandcamp' propagate 'beets': that would make some
amount of sense, but we don't let, say 'emacs-minion' propagate
'emacs', we don't let 'python-six' propagate 'python'. That is, when
the user asks to install a plugin, we currently only install the plugin
and not the application it extends.

A bug? Maybe, because a plugin is useless without the corresponding
applications. Maybe not, because propagated-inputs are inconvenient in
many ways in Guix; they are a work-around to be avoided. In any case,
this is not specific to beets, so if you'd like to change this, I
suggest starting a thread on guix-devel.

> If so, I am unsure why the other originally propagated-inputs were 
> listed as such when they weren't needed for beets to function.

(The non-optional ones) are needed for beets to function.
How did 'beets' find its dependencies then when they aren't propagated?
Because 'beets' was wrapped to set GUIX_PYTHONPATH (see 'wrap' in
python-build-system.scm), and because of references (see above).

>  I just 
> built beets-bandcamp with everything listed as a propagated-input in my 
> patch moved to an input, and it built fine. Is there a way I could 
> install that built version to test it, to ensure none of the inputs need 
> to be propagated-inputs (aside from 'beets')?

A pure temporary environment
(guix shell --pure beets beets-bandcamp -- beet $do-stuff-with-
bandcamp) can be useful. Not sure about what '$do-stuff-with-bandcamp'
would be, I'm not familiar with beets.

> Please let me know if I'm way off base here; I'm very new to packaging 
> in GNU/Guix! (And thank You for the help while I learn!)
> 
> As for the GUIX_PYTHONPATH and GUIX_BEETSPATH idea, I would love to 
> implement something like that here, but I am running against my 
> inexperience here, and was unable to find useful docs on defining PATHs 
> or 'wrap-program' (I haven't looked exhaustively yet, but only have so 
> much time in the day to do so, unfortunately).
> 
> Could You point me to some resources to explain the mechanisms involved 
> in defining PATHs, or on the 'wrap-program' function? I am more than 
> willing to learn.

It appears that 'search-path-specification' is not documented in the
manual, but you could look at (guix search-paths) and look for examples
in packages (git grep -F search-path is useful). 'wrap-program' is
unfortunately undocumented as well.

Anyway, a separate GUIX_BEETSPATH isn't strictly necessary, though I
believe it would be a slight improvement. But now I think about a bit
more, I don't think it would be useful, because GUIX_BEETSPATH and
GUIX_PYTHONPATH would look in the same subdirectories anyway ...

(Search paths are bound to the ‘consuming’ package, not the ‘providing’
package.)


Greetings,
Maxime.