Re: Finding Dependencies at Run Time

[Thiago Jung Bauermann <bauermann@kolabnow.com>]

Hello Peter,

Peter Polidoro <peter@polidoro.io> writes:

>> Remember the difference between inputs and propagated inputs: they're
>> the same, but when you create a profile, inputs are not part of the
>> profile (so they need a direct store reference, such as RPATH or a
>> wrapper), whereas propagated inputs are part of the profile, so an
>> environment variable allows to find them.
>
> Thank you for patiently explaining, I think I am starting to
> understand now, but please correct me if I am still wrong.

I think there are still some misunderstandings, or perhaps at least to
me it's not clear what your question is. I think if you could provide a
concrete example of what you are trying to understand it would be
helpful.

> So there is package build-time, profile build-time, and run-time.
>
> Wrappers should be used to set environment variables when the value
> can be determined at package build-time, such as absolute paths to
> inputs.

No. Ideally wrappers shouldn't be necessary and wouldn't be used, but
for some software they are a necessary/useful workaround. At least
that's my understanding.

In the ideal situation, a Guix-package program uses absolute paths to
find the resources (libraries, scripts, other files) that it needs. For
example: how does a compiled program finds the library it was linked
with? It has the full path to glibc hard-coded into the ELF's library
runpath:

popigai: readelf --dynamic ~/.guix-profile/bin/htop | grep runpath
 0x000000000000001d (RUNPATH)            Library runpath: [/gnu/store/9rrnm5hdjw7cy96a2a9rfgh6y08wsbmf-ncurses-6.2.20210619/lib:/gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib:/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib:/gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/10.3.0/../../..]

Or if it's a shell script, it has the full path to its interpreter:

popigai: head -n1 ~/.guix-profile/bin/vcsh
#!/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/sh

Sometimes the original program doesn't use absolute paths, but we can
patch it so that it does. This is the case for example in public-inbox,
which invokes “xapian-compact” and expects it to be found in PATH, and
we change it to use the full path (this is in
guix/gnu/packages/mail.scm, line 4111 in the revision of Guix I'm
looking at):

--8<---------------cut here---------------start------------->8---
  ;; Use absolute paths for 'xapian-compact'.
  (substitute* "lib/PublicInbox/Xapcmd.pm"
    (("'xapian-compact'")
    (format #f "'~a'" (search-input-file inputs
                                          "/bin/xapian-compact"))))
--8<---------------cut here---------------end--------------->8---

But sometimes such changes aren't possible so in that case we do use
wrappers to configure environment variables that will cause the program
to find the resource. This also happens in public-inbox, which around
line 4160 has a ‘wrap-programs’ phase that creates a wrapper so that it
can find Perl modules and also the git and curl executables.

One thing that is implicit from the above is that at runtime, Guix
doesn't do anything to help a program find its dependencies. It has
already done its job at build time by hard-coding paths so that the
program can find them on its own or setting up a wrapper, and at profile
build-time, by setting up any relevant environment variables.

> Search paths should be used to set environment variables when the
> value will be determined at profile build-time, such as relative paths
> to either propagated-inputs or packages that also installed in the
> same profile, but not listed explicitly as propagated-inputs to the
> package declaring the search paths.

I'd say that search paths should be used when the set of resources that
a program will need can't be determined at build time. For example, the
Python interpreter can load a huge number of libraries. It doesn't make
sense to hard-code the absolute paths of all of them in the Python
interpreter. Rather, it should look for the ones that are installed in
the profile.

Another use case is when a program's dependency is optional and it can
work well enough without the dependency being available. That way, the
user can install the dependencies they want and the program will use
them.

> I am still a little unclear about the difference between search-paths
> and native-search-paths, though. If not cross-compiling then they act
> the same?

Yes, that's correct.

> If you are cross-compiling, though, then it only finds packages that
> are listed as native-inputs to the package declaring the search paths?

I don't know about cross-compilation profiles, but hopefully it's
similar to the build environment set up by Guix when cross-compiling a
package. Because of that, I'll answer some of your questions in this
section of the email but not all of them.

When Guix cross-compiles a package, native inputs are the ones that will
be used during the process of building the package (and not at runtime),
so they should be built for the architecture of the build machine. For
example, the compiler itself, the make tool, etc. The regular inputs
will be used by the package at runtime and thus will be built for the
target architecture.

> Native-inputs are not normally installed in a profile, though,
> correct?

In the case of a non-cross build, the native inputs will also be
installed but will be of the same architecture as the normal inputs.
In this case, it doesn't make any difference whether a dependency is
listed in the native-inputs or inputs field of a package.

I hope I haven't added to the confusion. :-)

-- 
Thanks
Thiago