What features does a language/runtime need to use guix as a package manager?

What features does a language/runtime need to use guix as a package manager?

[Vinícius dos Santos Oliveira]

Hi,

I was curious about how a language/runtime module system needs to be
designed to use guix as a package manager. I'm developing my own
require() function for a Lua host and I'd like to make sure it can use
guix as a package manager.

From a tutorial by Andrew Tropin[1] I learned about
native-search-paths. So I changed my module system to use a *_PATH
environment variable so guix could teach the Lua-based runtime where
to find the modules.

Then there is the next question: how does the module know where to
install itself to? A similar problem is found in GStreamer, so I'll
refer to the GStreamer scenario from now on.

The guix GStreamer package defines native-search-paths to
GST_PLUGIN_SYSTEM_PATH=lib/gstreamer-1.0. As far as I tested, if a
package gstreamer-foobar depends on gstreamer and install files to
${prefix}/lib/gstreamer-1.0, then GST_PLUGIN_SYSTEM_PATH will be
automatically updated to also contain gstreamer-foobar's
/gnu/store/*/lib/gstreamer-1.0.

My question here is: how should the software packaged in
gstreamer-foobar find out the proper directory to install its plugins?
Should it be ${prefix}/lib/gstreamer-1.0? Should it be
${prefix}/lib/gstreamer-1.1? That's something that shouldn't be
hardcoded. As far as I see, it can just use the pluginsdir from
GStreamer's pkg-config definition. Like so:

$ pkg-config --variable=pluginsdir gstreamer-1.0

However this command will print the wrong dir in guix. It'll print
something like:

/gnu/store/9if71w58d5mkxfxyc7fpz289qssnkqsv-gstreamer-1.18.2/lib/gstreamer-1.0

But that's the "namespace" for the gstreamer package, not for the
gstreamer-foobar package. A solution would be to invoke pkg-config as
follows:

$ pkg-config --define-variable="prefix=${prefix}"
--variable=pluginsdir gstreamer-1.0

This will actually print the proper ${prefix}/lib/gstreamer-1.0. But
that's my question here. How should pkg-config be invoked properly? I
could just as well invoke it as:

$ pkg-config --define-variable="prefix=${prefix}"
--define-variable=libdir='${prefix}/lib' --variable=pluginsdir
gstreamer-1.0


[1] https://youtu.be/R8DtPnP4eL8

-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: What features does a language/runtime need to use guix as a package manager?

[Leo Prikler]

Hello Vinícius

Am Mittwoch, den 03.03.2021, 15:31 -0300 schrieb Vinícius dos Santos
Oliveira:
> Hi,
> 
> I was curious about how a language/runtime module system needs to be
> designed to use guix as a package manager. I'm developing my own
> require() function for a Lua host and I'd like to make sure it can
> use
> guix as a package manager.
As a rule of thumb, the less "smart" your language/runtime is, the
easier it gets.  Now, this is only true to a certain extent – if your
language/runtime does not understand the concept of environment
variables at all, you may need to overthink it a little, but if it
rolls out its own package manager (see also Rust, NPM, Go) things will
get needlessly complicated.

> From a tutorial by Andrew Tropin[1] I learned about
> native-search-paths. So I changed my module system to use a *_PATH
> environment variable so guix could teach the Lua-based runtime where
> to find the modules.
If you're planning to do cross-compilation, you might add a search-path 
in lieu of/addition to the native-search-path, but that sounds like a
good idea.

> Then there is the next question: how does the module know where to
> install itself to? A similar problem is found in GStreamer, so I'll
> refer to the GStreamer scenario from now on.
> 
> The guix GStreamer package defines native-search-paths to
> GST_PLUGIN_SYSTEM_PATH=lib/gstreamer-1.0. As far as I tested, if a
> package gstreamer-foobar depends on gstreamer and install files to
> ${prefix}/lib/gstreamer-1.0, then GST_PLUGIN_SYSTEM_PATH will be
> automatically updated to also contain gstreamer-foobar's
> /gnu/store/*/lib/gstreamer-1.0.
> 
> My question here is: how should the software packaged in
> gstreamer-foobar find out the proper directory to install its
> plugins?
> Should it be ${prefix}/lib/gstreamer-1.0? Should it be
> ${prefix}/lib/gstreamer-1.1? That's something that shouldn't be
> hardcoded. 
Why not?  See [1] for how GStreamer itself does this.
More generally speaking, GNU has standard for Makefiles [2], that
should probably also be followed by your build tool of choice. 

> As far as I see, it can just use the pluginsdir from
> GStreamer's pkg-config definition. Like so:
> 
> $ pkg-config --variable=pluginsdir gstreamer-1.0
> 
> However this command will print the wrong dir in guix. It'll print
> something like:
> 
> /gnu/store/9if71w58d5mkxfxyc7fpz289qssnkqsv-gstreamer-
> 1.18.2/lib/gstreamer-1.0
> 
> But that's the "namespace" for the gstreamer package, not for the
> gstreamer-foobar package. A solution would be to invoke pkg-config as
> follows:
Using pkg-config in such a manner is a hack.  Maintainers of
traditional distros know (and ignore) this.

> $ pkg-config --define-variable="prefix=${prefix}"
> --variable=pluginsdir gstreamer-1.0
> 
> This will actually print the proper ${prefix}/lib/gstreamer-1.0. But
> that's my question here. How should pkg-config be invoked properly? I
> could just as well invoke it as:
> 
> $ pkg-config --define-variable="prefix=${prefix}"
> --define-variable=libdir='${prefix}/lib' --variable=pluginsdir
> gstreamer-1.0
I think you're trying to solve a problem, that does not really need
solving here.  Instead of trying to fix your calls to pkg-config, do
not use pkg-config to find locations to write to!

On a related note, you should probably not change versions in versioned
directories, such as lib/gstreamer-1.0, unless you also break the API
in some way.  I think Python serves as a somewhat good negative
example, since most packages would support 3.x, for some values of x,
but all get shoved into whatever Python was used during build.  Emacs
has a similar issue, which did lead many people into strange situations
during the 27 upgrade or when trying out emacs-next.  This does not
mean, that you can't version stuff like Python or Emacs do, just that
you should be careful and aware of the consequences.  (Also both Python
and Emacs are somewhat justified in doing so, since their minor
versions typically add features, that weren't present prior.)

Regards,
Leo

[1] 
https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/blob/552da8569bca9ea5238bcbdad7432b8f7c0837b9/meson.build#L25-27
[2] 
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html