Re: R and R modules (and a Ruby twist)

[Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On Thu, Sep 24, 2015 at 11:40:57AM +0200, Ricardo Wurmus wrote:
>> Maybe it would be best to append the R version to the site-library
>> directory.  I don’t think we should go further than that and bring in
>> the Guix hash, because I’m willing to trust that packages built with
>> version 3.2.2 are compatible with R 3.2.2, even if the inputs to our R
>> package changed and thus the hash is different.
>
> The exception I can think of is when R provides compile time switches
> for blas or ssl (for example). We don't do that now (Nix does!), but
> if you had two R's with the same version number, it could just be that
> a module 'lifts' that dependency and strictly works with one R (and
> not the other).

Isn’t that expected, though?  That’s a property of the used version of
R, then, not a problem with the package.

> It is the same for Ruby, Perl, Python, Apache, Firefox, etc. Anything
> that allows for building 'site' modules.

I don’t disagree in general.  There may be cases where the variant of
the build-time dependency must be identical to that used at runtime.
But I don’t think this is true for more than a few special packages.
Take R as an example.  Most packages are written in pure R, and thus
only depend on features provided by R.  What features are provided by
the language depends only on the version, not on configure flags.

If a user builds a variant of R that lacks Cairo, for example, then
certain packages won’t work as intended.  But does this mean that we
need to disallow installing packages that would have reduced feature
sets for a mutilated version of R in that case?

> I know this is mostly theoretical at this stage, but why not encourage
> strict isolation of interpreter+modules? That is the only way we'll
> guarantee independence between graphs. Nix/Guix does such a great job
> there, and now we allow interpreters to 'leak' their environments,
> just because of their convention and our trust in things that ought to
> work. And all it costs us is a partial SHA added to the path. So for
> Ruby it would be
>
>   ~/.guix-profile/lib/ruby/2.2.0-edb92950/
>
> instead of
>
>   ~/.guix-profile/lib/ruby/2.2.0/
>
> Personally I can live with the status quo, but somehow I prefer the
> exact isolation. Maybe it will come when someone gets hurt.

For R, Perl, Ruby and Python we are often forced to propagate inputs, so
that they end up in the profile and can be loaded by looking up the path
to the union in some environment variable, such as R_LIBS_SITE,
GEM_HOME, or PYTHONPATH.  These environment variables do not make a
distinction between versions or variants.  (Only Perl allows for a
distinction between major versions by having the major version number as
part of its environment variable: PERL5LIB.)

How would a *user* make sure to use different sets of packages with
different variants of languages?  At the moment, the only way is to
manually set the environment variable to point to the desired path.

With propagated inputs we cannot achieve as much isolation as we would
like to.  There might be a way to actually patch the mechanisms that
these languages use to load additional libraries/packages, patching them
such that they load dependencies by full path rather than by simple
name, similar to how we patch ‘dlopen()’ calls in C programmes.

Only if we can avoid using these inflexible environment variables can we
achieve the kind of isolation you try to get by adding a partial hash to
the output directory.

Just a data point: last time I checked Ruby’s “require” directive allows
for a full path to be given instead of a simple string.  Might there be
a way to forego propagating inputs by patching all “require $string”
statements in Ruby sources in a build phase, much like we automatically
patch shebangs?

To note: this would make it impossible for users to override
libraries/modules by adding an alternative directory containing a
modified version of a module to the list of search paths in the
consulted environment variable.  That’s akin to disabling the
LD_LIBRARY_PATH feature in C programmes.

~~ Ricardo