quirky behaviour of “guix environment”

quirky behaviour of “guix environment”

[Ricardo Wurmus]

Hi Guix,

for a few days now I have been testing the glibc graft to make software
work on the cluster running CentOS 6.8 (with a heavily patched Linux
that has the nominal version 2.6.32).

The graft seems to work fine (with the exception of a Guile GC warning I
get when running Guix), except when using “guix environment -l”.

In this particular instance a user informed me that running a
“configure” script inside of an environment spawned by “guix environment
-l guix.scm” something wasn’t quite right.  The script would check for R
packages and print something like this:

    checking for R package scater… FATAL: kernel too old
    yes

The check succeeded but somewhere a program was spawned that used the
ungrafted glibc!  I entered the environment myself and noticed that it
contained “ldd” from the glibc package.  And sure enough, running “ldd”
printed “FATAL: kernel too old”.

There are two things I find odd:

1.) the environment includes glibc and its executables.  Is this ever
    desired?  When loading an environment from a file or from a package
    (i.e. when “--ad-hoc” is NOT provided) “guix environment” uses
    “package-environment-inputs”, which runs “package->bag” and then
    “bag-transitive-inputs”.  The resulting list of packages is then
    used as the inputs for a profile derivation.  That seems a bit
    excessive.

    Would it not be sufficient to use only the direct inputs of the
    package as the inputs to the profile derivation?  That way “guix
    environment foo” would behave just like “guix environment --ad-hoc
    input-a-of-foo input-b-of-foo input-c-of-foo”.

    Is there a reason why it creates a whole bag and dumps its contents
    into the inputs of the profile derivation?

2.) How could the *old* glibc end up in the environment?  I expected the
    grafted glibc, which comes with a patch to avoid “FATAL: kernel too
    old”.

I have been trying to reproduce this with a grafted package that is
already in the repository, but I failed.  I tried with “raptor2”, which
has “curl” among its direct inputs.  “curl” has a replacement.  The
replacement ended up in the environment as expected.

I wonder if the difference in behaviour can be explained by the fact
that “curl” is a direct inputs whereas “glibc” is a build system input.

Any ideas?

--
Ricardo

Re: quirky behaviour of “guix environment”

[Björn Höfling]

Hi Ricardo,

On Thu, 22 Feb 2018 14:17:53 +0100
Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> wrote:

> Hi Guix,
> 
> for a few days now I have been testing the glibc graft to make
> software work on the cluster running CentOS 6.8 (with a heavily
> patched Linux that has the nominal version 2.6.32).
> 
> The graft seems to work fine (with the exception of a Guile GC
> warning I get when running Guix), except when using “guix environment
> -l”.
> 

[...]

> 2.) How could the *old* glibc end up in the environment?  I expected
> the grafted glibc, which comes with a patch to avoid “FATAL: kernel
> too old”.


I don't know if this is related, but in the last days/weeks I noticed this:

We all know that there are sometimes those collisions:

warning: collision encountered:
/gnu/store/w27inn24x162y9m4hy5d3k7msn6zip47-ld-wrapper-0/bin/ld
/gnu/store/b31l1fzq9xcafhigjm2a11w8y5mzk8n9-binutils-2281/bin/ld
warning: arbitrarily choosing /gnu/store/w27inn24x162y9m4hy5d3k7msn6zip47-ld-wrapper-0/bin/ld

OK, that's fine, they are "normal". When I did a 

guix environment ... (something without the "-l")

I got hundreds of such collisions in the form of

/gnu/store/hash1-package/bin/a-command
/gnu/store/hash2-package/bin/a-command

i.e. only the hash was different. I can't remember the package/commmands nor can I
reproduce it right now. But I also had the idea that there was a graft and its original
version in the environment.

I will have an eye on it and post the details, if I stumple upon it again.

Björn

Re: quirky behaviour of “guix environment”

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1238 bytes --]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

> 1.) the environment includes glibc and its executables.  Is this ever
>     desired?  When loading an environment from a file or from a package
>     (i.e. when “--ad-hoc” is NOT provided) “guix environment” uses
>     “package-environment-inputs”, which runs “package->bag” and then
>     “bag-transitive-inputs”.  The resulting list of packages is then
>     used as the inputs for a profile derivation.  That seems a bit
>     excessive.
>
>     Would it not be sufficient to use only the direct inputs of the
>     package as the inputs to the profile derivation?  That way “guix
>     environment foo” would behave just like “guix environment --ad-hoc
>     input-a-of-foo input-b-of-foo input-c-of-foo”.
>
>     Is there a reason why it creates a whole bag and dumps its contents
>     into the inputs of the profile derivation?

My interpretation of the intended behavior of "guix environment foo" is
that is that only the inputs of (the bag of) foo should show up in the
environment, not the transitive closure of inputs.  I am surprised to
hear that that is not the case, but perhaps I am missing something.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: quirky behaviour of “guix environment”

[Ricardo Wurmus]

Chris Marusich <cmmarusich@gmail.com> writes:

> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:
>
>> 1.) the environment includes glibc and its executables.  Is this ever
>>     desired?  When loading an environment from a file or from a package
>>     (i.e. when “--ad-hoc” is NOT provided) “guix environment” uses
>>     “package-environment-inputs”, which runs “package->bag” and then
>>     “bag-transitive-inputs”.  The resulting list of packages is then
>>     used as the inputs for a profile derivation.  That seems a bit
>>     excessive.
>>
>>     Would it not be sufficient to use only the direct inputs of the
>>     package as the inputs to the profile derivation?  That way “guix
>>     environment foo” would behave just like “guix environment --ad-hoc
>>     input-a-of-foo input-b-of-foo input-c-of-foo”.
>>
>>     Is there a reason why it creates a whole bag and dumps its contents
>>     into the inputs of the profile derivation?
>
> My interpretation of the intended behavior of "guix environment foo" is
> that is that only the inputs of (the bag of) foo should show up in the
> environment, not the transitive closure of inputs.  I am surprised to
> hear that that is not the case, but perhaps I am missing something.

Yeah, this was also quite a surprise to me.

David, would it be wrong for us to change the behaviour such that only
direct inputs end up in the environment?

Aside from this issue, I find it worrying that the graft for glibc does
not end up in the environment.  This is a serious problem for those
who use “guix environment” on RHEL 6.

Ludo, do you know if this is a more general bug or if it is due to the
design of “guix environment”?

--
Ricardo

Re: quirky behaviour of “guix environment”

[Ludovic Courtès]

Heya,

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Aside from this issue, I find it worrying that the graft for glibc does
> not end up in the environment.  This is a serious problem for those
> who use “guix environment” on RHEL 6.

As you reported on IRC, the problem turned out to be that ‘glibc-final’
was not being grafted.  This should now be fixed by commit
7c788ed2276957ea52858d80faeca2962cd26f8d.

Hopefully substitutes for the ‘glibc-final’ replacement will be
available soon.

(Some other issues you mentioned in your message still deserve
investigation.)

Thanks,
Ludo’.