Re: guix environment

Re: guix environment

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> I believe that "guix environment" does not consider all outputs
> properly. As one example, when I execute:
>
> guix environment libpeas
>
> the $PATH doesn't include /gnu/store/...glib-2.42.1-bin/bin where
> ("glib:bin" ,glib "bin") is one of the native-inputs of the package.
>
> Is this intentional?

I don’t think so.

On closer inspection, I see two issues:

  (define (packages->transitive-inputs packages)
    "Return a list of the transitive inputs for all PACKAGES."
    (define (transitive-inputs package)
      (filter-map (match-lambda
                   ((_ (? package? package)) package)
                   (_ #f))    ; <---- !
                  (bag-transitive-inputs
                   (package->bag package))))
    (delete-duplicates
     (append-map transitive-inputs packages)))

Here only inputs of the form ("foo" PKG) are considered; things like
("glib" ,glib "bin") are discarded.

  (define (for-each-search-path proc inputs derivations pure?)
    (let ((paths (map derivation->output-path derivations))) ; <-- !
      [...]

Above, ‘derivation->output-path’ considers only the “out” output,
ignoring others if they are needed.

I think these would need to be adjusted.  Any takers?  :-)

Ludo’.

Re: guix environment

[David Thompson]

Ludovic Courtès <ludo@gnu.org> writes:

> Any takers?  :-)

I can try, unless someone beats me to it.

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

Re: guix environment

[David Thompson]

[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> On closer inspection, I see two issues:
>
>   (define (packages->transitive-inputs packages)
>     "Return a list of the transitive inputs for all PACKAGES."
>     (define (transitive-inputs package)
>       (filter-map (match-lambda
>                    ((_ (? package? package)) package)
>                    (_ #f))    ; <---- !
>                   (bag-transitive-inputs
>                    (package->bag package))))
>     (delete-duplicates
>      (append-map transitive-inputs packages)))
>
> Here only inputs of the form ("foo" PKG) are considered; things like
> ("glib" ,glib "bin") are discarded.
>
>   (define (for-each-search-path proc inputs derivations pure?)
>     (let ((paths (map derivation->output-path derivations))) ; <-- !
>       [...]
>
> Above, ‘derivation->output-path’ considers only the “out” output,
> ignoring others if they are needed.

Here's a patch.  WDYT?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-guix-environment-Consider-all-package-outputs.patch --]
[-- Type: text/x-diff, Size: 1848 bytes --]

From 9609806fb78557d74cf5b3fb47802898ef9d1ecf Mon Sep 17 00:00:00 2001
From: David Thompson <dthompson2@worcester.edu>
Date: Thu, 29 Jan 2015 17:53:17 -0500
Subject: [PATCH] guix: environment: Consider all package outputs.

* guix/scripts/environment.scm (for-each-search-path): Iterate over all
  derivation output paths.
  (packages->transitive-inputs): Process inputs that specify an output, too.
---
 guix/scripts/environment.scm | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index ffa3a09..bb2ce53 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -40,7 +40,12 @@
 Use the output paths of DERIVATIONS to build each search path.  When PURE? is
 #t, the existing search path value is ignored.  Otherwise, the existing search
 path value is appended."
-  (let ((paths (map derivation->output-path derivations)))
+  (let ((paths (append-map (lambda (drv)
+                             (map (match-lambda
+                                   ((_ . output)
+                                    (derivation-output-path output)))
+                                  (derivation-outputs drv)))
+                           derivations)))
     (for-each (match-lambda
                (($ <search-path-specification>
                    variable directories separator)
@@ -177,7 +182,9 @@ packages."
   "Return a list of the transitive inputs for all PACKAGES."
   (define (transitive-inputs package)
     (filter-map (match-lambda
-                 ((_ (? package? package)) package)
+                 ((or (_ (? package? package))
+                      (_ (? package? package) _))
+                  package)
                  (_ #f))
                 (bag-transitive-inputs
                  (package->bag package))))
-- 
2.1.4


[-- Attachment #3: Type: text/plain, Size: 136 bytes --]


-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

Re: guix environment

[Ludovic Courtès]

David Thompson <dthompson2@worcester.edu> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> On closer inspection, I see two issues:
>>
>>   (define (packages->transitive-inputs packages)
>>     "Return a list of the transitive inputs for all PACKAGES."
>>     (define (transitive-inputs package)
>>       (filter-map (match-lambda
>>                    ((_ (? package? package)) package)
>>                    (_ #f))    ; <---- !
>>                   (bag-transitive-inputs
>>                    (package->bag package))))
>>     (delete-duplicates
>>      (append-map transitive-inputs packages)))
>>
>> Here only inputs of the form ("foo" PKG) are considered; things like
>> ("glib" ,glib "bin") are discarded.
>>
>>   (define (for-each-search-path proc inputs derivations pure?)
>>     (let ((paths (map derivation->output-path derivations))) ; <-- !
>>       [...]
>>
>> Above, ‘derivation->output-path’ considers only the “out” output,
>> ignoring others if they are needed.
>
> Here's a patch.  WDYT?
>
>
> From 9609806fb78557d74cf5b3fb47802898ef9d1ecf Mon Sep 17 00:00:00 2001
> From: David Thompson <dthompson2@worcester.edu>
> Date: Thu, 29 Jan 2015 17:53:17 -0500
> Subject: [PATCH] guix: environment: Consider all package outputs.
>
> * guix/scripts/environment.scm (for-each-search-path): Iterate over all
>   derivation output paths.
>   (packages->transitive-inputs): Process inputs that specify an output, too.

[...]

> --- a/guix/scripts/environment.scm
> +++ b/guix/scripts/environment.scm
> @@ -40,7 +40,12 @@
>  Use the output paths of DERIVATIONS to build each search path.  When PURE? is
>  #t, the existing search path value is ignored.  Otherwise, the existing search
>  path value is appended."
> -  (let ((paths (map derivation->output-path derivations)))
> +  (let ((paths (append-map (lambda (drv)
> +                             (map (match-lambda
> +                                   ((_ . output)
> +                                    (derivation-output-path output)))
> +                                  (derivation-outputs drv)))
> +                           derivations)))
>      (for-each (match-lambda
>                 (($ <search-path-specification>
>                     variable directories separator)
> @@ -177,7 +182,9 @@ packages."
>    "Return a list of the transitive inputs for all PACKAGES."
>    (define (transitive-inputs package)
>      (filter-map (match-lambda
> -                 ((_ (? package? package)) package)
> +                 ((or (_ (? package? package))
> +                      (_ (? package? package) _))
> +                  package)
>                   (_ #f))

LGTM, please push!

There’s another problem, though.  When a dependency is a multiple-output
package, all its outputs are added to the environment, because
‘package->transitive-inputs’ discards the information of which output is
needed.

So for instance, both the ‘out’ and the ‘debug’ output of Coreutils end
up being downloaded and added to the environment, even though only ‘out’
is an input.

Now, the problem is that ‘build-derivations’ can only build *all* the
outputs of the given derivation.  This could be worked around either:

  1. by creating a “sink” derivation, for instance with
     ‘profile-derivation’, that could refer precisely to the output(s)
     needed; not ideal.

  2. by using (build-things (list "/the/output/path")) and resorting to
     ‘build-derivations’ only if the ‘build-things’ call did nothing
     (when passed a non-.drv store item, ‘build-things’ tries to
     substitute and does nothing if that fails.)

Thoughts?

Ludo’.

Re: guix environment

[David Thompson]

Ludovic Courtès <ludo@gnu.org> writes:

> LGTM, please push!

Pushed.

> There’s another problem, though.  When a dependency is a multiple-output
> package, all its outputs are added to the environment, because
> ‘package->transitive-inputs’ discards the information of which output is
> needed.

Yes, I had thought about this, but couldn't think of a way to preserve
that information throughout the program.

> So for instance, both the ‘out’ and the ‘debug’ output of Coreutils end
> up being downloaded and added to the environment, even though only ‘out’
> is an input.
>
> Now, the problem is that ‘build-derivations’ can only build *all* the
> outputs of the given derivation.  This could be worked around either:
>
>   1. by creating a “sink” derivation, for instance with
>      ‘profile-derivation’, that could refer precisely to the output(s)
>      needed; not ideal.
>
>   2. by using (build-things (list "/the/output/path")) and resorting to
>      ‘build-derivations’ only if the ‘build-things’ call did nothing
>      (when passed a non-.drv store item, ‘build-things’ tries to
>      substitute and does nothing if that fails.)
>
> Thoughts?

I don't like option #1.  I haven't fully grokked option #2, but I guess
that will be the next thing to try.

Thanks for the nudge in the right direction.

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

bug#19641: guix environment

[Ludovic Courtès]

AFAIK the problem that Federico reported is fixed by 4b7ad2e3, so I’m
closing this bug, and opening a new one for the other issue I raised.

Thanks,
Ludo’.