bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 336 bytes --]


The 'wrap' phase of 'python-build-system' adds native-inputs to
PYTHONPATH of the wrapped programs. This causes unnecessary runtime
dependencies and may create problems when cross-compiling.

Given that all python dependencies are now supposed to be propagated,
perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Ludovic Courtès]

Marius Bakke <mbakke@fastmail.com> skribis:

> The 'wrap' phase of 'python-build-system' adds native-inputs to
> PYTHONPATH of the wrapped programs. This causes unnecessary runtime
> dependencies and may create problems when cross-compiling.
>
> Given that all python dependencies are now supposed to be propagated,
> perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.

We cannot simply remove it: it is meant for packages that provide
standalone programs, such as ‘certbot’.  For these, it automatically
wraps bin/* and we don’t even have to have propagated inputs, which is
pretty cool.

However, you are right that native inputs shouldn’t be there.  Probably
we need to add some filtering in the ‘wrap’ phase?

Ludo’.

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Leo Famulari]

On Tue, Dec 20, 2016 at 03:00:50PM +0100, Ludovic Courtès wrote:
> Marius Bakke <mbakke@fastmail.com> skribis:
> 
> > The 'wrap' phase of 'python-build-system' adds native-inputs to
> > PYTHONPATH of the wrapped programs. This causes unnecessary runtime
> > dependencies and may create problems when cross-compiling.
> >
> > Given that all python dependencies are now supposed to be propagated,
> > perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.
> 
> We cannot simply remove it: it is meant for packages that provide
> standalone programs, such as ‘certbot’.  For these, it automatically
> wraps bin/* and we don’t even have to have propagated inputs, which is
> pretty cool.
> 
> However, you are right that native inputs shouldn’t be there.  Probably
> we need to add some filtering in the ‘wrap’ phase?

While looking at the beets package definition with fresh eyes, I realize
that it will surely break once this change is made. So it will make a
good test case for this improvement :)

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Arun Isaac]

I'm working on fixing this bug. I have modified
guix/build/python-build-system.scm for the same. In particular, I have
added #:use-module (guix packages) because I need the functions
`package-name' and `package-transitive-target-inputs'. But, when I try
building any python-build-system package with something like
"./pre-inst-env guix build scons", I get a ("no code for module" (guix
packages)) error. What am I missing?

Full backtrace follows:

Backtrace:
In ice-9/eval.scm:
 432: 19 [eval # ()]
In system/base/compile.scm:
 152: 18 [compile-file "/gnu/store/38rxls2y8v8zjva7xq7zv99rv6c81bq3-module-import/guix/build/python-build-system.scm" ...]
  43: 17 [call-once #<procedure ae84c0 at system/base/compile.scm:56:5 ()>]
In ice-9/boot-9.scm:
 174: 16 [with-throw-handler #t ...]
In system/base/compile.scm:
  59: 15 [#<procedure ae8480 at system/base/compile.scm:58:9 ()>]
 155: 14 [#<procedure ae8500 at system/base/compile.scm:153:8 (port)> #<closed: file 0>]
 218: 13 [read-and-compile #<input: /gnu/store/38rxls2y8v8zjva7xq7zv99rv6c81bq3-module-import/guix/build/python-build-system.scm 6> ...]
 234: 12 [lp () #f #<module (#{ g152}#) ae7a20>]
 182: 11 [lp (#<procedure compile-tree-il (x e opts)>) (define-module # # ...) ...]
In ice-9/boot-9.scm:
2404: 10 [save-module-excursion #<procedure c967e0 at language/scheme/compile-tree-il.scm:29:3 ()>]
In language/scheme/compile-tree-il.scm:
  31: 9 [#<procedure c967e0 at language/scheme/compile-tree-il.scm:29:3 ()>]
In ice-9/psyntax.scm:
1106: 8 [expand-top-sequence ((define-module # # # ...)) () ((top)) ...]
 989: 7 [scan ((define-module # # # ...)) () ((top)) ...]
 279: 6 [scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
In ice-9/eval.scm:
 411: 5 [eval # ()]
In ice-9/boot-9.scm:
2954: 4 [define-module* (guix build python-build-system) #:filename ...]
2929: 3 [resolve-imports ((# # gnu:) (#) (#) (#) ...)]
2870: 2 [resolve-interface (guix packages) #:select ...]
In unknown file:
   ?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (guix packages)) #f]
In ice-9/boot-9.scm:
 109: 0 [#<procedure ae8440 at ice-9/boot-9.scm:100:6 (thrown-k . args)> misc-error ...]

ice-9/boot-9.scm:109:20: In procedure #<procedure ae8440 at ice-9/boot-9.scm:100:6 (thrown-k . args)>:
ice-9/boot-9.scm:109:20: no code for module (guix packages)
builder for `/gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv' failed with exit code 1
@ build-failed /gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv - 1 builder for `/gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv' failed with exit code 1
cannot build derivation `/gnu/store/pm2r6cbcfij8d7nzdxkzqx1dq880mhg8-scons-2.3.4.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/pm2r6cbcfij8d7nzdxkzqx1dq880mhg8-scons-2.3.4.drv' failed

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1783 bytes --]

Hello,

Arun Isaac <arunisaac@systemreboot.net> skribis:

> I'm working on fixing this bug. I have modified
> guix/build/python-build-system.scm for the same. In particular, I have
> added #:use-module (guix packages) because I need the functions
> `package-name' and `package-transitive-target-inputs'. But, when I try
> building any python-build-system package with something like
> "./pre-inst-env guix build scons", I get a ("no code for module" (guix
> packages)) error. What am I missing?
>
> Full backtrace follows:

[...]

> 2954: 4 [define-module* (guix build python-build-system) #:filename ...]
> 2929: 3 [resolve-imports ((# # gnu:) (#) (#) (#) ...)]
> 2870: 2 [resolve-interface (guix packages) #:select ...]
> In unknown file:
>    ?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (guix packages)) #f]

“Build-side” modules, which typically live in (guix build …), should not
depend on “host-side” modules such as (guix packages).  That’s because
if we did that, we’d effectively end up importing all of Guix on the
build side, but then we’d also have to serialize data structures such as
packages to pass them from one side to the other.  (I hope this makes
sense to you, but if it doesn’t maybe the intro of
<https://gnu.org/s/guix/manual/html_node/G_002dExpressions.html> can
shed some light.)

So in short, we cannot use ‘package-name’ and
‘package-transitive-target-inputs’ in this module.

(Time passes…)

I wasn’t sure how to fix this bug myself so I gave it a try and ended up
with the patch below, but I haven’t tested in detail.  (You’ll notice
(guix build-system python) is hard to work with because it doesn’t use
gexps yet.)

How does it look?

Thanks,
Ludo’.


[-- Attachment #2: Type: text/x-patch, Size: 6041 bytes --]

diff --git a/guix/build-system/python.scm b/guix/build-system/python.scm
index 17173f121..a05fd5a79 100644
--- a/guix/build-system/python.scm
+++ b/guix/build-system/python.scm
@@ -153,7 +153,7 @@ pre-defined variants."
                 #:rest arguments)
   "Return a bag for NAME."
   (define private-keywords
-    '(#:source #:target #:python #:inputs #:native-inputs))
+    '(#:source #:target #:python #:inputs))
 
   (and (not target)                               ;XXX: no cross-compilation
        (bag
@@ -174,6 +174,9 @@ pre-defined variants."
 
 (define* (python-build store name inputs
                        #:key
+                       (native-inputs '())
+                       ;; TODO: Something like this:
+                       ;; (disallowed-references native-inputs)
                        (tests? #t)
                        (test-target "test")
                        (use-setuptools? #t)
@@ -189,17 +192,24 @@ pre-defined variants."
                                   (guix build utils))))
   "Build SOURCE using PYTHON, and with INPUTS.  This assumes that SOURCE
 provides a 'setup.py' file as its build system."
+  (define canonicalize-reference
+    (match-lambda
+      (((? derivation? source))
+       (derivation->output-path source))
+      (((? package? package))
+       (derivation->output-path
+        (package-derivation store package system)))
+      ((source)
+       source)
+      (source
+       source)))
+
   (define builder
     `(begin
        (use-modules ,@modules)
        (python-build #:name ,name
-                     #:source ,(match (assoc-ref inputs "source")
-                                 (((? derivation? source))
-                                  (derivation->output-path source))
-                                 ((source)
-                                  source)
-                                 (source
-                                  source))
+                     #:source ,(canonicalize-reference
+                                (assoc-ref inputs "source"))
                      #:configure-flags ,configure-flags
                      #:system ,system
                      #:test-target ,test-target
@@ -209,7 +219,17 @@ provides a 'setup.py' file as its build system."
                      #:outputs %outputs
                      #:search-paths ',(map search-path-specification->sexp
                                            search-paths)
-                     #:inputs %build-inputs)))
+                     #:inputs %build-inputs
+
+                     ;; We call them "native inputs" but there's no
+                     ;; cross-compilation here, so that really means
+                     ;; "build-time-only" inputs, things should not be
+                     ;; run-time dependencies.
+                     #:native-inputs ',(map (match-lambda
+                                              ((name . rest)
+                                               `(,name
+                                                 . ,(canonicalize-reference rest))))
+                                            native-inputs))))
 
   (define guile-for-build
     (match guile
@@ -222,6 +242,8 @@ provides a 'setup.py' file as its build system."
 
   (build-expression->derivation store name builder
                                 #:inputs inputs
+                                ;; TODO:
+                                ;; #:disallowed-references disallowed-references
                                 #:system system
                                 #:modules imported-modules
                                 #:outputs outputs
diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index dd07986b9..1ca26104b 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2013, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013, 2015, 2016, 2017 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2013 Andreas Enge <andreas@enge.fr>
 ;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
 ;;; Copyright © 2015 Mark H Weaver <mhw@netris.org>
@@ -184,7 +184,7 @@ when running checks after installing the package."
                          configure-flags)))
     (call-setuppy "install" params use-setuptools?)))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (map (cut string-append dir "/" <>)
          (or (scandir dir (lambda (f)
@@ -199,14 +199,27 @@ when running checks after installing the package."
                         (string-append dir "/sbin"))))
                 outputs))
 
+  (define build-time-inputs
+    ;; Built-time-only dependencies.
+    (match native-inputs
+      (((names . directories) ...)
+       directories)))
+
+  (define (build-time-dependency? item)
+    (any (cut string-prefix? <> item)
+         build-time-inputs))
+
+  ;; Wrap binaries such that PYTHONPATH is set appropriately, but remove
+  ;; build-time-only dependencies (aka. #:native-inputs) from the search path.
   (let* ((out  (assoc-ref outputs "out"))
          (python (assoc-ref inputs "python"))
+         (path (search-path-as-string->list
+                (or (getenv "PYTHONPATH") "")))
          (var `("PYTHONPATH" prefix
                 ,(cons (string-append out "/lib/python"
                                       (get-python-version python)
                                       "/site-packages")
-                       (search-path-as-string->list
-                        (or (getenv "PYTHONPATH") ""))))))
+                       (remove build-time-dependency? path)))))
     (for-each (lambda (dir)
                 (let ((files (list-of-files dir)))
                   (for-each (cut wrap-program <> var)

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Arun Isaac]

> “Build-side” modules, which typically live in (guix build …), should not
> depend on “host-side” modules such as (guix packages).  That’s because
> if we did that, we’d effectively end up importing all of Guix on the
> build side, but then we’d also have to serialize data structures such as
> packages to pass them from one side to the other.  (I hope this makes
> sense to you, but if it doesn’t maybe the intro of
> <https://gnu.org/s/guix/manual/html_node/G_002dExpressions.html> can
> shed some light.)

It makes some sense, but I'll read up more.

> So in short, we cannot use ‘package-name’ and
> ‘package-transitive-target-inputs’ in this module.

Ok.

> (Time passes…)
>
> I wasn’t sure how to fix this bug myself so I gave it a try and ended up
> with the patch below, but I haven’t tested in detail.  (You’ll notice
> (guix build-system python) is hard to work with because it doesn’t use
> gexps yet.)
>
> How does it look?

I'm travelling now. I'll get back on Monday and study this patch.

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Arun Isaac]

> How does it look?

I tried your patch. But native-inputs are still there in the wrapped
PYTHONPATH. I printed out the results from various steps from the wrap
phase. They are as follows:

`build-time-inputs' is

("/gnu/store/5wbi6vsi12bmpzygwhys3c0cc6sjdw40-python-beautifulsoup4-4.5.3"
"/gnu/store/kc5gl1j85y9ispjsgbg5f5c6jdy1k32s-python-flask-0.11.1"
"/gnu/store/3ib43rpca6qzwfcav8s9ivm3dikpxq87-python-mock-1.0.1"
"/gnu/store/6q4hszis7z6x44ig2zlwjwlcjy3vvfmp-python-mpd2-0.5.5"
"/gnu/store/5wx6yffx6dppd2dr91qfi1jlpqlp9f5m-python-nose-1.3.7"
"/gnu/store/kqjfgfwzm7sq43fnng1w23mjyr2xrd75-python-pathlib-1.0.1"
"/gnu/store/b9yn8bfd43pvrp2wyinf6rf2qw113r5p-python-pyxdg-0.25"
"/gnu/store/g4vfl4wy81y14fjhlpmgqh0i01cg37mb-python-pylast-1.6.0"
"/gnu/store/nim5r75b1kzawln3a1bgag2f12vdmb8x-python-rarfile-2.8"
"/gnu/store/wbaaacm1m7bzbhi3z3scwbz2p4pf2ybg-python-responses-0.5.1")

`path' is

("/gnu/store/5wbi6vsi12bmpzygwhys3c0cc6sjdw40-python-beautifulsoup4-4.5.3/lib/python3.5/site-packages"
"/gnu/store/r3f04sda4gh5vbsa7zb5x0ygf326cp75-python-flask-0.11.1/lib/python3.5/site-packages"
"/gnu/store/3ib43rpca6qzwfcav8s9ivm3dikpxq87-python-mock-1.0.1/lib/python3.5/site-packages"
"/gnu/store/5mkb6im3dijkw4f5146mjz4c49qgcklq-python-mpd2-0.5.5/lib/python3.5/site-packages"
"/gnu/store/9z450rcsza3x448ky83fkd5636ck9aqd-python-nose-1.3.7/lib/python3.5/site-packages"
"/gnu/store/kqjfgfwzm7sq43fnng1w23mjyr2xrd75-python-pathlib-1.0.1/lib/python3.5/site-packages"
"/gnu/store/b9yn8bfd43pvrp2wyinf6rf2qw113r5p-python-pyxdg-0.25/lib/python3.5/site-packages"
"/gnu/store/g4vfl4wy81y14fjhlpmgqh0i01cg37mb-python-pylast-1.6.0/lib/python3.5/site-packages"
"/gnu/store/nim5r75b1kzawln3a1bgag2f12vdmb8x-python-rarfile-2.8/lib/python3.5/site-packages"
"/gnu/store/wbaaacm1m7bzbhi3z3scwbz2p4pf2ybg-python-responses-0.5.1/lib/python3.5/site-packages"
"/gnu/store/m6ir6mj7laxcwsjrz0lvxm7q04w0anhp-python-discogs-client-2.2.1/lib/python3.5/site-packages"
"/gnu/store/v3xiyw0cj08qmpn08jaiznd8ak3nvdga-python-jellyfish-0.5.6/lib/python3.5/site-packages"
"/gnu/store/pwxvvi24950x6rgvl4f46z89sbpa2b7a-python-munkres-1.0.8/lib/python3.5/site-packages"
"/gnu/store/nws3867b5q78ri461wr3rdwgqzasvv2a-python-musicbrainzngs-0.6/lib/python3.5/site-packages"
"/gnu/store/wsw2nr30nhz29dww1hfjb6lgqn2ckrgy-python-mutagen-1.36/lib/python3.5/site-packages"
"/gnu/store/n7fh268b1g1g87s4djvm672p2xmwkd5r-python-pyyaml-3.12/lib/python3.5/site-packages"
"/gnu/store/y3ncl6ia04vqdkhnhgzb3iabcsczj0zb-python-unidecode-0.04.20/lib/python3.5/site-packages"
"/gnu/store/alk9r3rir93pjmv8im20f8xrvv90219z-python-3.5.2/lib/python3.5/site-packages"
"/gnu/store/z2mmzglkykws7hc9k2sr40fidiv62cpa-python-werkzeug-0.11.15/lib/python3.5/site-packages"
"/gnu/store/5w60y9zyjskv179sf4pmq2kzpff887dv-python-click-6.7/lib/python3.5/site-packages"
"/gnu/store/f4qqgaql0f95alxrz9r52n3m5ahjrc1s-python-jinja2-2.8/lib/python3.5/site-packages"
"/gnu/store/n0l3szawn661n2rjk6nnbajzzc02pmfj-python-itsdangerous-0.24/lib/python3.5/site-packages"
"/gnu/store/rpnfm1v7vnwxl3wpbadz2bz25s2x1bbg-python-six-1.10.0/lib/python3.5/site-packages"
"/gnu/store/64ahrdmjmmm8id5327i5kqjj8ppvi2kl-python-cookies-2.2.1/lib/python3.5/site-packages"
"/gnu/store/2iyrxbr6k1qdcrhfpv3cz1nh1znwnga5-python-requests-2.13.0/lib/python3.5/site-packages"
"/gnu/store/0c28plwcf33lnibf27h8ijig467pphiw-python-oauthlib-1.0.3/lib/python3.5/site-packages"
"/gnu/store/m48bcrmrzrcb79p6dhaq0diwr3aacfq3-python-markupsafe-0.23/lib/python3.5/site-packages")

(remove build-time-dependency? path) is

("/gnu/store/r3f04sda4gh5vbsa7zb5x0ygf326cp75-python-flask-0.11.1/lib/python3.5/site-packages"
"/gnu/store/5mkb6im3dijkw4f5146mjz4c49qgcklq-python-mpd2-0.5.5/lib/python3.5/site-packages"
"/gnu/store/9z450rcsza3x448ky83fkd5636ck9aqd-python-nose-1.3.7/lib/python3.5/site-packages"
"/gnu/store/m6ir6mj7laxcwsjrz0lvxm7q04w0anhp-python-discogs-client-2.2.1/lib/python3.5/site-packages"
"/gnu/store/v3xiyw0cj08qmpn08jaiznd8ak3nvdga-python-jellyfish-0.5.6/lib/python3.5/site-packages"
"/gnu/store/pwxvvi24950x6rgvl4f46z89sbpa2b7a-python-munkres-1.0.8/lib/python3.5/site-packages"
"/gnu/store/nws3867b5q78ri461wr3rdwgqzasvv2a-python-musicbrainzngs-0.6/lib/python3.5/site-packages"
"/gnu/store/wsw2nr30nhz29dww1hfjb6lgqn2ckrgy-python-mutagen-1.36/lib/python3.5/site-packages"
"/gnu/store/n7fh268b1g1g87s4djvm672p2xmwkd5r-python-pyyaml-3.12/lib/python3.5/site-packages"
"/gnu/store/y3ncl6ia04vqdkhnhgzb3iabcsczj0zb-python-unidecode-0.04.20/lib/python3.5/site-packages"
"/gnu/store/alk9r3rir93pjmv8im20f8xrvv90219z-python-3.5.2/lib/python3.5/site-packages"
"/gnu/store/z2mmzglkykws7hc9k2sr40fidiv62cpa-python-werkzeug-0.11.15/lib/python3.5/site-packages"
"/gnu/store/5w60y9zyjskv179sf4pmq2kzpff887dv-python-click-6.7/lib/python3.5/site-packages"
"/gnu/store/f4qqgaql0f95alxrz9r52n3m5ahjrc1s-python-jinja2-2.8/lib/python3.5/site-packages"
"/gnu/store/n0l3szawn661n2rjk6nnbajzzc02pmfj-python-itsdangerous-0.24/lib/python3.5/site-packages"
"/gnu/store/rpnfm1v7vnwxl3wpbadz2bz25s2x1bbg-python-six-1.10.0/lib/python3.5/site-packages"
"/gnu/store/64ahrdmjmmm8id5327i5kqjj8ppvi2kl-python-cookies-2.2.1/lib/python3.5/site-packages"
"/gnu/store/2iyrxbr6k1qdcrhfpv3cz1nh1znwnga5-python-requests-2.13.0/lib/python3.5/site-packages"
"/gnu/store/0c28plwcf33lnibf27h8ijig467pphiw-python-oauthlib-1.0.3/lib/python3.5/site-packages"
"/gnu/store/m48bcrmrzrcb79p6dhaq0diwr3aacfq3-python-markupsafe-0.23/lib/python3.5/site-packages")

Somehow, the paths in `build-time-inputs' are not the same as those in
`path'. Notice the different hashes. Hence `remove' does not succeed in
filtering them out.

> +                       ;; TODO: Something like this:
> +                       ;; (disallowed-references native-inputs)

What is this part for?

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Ricardo Wurmus]

Arun Isaac <arunisaac@systemreboot.net> writes:

>> +                       ;; TODO: Something like this:
>> +                       ;; (disallowed-references native-inputs)
>
> What is this part for?

This is to ensure that no references to any native input are retained in
the output.  (Think of it as an assertion that is checked at the end of
the build.)

~~ Ricardo

bug#25235: (no subject)

[Ricardo Wurmus via web]

Have you been able to figure out why some of the items differ in the two lists?

bug#25235: (no subject)

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 208 bytes --]


> Have you been able to figure out why some of the items differ in the two lists?

Hi! Are you asking me? I haven't worked on this in a very long
time. But I'll give it another shot now and get back to you.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#25235: Wrapped python programs get native-inputs in PYTHONPATH

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 3115 bytes --]


Hi,

> Have you been able to figure out why some of the items differ in the
> two lists?

The two lists are the same if `guix build' is invoked with the
`--no-grafts' flag. I don't understand grafts very well and am not sure
how that can be fixed. Any ideas?

See different output below with and without the `--no-grafts' flag.

$ guix build python-mock

--8<---------------cut here---------------start------------->8---
`build-time-inputs' is
(/gnu/store/dhs3v8rfzazq9cb37pqs7fdm5hd7zmf0-python-unittest2-1.1.0)

`path' is
(/gnu/store/v70qcgga1r2m6scnard8s61dqh859mn2-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/w408dkc1lynq5kqi2cxkvikalp6w35qw-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/bwqidwykphzv10n454kxv3hfippq8qrx-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/k4a9433nghpjhs101cm9v8k64bmd8iqf-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/3nl9hk7f59vsxgr7xmnqgc15y9c3rwpg-python-linecache2-1.0.0/lib/python3.7/site-packages)

`(remove build-time-dependency? path)' is
(/gnu/store/v70qcgga1r2m6scnard8s61dqh859mn2-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/w408dkc1lynq5kqi2cxkvikalp6w35qw-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/bwqidwykphzv10n454kxv3hfippq8qrx-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/k4a9433nghpjhs101cm9v8k64bmd8iqf-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/3nl9hk7f59vsxgr7xmnqgc15y9c3rwpg-python-linecache2-1.0.0/lib/python3.7/site-packages)
--8<---------------cut here---------------end--------------->8---

$ guix build --no-grafts python-mock

--8<---------------cut here---------------start------------->8---
`build-time-inputs' is
(/gnu/store/m01qbb9sgymxh2xqwpnbv58h60yl4pz0-python-unittest2-1.1.0)

`path' is
(/gnu/store/m01qbb9sgymxh2xqwpnbv58h60yl4pz0-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/9k5hyd9d7b3r4yzadd6phphf12l2mfc8-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/p8p65yd0pf95yn50iib0jw05p0n4mr9p-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/dvn1fz82ccjg7dnqqmkxhwp720wnmriw-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/vnbk5jswscv68qasfbhmh7wqlsibyism-python-linecache2-1.0.0/lib/python3.7/site-packages)

`(remove build-time-dependency? path)' is
(/gnu/store/9k5hyd9d7b3r4yzadd6phphf12l2mfc8-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/p8p65yd0pf95yn50iib0jw05p0n4mr9p-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/dvn1fz82ccjg7dnqqmkxhwp720wnmriw-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/vnbk5jswscv68qasfbhmh7wqlsibyism-python-linecache2-1.0.0/lib/python3.7/site-packages)
--8<---------------cut here---------------end--------------->8---

Regards,
Arun

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#25235: [PATCH 0/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

This fixes a longstanding issue.  The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.


Maxim Cournoyer (1):
  build-system/pyproject: Do not wrap native-inputs.

 guix/build/pyproject-build-system.scm | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)


base-commit: 9da36aa1e207ff8a8fb2af8bc9442f9c935dc8db
-- 
2.38.1

bug#25235: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

Fixes <https://issues.guix.gnu.org/25235>.

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

 guix/build/pyproject-build-system.scm | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..cd418f7ec9 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
   "A phase that just wraps the 'add-installed-pythonpath' procedure."
   (add-installed-pythonpath native-inputs outputs))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (find-files dir (lambda (file stat)
                       (and (eq? 'regular (stat:type stat))
@@ -464,9 +464,17 @@ (define bindirs
   (define %guile (delay (search-input-file inputs "bin/guile")))
   (define (guile) (force %guile))
 
-  (let* ((var `("GUIX_PYTHONPATH" prefix
-                ,(search-path-as-string->list
-                  (or (getenv "GUIX_PYTHONPATH") "")))))
+  ;; Use the same strategy to compute the native-input file names.
+  (define %native-input-dirs (delay (match native-inputs
+                                      (((_ . dir) ...)
+                                       dir))))
+  (define (native-input-dirs) (force %native-input-dirs))
+
+  (let ((var `("GUIX_PYTHONPATH" prefix
+               ,(remove (lambda (x)
+                          (any (cut string-prefix? <> x) (native-input-dirs)))
+                        (search-path-as-string->list
+                         (or (getenv "GUIX_PYTHONPATH") ""))))))
     (for-each (lambda (dir)
                 (let ((files (list-of-files dir)))
                   (for-each (cut wrap-script <> #:guile (guile) var)
-- 
2.38.1

bug#25235: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Fixes <https://issues.guix.gnu.org/25235>.
>
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.

Yay, great to see this fixed!

>  guix/build/pyproject-build-system.scm | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
> index a66c1fb34a..cd418f7ec9 100644
> --- a/guix/build/pyproject-build-system.scm
> +++ b/guix/build/pyproject-build-system.scm
> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
>    "A phase that just wraps the 'add-installed-pythonpath' procedure."
>    (add-installed-pythonpath native-inputs outputs))
>  
> -(define* (wrap #:key inputs outputs #:allow-other-keys)
> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
>    (define (list-of-files dir)
>      (find-files dir (lambda (file stat)
>                        (and (eq? 'regular (stat:type stat))
> @@ -464,9 +464,17 @@ (define bindirs
>    (define %guile (delay (search-input-file inputs "bin/guile")))
>    (define (guile) (force %guile))
>  
> -  (let* ((var `("GUIX_PYTHONPATH" prefix
> -                ,(search-path-as-string->list
> -                  (or (getenv "GUIX_PYTHONPATH") "")))))
> +  ;; Use the same strategy to compute the native-input file names.
> +  (define %native-input-dirs (delay (match native-inputs
> +                                      (((_ . dir) ...)
> +                                       dir))))
> +  (define (native-input-dirs) (force %native-input-dirs))
> +
> +  (let ((var `("GUIX_PYTHONPATH" prefix
> +               ,(remove (lambda (x)
> +                          (any (cut string-prefix? <> x) (native-input-dirs)))
> +                        (search-path-as-string->list
> +                         (or (getenv "GUIX_PYTHONPATH") ""))))))

I think there’s no need to delay+thunk things since we’re using them
right away.  How about:

  (define native-input-directories
    (match native-inputs
      …))

  (let ((var `("GUIX_PYTHONPATH" prefix
               ,(remove (lambda (x) … native-input-directories …) …))))
    …)

?

(Same for ‘%guile’ above, but I don’t see it in ‘master’; maybe that’s
from another branch?)

Thanks,
Ludo’.

bug#25235: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Fixes <https://issues.guix.gnu.org/25235>.
>>
>> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
>> Filter out native inputs from the values in GUIX_PYTHONPATH.
>
> Yay, great to see this fixed!
>
>>  guix/build/pyproject-build-system.scm | 16 ++++++++++++----
>>  1 file changed, 12 insertions(+), 4 deletions(-)
>>
>> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
>> index a66c1fb34a..cd418f7ec9 100644
>> --- a/guix/build/pyproject-build-system.scm
>> +++ b/guix/build/pyproject-build-system.scm
>> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
>>    "A phase that just wraps the 'add-installed-pythonpath' procedure."
>>    (add-installed-pythonpath native-inputs outputs))
>>
>> -(define* (wrap #:key inputs outputs #:allow-other-keys)
>> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
>>    (define (list-of-files dir)
>>      (find-files dir (lambda (file stat)
>>                        (and (eq? 'regular (stat:type stat))
>> @@ -464,9 +464,17 @@ (define bindirs
>>    (define %guile (delay (search-input-file inputs "bin/guile")))
>>    (define (guile) (force %guile))
>>
>> -  (let* ((var `("GUIX_PYTHONPATH" prefix
>> -                ,(search-path-as-string->list
>> -                  (or (getenv "GUIX_PYTHONPATH") "")))))
>> +  ;; Use the same strategy to compute the native-input file names.
>> +  (define %native-input-dirs (delay (match native-inputs
>> +                                      (((_ . dir) ...)
>> +                                       dir))))
>> +  (define (native-input-dirs) (force %native-input-dirs))
>> +
>> +  (let ((var `("GUIX_PYTHONPATH" prefix
>> +               ,(remove (lambda (x)
>> +                          (any (cut string-prefix? <> x) (native-input-dirs)))
>> +                        (search-path-as-string->list
>> +                         (or (getenv "GUIX_PYTHONPATH") ""))))))
>
> I think there’s no need to delay+thunk things since we’re using them
> right away.  How about:
>
>   (define native-input-directories
>     (match native-inputs
>       …))
>
>   (let ((var `("GUIX_PYTHONPATH" prefix
>                ,(remove (lambda (x) … native-input-directories …) …))))
>     …)
>
> ?

It's a (small) optimization, that was intended to delay things until
they are really needed that was already used for %bash (now %guile) in
the wrap phase of (guix build python-build-system).

The idea is to delay the evaluation of native-input-directories or the
lookup for guile in the inputs until it's really necessary.  For
example, the computations are avoided when there are no "bin" or "sbin"
sub-directories in the outputs.

> (Same for ‘%guile’ above, but I don’t see it in ‘master’; maybe that’s
> from another branch?)

Yes, the change build on both #60847 (Enable cross-compilation for the
pyproject-build-system) and #60868 (Use wrap-script instead of
wrap-program), which should be applied first.  %guile appears in #60868,
and changes that make the above easy (native-inputs always appearing as
native-inputs, even for native builds) are from #60847.

-- 
Thanks,
Maxim

bug#25235: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2744 bytes --]

I'm still on my first cup of coffee...

If a python input is in inputs and in native-inputs would it be included
in the wrapper or not? At first glance I'd say no. Does the search path
do deduplication? If it doesn't then it'd work to only remove the first
instance matching from native-input-dirs.

On Mon, Jan 16, 2023 at 04:29:19PM -0500, Maxim Cournoyer wrote:
> Fixes <https://issues.guix.gnu.org/25235>.
> 
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.
> 
> ---
> 
>  guix/build/pyproject-build-system.scm | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
> index a66c1fb34a..cd418f7ec9 100644
> --- a/guix/build/pyproject-build-system.scm
> +++ b/guix/build/pyproject-build-system.scm
> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
>    "A phase that just wraps the 'add-installed-pythonpath' procedure."
>    (add-installed-pythonpath native-inputs outputs))
>  
> -(define* (wrap #:key inputs outputs #:allow-other-keys)
> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
>    (define (list-of-files dir)
>      (find-files dir (lambda (file stat)
>                        (and (eq? 'regular (stat:type stat))
> @@ -464,9 +464,17 @@ (define bindirs
>    (define %guile (delay (search-input-file inputs "bin/guile")))
>    (define (guile) (force %guile))
>  
> -  (let* ((var `("GUIX_PYTHONPATH" prefix
> -                ,(search-path-as-string->list
> -                  (or (getenv "GUIX_PYTHONPATH") "")))))
> +  ;; Use the same strategy to compute the native-input file names.
> +  (define %native-input-dirs (delay (match native-inputs
> +                                      (((_ . dir) ...)
> +                                       dir))))
> +  (define (native-input-dirs) (force %native-input-dirs))
> +
> +  (let ((var `("GUIX_PYTHONPATH" prefix
> +               ,(remove (lambda (x)
> +                          (any (cut string-prefix? <> x) (native-input-dirs)))
> +                        (search-path-as-string->list
> +                         (or (getenv "GUIX_PYTHONPATH") ""))))))
>      (for-each (lambda (dir)
>                  (let ((files (list-of-files dir)))
>                    (for-each (cut wrap-script <> #:guile (guile) var)
> -- 
> 2.38.1
> 
> 
> 
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#25235: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

Hi Efraim,

Efraim Flashner <efraim@flashner.co.il> writes:

> I'm still on my first cup of coffee...
>
> If a python input is in inputs and in native-inputs would it be included
> in the wrapper or not? At first glance I'd say no. Does the search path
> do deduplication? If it doesn't then it'd work to only remove the first
> instance matching from native-input-dirs.

It wouldn't be included with the current simple logic, which simply
remove any native input from the items of GUIX_PYTHONPATH.  Good catch!
I'll send a v2 with the refined check.

-- 
Thanks,
Maxim

bug#25235: [PATCH v2 0/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

This fixes a longstanding issue.  The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and
inputs
- Enclose wrap computations in an 'unless' form and streamline

Maxim Cournoyer (1):
  build-system/pyproject: Do not wrap native-inputs.

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)


base-commit: 9da36aa1e207ff8a8fb2af8bc9442f9c935dc8db
-- 
2.39.1

bug#25235: [PATCH v2 1/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

Fixes <https://issues.guix.gnu.org/25235>.

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..9da86bfc54 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net>
 ;;; Copyright © 2022 Marius Bakke <marius@gnu.org>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -445,7 +446,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
   "A phase that just wraps the 'add-installed-pythonpath' procedure."
   (add-installed-pythonpath native-inputs outputs))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (find-files dir (lambda (file stat)
                       (and (eq? 'regular (stat:type stat))
@@ -458,20 +459,28 @@ (define bindirs
                          (string-append dir "/sbin"))))
                 outputs))
 
-  ;; Do not require "guile" to be present in the package inputs
-  ;; even when there is nothing to wrap.
-  ;; Also, calculate (guile) only once to prevent some I/O.
-  (define %guile (delay (search-input-file inputs "bin/guile")))
-  (define (guile) (force %guile))
-
-  (let* ((var `("GUIX_PYTHONPATH" prefix
-                ,(search-path-as-string->list
-                  (or (getenv "GUIX_PYTHONPATH") "")))))
-    (for-each (lambda (dir)
-                (let ((files (list-of-files dir)))
-                  (for-each (cut wrap-script <> #:guile (guile) var)
-                            files)))
-              bindirs)))
+  (unless (null? bindirs)
+    (let* ((guile (search-input-file inputs "bin/guile"))
+           (native-input-dirs (match native-inputs
+                                (((_ . dir) ...)
+                                 dir)))
+           (input-dirs (match inputs
+                         (((_ . dir) ...)
+                          dir)))
+           (build-inputs (lset-difference string=? native-input-dirs
+                                          input-dirs))
+           ;; A build input is an input listed in native-inputs and NOT in
+           ;; inputs.
+           (build-input? (lambda (x)
+                           (any (cut string-prefix? <> x) build-inputs)))
+           (var `("GUIX_PYTHONPATH" prefix
+                  ,(remove build-input? (search-path-as-string->list
+                                         (or (getenv "GUIX_PYTHONPATH") ""))))))
+      (for-each (lambda (dir)
+                  (let ((files (list-of-files dir)))
+                    (for-each (cut wrap-script <> #:guile guile var)
+                              files)))
+                bindirs))))
 
 (define* (rename-pth-file #:key name native-inputs outputs #:allow-other-keys
                           #:rest args)
-- 
2.39.1

bug#25235: [PATCH v2 1/1] build-system/pyproject: Do not wrap native-inputs.

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Fixes <https://issues.guix.gnu.org/25235>.
>
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.

LGTM!

> -  ;; Do not require "guile" to be present in the package inputs
> -  ;; even when there is nothing to wrap.
> -  ;; Also, calculate (guile) only once to prevent some I/O.
> -  (define %guile (delay (search-input-file inputs "bin/guile")))
> -  (define (guile) (force %guile))

I still don’t see what branch this snippet is on, what am I missing?
:-)

Thanks,
Ludo’.

bug#25235: [PATCH v2 1/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Fixes <https://issues.guix.gnu.org/25235>.
>>
>> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
>> Filter out native inputs from the values in GUIX_PYTHONPATH.
>
> LGTM!
>
>> -  ;; Do not require "guile" to be present in the package inputs
>> -  ;; even when there is nothing to wrap.
>> -  ;; Also, calculate (guile) only once to prevent some I/O.
>> -  (define %guile (delay (search-input-file inputs "bin/guile")))
>> -  (define (guile) (force %guile))
>
> I still don’t see what branch this snippet is on, what am I missing?
> :-)

It's based on https://issues.guix.gnu.org/60868, itself based on
https://issues.guix.gnu.org/60847.  I've tried to express this
relationship by marking the dependent changes blocked by the submissions
they are based on.

I've just rebased the 3 submissions and added you in CC; hopefully that
helps!

-- 
Thanks,
Maxim

bug#25235: [PATCH v3 0/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

This fixes a longstanding issue.  The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.

Changes in v3:
- Rebase

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

Maxim Cournoyer (1):
  build-system/pyproject: Do not wrap native-inputs.

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)


base-commit: 490e386e10ca2b7496c3da7ed85299ed3e9c2404
-- 
2.39.1

bug#25235: [PATCH v3 1/1] build-system/pyproject: Do not wrap native-inputs.

[Maxim Cournoyer]

Fixes <https://issues.guix.gnu.org/25235>.

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

Changes in v3:
- Rebase

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..9da86bfc54 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net>
 ;;; Copyright © 2022 Marius Bakke <marius@gnu.org>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -445,7 +446,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
   "A phase that just wraps the 'add-installed-pythonpath' procedure."
   (add-installed-pythonpath native-inputs outputs))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (find-files dir (lambda (file stat)
                       (and (eq? 'regular (stat:type stat))
@@ -458,20 +459,28 @@ (define bindirs
                          (string-append dir "/sbin"))))
                 outputs))
 
-  ;; Do not require "guile" to be present in the package inputs
-  ;; even when there is nothing to wrap.
-  ;; Also, calculate (guile) only once to prevent some I/O.
-  (define %guile (delay (search-input-file inputs "bin/guile")))
-  (define (guile) (force %guile))
-
-  (let* ((var `("GUIX_PYTHONPATH" prefix
-                ,(search-path-as-string->list
-                  (or (getenv "GUIX_PYTHONPATH") "")))))
-    (for-each (lambda (dir)
-                (let ((files (list-of-files dir)))
-                  (for-each (cut wrap-script <> #:guile (guile) var)
-                            files)))
-              bindirs)))
+  (unless (null? bindirs)
+    (let* ((guile (search-input-file inputs "bin/guile"))
+           (native-input-dirs (match native-inputs
+                                (((_ . dir) ...)
+                                 dir)))
+           (input-dirs (match inputs
+                         (((_ . dir) ...)
+                          dir)))
+           (build-inputs (lset-difference string=? native-input-dirs
+                                          input-dirs))
+           ;; A build input is an input listed in native-inputs and NOT in
+           ;; inputs.
+           (build-input? (lambda (x)
+                           (any (cut string-prefix? <> x) build-inputs)))
+           (var `("GUIX_PYTHONPATH" prefix
+                  ,(remove build-input? (search-path-as-string->list
+                                         (or (getenv "GUIX_PYTHONPATH") ""))))))
+      (for-each (lambda (dir)
+                  (let ((files (list-of-files dir)))
+                    (for-each (cut wrap-script <> #:guile guile var)
+                              files)))
+                bindirs))))
 
 (define* (rename-pth-file #:key name native-inputs outputs #:allow-other-keys
                           #:rest args)
-- 
2.39.1

bug#25235: [Maxim Cournoyer] bug#25235: [PATCH v3 1/1] build-system/pyproject: Do not wrap native-inputs.

[Nicolas Graves via Bug reports for GNU Guix]

Hi Lars, Ricardo,

I'm advancing with my patch series, which I can submit soon. I was
curious about why this 25235 patch isn't in python-team branch yet since
it's also a very welcome change to the pyproject-build-system.

Cheers,

Nicolas

-------------------- Start of forwarded message --------------------
Subject: bug#25235: [PATCH v3 1/1] build-system/pyproject: Do not wrap
 native-inputs.
To: 25235@debbugs.gnu.org
Cc: arunisaac@systemreboot.net, Maxim Cournoyer <maxim.cournoyer@gmail.com>,
 ludo@gnu.org, Lars-Dominik Braun <lars@6xq.net>, efraim@flashner.co.il,
 rekado@elephly.net, jgart <jgart@dismail.de>, marius@gnu.org
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Mon, 23 Jan 2023 08:34:33 -0500

Fixes <https://issues.guix.gnu.org/25235>.

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

Changes in v3:
- Rebase

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

 guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..9da86bfc54 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net>
 ;;; Copyright © 2022 Marius Bakke <marius@gnu.org>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -445,7 +446,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
   "A phase that just wraps the 'add-installed-pythonpath' procedure."
   (add-installed-pythonpath native-inputs outputs))
 
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
     (find-files dir (lambda (file stat)
                       (and (eq? 'regular (stat:type stat))
@@ -458,20 +459,28 @@ (define bindirs
                          (string-append dir "/sbin"))))
                 outputs))
 
-  ;; Do not require "guile" to be present in the package inputs
-  ;; even when there is nothing to wrap.
-  ;; Also, calculate (guile) only once to prevent some I/O.
-  (define %guile (delay (search-input-file inputs "bin/guile")))
-  (define (guile) (force %guile))
-
-  (let* ((var `("GUIX_PYTHONPATH" prefix
-                ,(search-path-as-string->list
-                  (or (getenv "GUIX_PYTHONPATH") "")))))
-    (for-each (lambda (dir)
-                (let ((files (list-of-files dir)))
-                  (for-each (cut wrap-script <> #:guile (guile) var)
-                            files)))
-              bindirs)))
+  (unless (null? bindirs)
+    (let* ((guile (search-input-file inputs "bin/guile"))
+           (native-input-dirs (match native-inputs
+                                (((_ . dir) ...)
+                                 dir)))
+           (input-dirs (match inputs
+                         (((_ . dir) ...)
+                          dir)))
+           (build-inputs (lset-difference string=? native-input-dirs
+                                          input-dirs))
+           ;; A build input is an input listed in native-inputs and NOT in
+           ;; inputs.
+           (build-input? (lambda (x)
+                           (any (cut string-prefix? <> x) build-inputs)))
+           (var `("GUIX_PYTHONPATH" prefix
+                  ,(remove build-input? (search-path-as-string->list
+                                         (or (getenv "GUIX_PYTHONPATH") ""))))))
+      (for-each (lambda (dir)
+                  (let ((files (list-of-files dir)))
+                    (for-each (cut wrap-script <> #:guile guile var)
+                              files)))
+                bindirs))))
 
 (define* (rename-pth-file #:key name native-inputs outputs #:allow-other-keys
                           #:rest args)
-- 
2.39.1




-------------------- End of forwarded message --------------------

-- 
Best regards,
Nicolas Graves

bug#25235: [Maxim Cournoyer] bug#25235: [PATCH v3 1/1] build-system/pyproject: Do not wrap native-inputs.

[Lars-Dominik Braun]

Hi,

> I'm advancing with my patch series, which I can submit soon. I was
> curious about why this 25235 patch isn't in python-team branch yet since
> it's also a very welcome change to the pyproject-build-system.

I believe this change is based on #60847, which Ludo objected to
(https://issues.guix.gnu.org/60847#19-lineno32). Thus it cannot be merged.

Lars