[bug#34358] [PATCH] gnu: python@2.7: Honor NIX_STORE.

[bug#34358] [PATCH] gnu: python@2.7: Honor NIX_STORE.

[Caleb Ristvedt]

[-- Attachment #1: Type: text/plain, Size: 1279 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Perhaps in the future you could email guix-patches@gnu.org for specific
> commits like this one, especially when it’s not strictly related to the
> daemon?

...

> Could you send an updated version of the patch?

Here it is!

> ./configure does not generate any patch files based on patch templates,
> and that’s on purpose.  Instead, when we need something like this, we
> handle it the way ld-wrapper.in is handled: by doing the substitution
> when creating the derivation.

"When creating the derivation" sounds like it's when the package is lowered
to a derivation, but from what I can see of ld-wrapper in (gnu packages
base) the actual substitution is done when the derivation is built. I
am curious how one would go about doing the substitution when the
package is lowered to a derivation, though. Anyway, for now I'm doing
the substitution at derivation-build-time.

> Last, the patch would need to go to ‘core-updates’ because of the number
> of rebuilds it entails.

Should I mention this somewhere?

Also, I should add that "guix lint" and indent-code.el both want changes
to gnu/packages/python.scm, but not due to changes I made. Should a
separate patch address those?

- reepca


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnu: python@2.7: Honor NIX_STORE. --]
[-- Type: text/x-patch, Size: 7008 bytes --]

From 62e9e9a336ab5608405df8114f78c3cbb9dc3a39 Mon Sep 17 00:00:00 2001
From: Caleb Ristvedt <caleb.ristvedt@cune.org>
Date: Wed, 30 Jan 2019 17:00:32 -0600
Subject: [PATCH] gnu: python@2.7: Honor NIX_STORE.

Previously various python packages would fail to work unless the store they
were kept in was /gnu/store. This fixes that.

* gnu/packages/patches/python-2.7-site-prefixes.patch.in: New file that causes
  python@2.7 to honor NIX_STORE at runtime or, if it isn't set, to use the
  NIX_STORE available when it was built.

* gnu/packages/patches/python-2.7-site-prefixes.patch: Removed.

* gnu/packages/python.scm (python-2.7): generates a patch from
  python-2.7-site-prefixes.patch.in at build-time and applies it.
  (python-3.7): don't apply that patch.
  (python2-minimal): inputs still need to include the patch utility and the
  patch.

* gnu/local.mk: adjust patch name since it's been suffixed with ".in".
---
 gnu/local.mk                                  |  2 +-
 ...atch => python-2.7-site-prefixes.patch.in} |  9 ++++--
 gnu/packages/python.scm                       | 29 ++++++++++++++++---
 3 files changed, 32 insertions(+), 8 deletions(-)
 rename gnu/packages/patches/{python-2.7-site-prefixes.patch => python-2.7-site-prefixes.patch.in} (81%)

diff --git a/gnu/local.mk b/gnu/local.mk
index 57595721cd..af18741a55 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1164,7 +1164,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/python-2-deterministic-build-info.patch	\
   %D%/packages/patches/python-2.7-adjust-tests.patch		\
   %D%/packages/patches/python-2.7-search-paths.patch		\
-  %D%/packages/patches/python-2.7-site-prefixes.patch		\
+  %D%/packages/patches/python-2.7-site-prefixes.patch.in	\
   %D%/packages/patches/python-2.7-source-date-epoch.patch	\
   %D%/packages/patches/python-3-deterministic-build-info.patch	\
   %D%/packages/patches/python-3-search-paths.patch		\
diff --git a/gnu/packages/patches/python-2.7-site-prefixes.patch b/gnu/packages/patches/python-2.7-site-prefixes.patch.in
similarity index 81%
rename from gnu/packages/patches/python-2.7-site-prefixes.patch
rename to gnu/packages/patches/python-2.7-site-prefixes.patch.in
index 9e3066508f..0ac0da46ce 100644
--- a/gnu/packages/patches/python-2.7-site-prefixes.patch
+++ b/gnu/packages/patches/python-2.7-site-prefixes.patch.in
@@ -5,7 +5,7 @@ site-packages (and .pth files) are searched.
 --- Python-2.7.11/Lib/site.py	2016-10-17 23:44:51.930871644 +0200
 ***************
 *** 65,70 ****
---- 65,82 ----
+--- 65,85 ----
   
   # Prefixes for site-packages; add additional prefixes like /usr/local here
   PREFIXES = [sys.prefix, sys.exec_prefix]
@@ -16,9 +16,12 @@ site-packages (and .pth files) are searched.
 + # This is necessary if the packages are not merged into a single
 + # `site-packages` directory (like when using `guix environment`) but
 + # listed in PYTHONPATH (like when running `guix build`).
++ guix_store = os.getenv("NIX_STORE")
++ if not guix_store:
++     guix_store = '@storedir@'
 + for p in sys.path:
-+     if p.startswith('/gnu/store/'):
-+         PREFIXES.append(p[:p.find('/', 44)]) # find first pathsep after hash
++     if p.startswith(guix_store):
++         PREFIXES.append(p[:p.find('/', 34 + len(guix_store))]) # find first pathsep after hash
 + del p
 + 
   # Enable per user site-packages directory
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 1c4ea720f1..7c53e6a4ef 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -76,6 +76,7 @@
 (define-module (gnu packages python)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (gnu packages)
+  #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages dbm)
@@ -106,7 +107,6 @@
         "0x2mvz9dp11wj7p5ccvmk9s0hzjk2fa1m462p395l4r6bfnb3n92"))
       (patches (search-patches "python-2.7-search-paths.patch"
                                "python-2-deterministic-build-info.patch"
-                               "python-2.7-site-prefixes.patch"
                                "python-2.7-source-date-epoch.patch"
                                "python-2.7-adjust-tests.patch"))
       (modules '((guix build utils)))
@@ -138,7 +138,10 @@
                             (assoc-ref %outputs "out") "/lib"))
 
         #:modules ((ice-9 ftw) (ice-9 match)
-                   (guix build utils) (guix build gnu-build-system))
+                   (guix build utils) (guix build gnu-build-system)
+                   (guix config))
+        #:imported-modules ,(cons '(guix config)
+                                    %gnu-build-system-modules)
         #:phases
         (modify-phases %standard-phases
           (add-before
@@ -188,6 +191,19 @@
                           (utime file circa-1980 circa-1980)
                           #t))
                #t)))
+          (add-after 'unpack 'apply-templated-patch
+           (lambda* (#:key inputs #:allow-other-keys)
+             (let ((patch-template (assoc-ref inputs "site-patch"))
+                   (patch "python-2.7-site-prefixes.patch"))
+               ;; generate actual patch
+               (copy-file patch-template patch)
+               (substitute* patch
+                 (("@storedir@")
+                  %store-directory))
+               ;; apply it (taken from (guix packages))
+               (invoke (string-append (assoc-ref inputs "patch") "/bin/patch")
+                       "--force" "--no-backup-if-mismatch"
+                       "-p1" "--input" patch))))
           (add-after 'install 'remove-tests
             ;; Remove 25 MiB of unneeded unit tests.  Keep test_support.*
             ;; because these files are used by some libraries out there.
@@ -244,7 +260,9 @@
        ("readline" ,readline)
        ("zlib" ,zlib)
        ("tcl" ,tcl)
-       ("tk" ,tk)))                               ; for tkinter
+       ("tk" ,tk)                                 ; for tkinter
+       ("site-patch" ,(search-patch "python-2.7-site-prefixes.patch.in"))
+       ("patch" ,patch)))
     (native-inputs
      `(("pkg-config" ,pkg-config)))
     (native-search-paths
@@ -320,6 +338,7 @@ data types.")
      (substitute-keyword-arguments (package-arguments python-2)
        ((#:phases phases)
        `(modify-phases ,phases
+          (delete 'apply-templated-patch)
           ;; Unset SOURCE_DATE_EPOCH while running the test-suite and set it
           ;; again afterwards.  See <https://bugs.python.org/issue34022>.
           (add-before 'check 'unset-SOURCE_DATE_EPOCH
@@ -384,7 +403,9 @@ data types.")
     ;; is invoked upon 'make install'.  'pip' also expects 'ctypes' and thus
     ;; libffi.
     (inputs `(("libffi" ,libffi)
-              ("zlib" ,zlib)))))
+              ("zlib" ,zlib)
+              ("site-patch" ,(search-patch "python-2.7-site-prefixes.patch.in"))
+              ("patch" ,patch)))))
 
 (define-public python-minimal
   (package/inherit python
-- 
2.20.0

[bug#34358] [PATCH] gnu: python@2.7: Honor NIX_STORE.

[Sarah Morgensen]

Hello,

Caleb Ristvedt <caleb.ristvedt@cune.org> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Perhaps in the future you could email guix-patches@gnu.org for specific
>> commits like this one, especially when it’s not strictly related to the
>> daemon?
>
> ...
>
>> Could you send an updated version of the patch?
>
> Here it is!
>
>> ./configure does not generate any patch files based on patch templates,
>> and that’s on purpose.  Instead, when we need something like this, we
>> handle it the way ld-wrapper.in is handled: by doing the substitution
>> when creating the derivation.
>
> "When creating the derivation" sounds like it's when the package is lowered
> to a derivation, but from what I can see of ld-wrapper in (gnu packages
> base) the actual substitution is done when the derivation is built. I
> am curious how one would go about doing the substitution when the
> package is lowered to a derivation, though. Anyway, for now I'm doing
> the substitution at derivation-build-time.
>
>> Last, the patch would need to go to ‘core-updates’ because of the number
>> of rebuilds it entails.
>
> Should I mention this somewhere?
>
> Also, I should add that "guix lint" and indent-code.el both want changes
> to gnu/packages/python.scm, but not due to changes I made. Should a
> separate patch address those?
>
> - reepca
>
>From 62e9e9a336ab5608405df8114f78c3cbb9dc3a39 Mon Sep 17 00:00:00 2001
>From: Caleb Ristvedt <caleb.ristvedt@cune.org>
>Date: Wed, 30 Jan 2019 17:00:32 -0600
>Subject: [PATCH] gnu: python@2.7: Honor NIX_STORE.
>
>Previously various python packages would fail to work unless the store they
>were kept in was /gnu/store. This fixes that.
>
>* gnu/packages/patches/python-2.7-site-prefixes.patch.in: New file that causes
>  python@2.7 to honor NIX_STORE at runtime or, if it isn't set, to use the
>  NIX_STORE available when it was built.
>
>* gnu/packages/patches/python-2.7-site-prefixes.patch: Removed.
>
>* gnu/packages/python.scm (python-2.7): generates a patch from
>  python-2.7-site-prefixes.patch.in at build-time and applies it.
>  (python-3.7): don't apply that patch.
>  (python2-minimal): inputs still need to include the patch utility and the
>  patch.
>
>* gnu/local.mk: adjust patch name since it's been suffixed with ".in".

Given that Python 2.7 is now EOL and Python 3 doesn't seem to use this
NIX_STORE patch in the first place, is this patch still relevant?

--
Sarah

bug#34358: [PATCH] gnu: python@2.7: Honor NIX_STORE.

[Ludovic Courtès]

Hi Sarah,

Sarah Morgensen <iskarian@mgsn.dev> skribis:

>>From 62e9e9a336ab5608405df8114f78c3cbb9dc3a39 Mon Sep 17 00:00:00 2001
>>From: Caleb Ristvedt <caleb.ristvedt@cune.org>
>>Date: Wed, 30 Jan 2019 17:00:32 -0600
>>Subject: [PATCH] gnu: python@2.7: Honor NIX_STORE.
>>
>>Previously various python packages would fail to work unless the store they
>>were kept in was /gnu/store. This fixes that.
>>
>>* gnu/packages/patches/python-2.7-site-prefixes.patch.in: New file that causes
>>  python@2.7 to honor NIX_STORE at runtime or, if it isn't set, to use the
>>  NIX_STORE available when it was built.
>>
>>* gnu/packages/patches/python-2.7-site-prefixes.patch: Removed.
>>
>>* gnu/packages/python.scm (python-2.7): generates a patch from
>>  python-2.7-site-prefixes.patch.in at build-time and applies it.
>>  (python-3.7): don't apply that patch.
>>  (python2-minimal): inputs still need to include the patch utility and the
>>  patch.
>>
>>* gnu/local.mk: adjust patch name since it's been suffixed with ".in".
>
> Given that Python 2.7 is now EOL and Python 3 doesn't seem to use this
> NIX_STORE patch in the first place, is this patch still relevant?

Let’s close it and Caleb or anyone is welcome to reopen it if there’s
interest.

Thanks,
Ludo’.