bug#67048: guix refresh -u -L does not work with relative path

bug#67048: guix refresh -u -L does not work with relative path

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 740 bytes --]

Hello,

to reproduce this weird (and very specific!) problem, do the following:
   cd /tmp
   mkdir proj
copy the attached example.scm into /tmp/proj

Now
   guix refresh -u -L proj python-numpy-illustrated
yields the error
proj/example.scm:10:2: python-numpy-illustrated: updating from version 0.3 to version 0.3.1...
proj/example.scm:10:2: warning: python-numpy-illustrated: no `version' field in source; skipping
and does not update the package; whereas
   guix refresh -u -L /tmp/proj python-numpy-illustrated
works as expected.

Without the "-u" things work with a relative path (as indicated by the
first line before the error message above), and I have not found other
guix commands that pose problems with relative paths.

Andreas


[-- Attachment #2: example.scm --]
[-- Type: text/plain, Size: 1049 bytes --]

(define-module (example)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix download)
  #:use-module (guix packages)
  #:use-module (guix build-system pyproject)
  #:use-module (guix build-system python)
  #:use-module (gnu packages python-xyz))

(define-public python-numpy-illustrated
  (package
    (name "python-numpy-illustrated")
    (version "0.3.1")
    (source
     (origin
       (method url-fetch)
       (uri (pypi-uri "numpy-illustrated" version))
       (sha256
        (base32 "0s7ki6lm9xwd4pj7rx6al230wbywqk11wjvgdk44lbdq2fz7kfxd"))))
    (build-system pyproject-build-system)
    (propagated-inputs (list python-numpy))
    (home-page "https://github.com/axil/numpy-illustrated")
    (synopsis "Helper functions from the NumPy Illustrated guide")
    (description "This package provides helper functions for the
@url{https://betterprogramming.pub/numpy-illustrated-the-visual-guide-to-numpy-3b1d4976de1d?sk=57b908a77aa44075a49293fa1631dd9b,
NumPy Illustrated} programming guide.")
    (license license:expat)))

bug#67048: Acknowledgement (guix refresh -u -L does not work with relative path)

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 154 bytes --]

My mail client has secretly updated the file while I was carrying out
the experiment; the attached version should be the one before refreshing.

Andreas


[-- Attachment #2: example.scm --]
[-- Type: text/plain, Size: 1047 bytes --]

(define-module (example)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix download)
  #:use-module (guix packages)
  #:use-module (guix build-system pyproject)
  #:use-module (guix build-system python)
  #:use-module (gnu packages python-xyz))

(define-public python-numpy-illustrated
  (package
    (name "python-numpy-illustrated")
    (version "0.3")
    (source
     (origin
       (method url-fetch)
       (uri (pypi-uri "numpy-illustrated" version))
       (sha256
        (base32 "1x3gd19hhmlg51i5aj070y7w0lk47gx6yx3r3f45396bgdfnsw4i"))))
    (build-system pyproject-build-system)
    (propagated-inputs (list python-numpy))
    (home-page "https://github.com/axil/numpy-illustrated")
    (synopsis "Helper functions from the NumPy Illustrated guide")
    (description "This package provides helper functions for the
@url{https://betterprogramming.pub/numpy-illustrated-the-visual-guide-to-numpy-3b1d4976de1d?sk=57b908a77aa44075a49293fa1631dd9b,
NumPy Illustrated} programming guide.")
    (license license:expat)))

bug#67048: guix refresh -u -L does not work with relative path

[Simon Tournier]

Hi,

On Fri, 10 Nov 2023 at 18:11, Andreas Enge <andreas@enge.fr> wrote:

>    guix refresh -u -L proj python-numpy-illustrated
> yields the error
> proj/example.scm:10:2: python-numpy-illustrated: updating from version 0.3 to version 0.3.1...
> proj/example.scm:10:2: warning: python-numpy-illustrated: no `version' field in source; skipping

The issue is from several layers; (search-path %load-path file) returns
#false.  See:

 1. package-field-location from (guix packages)
    called by update-package-source from (guix upstream)

 2. update-package-source from (guix upstream)
    called by package-update from same module
    called by update-package from (guix scripts refresh)

The issue looks similar as #66901 [1].  Well, I have tried to improve
the situation by setting some ’canonicalize-path’ here or there.
However, then the next issue is from the call ’(absolute-location loc)’
in ’update-package-source’ from module (guix upstream); it returns:

--8<---------------cut here---------------start------------->8---
Backtrace:
In ice-9/boot-9.scm:
    724:2 19 (call-with-prompt _ _ #<procedure default-prompt-handler (k proc)>)
In ice-9/eval.scm:
    619:8 18 (_ #(#(#<directory (guile-user) 7f45f4f17c80>)))
In guix/ui.scm:
   2324:7 17 (run-guix . _)
  2287:10 16 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  1752:10 15 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
  1752:10 14 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In guix/store.scm:
   659:37 13 (thunk)
  2168:25 12 (run-with-store #<store-connection 256.99 7f45d56dc1e0> #<procedure 7f45d5385ca0 at ice-9/eval.scm:333:13 (a)> #:guile-for-build _ #:system _ …)
In ice-9/eval.scm:
   191:27 11 (_ #(#(#<directory (guix scripts refresh) 7f45ea7c1d20> #<procedure 7f45d5385d60 at ice-9/eval.scm:333:13 (a)>) (#<<update-spec> package…>) …))
In ice-9/boot-9.scm:
    152:2 10 (with-fluid* _ _ _)
    152:2  9 (with-fluid* _ _ _)
    152:2  8 (with-fluid* _ _ _)
In ice-9/eval.scm:
    619:8  7 (_ #(#(#<directory (guix scripts refresh) 7f45ea7c1d20> (#<<update-spec> package: #<package python-numpy-illustrated@0.3 proj/example.s…>) …)))
In srfi/srfi-1.scm:
    634:9  6 (for-each #<procedure 7f45d41d1280 at ice-9/eval.scm:333:13 (a)> (#<<update-spec> package: #<package python-numpy-illustrated@0.3 proj/exam…>))
In ice-9/eval.scm:
    619:8  5 (_ #(#(#(#(#(#(#(#(#(#(#(#(#<directory (guix upstream) 7f45e9af08c0>) #<package python-numpy-illustrated@0.…> …) …) …) …) …) …) …) …) …) …) …))
    155:9  4 (_ #(#(#(#(#(#(#(#(#(#(#(#(#<directory (guix upstream) 7f45e9af08c0>) #<package python-numpy-illustrated@0.…> …) …) …) …) …) …) …) …) …) …) …))
In guix/diagnostics.scm:
   354:20  3 (absolute-location #<<location> file: "proj/example.scm" line: 10 column: 2>)
In unknown file:
           2 (raise #<&formatted-message format: "file '~a' not found on load path\n" arguments: ("proj/example.scm")>)
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Wrong type (expecting exact integer): #<&formatted-message format: "file '~a' not found on load path\n" arguments: ("proj/example.scm")>
--8<---------------cut here---------------end--------------->8---

Hum, it needs to be investigated…

1: https://issues.guix.gnu.org/66901

Cheers,
simon

bug#67048: [PATCH] DRAFT guix: upstream: Allow relative file name.

[Simon Tournier]

Fixes <https://issues.guix.gnu.org/issue/67048>.
Reported by Andreas Enge <andreas@enge.fr>.

* guix/diagnostics.scm (absolute-location): Return FILE from
'canonicalize-path' when 'search-path' fails.
* guix/packages.scm (package-field-location): New procedure 'file-name' and
use it.  When 'search-path' does not find FILE in %LOAD-PATH, try
'canonicalize-path'.
* guix/upstream.scm (update-package-source): When 'search-path' fails, test if
FILE exists.

Change-Id: I9337041b43e17ace82416db5840f04113f9544fc
---
 guix/diagnostics.scm | 13 +++++-----
 guix/packages.scm    | 57 +++++++++++++++++++++++++-------------------
 guix/upstream.scm    |  6 +++--
 3 files changed, 43 insertions(+), 33 deletions(-)

diff --git a/guix/diagnostics.scm b/guix/diagnostics.scm
index 3f1f527b43..f79df1ca2d 100644
--- a/guix/diagnostics.scm
+++ b/guix/diagnostics.scm
@@ -349,12 +349,13 @@ (define (absolute-location loc)
                 ;; 'search-path' might return #f in obscure cases, such as
                 ;; when %LOAD-PATH includes "." or ".." and LOC comes from a
                 ;; file in a subdirectory thereof.
-                (match (search-path %load-path (location-file loc))
-                  (#f
-                   (raise (formatted-message
-                           (G_ "file '~a' not found on load path")
-                           (location-file loc))))
-                  (str str)))
+                (let ((file (location-file loc)))
+                  (or (search-path %load-path file)
+                      (and (file-exists? file)
+                           (canonicalize-path file))
+                      (raise (formatted-message
+                              (G_ "file '~a' not found on load path")
+                              file)))))
             (location-line loc)
             (location-column loc)))
 
diff --git a/guix/packages.scm b/guix/packages.scm
index e2e82692ad..ea05b739a8 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -750,37 +750,44 @@ (define (deprecated-package old-name p)
 (define (package-field-location package field)
   "Return the source code location of the definition of FIELD for PACKAGE, or
 #f if it could not be determined."
+  (define (file-name relative-file file-found line column)
+    (catch 'system-error
+      (lambda ()
+        ;; In general we want to keep relative file names for modules.
+        (call-with-input-file file-found
+          (lambda (port)
+            (go-to-location port line column)
+            (match (read port)
+              ((or ('package inits ...)
+                   ('package/inherit _ inits ...))
+               (let ((field (assoc field inits)))
+                 (match field
+                   ((_ value)
+                    (let ((loc (and=> (source-properties value)
+                                      source-properties->location)))
+                      (and loc
+                           ;; Preserve the original file name, which may be a
+                           ;; relative file name.
+                           (set-field loc (location-file) relative-file))))
+                   (_
+                    #f))))
+              (_
+               #f)))))
+      (lambda _
+        #f)))
+
   (match (package-location package)
     (($ <location> file line column)
      (match (search-path %load-path file)
        ((? string? file-found)
-        (catch 'system-error
-          (lambda ()
-            ;; In general we want to keep relative file names for modules.
-            (call-with-input-file file-found
-              (lambda (port)
-                (go-to-location port line column)
-                (match (read port)
-                  ((or ('package inits ...)
-                       ('package/inherit _ inits ...))
-                   (let ((field (assoc field inits)))
-                     (match field
-                       ((_ value)
-                        (let ((loc (and=> (source-properties value)
-                                          source-properties->location)))
-                          (and loc
-                               ;; Preserve the original file name, which may be a
-                               ;; relative file name.
-                               (set-field loc (location-file) file))))
-                       (_
-                        #f))))
-                  (_
-                   #f)))))
-          (lambda _
-            #f)))
+        (file-name file file-found line column))
        (#f
         ;; FILE could not be found in %LOAD-PATH.
-        #f)))
+        (let ((file-found (and (file-exists? file)
+                               (canonicalize-path file))))
+          (if file-found
+              (file-name file file-found line column)
+              #f)))))
     (_ #f)))
 
 (define-syntax-rule (this-package-input name)
diff --git a/guix/upstream.scm b/guix/upstream.scm
index e28ae12f3f..5403aa833d 100644
--- a/guix/upstream.scm
+++ b/guix/upstream.scm
@@ -645,8 +645,10 @@ (define* (update-package-source package source hash)
                               ((? git-reference? ref)
                                (git-reference-commit ref))
                               (_ #f)))
-               (file        (and=> (location-file loc)
-                                   (cut search-path %load-path <>))))
+               (file        (or (and=> (location-file loc)
+                                       (cut search-path %load-path <>))
+                                (and=> (location-file loc)
+                                       file-exists?))))
           (if file
               ;; Be sure to use absolute filename.  Replace the URL directory
               ;; when OLD-URL is available; this is useful notably for

base-commit: 3d15e9e5bcd7cdad33f9832e4956f494c47e1937
-- 
2.41.0

bug#67048: guix refresh -u -L does not work with relative path

[Simon Tournier]

Hi,

On Mon, 13 Nov 2023 at 19:28, Simon Tournier <zimon.toutoune@gmail.com> wrote:

>>    guix refresh -u -L proj python-numpy-illustrated
>> yields the error
>> proj/example.scm:10:2: python-numpy-illustrated: updating from version 0.3 to version 0.3.1...
>> proj/example.scm:10:2: warning: python-numpy-illustrated: no `version' field in source; skipping

As mentioned in [1], the issue is from ’search-path’ – it returns #false
– it is involved in several layers.

See <https://issues.guix.gnu.org/67048#3> for a patch proposal.


Some details: When invoking “guix refresh” (guix scripts refresh):

 + It calls the procedure ’update-package’

  + which calls ’update-package-source’; see guix/upstream.scm

    + which calls ’package-field-location’; see guix/package.scm

     (match (search-path %load-path file)
       ((? string? file-found)
[...]
       (#f
        ;; FILE could not be found in %LOAD-PATH.
        #f)))

  -> Therefore in ’update-package-source’, the variable ’version-loc’ is
     false:

            (version-loc (package-field-location package 'version)))
        (if version-loc

  + Once fixed, later in ’update-package-source’, it reads,

               (file        (and=> (location-file loc)
                                   (cut search-path %load-path <>))))
          (if file

  -> Therefore, the variable ’file’ is false.

     + Once fixed, later in ’update-package-source’ it calls
     ’absolute-location’; see guix/diagnostics.scm.

            (if (string-prefix? "/" (location-file loc))
[...]
                (match (search-path %load-path (location-file loc))
                  (#f
                   (raise (formatted-message



Well, I do not know how to do better than
<https://issues.guix.gnu.org/67048#3>.

WDYT?

Cheers,
simon


1: bug#67048: guix refresh -u -L does not work with relative path
Simon Tournier <zimon.toutoune@gmail.com>
Mon, 13 Nov 2023 19:28:25 +0100
id:8734x9fqye.fsf@gmail.com
https://issues.guix.gnu.org/67048
https://issues.guix.gnu.org/msgid/8734x9fqye.fsf@gmail.com
https://yhetil.org/guix/8734x9fqye.fsf@gmail.com