[bug#41219] [PATCH 2/2] guix: Enforce package.json "files" directive.

[paul <goodoldpaul@autistici.org>]

[-- Attachment #1: Type: text/plain, Size: 4665 bytes --]

Hi Jelle,

On 9/20/20 9:51 PM, Jelle Licht wrote:
> Hey Giacomo,
>
> Apologies for the delay! Better late than never, a review just for you.
No problem really, I spent some time AFK this summer and didn't ping 
soon enough.
> Perhaps 'pattern-list'? I keep reading this as patron-list. We could
> also build the patterns here. Mapping over the pattern-list + 'default
> patterns' here might also be a wee bit faster.
Yeah I actually don't know why I avoided to type two more letters in the 
first place. I didn't build the patterns here because that would have 
required storing the match result in a separate variable binding and 
requiring to check twice if the binding was false (which is the way I 
went in the new patch. The only slight downside in the new patch is that 
if the match result is #f then patterns is #<unspecified> but is also 
provably never accessed. If you can think of a better way to solve this, 
please do tell me), but mapping first is still more efficient, so I 
changed it.
>
>> +                     (#f #f)))
>> +         (main (match (assoc-ref data "main")
>> +                     ("" #f)
>> +                     ((? string? main-module) main-module)
>> +                     (#f #f)))
>> +         (install-dir (string-append target "/node_modules/" modulename))
>> +         (install-files (lambda (files directory)
>                                            ^
> You only use install-dir here: you could hard-code it in the lambda.
Definitely, I just fixed that.
>
>> +                          (for-each (lambda (file)
>> +                                      (install-file
>> +                                       file
>> +                                       (string-append directory "/"
>> +                                                      (dirname file))))
>> +                                    files))))
>>       (mkdir-p target)
>> -    (copy-recursively "." (string-append target "/node_modules/" modulename))
>> -    ;; Remove references to dependencies
>> -    (delete-file-recursively
>> -      (string-append target "/node_modules/" modulename "/node_modules"))
>> +    (if patterns
>> +        (install-files
>> +         (filter (lambda (file)
>> +                   (any (lambda (pattern)
>> +                          (glob-match?
>> +                           (string->compiled-sglob pattern)
>> +                           file))
>> +                        (append
>> +                         patterns
>> +                         '("package.json"
>> +                           ;; These files get installed no
>> +                           ;; matter the case or extension.
>> +                           "[rR][eE][aA][dD][mM][eE]*"
>> +                           "[cC][hH][aA][nN][gG][eE][sS]*"
>> +                           "[cC][hH][aA][nN][gG][eE][lL][oO][gG]*"
>> +                           "[hH][iI][sS][tT][oO][rR][yY]*"
>> +                           "[nN][oO][tT][iI][cC][eE]*"))))
>> +                 (map (lambda (path)
>> +                        (string-drop path 2))
>                             ^
>                           If this is meant to drop the "./" prefix, you
>                           should be able to leave it out.
>
>> +                      (find-files ".")))
> `find-files' accepts an optional second argument called PRED, so you can
> do that instead of the earlier 'filter'.
Thanks, I didn't know. Fixed :).
>
>> +         install-dir)
>> +        (begin
>> +          (copy-recursively "." install-dir)
>> +          ;; Remove references to dependencies
>> +          (delete-file-recursively
>> +           (string-append install-dir "/node_modules"))))
>> +    (if (and main
>> +             (not (file-exists?
>> +                   (string-append
>> +                    install-dir "/" (dirname main)))))
>> +        (install-files (list main) install-dir))
>             ^
>
> This should not be needed if we use the 'old' (=non-files) approach of
> installing. Do you think it makes sense to pull it into the previous
> block that only runs on using the 'files' directive?
I put this because also the "main" field from package.json is also 
guaranteed to be installed by NPM, according to 
https://docs.npmjs.com/files/package.json#main . Thus if a developer 
populates the "files" field without including the main file in that 
list, but they do insert it in the "main" field the file should be 
installed. Does it make sense?
> Thanks for you patience, and thanks again for working on this.
>
> HTH,
>
>   - Jelle

Thank you for your patience in reviewing this patch. I'm attaching an 
updated version of the second patch.

Cheers,

Giacomo


[-- Attachment #2: 0002-guix-Enforce-package.json-files-directive.patch --]
[-- Type: text/x-patch, Size: 6369 bytes --]

From 329ad1227ee537a630b3823e8d37db4862e023d5 Mon Sep 17 00:00:00 2001
From: Giacomo Leidi <goodoldpaul@autistici.org>
Date: Mon, 21 Sep 2020 22:18:19 +0200
Subject: [PATCH 2/2] guix: Enforce package.json "files" directive.

This fixes https://issues.guix.gnu.org/40710 by implementing support for the
"files" directive from https://docs.npmjs.com/files/package.json#files .

* guix/build/node-build-system.scm (install): Enforce package.json
"files" directive.
* guix/build-system/node.scm (%node-build-system-modules)
(node-build)[modules]: Add (guix glob).
---
 guix/build-system/node.scm       |  4 +-
 guix/build/node-build-system.scm | 67 +++++++++++++++++++++++++-------
 2 files changed, 57 insertions(+), 14 deletions(-)

diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 05c24c47d5..05bc9f2087 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -42,6 +42,7 @@ registry."
   `((guix build node-build-system)
     (guix build json)
     (guix build union)
+    (guix glob)
     ,@%gnu-build-system-modules)) ;; TODO: Might be not needed
 
 (define (default-node)
@@ -90,7 +91,8 @@ registry."
                      (modules '((guix build node-build-system)
 				(guix build json)
 				(guix build union)
-                                (guix build utils))))
+                                (guix build utils)
+                                (guix glob))))
   "Build SOURCE using NODE and INPUTS."
   (define builder
     `(begin
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 7799f03595..6e11d1c142 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2015 David Thompson <davet@gnu.org>
 ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
+;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -22,6 +23,7 @@
   #:use-module (guix build json)
   #:use-module (guix build union)
   #:use-module (guix build utils)
+  #:use-module (guix glob)
   #:use-module (ice-9 match)
   #:use-module (ice-9 popen)
   #:use-module (ice-9 regex)
@@ -110,18 +112,59 @@ the @file{bin} directory."
 				 (#f #f)))
          (dependencies (match (assoc-ref data "dependencies")
                          (('@ deps ...) deps)
-                         (#f #f))))
+                         (#f #f)))
+         (file-list (match (assoc-ref data "files")
+                      (() #f)
+                      ((? list? pattern-list) pattern-list)
+                      (#f #f)))
+         (patterns
+          (when file-list
+            (map (lambda (pattern)
+                   (string->compiled-sglob pattern))
+                 (append file-list
+                         '("package.json"
+                           ;; These files get installed no
+                           ;; matter the case or extension.
+                           "[rR][eE][aA][dD][mM][eE]*"
+                           "[cC][hH][aA][nN][gG][eE][sS]*"
+                           "[cC][hH][aA][nN][gG][eE][lL][oO][gG]*"
+                           "[hH][iI][sS][tT][oO][rR][yY]*"
+                           "[nN][oO][tT][iI][cC][eE]*")))))
+         (main (match (assoc-ref data "main")
+                 ("" #f)
+                 ((? string? main-module) main-module)
+                 (#f #f)))
+         (install-dir (string-append target "/node_modules/" modulename))
+         (install-files (lambda (files)
+                          (for-each (lambda (file)
+                                      (install-file
+                                       file
+                                       (string-append install-dir "/"
+                                                      (dirname file))))
+                                    files))))
     (mkdir-p target)
-    (copy-recursively "." (string-append target "/node_modules/" modulename))
-    ;; Remove references to dependencies
-    (delete-file-recursively
-      (string-append target "/node_modules/" modulename "/node_modules"))
+    (if file-list
+        (install-files
+         (find-files "." (lambda (file stat)
+                           (any (lambda (pattern)
+                                  (glob-match? pattern
+                                               (string-drop file 2)))
+                                patterns))))
+        (begin
+          (copy-recursively "." install-dir)
+          ;; Remove references to dependencies
+          (delete-file-recursively
+           (string-append install-dir "/node_modules"))))
+    (if (and main
+             (not (file-exists?
+                   (string-append
+                    install-dir "/" (dirname main)))))
+        (install-files (list main)))
     (cond
       ((string? binary-configuration)
        (begin
          (mkdir-p binaries)
-         (symlink (string-append target "/node_modules/" modulename "/"
-				 binary-configuration)
+         (symlink (string-append install-dir "/" binary-configuration)
                   (string-append binaries "/" modulename))))
       ((list? binary-configuration)
        (for-each
@@ -130,21 +173,19 @@ the @file{bin} directory."
              ((key . value)
               (begin
                 (mkdir-p (dirname (string-append binaries "/" key)))
-                (symlink (string-append target "/node_modules/" modulename "/"
-					value)
+                (symlink (string-append install-dir "/" value)
                          (string-append binaries "/" key))))))
-         binary-configuration)))
+        binary-configuration)))
     (when dependencies
       (mkdir-p
-        (string-append target "/node_modules/" modulename "/node_modules"))
+        (string-append install-dir "/node_modules"))
       (for-each
         (lambda (dependency)
           (let ((dependency (car dependency)))
             (symlink
               (string-append (assoc-ref inputs (string-append "node-" dependency))
                              "/lib/node_modules/" dependency)
-              (string-append target "/node_modules/" modulename
-                             "/node_modules/" dependency))))
+              (string-append install-dir "/node_modules/" dependency))))
         dependencies))
     #t))
 
-- 
2.28.0