unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: Christopher Baines <mail@cbaines.net>
To: 45409@debbugs.gnu.org
Subject: [bug#45409] [PATCH v3 2/3] guix: Move narinfo code from substitute script to module.
Date: Mon,  4 Jan 2021 21:19:26 +0000	[thread overview]
Message-ID: <20210104211927.14959-2-mail@cbaines.net> (raw)
In-Reply-To: <20210104211927.14959-1-mail@cbaines.net>

This separation between the code for dealing with narinfos from the code doing
that for a purpose should make things clearer, and better support components
other that the substitute script in using this code.

This is just moving the code around, no code should have been significantly
changed.

* guix/scripts/substitute.scm (<narinfo>): Move record type to (guix narinfo).
(fields->alist, narinfo-hash-algorithm+value, narinfo-hash->sha256,
narinfo-signature->canonical-sexp, narinfo-maker, read-narinfo,
narinfo-sha256, valid-narinfo?, write-narinfo, narinfo->string,
string->narinfo, equivalent-narinfo?, supported-compression?,
compresses-better?, narinfo-best-uri): Move procedures to (guix narinfo).
(%compression-methods): Move variable to (guix narinfo).
* guix/narinfo.scm: New file.
* Makefile.am (MODULES): Add it.
* po/guix/POTFILES.in: Add 'guix/narinfo.scm'.
---
 Makefile.am                 |   1 +
 guix/narinfo.scm            | 324 ++++++++++++++++++++++++++++++++++++
 guix/scripts/challenge.scm  |   1 +
 guix/scripts/substitute.scm | 281 +------------------------------
 guix/scripts/weather.scm    |   1 +
 po/guix/POTFILES.in         |   1 +
 tests/challenge.scm         |   2 +-
 tests/substitute.scm        |   1 +
 8 files changed, 332 insertions(+), 280 deletions(-)
 create mode 100644 guix/narinfo.scm

diff --git a/Makefile.am b/Makefile.am
index aec2bb1474..69166a2ea1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -103,6 +103,7 @@ MODULES =					\
   guix/profiles.scm				\
   guix/serialization.scm			\
   guix/nar.scm					\
+  guix/narinfo.scm				\
   guix/derivations.scm				\
   guix/grafts.scm				\
   guix/repl.scm					\
diff --git a/guix/narinfo.scm b/guix/narinfo.scm
new file mode 100644
index 0000000000..5965758bff
--- /dev/null
+++ b/guix/narinfo.scm
@@ -0,0 +1,324 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2014 Nikita Karetnikov <nikita@karetnikov.org>
+;;; Copyright © 2018 Kyle Meyer <kyle@kyleam.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix narinfo)
+  #:use-module (guix pki)
+  #:use-module (guix i18n)
+  #:use-module (guix base32)
+  #:use-module (guix base64)
+  #:use-module (guix records)
+  #:use-module (guix diagnostics)
+  #:use-module (guix scripts substitute)
+  #:use-module (gcrypt hash)
+  #:use-module (gcrypt pk-crypto)
+  #:use-module (rnrs bytevectors)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-9)
+  #:use-module (srfi srfi-26)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 binary-ports)
+  #:use-module (web uri)
+  #:export (narinfo-signature->canonical-sexp
+
+            narinfo?
+            narinfo-path
+            narinfo-uris
+            narinfo-uri-base
+            narinfo-compressions
+            narinfo-file-hashes
+            narinfo-file-sizes
+            narinfo-hash
+            narinfo-size
+            narinfo-references
+            narinfo-deriver
+            narinfo-system
+            narinfo-signature
+
+            narinfo-hash-algorithm+value
+
+            narinfo-hash->sha256
+            narinfo-best-uri
+
+            valid-narinfo?
+
+            read-narinfo
+            write-narinfo
+
+            string->narinfo
+            narinfo->string
+
+            equivalent-narinfo?))
+
+(define-record-type <narinfo>
+  (%make-narinfo path uri-base uris compressions file-sizes file-hashes
+                 nar-hash nar-size references deriver system
+                 signature contents)
+  narinfo?
+  (path         narinfo-path)
+  (uri-base     narinfo-uri-base)        ;URI of the cache it originates from
+  (uris         narinfo-uris)            ;list of strings
+  (compressions narinfo-compressions)    ;list of strings
+  (file-sizes   narinfo-file-sizes)      ;list of (integers | #f)
+  (file-hashes  narinfo-file-hashes)
+  (nar-hash     narinfo-hash)
+  (nar-size     narinfo-size)
+  (references   narinfo-references)
+  (deriver      narinfo-deriver)
+  (system       narinfo-system)
+  (signature    narinfo-signature)      ; canonical sexp
+  ;; The original contents of a narinfo file.  This field is needed because we
+  ;; want to preserve the exact textual representation for verification purposes.
+  ;; See <https://lists.gnu.org/archive/html/guix-devel/2014-02/msg00340.html>
+  ;; for more information.
+  (contents     narinfo-contents))
+
+(define (narinfo-hash-algorithm+value narinfo)
+  "Return two values: the hash algorithm used by NARINFO and its value as a
+bytevector."
+  (match (string-tokenize (narinfo-hash narinfo)
+                          (char-set-complement (char-set #\:)))
+    ((algorithm base32)
+     (values (lookup-hash-algorithm (string->symbol algorithm))
+             (nix-base32-string->bytevector base32)))
+    (_
+     (raise (formatted-message
+             (G_ "invalid narinfo hash: ~s") (narinfo-hash narinfo))))))
+
+(define (narinfo-hash->sha256 hash)
+  "If the string HASH denotes a sha256 hash, return it as a bytevector.
+Otherwise return #f."
+  (and (string-prefix? "sha256:" hash)
+       (nix-base32-string->bytevector (string-drop hash 7))))
+
+(define (narinfo-signature->canonical-sexp str)
+  "Return the value of a narinfo's 'Signature' field as a canonical sexp."
+  (match (string-split str #\;)
+    ((version host-name sig)
+     (let ((maybe-number (string->number version)))
+       (cond ((not (number? maybe-number))
+              (leave (G_ "signature version must be a number: ~s~%")
+                     version))
+             ;; Currently, there are no other versions.
+             ((not (= 1 maybe-number))
+              (leave (G_ "unsupported signature version: ~a~%")
+                     maybe-number))
+             (else
+              (let ((signature (utf8->string (base64-decode sig))))
+                (catch 'gcry-error
+                  (lambda ()
+                    (string->canonical-sexp signature))
+                  (lambda (key proc err)
+                    (leave (G_ "signature is not a valid \
+s-expression: ~s~%")
+                           signature))))))))
+    (x
+     (leave (G_ "invalid format of the signature field: ~a~%") x))))
+
+(define (narinfo-maker str cache-url)
+  "Return a narinfo constructor for narinfos originating from CACHE-URL.  STR
+must contain the original contents of a narinfo file."
+  (lambda (path urls compressions file-hashes file-sizes
+                nar-hash nar-size references deriver system
+                signature)
+    "Return a new <narinfo> object."
+    (define len (length urls))
+    (%make-narinfo path cache-url
+                   ;; Handle the case where URL is a relative URL.
+                   (map (lambda (url)
+                          (or (string->uri url)
+                              (string->uri
+                               (string-append cache-url "/" url))))
+                        urls)
+                   compressions
+                   (match file-sizes
+                     (()        (make-list len #f))
+                     ((lst ...) (map string->number lst)))
+                   (match file-hashes
+                     (()        (make-list len #f))
+                     ((lst ...) (map string->number lst)))
+                   nar-hash
+                   (and=> nar-size string->number)
+                   (string-tokenize references)
+                   (match deriver
+                     ((or #f "") #f)
+                     (_ deriver))
+                   system
+                   (false-if-exception
+                    (and=> signature narinfo-signature->canonical-sexp))
+                   str)))
+
+(define fields->alist
+  ;; The narinfo format is really just like recutils.
+  recutils->alist)
+
+(define* (read-narinfo port #:optional url
+                       #:key size)
+  "Read a narinfo from PORT.  If URL is true, it must be a string used to
+build full URIs from relative URIs found while reading PORT.  When SIZE is
+true, read at most SIZE bytes from PORT; otherwise, read as much as possible.
+
+No authentication and authorization checks are performed here!"
+  (let ((str (utf8->string (if size
+                               (get-bytevector-n port size)
+                               (get-bytevector-all port)))))
+    (alist->record (call-with-input-string str fields->alist)
+                   (narinfo-maker str url)
+                   '("StorePath" "URL" "Compression"
+                     "FileHash" "FileSize" "NarHash" "NarSize"
+                     "References" "Deriver" "System"
+                     "Signature")
+                   '("URL" "Compression" "FileSize" "FileHash"))))
+
+(define (narinfo-sha256 narinfo)
+  "Return the sha256 hash of NARINFO as a bytevector, or #f if NARINFO lacks a
+'Signature' field."
+  (define %mandatory-fields
+    ;; List of fields that must be signed.  If they are not signed, the
+    ;; narinfo is considered unsigned.
+    '("StorePath" "NarHash" "References"))
+
+  (let ((contents (narinfo-contents narinfo)))
+    (match (string-contains contents "Signature:")
+      (#f #f)
+      (index
+       (let* ((above-signature (string-take contents index))
+              (signed-fields (match (call-with-input-string above-signature
+                                      fields->alist)
+                               (((fields . values) ...) fields))))
+         (and (every (cut member <> signed-fields) %mandatory-fields)
+              (sha256 (string->utf8 above-signature))))))))
+
+(define* (valid-narinfo? narinfo #:optional (acl (current-acl))
+                         #:key verbose?)
+  "Return #t if NARINFO's signature is not valid."
+  (let ((hash      (narinfo-sha256 narinfo))
+        (signature (narinfo-signature narinfo))
+        (uri       (uri->string (first (narinfo-uris narinfo)))))
+    (and hash signature
+         (signature-case (signature hash acl)
+           (valid-signature #t)
+           (invalid-signature
+            (when verbose?
+              (format (current-error-port)
+                      "invalid signature for substitute at '~a'~%"
+                      uri))
+            #f)
+           (hash-mismatch
+            (when verbose?
+              (format (current-error-port)
+                      "hash mismatch for substitute at '~a'~%"
+                      uri))
+            #f)
+           (unauthorized-key
+            (when verbose?
+              (format (current-error-port)
+                      "substitute at '~a' is signed by an \
+unauthorized party~%"
+                      uri))
+            #f)
+           (corrupt-signature
+            (when verbose?
+              (format (current-error-port)
+                      "corrupt signature for substitute at '~a'~%"
+                      uri))
+            #f)))))
+
+(define (write-narinfo narinfo port)
+  "Write NARINFO to PORT."
+  (put-bytevector port (string->utf8 (narinfo-contents narinfo))))
+
+(define (narinfo->string narinfo)
+  "Return the external representation of NARINFO."
+  (call-with-output-string (cut write-narinfo narinfo <>)))
+
+(define (string->narinfo str cache-uri)
+  "Return the narinfo represented by STR.  Assume CACHE-URI as the base URI of
+the cache STR originates form."
+  (call-with-input-string str (cut read-narinfo <> cache-uri)))
+
+(define (equivalent-narinfo? narinfo1 narinfo2)
+  "Return true if NARINFO1 and NARINFO2 are equivalent--i.e., if they describe
+the same store item.  This ignores unnecessary metadata such as the Nar URL."
+  (and (string=? (narinfo-hash narinfo1)
+                 (narinfo-hash narinfo2))
+
+       ;; The following is not needed if all we want is to download a valid
+       ;; nar, but it's necessary if we want valid narinfo.
+       (string=? (narinfo-path narinfo1)
+                 (narinfo-path narinfo2))
+       (equal? (narinfo-references narinfo1)
+               (narinfo-references narinfo2))
+
+       (= (narinfo-size narinfo1)
+          (narinfo-size narinfo2))))
+
+(define %compression-methods
+  ;; Known compression methods and a thunk to determine whether they're
+  ;; supported.  See 'decompressed-port' in (guix utils).
+  `(("gzip"  . ,(const #t))
+    ("lzip"  . ,(const #t))
+    ("xz"    . ,(const #t))
+    ("bzip2" . ,(const #t))
+    ("none"  . ,(const #t))))
+
+(define (supported-compression? compression)
+  "Return true if COMPRESSION, a string, denotes a supported compression
+method."
+  (match (assoc-ref %compression-methods compression)
+    (#f         #f)
+    (supported? (supported?))))
+
+(define (compresses-better? compression1 compression2)
+  "Return true if COMPRESSION1 generally compresses better than COMPRESSION2;
+this is a rough approximation."
+  (match compression1
+    ("none" #f)
+    ("gzip" (string=? compression2 "none"))
+    (_      (or (string=? compression2 "none")
+                (string=? compression2 "gzip")))))
+
+(define (narinfo-best-uri narinfo)
+  "Select the \"best\" URI to download NARINFO's nar, and return three values:
+the URI, its compression method (a string), and the compressed file size."
+  (define choices
+    (filter (match-lambda
+              ((uri compression file-size)
+               (supported-compression? compression)))
+            (zip (narinfo-uris narinfo)
+                 (narinfo-compressions narinfo)
+                 (narinfo-file-sizes narinfo))))
+
+  (define (file-size<? c1 c2)
+    (match c1
+      ((uri1 compression1 (? integer? file-size1))
+       (match c2
+         ((uri2 compression2 (? integer? file-size2))
+          (< file-size1 file-size2))
+         (_ #t)))
+      ((uri compression1 #f)
+       (match c2
+         ((uri2 compression2 _)
+          (compresses-better? compression1 compression2))))
+      (_ #f)))                                    ;we can't tell
+
+  (match (sort choices file-size<?)
+    (((uri compression file-size) _ ...)
+     (values uri compression file-size))))
+
diff --git a/guix/scripts/challenge.scm b/guix/scripts/challenge.scm
index d0a456ac1d..cc9cbe6f27 100644
--- a/guix/scripts/challenge.scm
+++ b/guix/scripts/challenge.scm
@@ -28,6 +28,7 @@
   #:use-module ((guix progress) #:hide (dump-port*))
   #:use-module (guix serialization)
   #:use-module (guix scripts substitute)
+  #:use-module (guix narinfo)
   #:use-module (rnrs bytevectors)
   #:autoload   (guix http-client) (http-fetch)
   #:use-module ((guix build syscalls) #:select (terminal-columns))
diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
index 14fb848880..f9bcead045 100755
--- a/guix/scripts/substitute.scm
+++ b/guix/scripts/substitute.scm
@@ -22,6 +22,7 @@
 (define-module (guix scripts substitute)
   #:use-module (guix ui)
   #:use-module (guix scripts)
+  #:use-module (guix narinfo)
   #:use-module (guix store)
   #:use-module (guix utils)
   #:use-module (guix combinators)
@@ -68,29 +69,8 @@
   #:use-module (web request)
   #:use-module (web response)
   #:use-module (guix http-client)
-  #:export (narinfo-signature->canonical-sexp
-
-            narinfo?
-            narinfo-path
-            narinfo-uris
-            narinfo-uri-base
-            narinfo-compressions
-            narinfo-file-hashes
-            narinfo-file-sizes
-            narinfo-hash
-            narinfo-size
-            narinfo-references
-            narinfo-deriver
-            narinfo-system
-            narinfo-signature
-
-            narinfo-hash->sha256
-            narinfo-best-uri
-
-            lookup-narinfos
+  #:export (lookup-narinfos
             lookup-narinfos/diverse
-            read-narinfo
-            write-narinfo
 
             %allow-unauthenticated-substitutes?
             %error-to-file-descriptor-4?
@@ -150,10 +130,6 @@ disabled!~%"))
   ;; How often we want to remove files corresponding to expired cache entries.
   (* 7 24 3600))
 
-(define fields->alist
-  ;; The narinfo format is really just like recutils.
-  recutils->alist)
-
 (define %fetch-timeout
   ;; Number of seconds after which networking is considered "slow".
   5)
@@ -237,190 +213,6 @@ connection (typically PORT) is kept open once data has been fetched from URI."
      (leave (G_ "unsupported substitute URI scheme: ~a~%")
             (uri->string uri)))))
 
-\f
-(define-record-type <narinfo>
-  (%make-narinfo path uri-base uris compressions file-sizes file-hashes
-                 nar-hash nar-size references deriver system
-                 signature contents)
-  narinfo?
-  (path         narinfo-path)
-  (uri-base     narinfo-uri-base)        ;URI of the cache it originates from
-  (uris         narinfo-uris)            ;list of strings
-  (compressions narinfo-compressions)    ;list of strings
-  (file-sizes   narinfo-file-sizes)      ;list of (integers | #f)
-  (file-hashes  narinfo-file-hashes)
-  (nar-hash     narinfo-hash)
-  (nar-size     narinfo-size)
-  (references   narinfo-references)
-  (deriver      narinfo-deriver)
-  (system       narinfo-system)
-  (signature    narinfo-signature)      ; canonical sexp
-  ;; The original contents of a narinfo file.  This field is needed because we
-  ;; want to preserve the exact textual representation for verification purposes.
-  ;; See <https://lists.gnu.org/archive/html/guix-devel/2014-02/msg00340.html>
-  ;; for more information.
-  (contents     narinfo-contents))
-
-(define (narinfo-hash-algorithm+value narinfo)
-  "Return two values: the hash algorithm used by NARINFO and its value as a
-bytevector."
-  (match (string-tokenize (narinfo-hash narinfo)
-                          (char-set-complement (char-set #\:)))
-    ((algorithm base32)
-     (values (lookup-hash-algorithm (string->symbol algorithm))
-             (nix-base32-string->bytevector base32)))
-    (_
-     (raise (formatted-message
-             (G_ "invalid narinfo hash: ~s") (narinfo-hash narinfo))))))
-
-(define (narinfo-hash->sha256 hash)
-  "If the string HASH denotes a sha256 hash, return it as a bytevector.
-Otherwise return #f."
-  (and (string-prefix? "sha256:" hash)
-       (nix-base32-string->bytevector (string-drop hash 7))))
-
-(define (narinfo-signature->canonical-sexp str)
-  "Return the value of a narinfo's 'Signature' field as a canonical sexp."
-  (match (string-split str #\;)
-    ((version host-name sig)
-     (let ((maybe-number (string->number version)))
-       (cond ((not (number? maybe-number))
-              (leave (G_ "signature version must be a number: ~s~%")
-                     version))
-             ;; Currently, there are no other versions.
-             ((not (= 1 maybe-number))
-              (leave (G_ "unsupported signature version: ~a~%")
-                     maybe-number))
-             (else
-              (let ((signature (utf8->string (base64-decode sig))))
-                (catch 'gcry-error
-                  (lambda ()
-                    (string->canonical-sexp signature))
-                  (lambda (key proc err)
-                    (leave (G_ "signature is not a valid \
-s-expression: ~s~%")
-                           signature))))))))
-    (x
-     (leave (G_ "invalid format of the signature field: ~a~%") x))))
-
-(define (narinfo-maker str cache-url)
-  "Return a narinfo constructor for narinfos originating from CACHE-URL.  STR
-must contain the original contents of a narinfo file."
-  (lambda (path urls compressions file-hashes file-sizes
-                nar-hash nar-size references deriver system
-                signature)
-    "Return a new <narinfo> object."
-    (define len (length urls))
-    (%make-narinfo path cache-url
-                   ;; Handle the case where URL is a relative URL.
-                   (map (lambda (url)
-                          (or (string->uri url)
-                              (string->uri
-                               (string-append cache-url "/" url))))
-                        urls)
-                   compressions
-                   (match file-sizes
-                     (()        (make-list len #f))
-                     ((lst ...) (map string->number lst)))
-                   (match file-hashes
-                     (()        (make-list len #f))
-                     ((lst ...) (map string->number lst)))
-                   nar-hash
-                   (and=> nar-size string->number)
-                   (string-tokenize references)
-                   (match deriver
-                     ((or #f "") #f)
-                     (_ deriver))
-                   system
-                   (false-if-exception
-                    (and=> signature narinfo-signature->canonical-sexp))
-                   str)))
-
-(define* (read-narinfo port #:optional url
-                       #:key size)
-  "Read a narinfo from PORT.  If URL is true, it must be a string used to
-build full URIs from relative URIs found while reading PORT.  When SIZE is
-true, read at most SIZE bytes from PORT; otherwise, read as much as possible.
-
-No authentication and authorization checks are performed here!"
-  (let ((str (utf8->string (if size
-                               (get-bytevector-n port size)
-                               (get-bytevector-all port)))))
-    (alist->record (call-with-input-string str fields->alist)
-                   (narinfo-maker str url)
-                   '("StorePath" "URL" "Compression"
-                     "FileHash" "FileSize" "NarHash" "NarSize"
-                     "References" "Deriver" "System"
-                     "Signature")
-                   '("URL" "Compression" "FileSize" "FileHash"))))
-
-(define (narinfo-sha256 narinfo)
-  "Return the sha256 hash of NARINFO as a bytevector, or #f if NARINFO lacks a
-'Signature' field."
-  (define %mandatory-fields
-    ;; List of fields that must be signed.  If they are not signed, the
-    ;; narinfo is considered unsigned.
-    '("StorePath" "NarHash" "References"))
-
-  (let ((contents (narinfo-contents narinfo)))
-    (match (string-contains contents "Signature:")
-      (#f #f)
-      (index
-       (let* ((above-signature (string-take contents index))
-              (signed-fields (match (call-with-input-string above-signature
-                                      fields->alist)
-                               (((fields . values) ...) fields))))
-         (and (every (cut member <> signed-fields) %mandatory-fields)
-              (sha256 (string->utf8 above-signature))))))))
-
-(define* (valid-narinfo? narinfo #:optional (acl (current-acl))
-                         #:key verbose?)
-  "Return #t if NARINFO's signature is not valid."
-  (let ((hash      (narinfo-sha256 narinfo))
-        (signature (narinfo-signature narinfo))
-        (uri       (uri->string (first (narinfo-uris narinfo)))))
-    (and hash signature
-         (signature-case (signature hash acl)
-           (valid-signature #t)
-           (invalid-signature
-            (when verbose?
-              (format (current-error-port)
-                      "invalid signature for substitute at '~a'~%"
-                      uri))
-            #f)
-           (hash-mismatch
-            (when verbose?
-              (format (current-error-port)
-                      "hash mismatch for substitute at '~a'~%"
-                      uri))
-            #f)
-           (unauthorized-key
-            (when verbose?
-              (format (current-error-port)
-                      "substitute at '~a' is signed by an \
-unauthorized party~%"
-                      uri))
-            #f)
-           (corrupt-signature
-            (when verbose?
-              (format (current-error-port)
-                      "corrupt signature for substitute at '~a'~%"
-                      uri))
-            #f)))))
-
-(define (write-narinfo narinfo port)
-  "Write NARINFO to PORT."
-  (put-bytevector port (string->utf8 (narinfo-contents narinfo))))
-
-(define (narinfo->string narinfo)
-  "Return the external representation of NARINFO."
-  (call-with-output-string (cut write-narinfo narinfo <>)))
-
-(define (string->narinfo str cache-uri)
-  "Return the narinfo represented by STR.  Assume CACHE-URI as the base URI of
-the cache STR originates form."
-  (call-with-input-string str (cut read-narinfo <> cache-uri)))
-
 (define (narinfo-cache-file cache-url path)
   "Return the name of the local file that contains an entry for PATH.  The
 entry is stored in a sub-directory specific to CACHE-URL."
@@ -742,22 +534,6 @@ information is available locally."
         (let ((missing (fetch-narinfos cache missing)))
           (append cached (or missing '()))))))
 
-(define (equivalent-narinfo? narinfo1 narinfo2)
-  "Return true if NARINFO1 and NARINFO2 are equivalent--i.e., if they describe
-the same store item.  This ignores unnecessary metadata such as the Nar URL."
-  (and (string=? (narinfo-hash narinfo1)
-                 (narinfo-hash narinfo2))
-
-       ;; The following is not needed if all we want is to download a valid
-       ;; nar, but it's necessary if we want valid narinfo.
-       (string=? (narinfo-path narinfo1)
-                 (narinfo-path narinfo2))
-       (equal? (narinfo-references narinfo1)
-               (narinfo-references narinfo2))
-
-       (= (narinfo-size narinfo1)
-          (narinfo-size narinfo2))))
-
 (define (lookup-narinfos/diverse caches paths authorized?)
   "Look up narinfos for PATHS on all of CACHES, a list of URLS, in that order.
 That is, when a cache lacks an AUTHORIZED? narinfo, look it up in the next
@@ -943,59 +719,6 @@ authorized substitutes."
     (wtf
      (error "unknown `--query' command" wtf))))
 
-(define %compression-methods
-  ;; Known compression methods and a thunk to determine whether they're
-  ;; supported.  See 'decompressed-port' in (guix utils).
-  `(("gzip"  . ,(const #t))
-    ("lzip"  . ,(const #t))
-    ("xz"    . ,(const #t))
-    ("bzip2" . ,(const #t))
-    ("none"  . ,(const #t))))
-
-(define (supported-compression? compression)
-  "Return true if COMPRESSION, a string, denotes a supported compression
-method."
-  (match (assoc-ref %compression-methods compression)
-    (#f         #f)
-    (supported? (supported?))))
-
-(define (compresses-better? compression1 compression2)
-  "Return true if COMPRESSION1 generally compresses better than COMPRESSION2;
-this is a rough approximation."
-  (match compression1
-    ("none" #f)
-    ("gzip" (string=? compression2 "none"))
-    (_      (or (string=? compression2 "none")
-                (string=? compression2 "gzip")))))
-
-(define (narinfo-best-uri narinfo)
-  "Select the \"best\" URI to download NARINFO's nar, and return three values:
-the URI, its compression method (a string), and the compressed file size."
-  (define choices
-    (filter (match-lambda
-              ((uri compression file-size)
-               (supported-compression? compression)))
-            (zip (narinfo-uris narinfo)
-                 (narinfo-compressions narinfo)
-                 (narinfo-file-sizes narinfo))))
-
-  (define (file-size<? c1 c2)
-    (match c1
-      ((uri1 compression1 (? integer? file-size1))
-       (match c2
-         ((uri2 compression2 (? integer? file-size2))
-          (< file-size1 file-size2))
-         (_ #t)))
-      ((uri compression1 #f)
-       (match c2
-         ((uri2 compression2 _)
-          (compresses-better? compression1 compression2))))
-      (_ #f)))                                    ;we can't tell
-
-  (match (sort choices file-size<?)
-    (((uri compression file-size) _ ...)
-     (values uri compression file-size))))
-
 (define %max-cached-connections
   ;; Maximum number of connections kept in cache by
   ;; 'open-connection-for-uri/cached'.
diff --git a/guix/scripts/weather.scm b/guix/scripts/weather.scm
index f28070ddc4..97e4a73802 100644
--- a/guix/scripts/weather.scm
+++ b/guix/scripts/weather.scm
@@ -33,6 +33,7 @@
   #:use-module ((guix build syscalls) #:select (terminal-columns))
   #:use-module ((guix build utils) #:select (every*))
   #:use-module (guix scripts substitute)
+  #:use-module (guix narinfo)
   #:use-module (guix http-client)
   #:use-module (guix ci)
   #:use-module (guix sets)
diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in
index 1aec3bef3c..666e630adf 100644
--- a/po/guix/POTFILES.in
+++ b/po/guix/POTFILES.in
@@ -87,6 +87,7 @@ guix/ui.scm
 guix/status.scm
 guix/http-client.scm
 guix/nar.scm
+guix/narinfo.scm
 guix/channels.scm
 guix/profiles.scm
 guix/git.scm
diff --git a/tests/challenge.scm b/tests/challenge.scm
index 9c6d6e0d58..fdd5fd238e 100644
--- a/tests/challenge.scm
+++ b/tests/challenge.scm
@@ -27,8 +27,8 @@
   #:use-module (guix packages)
   #:use-module (guix gexp)
   #:use-module (guix base32)
+  #:use-module (guix narinfo)
   #:use-module (guix scripts challenge)
-  #:use-module (guix scripts substitute)
   #:use-module ((guix build utils) #:select (find-files))
   #:use-module (gnu packages bootstrap)
   #:use-module (srfi srfi-1)
diff --git a/tests/substitute.scm b/tests/substitute.scm
index 542aaf603f..697abc4684 100644
--- a/tests/substitute.scm
+++ b/tests/substitute.scm
@@ -19,6 +19,7 @@
 
 (define-module (test-substitute)
   #:use-module (guix scripts substitute)
+  #:use-module (guix narinfo)
   #:use-module (guix base64)
   #:use-module (gcrypt hash)
   #:use-module (guix serialization)
-- 
2.29.2





  reply	other threads:[~2021-01-04 21:20 UTC|newest]

Thread overview: 56+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-24 17:17 [bug#45409] [PATCH 0/3] Move some (guix scripts substitute) code to two new modules Christopher Baines
2020-12-24 17:22 ` [bug#45409] [PATCH 1/3] guix: Move narinfo code from substitute script to module Christopher Baines
2020-12-24 17:22   ` [bug#45409] [PATCH 2/3] guix: Untangle (guix narinfo) from (guix scripts substitute) Christopher Baines
2020-12-24 17:22   ` [bug#45409] [PATCH 3/3] guix: Split (guix substitute) " Christopher Baines
2021-01-03 15:08     ` Ludovic Courtès
2021-01-03 18:19       ` Christopher Baines
2021-01-03 15:03   ` [bug#45409] [PATCH 1/3] guix: Move narinfo code from substitute script to module Ludovic Courtès
2021-01-03 18:16     ` Christopher Baines
2021-01-04 21:24       ` Christopher Baines
2021-01-03 17:59 ` [bug#45409] [PATCH v2 1/3] substitute: Untangle skipping authentication from valid-narinfo? Christopher Baines
2021-01-03 17:59   ` [bug#45409] [PATCH v2 2/3] guix: Move narinfo code from substitute script to module Christopher Baines
2021-01-03 17:59   ` [bug#45409] [PATCH v2 3/3] guix: Split (guix substitutes) from (guix scripts substitute) Christopher Baines
2021-01-04 21:19 ` [bug#45409] [PATCH v3 1/3] substitute: Untangle skipping authentication from valid-narinfo? Christopher Baines
2021-01-04 21:19   ` Christopher Baines [this message]
2021-01-05 21:58     ` [bug#45409] [PATCH v3 2/3] guix: Move narinfo code from substitute script to module Ludovic Courtès
2021-01-04 21:19   ` [bug#45409] [PATCH v3 3/3] guix: Split (guix substitutes) from (guix scripts substitute) Christopher Baines
2021-01-05 22:03     ` Ludovic Courtès
2021-01-07 22:29       ` Christopher Baines
2021-01-11 13:26         ` Ludovic Courtès
2021-01-16 14:18           ` Christopher Baines
2021-02-13 13:56             ` Christopher Baines
2021-02-22 22:21               ` Christopher Baines
2021-02-23 20:46                 ` Christopher Baines
2021-01-05 21:57   ` [bug#45409] [PATCH v3 1/3] substitute: Untangle skipping authentication from valid-narinfo? Ludovic Courtès
2021-01-05 22:58     ` Christopher Baines
2021-01-06  8:37       ` Ludovic Courtès
2021-01-16 13:57 ` [bug#45409] [PATCH v4 01/13] substitute: Remove buffer handling from fetch Christopher Baines
2021-01-16 13:57   ` [bug#45409] [PATCH v4 02/13] substitute: Remove connection " Christopher Baines
2021-01-16 13:57   ` [bug#45409] [PATCH v4 03/13] substitute: Remove redundant let block " Christopher Baines
2021-01-16 13:57   ` [bug#45409] [PATCH v4 04/13] guix: Move http-multiple-get to (guix http-client) Christopher Baines
2021-01-16 13:57   ` [bug#45409] [PATCH v4 05/13] http-client: Add error handling to http-multiple-get Christopher Baines
2021-01-16 13:57   ` [bug#45409] [PATCH v4 06/13] substitute: open-connection-for-uri/maybe add #:verify-certificate? Christopher Baines
2021-01-16 13:57   ` [bug#45409] [PATCH v4 07/13] substitute: Stop using call-with-cached-connection in fetch-narinfos Christopher Baines
2021-01-16 13:57   ` [bug#45409] [PATCH v4 08/13] http-client: Accept #:open-connection in http-fetch Christopher Baines
2021-01-16 13:57   ` [bug#45409] [PATCH v4 09/13] substitute: Change connection cache handling in process-substitution Christopher Baines
2021-01-16 13:58   ` [bug#45409] [PATCH v4 10/13] substitute: Remove now redundant connection caching helpers Christopher Baines
2021-01-16 13:58   ` [bug#45409] [PATCH v4 11/13] substitute: Remove redundant fetch arguments Christopher Baines
2021-01-16 13:58   ` [bug#45409] [PATCH v4 12/13] substitute: Inline fetch in to process-substitutes Christopher Baines
2021-01-16 13:58   ` [bug#45409] [PATCH v4 13/13] substitute: Remove fetch-narinfos use open-connection-for-uri/maybe Christopher Baines
2021-02-13 13:47 ` [bug#45409] [PATCH v5 01/14] substitute: Remove buffer handling from fetch Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 02/14] substitute: Remove connection " Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 03/14] substitute: Remove redundant let block " Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 04/14] guix: Move http-multiple-get to (guix http-client) Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 05/14] http-client: Add error handling to http-multiple-get Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 06/14] substitute: open-connection-for-uri/maybe add #:verify-certificate? Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 07/14] substitute: Stop using call-with-cached-connection in fetch-narinfos Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 08/14] http-client: Accept #:open-connection in http-fetch Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 09/14] substitute: Change connection cache handling in process-substitution Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 10/14] substitute: Remove now redundant connection caching helpers Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 11/14] substitute: Remove redundant fetch arguments Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 12/14] substitute: Inline fetch in to process-substitutes Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 13/14] substitute: Remove fetch-narinfos use open-connection-for-uri/maybe Christopher Baines
2021-02-13 13:47   ` [bug#45409] [PATCH v5 14/14] substitute: Rework connection error handling Christopher Baines
2021-02-23 19:59 ` [bug#45409] [PATCH 1/2] guix: Split (guix substitutes) from (guix scripts substitute) Christopher Baines
2021-02-23 19:59   ` [bug#45409] [PATCH 2/2] substitute: Print backtraces to (current-error-port) Christopher Baines
2021-03-06  0:57 ` bug#45409: [PATCH 0/3] Move some (guix scripts substitute) code to two new modules Christopher Baines

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210104211927.14959-2-mail@cbaines.net \
    --to=mail@cbaines.net \
    --cc=45409@debbugs.gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).