unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: Maxime Devos <maximedevos@telenet.be>
To: 50072@debbugs.gnu.org
Cc: Sarah Morgensen <iskarian@mgsn.dev>,
	Maxime Devos <maximedevos@telenet.be>
Subject: [bug#50072] [PATCH v5 1/4] guix hash: Extract file hashing procedures.
Date: Wed,  5 Jan 2022 14:07:47 +0000	[thread overview]
Message-ID: <20220105140750.18214-1-maximedevos@telenet.be> (raw)
In-Reply-To: <cover.1629068119.git.iskarian@mgsn.dev>

From: Sarah Morgensen <iskarian@mgsn.dev>

* guix/scripts/hash.scm (guix-hash)[vcs-file?] (nar-hash, default-hash):
  Extract hashing logic to...
* guix/hash.scm (vcs-file?, file-hash*): ... these new procedures in this
  new file.

Modified-by: Maxime Devos <maximedevos@telenet.be>
---
 Makefile.am           |  1 +
 guix/hash.scm         | 73 +++++++++++++++++++++++++++++++++++++++++++
 guix/scripts/hash.scm | 22 +++----------
 3 files changed, 78 insertions(+), 18 deletions(-)
 create mode 100644 guix/hash.scm

diff --git a/Makefile.am b/Makefile.am
index 8c5682a1c6..bc3d0087d0 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -99,6 +99,7 @@ MODULES =					\
   guix/extracting-download.scm			\
   guix/git-download.scm				\
   guix/hg-download.scm				\
+  guix/hash.scm					\
   guix/swh.scm					\
   guix/monads.scm				\
   guix/monad-repl.scm				\
diff --git a/guix/hash.scm b/guix/hash.scm
new file mode 100644
index 0000000000..3cb68e5c44
--- /dev/null
+++ b/guix/hash.scm
@@ -0,0 +1,73 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
+;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix hash)
+  #:use-module (gcrypt hash)
+  #:use-module (guix serialization)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
+  #:export (vcs-file?
+            file-hash*))
+
+(define (vcs-file? file stat)
+  "Returns true if FILE is a version control system file."
+  (case (stat:type stat)
+    ((directory)
+     (member (basename file) '(".bzr" ".git" ".hg" ".svn" "CVS")))
+    ((regular)
+     ;; Git sub-modules have a '.git' file that is a regular text file.
+     (string=? (basename file) ".git"))
+    (else
+     #f)))
+
+(define* (file-hash* file #:key
+                     (algorithm (hash-algorithm sha256))
+                     (recursive? 'auto)
+                     (select? (negate vcs-file?)))
+  "Compute the hash of FILE with ALGORITHM.
+
+Symbolic links are only dereferenced if RECURSIVE? is false.
+Directories are only supported if RECURSIVE? is #true or 'auto'.
+The executable bit is only recorded if RECURSIVE? is #true.
+If FILE is a symbolic link, it is only followed if RECURSIVE? is false.
+
+For regular files, there are two different hashes when the executable
+hash isn't recorded: the regular hash and the nar hash. In most situations,
+the regular hash is desired and setting RECURSIVE? to 'auto' does the right
+thing for both regular files and directories.
+
+This procedure must only be used under controlled circumstances;
+the detection of symbolic links in FILE is racy.
+
+When FILE is a directory, the procedure SELECT? called as (SELECT? FILE STAT)
+decides which files to include. By default, version control files are
+excluded. To include everything, SELECT? can be set to (const #true)."
+  (if (or (eq? recursive? #true)
+          (and (eq? recursive? 'auto)
+               ;; Don't change this to (eq? 'directory ...), because otherwise
+               ;; if 'file' denotes a symbolic link, the 'file-hash' below
+               ;; would dereference it -- dereferencing symbolic links would
+               ;; open an avoidable can of potential worms.
+               (not (eq? 'regular (stat:type (lstat file))))))
+      (let-values (((port get-hash)
+                    (open-hash-port algorithm)))
+        (write-file file port #:select? select?)
+        (force-output port)
+        (get-hash))
+      (file-hash algorithm file)))
diff --git a/guix/scripts/hash.scm b/guix/scripts/hash.scm
index d73e3d13dd..28d587b944 100644
--- a/guix/scripts/hash.scm
+++ b/guix/scripts/hash.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2016 Jan Nieuwenhuizen <janneke@gnu.org>
 ;;; Copyright © 2018 Tim Gesthuizen <tim.gesthuizen@yahoo.de>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -24,6 +25,7 @@
   #:use-module (gcrypt hash)
   #:use-module (guix serialization)
   #:use-module (guix ui)
+  #:use-module (guix hash)
   #:use-module (guix scripts)
   #:use-module (guix base16)
   #:use-module (guix base32)
@@ -46,20 +48,14 @@
 (define* (nar-hash file #:optional
                    (algorithm (assoc-ref %default-options 'hash-algorithm))
                    select?)
-  (let-values (((port get-hash)
-                (open-hash-port algorithm)))
-    (write-file file port #:select? select?)
-    (force-output port)
-    (get-hash)))
+  (file-hash* file #:algorithm algorithm #:select? select? #:recursive? #true))
 
 (define* (default-hash file #:optional
                        (algorithm (assoc-ref %default-options 'hash-algorithm))
                        select?)
   (match file
     ("-" (port-hash algorithm (current-input-port)))
-    (_
-     (call-with-input-file file
-       (cute port-hash algorithm <>)))))
+    (_ (file-hash* file #:algorithm algorithm #:recursive? #false))))
 
 (define* (git-hash file #:optional
                        (algorithm (assoc-ref %default-options 'hash-algorithm))
@@ -181,16 +177,6 @@ use '--serializer' instead~%"))
     (parse-command-line args %options (list %default-options)
                         #:build-options? #f))
 
-  (define (vcs-file? file stat)
-    (case (stat:type stat)
-      ((directory)
-       (member (basename file) '(".bzr" ".git" ".hg" ".svn" "CVS")))
-      ((regular)
-       ;; Git sub-modules have a '.git' file that is a regular text file.
-       (string=? (basename file) ".git"))
-      (else
-       #f)))
-
   (let* ((opts (parse-options))
          (args (filter-map (match-lambda
                             (('argument . value)

base-commit: 9708681f1a9f221ae6cad64625ba8309b6742653
-- 
2.30.2





  parent reply	other threads:[~2022-01-05 15:14 UTC|newest]

Thread overview: 66+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-15 23:16 [bug#50072] [PATCH WIP 0/4] Add upstream updater for git-fetch origins Sarah Morgensen
2021-08-15 23:25 ` [bug#50072] [PATCH WIP 1/4] guix hash: Extract file hashing procedures Sarah Morgensen
2021-08-15 23:25 ` [bug#50072] [PATCH WIP 2/4] import: Factorize file hashing Sarah Morgensen
2021-08-15 23:25 ` [bug#50072] [PATCH WIP 3/4] refresh: Support non-tarball sources Sarah Morgensen
2021-08-15 23:25 ` [bug#50072] [PATCH WIP 4/4] upstream: Support updating git-fetch origins Sarah Morgensen
2021-08-16 10:46   ` Maxime Devos
2021-08-16 13:02     ` Xinglu Chen
2021-08-16 18:15       ` Maxime Devos
2021-08-18 14:45         ` Xinglu Chen
2021-08-16 19:56     ` [bug#50072] [PATCH WIP 0/4] Add upstream updater for " Sarah Morgensen
2021-08-17 10:18       ` Maxime Devos
2021-08-30 21:36         ` Maxime Devos
2021-09-06 10:23         ` Ludovic Courtès
2021-09-06 11:47           ` Maxime Devos
2021-09-07  1:16     ` [bug#50072] [PATCH WIP 4/4] upstream: Support updating " Sarah Morgensen
2021-09-07 10:00       ` Maxime Devos
2021-09-07 17:51         ` Sarah Morgensen
2021-09-07 20:58           ` Maxime Devos
2021-09-06 10:27   ` [bug#50072] [PATCH WIP 0/4] Add upstream updater for " Ludovic Courtès
2021-09-07  1:59     ` Sarah Morgensen
2021-09-29 21:28       ` Ludovic Courtès
2021-11-17 15:03         ` Ludovic Courtès
2022-01-01 17:35 ` Maxime Devos
2022-01-01 20:39 ` [bug#50072] [PATCH v2 " Maxime Devos
2022-01-01 20:39   ` [bug#50072] [PATCH v2 1/4] guix hash: Extract file hashing procedures Maxime Devos
2022-01-01 20:39   ` [bug#50072] [PATCH v2 2/4] import: Factorize file hashing Maxime Devos
2022-01-01 20:39   ` [bug#50072] [PATCH v2 3/4] refresh: Support non-tarball sources Maxime Devos
2022-01-03 13:55     ` Ludovic Courtès
2022-01-01 20:39   ` [bug#50072] [PATCH v2 4/4] upstream: Support updating 'git-fetch' origins Maxime Devos
2022-01-03 14:02     ` Ludovic Courtès
2022-01-04 15:09 ` [bug#50072] [PATCH v3 0/4] Add upstream updater for git-fetch origins Maxime Devos
2022-01-04 15:09   ` [bug#50072] [PATCH v3 1/4] guix hash: Extract file hashing procedures Maxime Devos
2022-01-04 15:09   ` [bug#50072] [PATCH v3 2/4] import: Factorize file hashing Maxime Devos
2022-01-04 15:09   ` [bug#50072] [PATCH v3 3/4] refresh: Support non-tarball sources Maxime Devos
2022-01-04 15:09   ` [bug#50072] [PATCH v3 4/4] upstream: Support updating and fetching 'git-fetch' origins Maxime Devos
2022-01-04 19:05   ` [bug#50072] [PATCH v3 0/4] Add upstream updater for git-fetch origins Maxime Devos
2022-01-04 20:06 ` [bug#50072] [PATCH v4 " Maxime Devos
2022-01-04 20:06   ` [bug#50072] [PATCH v4 1/4] guix hash: Extract file hashing procedures Maxime Devos
2022-01-04 22:22     ` [bug#50072] [PATCH WIP 0/4] Add upstream updater for git-fetch origins zimoun
2022-01-05 10:07       ` Maxime Devos
2022-01-05 11:48         ` zimoun
2022-01-05 12:10           ` Maxime Devos
2022-01-06 10:06             ` Ludovic Courtès
2022-01-05 12:27           ` Maxime Devos
2022-01-05 12:58             ` zimoun
2022-01-05 14:06               ` Maxime Devos
2022-01-05 15:08                 ` zimoun
2022-01-05 15:54                   ` Maxime Devos
2022-01-06 10:13                 ` Ludovic Courtès
2022-01-06 10:32                   ` Maxime Devos
2022-01-06 11:19                   ` zimoun
2022-01-05 10:09       ` Maxime Devos
2022-01-04 20:06   ` [bug#50072] [PATCH v4 2/4] import: Factorize file hashing Maxime Devos
2022-01-04 20:06   ` [bug#50072] [PATCH v4 3/4] refresh: Support non-tarball sources Maxime Devos
2022-01-04 20:06   ` [bug#50072] [PATCH v4 4/4] upstream: Support updating and fetching 'git-fetch' origins Maxime Devos
2022-01-05 14:07 ` Maxime Devos [this message]
2022-01-05 14:07   ` [bug#50072] [PATCH v5 2/4] import: Factorize file hashing Maxime Devos
2022-01-05 14:07   ` [bug#50072] [PATCH v5 3/4] refresh: Support non-tarball sources Maxime Devos
2022-01-05 14:07   ` [bug#50072] [PATCH v5 4/4] upstream: Support updating and fetching 'git-fetch' origins Maxime Devos
2022-01-05 15:57   ` [bug#50072] [PATCH v5 1/4] guix hash: Extract file hashing procedures zimoun
2022-01-05 15:56 ` Maxime Devos
2022-01-05 15:56   ` [bug#50072] [PATCH v5 2/4] import: Factorize file hashing Maxime Devos
2022-01-05 15:56   ` [bug#50072] [PATCH v5 3/4] refresh: Support non-tarball sources Maxime Devos
2022-01-05 15:56   ` [bug#50072] [PATCH v5 4/4] upstream: Support updating and fetching 'git-fetch' origins Maxime Devos
2022-01-06 10:20     ` bug#50072: [PATCH WIP 0/4] Add upstream updater for git-fetch origins Ludovic Courtès
2022-01-06 14:12       ` [bug#50072] " Maxime Devos

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220105140750.18214-1-maximedevos@telenet.be \
    --to=maximedevos@telenet.be \
    --cc=50072@debbugs.gnu.org \
    --cc=iskarian@mgsn.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).