From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 0DzuBH6clWHRQAAAgWs5BA (envelope-from ) for ; Thu, 18 Nov 2021 01:21:18 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id uN+tAH6clWE7IQAAbx9fmQ (envelope-from ) for ; Thu, 18 Nov 2021 00:21:18 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9916934ED5 for ; Thu, 18 Nov 2021 01:21:17 +0100 (CET) Received: from localhost ([::1]:51214 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mnVAy-0003lX-Q1 for larch@yhetil.org; Wed, 17 Nov 2021 19:21:16 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45194) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mnVAm-0003gr-5N for guix-patches@gnu.org; Wed, 17 Nov 2021 19:21:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:50832) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mnVAl-00088e-2R for guix-patches@gnu.org; Wed, 17 Nov 2021 19:21:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mnVAk-00070h-LO for guix-patches@gnu.org; Wed, 17 Nov 2021 19:21:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#51307] [PATCH v2 2/3] scripts: hash: Add 'serializer' option. Resent-From: zimoun Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 18 Nov 2021 00:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51307 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 51307@debbugs.gnu.org Cc: ludo@gnu.org, zimoun Received: via spool by 51307-submit@debbugs.gnu.org id=B51307.163719485026885 (code B ref 51307); Thu, 18 Nov 2021 00:21:02 +0000 Received: (at 51307) by debbugs.gnu.org; 18 Nov 2021 00:20:50 +0000 Received: from localhost ([127.0.0.1]:34141 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mnVAX-0006zS-KU for submit@debbugs.gnu.org; Wed, 17 Nov 2021 19:20:50 -0500 Received: from mail-wm1-f47.google.com ([209.85.128.47]:40566) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mnVAS-0006yg-Aw for 51307@debbugs.gnu.org; Wed, 17 Nov 2021 19:20:46 -0500 Received: by mail-wm1-f47.google.com with SMTP id az33-20020a05600c602100b00333472fef04so6184721wmb.5 for <51307@debbugs.gnu.org>; Wed, 17 Nov 2021 16:20:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cY2sj5YyVxW04ul2pRGYBDOUoW+p8Rb6I9dQaIubgiU=; b=mtFnMqTC5dLvlS7sLv464bD26ruJGgx+twwB9ufpPDolUi6K8xfUTXeRxOU0+Af0AB Tb30MnAuEO2tx0FmPjcQOn559pjlN+ih+DDplJB2JVYk1iLzzsPpCyYVRIMFVHiA2aIp wQRx/yCoqhzALKMZubHl12YSqOt8UnSCE0VDFHICxlEF0w1LbY9Yu54XD0zW8skXDfT2 OYdPO9GQcuGx8ASXfbBk58k4fXICL+312/HiOnShHkfcEMwdFvpcoc1E70+VBPIZUqtg jM+iRcNTkliop3DCnpXhTsvjDxsvgXonsv70/c5RhmZofRoeD2IMwR4Ki0DSXwp0gTc3 5ipw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cY2sj5YyVxW04ul2pRGYBDOUoW+p8Rb6I9dQaIubgiU=; b=t7P1Br90aDaD6297ArJtSofiCMzvrOKUyFbl7LAIIZT1FdxTlWC8r4SCiAjGhllejE fZG5nSLCFdz2ipU8hsZbfv94/QZ2oFImlkVj4C1WGG5RwWcEnx1Jnnx9p1uYySB4PhHd /SyMVxTcOWTNlgH+LmsKvtteD8P+iHaTL8utAu63BXjuPIDUsC3GnioksPFgdIMR1uHG QEhuvp+ZLbqeBdV4kXshLexHj/PV6NA/+EyytXWlGBcUZdyXiPRdzbC2B4tmiBvuE0kt wqvHwRmeejWssWKi7KWLkuNoHnXTXIGd9CpSFzRXQCUJ+bTZqbTd3BFwWdQaEb2yDFy0 eMqw== X-Gm-Message-State: AOAM5306IwEXUUB2I42ekeJPMfTt3KinFTdsLbR5nDukMczU59zT9Aok hBDgSPlhYUO2uMWMJKt5Lw5zaJ+3KQY= X-Google-Smtp-Source: ABdhPJwmmJck3m8o3lg0Sgmz/VUMJ7lF8Czwi9PjAniJEZtR2AwGLaolwWHIVTmFB2QuRAzRq6aZ7w== X-Received: by 2002:a7b:c2f7:: with SMTP id e23mr4766735wmk.92.1637194838831; Wed, 17 Nov 2021 16:20:38 -0800 (PST) Received: from localhost.localdomain ([193.48.40.117]) by smtp.gmail.com with ESMTPSA id y7sm1255927wrw.55.2021.11.17.16.20.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Nov 2021 16:20:38 -0800 (PST) From: zimoun Date: Thu, 18 Nov 2021 01:20:22 +0100 Message-Id: <20211118002023.3323307-3-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211118002023.3323307-1-zimon.toutoune@gmail.com> References: <20211118002023.3323307-1-zimon.toutoune@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1637194877; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=d2FIYeNIuetlp7IzAXCVKfY/B6A6LtyBnJ2/odSZStA=; b=C9QtUsOu/U/zcSZChzocZ7WtUadaBWAFw3UO5TcYpgnBGyscL/q5i1pT2DzeETk5zfb6MN Hk35GIfZGBwrr5mYbV6XYYx1tF1bewPlVORKzALeP3mHkWPBV61wM9PNGdt/9IRkrMhIw2 /WKbcsNbXyclGYJRAtak5Aeq8v9k/KurCzgsj3zROM0VsSatkRmVASUJoKHNN0xrj8IheQ R/WOfssFLItD0DPRQrrQubMVcqu/SYvbgT2ld+FpQSchpdJF74irlWKyklB0d0+6L83reT iGkdLQizHeO8CthY4pvRprpbICRGq2WqfXbh25br9jMkZr9gcxoCLlDZgvNFTw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1637194877; a=rsa-sha256; cv=none; b=Dfi78FF868LCyRQzYRqhUQlLKg4BJLvbVyQ4hV3ikKrDTl7MdzB+nlD4V36qSbFRokZ/ZT a5nAKc+8JWQ3Hqok/ZGMg2WBPuHt3n5VBvigZjFsyH5yWlwjjZyPwst4CwWLqii66SyYc8 +PynXYV6vadmKReAfJ5kTSWZXcnvRgKtD2Car4+4d9kMaeyWFs+wv1jAYykNJ3GehtD8do MKACrrBJ0tsEriHD2FDbgSUt0x6kRAE2/zPNjeIuijoA9TRxcSTt3lyFASA69fnLxa4iip 5m3XHhwCQb0Xf47f1xtl4/nDuAuHvwveAIIov8KXv3hWk8LS7rsZeTDs81gG6w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20210112 header.b=mtFnMqTC; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -1.76 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20210112 header.b=mtFnMqTC; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 9916934ED5 X-Spam-Score: -1.76 X-Migadu-Scanner: scn1.migadu.com X-TUID: NqGf3JG9QEQO * guix/scripts/hash.scm (%options): Deprecate 'recursive', add 'serializer'. (%default-options): Add 'serializer'. (nar-hash): New procedure. (default-hash): New procedure. (guix-hash)[file-hash]: Use them. (show-help): Adjust. * tests/guix-hash.scm: Adjust. * doc/guix.texi: Update. --- doc/guix.texi | 25 ++++++++----- guix/scripts/hash.scm | 82 +++++++++++++++++++++++++++++-------------- tests/guix-hash.sh | 14 +++++--- 3 files changed, 81 insertions(+), 40 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 89a970908d..20041c20b7 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -71,7 +71,7 @@ Copyright @copyright{} 2019 Kyle Andrews@* Copyright @copyright{} 2019 Alex Griffin@* Copyright @copyright{} 2019, 2020, 2021 Guillaume Le Vaillant@* Copyright @copyright{} 2020 Liliana Marie Prikler@* -Copyright @copyright{} 2019, 2020 Simon Tournier@* +Copyright @copyright{} 2019, 2020, 2021 Simon Tournier@* Copyright @copyright{} 2020 Wiktor Żelazny@* Copyright @copyright{} 2020 Damien Cassou@* Copyright @copyright{} 2020 Jakub Kądziołka@* @@ -11631,14 +11631,21 @@ in the definitions of packages. @item --recursive @itemx -r -Compute the hash on @var{file} recursively. - -In this case, the hash is computed on an archive containing @var{file}, -including its children if it is a directory. Some of the metadata of -@var{file} is part of the archive; for instance, when @var{file} is a -regular file, the hash is different depending on whether @var{file} is -executable or not. Metadata such as time stamps has no impact on the -hash (@pxref{Invoking guix archive}). +This option is deprecated in favor of @option{--serializer}. It is a +legacy alias for @var{type} sets to @code{nar}. + +@item --serializer=@var{type} +@itemx -S +Compute the hash on @var{file} using @var{type} serialization. + +Supported types: @code{none} and @code{nar}. + +When using @code{nar}, the hash is computed on an archive containing +@var{file}, including its children if it is a directory. Some of the +metadata of @var{file} is part of the archive; for instance, when +@var{file} is a regular file, the hash is different depending on whether +@var{file} is executable or not. Metadata such as time stamps has no +impact on the hash (@pxref{Invoking guix archive}). @c FIXME: Replace xref above with xref to an ``Archive'' section when @c it exists. diff --git a/guix/scripts/hash.scm b/guix/scripts/hash.scm index 12f542929b..d05ecb80ba 100644 --- a/guix/scripts/hash.scm +++ b/guix/scripts/hash.scm @@ -37,6 +37,29 @@ (define-module (guix scripts hash) #:use-module (srfi srfi-37) #:export (guix-hash)) + +;;; +;;; Serializers +;;; + +(define* (nar-hash file #:optional + (algorithm (assoc-ref %default-options 'hash-algorithm)) + select?) + (let-values (((port get-hash) + (open-hash-port algorithm))) + (write-file file port #:select? select?) + (force-output port) + (get-hash))) + +(define* (default-hash file #:optional + (algorithm (assoc-ref %default-options 'hash-algorithm)) + select?) + (match file + ("-" (port-hash algorithm (current-input-port))) + (_ + (call-with-input-file file + (cute port-hash algorithm <>))))) + ;;; ;;; Command-line options. @@ -45,7 +68,8 @@ (define-module (guix scripts hash) (define %default-options ;; Alist of default option values. `((format . ,bytevector->nix-base32-string) - (hash-algorithm . ,(hash-algorithm sha256)))) + (hash-algorithm . ,(hash-algorithm sha256)) + (serializer . ,default-hash))) (define (show-help) (display (G_ "Usage: guix hash [OPTION] FILE @@ -61,7 +85,7 @@ (define (show-help) (format #t (G_ " -f, --format=FMT write the hash in the given format")) (format #t (G_ " - -r, --recursive compute the hash on FILE recursively")) + -S, --serializer=TYPE compute the hash on FILE according to TYPE serialization")) (newline) (display (G_ " -h, --help display this help and exit")) @@ -102,7 +126,24 @@ (define fmt-proc (alist-delete 'format result)))) (option '(#\r "recursive") #f #f (lambda (opt name arg result) - (alist-cons 'recursive? #t result))) + (warning (G_ "'--recursive' is deprecated, \ +use '--serializer' instead~%")) + (alist-cons 'serializer nar-hash + (alist-delete 'serializer result)))) + (option '(#\S "serializer") #t #f + (lambda (opt name arg result) + (define serializer-proc + (match arg + ("none" + default-hash) + ("nar" + nar-hash) + (x + (leave (G_ "unsupported serializer type: ~a~%") + arg)))) + + (alist-cons 'serializer serializer-proc + (alist-delete 'serializer result)))) (option '(#\h "help") #f #f (lambda args (show-help) @@ -145,35 +186,24 @@ (define (vcs-file? file stat) (fmt (assq-ref opts 'format)) (select? (if (assq-ref opts 'exclude-vcs?) (negate vcs-file?) - (const #t)))) + (const #t))) + (algorithm (assoc-ref opts 'hash-algorithm)) + (serializer (assoc-ref opts 'serializer))) (define (file-hash file) ;; Compute the hash of FILE. - ;; Catch and gracefully report possible '&nar-error' conditions. - (if (assoc-ref opts 'recursive?) + ;; Catch and gracefully report possible error + (catch 'system-error + (lambda _ (with-error-handling - (let-values (((port get-hash) - (open-hash-port (assoc-ref opts 'hash-algorithm)))) - (write-file file port #:select? select?) - (force-output port) - (get-hash))) - (catch 'system-error - (lambda _ - (call-with-input-file file - (cute port-hash (assoc-ref opts 'hash-algorithm) - <>))) - (lambda args - (leave (G_ "~a ~a~%") - file - (strerror (system-error-errno args))))))) + (serializer file algorithm select?))) + (lambda args + (leave (G_ "~a ~a~%") + file + (strerror (system-error-errno args)))))) (define (formatted-hash thing) - (match thing - ("-" (with-error-handling - (fmt (port-hash (assoc-ref opts 'hash-algorithm) - (current-input-port))))) - (_ - (fmt (file-hash thing))))) + (fmt (file-hash thing))) (for-each (compose (cute format #t "~a~%" <>) formatted-hash) diff --git a/tests/guix-hash.sh b/tests/guix-hash.sh index c4461fa955..cdcfac19bc 100644 --- a/tests/guix-hash.sh +++ b/tests/guix-hash.sh @@ -42,25 +42,29 @@ chmod +x "$tmpdir/exe" ( cd "$tmpdir" ; ln -s exe symlink ) mkdir "$tmpdir/subdir" -test `guix hash -r "$tmpdir"` = 10k1lw41wyrjf9mxydi0is5nkpynlsvgslinics4ppir13g7d74p -test `guix hash -r "$tmpdir" -H sha512` = 301ra58c2vahczzxiyfin41mpyb0ljh4dh9zn3ijvwviaw1j40sfzw5skh9x945da88n3785ggifzig7acd6k72h0mpsc20m1f66m9n +test `guix hash -S nar "$tmpdir"` = 10k1lw41wyrjf9mxydi0is5nkpynlsvgslinics4ppir13g7d74p +test `guix hash -S nar "$tmpdir" -H sha512` = 301ra58c2vahczzxiyfin41mpyb0ljh4dh9zn3ijvwviaw1j40sfzw5skh9x945da88n3785ggifzig7acd6k72h0mpsc20m1f66m9n + +# Deprecated --recursive option +test `guix hash -r "$tmpdir" 2>/dev/null` = 10k1lw41wyrjf9mxydi0is5nkpynlsvgslinics4ppir13g7d74p +test `guix hash -r "$tmpdir" -H sha512 2>/dev/null` = 301ra58c2vahczzxiyfin41mpyb0ljh4dh9zn3ijvwviaw1j40sfzw5skh9x945da88n3785ggifzig7acd6k72h0mpsc20m1f66m9n # Without '-r', this should fail. ! guix hash "$tmpdir" # This should fail because /dev/null is a character device, which # the archive format doesn't support. -! guix hash -r /dev/null +! guix hash -S nar /dev/null # Adding a .git directory mkdir "$tmpdir/.git" touch "$tmpdir/.git/foo" # ...changes the hash -test `guix hash -r $tmpdir` = 0a50z04zyzf7pidwxv0nwbj82pgzbrhdy9562kncnvkcfvb48m59 +test `guix hash -S nar $tmpdir` = 0a50z04zyzf7pidwxv0nwbj82pgzbrhdy9562kncnvkcfvb48m59 # ...but remains the same when using `-x' -test `guix hash -r $tmpdir -x` = 10k1lw41wyrjf9mxydi0is5nkpynlsvgslinics4ppir13g7d74p +test `guix hash -S nar $tmpdir -x` = 10k1lw41wyrjf9mxydi0is5nkpynlsvgslinics4ppir13g7d74p # Without '-r', this should fail. ! guix hash "$tmpdir" -- 2.33.1