From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id OOiACe7QMWVc9AAAG6o9tA:P1 (envelope-from ) for ; Fri, 20 Oct 2023 02:59:26 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id OOiACe7QMWVc9AAAG6o9tA (envelope-from ) for ; Fri, 20 Oct 2023 02:59:26 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B10C05971F for ; Fri, 20 Oct 2023 02:59:25 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20230601 header.b=IoKCbPRX; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1697763565; a=rsa-sha256; cv=none; b=OP0afHSVMy5yTh6ovYwTlG9bxMvFr8HvutiaxNbklgZBp8zT4mwAtuJOtyZarVYOoOgCTf y4ZU0W1XjqQhCRUT6357HHmllchT7VR+/E813ZQ4ex6eZY+e1EQNiAqT4RgWw6XXV9igiX aVovPrrr9njgPy7ZAqjAOhnZK+8SKDHyknxl5XtQGmUSXJOoRAN0m/Ls8mieLwH71rwZtx UtinUDRpfpkaZx0SAdxt1o03feK3CECP96wpZMD7C3jhGu1JRhNTjMEa6+2XNqR1m+PLfn oVWDIiRJ1ETVSQJfPHWmKTgecHWuqksBE/QHuls7nYnoX2KfHA0RYsAOCyvJWQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20230601 header.b=IoKCbPRX; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1697763565; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=2eiEVVW1YIPDVs3/30xqVWj4WCnrhALxoLlMnwbevBc=; b=hESXuss4IW0LUmsidPqr2EczTTk77j931/wsX3MwAqAeXUydoVQhIHLyLUcARxLaZ+wITm oUq/Q5ufqd6ITAOcTu8rqb+xvne0y35fZBZexQ3DW0udlyGwq1JK0VdnLLirFOFoAt4kiW an7QKz8UvYc7XP2Ygy8gMrloDeEwQh+9KGKVZL91LoZEekv7ze0+IlUmdGOnm4dQ30srPG tYLm4AbM700ZCtaiMxegJjtPvbcqdgfybNtfWrR+mXJC4rsU/SBUSgcB/vB+JhWIvbbGcb U7W7kOrqP5Uc8uXCK6eMYtFOpFnU0eXdQkipN6IM7AkpV7VkO78CV7HpaT4Ogg== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtdqa-0003mB-NG; Thu, 19 Oct 2023 20:58:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtdqW-0003lQ-Kn for guix-patches@gnu.org; Thu, 19 Oct 2023 20:58:37 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtdqV-0007Mg-Pr; Thu, 19 Oct 2023 20:58:35 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qtdqw-0003kl-7C; Thu, 19 Oct 2023 20:59:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42146] [PATCH v3 2/3] build: substitute: Error when no substitutions were done. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: kuba@kadziolka.net, maxim.cournoyer@gmail.com, ludo@gnu.org, guix-patches@gnu.org Resent-Date: Fri, 20 Oct 2023 00:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42146 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42146@debbugs.gnu.org Cc: Jakub =?UTF-8?Q?K=C4=85dzio=C5=82ka?= , Maxim Cournoyer , Jakub =?UTF-8?Q?K=C4=85dzio=C5=82ka?= , Maxim Cournoyer , Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-Xcc: Jakub =?UTF-8?Q?K=C4=85dzio=C5=82ka?= , Maxim Cournoyer , Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 42146-submit@debbugs.gnu.org id=B42146.169776351414362 (code B ref 42146); Fri, 20 Oct 2023 00:59:02 +0000 Received: (at 42146) by debbugs.gnu.org; 20 Oct 2023 00:58:34 +0000 Received: from localhost ([127.0.0.1]:38243 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtdqT-0003jY-OW for submit@debbugs.gnu.org; Thu, 19 Oct 2023 20:58:34 -0400 Received: from mail-vs1-xe33.google.com ([2607:f8b0:4864:20::e33]:54714) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtdqR-0003jD-CA for 42146@debbugs.gnu.org; Thu, 19 Oct 2023 20:58:32 -0400 Received: by mail-vs1-xe33.google.com with SMTP id ada2fe7eead31-4580a2ec248so87551137.1 for <42146@debbugs.gnu.org>; Thu, 19 Oct 2023 17:58:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697763478; x=1698368278; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2eiEVVW1YIPDVs3/30xqVWj4WCnrhALxoLlMnwbevBc=; b=IoKCbPRX35cahet1V/Dzbj+HX18OnMOHFotZRlCTd0/B2kTHs+RJ+oWcO9lZYOmEtI IioWxCnUsx+iYskfZ/Ia0gFGf2Q4DBgTtfhTNflV+aZPRUk1HAt1WBGoGN5Yq22feKdL DpObCGX4Fe9zc5iB/Y+bFifLdMgNL7fzVrI3rfHdjH8c/hyq/JSzlywM1ZRXq9yLXd6p Uk3AYW2OWCneLucuGejW6+T4ZLJ0Eayg9Oqwhb81Tbun6IvD71Xc2Ahh8U7FInv9ZSRR 8eIu2PyHsF2U6Afqz/5YE0TA54GtoUe9LxXQdL7m5uXHub5U1QGX0SsVCnGvxIUS5z88 Bupg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697763478; x=1698368278; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2eiEVVW1YIPDVs3/30xqVWj4WCnrhALxoLlMnwbevBc=; b=rPZ7FPqphoYnjKWiQXsX5Jzd3eWsiX2jVdmppg82AQZIU9VccaN0uz59d/GqcA3/bV Vf2674tTO7TvV7/XY6lB7Pu1HYyVsGB5EOc+qhYYz+IxfpJW5n8QC0xS8ctSBE0d1+5t ZDvyOUgMzWt/NwqoXJRS2byes2CRnscOsjBcVS9IOcJAiVkSt5KCPB1T66zJroxxVEB1 BN1yO1QHkaiRXzfUHE6wYPCWGsibAOKX6NN0ZOTlhJT/aurOXisP/VBSskO5EKySsxRl +cTEfcbZ6H6HAZnMp/GKCh7RYLxIgvUshSLoCzVXV2eUQ3+jXQHYy0Ngv7Ia6qnMSWKZ Dk6g== X-Gm-Message-State: AOJu0Ywt1LJcsT05HvPH15a66wqfiunS6j/VTDmRqA+lZo4t762dKpcA yFeSz0uAXSxeq7qdmebYmXcgh5rgVsUp3g== X-Google-Smtp-Source: AGHT+IFfRU5dHNsQoMCxDMmAySoleppZPRIpE66fblrMLPEF5zzKO8dSii5CS3s278SeXBnLIP/A1A== X-Received: by 2002:a05:6102:20c4:b0:457:7138:fa74 with SMTP id i4-20020a05610220c400b004577138fa74mr552529vsr.35.1697763478521; Thu, 19 Oct 2023 17:57:58 -0700 (PDT) Received: from localhost.localdomain (dsl-156-111.b2b2c.ca. [66.158.156.111]) by smtp.gmail.com with ESMTPSA id h9-20020a0cf8c9000000b0065d0a4262e0sm279097qvo.70.2023.10.19.17.57.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 17:57:58 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 19 Oct 2023 20:57:39 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx1.migadu.com X-Migadu-Spam-Score: 7.34 X-Spam-Score: 7.34 X-Migadu-Queue-Id: B10C05971F X-TUID: iHMmEjkhiS8M From: Jakub Kądziołka * guix/build/utils.scm (substitute, substitute*) [require-matches?]: New argument. * tests/build-utils.scm ("substitute*"): New test group. ("substitute*, no match error") ("substitute*, partial no match error"): New tests. Co-authored-by: Maxim Cournoyer Change-Id: I66ed33d72aa73cd35e5642521efec70bf756f86e --- guix/build/utils.scm | 93 +++++++++++++++++++++++++++++++++---------- tests/build-utils.scm | 68 +++++++++++++++++++++---------- 2 files changed, 118 insertions(+), 43 deletions(-) diff --git a/guix/build/utils.scm b/guix/build/utils.scm index 2b3a8e278b..8e4b8321dd 100644 --- a/guix/build/utils.scm +++ b/guix/build/utils.scm @@ -6,7 +6,8 @@ ;;; Copyright © 2018, 2022 Arun Isaac ;;; Copyright © 2018, 2019 Ricardo Wurmus ;;; Copyright © 2020 Efraim Flashner -;;; Copyright © 2020, 2021 Maxim Cournoyer +;;; Copyright © 2020 Jakub Kądziołka +;;; Copyright © 2020, 2021, 2023 Maxim Cournoyer ;;; Copyright © 2021, 2022 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley ;;; Copyright © 2022 Simon Tournier @@ -111,8 +112,14 @@ (define-module (guix build utils) modify-phases with-atomic-file-replacement + %substitute-requires-matches? substitute substitute* + &substitute-error + substitute-error? + substitute-error-file + substitute-error-patterns + dump-port set-file-time patch-shebang @@ -971,24 +978,51 @@ (define (replace-char c1 c2 s) c)) s))) -(define (substitute file pattern+procs) +(define-condition-type &substitute-error &error + substitute-error? + (file substitute-error-file) + (patterns substitute-error-patterns)) + +(define %substitute-requires-matches? + (make-parameter #t)) + +(define* (substitute file pattern+procs + #:key (require-matches? (%substitute-requires-matches?))) "PATTERN+PROCS is a list of regexp/two-argument-procedure pairs. For each line of FILE, and for each PATTERN that it matches, call the corresponding PROC as (PROC LINE MATCHES); PROC must return the line that will be written as a substitution of the original line. Be careful about using '$' to match the -end of a line; by itself it won't match the terminating newline of a line." - (let ((rx+proc (map (match-lambda - (((? regexp? pattern) . proc) +end of a line; by itself it won't match the terminating newline of a line. + +By default, SUBSTITUTE will raise a &substitute-error condition if one of the +patterns fails to match. REQUIRE-MATCHES? can be set to false when lack of +matches is acceptable (e.g. if you have multiple potential patterns not +guaranteed to be found in FILE)." + (define (rx->pattern m) + (match m + ((? regexp? pattern) + "") + ((? regexp*? pattern) + (regexp*-pattern pattern)) + ((? string? pattern) + pattern))) + + (let ((rx+proc (map (match-lambda + (((or (? regexp? pattern) (? regexp*? pattern)) . proc) (cons pattern proc)) ((pattern . proc) - (cons (make-regexp pattern regexp/extended) - proc))) - pattern+procs))) + (cons (make-regexp* pattern regexp/extended) proc))) + pattern+procs))) (with-atomic-file-replacement file (lambda (in out) - (let loop ((line (read-line in 'concat))) + (let loop ((line (read-line in 'concat)) + (unmatched-regexps (map first rx+proc))) (if (eof-object? line) - #t + (when (and require-matches? (not (null? unmatched-regexps))) + (raise (condition + (&substitute-error + (file file) + (patterns (map rx->pattern unmatched-regexps)))))) ;; Work around the fact that Guile's regexp-exec does not handle ;; NUL characters (a limitation of the underlying GNU libc's ;; regexec) by temporarily replacing them by an unused private @@ -998,19 +1032,23 @@ (define (substitute file pattern+procs) (unused-private-use-code-point line)) #\nul)) (line* (replace-char #\nul nul* line)) - (line1* (fold (lambda (r+p line) - (match r+p - ((regexp . proc) - (match (list-matches regexp line) - ((and m+ (_ _ ...)) - (proc line m+)) - (_ line))))) - line* - rx+proc)) + (results ;line, unmatched-regexps + (fold (lambda (r+p results) + (let ((line (first results)) + (unmatched (second results))) + (match r+p + ((regexp . proc) + (match (list-matches* regexp line) + ((and m+ (_ _ ...)) + (list (proc line m+) + (delq regexp unmatched))) + (_ (list line unmatched))))))) + (list line* unmatched-regexps) + rx+proc)) + (line1* (first results)) (line1 (replace-char nul* #\nul line1*))) (display line1 out) - (loop (read-line in 'concat))))))))) - + (loop (read-line in 'concat) (second results))))))))) (define-syntax let-matches ;; Helper macro for `substitute*'. @@ -1048,9 +1086,19 @@ (define-syntax substitute* Alternatively, FILE may be a list of file names, in which case they are all subject to the substitutions. +By default, SUBSTITUTE* will raise a &message condition if one of the patterns +fails to match on one of the files; REQUIRE-MATCHES? may be set to false to +avoid an error being raised in such condition. + Be careful about using '$' to match the end of a line; by itself it won't match the terminating newline of a line." ((substitute* file ((regexp match-var ...) body ...) ...) + (substitute* file + ((regexp match-var ...) body ...) ... + #:require-matches? #t)) + ((substitute* file + ((regexp match-var ...) body ...) ... + #:require-matches? require-matches?) (let () (define (substitute-one-file file-name) (substitute @@ -1074,7 +1122,8 @@ (define-syntax substitute* (begin body ...) (substring l o (match:start m)) r)))))))) - ...))) + ...) + #:require-matches? require-matches?)) (match file ((files (... ...)) diff --git a/tests/build-utils.scm b/tests/build-utils.scm index 3babf5d544..35c66faa3c 100644 --- a/tests/build-utils.scm +++ b/tests/build-utils.scm @@ -1,7 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2012, 2015, 2016, 2019, 2020 Ludovic Courtès ;;; Copyright © 2019 Ricardo Wurmus -;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021, 2023 Maxim Cournoyer ;;; Copyright © 2021 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley ;;; @@ -289,26 +289,52 @@ (define (arg-test bash-args) (test-assert "wrap-script, argument handling, bash --norc" (arg-test " --norc")) -(test-equal "substitute*, text contains a NUL byte, UTF-8" - "c\0d" - (with-fluids ((%default-port-encoding "UTF-8") - (%default-port-conversion-strategy 'error)) - ;; The GNU libc is locale sensitive. Depending on the value of LANG, the - ;; test could fail with "string contains #\\nul character: ~S" or "cannot - ;; convert wide string to output locale". - (setlocale LC_ALL "en_US.UTF-8") - (call-with-temporary-output-file - (lambda (file port) - (format port "a\0b") - (flush-output-port port) - - (substitute* file - (("a") "c") - (("b") "d")) - - (with-input-from-file file - (lambda _ - (get-string-all (current-input-port)))))))) +(define-syntax-rule (define-substitute*-test test-type name expected + content clauses ...) + (test-type + name + expected + (with-fluids ((%default-port-encoding "UTF-8") + (%default-port-conversion-strategy 'error)) + ;; The GNU libc is locale sensitive. Depending on the value of LANG, + ;; the test could fail with "string contains #\\nul character: ~S" or + ;; "cannot convert wide string to output locale". + (setlocale LC_ALL "en_US.UTF-8") + (call-with-temporary-output-file + (lambda (file port) + (format port content) + (flush-output-port port) + + (substitute* file + clauses ...) + + (with-input-from-file file + (lambda _ + (get-string-all (current-input-port))))))))) + +(define-substitute*-test test-equal + "substitute*, text contains a NUL byte, UTF-8" + "c\0d" ;expected + "a\0b" ;content + (("a") "c") + (("b") "d")) + +(define-substitute*-test test-error "substitute*, no match error" + #t ;expected + "a\0b" ;content + (("Oops!") "c")) + +(define-substitute*-test test-equal "substitute*, no match, ignored" + "abc" ;expected + "abc" ;content + (("Oops!") "c") + #:require-matches? #f) + +(define-substitute*-test test-error "substitute*, partial no match error" + #t ;expected + "a\0b" ;content + (("a") "c" + ("Oops!") "c")) (test-equal "search-input-file: exception if not found" `((path) -- 2.41.0