From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:403:478a::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id iM6CNfOSMWVwLAAA9RJhRA:P1 (envelope-from ) for ; Thu, 19 Oct 2023 22:35:00 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:478a::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id iM6CNfOSMWVwLAAA9RJhRA (envelope-from ) for ; Thu, 19 Oct 2023 22:34:59 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 67DE363CA6 for ; Thu, 19 Oct 2023 22:34:59 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20230601 header.b=aigGrMq5; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1697747699; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=oTAbCUEBY4/PaAlLb3JRnahAdJdeX/zaSLEJfY7+90g=; b=fDR8GKuoijj5Tk/qkOKZezjxckzgDrOZPB5s+a7WcWY+dxpjNIW4hWeKNn9bv/aA/rkEb9 e0mheT0E/dYQmXoO+T4OM8N5mhLMaZku0Lo4nGnUwljml0mlXiE7Pv2U4M/zrlL5J0xSgY j3BBWcFMQIdlbn1xaYwFPYRYa0CbaCRQmO50Izk+ERlVCUTHKxK+Ahfug8uUY6MPD6YdFo Q1qozWzF2pKN/ZjeM9MZ1OhDiCIo+tAs0NTbi+pXqAEfDj1gLkvbXZfO8kDkRvD1vrCWg4 lq2aQ3mHb+26sTBxsOs6gO36u4hyGvNHOetAY6PlONRM2nsECzUpyGSXXNXDlw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1697747699; a=rsa-sha256; cv=none; b=e6jG+kfKVtxNYkQP7XcgIP2MyhL+7rGXb9/PmTsO0DmQz/Q1LUQ7bL/288cHQ8BYfnqIc8 sqXiGLL8C4x2tDIgEWuxz2U82JHvcjzcL7plUIO36B3b5Xa3RTsZFIeAxqjV+Q8ya76G84 tB23JQzESOkinvKLg/LQ6PEBZP3oTv+j81b703K/d4BrX6lcAuInUdPm7y571d2vCEnrjA vg+pF5ay4rl0JGBAnIMh4Gbmzt3IpAQ9yheD/GBY79TSIZ5kMduzZtY//gjL2juJAanhQg NWeVp03f50xkqjPSlsPLi+ozQR2MZs8wKs/qDZ7PM2mfJ7x+AZdQy9RdqQGLSA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20230601 header.b=aigGrMq5; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtZj3-0006Ke-RZ; Thu, 19 Oct 2023 16:34:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtZj1-0006Iu-VH for guix-patches@gnu.org; Thu, 19 Oct 2023 16:34:35 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtZj1-0006zz-MV for guix-patches@gnu.org; Thu, 19 Oct 2023 16:34:35 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qtZjS-0000wI-4d; Thu, 19 Oct 2023 16:35:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42146] [PATCH 2/3] build: substitute: Error when no substitutions were done. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: kuba@kadziolka.net, maxim.cournoyer@gmail.com, guix-patches@gnu.org Resent-Date: Thu, 19 Oct 2023 20:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42146 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42146@debbugs.gnu.org Cc: Jakub =?UTF-8?Q?K=C4=85dzio=C5=82ka?= , Maxim Cournoyer , Jakub =?UTF-8?Q?K=C4=85dzio=C5=82ka?= , Maxim Cournoyer X-Debbugs-Original-Xcc: Jakub =?UTF-8?Q?K=C4=85dzio=C5=82ka?= , Maxim Cournoyer Received: via spool by 42146-submit@debbugs.gnu.org id=B42146.16977476883573 (code B ref 42146); Thu, 19 Oct 2023 20:35:02 +0000 Received: (at 42146) by debbugs.gnu.org; 19 Oct 2023 20:34:48 +0000 Received: from localhost ([127.0.0.1]:37944 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtZjD-0000vU-8E for submit@debbugs.gnu.org; Thu, 19 Oct 2023 16:34:47 -0400 Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]:47290) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtZj6-0000uz-DZ for 42146@debbugs.gnu.org; Thu, 19 Oct 2023 16:34:41 -0400 Received: by mail-qk1-x72d.google.com with SMTP id af79cd13be357-7788db95652so3878185a.2 for <42146@debbugs.gnu.org>; Thu, 19 Oct 2023 13:34:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697747648; x=1698352448; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oTAbCUEBY4/PaAlLb3JRnahAdJdeX/zaSLEJfY7+90g=; b=aigGrMq5FpRYclutciNokFrhbtz5FbX8/+vifefVTLPWmE6/OA5/tVoojMfaJcXZ2Z 0hzO5oSgshZ5p4o2hS9/rksQRywXE7d4I3d8SODSiarDrLKn/3rEICH0Pjc9ndIg3QWF BN3UypVZSFu3Kfx1rPtHJY1DHNhG8rew6u1u68UeNp1R/yENLp0zyPu/jlePukzWmJHB Ag3rinz+5fn7QHn9mn5ZVpSLK//X9aVf1hx0vgrskD8itsVICYINn0Ok36PqoNHW7rkK XGrOhqXonWe/GrwpsT/B2KK+EiOzkPF/1hVJNGvURb9eggwx2NTerRgfxQWniKvLXrWT G98A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697747648; x=1698352448; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oTAbCUEBY4/PaAlLb3JRnahAdJdeX/zaSLEJfY7+90g=; b=IADZxhjW81S5sZXWI4KRaCu9Y7pSues3bHaW0ZtJrjeJRZszBHwDHNIFzLI5YF4h8W EFy6qCNR8+ElPAMt6yA/AI9bR+qUiTdrMNRMM3UNu5kLbOpPyuo4CaS851w9cnZ2+yxA qOWzTtoT5dcqdiS4+eCXLPg+z9ChpLRezZpwkNgb7nlNTJywQNNsf4cIpMPUCdJrIoDA wT79PgUYSww/sKDdt50u7lPDdO2hiy0HOawzhEPxigDIx5YqiaCbYOGS/5Ee+Al5DfE/ 9wDjpRJ5e2GFPQw8ADK2+B6KrFDyvvEkit99K+QjIAPKfIAOcplRAG+S6jzJ6/kOYKwx jWsw== X-Gm-Message-State: AOJu0Yzz7GcXz3qcHVlpl1ACsfT/UMSRLZwWXGhIwWgF0AbFDgLoP9XK LIhVf8UWPJ+YYfHtRSMp+ELYu5LRg7J3ZA== X-Google-Smtp-Source: AGHT+IH3r47NdrEpahDPXiMGSZCz+rO/hZa9dCzcOtNVbE0KEcgI/qFhbXrZPzm4LX87wc0K+K5MrA== X-Received: by 2002:a05:620a:29c2:b0:775:903e:388c with SMTP id s2-20020a05620a29c200b00775903e388cmr3062568qkp.2.1697747647630; Thu, 19 Oct 2023 13:34:07 -0700 (PDT) Received: from localhost.localdomain (dsl-156-111.b2b2c.ca. [66.158.156.111]) by smtp.gmail.com with ESMTPSA id 2-20020a05620a070200b0077413b342e9sm76591qkc.128.2023.10.19.13.34.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 13:34:07 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 19 Oct 2023 16:33:33 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Spam-Score: 7.35 X-Migadu-Queue-Id: 67DE363CA6 X-Migadu-Scanner: mx0.migadu.com X-Migadu-Spam-Score: 7.35 X-TUID: 7mMUf3stEeyQ From: Jakub Kądziołka * guix/build/utils.scm (substitute, substitute*) [require-matches?]: New argument. * tests/build-utils.scm ("substitute*"): New test group. ("substitute*, no match error") ("substitute*, partial no match error"): New tests. Co-authored-by: Maxim Cournoyer Change-Id: I66ed33d72aa73cd35e5642521efec70bf756f86e --- guix/build/utils.scm | 94 +++++++++++++++++++++++++++++++++---------- tests/build-utils.scm | 63 +++++++++++++++++++---------- 2 files changed, 114 insertions(+), 43 deletions(-) diff --git a/guix/build/utils.scm b/guix/build/utils.scm index 2b3a8e278b..7bfb6560e1 100644 --- a/guix/build/utils.scm +++ b/guix/build/utils.scm @@ -6,7 +6,8 @@ ;;; Copyright © 2018, 2022 Arun Isaac ;;; Copyright © 2018, 2019 Ricardo Wurmus ;;; Copyright © 2020 Efraim Flashner -;;; Copyright © 2020, 2021 Maxim Cournoyer +;;; Copyright © 2020 Jakub Kądziołka +;;; Copyright © 2020, 2021, 2023 Maxim Cournoyer ;;; Copyright © 2021, 2022 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley ;;; Copyright © 2022 Simon Tournier @@ -971,24 +972,53 @@ (define (replace-char c1 c2 s) c)) s))) -(define (substitute file pattern+procs) +(define* (substitute file pattern+procs #:key (require-matches? #t)) "PATTERN+PROCS is a list of regexp/two-argument-procedure pairs. For each line of FILE, and for each PATTERN that it matches, call the corresponding PROC as (PROC LINE MATCHES); PROC must return the line that will be written as a substitution of the original line. Be careful about using '$' to match the -end of a line; by itself it won't match the terminating newline of a line." - (let ((rx+proc (map (match-lambda - (((? regexp? pattern) . proc) +end of a line; by itself it won't match the terminating newline of a line. + +By default, SUBSTITUTE will raise a &message condition if one of the patterns +fails to match. REQUIRE-MATCHES? can be set to false when lack of matches is +acceptable (e.g. if you have multiple potential patterns not guaranteed to be +found in FILE)." + (define (rx->pattern m) + (match m + ((? regexp? pattern) + "") + ((? regexp*? pattern) + (regexp*-pattern pattern)) + ((? string? pattern) + pattern))) + + (define (make-condition failed-matches) + (condition + (&message + (message (format #f "substitute: `~a': no match for patterns `~a'" + file failed-matches))))) + + (let ((rx+proc (map (match-lambda + (((or (? regexp? pattern) (? regexp*? pattern)) . proc) (cons pattern proc)) ((pattern . proc) - (cons (make-regexp pattern regexp/extended) - proc))) - pattern+procs))) + (cons (make-regexp* pattern regexp/extended) proc))) + pattern+procs))) (with-atomic-file-replacement file (lambda (in out) - (let loop ((line (read-line in 'concat))) + (let loop ((line (read-line in 'concat)) + (ok-matches '()) + (failed-matches '())) (if (eof-object? line) - #t + (when require-matches? + (let ((failed-patterns (lset-difference + string=? + (delete-duplicates + (map rx->pattern failed-matches)) + (delete-duplicates + (map rx->pattern ok-matches))))) + (when (not (null? failed-patterns)) + (raise (make-condition failed-patterns))))) ;; Work around the fact that Guile's regexp-exec does not handle ;; NUL characters (a limitation of the underlying GNU libc's ;; regexec) by temporarily replacing them by an unused private @@ -998,19 +1028,30 @@ (define (substitute file pattern+procs) (unused-private-use-code-point line)) #\nul)) (line* (replace-char #\nul nul* line)) - (line1* (fold (lambda (r+p line) - (match r+p - ((regexp . proc) - (match (list-matches regexp line) - ((and m+ (_ _ ...)) - (proc line m+)) - (_ line))))) - line* - rx+proc)) + (results ;line, ok-matches and failed-matches + (fold (lambda (r+p results) + (let ((line (first results)) + (ok-matches (second results)) + (failed-matches (third results))) + (match r+p + ((regexp . proc) + (match (list-matches* regexp line) + ((and m+ (_ _ ...)) + (list (proc line m+) + (cons regexp ok-matches) + failed-matches)) + (_ + (list line + ok-matches + (cons regexp failed-matches)))))))) + (list line* '() '()) + rx+proc)) + (line1* (first results)) + (ok-matches (second results)) + (failed-matches (third results)) (line1 (replace-char nul* #\nul line1*))) (display line1 out) - (loop (read-line in 'concat))))))))) - + (loop (read-line in 'concat) ok-matches failed-matches)))))))) (define-syntax let-matches ;; Helper macro for `substitute*'. @@ -1048,9 +1089,17 @@ (define-syntax substitute* Alternatively, FILE may be a list of file names, in which case they are all subject to the substitutions. +By default, SUBSTITUTE* will raise a &message condition if one of the patterns +fails to match on one of the files; REQUIRE-MATCHES? may be set to false to +avoid an error being raised in such condition. + Be careful about using '$' to match the end of a line; by itself it won't match the terminating newline of a line." ((substitute* file ((regexp match-var ...) body ...) ...) + (substitute* file #:require-matches? #t + ((regexp match-var ...) body ...) ...)) + ((substitute* file #:require-matches? require-matches? + ((regexp match-var ...) body ...) ...) (let () (define (substitute-one-file file-name) (substitute @@ -1074,7 +1123,8 @@ (define-syntax substitute* (begin body ...) (substring l o (match:start m)) r)))))))) - ...))) + ...) + #:require-matches? require-matches?)) (match file ((files (... ...)) diff --git a/tests/build-utils.scm b/tests/build-utils.scm index 3babf5d544..890fbca16f 100644 --- a/tests/build-utils.scm +++ b/tests/build-utils.scm @@ -1,7 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2012, 2015, 2016, 2019, 2020 Ludovic Courtès ;;; Copyright © 2019 Ricardo Wurmus -;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021, 2023 Maxim Cournoyer ;;; Copyright © 2021 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley ;;; @@ -289,26 +289,47 @@ (define (arg-test bash-args) (test-assert "wrap-script, argument handling, bash --norc" (arg-test " --norc")) -(test-equal "substitute*, text contains a NUL byte, UTF-8" - "c\0d" - (with-fluids ((%default-port-encoding "UTF-8") - (%default-port-conversion-strategy 'error)) - ;; The GNU libc is locale sensitive. Depending on the value of LANG, the - ;; test could fail with "string contains #\\nul character: ~S" or "cannot - ;; convert wide string to output locale". - (setlocale LC_ALL "en_US.UTF-8") - (call-with-temporary-output-file - (lambda (file port) - (format port "a\0b") - (flush-output-port port) - - (substitute* file - (("a") "c") - (("b") "d")) - - (with-input-from-file file - (lambda _ - (get-string-all (current-input-port)))))))) +(test-group "substitute*" + (define-syntax-rule (define-substitute*-test test-type name expected + content clauses ...) + (test-type + name + expected + (with-fluids ((%default-port-encoding "UTF-8") + (%default-port-conversion-strategy 'error)) + ;; The GNU libc is locale sensitive. Depending on the value of LANG, + ;; the test could fail with "string contains #\\nul character: ~S" or + ;; "cannot convert wide string to output locale". + (setlocale LC_ALL "en_US.UTF-8") + (call-with-temporary-output-file + (lambda (file port) + (format port content) + (flush-output-port port) + + (substitute* file + clauses ...) + + (with-input-from-file file + (lambda _ + (get-string-all (current-input-port))))))))) + + (define-substitute*-test test-equal + "substitute*, text contains a NUL byte, UTF-8" + "c\0d" ;expected + "a\0b" ;content + (("a") "c") + (("b") "d")) + + (define-substitute*-test test-error "substitute*, no match error" + #t ;expected + "a\0b" ;content + (("Oops!") "c")) + + (define-substitute*-test test-error "substitute*, partial no match error" + #t ;expected + "a\0b" ;content + (("a") "c" + ("Oops!") "c"))) (test-equal "search-input-file: exception if not found" `((path) -- 2.41.0