From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id AFzLKCnqQWLZ4gAAgWs5BA (envelope-from ) for ; Mon, 28 Mar 2022 19:02:33 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id uD+6JSnqQWKbTQAAauVa8A (envelope-from ) for ; Mon, 28 Mar 2022 19:02:33 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3C6F5E955 for ; Mon, 28 Mar 2022 19:02:33 +0200 (CEST) Received: from localhost ([::1]:56258 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nYslD-0000Hp-Jb for larch@yhetil.org; Mon, 28 Mar 2022 13:02:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39962) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nYWFH-0004hp-Rc for bug-guix@gnu.org; Sun, 27 Mar 2022 13:00:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:34357) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nYWFH-0008SV-Gk for bug-guix@gnu.org; Sun, 27 Mar 2022 13:00:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nYWFH-0005uO-Fj for bug-guix@gnu.org; Sun, 27 Mar 2022 13:00:03 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#51466: [PATCH 1/1] environment: properly parse environment variables during --check Resent-From: Kevin Boulain Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 27 Mar 2022 17:00:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51466 X-GNU-PR-Package: guix X-GNU-PR-Keywords: moreinfo To: 51466@debbugs.gnu.org Received: via spool by 51466-submit@debbugs.gnu.org id=B51466.164840036922631 (code B ref 51466); Sun, 27 Mar 2022 17:00:03 +0000 Received: (at 51466) by debbugs.gnu.org; 27 Mar 2022 16:59:29 +0000 Received: from localhost ([127.0.0.1]:56486 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nYWEj-0005sx-30 for submit@debbugs.gnu.org; Sun, 27 Mar 2022 12:59:29 -0400 Received: from mail-ej1-f51.google.com ([209.85.218.51]:39771) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nYWEf-0005sR-O1 for 51466@debbugs.gnu.org; Sun, 27 Mar 2022 12:59:26 -0400 Received: by mail-ej1-f51.google.com with SMTP id dr20so24208355ejc.6 for <51466@debbugs.gnu.org>; Sun, 27 Mar 2022 09:59:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hh+/yP0VKKJunTnXq4xDB/1DhEK360hpOM44ZzirqOQ=; b=iLfx1r3KTEeotus7cz0D0Gy93uft+sBi0E8qdyE553d6CPY7o8zGZk+YqYa5ICxuWl 3wmkITesjT0edl9HkwZaUMGoS2M8NEPTLEDYxhoA9f3v3GQKaPiuyk1fRzgKiEcv1vZi TfiXhTiFnYwRn/QAwVN7c3Izk97EZXXvZpcyx3ZRWHaFb9JiH4JHLHSj+Gn9HI53xDtG YKDt1lwgJtHUIBcA4a8yt21YXuFbsAuMDaZNS/cl15oE5Gvy7FkoJEgh7kA9Ee49sISF 48nYacRcUoKjKs//Tk2mFYEVzLpuNC8YprGAlUyxEhAxvSqiiq+LFmbhOkj2zNOQsTCR ZmhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hh+/yP0VKKJunTnXq4xDB/1DhEK360hpOM44ZzirqOQ=; b=6vgxwDp8CMANoLT5dfKKImQvVurX5L3ZJW3llKVhZgCaB1G5l8psroco/OY+lHk3vy NqpbTFKBCnPjIu+7Yc9/dPKM2LVRLkm3cIUJY7fTIZ+M2duJyfiRecGiUqCznOa9Z5a8 i40KUiR+x9OwGcB/ryS6pNr/ZB9u5L16zcPXAvjmTNqFAGCO9lI0GyTWFbRxrZUUjr+7 l4eaH81HUQNJA8uJlEmprmqLOt+0UpXq0nqgoDo/YqpUhR7jle4USsVfbTZs1YXgJr7x ddJ7447Owpqb713pCrGahFD8cBzTI3Xmgc2nmwtKEymB/1iFhA0rgqiuWFkbM+XWqbuQ o+hg== X-Gm-Message-State: AOAM530vRZ3V9w93DxAYHugeg4G0FnZR55yQ32a490Rv6+89m+w7KtZP GB3ni7MPzcO/o1Q3YEYD8CNwKl659BQ= X-Google-Smtp-Source: ABdhPJxZ8EnEc88oxiRfZRs/ePai28lEZd2NIEv+OFRbV3TnfldkDAQwCVa2SehVoxzMaAzClX4Usg== X-Received: by 2002:a17:907:72cc:b0:6e0:2d3:bcba with SMTP id du12-20020a17090772cc00b006e002d3bcbamr22772349ejc.642.1648400359745; Sun, 27 Mar 2022 09:59:19 -0700 (PDT) Received: from localhost.localdomain ([51.154.70.17]) by smtp.gmail.com with ESMTPSA id q2-20020a170906144200b006ceb8723de9sm4786243ejc.120.2022.03.27.09.59.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Mar 2022 09:59:19 -0700 (PDT) From: Kevin Boulain Date: Sun, 27 Mar 2022 19:01:24 +0200 Message-Id: <20220327170124.2846-2-kevinboulain@gmail.com> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20220327170124.2846-1-kevinboulain@gmail.com> References: <20220327170124.2846-1-kevinboulain@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Mailman-Approved-At: Mon, 28 Mar 2022 13:02:19 -0400 X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Boulain Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1648486953; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=hh+/yP0VKKJunTnXq4xDB/1DhEK360hpOM44ZzirqOQ=; b=FC8X2C9wedFEQMiSiIvkpJ8+dMa+d5KX8RRmnJNJcYJHoUw0RHhOqzQQlPT1gQvVQRfzaO p/58ugVJn8mJvmzsudWMjkjPDm398IX/mogl16m2HopZSqZll76Ufrs1Z2Lk+wCFJsOPys cQzhDlJCFCUvCESRTlwRnHRS6WqjQR8TpM1fM9aXvT5MrDPW5MvrT5PKtUdUROQyfUNuJ2 YC8aUcIKwIh3y50rMOWivekoQuILpc56PdnIcmlK07cPcTCyATkWheu+ndRl3Uw0w0KFNg OVwqf8DpjCjOTM93FmVaMIQ+cQftx8WeOPfHjy6Xjb/lhl28n/lciAKlOSDzYA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1648486953; a=rsa-sha256; cv=none; b=gs6zBlNnWxPL304tz4ZHCwt+rFDQ97GNl818tAdA7RxUxA5XLDvB16YH5OHNAPB/iQvUcz 5olUJztdCMCAx5YRfMWW6XXPf+QpU+HoO/lJTeAz1uKgDOz6RUIvOTo3gyDfNBzG4Zkjd5 m6G5cCSJcRsHV+N7bHsiSQx4KsQTd1MHKbLZtj73Np4szQlo45dSIxxd1yUzY+/i61VX6W ZFljNITCqkI8zF1nsgbdEzUPi9fCYswyEHjuGCiyNWPdICYWEm4eP0TRFa+finLzc7Jlev fsRiOxRBcO9MQ70idGq2olmG4nVMzKs5EEzF2pn0nYs44wcuCsFp+XTkNv24GA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=iLfx1r3K; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 7.43 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=iLfx1r3K; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3C6F5E955 X-Spam-Score: 7.43 X-Migadu-Scanner: scn0.migadu.com X-TUID: KEykNVJU/Gat Should solve https://issues.guix.gnu.org/51466. This redirects the env command's output to a temporary file so that there is no way to get it mixed with the shell's output (like PS1). Additionnally: - Remove GUIX-CHECK-DONE and read: I don't think there is a need for them as discarding the output until the shell exits is enough to guarantee the environment has been dumped. Sadly, Linux doesn't report EOF but EIO when the shell exits and the pty gets closed hence the catch/throw. - Don't try to parse environment variables by splitting them on \n, instead use env's -0 option to separate environment variables with \0. This prevent incorrect parsing of some multi-line variables (e.g.: f() { echo "hello"; }; declare -fx f) but isn't standard. * guix/scripts/environment.scm (child-shell-environment): make environment variables parsing more robust. * tests/guix-environment.sh: add a simple test that was reliably failing on my machine. --- guix/scripts/environment.scm | 65 +++++++++++++++++++++++------------- tests/guix-environment.sh | 16 +++++++++ 2 files changed, 57 insertions(+), 24 deletions(-) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index ec071402f4..4ff13c6bde 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -48,7 +48,7 @@ (define-module (guix scripts environment) #:autoload (gnu packages bash) (bash) #:autoload (gnu packages bootstrap) (bootstrap-executable %bootstrap-guile) #:use-module (ice-9 match) - #:autoload (ice-9 rdelim) (read-line) + #:autoload (ice-9 rdelim) (read-delimited read-line) #:use-module (ice-9 vlist) #:use-module (srfi srfi-1) #:use-module (srfi srfi-11) @@ -413,16 +413,22 @@ (define (child-shell-environment shell profile manifest) "Create a child process, load PROFILE and MANIFEST, and then run SHELL in interactive mode in it. Return a name/value vhash for all the variables shown by running 'set' in the shell." - (define-values (controller inferior) - (openpty)) - - (define script - ;; Script to obtain the list of environment variable values. On a POSIX - ;; shell we can rely on 'set', but on fish we have to use 'env' (fish's - ;; 'set' truncates values and prints them in a different format.) - "env || /usr/bin/env || set; echo GUIX-CHECK-DONE; read x; exit\n") - (define lines + (let* ((environment-port (mkstemp "/tmp/guix-enviroment-XXXXXX")) + (environment-file (port-filename environment-port)) + ;; Script to obtain the list of environment variable values. On a + ;; POSIX shell we can rely on 'set', but on fish we have to use 'env' + ;; (fish's 'set' truncates values and prints them in a different + ;; format.) + ;; We rely on env --null to mark the end of environment variables. + ;; This is expected to be safe because POSIX defines environment + ;; variables end with '\0' (but does't document the --null option). + ;; Some shells, like Bash, allow to export functions, which will span + ;; multiple lines and break any trivial parsing relying on '\n'. + (script (format #f "env --null > ~a || /usr/bin/env --null > ~a; exit\n" + environment-file environment-file))) + + (let-values (((controller inferior) (openpty))) (match (primitive-fork) (0 (catch #t @@ -436,26 +442,37 @@ (define lines (primitive-exit 127)))) (pid (close-fdes inferior) - (let* ((port (fdopen controller "r+l")) - (result (begin + (let ((port (fdopen controller "r+l"))) (display script port) + (catch 'system-error + (lambda () + ;; We aren't interested in the output of the shell itself, + ;; drop it. + (while (not (eof-object? (read-line port))))) + (lambda args + (let ((errno (system-error-errno args))) + (cond + ((= errno EIO) + ;; On Linux, a read won't return EOF but will fail with EIO + ;; when the device is closed: + ;; https://bugs.python.org/issue5380#msg252544 + #t) + (#t + (apply throw args)))))) + (close-port port) + (waitpid pid))))) + + (let ((result (begin (let loop ((lines '())) - (match (read-line port) + (match (read-delimited "\0" environment-port) ((? eof-object?) (reverse lines)) - ("GUIX-CHECK-DONE\r" - (display "done\n" port) - (reverse lines)) (line - ;; Drop the '\r' from LINE. - (loop (cons (string-drop-right line 1) - lines)))))))) - (close-port port) - (waitpid pid) - result)))) + (loop (cons line lines)))))))) + (close-port environment-port) + (delete-file environment-file) + result))) (fold (lambda (line table) - ;; Note: 'set' in fish outputs "NAME VALUE" instead of "NAME=VALUE" - ;; but it also truncates values anyway, so don't try to support it. (let ((index (string-index line #\=))) (if index (vhash-cons (string-take line index) diff --git a/tests/guix-environment.sh b/tests/guix-environment.sh index 95fe95b437..114bf9fbf5 100644 --- a/tests/guix-environment.sh +++ b/tests/guix-environment.sh @@ -258,3 +258,19 @@ then guix gc --references "$profile" | grep "$dep" done fi + +# https://issues.guix.gnu.org/51466 +# guix environment --check was sometimes unable to find PKG_CONFIG_PATH +# because the env command is sent before the prompt gets printed and there +# is no proper way to deinterleave streams: +# ;;; (read-line "env || /usr/bin/env || set; echo GUIX-CHECK-DONE; read x; exit\r") +# ;;; (read-line "bash-5.1$ PKG_CONFIG_PATH=/gnu/store/0ysl5arpf0yf3pn15afr450k676lgdq3-profile/lib/pkgconfig\r") +# ;;; (read-line "PWD=/home/ether/source/guix\r") +bash=$(mktemp) +cat > "$bash" << 'BASH' +#!/usr/bin/env bash +exec bash --noprofile --rcfile /dev/null "$@" +BASH +chmod +x "$bash" +env SHELL=$bash guix environment --check --pure guix -- env +rm -f "$bash"