From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id GPHgAWkvs14NbwAA0tVLHw (envelope-from ) for ; Wed, 06 May 2020 21:43:05 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id iE4sOXQvs15yXwAA1q6Kng (envelope-from ) for ; Wed, 06 May 2020 21:43:16 +0000 Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:470:142::17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8832E940E99 for ; Wed, 6 May 2020 21:43:14 +0000 (UTC) Received: from localhost ([::1]:47076 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jWRox-0006nf-2G for larch@yhetil.org; Wed, 06 May 2020 17:43:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jWRok-0006m4-Bp for bug-guix@gnu.org; Wed, 06 May 2020 17:43:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:58165) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jWRok-0000Sv-31 for bug-guix@gnu.org; Wed, 06 May 2020 17:43:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jWRok-0005A5-1M for bug-guix@gnu.org; Wed, 06 May 2020 17:43:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#41028: Channel/inferior error with core-updates: Unbound variable: call-with-new-thread Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 06 May 2020 21:43:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41028 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Christopher Baines , Marius Bakke Received: via spool by 41028-submit@debbugs.gnu.org id=B41028.158880133419764 (code B ref 41028); Wed, 06 May 2020 21:43:01 +0000 Received: (at 41028) by debbugs.gnu.org; 6 May 2020 21:42:14 +0000 Received: from localhost ([127.0.0.1]:41478 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jWRnx-00058i-JH for submit@debbugs.gnu.org; Wed, 06 May 2020 17:42:14 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49458) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jWRnv-00058U-SB for 41028@debbugs.gnu.org; Wed, 06 May 2020 17:42:12 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36452) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jWRnq-000734-Io; Wed, 06 May 2020 17:42:06 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=34376 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jWRnp-0005wk-Bk; Wed, 06 May 2020 17:42:05 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87h7wymj8a.fsf@cbaines.net> Date: Wed, 06 May 2020 23:42:02 +0200 In-Reply-To: <87h7wymj8a.fsf@cbaines.net> (Christopher Baines's message of "Sat, 02 May 2020 16:47:49 +0100") Message-ID: <87a72kg2qd.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 41028@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 X-Spam-Score: -1.01 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 2001:470:142::17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Scan-Result: default: False [-1.01 / 13.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.49705419383529]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2001:470:142::/48:c]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.11), country: US(-0.00), ip: 2001:470:142::17(-0.50)]; DWL_DNSWL_FAIL(0.00)[2001:470:142::17:server fail]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; MAILLIST(-0.20)[mailman]; FREEMAIL_TO(0.00)[cbaines.net,fastmail.com]; RCVD_IN_DNSWL_FAIL(0.00)[2001:470:142::17:server fail]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:22989, ipnet:2001:470:142::/48, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[larch=yhetil.org]; ARC_NA(0.00)[]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; FROM_NEQ_ENVFROM(0.00)[ludo@gnu.org,bug-guix-bounces@gnu.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; URIBL_BLOCKED(0.00)[cbaines.net:email,gnu.org:email]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-patch]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+]; DMARC_NA(0.00)[gnu.org]; HAS_LIST_UNSUB(-0.01)[]; RCVD_COUNT_SEVEN(0.00)[9]; FORGED_SENDER_MAILLIST(0.00)[] X-TUID: aeSo4WPxpgc8 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Comrades, Christopher Baines skribis: > (use-modules (guix channels) > (guix inferior)) > > (define channels > (list (channel > (name 'guix) > (url "https://git.savannah.gnu.org/git/guix.git") > (commit > "e02c2f85b36ce1c733bd908a210ce1182bdd2560")))) > > (define inferior > (inferior-for-channels channels)) > > (first (lookup-inferior-packages inferior "linux-libre" "5.2.21")) > > > If you save that as a file then attempt to build it: > > > =E2=86=92 guix build -f test.scm=20 > Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.= org/git/guix.git'... > Backtrace: > 4 (primitive-load "/gnu/store/8mv5bpjgxg9c369xnbb5rf1kv9r?") > In ice-9/eval.scm: > 619:8 3 (_ #(#(#(#(#(#(#(#(#(#(#(?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?)) > 182:19 2 (proc #(#(#(#(#(#(#(#(#(#(# ?) ?) ?) ?) ?) ?) ?) ?) ?) ?)) > 142:16 1 (compile-top-call # ?) > In unknown file: > 0 (%resolve-variable (7 . call-with-new-thread) #) > > ERROR: In procedure %resolve-variable: > Unbound variable: call-with-new-thread The attached patches add a mechanism to patch the Guix source tree, and then use that mechanism to add the missing (ice-9 threads) import. With this I can do: ./pre-inst-env guix time-machine \ --commit=3De02c2f85b36ce1c733bd908a210ce1182bdd2560 -- build linux-lib= re =E2=80=A6 which is a simple way to do what the manifest above was about. If we think a bit long-term, I think the %quirks and %patches mechanisms are a necessary evil. The goal is for =E2=80=98build-self.scm=E2=80=99 to = use as little of the Guix API, as well as Guile APIs that are presumably here to stay. But as we=E2=80=99re already seeing, there are unavoidably incompatibilities that arise here and there. The good news is that we know today how to modify yesterday=E2=80=99s code to work well, or which Guile version to use= to get it to run. Thoughts? Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-channels-Add-latest-channel-instance.patch >From 4ec2c3b5527b2189b3f767a980f80dee3c6717ae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= Date: Wed, 6 May 2020 22:18:52 +0200 Subject: [PATCH 1/3] channels: Add 'latest-channel-instance'. * guix/channels.scm (latest-channel-instance): New procedure. (latest-channel-instances): Use it. --- guix/channels.scm | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/guix/channels.scm b/guix/channels.scm index 041fae2a9c..4ffc366d6a 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -199,6 +199,14 @@ description file or its default value." channel INSTANCE." (channel-metadata-dependencies (channel-instance-metadata instance))) +(define (latest-channel-instance store channel) + "Return the latest channel instance for CHANNEL." + (let-values (((checkout commit) + (latest-repository-commit store (channel-url channel) + #:ref (channel-reference + channel)))) + (channel-instance channel commit checkout))) + (define* (latest-channel-instances store channels #:optional (previous-channels '())) "Return a list of channel instances corresponding to the latest checkouts of CHANNELS and the channels on which they depend. PREVIOUS-CHANNELS is a list @@ -224,20 +232,16 @@ of previously processed channels." (G_ "Updating channel '~a' from Git repository at '~a'...~%") (channel-name channel) (channel-url channel)) - (let-values (((checkout commit) - (latest-repository-commit store (channel-url channel) - #:ref (channel-reference - channel)))) - (let ((instance (channel-instance channel commit checkout))) - (let-values (((new-instances new-channels) - (latest-channel-instances - store - (channel-instance-dependencies instance) - previous-channels))) - (values (append (cons channel new-channels) - previous-channels) - (append (cons instance new-instances) - instances)))))))) + (let ((instance (latest-channel-instance store channel))) + (let-values (((new-instances new-channels) + (latest-channel-instances + store + (channel-instance-dependencies instance) + previous-channels))) + (values (append (cons channel new-channels) + previous-channels) + (append (cons instance new-instances) + instances))))))) previous-channels '() ;instances channels)) -- 2.26.2 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0002-channels-Add-mechanism-to-patch-checkouts-of-the-gui.patch >From 18f2efd43b824f87b3d3a5dbcba9e8c363798f5b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= Date: Wed, 6 May 2020 22:45:31 +0200 Subject: [PATCH 2/3] channels: Add mechanism to patch checkouts of the 'guix channel. * guix/channels.scm (): New record type. (apply-patches): New procedure. (latest-channel-instance)[dot-git?]: New procedure. Use 'update-cached-checkout' and 'add-to-store' instead of 'latest-repository-commit'. Call 'apply-patches' when CHANNEL is the 'guix channel. (%patches): New variable. * guix/git.scm (url+commit->name): Make public. * tests/channels.scm ("latest-channel-instances includes channel dependencies") ("latest-channel-instances excludes duplicate channel dependencies"): Mock 'update-cached-checkout' instead of 'latest-repository-commit'. Wrap body in 'with-store' and pass the store to 'latest-channel-instances'. --- guix/channels.scm | 50 +++++++++++++++++++++++++++++++----- guix/git.scm | 1 + tests/channels.scm | 64 ++++++++++++++++++++++++---------------------- 3 files changed, 79 insertions(+), 36 deletions(-) diff --git a/guix/channels.scm b/guix/channels.scm index 4ffc366d6a..75b53c3a8e 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -199,13 +199,45 @@ description file or its default value." channel INSTANCE." (channel-metadata-dependencies (channel-instance-metadata instance))) -(define (latest-channel-instance store channel) +;; Patch to apply to a source tree. +(define-record-type + (patch predicate application) + patch? + (predicate patch-predicate) ;procedure + (application patch-application)) ;procedure + +(define (apply-patches checkout commit patches) + "Apply the matching PATCHES to CHECKOUT, modifying files in place. The +result is unspecified." + (let loop ((patches patches)) + (match patches + (() #t) + ((($ predicate modify) rest ...) + ;; PREDICATE is passed COMMIT so that it can choose to only apply to + ;; ancestors. + (when (predicate checkout commit) + (modify checkout)) + (loop rest))))) + +(define* (latest-channel-instance store channel + #:key (patches %patches)) "Return the latest channel instance for CHANNEL." + (define (dot-git? file stat) + (and (string=? (basename file) ".git") + (eq? 'directory (stat:type stat)))) + (let-values (((checkout commit) - (latest-repository-commit store (channel-url channel) - #:ref (channel-reference - channel)))) - (channel-instance channel commit checkout))) + (update-cached-checkout (channel-url channel) + #:ref (channel-reference channel)))) + (when (guix-channel? channel) + ;; Apply the relevant subset of PATCHES directly in CHECKOUT. This is + ;; safe to do because 'switch-to-ref' eventually does a hard reset. + (apply-patches checkout commit patches)) + + (let* ((name (url+commit->name (channel-url channel) commit)) + (checkout (add-to-store store name #t "sha256" checkout + #:select? (negate dot-git?)))) + (channel-instance channel commit checkout)))) (define* (latest-channel-instances store channels #:optional (previous-channels '())) "Return a list of channel instances corresponding to the latest checkouts of @@ -337,12 +369,18 @@ to '%package-module-path'." 'guile-2.2.4)) (define %quirks - ;; List of predicate/package pairs. This allows us provide information + ;; List of predicate/package pairs. This allows us to provide information ;; about specific Guile versions that old Guix revisions might need to use ;; just to be able to build and run the trampoline in %SELF-BUILD-FILE. See ;; `((,syscalls-reexports-local-variables? . ,guile-2.2.4))) +(define %patches + ;; Bits of past Guix revisions can become incompatible with newer Guix and + ;; Guile. This variable lists records for the Guix source tree that + ;; apply to the Guix source. + '()) + (define* (guile-for-source source #:optional (quirks %quirks)) "Return the Guile package to use when building SOURCE or #f if the default '%guile-for-build' should be good enough." diff --git a/guix/git.scm b/guix/git.scm index 5fffd429bd..92121156cf 100644 --- a/guix/git.scm +++ b/guix/git.scm @@ -40,6 +40,7 @@ with-repository update-cached-checkout + url+commit->name latest-repository-commit commit-difference diff --git a/tests/channels.scm b/tests/channels.scm index f5a7955483..910088ba15 100644 --- a/tests/channels.scm +++ b/tests/channels.scm @@ -135,44 +135,48 @@ (name 'test) (url "test"))) (test-dir (channel-instance-checkout instance--simple))) - (mock ((guix git) latest-repository-commit - (lambda* (store url #:key ref) + (mock ((guix git) update-cached-checkout + (lambda* (url #:key ref) (match url - ("test" (values test-dir 'whatever)) - (_ (values "/not-important" 'not-important))))) - (let ((instances (latest-channel-instances #f (list channel)))) - (and (eq? 2 (length instances)) - (lset= eq? - '(test test-channel) - (map (compose channel-name channel-instance-channel) - instances))))))) + ("test" (values test-dir "caf3cabba9e")) + (_ (values (channel-instance-checkout instance--no-deps) + "abcde1234"))))) + (with-store store + (let ((instances (latest-channel-instances store (list channel)))) + (and (eq? 2 (length instances)) + (lset= eq? + '(test test-channel) + (map (compose channel-name channel-instance-channel) + instances)))))))) (test-assert "latest-channel-instances excludes duplicate channel dependencies" (let* ((channel (channel (name 'test) (url "test"))) (test-dir (channel-instance-checkout instance--with-dupes))) - (mock ((guix git) latest-repository-commit - (lambda* (store url #:key ref) + (mock ((guix git) update-cached-checkout + (lambda* (url #:key ref) (match url - ("test" (values test-dir 'whatever)) - (_ (values "/not-important" 'not-important))))) - (let ((instances (latest-channel-instances #f (list channel)))) - (and (= 2 (length instances)) - (lset= eq? - '(test test-channel) - (map (compose channel-name channel-instance-channel) - instances)) - ;; only the most specific channel dependency should remain, - ;; i.e. the one with a specified commit. - (find (lambda (instance) - (and (eq? (channel-name - (channel-instance-channel instance)) - 'test-channel) - (string=? (channel-commit - (channel-instance-channel instance)) - "abc1234"))) - instances)))))) + ("test" (values test-dir "caf3cabba9e")) + (_ (values (channel-instance-checkout instance--no-deps) + "abcde1234"))))) + (with-store store + (let ((instances (latest-channel-instances store (list channel)))) + (and (= 2 (length instances)) + (lset= eq? + '(test test-channel) + (map (compose channel-name channel-instance-channel) + instances)) + ;; only the most specific channel dependency should remain, + ;; i.e. the one with a specified commit. + (find (lambda (instance) + (and (eq? (channel-name + (channel-instance-channel instance)) + 'test-channel) + (string=? (channel-commit + (channel-instance-channel instance)) + "abc1234"))) + instances))))))) (test-assert "channel-instances->manifest" ;; Compute the manifest for a graph of instances and make sure we get a -- 2.26.2 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0003-channels-Add-patch-for-https-bugs.gnu.org-41028.patch >From 66e2ef6ae0958fa45e14c3528c767cd84e24a480 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= Date: Wed, 6 May 2020 23:17:47 +0200 Subject: [PATCH 3/3] channels: Add patch for . Without this patch, we couldn't jump from here to commits before 05e783871c2c69b402e088863d46f5be7915ac74 because the 'compute-guix-derivation' script would crash with an unbound-variable error for 'call-with-new-thread'. Fixes . Reported by Christopher Baines . * guix/channels.scm (%bug-41028-patch): New variable. (%patches): Add it. --- guix/channels.scm | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/guix/channels.scm b/guix/channels.scm index 75b53c3a8e..0fa036446c 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -38,6 +38,7 @@ #:select (source-properties->location &error-location &fix-hint)) + #:use-module ((guix build utils) #:select (substitute*)) #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) #:use-module (srfi srfi-9) @@ -375,11 +376,35 @@ to '%package-module-path'." ;; `((,syscalls-reexports-local-variables? . ,guile-2.2.4))) + +(define %bug-41028-patch + ;; Patch for . The faulty code is the + ;; 'compute-guix-derivation' body, which uses 'call-with-new-thread' without + ;; importing (ice-9 threads). However, the 'call-with-new-thread' binding + ;; is no longer available in the default name space on Guile 3.0. + (let () + (define (missing-ice-9-threads-import? source commit) + ;; Return true if %SELF-BUILD-FILE is missing an (ice-9 threads) import. + (define content + (call-with-input-file (string-append source "/" %self-build-file) + read-string)) + + (and (string-contains content "(call-with-new-thread") + (not (string-contains content "(ice-9 threads)")))) + + (define (add-missing-ice-9-threads-import source) + ;; Add (ice-9 threads) import in the gexp of 'compute-guix-derivation'. + (substitute* (string-append source "/" %self-build-file) + (("^ +\\(use-modules \\(ice-9 match\\)\\)") + (object->string '(use-modules (ice-9 match) (ice-9 threads)))))) + + (patch missing-ice-9-threads-import? add-missing-ice-9-threads-import))) + (define %patches ;; Bits of past Guix revisions can become incompatible with newer Guix and ;; Guile. This variable lists records for the Guix source tree that ;; apply to the Guix source. - '()) + (list %bug-41028-patch)) (define* (guile-for-source source #:optional (quirks %quirks)) "Return the Guile package to use when building SOURCE or #f if the default -- 2.26.2 --=-=-=--