From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:45747) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hsPfu-0008PH-MP for guix-patches@gnu.org; Tue, 30 Jul 2019 06:48:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hsPfs-0005ZY-K9 for guix-patches@gnu.org; Tue, 30 Jul 2019 06:48:10 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:40791) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hsPfp-0005YS-5s for guix-patches@gnu.org; Tue, 30 Jul 2019 06:48:06 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hsPfm-000406-CW for guix-patches@gnu.org; Tue, 30 Jul 2019 06:48:05 -0400 Subject: [bug#36841] [PATCH v3] build/cargo-build-system: Patch cargo checksums. Resent-Message-ID: Date: Tue, 30 Jul 2019 13:46:58 +0300 From: Efraim Flashner Message-ID: <20190730104658.GC21431@E2140> References: <20190729190422.6834-1-efraim@flashner.co.il> <20190730081757.GB21431@E2140> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="pY3vCvL1qV+PayAL" Content-Disposition: inline In-Reply-To: <20190730081757.GB21431@E2140> List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Ivan Petkov Cc: 36841@debbugs.gnu.org --pY3vCvL1qV+PayAL Content-Type: multipart/mixed; boundary="at6+YcpfzWZg/htY" Content-Disposition: inline --at6+YcpfzWZg/htY Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 30, 2019 at 11:17:57AM +0300, Efraim Flashner wrote: > On Mon, Jul 29, 2019 at 06:44:31PM -0700, Ivan Petkov wrote: > > Hi Efraim, > >=20 > > > On Jul 29, 2019, at 12:04 PM, Efraim Flashner = wrote: > > >=20 > > > +;; After patching the 'patch-generated-file-shebangs phase any vendo= red crates > > > +;; will have a mismatch on their checksum. > > > +(define* (patch-cargo-checksums #:key > > > + (vendor-dir "guix-vendor") > > > + #:allow-other-keys) > >=20 > > [snip] > >=20 > > > + (replace 'install install) > > > + (add-after 'patch-generated-file-shebangs 'patch-cargo-checksums= patch-cargo-checksums))) > >=20 > > I can=E2=80=99t quite remember the order the phases run in off the top = of my head. Would it be possible to > > make the configure/checksum generation phase run after shebang-patching= (or ensure the patching > > happens first)? It would avoid having to checksum all the files twice t= hat way=E2=80=A6 >=20 > The 'configure phase could be renamed the plop-vendored-crates-into-place > phase. It actually can't come after the 'patch-generated-file-shebangs > phase since then there won't be any vendored crates to patch. >=20 > If we remove the generate-checksums call from 'configure then there > won't be a .cargo-checksum.json to remove and regenerate during the > 'patch-cargo-checksums phase, so I've changed that to search for > "Cargo.toml$" and not delete it. Not as robust as "for each top-level > directory in the 'vendor-dir'", but should be good enough. >=20 This one I'm pretty happy with. The checksums are only generated twice when there's a Cargo.lock file present and I've factored out the function to generate all the checksums. When that's moved to (guix build cargo-utils) it can be used by the rust compilers and icecat. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --at6+YcpfzWZg/htY Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="0001-build-cargo-build-system-Patch-cargo-checksums.patch" Content-Transfer-Encoding: quoted-printable =46rom d1252887da416b664090bfcee0d84729abb0fbc4 Mon Sep 17 00:00:00 2001 =46rom: Efraim Flashner Date: Mon, 29 Jul 2019 22:01:05 +0300 Subject: [PATCH] build/cargo-build-system: Patch cargo checksums. * guix/build/cargo-build-system.scm (generate-all-checksums): New procedure. (update-cargo-lock, patch-cargo-checksums): New phases. (%standard-phases): Add 'update=3Dcargo-lock after 'configure and 'patch-cargo-checksums after 'patch-generated-file-shebangs. --- guix/build/cargo-build-system.scm | 48 +++++++++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 3 deletions(-) diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-sys= tem.scm index f38de16cf7..7d363a18a5 100644 --- a/guix/build/cargo-build-system.scm +++ b/guix/build/cargo-build-system.scm @@ -2,6 +2,7 @@ ;;; Copyright =C2=A9 2016 David Craven ;;; Copyright =C2=A9 2017 Mathieu Othacehe ;;; Copyright =C2=A9 2019 Ivan Petkov +;;; Copyright =C2=A9 2019 Efraim Flashner ;;; ;;; This file is part of GNU Guix. ;;; @@ -39,6 +40,21 @@ ;; ;; Code: =20 +;; TODO: Move this to (guix build cargo-utils). Will cause a full rebuild +;; of all rust compilers. + +(define (generate-all-checksums dir-name) + (for-each + (lambda (filename) + (let* ((dir (dirname filename)) + (checksum-file (string-append dir "/.cargo-checksum.json"))) + (when (file-exists? checksum-file) (delete-file checksum-file)) + (display (string-append + "patch-cargo-checksums: generate-checksums for " + dir "\n")) + (generate-checksums dir))) + (find-files dir-name "Cargo.toml$"))) + (define (manifest-targets) "Extract all targets from the Cargo.toml manifest" (let* ((port (open-input-pipe "cargo read-manifest")) @@ -94,8 +110,7 @@ Cargo.toml file present at its root." ;; so that we can generate any cargo checksums. ;; The --strip-components argument is needed to prevent crea= ting ;; an extra directory within `crate-dir`. - (invoke "tar" "xvf" path "-C" crate-dir "--strip-components"= "1") - (generate-checksums crate-dir))))) + (invoke "tar" "xvf" path "-C" crate-dir "--strip-components"= "1"))))) inputs) =20 ;; Configure cargo to actually use this new directory. @@ -121,6 +136,31 @@ directory =3D '" port) (setenv "CC" (string-append (assoc-ref inputs "gcc") "/bin/gcc")) #t) =20 +;; The Cargo.lock file tells the build system which crates are required for +;; building and hardcodes their version and checksum. In order to build w= ith +;; the inputs we provide, we need to recreate the file with our inputs. +(define* (update-cargo-lock #:key + (vendor-dir "guix-vendor") + #:allow-other-keys) + "Regenerate the Cargo.lock file with the current build inputs." + (when (file-exists? "Cargo.lock") + (begin + ;; Unfortunately we can't generate a Cargo.lock file until the check= sums + ;; are generated, so we have an extra round of generate-all-checksum= s here. + (generate-all-checksums vendor-dir) + (delete-file "Cargo.lock") + (invoke "cargo" "generate-lockfile"))) + #t) + +;; After the 'patch-generated-file-shebangs phase any vendored crates who = have +;; their shebangs patched will have a mismatch on their checksum. +(define* (patch-cargo-checksums #:key + (vendor-dir "guix-vendor") + #:allow-other-keys) + "Patch the checksums of the vendored crates after patching their shebang= s." + (generate-all-checksums vendor-dir) + #t) + (define* (build #:key skip-build? (cargo-build-flags '("--release")) @@ -162,7 +202,9 @@ directory =3D '" port) (replace 'configure configure) (replace 'build build) (replace 'check check) - (replace 'install install))) + (replace 'install install) + (add-after 'configure 'update-cargo-lock update-cargo-lock) + (add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch= -cargo-checksums))) =20 (define* (cargo-build #:key inputs (phases %standard-phases) #:allow-other-keys #:rest args) --=20 2.22.0 --at6+YcpfzWZg/htY-- --pY3vCvL1qV+PayAL Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl1AIBkACgkQQarn3Mo9 g1GACA//e4dcPbOOFaUhlmGrBYhtHAWOQwP+EbIekBDrVjWVRHR2HZo8CJy/lvDG 8Xy04HTNPQBP3U1HRuU9jZWKOW+LLNKIWoJrLURcPjP6EcuCA7dnEXXG0qjk/wNk pa6yBI4TS62nHa0dB6SFWynUELBi2BVhjbUfKqlWvsnZ9A68A3QfJxc2A4AJJO/A iA+eTREkRWslils7KHmGoTdhcFCL17FTXBkX/xGVkO3hF6YVKJ1DCIgbt9lZONnP vn9bjmjNSu38ut6K5lJf0uI11YcByoArhZIL19r3ZJ7a6iD7Fe+oQMMsunuVjJ7t DuQLQdMCnOjoqNxSgmLGc/B/cFAn3FR8vIPY5hnMKQC2kkjH0kbpLqKG6hhWFfR0 +fI7oqIfppO5J61+t8odp7umZnporxdxXt8be7fRbYZBewRuRNY59EBdk3qQItOf desamHFqIpeuNWSGAZF82KU5vEx4OcPKHgkhgz4d+XnDJI/WrTbPN3Oif5Ggu5fM +qwseoFvPSwOraxwxlUz6N2Z7X+rW1dA3SRFBq3Wza4ox3OAhB0RsyX+ORE2qmWY Ik3Ch6sFA8QgUODgGNAD2t2swumQG6123HlmHkEt2GZYzLPE+bgKE5sW81rkE/zy wbIMWQ+bwV6rTBsG1RpzDUyGaofIYJ9VvW67HZcJtagwhz8OUZI= =mXxC -----END PGP SIGNATURE----- --pY3vCvL1qV+PayAL--