[bug#36841] [PATCH v3] build/cargo-build-system: Patch cargo checksums.

[Efraim Flashner <efraim@flashner.co.il>]

[-- Attachment #1.1: Type: text/plain, Size: 2083 bytes --]

On Tue, Jul 30, 2019 at 11:17:57AM +0300, Efraim Flashner wrote:
> On Mon, Jul 29, 2019 at 06:44:31PM -0700, Ivan Petkov wrote:
> > Hi Efraim,
> > 
> > > On Jul 29, 2019, at 12:04 PM, Efraim Flashner <efraim@flashner.co.il> wrote:
> > > 
> > > +;; After patching the 'patch-generated-file-shebangs phase any vendored crates
> > > +;; will have a mismatch on their checksum.
> > > +(define* (patch-cargo-checksums #:key
> > > +                                (vendor-dir "guix-vendor")
> > > +                                #:allow-other-keys)
> > 
> > [snip]
> > 
> > > +    (replace 'install install)
> > > +    (add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
> > 
> > I can’t quite remember the order the phases run in off the top of my head. Would it be possible to
> > make the configure/checksum generation phase run after shebang-patching (or ensure the patching
> > happens first)? It would avoid having to checksum all the files twice that way…
> 
> The 'configure phase could be renamed the plop-vendored-crates-into-place
> phase. It actually can't come after the 'patch-generated-file-shebangs
> phase since then there won't be any vendored crates to patch.
> 
> If we remove the generate-checksums call from 'configure then there
> won't be a .cargo-checksum.json to remove and regenerate during the
> 'patch-cargo-checksums phase, so I've changed that to search for
> "Cargo.toml$" and not delete it. Not as robust as "for each top-level
> directory in the 'vendor-dir'", but should be good enough.
> 
This one I'm pretty happy with. The checksums are only generated twice
when there's a Cargo.lock file present and I've factored out the
function to generate all the checksums. When that's moved to (guix build
cargo-utils) it can be used by the rust compilers and icecat.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #1.2: 0001-build-cargo-build-system-Patch-cargo-checksums.patch --]
[-- Type: text/plain, Size: 4436 bytes --]

From d1252887da416b664090bfcee0d84729abb0fbc4 Mon Sep 17 00:00:00 2001
From: Efraim Flashner <efraim@flashner.co.il>
Date: Mon, 29 Jul 2019 22:01:05 +0300
Subject: [PATCH] build/cargo-build-system: Patch cargo checksums.

* guix/build/cargo-build-system.scm (generate-all-checksums): New
procedure.
(update-cargo-lock, patch-cargo-checksums): New phases.
(%standard-phases): Add 'update=cargo-lock after 'configure and
'patch-cargo-checksums after 'patch-generated-file-shebangs.
---
 guix/build/cargo-build-system.scm | 48 +++++++++++++++++++++++++++++--
 1 file changed, 45 insertions(+), 3 deletions(-)

diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index f38de16cf7..7d363a18a5 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2016 David Craven <david@craven.ch>
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;; Copyright © 2019 Ivan Petkov <ivanppetkov@gmail.com>
+;;; Copyright © 2019 Efraim Flashner <efraim@flashner.co.il>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -39,6 +40,21 @@
 ;;
 ;; Code:
 
+;; TODO: Move this to (guix build cargo-utils). Will cause a full rebuild
+;; of all rust compilers.
+
+(define (generate-all-checksums dir-name)
+  (for-each
+    (lambda (filename)
+      (let* ((dir (dirname filename))
+             (checksum-file (string-append dir "/.cargo-checksum.json")))
+        (when (file-exists? checksum-file) (delete-file checksum-file))
+        (display (string-append
+                   "patch-cargo-checksums: generate-checksums for "
+                   dir "\n"))
+        (generate-checksums dir)))
+    (find-files dir-name "Cargo.toml$")))
+
 (define (manifest-targets)
   "Extract all targets from the Cargo.toml manifest"
   (let* ((port (open-input-pipe "cargo read-manifest"))
@@ -94,8 +110,7 @@ Cargo.toml file present at its root."
               ;; so that we can generate any cargo checksums.
               ;; The --strip-components argument is needed to prevent creating
               ;; an extra directory within `crate-dir`.
-              (invoke "tar" "xvf" path "-C" crate-dir "--strip-components" "1")
-              (generate-checksums crate-dir)))))
+              (invoke "tar" "xvf" path "-C" crate-dir "--strip-components" "1")))))
     inputs)
 
   ;; Configure cargo to actually use this new directory.
@@ -121,6 +136,31 @@ directory = '" port)
   (setenv "CC" (string-append (assoc-ref inputs "gcc") "/bin/gcc"))
   #t)
 
+;; The Cargo.lock file tells the build system which crates are required for
+;; building and hardcodes their version and checksum.  In order to build with
+;; the inputs we provide, we need to recreate the file with our inputs.
+(define* (update-cargo-lock #:key
+                            (vendor-dir "guix-vendor")
+                            #:allow-other-keys)
+  "Regenerate the Cargo.lock file with the current build inputs."
+  (when (file-exists? "Cargo.lock")
+    (begin
+      ;; Unfortunately we can't generate a Cargo.lock file until the checksums
+      ;; are generated, so we have an extra round of generate-all-checksums here.
+      (generate-all-checksums vendor-dir)
+      (delete-file "Cargo.lock")
+      (invoke "cargo" "generate-lockfile")))
+  #t)
+
+;; After the 'patch-generated-file-shebangs phase any vendored crates who have
+;; their shebangs patched will have a mismatch on their checksum.
+(define* (patch-cargo-checksums #:key
+                                (vendor-dir "guix-vendor")
+                                #:allow-other-keys)
+  "Patch the checksums of the vendored crates after patching their shebangs."
+  (generate-all-checksums vendor-dir)
+  #t)
+
 (define* (build #:key
                 skip-build?
                 (cargo-build-flags '("--release"))
@@ -162,7 +202,9 @@ directory = '" port)
     (replace 'configure configure)
     (replace 'build build)
     (replace 'check check)
-    (replace 'install install)))
+    (replace 'install install)
+    (add-after 'configure 'update-cargo-lock update-cargo-lock)
+    (add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
 
 (define* (cargo-build #:key inputs (phases %standard-phases)
                       #:allow-other-keys #:rest args)
-- 
2.22.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]