unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: Efraim Flashner <efraim@flashner.co.il>
To: Ivan Petkov <ivanppetkov@gmail.com>
Cc: 36841@debbugs.gnu.org
Subject: [bug#36841] [PATCH v3] build/cargo-build-system: Patch cargo checksums.
Date: Tue, 30 Jul 2019 13:46:58 +0300	[thread overview]
Message-ID: <20190730104658.GC21431@E2140> (raw)
In-Reply-To: <20190730081757.GB21431@E2140>


[-- Attachment #1.1: Type: text/plain, Size: 2083 bytes --]

On Tue, Jul 30, 2019 at 11:17:57AM +0300, Efraim Flashner wrote:
> On Mon, Jul 29, 2019 at 06:44:31PM -0700, Ivan Petkov wrote:
> > Hi Efraim,
> > 
> > > On Jul 29, 2019, at 12:04 PM, Efraim Flashner <efraim@flashner.co.il> wrote:
> > > 
> > > +;; After patching the 'patch-generated-file-shebangs phase any vendored crates
> > > +;; will have a mismatch on their checksum.
> > > +(define* (patch-cargo-checksums #:key
> > > +                                (vendor-dir "guix-vendor")
> > > +                                #:allow-other-keys)
> > 
> > [snip]
> > 
> > > +    (replace 'install install)
> > > +    (add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
> > 
> > I can’t quite remember the order the phases run in off the top of my head. Would it be possible to
> > make the configure/checksum generation phase run after shebang-patching (or ensure the patching
> > happens first)? It would avoid having to checksum all the files twice that way…
> 
> The 'configure phase could be renamed the plop-vendored-crates-into-place
> phase. It actually can't come after the 'patch-generated-file-shebangs
> phase since then there won't be any vendored crates to patch.
> 
> If we remove the generate-checksums call from 'configure then there
> won't be a .cargo-checksum.json to remove and regenerate during the
> 'patch-cargo-checksums phase, so I've changed that to search for
> "Cargo.toml$" and not delete it. Not as robust as "for each top-level
> directory in the 'vendor-dir'", but should be good enough.
> 
This one I'm pretty happy with. The checksums are only generated twice
when there's a Cargo.lock file present and I've factored out the
function to generate all the checksums. When that's moved to (guix build
cargo-utils) it can be used by the rust compilers and icecat.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #1.2: 0001-build-cargo-build-system-Patch-cargo-checksums.patch --]
[-- Type: text/plain, Size: 4436 bytes --]

From d1252887da416b664090bfcee0d84729abb0fbc4 Mon Sep 17 00:00:00 2001
From: Efraim Flashner <efraim@flashner.co.il>
Date: Mon, 29 Jul 2019 22:01:05 +0300
Subject: [PATCH] build/cargo-build-system: Patch cargo checksums.

* guix/build/cargo-build-system.scm (generate-all-checksums): New
procedure.
(update-cargo-lock, patch-cargo-checksums): New phases.
(%standard-phases): Add 'update=cargo-lock after 'configure and
'patch-cargo-checksums after 'patch-generated-file-shebangs.
---
 guix/build/cargo-build-system.scm | 48 +++++++++++++++++++++++++++++--
 1 file changed, 45 insertions(+), 3 deletions(-)

diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index f38de16cf7..7d363a18a5 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2016 David Craven <david@craven.ch>
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;; Copyright © 2019 Ivan Petkov <ivanppetkov@gmail.com>
+;;; Copyright © 2019 Efraim Flashner <efraim@flashner.co.il>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -39,6 +40,21 @@
 ;;
 ;; Code:
 
+;; TODO: Move this to (guix build cargo-utils). Will cause a full rebuild
+;; of all rust compilers.
+
+(define (generate-all-checksums dir-name)
+  (for-each
+    (lambda (filename)
+      (let* ((dir (dirname filename))
+             (checksum-file (string-append dir "/.cargo-checksum.json")))
+        (when (file-exists? checksum-file) (delete-file checksum-file))
+        (display (string-append
+                   "patch-cargo-checksums: generate-checksums for "
+                   dir "\n"))
+        (generate-checksums dir)))
+    (find-files dir-name "Cargo.toml$")))
+
 (define (manifest-targets)
   "Extract all targets from the Cargo.toml manifest"
   (let* ((port (open-input-pipe "cargo read-manifest"))
@@ -94,8 +110,7 @@ Cargo.toml file present at its root."
               ;; so that we can generate any cargo checksums.
               ;; The --strip-components argument is needed to prevent creating
               ;; an extra directory within `crate-dir`.
-              (invoke "tar" "xvf" path "-C" crate-dir "--strip-components" "1")
-              (generate-checksums crate-dir)))))
+              (invoke "tar" "xvf" path "-C" crate-dir "--strip-components" "1")))))
     inputs)
 
   ;; Configure cargo to actually use this new directory.
@@ -121,6 +136,31 @@ directory = '" port)
   (setenv "CC" (string-append (assoc-ref inputs "gcc") "/bin/gcc"))
   #t)
 
+;; The Cargo.lock file tells the build system which crates are required for
+;; building and hardcodes their version and checksum.  In order to build with
+;; the inputs we provide, we need to recreate the file with our inputs.
+(define* (update-cargo-lock #:key
+                            (vendor-dir "guix-vendor")
+                            #:allow-other-keys)
+  "Regenerate the Cargo.lock file with the current build inputs."
+  (when (file-exists? "Cargo.lock")
+    (begin
+      ;; Unfortunately we can't generate a Cargo.lock file until the checksums
+      ;; are generated, so we have an extra round of generate-all-checksums here.
+      (generate-all-checksums vendor-dir)
+      (delete-file "Cargo.lock")
+      (invoke "cargo" "generate-lockfile")))
+  #t)
+
+;; After the 'patch-generated-file-shebangs phase any vendored crates who have
+;; their shebangs patched will have a mismatch on their checksum.
+(define* (patch-cargo-checksums #:key
+                                (vendor-dir "guix-vendor")
+                                #:allow-other-keys)
+  "Patch the checksums of the vendored crates after patching their shebangs."
+  (generate-all-checksums vendor-dir)
+  #t)
+
 (define* (build #:key
                 skip-build?
                 (cargo-build-flags '("--release"))
@@ -162,7 +202,9 @@ directory = '" port)
     (replace 'configure configure)
     (replace 'build build)
     (replace 'check check)
-    (replace 'install install)))
+    (replace 'install install)
+    (add-after 'configure 'update-cargo-lock update-cargo-lock)
+    (add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
 
 (define* (cargo-build #:key inputs (phases %standard-phases)
                       #:allow-other-keys #:rest args)
-- 
2.22.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

  reply	other threads:[~2019-07-30 10:48 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-29 19:04 [bug#36841] [PATCH] build/cargo-build-system: Patch cargo checksums Efraim Flashner
2019-07-30  1:44 ` Ivan Petkov
2019-07-30  5:59   ` bug#36841: " Efraim Flashner
2019-07-30  8:17   ` [bug#36841] " Efraim Flashner
2019-07-30 10:46     ` Efraim Flashner [this message]
2019-08-01  3:00       ` [bug#36841] [PATCH v3] " Ivan Petkov
2019-08-01 11:15         ` Efraim Flashner
2019-08-04  8:57           ` Efraim Flashner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190730104658.GC21431@E2140 \
    --to=efraim@flashner.co.il \
    --cc=36841@debbugs.gnu.org \
    --cc=ivanppetkov@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).