reusing rust build artifacts

reusing rust build artifacts

[Efraim Flashner]

[-- Attachment #1.1: Type: text/plain, Size: 2857 bytes --]

I took another look at reusing rust build artifacts. I figured this time
it would be best to see if there was even overlap that would help.

The concept is simple, when you make changes to your code you should
only need to recompile the parts you made changes too. Not that much
difference from touching a few modules in Guix and re-running 'make'.

I looked at rust-aho-corasick-0.6 and rust-unicode-segmentation-1.6 and
1.3. All three of them use rust-quickcheck-0.7 as
cargo-development-inputs and therefore seemed like a worthwhile check to
see if they even produce the same artifacts.

The build artifacts produce a long string of letters and numbers to let
the Well Informed Reader™ know which features are included in the
artifact.

Rust-unicode-segmentation-1.6 and 1.3 each produced identical artifacts
for rust-quickcheck-0.7. This means that IF we were caching the
artifacts (for use with packages which depend on them) AND IF
rust-unicode-segmentation-1.6 depended on 1.3 (or the other way around)
THEN we could reuse the compiled rust-quickcheck-0.7. Nice to know that
it works, but on its own not useful.

rust-aho-corasick-0.6 and rust-unicode-segmentation-1.6 did not produce
the same rust-quickcheck-0.6 artifacts. A check of their Cargo.toml
files¹² shows that rust-aho-corasick-0.6 uses the
'default-features=false' build flag for rust-quickcheck-0.7 and
rust-unicode-segmentation-1.6 doesn't. Checking rust-quickcheck-0.7's
Cargo.toml³ shows that there are a number of different feature options.

Realizing I made a mistake here, I compared the 'target' directory from
rust-unicode-segmentation-1.6 and rust-quickcheck-0.7. By default we
build all creates with the 'default' feature set, so it turns out that
the 'target/release/deps' folder and the 'target/release/build' folder
are almost identical.

Given the above statement about the default feature set, it is likely
worthwhile to try to copy the 'target' directory to an output and then
copy it back into place for the next package which needs it as an input.

For comparison, on my (old) machine, building
rust-unicode-segmentation-1.6 takes 185 seconds. When I copy the output
from rust-quickcheck-0.7 to the 'target' directory this drops down to
195 seconds.

I'm still investigating why it is rebuilding all the bits it's already
built. I've attached a diff with what I've been working with.


¹ https://github.com/BurntSushi/aho-corasick/blob/0.6.10/Cargo.toml
² https://github.com/unicode-rs/unicode-segmentation/blob/v1.6.0/Cargo.toml
³ https://github.com/BurntSushi/quickcheck/blob/0.7.2/Cargo.toml

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #1.2: reuse-cargo-build-artifacts.diff --]
[-- Type: text/plain, Size: 2675 bytes --]

diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 0f0f0c28c9..4495e6422b 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -28021,7 +28021,22 @@ described in Unicode Standard Annex #15.")
     (build-system cargo-build-system)
     (arguments
      `(#:cargo-development-inputs
-       (("rust-quickcheck" ,rust-quickcheck-0.7))))
+       (("rust-quickcheck" ,rust-quickcheck-0.7))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'copy-target-into-place
+           (lambda* (#:key inputs #:allow-other-keys)
+             (copy-recursively (string-append (assoc-ref inputs "rust-quickcheck-0.7")
+                                              "/target")
+                               "target")
+             (for-each make-file-writable (find-files "target" "."))
+             ;; Change from the previous build(s) to the current directory.
+             (substitute* (find-files "target" "\\.d$")
+               (("/tmp/.*/target") (string-append (getcwd) "/target")))
+             #t)))
+       ))
+    (inputs
+     `(("rust-quickcheck-0.7" ,rust-quickcheck-0.7)))
     (home-page "https://github.com/unicode-rs/unicode-segmentation")
     (synopsis "Grapheme Cluster, Word and Sentence boundaries")
     (description
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 95e8dd772a..27c48b6c8d 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -123,6 +123,12 @@ directory = '" port)
   (setenv "LIBGIT2_SYS_USE_PKG_CONFIG" "1")
   (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
 
+  ;; By setting cargo.dep-info-basedir we get relative paths for the build
+  ;; files in the 'target/*/dep' directory, allowing us to reuse them.
+  ;; https://doc.rust-lang.org/cargo/reference/config.html#builddep-info-basedir
+  ;; https://doc.rust-lang.org/cargo/guide/build-cache.html#dep-info-files
+  ;(setenv "CARGO_BUILD_DEP_INFO_BASEDIR" (getcwd))
+
   ;; We don't use the Cargo.lock file to determine the package versions we use
   ;; during building, and in any case if one is not present it is created
   ;; during the 'build phase by cargo.
@@ -173,7 +179,9 @@ directory = '" port)
     (or skip-build?
         (not (has-executable-target?))
         (invoke "cargo" "install" "--path" "." "--root" out
-                "--features" (string-join features)))))
+                "--features" (string-join features)))
+    (copy-recursively "./target" (string-append out "/target")))
+  #t)
 
 (define %standard-phases
   (modify-phases gnu:%standard-phases

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]