From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id RHE/Jmp5V1+aMwAA0tVLHw (envelope-from ) for ; Tue, 08 Sep 2020 12:30:34 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id +OSSIWp5V19ZLwAAbx9fmQ (envelope-from ) for ; Tue, 08 Sep 2020 12:30:34 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 329B694036A for ; Tue, 8 Sep 2020 12:30:34 +0000 (UTC) Received: from localhost ([::1]:36918 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kFcld-0006Lk-42 for larch@yhetil.org; Tue, 08 Sep 2020 08:30:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33566) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kFcl3-0006HW-LT for guix-devel@gnu.org; Tue, 08 Sep 2020 08:29:57 -0400 Received: from flashner.co.il ([178.62.234.194]:42070) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kFcl1-0007mz-2l for guix-devel@gnu.org; Tue, 08 Sep 2020 08:29:57 -0400 Received: from localhost (unknown [31.210.181.177]) by flashner.co.il (Postfix) with ESMTPSA id 75762404EF for ; Tue, 8 Sep 2020 12:29:22 +0000 (UTC) Date: Tue, 8 Sep 2020 15:28:49 +0300 From: Efraim Flashner To: guix-devel@gnu.org Subject: reusing rust build artifacts Message-ID: <20200908122849.GJ1643@E5400> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="lZZ4ablUVnt2XgAh" Content-Disposition: inline X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Received-SPF: pass client-ip=178.62.234.194; envelope-from=efraim@flashner.co.il; helo=flashner.co.il X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/08 08:29:23 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -2.61 X-TUID: DJFtIQX4u5JL --lZZ4ablUVnt2XgAh Content-Type: multipart/mixed; boundary="W/D3X8sky0X3AmG5" Content-Disposition: inline --W/D3X8sky0X3AmG5 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I took another look at reusing rust build artifacts. I figured this time it would be best to see if there was even overlap that would help. The concept is simple, when you make changes to your code you should only need to recompile the parts you made changes too. Not that much difference from touching a few modules in Guix and re-running 'make'. I looked at rust-aho-corasick-0.6 and rust-unicode-segmentation-1.6 and 1.3. All three of them use rust-quickcheck-0.7 as cargo-development-inputs and therefore seemed like a worthwhile check to see if they even produce the same artifacts. The build artifacts produce a long string of letters and numbers to let the Well Informed Reader=E2=84=A2 know which features are included in the artifact. Rust-unicode-segmentation-1.6 and 1.3 each produced identical artifacts for rust-quickcheck-0.7. This means that IF we were caching the artifacts (for use with packages which depend on them) AND IF rust-unicode-segmentation-1.6 depended on 1.3 (or the other way around) THEN we could reuse the compiled rust-quickcheck-0.7. Nice to know that it works, but on its own not useful. rust-aho-corasick-0.6 and rust-unicode-segmentation-1.6 did not produce the same rust-quickcheck-0.6 artifacts. A check of their Cargo.toml files=C2=B9=C2=B2 shows that rust-aho-corasick-0.6 uses the 'default-features=3Dfalse' build flag for rust-quickcheck-0.7 and rust-unicode-segmentation-1.6 doesn't. Checking rust-quickcheck-0.7's Cargo.toml=C2=B3 shows that there are a number of different feature options. Realizing I made a mistake here, I compared the 'target' directory from rust-unicode-segmentation-1.6 and rust-quickcheck-0.7. By default we build all creates with the 'default' feature set, so it turns out that the 'target/release/deps' folder and the 'target/release/build' folder are almost identical. Given the above statement about the default feature set, it is likely worthwhile to try to copy the 'target' directory to an output and then copy it back into place for the next package which needs it as an input. For comparison, on my (old) machine, building rust-unicode-segmentation-1.6 takes 185 seconds. When I copy the output =66rom rust-quickcheck-0.7 to the 'target' directory this drops down to 195 seconds. I'm still investigating why it is rebuilding all the bits it's already built. I've attached a diff with what I've been working with. =C2=B9 https://github.com/BurntSushi/aho-corasick/blob/0.6.10/Cargo.toml =C2=B2 https://github.com/unicode-rs/unicode-segmentation/blob/v1.6.0/Cargo= =2Etoml =C2=B3 https://github.com/BurntSushi/quickcheck/blob/0.7.2/Cargo.toml --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --W/D3X8sky0X3AmG5 Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="reuse-cargo-build-artifacts.diff" Content-Transfer-Encoding: quoted-printable diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm index 0f0f0c28c9..4495e6422b 100644 --- a/gnu/packages/crates-io.scm +++ b/gnu/packages/crates-io.scm @@ -28021,7 +28021,22 @@ described in Unicode Standard Annex #15.") (build-system cargo-build-system) (arguments `(#:cargo-development-inputs - (("rust-quickcheck" ,rust-quickcheck-0.7)))) + (("rust-quickcheck" ,rust-quickcheck-0.7)) + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'copy-target-into-place + (lambda* (#:key inputs #:allow-other-keys) + (copy-recursively (string-append (assoc-ref inputs "rust-quic= kcheck-0.7") + "/target") + "target") + (for-each make-file-writable (find-files "target" ".")) + ;; Change from the previous build(s) to the current directory. + (substitute* (find-files "target" "\\.d$") + (("/tmp/.*/target") (string-append (getcwd) "/target"))) + #t))) + )) + (inputs + `(("rust-quickcheck-0.7" ,rust-quickcheck-0.7))) (home-page "https://github.com/unicode-rs/unicode-segmentation") (synopsis "Grapheme Cluster, Word and Sentence boundaries") (description diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-sys= tem.scm index 95e8dd772a..27c48b6c8d 100644 --- a/guix/build/cargo-build-system.scm +++ b/guix/build/cargo-build-system.scm @@ -123,6 +123,12 @@ directory =3D '" port) (setenv "LIBGIT2_SYS_USE_PKG_CONFIG" "1") (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1") =20 + ;; By setting cargo.dep-info-basedir we get relative paths for the build + ;; files in the 'target/*/dep' directory, allowing us to reuse them. + ;; https://doc.rust-lang.org/cargo/reference/config.html#builddep-info-b= asedir + ;; https://doc.rust-lang.org/cargo/guide/build-cache.html#dep-info-files + ;(setenv "CARGO_BUILD_DEP_INFO_BASEDIR" (getcwd)) + ;; We don't use the Cargo.lock file to determine the package versions we= use ;; during building, and in any case if one is not present it is created ;; during the 'build phase by cargo. @@ -173,7 +179,9 @@ directory =3D '" port) (or skip-build? (not (has-executable-target?)) (invoke "cargo" "install" "--path" "." "--root" out - "--features" (string-join features))))) + "--features" (string-join features))) + (copy-recursively "./target" (string-append out "/target"))) + #t) =20 (define %standard-phases (modify-phases gnu:%standard-phases --W/D3X8sky0X3AmG5-- --lZZ4ablUVnt2XgAh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl9XeP0ACgkQQarn3Mo9 g1H1NQ//XyEoK7Wd1pS7LpTK/ywWoZNubwlFdkNZvZ2obuEqDqIlOw8DWwGLbUVr ZJC/bXHh36y+lzoH/hQUmQSYbiDhdu0WzJlNvFgvpC2SCHEqF1F7uolHjFS1Nw8b JPSxdTFrX3mpRdy5VarhcoQne384w9mvFlKJX9/ZpdXceY2Q1uagGkI/fzRoaOvn cU4z4y5lysG5ild0hHxtpKd1I9Dk7UKEMaOp+i2hy33QmKQAkdsGAzDmVYyVlmss DnQ2PuZpPocKDFqhjZGDptpm+xlux9cKlGAngeTCThe9EGRMJ1vdJIjtBBzepxQg 1Q4gnB0OaZHuljFiQHgSN7l79+8jjSPLIMg0sKnXA+oLxm+p7ZnJb3Z4KH84dwrm ++H0D5QpBgbRzyZbaM9StP3whjRJB38w05If/ks80hvHa8RcPDBafwBvqSX39XPX dRfevWTitzWPANvlwl2ACt/6gy+JM8vBhYda057/MCHzSQADDOK5pVKsjBbwjf/h HYPZs1ZLzgMYvfv1oplawNg93KNJWnUgwv+UA1UEl7KRR4elAkaYHzaoKszk5unh 8Raq+7tgb5eiCcAVqLbWAuufKyTQXUg0E4EAKNqAXOg2jiv4s+37H7zn0f/5rBE5 lIofC3qPOsC11BxHQQfWKVU9UPVVPkh0MVHnBz6qZD28VojIyCM= =E5n2 -----END PGP SIGNATURE----- --lZZ4ablUVnt2XgAh--