From mboxrd@z Thu Jan 1 00:00:00 1970 From: Efraim Flashner Subject: Overhauling the cargo-build-system Date: Thu, 10 Oct 2019 18:50:56 +0300 Message-ID: <20191010155056.GD1301@E5400> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ylS2wUBXLOxYXZFQ" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:54500) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iIZmg-0004np-Lt for guix-devel@gnu.org; Thu, 10 Oct 2019 10:51:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iIZmf-0008Vw-BL for guix-devel@gnu.org; Thu, 10 Oct 2019 10:51:18 -0400 Received: from flashner.co.il ([178.62.234.194]:57960) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1iIZmf-0008VN-29 for guix-devel@gnu.org; Thu, 10 Oct 2019 10:51:17 -0400 Received: from localhost (unknown [94.230.83.61]) by flashner.co.il (Postfix) with ESMTPSA id A8F8D4020F for ; Thu, 10 Oct 2019 14:51:15 +0000 (UTC) Content-Disposition: inline List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org --ylS2wUBXLOxYXZFQ Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've spent a lot of time over the past few months hacking at packaging rust crates. I've found it time-consuming, soothing and, ultimately, likely a waste of time. When we started with the cargo-build-system the premise was 'packages are both libraries and source, we need them in source form to build the next library in the chain' and the example was our first three rust- packages, rust-quote, rust-syn and rust-proc-macro2, all three of which depend on the other two. I'd like to challenge the assumption that packages are both libraries and source. A 'library' in rust compiles into one of three types: a static library (libfoo.a), a shared library (libfoo.so), or a 'rust-library' (libfoo.rlib). A 'rust library', an rlib file, is like a shared library but with rust-specific metadata. I haven't looked into what exactly that means, but I can tell you practically how it works. Through some in depth searching online it seems it is possible to pre-compile rlibs and link them to a final binary, but it is not possible to link an rlib to another rlib. What this means is currently we have $(guix package -A ^rust- | wc -l) ~> 192 rust libraries packaged that no one will ever use in any form EXCEPT as source inputs for a future library or binary. Let me repeat that. We have 192 rust packages that no one needs or wants, except in source form. We have a couple of packages/variables which are defined in source-form only. We normally see them as inputs or native inputs as ("foo" ,(origin .... This is basically what we have with the rust packages. Upstream to build a binary they call 'cargo build' and it downloads the sources of the dependent libraries, and some of their dependants based on the use-case, compiles them, and then compiles the binary. In Guix, when we eventually have packaged enough libraries to get to a target binary, we'll be pulling in hundreds of extra libraries (transitive inputs of inputs) so a few dozen can be compiled. PROPOSAL: Change all the rust packages we have now to be source-only. Rename them =66rom rust-foo to rust-foo-src or rust-src-foo. The package 'alacritty' is perhaps not a great example choice, but it's Cargo.lock file, which describes the dependencies and their versions, lists 278 dependencies. (Generally you can care only about the major and minor part of the version.) It's package definition would explicitly list the 278 dependent libraries it wanted so they could be put in the 'guix-vendor' directory like now. It makes for a very large package definition, but we wouldn't have to ensure thousands of other rust libraries built so we can throw away the results and build alacritty in the end. Another package, starship, would have 124 rust dependencies, but I'm betting there's a fair amount of overlap between them. Since we never actually use the output of any of the rust packages we currently have it doesn't really make sense to 'build them' per se as we currently do. $ tree $(guix build rust-proc-macro2) /gnu/store/jjinj0dvvzqnlpn9li3nxcyrpv1fbbfp-rust-proc-macro2-0.4.30 `-- share `-- doc `-- rust-proc-macro2-0.4.30 |-- LICENSE-APACHE `-- LICENSE-MIT My plan is to hack on the cargo-build-system on a separate branch and see if I can pull all of this together. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --ylS2wUBXLOxYXZFQ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl2fU1wACgkQQarn3Mo9 g1FxzQ//d5Cu5WnFqE15nOEsml4lJbcJRgkojpssPge8GmzmrQZqK3U0fpARCyHp q+54SI8YHB6KJDvN2ageF3/+HPG+5idKcYgh8JYyJLLI3c6wcc2DJo9RDBBto+04 YhPlkw2XQ31yF/xdNCikdZC12QjOBuJKU7WdlfgTOvy7PIqwiO7kRtsjfPmJK/8F OvcaZNbJxhsWy919baoNo42VUNz9EYs/Sy+vpJaxC6ewIWJ/xlsy45aSYrtWDGCQ TL6palHs4/0NV+gQcZoBZcAuL6eqmq3i1iyrG5Eun022bx1YyPifWKRe6L2N+idS lSkoRNSu4sAds5HyOxgoIMJGAgZRhO4kFLT3DDu6ygAEhAUTi7gyFMR99BTYPpyk TsDGTqMf1wewd6m5zipZoL+7K1ab+d1RMXIKylBgvTflbnCWpmK7Si4o9ECwA9wD I5VLpueBSWbfwWTTEV1enNYPkw/TjTAlwCh9PvrdzZwCSGGV3yaL1KDh+dOovAXE 7b68Ae6DBMWlhudyvz70UxAk/AJLtJDXSCnqy4ZQvMFeiNfQPVND9v440mnEoZj2 ygpWmR0uZ4G3DW96tfFDniwbd+PySkcyrOafs53DxquxPy96kPhuSwMP1mQGuRYp W7X3v7q4SoLLakRGIk3PpPeQ4PY/CaAIqHBPgKNpJUGblhxJYxo= =vatr -----END PGP SIGNATURE----- --ylS2wUBXLOxYXZFQ--