From mboxrd@z Thu Jan  1 00:00:00 1970
From: Efraim Flashner <efraim@flashner.co.il>
Subject: Overhauling the cargo-build-system
Date: Thu, 10 Oct 2019 18:50:56 +0300
Message-ID: <20191010155056.GD1301@E5400>
Mime-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature"; boundary="ylS2wUBXLOxYXZFQ"
Return-path: <guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org>
Received: from eggs.gnu.org ([2001:470:142:3::10]:54500)
 by lists.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <efraim@flashner.co.il>) id 1iIZmg-0004np-Lt
 for guix-devel@gnu.org; Thu, 10 Oct 2019 10:51:19 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <efraim@flashner.co.il>) id 1iIZmf-0008Vw-BL
 for guix-devel@gnu.org; Thu, 10 Oct 2019 10:51:18 -0400
Received: from flashner.co.il ([178.62.234.194]:57960)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <efraim@flashner.co.il>) id 1iIZmf-0008VN-29
 for guix-devel@gnu.org; Thu, 10 Oct 2019 10:51:17 -0400
Received: from localhost (unknown [94.230.83.61])
 by flashner.co.il (Postfix) with ESMTPSA id A8F8D4020F
 for <guix-devel@gnu.org>; Thu, 10 Oct 2019 14:51:15 +0000 (UTC)
Content-Disposition: inline
List-Id: "Development of GNU Guix and the GNU System distribution."
 <guix-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/guix-devel>,
 <mailto:guix-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/guix-devel>
List-Post: <mailto:guix-devel@gnu.org>
List-Help: <mailto:guix-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/guix-devel>,
 <mailto:guix-devel-request@gnu.org?subject=subscribe>
Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org
Sender: "Guix-devel" <guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org>
To: guix-devel@gnu.org


--ylS2wUBXLOxYXZFQ
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

I've spent a lot of time over the past few months hacking at packaging
rust crates. I've found it time-consuming, soothing and, ultimately,
likely a waste of time. When we started with the cargo-build-system the
premise was 'packages are both libraries and source, we need them in
source form to build the next library in the chain' and the example was
our first three rust- packages, rust-quote, rust-syn and
rust-proc-macro2, all three of which depend on the other two.

I'd like to challenge the assumption that packages are both libraries
and source. A 'library' in rust compiles into one of three types: a
static library (libfoo.a), a shared library (libfoo.so), or a
'rust-library' (libfoo.rlib). A 'rust library', an rlib file, is like a
shared library but with rust-specific metadata. I haven't looked into
what exactly that means, but I can tell you practically how it works.
Through some in depth searching online it seems it is possible to
pre-compile rlibs and link them to a final binary, but it is not
possible to link an rlib to another rlib. What this means is currently
we have $(guix package -A ^rust- | wc -l) ~> 192 rust libraries packaged
that no one will ever use in any form EXCEPT as source inputs for a
future library or binary.

Let me repeat that. We have 192 rust packages that no one needs or
wants, except in source form.

We have a couple of packages/variables which are defined in source-form
only. We normally see them as inputs or native inputs as
("foo" ,(origin .... This is basically what we have with the rust
packages. Upstream to build a binary they call 'cargo build' and it
downloads the sources of the dependent libraries, and some of their
dependants based on the use-case, compiles them, and then compiles the
binary. In Guix, when we eventually have packaged enough libraries to
get to a target binary, we'll be pulling in hundreds of extra libraries
(transitive inputs of inputs) so a few dozen can be compiled.

PROPOSAL:
Change all the rust packages we have now to be source-only. Rename them
=66rom rust-foo to rust-foo-src or rust-src-foo. The package 'alacritty'
is perhaps not a great example choice, but it's Cargo.lock file, which
describes the dependencies and their versions, lists 278 dependencies.
(Generally you can care only about the major and minor part of the
version.) It's package definition would explicitly list the 278
dependent libraries it wanted so they could be put in the 'guix-vendor'
directory like now. It makes for a very large package definition, but we
wouldn't have to ensure thousands of other rust libraries built so we
can throw away the results and build alacritty in the end. Another
package, starship, would have 124 rust dependencies, but I'm betting
there's a fair amount of overlap between them.

Since we never actually use the output of any of the rust packages we
currently have it doesn't really make sense to 'build them' per se as we
currently do.

$ tree $(guix build rust-proc-macro2)
/gnu/store/jjinj0dvvzqnlpn9li3nxcyrpv1fbbfp-rust-proc-macro2-0.4.30
`-- share
    `-- doc
         `-- rust-proc-macro2-0.4.30
             |-- LICENSE-APACHE
             `-- LICENSE-MIT

My plan is to hack on the cargo-build-system on a separate branch and
see if I can pull all of this together.

--=20
Efraim Flashner   <efraim@flashner.co.il>   =D7=90=D7=A4=D7=A8=D7=99=D7=9D =
=D7=A4=D7=9C=D7=A9=D7=A0=D7=A8
GPG key =3D A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

--ylS2wUBXLOxYXZFQ
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl2fU1wACgkQQarn3Mo9
g1FxzQ//d5Cu5WnFqE15nOEsml4lJbcJRgkojpssPge8GmzmrQZqK3U0fpARCyHp
q+54SI8YHB6KJDvN2ageF3/+HPG+5idKcYgh8JYyJLLI3c6wcc2DJo9RDBBto+04
YhPlkw2XQ31yF/xdNCikdZC12QjOBuJKU7WdlfgTOvy7PIqwiO7kRtsjfPmJK/8F
OvcaZNbJxhsWy919baoNo42VUNz9EYs/Sy+vpJaxC6ewIWJ/xlsy45aSYrtWDGCQ
TL6palHs4/0NV+gQcZoBZcAuL6eqmq3i1iyrG5Eun022bx1YyPifWKRe6L2N+idS
lSkoRNSu4sAds5HyOxgoIMJGAgZRhO4kFLT3DDu6ygAEhAUTi7gyFMR99BTYPpyk
TsDGTqMf1wewd6m5zipZoL+7K1ab+d1RMXIKylBgvTflbnCWpmK7Si4o9ECwA9wD
I5VLpueBSWbfwWTTEV1enNYPkw/TjTAlwCh9PvrdzZwCSGGV3yaL1KDh+dOovAXE
7b68Ae6DBMWlhudyvz70UxAk/AJLtJDXSCnqy4ZQvMFeiNfQPVND9v440mnEoZj2
ygpWmR0uZ4G3DW96tfFDniwbd+PySkcyrOafs53DxquxPy96kPhuSwMP1mQGuRYp
W7X3v7q4SoLLakRGIk3PpPeQ4PY/CaAIqHBPgKNpJUGblhxJYxo=
=vatr
-----END PGP SIGNATURE-----

--ylS2wUBXLOxYXZFQ--