From mboxrd@z Thu Jan 1 00:00:00 1970 From: Efraim Flashner Subject: Re: Overhauling the cargo-build-system Date: Fri, 11 Oct 2019 17:13:42 +0300 Message-ID: <20191011141342.GC13364@E5400> References: <20191010155056.GD1301@E5400> <87d0f4p6xd.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="adJ1OR3c6QgCpb/j" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:43398) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iIvfv-00020s-FR for guix-devel@gnu.org; Fri, 11 Oct 2019 10:13:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iIvft-0007TR-V1 for guix-devel@gnu.org; Fri, 11 Oct 2019 10:13:47 -0400 Content-Disposition: inline In-Reply-To: <87d0f4p6xd.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel@gnu.org --adJ1OR3c6QgCpb/j Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 11, 2019 at 12:33:18AM +0200, Ludovic Court=C3=A8s wrote: > Hello! >=20 > Efraim Flashner skribis: >=20 > > I'd like to challenge the assumption that packages are both libraries > > and source. A 'library' in rust compiles into one of three types: a > > static library (libfoo.a), a shared library (libfoo.so), or a > > 'rust-library' (libfoo.rlib). >=20 > Why don=E2=80=99t we create .so files, then? They have NEEDED and RUNPAT= H, so > that could work like for C, no? >=20 It is possible to force a library to produce .so files, but then we'd have to force all the other packages to link against them, and then at that point we're basically rewriting 'cargo build'. For some packages, like icecat, that does seem to be what they do. > > Let me repeat that. We have 192 rust packages that no one needs or > > wants, except in source form. >=20 > Ouch! So the rlib file is never actually used?! >=20 As I understand it, it gets treated as a build artifact. It'll be compiled as libfoo_XXXXXXXXX.rlib and then, if some other program comes around as asks for "foo with feature baz" it'll be reused. At some point it gets compiled directly into the final binary, so it's really more of an intermediary stage. > You said =E2=80=9Cit is not possible to link an rlib to another rlib=E2= =80=9D, but > that=E2=80=99s not necessarily a problem, it=E2=80=99s like .a libraries,= no? >=20 if libfoo depends on libbar, then pre-building libbar_XXXXX.rlib doesn't allow us to call "rustc --crate-name foo src/lib.rs --crate-type lib --extern (string-append bar=3D(assoc-ref %build-inputs bar) /lib/libbar.rlib)" (note the --crate-type lib in there), it'll just not work and want to recompile libbar for just the few functions it uses =66rom it. The only difference I've seen is if our foo is also a "binary crate", ie produces a binary, then it'll sometimes accept rlibs for its own rlib lib output. Going back to forcing it to produce .so files, if we changed the crate-type to dylib then we'd get libfoo.so. The two parts of the problem is now we need to change all the other libraries and programs that depend on foo to take our libfoo.so and not expect an rlib, and that if foo has 3 options then we have to decide how to build libfoo so it's useful for everything (obviously all 3) but without dragging in everything (easiest with none of the 3), or to have some combination, which becomes a nightmare very quickly. That sentence was way too long. > > PROPOSAL: > > Change all the rust packages we have now to be source-only. Rename them > > from rust-foo to rust-foo-src or rust-src-foo. >=20 > In the current scheme, can you actually do, say: >=20 > guix environment --ad-hoc rust rust-foo rust-bar >=20 > and then (pseudo syntax): >=20 > rustc mystuff.rust -lfoo -lbar >=20 > ? Ignoring that we don't actually install the libraries here's a copy of a check phase I wrote trying to use installed libraries. I think in the end it did consume the rlibs as requested, but it never did link to them, it just absorbed them into the final binary. (replace 'check (lambda* (#:key inputs #:allow-other-keys) (let ((ndarray (assoc-ref inputs "ndarray")) (rand (assoc-ref inputs "rand")) (rayon (assoc-ref inputs "rayon")) (serde (assoc-ref inputs "serde")) (serde-json (assoc-ref inputs "serde-json")) (structopt (assoc-ref inputs "structopt"))) (invoke "rustc" "--edition=3D2018" "--crate-name" "qtlreaper" "src/lib.rs" "-C" "opt-level=3D3" "--test" "-C" "metadata=3Dtest" "-C" "extra-filename=3D-test" "--out-dir" "target/release/deps" "--extern" (string-append "ndarray=3D" ndarray "/lib/libndarr= ay.rlib") "--extern" (string-append "rand=3D" rand "/lib/librand.rlib") "--extern" (string-append "rayon=3D" rayon "/lib/librayon.rli= b") "--extern" (string-append "serde=3D" serde "/lib/libserde.rli= b") "--extern" (string-append "serde_json=3D" serde-json "/lib/li= bserde_json.rlib") "--extern" (string-append "structopt=3D" structopt "/lib/stru= ctopt.rlib") "--cap-lints" "allow" "-C" "rpath" "-C" "prefer-dynamic") (invoke "rustc" "--edition=3D2018" "--crate-name" "qtlreaper" "src/main.rs" "-C" "opt-level=3D3" "--test" "-C" "metadata=3Dtest" "-C" "extra-filename=3D-test" "--out-dir" "target/release/deps" "--extern" "qtlreaper=3Dtarget/release/deps/libqtlreaper-test= =2Erlib" "--extern" (string-append "ndarray=3D" ndarray "/lib/libndarr= ay.rlib") "--extern" (string-append "rand=3D" rand "/lib/librand.rlib") "--extern" (string-append "rayon=3D" rayon "/lib/librayon.rli= b") "--extern" (string-append "serde=3D" serde "/lib/libserde.rli= b") "--extern" (string-append "serde_json=3D" serde-json "/lib/li= bserde_json.rlib") "--extern" (string-append "structopt=3D" structopt "/lib/stru= ctopt.rlib") "--cap-lints" "allow" "-C" "rpath" "-C" "prefer-dynamic") (invoke "./target/release/deps/qtlreaper-test") (invoke "rustdoc" "--edition=3D2018" "--test" "src/lib" "--crate-name= " "qtlreaper" "--extern" "qtlreaper=3Dtarget/release/deps/libqtlreaper-test= =2Erlib" "--extern" (string-append "ndarray=3D" ndarray "/libndarray.r= lib") "--extern" (string-append "rand=3D" rand "/librand.rlib") "--extern" (string-append "rayon=3D" rayon "/lib/librayon.rli= b") "--extern" (string-append "serde=3D" serde "/lib/libserde.rli= b") "--extern" (string-append "serde_json=3D" serde-json "/lib/li= bserde_json.rlib") "--extern" (string-append "structopt=3D" structopt "/lib/stru= ctopt.rlib")))))))) >=20 > Thanks, > Ludo=E2=80=99. I was thinking of making the build phases for the cargo-build-system use the --verbose flag, I found it very informative to see what it was doing along the way. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --adJ1OR3c6QgCpb/j Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl2gjhMACgkQQarn3Mo9 g1Fv3BAAn2v15fMuFd2lBWdt3SO/MF64rgXWFDiIKAPNlLit+Au1cZGNQ7WP123m 436XXt6y53qqGlSDbtF/xExAZlRsnBRPjTEuU3ZLMy+01XKLzMTR9ppxzWObVFjo Bz3rjraE4a1ZdNHaEK8JazvDiOnrd5ynA4dIuSxuWQyu1sYxhfkQT+KfnAdCn27r 62vS9uBGPpWedx8g1vVSyqX7yNdBn1oQxrdSJfemd5qZKq1Il+V1rycDAys7QEnq Hc7an8XeSXb/aV+7wun1hHCtEC0shF4WSfjetfrtlx1m+WG+o0zLlhtyWu7Ey6XE z/lKRfLeA4Ez1qEUIGwNRSF5u0ADEeEGXqiKRBJraHHfUaSoAOlPyty/4nj0zN/w 2q6rpzYelZL15UYnfqI7xIPBBncqd45hoZHDaXt7gJ4d99vWmIC3xUYXh5Q8yWPy 63TpOAChnLKOptOf6gjClc0j2p/cbESHR3NYIfRdyHUMzi89ojxdw4cYGDlicNJS 9oaBU9LbPVWM9MHDzSrObEu077AK5JltOV+bdhyy/4j4JEX3Ew2Eko8Yf4bXhR3r g9Qcx516xtiZ5hFku97RtTvviw3NGynXwbq0P4DgFx868EWI6aYp2Bh6olzQJ96S +j9xbJfd7aPwm+jFT/JFDZeytiHp0FOIEcfGHfpb6P6ZhbkJAQU= =W/84 -----END PGP SIGNATURE----- --adJ1OR3c6QgCpb/j--