all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Efraim Flashner <efraim@flashner.co.il>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: guix-devel@gnu.org
Subject: Re: Overhauling the cargo-build-system
Date: Fri, 11 Oct 2019 17:13:42 +0300	[thread overview]
Message-ID: <20191011141342.GC13364@E5400> (raw)
In-Reply-To: <87d0f4p6xd.fsf@gnu.org>

[-- Attachment #1: Type: text/plain, Size: 6322 bytes --]

On Fri, Oct 11, 2019 at 12:33:18AM +0200, Ludovic Courtès wrote:
> Hello!
> 
> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
> > I'd like to challenge the assumption that packages are both libraries
> > and source. A 'library' in rust compiles into one of three types: a
> > static library (libfoo.a), a shared library (libfoo.so), or a
> > 'rust-library' (libfoo.rlib).
> 
> Why don’t we create .so files, then?  They have NEEDED and RUNPATH, so
> that could work like for C, no?
> 

It is possible to force a library to produce .so files, but then we'd
have to force all the other packages to link against them, and then at
that point we're basically rewriting 'cargo build'. For some packages,
like icecat, that does seem to be what they do.

> > Let me repeat that. We have 192 rust packages that no one needs or
> > wants, except in source form.
> 
> Ouch!  So the rlib file is never actually used?!
> 

As I understand it, it gets treated as a build artifact. It'll be
compiled as libfoo_XXXXXXXXX.rlib and then, if some other program comes
around as asks for "foo with feature baz" it'll be reused. At some point
it gets compiled directly into the final binary, so it's really more of
an intermediary stage.

> You said “it is not possible to link an rlib to another rlib”, but
> that’s not necessarily a problem, it’s like .a libraries, no?
> 

if libfoo depends on libbar, then pre-building libbar_XXXXX.rlib doesn't
allow us to call "rustc --crate-name foo src/lib.rs --crate-type lib
--extern (string-append bar=(assoc-ref %build-inputs bar)
/lib/libbar.rlib)" (note the --crate-type lib in there), it'll just not
work and want to recompile libbar for just the few functions it uses
from it. The only difference I've seen is if our foo is also a "binary
crate", ie produces a binary, then it'll sometimes accept rlibs for its
own rlib lib output.

Going back to forcing it to produce .so files, if we changed the
crate-type to dylib then we'd get libfoo.so. The two parts of the
problem is now we need to change all the other libraries and programs
that depend on foo to take our libfoo.so and not expect an rlib, and
that if foo has 3 options then we have to decide how to build libfoo so
it's useful for everything (obviously all 3) but without dragging in
everything (easiest with none of the 3), or to have some combination,
which becomes a nightmare very quickly.

That sentence was way too long.

> > PROPOSAL:
> > Change all the rust packages we have now to be source-only. Rename them
> > from rust-foo to rust-foo-src or rust-src-foo.
> 
> In the current scheme, can you actually do, say:
> 
>   guix environment --ad-hoc rust rust-foo rust-bar
> 
> and then (pseudo syntax):
> 
>   rustc mystuff.rust -lfoo -lbar
> 
> ?

Ignoring that we don't actually install the libraries here's a copy of
a check phase I wrote trying to use installed libraries. I think in the
end it did consume the rlibs as requested, but it never did link to
them, it just absorbed them into the final binary.

(replace 'check
  (lambda* (#:key inputs #:allow-other-keys)
    (let ((ndarray    (assoc-ref inputs "ndarray"))
          (rand       (assoc-ref inputs "rand"))
          (rayon      (assoc-ref inputs "rayon"))
          (serde      (assoc-ref inputs "serde"))
          (serde-json (assoc-ref inputs "serde-json"))
          (structopt  (assoc-ref inputs "structopt")))
      (invoke "rustc" "--edition=2018" "--crate-name" "qtlreaper"
              "src/lib.rs"
              "-C" "opt-level=3"
              "--test" "-C" "metadata=test" "-C" "extra-filename=-test"
              "--out-dir" "target/release/deps"
              "--extern" (string-append "ndarray=" ndarray "/lib/libndarray.rlib")
              "--extern" (string-append "rand=" rand "/lib/librand.rlib")
              "--extern" (string-append "rayon=" rayon "/lib/librayon.rlib")
              "--extern" (string-append "serde=" serde "/lib/libserde.rlib")
              "--extern" (string-append "serde_json=" serde-json "/lib/libserde_json.rlib")
              "--extern" (string-append "structopt=" structopt "/lib/structopt.rlib")
              "--cap-lints" "allow" "-C" "rpath" "-C" "prefer-dynamic")
      (invoke "rustc" "--edition=2018" "--crate-name" "qtlreaper"
              "src/main.rs"
              "-C" "opt-level=3"
              "--test" "-C" "metadata=test" "-C" "extra-filename=-test"
              "--out-dir" "target/release/deps"
              "--extern" "qtlreaper=target/release/deps/libqtlreaper-test.rlib"
              "--extern" (string-append "ndarray=" ndarray "/lib/libndarray.rlib")
              "--extern" (string-append "rand=" rand "/lib/librand.rlib")
              "--extern" (string-append "rayon=" rayon "/lib/librayon.rlib")
              "--extern" (string-append "serde=" serde "/lib/libserde.rlib")
              "--extern" (string-append "serde_json=" serde-json "/lib/libserde_json.rlib")
              "--extern" (string-append "structopt=" structopt "/lib/structopt.rlib")
              "--cap-lints" "allow" "-C" "rpath" "-C" "prefer-dynamic")
      (invoke "./target/release/deps/qtlreaper-test")
      (invoke "rustdoc" "--edition=2018" "--test" "src/lib" "--crate-name" "qtlreaper"
              "--extern" "qtlreaper=target/release/deps/libqtlreaper-test.rlib"
              "--extern" (string-append "ndarray=" ndarray "/libndarray.rlib")
              "--extern" (string-append "rand=" rand "/librand.rlib")
              "--extern" (string-append "rayon=" rayon "/lib/librayon.rlib")
              "--extern" (string-append "serde=" serde "/lib/libserde.rlib")
              "--extern" (string-append "serde_json=" serde-json "/lib/libserde_json.rlib")
              "--extern" (string-append "structopt=" structopt "/lib/structopt.rlib"))))))))

> 
> Thanks,
> Ludo’.

I was thinking of making the build phases for the cargo-build-system use
the --verbose flag, I found it very informative to see what it was doing
along the way.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

  reply	other threads:[~2019-10-11 14:13 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-10 15:50 Overhauling the cargo-build-system Efraim Flashner
2019-10-10 22:33 ` Ludovic Courtès
2019-10-11 14:13   ` Efraim Flashner [this message]
2019-11-16  6:31     ` Martin Becze
2019-11-16 16:37       ` John Soo
2019-11-16 18:44         ` Martin Becze
2019-11-16 21:33       ` Ludovic Courtès
2019-11-17  2:35         ` Martin Becze
2019-11-17  7:19         ` Efraim Flashner
2019-11-17 21:22           ` Ludovic Courtès
2019-11-18 10:20             ` Efraim Flashner
2019-11-23 17:27               ` Ludovic Courtès
2019-12-09  4:45           ` Chris Marusich
2019-12-09 20:14             ` Martin Becze
2019-12-19 16:10               ` Ludovic Courtès
2019-12-19 16:09             ` Ludovic Courtès
2019-12-19 17:23               ` John Soo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191011141342.GC13364@E5400 \
    --to=efraim@flashner.co.il \
    --cc=guix-devel@gnu.org \
    --cc=ludo@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.