On Wed, Mar 6, 2024 at 4:06 AM Efraim Flashner <efraim@flashner.co.il>
wrote:

> On Mon, Feb 26, 2024 at 09:24:29PM -0500, Jason Conroy wrote:
> > Hello Efraim,
> >
> > Thanks for investigating this - a Rust development workflow using only
> > Guix-native crates is something I've been waiting for!
> >
> > I was experimenting with your patches and it seems that they do pull in
> the
> > source crates for requested packages, but not their dependencies (example
> > below). Is there something I'm missing?
>
> When you say they pull in the source crates do you mean the sources of
> the other rust packages needed by rust-rand? I didn't test that, but I
> assumed it wouldn't.
>

That's right, I'm interested in having `guix shell` populate the full set
of transitive dependencies in the registry, similar to what happens when
using `cargo build` in the conventional way (but instead using
Guix-packaged crates). I interpreted your message up-thread to mean that
you were trying to accomplish the same, but please correct me if not.

I took rust-rand as an example only because it does have some dependencies:

$ cd $CARGO_PROJECT
$ cat Cargo.toml
[package]
name = "test_prog"
...
[dependencies]
rand = "0.8.5"

$ cargo build
    Updating crates.io index
  Downloaded cfg-if v1.0.0
  Downloaded rand_chacha v0.3.1
  Downloaded rand v0.8.5
  Downloaded ppv-lite86 v0.2.17
  Downloaded rand_core v0.6.4
  Downloaded getrandom v0.2.12
  Downloaded libc v0.2.153
  Downloaded 7 crates (932.0 KB) in 0.48s
   Compiling libc v0.2.153
   Compiling cfg-if v1.0.0
   Compiling ppv-lite86 v0.2.17
   Compiling getrandom v0.2.12
   Compiling rand_core v0.6.4
   Compiling rand_chacha v0.3.1
   Compiling rand v0.8.5
   Compiling test_prog v0.1.0 (/home/...)


> Currently if you were to pull in rust-rand-0.8 and rust-rand-0.7 then
> you'd have both rand-0.*.crate files in the registry but only one of
> them would be listed in share/cargo/registry/index/ra/nd/rand. I need to
> adjust the generation of that file to combine multiple sources if they
> exist, and sort them (I'm not sure it's necessary, but wouldn't be
> surprised if we hit undefined behaviour if they were listed multiple
> times or out of order).


I'm somewhat new to rust, but it appears that outside of Guix, the
local-only development workflow looks like this:

$ cd $CARGO_PROJECT
$ mkdir $VENDOR
$ cargo vendor $VENDOR

After downloading and unpacking all of the crates into $VENDOR, this last
command instructs me to add the following in ~/cargo/config.toml.
Then, after opening a new guix shell without network access, I can confirm
that `cargo build` works fine with the vendored crates.

[source.crates-io]
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "<VENDOR>"

Getting back to your patch set: would it make sense to emulate this vendor
workflow instead of trying to construct a registry directly? Even assuming
that all details of the registry structure are stable and documented, the
layout of the vendor directory appears much simpler. And IIUC the code for
setting up vendored libraries already exists in cargo-build-system.

I also need to figure out something with a
> config.toml to see if it's possible to generate one that could be
> included from another one, since you can't add 'local-registry =
> $GUIX_PROFILE/...' in a toml file.
>

You've probably researched this more than I have, but it seems that this
use case is explicitly unsupported in the TOML language spec:
https://github.com/toml-lang/toml/issues/397

With that option off the table, I can't think of any elegant solutions.
Maybe a wrapper for the cargo binary that pre-processes cargo.toml and then
calls the real cargo?