From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id iJ1NGbSS+GW3kgAAe85BDQ:P1 (envelope-from ) for ; Mon, 18 Mar 2024 20:15:00 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id iJ1NGbSS+GW3kgAAe85BDQ (envelope-from ) for ; Mon, 18 Mar 2024 20:15:00 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1710789300; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=cQiFZ2UMJDmK4cg0okPLqQ4TosM9f/ZSIDHmJpbHz88=; b=hSH+eFuETsX2j1S7jFaxY8Omw4Fn2Yx91xBovfzsBS0wjuAwY2BQlTtXIcOk2P/03VXx6m 8Ev17oOvnani7kX3i3yooZoWKsLbT1Cs3Od4FAxJNRC7yyMNhqwXR3dy4ZJxIxHqUa524X L4LLVCL/py9yclgQx3+Gi8amSF3klHnxZqt93xIu93g4krRjXE0DudbOQuKxX4cnxSw31G nX1QQQhT8Quh6AK0vj/VxQiTenQ51e6yPY9cBeXW72sDbHvh5K4VHYepqjFzBpR/cnMK8i wiz+t5mPs2qszH304K9DM9w13OavBC0I4Np1XTVarnkAHbcvNwYRHfPpYy6wGw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1710789300; a=rsa-sha256; cv=none; b=D1SI5MLjjd5WaJj4p6e/3Rrax58f59KRjQgJjCDdd4WOzcnuJZ/iKI68NYAAnbh4Yq8Mz8 mFuWsV4rTM1if+soWiPP+6LQ2arQQqc6PGr75xd5niI9NBvkIPCPQCF4Dem9Ev+OjaDrrD ZB96jJ0ussPOkFiMBPhKMicwg9SbkKE/l2CnTDXDS5OnkFMp3yuVzR8uZrxt091o7OVPNT OZPGjIHppyf0ZDWzypQsm++hS4oZjGrbrqCGEYD8JGSZSFukJ8VPBu0YOACaGrNiJhAG1K U2KMd6EpN4C7bz4VlkD8csVuC6JF/21W3C1Mu4ZuADTG48du28fGHSzx9Q3u7g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=gmail.com (policy=none) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 295553B5B6 for ; Mon, 18 Mar 2024 20:15:00 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rmIRA-0000MQ-1g; Mon, 18 Mar 2024 15:14:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhvTE-0001kI-HJ for guix-devel@gnu.org; Wed, 06 Mar 2024 12:54:24 -0500 Received: from mail-qv1-xf2f.google.com ([2607:f8b0:4864:20::f2f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhvTB-0006iL-C5 for guix-devel@gnu.org; Wed, 06 Mar 2024 12:54:23 -0500 Received: by mail-qv1-xf2f.google.com with SMTP id 6a1803df08f44-68facd7ea4aso583636d6.0 for ; Wed, 06 Mar 2024 09:54:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709747660; x=1710352460; darn=gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=cQiFZ2UMJDmK4cg0okPLqQ4TosM9f/ZSIDHmJpbHz88=; b=kqT9pQ+jUjNbNjon9Yp2AQaUAwrZUHmtXB/II+UzS5XdQ1XaFTY1r9ZCCF6Y0iyLSC crFk1XLbUaHb8lpLIu0AYVdTaj3BhIItFKYiJhKVhOzlanv29Qv6AgaYJ5z6F8jHWQRr TKn37GgvqgCqTi+8YD9m4Ta/Eb0Cl/toHg0VoL3jrq6ULYZaXYD2hNUXIotkZeclKfno Ln0D1TGCSmUSWiC/gov7hkGE4SiMjdk7QAVNcl36yjDkWd1xiyo5hSA7lHJ6A+jORwj7 XlcSXQk48GS7LDxno4MIIa5nY088L/M/sLensog96Q0/bncqdZxB+Nt97giqQXUPtxpN 1RVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709747660; x=1710352460; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cQiFZ2UMJDmK4cg0okPLqQ4TosM9f/ZSIDHmJpbHz88=; b=k2uCx2gJB+AbUBDnbMK4CLUaiTcvQdiulyrqkPnUGepzTH4GbJP1YFgr/RUhMZQiZp epW5OZ13lJhPBANdIt7MqWNTzT/cDuaKNI1/XnTHjLJX4WpE5K3sa444uioaACUHF/L5 u3bI6J3pI/jlJcmUSq/hvtI5lMt/QHzOpdJl1CNLoGHIhw/TwVL6AEmHtgK/tdYHqOo/ qV05aTW8ZxyXee/aZX7c30alzwl/GlJUPOA9H+6qQU0NSi6fUAR93eBtBNBF8B7dFyUL kuutzvgJpZXbCFjRVuUTVkBgvKlhzjjlrAZN3nTkLmPHdHUZ6c9LK96LjIP+CMf4+OGC MCTg== X-Gm-Message-State: AOJu0Yx11F7fHNPvy7mbf3DRgF2B1Ub3ks2Jhf4ZdpUWgJ3btmP+IYfp CG/Sy0/lSQ0DXQ8dzG27pNZoslbeJl0/EHTKllh7xTpEKCGbsGmlY7ubKbK6rR6F4mWTIzuXZue VFM0Av6TcwN+m19YOChthW6rvhM8= X-Google-Smtp-Source: AGHT+IHNfde72aq8l+8d5dvNKREtzyZdcs3d4qJ1RzJrh91JaZjG2VwR9nUrnQ3BZGesfvyhb4zZFtHyHciTFoXTNho= X-Received: by 2002:a05:6214:968:b0:690:7e61:9a60 with SMTP id do8-20020a056214096800b006907e619a60mr1449871qvb.12.1709747659970; Wed, 06 Mar 2024 09:54:19 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Jason Conroy Date: Wed, 6 Mar 2024 12:53:44 -0500 Message-ID: Subject: Re: rust-team branch merged To: Jason Conroy , Efraim Flashner Cc: guix-devel@gnu.org Content-Type: multipart/alternative; boundary="000000000000ed9ed2061301a5bc" Received-SPF: pass client-ip=2607:f8b0:4864:20::f2f; envelope-from=conjaroy@gmail.com; helo=mail-qv1-xf2f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 18 Mar 2024 15:14:18 -0400 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -5.21 X-Spam-Score: -5.21 X-Migadu-Queue-Id: 295553B5B6 X-Migadu-Scanner: mx11.migadu.com X-TUID: zv0xFP+cFJAe --000000000000ed9ed2061301a5bc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Mar 6, 2024 at 4:06=E2=80=AFAM Efraim Flashner wrote: > On Mon, Feb 26, 2024 at 09:24:29PM -0500, Jason Conroy wrote: > > Hello Efraim, > > > > Thanks for investigating this - a Rust development workflow using only > > Guix-native crates is something I've been waiting for! > > > > I was experimenting with your patches and it seems that they do pull in > the > > source crates for requested packages, but not their dependencies (examp= le > > below). Is there something I'm missing? > > When you say they pull in the source crates do you mean the sources of > the other rust packages needed by rust-rand? I didn't test that, but I > assumed it wouldn't. > That's right, I'm interested in having `guix shell` populate the full set of transitive dependencies in the registry, similar to what happens when using `cargo build` in the conventional way (but instead using Guix-packaged crates). I interpreted your message up-thread to mean that you were trying to accomplish the same, but please correct me if not. I took rust-rand as an example only because it does have some dependencies: $ cd $CARGO_PROJECT $ cat Cargo.toml [package] name =3D "test_prog" ... [dependencies] rand =3D "0.8.5" $ cargo build Updating crates.io index Downloaded cfg-if v1.0.0 Downloaded rand_chacha v0.3.1 Downloaded rand v0.8.5 Downloaded ppv-lite86 v0.2.17 Downloaded rand_core v0.6.4 Downloaded getrandom v0.2.12 Downloaded libc v0.2.153 Downloaded 7 crates (932.0 KB) in 0.48s Compiling libc v0.2.153 Compiling cfg-if v1.0.0 Compiling ppv-lite86 v0.2.17 Compiling getrandom v0.2.12 Compiling rand_core v0.6.4 Compiling rand_chacha v0.3.1 Compiling rand v0.8.5 Compiling test_prog v0.1.0 (/home/...) > Currently if you were to pull in rust-rand-0.8 and rust-rand-0.7 then > you'd have both rand-0.*.crate files in the registry but only one of > them would be listed in share/cargo/registry/index/ra/nd/rand. I need to > adjust the generation of that file to combine multiple sources if they > exist, and sort them (I'm not sure it's necessary, but wouldn't be > surprised if we hit undefined behaviour if they were listed multiple > times or out of order). I'm somewhat new to rust, but it appears that outside of Guix, the local-only development workflow looks like this: $ cd $CARGO_PROJECT $ mkdir $VENDOR $ cargo vendor $VENDOR After downloading and unpacking all of the crates into $VENDOR, this last command instructs me to add the following in ~/cargo/config.toml. Then, after opening a new guix shell without network access, I can confirm that `cargo build` works fine with the vendored crates. [source.crates-io] replace-with =3D "vendored-sources" [source.vendored-sources] directory =3D "" Getting back to your patch set: would it make sense to emulate this vendor workflow instead of trying to construct a registry directly? Even assuming that all details of the registry structure are stable and documented, the layout of the vendor directory appears much simpler. And IIUC the code for setting up vendored libraries already exists in cargo-build-system. I also need to figure out something with a > config.toml to see if it's possible to generate one that could be > included from another one, since you can't add 'local-registry =3D > $GUIX_PROFILE/...' in a toml file. > You've probably researched this more than I have, but it seems that this use case is explicitly unsupported in the TOML language spec: https://github.com/toml-lang/toml/issues/397 With that option off the table, I can't think of any elegant solutions. Maybe a wrapper for the cargo binary that pre-processes cargo.toml and then calls the real cargo? --000000000000ed9ed2061301a5bc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Wed, Mar 6, 2024 at 4:06=E2=80=AFA= M Efraim Flashner <efraim@flash= ner.co.il> wrote:
On Mon, Feb 26, 2024 at 09:24:29PM -0500, Jason Conroy wrote: > Hello Efraim,
>
> Thanks for investigating this - a Rust development workflow using only=
> Guix-native crates is something I've been waiting for!
>
> I was experimenting with your patches and it seems that they do pull i= n the
> source crates for requested packages, but not their dependencies (exam= ple
> below). Is there something I'm missing?

When you say they pull in the source crates do you mean the sources of
the other rust packages needed by rust-rand? I didn't test that, but I<= br> assumed it wouldn't.

That's rig= ht, I'm interested in having `guix shell` populate the full set of tran= sitive dependencies in the registry, similar to what happens when using `ca= rgo build` in the conventional way (but instead using Guix-packaged crates)= . I interpreted your message up-thread to mean that you were trying to acco= mplish the same, but please correct me if not.

I took rust-rand as an example only because it does have some dependencies= :

$ cd $CARGO_PROJECT
$ cat Cargo.to= ml
[package]
name =3D "test_prog"
...<= /div>
[dependencies]
rand =3D "0.8.5"

$ cargo build
=C2=A0 =C2=A0 Updating crates.io index
=C2=A0 Downloaded cfg-if v1.0.0
=C2=A0 Downloa= ded rand_chacha v0.3.1
=C2=A0 Downloaded rand v0.8.5
=C2=A0 Downloade= d ppv-lite86 v0.2.17
=C2=A0 Downloaded rand_core v0.6.4
=C2=A0 Downlo= aded getrandom v0.2.12
=C2=A0 Downloaded libc v0.2.153
=C2=A0 Downloa= ded 7 crates (932.0 KB) in 0.48s
=C2=A0 =C2=A0Compiling libc v0.2.153=C2=A0 =C2=A0Compiling cfg-if v1.0.0
=C2=A0 =C2=A0Compiling ppv-lite86 = v0.2.17
=C2=A0 =C2=A0Compiling getrandom v0.2.12
=C2=A0 =C2=A0Compili= ng rand_core v0.6.4
=C2=A0 =C2=A0Compiling rand_chacha v0.3.1
=C2=A0 = =C2=A0Compiling rand v0.8.5
=C2=A0 =C2=A0Compiling test_prog v0.1.0 (/ho= me/...)
=C2=A0
Currently if you were to pull in rust-rand-0.8 and rust-rand-0.7 then
you'd have both rand-0.*.crate files in the registry but only one of them would be listed in share/cargo/registry/index/ra/nd/rand. I need to adjust the generation of that file to combine multiple sources if they
exist, and sort them (I'm not sure it's necessary, but wouldn't= be
surprised if we hit undefined behaviour if they were listed multiple
times or out of order).=C2=A0

I'm some= what new to rust, but it appears that outside of Guix, the local-only devel= opment workflow looks like this:

$ cd $CARGO_PROJE= CT
$ mkdir $VENDOR
$ cargo vendor $VENDOR

After downloading and unpacking all of the crates into $VEN= DOR, this last command instructs me to add the following in ~/cargo/config.= toml.
Then, after opening a new guix shell without network access,= I can=20 confirm that `cargo build` works fine with the vendored crates.

[source.crates-io]
replace-with =3D "vendor= ed-sources"

[source.vendored-sources]
directory =3D "&l= t;VENDOR>"

Getting back to your patch set: woul= d it make sense to emulate this vendor workflow instead of trying to constr= uct a registry directly? Even assuming that all details of the registry str= ucture are stable and documented, the layout of the vendor directory appear= s much simpler. And IIUC the code for setting up vendored libraries already= exists in cargo-build-system.
I also need to fig= ure out something with a
config.toml to see if it's possible to generate one that could be
included from another one, since you can't add 'local-registry =3D<= br> $GUIX_PROFILE/...' in a toml file.

= You've probably researched this more than I have, but it seems that thi= s use case is explicitly unsupported in the TOML language spec: https://github.com/toml-lang/= toml/issues/397

With that option off the table= , I can't think of any elegant solutions. Maybe a wrapper for the cargo= binary that pre-processes cargo.toml and then calls the real cargo?
--000000000000ed9ed2061301a5bc--