From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id aBfsB7iS+GWPVgEAqHPOHw:P1 (envelope-from ) for ; Mon, 18 Mar 2024 20:15:04 +0100 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id aBfsB7iS+GWPVgEAqHPOHw (envelope-from ) for ; Mon, 18 Mar 2024 20:15:04 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1710789303; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Jut8wNoqgE67CxHtfMt/x240J+1IkFPtpbNWPtBN9cM=; b=reggMjLIM5dG1y3l0nY/2OHWGAW4nk1mpqQojBGEkz0GeaFyCS2/VFHOZScCG5cVI7SXhO j+2lEKcXf81W/AtBOKHr5v89lPa9e/bq3DlTwUseQ0s0WsBVEG7Y3NOtuVentYGd6yHaE3 EgPKF4P4Py7F8uoBoeYEoxQAcnxgq1TMEhKwTlNqVLAxVi3UOlpb1DyBnr9AHArRV/xgXk i6Zo6CWPygxHSGovhjYzUU8zXIiBKGay2XiEf16+ilHPWisxPW2HJuBSTZDor2LIiwYVPl TxHTcDgUGyd4qgInRikHSXexlMhxy2FJNxYUi3w4mC4wSN0nGft6caaESq8OGA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1710789303; a=rsa-sha256; cv=none; b=a5fpt5nlMeDQNwNGLOCO98uXUIgia7WCe6hIAUEmjBMOydDF1sxI7SPVA7EbsOJ+WP16fC oQVhn222pmRwfGcRa7gc/EPjLh47TiG+rR5FkoeAucRGF/K/qKjfLd2TTjodiecpcAG5YI XbjA1xlYNGJnD7PJZgp3ck4i6cEehDBiYS3aJzsIP7P4TXoYSxaj5ors0MFMS3RZF/0vKw wlTHxlLeSn5GAKAg01uAzjNolTzrkRaffS7l8/HwpMB3h9rMhKqi3wf0RC8n2Urunl8LFx /8z8ZC3i42xOJnjvLI1aO++hMxMJhsLlPdjsBQFh7m59cJO4RzDbQegewpLDlQ== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 916D54684F for ; Mon, 18 Mar 2024 20:15:03 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rmIRA-0000Mc-Ci; Mon, 18 Mar 2024 15:14:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1riGw3-0007qa-PB for guix-devel@gnu.org; Thu, 07 Mar 2024 11:49:36 -0500 Received: from mail-yw1-x112b.google.com ([2607:f8b0:4864:20::112b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1riGvz-0002jd-9r for guix-devel@gnu.org; Thu, 07 Mar 2024 11:49:35 -0500 Received: by mail-yw1-x112b.google.com with SMTP id 00721157ae682-60822b444c9so8606767b3.2 for ; Thu, 07 Mar 2024 08:49:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709830169; x=1710434969; darn=gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Jut8wNoqgE67CxHtfMt/x240J+1IkFPtpbNWPtBN9cM=; b=Sk1BvIGW+cJ460zsxqIs9MHIXjN7zZ63bTXAZPD3/GQSiQx6fR1PBvyh3Dh//fx7Q/ 3R+ht969eyRdOM0M1/p3niJ2Dl5W6blSeLvZMDChibZgZSVvYloI6NBnRqS4gyLncxOp 22fzqE6U3MSc+UOS6CsdGsWixsxZhuOTQZDI9UtkPQ33zDMXl3xfTrW3V+IRWeH7cL6I DGRo2wzv3QjcTA0AJjtwzErA8GNBXL+5ZFgNjvrYFcZajN5hNoSJusLVvvYZ4HEAWkhT eTao8K0o7L7+J/ZmsAUOgmu5wiA+mAiv/jwy0Dy9pFu59j6V5/bNA9NxmA07JONnOjhs Zr2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709830169; x=1710434969; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Jut8wNoqgE67CxHtfMt/x240J+1IkFPtpbNWPtBN9cM=; b=HfaCJunYBPANs7jJTdr4PPfo5dCFAtbg4XrFm3mLZh5knYUdOynsmIU15MbBhg5xKU nYxO+MmkQeXf94thIzsy6j6p02fpilUa8JdKZEeorge48b8Nda+8O/WTSTY5tdDXn4yp FULJjkJt5qLEN/ODYqdjHUsTw+ZeWn1iBQgD2rCPTyzOiEmnNsu9Tn1LHSfjd3ZeSESi 2bakDYeaezDbDxvCky6Ob6Z1ldhLRjOIhkmDFq9o7kh3oAXpi/W8TPNRxGB/meLBROfZ U8rY1nvFDwyPVCXLSDs9jquyB8Y+XIZTaGnW0jYrYtMjGToIH0QHjwrmMDQ4dMFNr6QU G5wA== X-Gm-Message-State: AOJu0YyWVlxxcY/RBGoI0pIkoQjx3upsxK94640zwMA9LUCQltB2jnWN WNzExljRzuhG9P8oMhn9HLkvEpuab7x7P/szQn1IoaHL/8pqe/BMfGCAohciG2n7SD9gj8xG5Px x4k+mq0ANg8prFCy3PCRx888wyt6n4hI7Bfg= X-Google-Smtp-Source: AGHT+IEAjdByd3A91lK5FK19+LWV+JYgMtAs9WRfRbMWexyuIGwZxYX4D8jvj4RAi7FtTVvkooQy/VDgT+RYKHzmvF8= X-Received: by 2002:a0d:c046:0:b0:609:254f:766d with SMTP id b67-20020a0dc046000000b00609254f766dmr19864693ywd.26.1709830169390; Thu, 07 Mar 2024 08:49:29 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Jason Conroy Date: Thu, 7 Mar 2024 11:48:53 -0500 Message-ID: Subject: Re: rust-team branch merged To: Jason Conroy , Efraim Flashner Cc: guix-devel@gnu.org Content-Type: multipart/alternative; boundary="000000000000df788c061314dbfb" Received-SPF: pass client-ip=2607:f8b0:4864:20::112b; envelope-from=conjaroy@gmail.com; helo=mail-yw1-x112b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 18 Mar 2024 15:14:18 -0400 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -5.21 X-Spam-Score: -5.21 X-Migadu-Queue-Id: 916D54684F X-Migadu-Scanner: mx13.migadu.com X-TUID: kEsgIO/8qVBM --000000000000df788c061314dbfb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Mar 7, 2024 at 3:08=E2=80=AFAM Efraim Flashner wrote: > The transitive dependencies getting pulled in automatically should work > automatically if we ever finish the antioxidant-build-system. Since you bring up antioxidant, I'm kind of curious whether that stalled mainly due to shifts in contributor priorities, or due to significant technical issues. > Until then > I've been experimenting by manually listing the other crates I've needed > but in theory we could try to make `guix shell --development` pull in > the needed crates. > I was considering another option that falls somewhere in between: since I'm already building shells from manifest files, it should suffice to have a Scheme utility function that calculates the transitive dependencies for a given list of library packages. Similar logic seems to exist already in `(guix build-system cargo)` but it's not exposed publicly. As interim solutions go, what do you think about this one versus modifying `guix shell`? > > I misread what you wrote as it was working. It's definitely something I > want but wasn't ready to work on yet. > > > I took rust-rand as an example only because it does have some > dependencies: > > > > $ cd $CARGO_PROJECT > > $ cat Cargo.toml > > [package] > > name =3D "test_prog" > > ... > > [dependencies] > > rand =3D "0.8.5" > > > > $ cargo build > > Updating crates.io index > > Downloaded cfg-if v1.0.0 > > Downloaded rand_chacha v0.3.1 > > Downloaded rand v0.8.5 > > Downloaded ppv-lite86 v0.2.17 > > Downloaded rand_core v0.6.4 > > Downloaded getrandom v0.2.12 > > Downloaded libc v0.2.153 > > Downloaded 7 crates (932.0 KB) in 0.48s > > Compiling libc v0.2.153 > > Compiling cfg-if v1.0.0 > > Compiling ppv-lite86 v0.2.17 > > Compiling getrandom v0.2.12 > > Compiling rand_core v0.6.4 > > Compiling rand_chacha v0.3.1 > > Compiling rand v0.8.5 > > Compiling test_prog v0.1.0 (/home/...) > > > > > > > Currently if you were to pull in rust-rand-0.8 and rust-rand-0.7 then > > > you'd have both rand-0.*.crate files in the registry but only one of > > > them would be listed in share/cargo/registry/index/ra/nd/rand. I need > to > > > adjust the generation of that file to combine multiple sources if the= y > > > exist, and sort them (I'm not sure it's necessary, but wouldn't be > > > surprised if we hit undefined behaviour if they were listed multiple > > > times or out of order). > > > > > > I'm somewhat new to rust, but it appears that outside of Guix, the > > local-only development workflow looks like this: > > > > $ cd $CARGO_PROJECT > > $ mkdir $VENDOR > > $ cargo vendor $VENDOR > > > > After downloading and unpacking all of the crates into $VENDOR, this la= st > > command instructs me to add the following in ~/cargo/config.toml. > > Then, after opening a new guix shell without network access, I can > confirm > > that `cargo build` works fine with the vendored crates. > > > > [source.crates-io] > > replace-with =3D "vendored-sources" > > > > [source.vendored-sources] > > directory =3D "" > > I wanted local-registry over replace-with because IIRC replace-with > won't fall back to downloading from crates.io if there's missing crates, > while local-registry will check there first and then download any > missing crates. The use-case I was looking at for that was adding a new > dependency to a project and then not needing to re-create a shell or > package the new crates before continuing on. > I see, thanks. My preferred workflow is different but I acknowledge that use case. The link below claims that one can update the vendor directory in a similar way by re-running `cargo vendor` after adding a dependency to Cargo.toml, but for your use case I agree that it's nicer if `cargo build` can pull to the registry automatically. https://old.reddit.com/r/rust/comments/uvvmjy/how_to_include_vendored_crate= s_into_a_project/ > The registry as setup in the patches is actually mostly correct, minus > the multiple versions of a crate, it just needs to be fed the crates. > > > Getting back to your patch set: would it make sense to emulate this > vendor > > workflow instead of trying to construct a registry directly? Even > assuming > > that all details of the registry structure are stable and documented, t= he > > layout of the vendor directory appears much simpler. And IIUC the code > for > > setting up vendored libraries already exists in cargo-build-system. > > > > I also need to figure out something with a > > > config.toml to see if it's possible to generate one that could be > > > included from another one, since you can't add 'local-registry =3D > > > $GUIX_PROFILE/...' in a toml file. > > > > > > > You've probably researched this more than I have, but it seems that thi= s > > use case is explicitly unsupported in the TOML language spec: > > https://github.com/toml-lang/toml/issues/397 > > > > With that option off the table, I can't think of any elegant solutions. > > Maybe a wrapper for the cargo binary that pre-processes cargo.toml and > then > > calls the real cargo? > > Thanks, I didn't see that one. So it looks like environment variables > and INCLUDEs are off the table. An autoconf style macro that take a > config.toml.in and spits out a config.toml with the correct directory > would work, but from my understanding that's not really in line with any > bit of how cargo and the rust ecosystem works. > > Another option would be to symlink > $GUIX_ENVIRONMENT/share/cargo/registry to ./local-registry and then add > the line 'local-registry =3D 'local-registry' in a config.toml. > > -- > Efraim Flashner =D7=A8=D7=A0=D7=A9=D7=9C=D7= =A4 =D7=9D=D7=99=D7=A8=D7=A4=D7=90 > GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 > Confidentiality cannot be guaranteed on emails sent or received unencrypt= ed > --000000000000df788c061314dbfb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Mar 7, 2024 at 3:08=E2=80=AFA= M Efraim Flashner <efraim@flash= ner.co.il> wrote:
The transitive dependencies getting pulled in automatically shou= ld work
automatically if we ever finish the antioxidant-build-system.
=

Since you bring up antioxidant, I'm kind of curious= whether that stalled mainly due to shifts in contributor priorities, or du= e to significant technical issues.
=C2=A0
Until then
I've been experimenting by manually listing the other crates I've n= eeded
but in theory we could try to make `guix shell --development` pull in
the needed crates.

I was considering an= other option that falls somewhere in between: since I'm already buildin= g shells from manifest files, it should suffice to have a Scheme utility fu= nction that calculates the transitive dependencies for a given list of libr= ary packages. Similar logic seems to exist already in `(guix build-system c= argo)` but it's not exposed publicly. As interim solutions go, what do = you think about this one versus modifying `guix shell`?
=C2= =A0

I misread what you wrote as it was working. It's definitely something I=
want but wasn't ready to work on yet.

> I took rust-rand as an example only because it does have some dependen= cies:
>
> $ cd $CARGO_PROJECT
> $ cat Cargo.toml
> [package]
> name =3D "test_prog"
> ...
> [dependencies]
> rand =3D "0.8.5"
>
> $ cargo build
>=C2=A0 =C2=A0 =C2=A0Updating crates.io index
>=C2=A0 =C2=A0Downloaded cfg-if v1.0.0
>=C2=A0 =C2=A0Downloaded rand_chacha v0.3.1
>=C2=A0 =C2=A0Downloaded rand v0.8.5
>=C2=A0 =C2=A0Downloaded ppv-lite86 v0.2.17
>=C2=A0 =C2=A0Downloaded rand_core v0.6.4
>=C2=A0 =C2=A0Downloaded getrandom v0.2.12
>=C2=A0 =C2=A0Downloaded libc v0.2.153
>=C2=A0 =C2=A0Downloaded 7 crates (932.0 KB) in 0.48s
>=C2=A0 =C2=A0 Compiling libc v0.2.153
>=C2=A0 =C2=A0 Compiling cfg-if v1.0.0
>=C2=A0 =C2=A0 Compiling ppv-lite86 v0.2.17
>=C2=A0 =C2=A0 Compiling getrandom v0.2.12
>=C2=A0 =C2=A0 Compiling rand_core v0.6.4
>=C2=A0 =C2=A0 Compiling rand_chacha v0.3.1
>=C2=A0 =C2=A0 Compiling rand v0.8.5
>=C2=A0 =C2=A0 Compiling test_prog v0.1.0 (/home/...)
>
>
> > Currently if you were to pull in rust-rand-0.8 and rust-rand-0.7 = then
> > you'd have both rand-0.*.crate files in the registry but only= one of
> > them would be listed in share/cargo/registry/index/ra/nd/rand. I = need to
> > adjust the generation of that file to combine multiple sources if= they
> > exist, and sort them (I'm not sure it's necessary, but wo= uldn't be
> > surprised if we hit undefined behaviour if they were listed multi= ple
> > times or out of order).
>
>
> I'm somewhat new to rust, but it appears that outside of Guix, the=
> local-only development workflow looks like this:
>
> $ cd $CARGO_PROJECT
> $ mkdir $VENDOR
> $ cargo vendor $VENDOR
>
> After downloading and unpacking all of the crates into $VENDOR, this l= ast
> command instructs me to add the following in ~/cargo/config.toml.
> Then, after opening a new guix shell without network access, I can con= firm
> that `cargo build` works fine with the vendored crates.
>
> [source.crates-io]
> replace-with =3D "vendored-sources"
>
> [source.vendored-sources]
> directory =3D "<VENDOR>"

I wanted local-registry over replace-with because IIRC replace-with
won't fall back to downloading from crates.io if there's missing crates,=
while local-registry will check there first and then download any
missing crates.=C2=A0 The use-case I was looking at for that was adding a n= ew
dependency to a project and then not needing to re-create a shell or
package the new crates before continuing on.

I see, thanks. My preferred workflow is different but I acknowledge t= hat use case.=C2=A0

The link below claims that one= can update the vendor directory in a similar way by re-running `cargo vend= or` after adding a dependency to Cargo.toml, but for your use case I agree = that it's nicer if `cargo build` can pull to the registry automatically= .
<= div>

The registry as setup in the patches is actually mostly correct, minus
the multiple versions of a crate, it just needs to be fed the crates.

> Getting back to your patch set: would it make sense to emulate this ve= ndor
> workflow instead of trying to construct a registry directly? Even assu= ming
> that all details of the registry structure are stable and documented, = the
> layout of the vendor directory appears much simpler. And IIUC the code= for
> setting up vendored libraries already exists in cargo-build-system. >
> I also need to figure out something with a
> > config.toml to see if it's possible to generate one that coul= d be
> > included from another one, since you can't add 'local-reg= istry =3D
> > $GUIX_PROFILE/...' in a toml file.
> >
>
> You've probably researched this more than I have, but it seems tha= t this
> use case is explicitly unsupported in the TOML language spec:
> https://github.com/toml-lang/toml/issues/397 >
> With that option off the table, I can't think of any elegant solut= ions.
> Maybe a wrapper for the cargo binary that pre-processes cargo.toml and= then
> calls the real cargo?

Thanks, I didn't see that one.=C2=A0 So it looks like environment varia= bles
and INCLUDEs are off the table.=C2=A0 An autoconf style macro that take a conf= ig.toml.in and spits out a config.toml with the correct directory
would work, but from my understanding that's not really in line with an= y
bit of how cargo and the rust ecosystem works.

Another option would be to symlink
$GUIX_ENVIRONMENT/share/cargo/registry to ./local-registry and then add
the line 'local-registry =3D 'local-registry' in a config.toml.=

--
Efraim Flashner=C2=A0 =C2=A0<efraim@flashner.co.il>=C2=A0 =C2=A0=D7=A8=D7=A0=D7= =A9=D7=9C=D7=A4 =D7=9D=D7=99=D7=A8=D7=A4=D7=90
GPG key =3D A28B F40C 3E55 1372 662D=C2=A0 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted=
--000000000000df788c061314dbfb--