From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: Re: Overhauling the cargo-build-system Date: Thu, 19 Dec 2019 17:09:20 +0100 Message-ID: <87pngkpaa7.fsf@gnu.org> References: <20191010155056.GD1301@E5400> <87d0f4p6xd.fsf@gnu.org> <20191011141342.GC13364@E5400> <20191010155056.GD1301@E5400> <87d0f4p6xd.fsf@gnu.org> <20191011141342.GC13364@E5400> <87imnjtsk3.fsf@gnu.org> <20191117071934.GC12423@E5400> <20191010155056.GD1301@E5400> <87d0f4p6xd.fsf@gnu.org> <20191011141342.GC13364@E5400> <87imnjtsk3.fsf@gnu.org> <87imnjtsk3.fsf@gnu.org> <87h82ap0oc.fsf_-_@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:47078) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihyMg-0005SF-R8 for guix-devel@gnu.org; Thu, 19 Dec 2019 11:09:28 -0500 In-Reply-To: <87h82ap0oc.fsf_-_@gmail.com> (Chris Marusich's message of "Sun, 08 Dec 2019 20:45:07 -0800") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Chris Marusich Cc: guix-devel@gnu.org Hi Chris, Chris Marusich skribis: > Ludovic Court=C3=A8s writes: > >> What I would have liked is to somehow replace the #:cargo-inputs >> argument (which is build-system-specific and thus =E2=80=9Copaque=E2=80= =9D) with regular >> =E2=80=98native-inputs=E2=80=99 or =E2=80=98inputs=E2=80=99 field. > > That would be nice. However, it doesn't seem possible to express > Cargo's "dependencies" and "dev-dependencies" concepts using Guix's > current package DSL. > > Consider the proc-macro2 and quote crates. We added these two crates in > commit 2444abd9c124cc55f8f19a0462e06a2094f25a9d, in the same patch > series where we added #:cargo-inputs and #:cargo-development-inputs: > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D35318 > > Here is the Cargo.toml file for proc-macro2: > > https://github.com/alexcrichton/proc-macro2/blob/master/Cargo.toml > > [dev-dependencies] > quote =3D { version =3D "1.0", default_features =3D false } > > And here is the Cargo.toml file for quote: > > https://github.com/dtolnay/quote/blob/master/Cargo.toml > > [dependencies] > proc-macro2 =3D { version =3D "1.0", default-features =3D false } > > Here is a diagram of their dependency relationship: > > +---------------+ > | quote | <+ > +---------------+ | > | | > | dependencies | dev-dependencies > v | > +---------------+ | > | proc-macro2 | -+ > +---------------+ > > To Cargo, this cycle is not a problem, since "dev-dependencies" are > treated differently from "dependencies": > > https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html > > "Dev-dependencies are not used when compiling a package for building, > but are used for compiling tests, examples, and benchmarks. > > These dependencies are not propagated to other packages which depend > on this package." > > The reason proc-macro2 declares a "dev-dependency" on quote is because > proc-macro2 uses quote in its doc tests: > > https://github.com/alexcrichton/proc-macro2/blob/e82e8571460a0a0e00f52f= 011a74a5e0359acf3e/src/lib.rs#L785 I see. > This relationship between proc-macro2 and quote cannot be readily > expressed using the current package DSL in Guix. If you try to model > "dependencies" and "dev-dependencies" as "inputs" (or "native-inputs", > or some combination of the two), Guix will fail due to the cycle. True, but we have the same problem with many non-Rust packages, which we address in various way=E2=80=94e.g., via an intermediate =E2=80=9C-minimal= =E2=80=9D variant. > Presumably, proc-macro2 just needs the source of quote (and the source > of proc-macro2's other dependency, unicode-xid). When Cargo builds > proc-macro2, it will take care of building quote and making it available > during proc-macro2's tests. Guix "just" needs to provide proc-macro2 > with the quote source. You might think this poses a bootstrapping > problem for Cargo, but I guess it doesn't. As long as Cargo has the > source for proc-macro2, quote, and unicode-xid, I guess it can build > proc-macro2 and quote in any order. In that case, would it work to turn =E2=80=9Cdev-dependencies=E2=80=9D into= dependencies on the source rather than on the package? > Unless we missed something in our discussion of patch 35318, there is no > easy way to express the relationship between proc-macro2 and quote > without changing (or mis-using) the existing package DSL. In the same > way that the package DSL introduced "native-inputs" and "inputs" as > concepts to facilitate cross-compilation, one way to solve this problem > might be to introduce a new concept to the package DSL that would make > it possible for Guix to express this kind of relationship correctly. > > However, in the discussion of patch 35318, everyone (myself included) > seemed opposed to changing the package DSL if we didn't have to. For > example, in response to an earlier version of the patch series in which > we tried to map "dependencies" and "dev-dependencies" onto the "inputs" > and "native-inputs" concepts (which was probably an abuse of the package > DSL, since "native-inputs" is a cross-compilation concept), you said: "I > don't understand yet why you change the role of 'inputs' compared to how > it is in the rest of Guix." Ultimately, we decided not to modify the > package DSL or the meaning of "inputs". Instead, we decided to encode > the necessary information about dependencies in the cargo-build-system's > arguments. That is how we arrived at #:cargo-inputs and > #:cargo-development-inputs. OK, thanks for the reminder. :-) I=E2=80=99d be curious to hear what Ivan and others think of in hindsight. > By introducing #:cargo-inputs and #:cargo-development-inputs as package > arguments to the cargo-build-system, we were able to solve the cyclic > dependency problem in one specific way. Perhaps there are better ways. > I agree it would be nice if it were integrated into the package DSL. I > think that changing the package DSL to suit our needs might work, but > I'm not sure how to change it without making it too Cargo-specific. Still, the notion of dependency definitely exists in Cargo, so it would seem natural to use =E2=80=98inputs=E2=80=99 or =E2=80=98native-inputs=E2= =80=99 to express that. I realize my understanding of Crates is still too limited, but I think our goal should be to somehow have something that=E2=80=99s closer to normal objects. We=E2=80=99ll probably still need Cargo-specific extens= ions, but it=E2=80=99d be nice if the graph for Crates was meaningful. Also, =E2=80=98guix import crates=E2=80=99 has all the info so we can alway= s tweak it to generate something different. Thanks! Ludo=E2=80=99.