From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ivan Petkov Subject: Re: Current state of cargo-build-system Date: Fri, 25 Jan 2019 15:57:03 -0800 Message-ID: <2E175E19-40C4-4954-A282-EE31C77D7A5D@gmail.com> References: <0FA91CF1-5040-4175-9347-7FEF74CCD725@gmail.com> <20190123181000.6c5fc532@scratchpost.org> Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([209.51.188.92]:52925) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gnBLr-0004Cu-G8 for guix-devel@gnu.org; Fri, 25 Jan 2019 18:57:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gnBLa-0006xX-PK for guix-devel@gnu.org; Fri, 25 Jan 2019 18:57:21 -0500 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]:45309) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gnBLW-0006qh-Qg for guix-devel@gnu.org; Fri, 25 Jan 2019 18:57:16 -0500 Received: by mail-pf1-x436.google.com with SMTP id g62so5429734pfd.12 for ; Fri, 25 Jan 2019 15:57:08 -0800 (PST) In-Reply-To: <20190123181000.6c5fc532@scratchpost.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Danny Milosavljevic Cc: guix-devel@gnu.org Thank you all for the responses! > Cycles. Also, often Cargo.lock specifies exact versions of = dependencies (in > programs, at least) Yes, this is pretty much the main issue I immediately hit when doing a = naive crate import, more on this further down. > First, we'd have to find out what kind of things Cargo can build and = how to > detect it. Cargo supports a subcommand called `metadata` which essentially dumps = its manifest in JSON format, which will include build scripts, binary = outputs, etc. I think this should give us enough information to figure out which = packages have targets to install (right now we try to install every single package = which doesn't work for libraries) among other things. > After that, we can decide whether to compile libraries or just ship = them as > source (right now, we do the latter pretty often - almost everything = in Guix > Rust library packages ends up in the "src" output). > =20 > Getting the unit tests to run often requires breaking many cycles, so = maybe > skip that at first. I tried breaking up cycles by hand for a couple days on another sample = import, but its just too tedious and error prone, especially when cycles span = multiple packages... I'd really like for us to solve the issue altogether if = possible. Cargo (thankfully) checks and rejects an true dependency cycles, but its notion of dev-dependencies (deps only used for running tests) is what = throws guix in for a loop. Dev-depencency cycles are permitted by cargo because = it builds the target crate independently, then it builds any extra = dependencies which may or may not depend on it, and then it builds a test crate which = depends on both. There are potentially legitimate cases which trigger this = scenario (such as testing your own public API through an upstream crate, see = proc-macro2 for an example), or are just exacerbated by optional dependencies. I think depending on the "src" output of crates is the right way to get = things started (this is what cargo essentially does anyway, it figures out what = crates are in the dependency, pulls their sources down and builds them), we can = always optimize this in the future to avoid recursively rebuilding crates over = and over. However, it appears that guix still insists on building the entire = package even if we only depend on the "src" output. Is it possible to lazily build = packages based on the type of dependency? Is this something the build system can = finagle, or is this an inherent limitation to guix? --Ivan=