From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id yNYeLgDAPGJitgAAgWs5BA (envelope-from ) for ; Thu, 24 Mar 2022 20:01:20 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id CBFrKwDAPGLrFwEA9RJhRA (envelope-from ) for ; Thu, 24 Mar 2022 20:01:20 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0B08D3C2D2 for ; Thu, 24 Mar 2022 20:01:20 +0100 (CET) Received: from localhost ([::1]:39496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nXShy-0003BO-Nu for larch@yhetil.org; Thu, 24 Mar 2022 15:01:18 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38056) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nXSX4-00071v-IK for bug-guix@gnu.org; Thu, 24 Mar 2022 14:50:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:55509) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nXSX3-0000Of-WA for bug-guix@gnu.org; Thu, 24 Mar 2022 14:50:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nXSX3-0004AJ-NV for bug-guix@gnu.org; Thu, 24 Mar 2022 14:50:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#25327: windows-related dependencies in crates References: <87zijathke.fsf@wasp.i-did-not-set--mail-host-address--so-tickle-me> In-Reply-To: <87zijathke.fsf@wasp.i-did-not-set--mail-host-address--so-tickle-me> Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 24 Mar 2022 18:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 25327 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 25327@debbugs.gnu.org Cc: Maxime Devos Received: via spool by 25327-submit@debbugs.gnu.org id=B25327.164814777915978 (code B ref 25327); Thu, 24 Mar 2022 18:50:01 +0000 Received: (at 25327) by debbugs.gnu.org; 24 Mar 2022 18:49:39 +0000 Received: from localhost ([127.0.0.1]:49406 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nXSWg-00049e-OA for submit@debbugs.gnu.org; Thu, 24 Mar 2022 14:49:39 -0400 Received: from 14.mo581.mail-out.ovh.net ([178.33.251.19]:60779) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nXSWe-00049V-JC for 25327@debbugs.gnu.org; Thu, 24 Mar 2022 14:49:37 -0400 Received: from player687.ha.ovh.net (unknown [10.111.208.149]) by mo581.mail-out.ovh.net (Postfix) with ESMTP id D503C24403 for <25327@debbugs.gnu.org>; Thu, 24 Mar 2022 18:49:34 +0000 (UTC) Received: from ngraves.fr (lfbn-lil-1-745-89.w81-254.abo.wanadoo.fr [81.254.78.89]) (Authenticated sender: ngraves@ngraves.fr) by player687.ha.ovh.net (Postfix) with ESMTPSA id 2920928B86964; Thu, 24 Mar 2022 18:49:33 +0000 (UTC) X-OVh-ClientIp: 81.254.78.89 User-agent: mu4e 1.6.10; emacs 28.0.92 Date: Thu, 24 Mar 2022 18:18:59 +0100 Message-ID: <87h77nyymd.fsf@ngraves.fr> MIME-Version: 1.0 Content-Type: text/plain X-Ovh-Tracer-Id: 13636336725616943832 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvvddrudegledguddufecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpegfhffvufffkfggtgesthdtredttdertdenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpeffgffhhfdtueduhfeghfegffeivdeijeejledtuefffeefleeigeekudeigfduheenucfkpheptddrtddrtddrtddpkedurddvheegrdejkedrkeelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmohguvgepshhmthhpohhuthdphhgvlhhopehplhgrhigvrheikeejrdhhrgdrohhvhhdrnhgvthdpihhnvghtpedtrddtrddtrddtpdhmrghilhhfrhhomhepnhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrpdhnsggprhgtphhtthhopedupdhrtghpthhtohepvdehfedvjeesuggvsggsuhhgshdrghhnuhdrohhrgh X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" Reply-to: Nicolas Graves From: Nicolas Graves via Bug reports for GNU Guix X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1648148480; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post; bh=RfFKQiVIX2yjj5a5s/JCiUXD1mpRzyGxB9H82s08shY=; b=EnEptz6tXBaQUVk2caDxiR8NSTqJxqulpdsKBHEx7SMb1oLqIEUhi2piEimTUsqvPa8J4Y fQBUVzq8xR2jgUmu0/ChGhTJpBN+tk65iaUrpq9wJ83/fxc0MscEie39JCrHBIwryXCCqt scfRCMxsMZdX/HhOQQNBrIco+FfRSsiNAv2cd88esSabg3zmsGWxYD8sUJ1TLwpaNMZT6g XmeuB72/CyN/aiqzro8fS1I6k5luVVoSv1cixGlXmIKLbh6idrdOcLBjq6+8IoK8Unf0ks Qh7Fltv0kTexkGTnMQR/EWiU0HG5xhpsngU4Uokqc9DKb3NN5AcEQZtJlbZZjw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1648148480; a=rsa-sha256; cv=none; b=Qmo3W1mPSViykjY1whZxU4m9uocPzpvQhGYOs+WiKgPRTMkZYyWsSrxGpWNzw13Wz3RD+k l5PjhdIuaEJI84Yges+a70zqPA3aUiTXwjMjQVl31hBjMXT+HesZu9RJrl4NVaqVF/rPdK yHU5f2LEx2JQgx0LWyaqGNnvA88DifxMyj2sBoNRdsT9FT77wPoLQYMEfdqRMmJyb3Y0xM /vE7tpJNF52G+L5mZM+/QQA0A4Cb0FKsgGUda/IvAwakzw2Sww3vebSrqPoPwJiiOWblAc oRE0pxHYeLeo4C9VQlTNCIzP9mc5jXOUju1+Ptmwfp6COqHGFXpLXG5pYkIEAQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.69 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 0B08D3C2D2 X-Spam-Score: -4.69 X-Migadu-Scanner: scn0.migadu.com X-TUID: 5TXn9gtCk5my Hi all, I've delved into the subject the past week and now have a clearer view on it : - windows (and mac) dependencies are not needed when compiling a single package. It is possible to bypass it without changing the build-system, by a few means : - either by using the #:cargo-build-flags argument (doesn't work everytime, seems to work only seldomly with --target unix) In this option, one would also need to modify Cargo.toml file before the package phase, since `cargo package` vendors all possible dependencies to ensure a package on all environments. For instance (1) : #:phases (modify-phases %standard-phases (add-before 'package 'unix-cargo-toml (lambda _ (delete-file "Cargo.toml") (substitute* "Cargo.toml.orig" (("^mac-notification-sys.*") "") (("^chrono.*") "") (("^winrt-notification.*") "")) (rename-file "Cargo.toml.orig" "Cargo.toml") #t))) - either by using the environment variable "CARGO_BUILD_RUSTFLAGS" during the build phase, like that for instance (2) : #:phases (modify-phases %standard-phases (add-before 'build 'unix-cargo-toml (setenv "CARGO_BUILD_RUSTFLAGS" (string-append (getenv "CARGO_BUILD_RUSTFLAGS") "--cfg 'target_os=\"linux\"'")))) In this case, the package phase would also require the same change as in (1). Note that this might not be backward compatible further down the line (seen in the cargo manual). For some reason, these two means don't seem to propagate when using such packages as inputs. There are also the options to use a snippet or a patch on the source itself. The snippet on the source could look like (1), or the patch as its effects, and we would still need to use (2). Haven't experimented much with that yet. In any case, all this seems kinda bulky for an edit on an imported base, and since we have lots of information with `cargo metadata`, I imagine it would be possible to modify the build-system itself, so that it only considers `target_os = "linux"` or possibly redox / bsd / dragonfly if guix considers porting to these platforms. I have 2 possible ideas : 1) Maybe something like adding a phase after unpack, which would : - parse `cargo metadata` to isolate only needed dependencies (by maybe generating and applying a patch ? or maybe there's a cargo command for generating a Cargo.toml based on modified metadata ?). In any case, modifying the Cargo.toml to only keep relevant information. 2) Maybe rethinking the package phase entirely instead of relying on `cargo package --no-verify --no-metadata` which seems to not allow options without interferring with the Cargo.toml file. In both cases, we would then need to : - modify the phase were environment variables are defined to add (2). - modify the crate import script accordingly (not trivial). As for the second hypothesis, according to the cargo manual, what `cargo package` does is the following : 1. Load and check the current workspace, performing some basic checks. Path dependencies are not allowed unless they have a version key. Cargo will ignore the path key for dependencies in published packages. dev-dependencies do not have this restriction. 2. Create the compressed .crate file. a. The original Cargo.toml file is rewritten and normalized. b. [patch], [replace], and [workspace] sections are removed from the manifest. c. Cargo.lock is automatically included if the package contains an executable binary or example target. cargo-install(1) will use the packaged lock file if the --locked flag is used. d. A .cargo_vcs_info.json file is included that contains information about the current VCS checkout hash if available (not included with --allow-dirty). 3. Extract the .crate file and build it to verify it can build. - This will rebuild your package from scratch to ensure that it can be built from a pristine state. The --no-verify flag can be used to skip this step. 4. Check that build scripts did not modify any source files. Step 3 and 4 are actually not performed, since the build-system calls cargo package with the --no-verify flag. The checks in step 1 doesn't seem that essential on guix, but I imagine we could replicate them. Clearly the essential step here is 2, but since we get a crate, steps a. and b. should actually already be done at the source level, and step d. is not essential because of origin's field version. Step 2c is not essential either since Cargo.lock file is actually deleted during the build system. I might try to implement these changes, although I do not have enough time and experience to do that quickly, neither enough horsepower to try to rebuild everything after that. I included Maxime Devos in the conversation, would it be possible to fork guix to develop this and try to rebuild every rust package after that on a guix build farm ? Sorry, haven't got in touch with guix developers yet, still imposter's syndrome effect, but implementing this doesn't seem too hard. What would I need to do to get going on that ? Cheers, Nicolas Graves