From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id MGk3DJjw3F+oEgAA0tVLHw (envelope-from ) for ; Fri, 18 Dec 2020 18:10:32 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id CHMJCJjw3F+5PAAA1q6Kng (envelope-from ) for ; Fri, 18 Dec 2020 18:10:32 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D3B769402B5 for ; Fri, 18 Dec 2020 18:10:31 +0000 (UTC) Received: from localhost ([::1]:48738 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kqKD0-0002VC-JL for larch@yhetil.org; Fri, 18 Dec 2020 13:10:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kqKBB-0000U4-2i for guix-devel@gnu.org; Fri, 18 Dec 2020 13:08:38 -0500 Received: from mail-out.m-online.net ([212.18.0.9]:52485) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kqKB7-00046y-5Z for guix-devel@gnu.org; Fri, 18 Dec 2020 13:08:36 -0500 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4CyH0s23K7z1qskD; Fri, 18 Dec 2020 19:08:29 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4CyH0r6sVYz1qql8; Fri, 18 Dec 2020 19:08:28 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id Q8WXOjgd-_67; Fri, 18 Dec 2020 19:08:27 +0100 (CET) Received: from hermia.goebel-consult.de (ppp-188-174-59-23.dynamic.mnet-online.de [188.174.59.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Fri, 18 Dec 2020 19:08:27 +0100 (CET) Received: from lenashee.goebel-consult.de (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id 3829160171; Fri, 18 Dec 2020 19:14:35 +0100 (CET) Subject: Discussion: How to package rust crates now and in future? To: Efraim Flashner , guix-devel References: <87tuskmq7l.fsf@nicolasgoaziou.fr> From: Hartmut Goebel Organization: crazy-compilers.com Message-ID: <995a1d66-1c65-15fd-ea61-669e2160bc71@crazy-compilers.com> Date: Fri, 18 Dec 2020 19:08:24 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------B8A53917B7C92B1FFA5BDC30" Content-Language: en-US Received-SPF: none client-ip=212.18.0.9; envelope-from=h.goebel@crazy-compilers.com; helo=mail-out.m-online.net X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicolas Goaziou Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.32 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: D3B769402B5 X-Spam-Score: -1.32 X-Migadu-Scanner: scn0.migadu.com X-TUID: pC26FNrU2QjD This is a multi-part message in MIME format. --------------B8A53917B7C92B1FFA5BDC30 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, I suggest to discuss this with a broader community. *TIL: Shall rust packages be packaged with #:skip-build #t? Shall tests be run for all crates?* *My vote: skip-build #t, no tests* Background: I just submitted some patches for some crates, setting #:skip-build for those I touched or added. My rational: 1) Building crate "libraries" is of no use. Rust still has no notion of "libraries", neither shared not static. it does not even provide any means to use "object"-files from another package. All crates will be build again and again for each package using it. And you will notice that the output of most crates will be almost empty (only exception: if the crate build a program). 2) This is what the crates importer does: It sets skip-build for all packages it imports as dependencies. It also does not add the crate-build-dependencies for these packages. (Please note that while I made the crates importer to honor semver versions, this has already been prepared in other patches and was not argued about. Am 17.12.20 um 21:08 schrieb Efraim Flashner: > I'm in favor of building the packages anyway, it serves as a check that > the inputs are actually correct. When I started packaging crates, I did this too. But then I learned, that others do not. So we should define how this should be handled in the future - and adjust the importer accordingly. This might not be possible - as there is another issue in the Rust ecosystem: The language is still moving fast. 3) If some packages requires rustc 1.46, while our default rustc is still 1.40, we need to add rustc-1.46 as an input to this package and to many of it's dependents. (AFAIR the package will even depend on *both* version then.) Now if we move on to rustc 1.50, extra care has to be taken to remove these dependencies. Even worse: All packages depending on such a package on will also depend on rustc 1.46, and all changes to rustc 1.46 will trigger a rebuild - without *any* use. 4) Since (2) building rust packages costs *a lot* of resources: time, memory and electrical power. As an example, building sequoia takes about 20 Minutes on my machine. Most of the time is spend compiling dependencies of dependencies. And all these dependencies of dependencies will be compiled over and over again. 5) *If* we decide to build dependencies, we should restrict this to *one* build. This means: either not run the tests or only do a test build. The reason is: When running the tests, all the code, including the dependencies, is compiled again with some "test" flag set. This will add yet another huge amount of time, memory and electrical power. To give you some figure: A release and test build for sequoia takes about 45 minutes on my machine, requiring 9 GB of space in /tmp. So this is double the time if the release build only. I can't imaging how many hours it would take to rebuild sequoia is one of the lower level dependencies changes - which is quite often the case in rust. 6) This not only effect berlin, but also every user out there requiring a rebuild for some reason. This will lead to a very, very bad user experience - practically kicking out users with less powerful equipment. 7) Given the rushing climate crisis, we MUST NOT waste this gigantic amount of electrical power. We are in a position of huge impact. If we decide to save power, hundreds of Guix users will save power (and money). If we decide to waste power, this will multiply by the number of Guix users. *It's our responsibility to protect the earth!**Yes to #:skip-build #t.* -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | --------------B8A53917B7C92B1FFA5BDC30 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit
Hi,

I suggest to discuss this with a broader community.

TIL: Shall rust packages be packaged with #:skip-build #t? Shall tests be run for all crates?
My vote: skip-build #t, no tests

Background: I just submitted some patches for some crates, setting #:skip-build for those I touched or added. My rational:

1) Building crate "libraries" is of no use. Rust still has no notion of "libraries", neither shared not static. it does not even provide any means to use "object"-files from another package. All crates will be build again and again for each package using it. And you will notice that the output of most crates will be almost empty (only exception: if the crate build a program).

2) This is what the crates importer does: It sets skip-build for all packages it imports as dependencies. It also does not add the crate-build-dependencies for these packages. (Please note that while I made the crates importer to honor semver versions, this has already been prepared in other patches and was not argued about.


Am 17.12.20 um 21:08 schrieb Efraim Flashner:
I'm in favor of building the packages anyway, it serves as a check that
the inputs are actually correct.

When I started packaging crates, I did this too. But then I learned, that others do not. So we should define how this should be handled in the future - and adjust the importer accordingly.

This might not be possible - as there is another issue in the  Rust ecosystem: The language is still moving fast.

3) If some packages requires rustc 1.46, while our default rustc is still 1.40, we need to add rustc-1.46 as an input to this package and to many of it's dependents. (AFAIR the package will even depend on *both* version then.) Now if we move on to rustc 1.50, extra care has to be taken to remove these dependencies.

Even worse: All packages depending on such a package on will also depend on rustc 1.46, and all changes to rustc 1.46 will trigger a rebuild - without *any* use.

4) Since (2) building rust packages costs *a lot* of resources: time, memory and electrical power. As an example, building sequoia takes about 20 Minutes on my machine. Most of the time is spend compiling dependencies of dependencies. And all these dependencies of dependencies will be compiled over and over again.

5) *If* we decide to build dependencies, we should restrict this to *one* build. This means: either not run the tests or only do a test build. The reason is: When running the tests, all the code, including the dependencies, is compiled again with some "test" flag set. This will add yet another huge amount of time, memory and electrical power.

To give you some figure: A release and test build for sequoia takes about 45 minutes on my machine, requiring 9 GB of space in /tmp. So this is double the time if the release build only.

I can't imaging how many hours it would take to rebuild sequoia is one of the lower level dependencies changes - which is quite often the case in rust.

6) This not only effect berlin, but also every user out there requiring a rebuild for some reason. This will lead to a very, very bad user experience - practically kicking out users with less powerful equipment.

7) Given the rushing climate crisis, we MUST NOT waste this gigantic amount of electrical power. We are in a position of huge impact. If we decide to save power, hundreds of Guix users will save power (and money). If we decide to waste power, this will multiply by the number of Guix users.

It's our responsibility to protect the earth! Yes to #:skip-build #t.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |
--------------B8A53917B7C92B1FFA5BDC30--