From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id 0AgkNbbmHmXeyQAAG6o9tA:P1 (envelope-from ) for ; Thu, 05 Oct 2023 18:39:19 +0200 Received: from aspmx1.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id 0AgkNbbmHmXeyQAAG6o9tA (envelope-from ) for ; Thu, 05 Oct 2023 18:39:18 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A64BC3B11A for ; Thu, 5 Oct 2023 18:39:18 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lease-up.com header.s=2017 header.b=fuf5hLW5; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1696523958; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=NOCWmJ3tmIUYxdWQZy+KCQPByh5IBJ88GziRJWZsOfQ=; b=bBQa/08Y3AuP8TkyWHf7JvwRK144IMzRlw5YJFuzGlrjllD/9geapavSWD/CBBYuyot6U3 rnkh6PBp3v7Ojofnytt4VRNUexvIuKOp6ksBI66MtdtOgz9S+PqbJMbN4vcdo0AANF360M lijy/Dnqm4PLCz3mFU8uIqjCwiTSgJkOkZA4nvWH96u/lTTwaj8VBbc3xTcDMLPkknO1Bc ANg/TcF8aDhsAX+qlVjvMlWot92nGePpteXzkzWTkPqJC9biGDw7Eiw8kGVcCDpXjSQY87 /AsYukpGrEPzTo9w6A7eIkWgF9MYtgwlNiaLG9Abw1R4Fsid4H2ROjvYXAvTYg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lease-up.com header.s=2017 header.b=fuf5hLW5; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Seal: i=1; s=key1; d=yhetil.org; t=1696523958; a=rsa-sha256; cv=none; b=nfSdv4v/UGm2UBxx2wXZIIw4v1+QHtiDfcfrCGHTh8yRJ7+MR99KO99sS5LmHzDDSFpHIC utS0tQOsXlwFyxBpXDKAAbpAPQJWlYfgv2BfnnspuaWE4T5R4TVmDieIGhnM2jXJO28hpO jQ6BMvXa847XlV9FKGtvKYPydxnEi3g1xWZC9/V5wT6xtTCnM5cwLa03otelRHWxXC7ZfT JXnvtO8qMpen4KvEpxyyw5j1cuTjYxpRBC2QFd1AG4+07XjFPjTXWgvmiNmKKKC4qcq0Bs 6UqDoS8PsiKsGfhea0GHcKeEeQcWMKc8QRO9W19zg9ggImTN3H4mkeZrLdgcSQ== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qoRN1-0003rz-0C; Thu, 05 Oct 2023 12:38:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qoRMv-0003qj-2v for guix-devel@gnu.org; Thu, 05 Oct 2023 12:38:33 -0400 Received: from sail-ipv4.us-core.com ([208.82.101.137]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256) (Exim 4.90_1) (envelope-from ) id 1qoRMs-0001gq-Po for guix-devel@gnu.org; Thu, 05 Oct 2023 12:38:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; s=2017; bh=NOCWmJ3tmIUYxdW QZy+KCQPByh5IBJ88GziRJWZsOfQ=; h=date:references:in-reply-to:subject: cc:to:from; d=lease-up.com; b=fuf5hLW5/izZi6JBbj59euQsmLt2zzFfpTKpu1Ra c7cZLukxglOUvWsarvehY9rh1swmcLGuUlmZG7g3O8YaYeRi4M9QXtRvpk7vCIOlA4TwO0 5MJygZhh3ke/MID/O0ICWI5N+QpFKt7AoBfpH2dsNqu5HVM/nUAwUxfDPx47c= Received: by sail-ipv4.us-core.com (OpenSMTPD) with ESMTPSA id 308c310e (TLSv1.3:TLS_CHACHA20_POLY1305_SHA256:256:NO); Thu, 5 Oct 2023 16:38:27 +0000 (UTC) To: Efraim Flashner , guix-devel@gnu.org Cc: Subject: Re: [RFC]: Skipping rust crate tests by default In-Reply-To: References: Date: Thu, 05 Oct 2023 09:38:27 -0700 Message-ID: <87edi9owho.fsf@lease-up.com> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=208.82.101.137; envelope-from=felix.lechner@lease-up.com; helo=sail-ipv4.us-core.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Felix Lechner From: Felix Lechner via "Development of GNU Guix and the GNU System distribution." Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -6.69 X-Migadu-Scanner: mx2.migadu.com X-Migadu-Queue-Id: A64BC3B11A X-Spam-Score: -6.69 X-TUID: yf/BQkePSU8a Hi Ephraim, On Thu, Oct 05 2023, Efraim Flashner wrote: > I propose changing the cargo-build-system to have '#:tests? #f' by > default At first sight, it appears improper to turn off tests because they fail. Please allow me to remind everyone that build-time tests cover only a small proportion of problems actually encountered by users. Most packaging errors, like improper permissions or the installation of components in a wrong path, usually go undetected. One of Debian's solutions to that realization is autopkgtest [1] which allows maintainers to specify a test bed that tests the *installed* versions and not the versions in the build trees. A common strategy for Debian maintainers is to convert the build-time tests to an autopkgtest suite. That way, folks get the benefit of both. Unfortunately, the setup of test beds is complicated in Debian, as the installation of the packages being tested has to take place in containers. In Guix, package installations are decoupled from the running system. Guix would make that process a lot easier, faster and more reliable! In summary, I believe that #:test? should be turned off for all build systems. Guix should instead test installed versions like Debian's autopkgtest. It would be an extra burden on contributors because such a 'test-installed phase would require more attention. It may be worthwhile, however, because than packages could be built without testing them---as Ephraim would like to do here. In addition, pre-built substitutes could be tested by consumers on their own systems. The substitutes could even be tested before they become part of any Guix profile. For Debian's QA tool Lintian, which I maintained for several years, the speed-up in the development process was remarkable. As a Perl script, the build toook seven minutes, while the build-time tests took seven hours. The builds were initiated with each commit in Salsa (in an online runner sponsored by a large company). Lintian was extreme because thousands of tests replicated much of what happens daily in Debian. The extreme duration of build-time the tests also took up further resources in downstream distributions. The tests ran for each backport and for each derivative, such as Ubuntu. For Guix, which relies on frequent rebuilds, the speed benefit could be remarkable. Substitutes could become available for testing in perhaps half the time. That being said, old habits die hard. The attachment to build-time tests is formidable. The people who maintained Lintian after me enabled them again. Kind regards Felix [1] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst