From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id 4C/XLGoNsWZY5AAAe85BDQ:P1 (envelope-from ) for ; Mon, 05 Aug 2024 17:35:38 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id 4C/XLGoNsWZY5AAAe85BDQ (envelope-from ) for ; Mon, 05 Aug 2024 19:35:38 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=marekpasnikowski.pl header.s=dkim header.b=Y+rIPy0v; dmarc=pass (policy=reject) header.from=marekpasnikowski.pl; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1722879338; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=060ZyV0QVOcWziyMIkGGNytLySNYsajGtwFppbSHTto=; b=jEuI9nSSJGWJyGCNquLp9I8b4Ab4pH8CufKrVlZamLKc1K4z1JxSEWV8Qr9bCHQAYBE4as GAQ2tj3tz89ogrltc/LYt4Wxp8+kYH8pi8tjRANSlRQ4dgKyk7vaxJLaLbial8DfDLxZf5 hSlzJ/izjYucGPccw8Js/3A+nRV47G8Xk7QnDRDRTntC9TbZcnTcSXNYlzTmDfEkG/bM80 aa7L0nS90czN2ONxmR/bs50bzQrbOjbCpv7LE4ElBQyeed8fkFoIGUruIM6yxyOjOrz+YC Tw8DbLvFnAalmMOpXzfOqp5VWldRhCGaHapz3zDgZ4jqVcXSIe8gfg3Dt33brg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1722879338; a=rsa-sha256; cv=none; b=MniZFWkz9reFtMYr9dc9i6zsXk3CQ0NDPooDIMl4hF68YJ5+QQtsyoLtDbaEQW+XrbZkt4 rTMU5WvNt4GHg2WNSo0oPvawhXsaL5WOvt5hvtQ8CvcR2PQfghJ+DcZJ7yJ7BfAxmShHnv se1cMJ3mH52g5gYqM0InEjqZYhhlfzCADhhcRcyx1dbjAfsUVOtopnPkprHIOL0WIg+3jh zowAYqJEoQK2OwYbssUvcSWTQERfs4u0g+g4z+EqxDmoe2r7+XPQNsNGO6/RZ+3MT0wwqc S2njKQvDqROPGVIiBrPn2QefPp32EbmbKqlQlgtkQQKVSQNlxd173WR/dK1Avw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=marekpasnikowski.pl header.s=dkim header.b=Y+rIPy0v; dmarc=pass (policy=reject) header.from=marekpasnikowski.pl; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CD2551F78D for ; Mon, 5 Aug 2024 19:35:37 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sb1cA-0004Ne-0J; Mon, 05 Aug 2024 13:35:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZpNY-0004yH-FC for guix-devel@gnu.org; Fri, 02 Aug 2024 06:19:22 -0400 Received: from [81.190.248.246] (helo=marekpasnikowski.pl) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZpNW-0003iz-7x for guix-devel@gnu.org; Fri, 02 Aug 2024 06:19:20 -0400 Received: from localhost (localhost.local [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id b9a80662 for ; Fri, 2 Aug 2024 10:19:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=marekpasnikowski.pl; h= from:to:subject:in-reply-to:references:date:message-id :mime-version:content-type:content-transfer-encoding; s=dkim; bh=060ZyV0QVOcWziyMIkGGNytLySNYsajGtwFppbSHTto=; b=Y+rIPy0vKiu4 CJjxHh/Tq0FboyJyD7mhDGMNMZYOYJtT+Z0ItIcTnC0sVxJCnXQMBtBGXM22svDi cQe6dZcIdtPKJj61O5jC0u60IMsoK8mw42VdL+v8ALw63y4sWcHK2uFoA9GULRwq 2sev2iBAggmJUOSAhTlluINFwpuPanVwEHaAA36kXy6tVTbZ2Lbf5kLu1L85Y0hB hHCkOq3NfsSdS/fKTU+tBIvqCrZkUWGOXRDlEpHF9bZr8zKnkSNwqRNg8IzaDTKI 6vFduXUjcxRpBrE0wroaTuFINSnlyCvHJVaqoCJhLOEmtvIYXOijmBUeZ0uM8xrn /nVfipMT6w== Received: by localhost (OpenSMTPD) with ESMTPSA id 237476e3 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Fri, 2 Aug 2024 10:19:11 +0000 (UTC) From: =?utf-8?Q?Marek_Pa=C5=9Bnikowski?= To: guix-devel@gnu.org Subject: Re: Indication of build failure from substitute servers? In-Reply-To: (Jonathan Frederickson's message of "Mon, 22 Jul 2024 22:38:35 -0400") References: Date: Fri, 02 Aug 2024 12:19:10 +0200 Message-ID: <87o76bgrc1.fsf@marekpasnikowski.pl> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 81.190.248.246 (failed) Received-SPF: pass client-ip=81.190.248.246; envelope-from=marek@marekpasnikowski.pl; helo=marekpasnikowski.pl X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 05 Aug 2024 13:35:17 -0400 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Queue-Id: CD2551F78D X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -7.81 X-Spam-Score: -7.81 X-TUID: yT0bUGhR2FGT "Jonathan Frederickson" writes: > I frequently end up with Guix attempting to build packages on my > lower-powered machines when there are no substitutes > available. However, a common reason that substitutes aren't available > for a package is that the package failed to build in CI! And I usually > discover this when the package fails to build locally, usually for the > same reason, and usually after a relatively long build process. I am also annoyed by this artifact of nix-based systems. Some systems are physically incapable of building their binaries; for example, kernel of a microcomputer =E2=80=94 absolutely necessary, yet the device does not = have enough memory. This is why I believe that a clean solution is to guarantee proper substitute availability for systems that require it. > Would it make sense to have some mechanism for substitute servers to > be able to provide a sort of "non-existence proof" for a given > package? Something that the CI system could publish to indicate that > its build attempt for that package failed, and that clients could use > to optionally abort without attempting a local build? I have carried the following idea for a long time with the intent of actually implementing it before sharing it ("if you want something done, do it yourself" mentality). But seeing other's frustration with this problem I could at least share it. Here it is: The proof of availability is in workflow itself. The project committers NEVER commit anything to the master branch. Only the CI system does. Instead, the committers push to a "pre-main" branch, and the CI system picks the commits up one by one and attempts to build them as usual. IMPORTANT POINT: *if* the commit builds correctly, it gets pushed by CI to master branch, and the substitute is already available. *If* the commit does not build, it gets rejected, and it never goes to master. I currently do not know enough about Git to confidently propose a solution to the problem of how to handle the reordering of the queued work on a build failure, but I have a feeling it is not that hard to solve. One could argue that this process delays availability of software updates, but I believe this is the correct price to pay. The CI latency would still be neglible when compared to the latency of developers who perform the real work of software maintenance. There is also the issue of software bugs which cause problems at runtime. However, this is an independent problems, which should be managed by other QA processes. The art of good engineering is to find the simplest mechanisms possible that achieve the tasks well. And this requires to break down problems into atomic parts.