From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 8K4pD5CM/mFcSQEAgWs5BA (envelope-from ) for ; Sat, 05 Feb 2022 15:41:20 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id GAz3B5CM/mHRbgAAG6o9tA (envelope-from ) for ; Sat, 05 Feb 2022 15:41:20 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C92D935836 for ; Sat, 5 Feb 2022 15:41:19 +0100 (CET) Received: from localhost ([::1]:54370 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nGMFb-0001iV-38 for larch@yhetil.org; Sat, 05 Feb 2022 09:41:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:53264) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nGMF4-0001iM-HK for guix-devel@gnu.org; Sat, 05 Feb 2022 09:40:47 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:36580) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nGMEq-0007ON-Sg for guix-devel@gnu.org; Sat, 05 Feb 2022 09:40:35 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id B2F76140; Sat, 5 Feb 2022 15:40:23 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FYVa3fZIp1lN; Sat, 5 Feb 2022 15:40:22 +0100 (CET) Received: from ribbon (91-160-117-201.subs.proxad.net [91.160.117.201]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 9F81C4A; Sat, 5 Feb 2022 15:40:22 +0100 (CET) From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zimoun Subject: Re: failure when rebuilding the past: long term? References: <87zgn89agt.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 17 =?utf-8?Q?Pluvi=C3=B4se?= an 230 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 05 Feb 2022 15:40:22 +0100 In-Reply-To: <87zgn89agt.fsf@gmail.com> (zimoun's message of "Wed, 02 Feb 2022 23:20:50 +0100") Message-ID: <87czk1gyw9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / X-Rspamd-Server: hera X-Rspamd-Queue-Id: B2F76140 X-Spamd-Result: default: False [0.90 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_ENVRCPT(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] Received-SPF: softfail client-ip=185.233.100.1; envelope-from=ludo@gnu.org; helo=hera.aquilenet.fr X-Spam_score_int: -11 X-Spam_score: -1.2 X-Spam_bar: - X-Spam_report: (-1.2 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.665, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1644072079; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=jFnQlR3MKVZ1a1gZE72fdmDacnT69NRBM21HGx/oTno=; b=tdmo2Woc3874UbCPrXckWM2xdJmBvD84CXODmzYFy32wWhASEcAmtuN9Od84lguXhI1WvP wFJ9weluWCJfub2/W89wzi5QCYsBoIs7YC5Bv+yDded0w4IKbNdgjhSBwJ4JQjhsKiXgoq Nmu7LnIeHVD1S/fuVRHES1ZNKiUnNLzHYrBxF4koxrcvHNCT0horMLpiFue6Y/1m415NFs UgnFxgzRujq34A7rU0E1ogPruJ1QGGBaS86kvm48LzDVl2Scq3EvmvOgZAcHfSOayWJgIa +TdqVp2wjZt4lTkUlnq1A36mts6w/os1jznOsgQ57GSH6qluQjVr4BT3DhXlzw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1644072079; a=rsa-sha256; cv=none; b=lSA9NcWmnJxGEvHTwMQe3CmXf7xe1dieaGXgZoup1Gx4TgdHEZmcAHNTRqgeU9x/fP1yXw z2RCnELogcllBEWlf0NU3vVS/SRZcO7deBZg5rOhnBJXVyq8Mm9RPcd1BWACM+lxit3Fk2 b5Cz+pRrlElK0Z8rZUOUosZwhti2M6E9oRd4OVkCt4RIZM4/N8EV9LlJ+39Y/QsgE+2ovL Myx9uI86X0uwBjgVss4nYzy6W3WYm24uT7UNCO9G8rW8ush/40m1PXxi3BhMCub6KYMe/j t7QBzGJ5+ssyyDdcpmIVW1gg1vwENV85dQ1itbgt1w5i/Qff4dV9vbyt1onoDw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.13 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: C92D935836 X-Spam-Score: -3.13 X-Migadu-Scanner: scn0.migadu.com X-TUID: hXsnL8Z49+K3 Hi! As a preamble, I=E2=80=99d like to say that what we=E2=80=99re doing in ter= ms of time traveling is ambitious; it=E2=80=99s never been done before, AFAIK. And there are many pitfalls, as you write, mainly: disappearing source (we=E2=80=99re working on it!), and, a bigger concern IMO, =E2=80=9Cnon-det= erministic=E2=80=9D build processes=E2=80=94in a broad sense: that includes build processes that depend on CPU features, kernel versions, and other things not specified in derivations. I think we can address the latter, indirectly, with early cutoffs: if two different derivations yield the same output, then we can consider them equivalent and avoid rebuilding anything that depends on it, instead just grafting them. It helps in that it would allow users to build =E2=80=98--without-tests=E2=80=99 and get an equivalent result, or to= build with a hypothetical =E2=80=98--in-virtual-machine=E2=80=99 or =E2=80=98--with-curr= ent-time=3D2020-01-01=E2=80=99 option. zimoun skribis: > 3. Having all the sources is one thing, but being able to rebuild is > another. Failure of OpenBLAS [0] is one example, of some mesboot [1] > or of texlive [2] are others. It appears to me that something is > inadequate with the current workflow pushing all to master without any > automated* checks. Other said, failures as 8f9fd9b70c (value "Unbound > variable: ~S") (value (r-biobase) seems wrong by design. Well, > ac6f677249 is another recent example. Somehow, because the package > collection is becoming larger and larger (which is good!) then it is > becoming harder and harder to maintain the consistency both forward and > backward. For the last Guix revision, my rough estimate is that ~5% of > packages are broken and my guess is that this number is =E2=80=9Cindepen= dant=E2=80=9D > of the package collection size. However, I already have some > collection of unreachable commits by the time-machine and then for some > reachable commits, I do not have numbers for what is effectively > rebuildable. As discussed in this thread [3], maybe Guix is moving too > fast; or better worded, maybe the current workflow is inadequate with > some goals of long term and build all from sources. I do not know=E2=80= =A6 My > point here: do we provide a list of commits (release, others) where we > apply more care for long term? It=E2=80=99s hard for committers to push evidently broken code, such as unb= ound variables, because =E2=80=98guix lint=E2=80=99, =E2=80=98make=E2=80=99, & c= o. catch it. Still, it would be great if it never happened at all, and for that enforcing server-side checks could help. The next issue is broken builds. Automation can definitely help, and Chris Baines=E2=80=99 work on automated patch handling or anything along th= ese lines would be a step in the right direction. Non-deterministic builds are probably harder to automatically identify, yet it=E2=80=99s a bigger concern. Besides, I think we should do some sort of CI for time traveling, to make sure we can travel forward and backward for different revisions. (I would love to see research institutes invest in this work.) My 2=C2=A2, Ludo=E2=80=99.