From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id gBuuH4uqSGBjCAAA0tVLHw (envelope-from ) for ; Wed, 10 Mar 2021 11:16:27 +0000 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 4KmCG4uqSGAYOQAA1q6Kng (envelope-from ) for ; Wed, 10 Mar 2021 11:16:27 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E96EB2C7A7 for ; Wed, 10 Mar 2021 12:16:26 +0100 (CET) Received: from localhost ([::1]:35806 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJwpF-0007Sj-L0 for larch@yhetil.org; Wed, 10 Mar 2021 06:16:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:48352) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJwp1-0007SQ-23 for guix-devel@gnu.org; Wed, 10 Mar 2021 06:16:11 -0500 Received: from lepiller.eu ([89.234.186.109]:48376) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJwox-0002w6-Ks; Wed, 10 Mar 2021 06:16:10 -0500 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 2daab039; Wed, 10 Mar 2021 11:16:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date :in-reply-to:references:mime-version:content-type :content-transfer-encoding:subject:to:from:message-id; s=dkim; bh=faFQLU1L+uG0JwpMIyYenM2/u5jZWiplvkTPdv0Otrk=; b=dfhOGWsVp4+1 dufyvdKu8lnDTi+h3gdFZvo0t/FWpojvh3qhkj6CrLlTEidU/8UtmHdFWWNNWTWq Yh0uNmTOv7RPhYhcQW8NSVNelhAzKYq6MNM3MWwjIQ+/wk5q+DxCy8X8V5B1tPC/ byAW/5jPlC+XRFYUQZtZJTsuwOqedyxueVNAec+g3Z2SlTwAiZhPMds7Vp0ni9/O 58AcY/0g9zrijfGsw5XjJdgMRX9SY0OJmMFob6TpQwXJzvXgbwHgquF4vui6zSkT pidWzCgnMY+VHnJzAEirqw+flhsUTKA1V302X1no9DJxqHzfgtRwxyWLMRZgl+Xq RrKDOcrQfA== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id f752b14f (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Wed, 10 Mar 2021 11:15:59 +0000 (UTC) Date: Wed, 10 Mar 2021 06:15:42 -0500 User-Agent: K-9 Mail for Android In-Reply-To: <87eegn4658.fsf@gnu.org> References: <87im6ojbbx.fsf@gnu.org> <87eegn4658.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----N6KF31FYAH8L61CY2X562JU47HN7N3" Content-Transfer-Encoding: 7bit Subject: Re: Will 2021 be the year of build systems on gexps? To: guix-devel@gnu.org,=?ISO-8859-1?Q?Ludovic_Court=E8s?= From: Julien Lepiller Message-ID: Received-SPF: pass client-ip=89.234.186.109; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1615374987; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=be6EzUz8HCJlvoABcKGfEm7GM8hxHnEmFZywzpHSMTc=; b=NNyRvhy/Exp/EODIryYFfeO9hUoNPEIxhsAX4XV98kySNIOkdq9fewY71Zae1yzmLveFRb 2sJnI6Tqp6rxOx/0qC8Lk2MQzvWN+kEYYr2m4TUvJugyBjTQYl6F5DtTqloATeUlBl26pJ AsNV+UqWkVyja8eW1GGSQ9+TjVy2j2tHgkS3PJI4+Jq8pPyCmMdH0hTW/JnbLVdiSy5l8u MathIGGWkJ0aUDMJyIj+6q2rcmb28MHm5iwZcf6WxtdD05ez/l2Ir6nv790FkoOmn6DWO7 rQuEdNW6Hx7zFI8c86CcTdAM+hpKBpfMBQwmduZvqgbpXHrKRdwBaIFJxhNifA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1615374987; a=rsa-sha256; cv=none; b=bfsh/vGNRAkUjOB6W/369p3/mq5bC5M/lNuHAk0Wu8nhqLJdLwWnsIqWDkarMGEZjkgv2y J0aUK4d8W6P5ejLoFMTAv3N1EL84zykYUciy2bucIeL+yG05yCIjywhJS8RijLPWb0wtYT u6lCRf2+FSV3dcEAHoYQYjMfV/YU9uP7JrznOYzxxN/qVaIi6cxA7uHrbmjSF2kTqZDhRR 9mcvxkJS43/ANGLX/KZSCGIEWdjK8YcKlZp6Q/I8q7sF+7bLy49nAZcxiYnpdqnpkaJ8aF ujS2K4vykU5HQw9A1Uolyqx2vHb2YUlxQZwd2eVidnvVSh816DmLXpxbpWK4Ig== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=lepiller.eu header.s=dkim header.b=dfhOGWsV; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -2.09 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=lepiller.eu header.s=dkim header.b=dfhOGWsV; dmarc=pass (policy=none) header.from=lepiller.eu; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: E96EB2C7A7 X-Spam-Score: -2.09 X-Migadu-Scanner: scn0.migadu.com X-TUID: KABVcaaF/y7A ------N6KF31FYAH8L61CY2X562JU47HN7N3 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable If we aim at merging c-u before next release, we should probably wait next = cycle, as this introduces quite a lot of changes, and packages might break = in subtle ways=2E 10% increase for computing derivations is not great :/ it already takes a = long time to do that on my arm system ^^"=2E I wonder how we could improve = that? Le 10 mars 2021 06:09:23 GMT-05:00, "Ludovic Court=C3=A8s" a =C3=A9crit : >Hello! > >Ludovic Court=C3=A8s skribis: > >> Over the last few days I=E2=80=99ve been head-down working on >> =E2=80=98wip-build-systems-gexp=E2=80=99, the mythical branch that brin= gs gexps to >build >> systems and packages, so we can say goodbye to >> =E2=80=98build-expression->derivation=E2=80=99=2E And=E2=80=A6 it=E2= =80=99s quite a ride! > >The current tip of =E2=80=98wip-build-systems-gexp=E2=80=99 Just Works; i= t=E2=80=99s being >built, >it can build =E2=80=98guix=E2=80=99 and cross-build things like =E2=80=98= sed=E2=80=99: > >https://data=2Eguix-patches=2Ecbaines=2Enet/repository/2/branch/wip-build= -systems-gexp > > https://ci=2Eguix=2Egnu=2Eorg/jobset/wip-build-systems-gexp (though Cui= rass > currently has unrelated problems) > >In terms of performance, there=E2=80=99s still a ~10% slowdown when compu= ting >derivations compared to the =E2=80=98core-updates=E2=80=99 revision the b= ranch is based >on=2E > >Here=E2=80=99s what I=E2=80=99d like to do in the coming days, if that do= esn=E2=80=99t >interfere >with what others have in mind for the upcoming release: > > =E2=80=A2 Monitor build failures due to typos/thinkos made while adjust= ing > build systems; > > =E2=80=A2 Merge on =E2=80=98core-updates=E2=80=99=2E > >Then there are optimizations to work on, but that can take a bit >longer=2E >In particular, in =E2=80=98gexp->derivation=E2=80=99, allow file-like obj= ects to be >specified as environment variable values=2E In turn, use that so that, >say, =E2=80=98gnu-build-system=E2=80=99 has a single builder for all its = packages and >just calls =E2=80=98getenv=E2=80=99 to get the value of its various param= eters, similar >to what (guix git-download) does=2E > >That said, if people think it=E2=80=99s too late in the cycle, we could j= ust as >well keep it for the next cycle=2E > >Thoughts? > >Thanks, >Ludo=E2=80=99=2E ------N6KF31FYAH8L61CY2X562JU47HN7N3 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable If we aim at merging c-u before next release, we s= hould probably wait next cycle, as this introduces quite a lot of changes, = and packages might break in subtle ways=2E

10% increase for computin= g derivations is not great :/ it already takes a long time to do that on my= arm system ^^"=2E I wonder how we could improve that?

Le 10 mars 2021 06:09:23 GMT-05:00, "Ludovic Court=C3=A8s" &l= t;ludo@gnu=2Eorg> a =C3=A9crit :
Hello!

Ludovic Court=C3=A8s <ludo@gnu=2Eor=
g> skribis:

Over=
 the last few days I=E2=80=99ve been head-down working on
=E2=80=98wip-b=
uild-systems-gexp=E2=80=99, the mythical branch that brings gexps to build<=
br>systems and packages, so we can say goodbye to
=E2=80=98build-express=
ion->derivation=E2=80=99=2E  And=E2=80=A6 it=E2=80=99s quite a ride!
=


The current tip of =E2=80=98wip-build-systems-gexp=E2=80=
=99 Just Works; it=E2=80=99s being built,
it can build =E2=80=98guix=E2=
=80=99 and cross-build things like =E2=80=98sed=E2=80=99:

  https://data=2Eguix-patches=2Ecbaines=2Enet/repository/2/b=
ranch/wip-build-systems-gexp

  https://ci=2Eguix=2Egnu=2Eorg/jobset=
/wip-build-systems-gexp (though Cuirass
  currently has unrelated pr=
oblems)

In terms of performance, there=E2=80=99s still a ~10% slowdo=
wn when computing
derivations compared to the =E2=80=98core-updates=E2=
=80=99 revision the branch is based
on=2E

Here=E2=80=99s what I=
=E2=80=99d like to do in the coming days, if that doesn=E2=80=99t interfere=

with what others have in mind for the upcoming release:

  =E2=80=
=A2 Monitor build failures due to typos/thinkos made while adjusting
   =
 build systems;

  =E2=80=A2 Merge on =E2=80=98core-updates=E2=80=99=
=2E

Then there are optimizations to work on, but that can take a bit=
 longer=2E
In particular, in =E2=80=98gexp->derivation=E2=80=99, allo=
w file-like objects to be
specified as environment variable values=2E  I=
n turn, use that so that,
say, =E2=80=98gnu-build-system=E2=80=99 has a =
single builder for all its packages and
just calls =E2=80=98getenv=E2=80=
=99 to get the value of its various parameters, similar
to what (guix gi=
t-download) does=2E

That said, if people think it=E2=80=99s too late=
 in the cycle, we could just as
well keep it for the next cycle=2E
Thoughts?

Thanks,
Ludo=E2=80=99=2E

------N6KF31FYAH8L61CY2X562JU47HN7N3--