From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id +B1LITaiP2XcdgEAG6o9tA:P1 (envelope-from ) for ; Mon, 30 Oct 2023 13:31:50 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id +B1LITaiP2XcdgEAG6o9tA (envelope-from ) for ; Mon, 30 Oct 2023 13:31:50 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2D16F27DC6 for ; Mon, 30 Oct 2023 13:31:50 +0100 (CET) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=deeplinks-com.20230601.gappssmtp.com header.s=20230601 header.b=stHBXofn; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1698669110; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=gjT57gpXGbIfTlmFPgnDq4sD35puDUwQOKMe594bwLg=; b=ptqc+UpslW28tos2WCbJU1Ag9zJbrLyMqyvwtaXRzfNW+gq4t1f8lMrb+uVlU2aLNt36Se tP8E0NfefQgDrN1w96sVPOZ3Pq/ZLAEWYxlvy7kUF4anzodoVlhKWB0Oh3o2/Dmyuj6Ahi MvfhKvb2QQ9zHnL/cxdmBHXWgugHAyUebq6ZIAOpdINzhfGA7qJP3J123m47GV81w7mAf8 DzH7SfffuR7USutWNDEvgNd/5hTkrbRWCyIUOMYcDSnkRZ3xjuIqeY7SVKeuwza4b9PsbL ClmCHJYO5e9LTuHBlKLKOtq5lW+hFFWblmdbgFWzfo73A2i4Xvy4KpWJAuXwFw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=deeplinks-com.20230601.gappssmtp.com header.s=20230601 header.b=stHBXofn; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1698669110; a=rsa-sha256; cv=none; b=CWso2YjgaitTLKeA5MdHlMNr9DMSD21ZZiENCekGsoI0CC6OMdw6IJpYC6kRaml7T2gIqZ oAmqvwOlhwQMBcMUqp880X/pBfYX2saKrnnaoagKYaNOmeuvV3aLX+wkE7O8OM3Z40oZG0 9kpy1J40DBVzI0dY/xWPOMMdTc9f6m6S6V4E30VLUfgYSLr2i4pRFw2IA/VZ0Rq+LqfqlV JmmZFrK6X7qZz0MJc1MJvGMpfCDesu9+2WO+MiNt0WthmE/kOil3AF+vTMdbgxZRiF94BQ iNuKuEosrZd1sqLh6m2azPLefD84oIjbU7Q/kHAZUEKsAojEiqEFwJsn37LBLg== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxRQa-00022I-5l; Mon, 30 Oct 2023 08:31:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxRQX-000222-Tx for guix-patches@gnu.org; Mon, 30 Oct 2023 08:31:30 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qxRQX-00005a-K8 for guix-patches@gnu.org; Mon, 30 Oct 2023 08:31:29 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qxRR4-0000Mm-DT for guix-patches@gnu.org; Mon, 30 Oct 2023 08:32:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#66801] [PATCH va3e5ae0f..37252e07 01/32] rebar-build-system and packages. Resent-From: Pierre-Henry =?UTF-8?Q?Fr=C3=B6hring?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 30 Oct 2023 12:32:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66801 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Liliana Marie Prikler Cc: 66801@debbugs.gnu.org Received: via spool by 66801-submit@debbugs.gnu.org id=B66801.16986690771352 (code B ref 66801); Mon, 30 Oct 2023 12:32:02 +0000 Received: (at 66801) by debbugs.gnu.org; 30 Oct 2023 12:31:17 +0000 Received: from localhost ([127.0.0.1]:44465 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qxRQK-0000Lk-Ar for submit@debbugs.gnu.org; Mon, 30 Oct 2023 08:31:17 -0400 Received: from mail-yb1-xb32.google.com ([2607:f8b0:4864:20::b32]:59782) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qxRQH-0000LX-To for 66801@debbugs.gnu.org; Mon, 30 Oct 2023 08:31:14 -0400 Received: by mail-yb1-xb32.google.com with SMTP id 3f1490d57ef6-da0344eb3fdso3726036276.3 for <66801@debbugs.gnu.org>; Mon, 30 Oct 2023 05:30:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=deeplinks-com.20230601.gappssmtp.com; s=20230601; t=1698669035; x=1699273835; darn=debbugs.gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=gjT57gpXGbIfTlmFPgnDq4sD35puDUwQOKMe594bwLg=; b=stHBXofnnuXMI7sNGtbNBd8r3nYWsbxM+p/a9thc6+1ZGYiz0FP7zqrXpLJrMa1j43 vi18+kswymcOgEFCeR6BezJjb3KnPBhc1NSvyRBVKtFgO/eC2zoMDz0xnm09wZB1WLgC KMrXydxcC6t+FgnuY4P/j57ZWINEEi4vnVj+QZ0fqKJH4JiWIMWMepVpMie0CTcK/FPh sCGeUHjWiRXLvyxsi7qvHS+wKnlahucTK6oaIDDrjdvTN7CTjemMZrgW2ECnqIbG47RP UJUUcXmVDeSHxuvMN/yAm5rrYI2l9fsHmv3VTPAJjtBEsje+l8OiGtVXZYNM4Tkkid/b JxYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698669035; x=1699273835; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gjT57gpXGbIfTlmFPgnDq4sD35puDUwQOKMe594bwLg=; b=CWIURSB9Hh/UAhA4gO7vxMvOhhZQlEz3Yz5mlFHtgYyom0rt8Nihai0HCP7Tuxahgd fYQbFhBhbiG9QcDKgbLTxNOtiYrqpwwa4JmDXgE2EzLaGqPmSfVSdVGth+sZUBOne5Wx oX5zaqHp7tmYzdyBwom9xhuk+EWGU8vZkxQKKiRoRI1lU5NmfjoYTwwruiBCtG7Axb5L UEOvnHkjSFto6/nMRYz8wnn79WhQPBhHE/osUBBD3xBJ3cU9PsufYOKwxep8qqq3U4Mz 9SpqtApad2mDg4K1p7DOYvfGk9Bk7VP7EpSFLoYHoiiMuw1qwjJ3KA2u3CGSP0u6f8QJ o6mA== X-Gm-Message-State: AOJu0Yws4jjO2s/c2A/0OLnAi+vupr6aXKbjQifbZm+mbUJ0N3eFop/R Xf8IAgpCa4ZL2WJuZho/fpKhXaFpIPLrNLAzlNLwLbFt0+d+cyeZJ5M= X-Google-Smtp-Source: AGHT+IE1KIw6za6mpFBP/ttEYlcr7YZvpRCG0t46sQjqqD8MEmop7Tr/PcfLSAsXIv11bzsRsCRi2/TOPMPkMXKhm+A= X-Received: by 2002:a25:abb0:0:b0:da0:e250:15ad with SMTP id v45-20020a25abb0000000b00da0e25015admr9269103ybi.30.1698669034775; Mon, 30 Oct 2023 05:30:34 -0700 (PDT) MIME-Version: 1.0 References: <7ceab0dcbe069f664377786b5bb531e2196fffc1.camel@gmail.com> In-Reply-To: From: Pierre-Henry =?UTF-8?Q?Fr=C3=B6hring?= Date: Mon, 30 Oct 2023 13:30:23 +0100 Message-ID: Content-Type: multipart/alternative; boundary="00000000000068b84b0608ee3490" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -2.10 X-Spam-Score: -2.10 X-Migadu-Queue-Id: 2D16F27DC6 X-Migadu-Scanner: mx10.migadu.com X-TUID: p7Ild0J53pKX --00000000000068b84b0608ee3490 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I've collected your comments in an org-file so that it's easier (at least for me, but I hope it's the case for you too) to follow multiple discussions at the same time. I've only included the ones that are still open to discussion. The structure should be self-evident. The most important comment may be Comment 9. Cheers. * TODO Comment 4 ** LMP This reeks of the hack that we need for cargo-build-system, except with a worse variable name. I strongly suggest looking into ways we can do withou= t it. ** PHF This idea came from a discussion with jpoiret. See: https://logs.guix.gnu.org/guix/2023-10-24.log#180111. It seems that the ide= a you suggest is to use =3Dsearch-path-as-list=3D as hinted below (=3DComment= 9=3D). Is this correct? * TODO Comment 6 ** LMP Uhm, how are you improving the status quo here? ** PHF Comment updated with: #+begin_example ;; If these directories exist, then no error occurs. So, we make sure ;; they exist. #+end_example Is this OK? I don't see how to prevent rebar3 to do that. It's apparently an opened issue: https://github.com/erlang/rebar3/issues/1173 * TODO Comment 9 ** LMP You might want to look into possible PATH variables or put these sources into a special folder so that you can use search-path-as-list. ** PHF Perhaps an idea: 1) If we require all Erlang packages to have an output =E2=80=9Csrc=E2=80= =9D something like: /gnu/store/=E2=80=A6elixir-pkg-1.2.3/src/elixir/src/=E2=80=A6, 2) then (search-path-as-list '("src/elixir/src") '("/gnu/store=E2=80=A6elixir-pkg-1.2.3" =E2=80=A6)) would return '("/gnu/store=E2=80=A6elixir-pkg-1.2.3/src/elixir/src" =E2=80=A6) =E2=89=A1= lst-src. 3) Given lst-src, it would be enough to install each source under _checkouts, i.e., _checkouts/lib-name/src. It is probably feasible to retrieve lib-name from somewhere. What do you think? ** LMP Rather than require, you can add a phase to ensure that this is the case. I'm not sure whether we should make that an extra output, however; there might be many packages for which we don't need those sources and where we d= o need them, we could potentially add them as native-inputs. ** PHF Agreed. ** LMP Another alternative would be to keep the sources in =3Dlib/erlang/lib/lib-name/src=3D so that it gets symlinked by the phase we have. Though at that point we can surely go with a less surprising install directory. ** PHF Here is the approach taken so far: 1) The objective is to ensure that =3Dmix compile=3D does not attempt to us= e the network and, as a result, fails to compile. 2) For the above to be true, it is necessary to have =E2=80=94 at build tim= e =E2=80=94 the sources of all Erlang input packages and install them as checkouts. 3) Thus, the question becomes: How to access all the Erlang package sources at build time? 4) One idea was to have the client side of the build code send all the sources to the server through the poorly named argument =3D#:sources-erlang=3D. = This led to the current code. It relies on the fact that for a given Erlang package, it is possible to access its source in the store, for example, =3D/gnu/store/=E2=80=A6erlang-kv/erlang-kv.tar=3D. 5) Instead, you propose modifying the installation process so that sources are installed along with the built libraries. The source might be collected with =3Dsearch-path-as-list=3D. The downside seems to be that the source code is stored twice: first in the archive, then in the package. However, this could lead to a much cleaner method of passing the sources to the build-side code, that is, the source code would not be passed throug= h =3Darguments=3D. I'm sending a patch based on this latter idea. Are you OK with that? * TODO Comment 10 ** LMP Also, IIUC, erlang-depends already does something rather similar. Is there any reason it's broken for you? ** PHF Without that (i.e., _checkouts/lib-name/src), rebar3 will not compile things. Is this a satisfying explanation? ** LMP I mean, ideally we would store these things in some other directory, given that _checkouts/lib-name already contains the precompiled stuff after erlang-depends. However, if the tooling insists on storing both in the same place, you could simply amend the existing erlang-depends phase to: 1. Copy instead of symlinking 2. Also copy the sources ** PHF If I understand correctly, this should be addressed with the patch from =3DComment 9=3D section above. --00000000000068b84b0608ee3490 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I've collected your comments in an org-file so that it= 's easier (at least for me, but I hope it's the case for you too)= =C2=A0
to follow multiple discussions at the same time. I've only i= ncluded the ones that are still open to discussion.=C2=A0
The str= ucture should be self-evident. The most important comment may be Comment 9.=
Cheers.

* TODO Comment 4
** LMP
T= his reeks of the hack that we need for cargo-build-system, except with aworse variable name.=C2=A0 I strongly suggest looking into ways we can do = without
it.

** PHF
This idea came from a discussion with jpoir= et. See:
https://logs.guix.gnu.org/guix/2023-10-24.log#180111. It seems that t= he idea
you suggest is to use =3Dsearch-path-as-list=3D as hinted below = (=3DComment 9=3D). Is this
correct?

* TODO Comment 6
** LMPUhm, how are you improving the status quo here?

** PHF
Comment u= pdated with:
#+begin_example
;; =C2=A0 If these directories exist, th= en no error occurs. So, we make sure
;; =C2=A0 they exist.
#+end_exam= ple
Is this OK?

I don't see how to prevent rebar3 to do that.= It's apparently an opened issue:
https://github.com/erlang/rebar3/issues/1173
=
* TODO Comment 9
** LMP
You might want to look into possible PATH= variables or put these sources into
a special folder so that you can us= e search-path-as-list.

** PHF
Perhaps an idea:
=C2=A0 1) If we= require all Erlang packages to have an output =E2=80=9Csrc=E2=80=9D someth= ing like: /gnu/store/=E2=80=A6elixir-pkg-1.2.3/src/elixir/src/=E2=80=A6,=C2=A0 2) then (search-path-as-list '("src/elixir/src") '= ;("/gnu/store=E2=80=A6elixir-pkg-1.2.3" =E2=80=A6)) would return = '("/gnu/store=E2=80=A6elixir-pkg-1.2.3/src/elixir/src" =E2=80= =A6) =E2=89=A1 lst-src.
=C2=A0 3) Given lst-src, it would be enough to i= nstall each source under _checkouts,
=C2=A0 =C2=A0 =C2=A0i.e., _checkout= s/lib-name/src. It is probably feasible to retrieve lib-name
=C2=A0 =C2= =A0 =C2=A0from somewhere.
What do you think?

** LMP
Rather tha= n require, you can add a phase to ensure that this is the case.
I'm = not sure whether we should make that an extra output, however; there
mig= ht be many packages for which we don't need those sources and where we = do
need them, we could potentially add them as native-inputs.

** = PHF
Agreed.

** LMP
Another alternative would be to keep the so= urces in
=3Dlib/erlang/lib/lib-name/src=3D so that it gets symlinked by = the phase we have.
Though at that point we can surely go with a less sur= prising install
directory.

** PHF
Here is the approach taken s= o far:
1) The objective is to ensure that =3Dmix compile=3D does not att= empt to use the
=C2=A0 =C2=A0network and, as a result, fails to compile.=
2) For the above to be true, it is necessary to have =E2=80=94 at build= time =E2=80=94 the
=C2=A0 =C2=A0sources of all Erlang input packages an= d install them as checkouts.
3) Thus, the question becomes: How to acces= s all the Erlang package sources at
=C2=A0 =C2=A0build time?
4) One i= dea was to have the client side of the build code send all the sources
= =C2=A0 =C2=A0to the server through the poorly named argument =3D#:sources-e= rlang=3D. This led
=C2=A0 =C2=A0to the current code. It relies on the fa= ct that for a given Erlang package,
=C2=A0 =C2=A0it is possible to acces= s its source in the store, for example,
=C2=A0 =C2=A0=3D/gnu/store/=E2= =80=A6erlang-kv/erlang-kv.tar=3D.
5) Instead, you propose modifying the = installation process so that sources are
=C2=A0 =C2=A0installed along wi= th the built libraries. The source might be collected
=C2=A0 =C2=A0with = =3Dsearch-path-as-list=3D.

=C2=A0 =C2=A0The downside seems to be tha= t the source code is stored twice: first in the
=C2=A0 =C2=A0archive, th= en in the package.

=C2=A0 =C2=A0However, this could lead to a much c= leaner method of passing the sources to
=C2=A0 =C2=A0the build-side code= , that is, the source code would not be passed through
=C2=A0 =C2=A0=3Da= rguments=3D.

I'm sending a patch based on this latter idea. Are = you OK with that?

* TODO Comment 10
** LMP
Also, IIUC, erlang-= depends already does something rather similar.=C2=A0 Is there
any reason= it's broken for you?

** PHF
Without that (i.e., _checkouts/l= ib-name/src), rebar3 will not compile
things. Is this a satisfying expla= nation?

** LMP
I mean, ideally we would store these things in som= e other directory,
given that _checkouts/lib-name already contains the p= recompiled stuff
after erlang-depends.=C2=A0 However, if the tooling ins= ists on storing both
in the same place, you could simply amend the exist= ing erlang-depends
phase to:
1. Copy instead of symlinking
2. Also= copy the sources

** PHF
If I understand correctly, this should b= e addressed with the patch from
=3DComment 9=3D section above.
=
--00000000000068b84b0608ee3490--