From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id WJtzDDEYV2URNgAAG6o9tA:P1 (envelope-from ) for ; Fri, 17 Nov 2023 08:37:21 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id WJtzDDEYV2URNgAAG6o9tA (envelope-from ) for ; Fri, 17 Nov 2023 08:37:21 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A9020581BC for ; Fri, 17 Nov 2023 08:37:20 +0100 (CET) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=deeplinks-com.20230601.gappssmtp.com header.s=20230601 header.b=t91aLzHi; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1700206641; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=uvpQq7eJrJgNpMlv5/+XtlTwIPy+bN4liSzhhJZrVpI=; b=ORGRPc3dKgtMjxeA3TacRgBGQn6e6bpLt2Mby6C0/q2u3fuH1+gb9W5IRAjqzXJ3Ehn2MK oH8KbzCm4/uZdrE8hfDCZJHLoaGeNLcPWmRfFOJh7NXK7DHqcnV1XznFhVtKFP9VsDK7uB qZZIgZKKRyuE27b+Bb8EU0iIam8Muxuv6fOymB7m0Ffkp8Q+ZTrFOjNDkAQ3510/hKYs/z jJDmpdc8gW+hrxVM1kyqpzENG6ggPmLKxzYoB5lLPzTLmxx7esNey33uElbzOmVwoWvOOx pxS96vuTbMEdW3CuYZ8Cgg1xYed3vLMF8avIH4DInehiNFOFkLYjMthTDW/yFQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=deeplinks-com.20230601.gappssmtp.com header.s=20230601 header.b=t91aLzHi; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1700206641; a=rsa-sha256; cv=none; b=iya5X//pNt9cMPH2+FCf0M9u83tVd3SwQCa0t2bAxtbsBzTvOZgCUn/UFV2ikNj/3G02yc A4kD3bSpPDwS7dStEXECdAPuzNo/qqgIAXl0z0Hje7HcHAmCLsiyZSN6NSgiISlhdxXk6J hs2WKUz9mT9fEco15SHv5LfjGKd40Lka+5uJi7m6v1/g/z5ahthWzUp2dYWR88zhcees1b WztejdWcztRV2ohh1J/vYm/dvlJGs+ujDJojUWn8RQrI+KRTG2Ogg8WPX7etSIkOAi/5g8 zzBtjYxaw8HdvFKBfFusz93ffWu8cQfLktfp265OD1DxpvJewCcCOeN8vwVkNQ== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r3tPW-0001XM-Pd; Fri, 17 Nov 2023 02:37:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r3tPU-0001QC-AQ for guix-patches@gnu.org; Fri, 17 Nov 2023 02:37:04 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r3tPU-00044m-1i for guix-patches@gnu.org; Fri, 17 Nov 2023 02:37:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1r3tPS-00021a-C5 for guix-patches@gnu.org; Fri, 17 Nov 2023 02:37:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#66801] [PATCH v3 01/14] build-system: Add mix-build-system. Resent-From: Pierre-Henry =?UTF-8?Q?Fr=C3=B6hring?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 17 Nov 2023 07:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66801 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Liliana Marie Prikler Cc: 66801@debbugs.gnu.org Received: via spool by 66801-submit@debbugs.gnu.org id=B66801.17002066077746 (code B ref 66801); Fri, 17 Nov 2023 07:37:02 +0000 Received: (at 66801) by debbugs.gnu.org; 17 Nov 2023 07:36:47 +0000 Received: from localhost ([127.0.0.1]:45021 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r3tPC-00020r-By for submit@debbugs.gnu.org; Fri, 17 Nov 2023 02:36:46 -0500 Received: from mail-yb1-xb35.google.com ([2607:f8b0:4864:20::b35]:61625) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r3tP6-00020U-4J for 66801@debbugs.gnu.org; Fri, 17 Nov 2023 02:36:44 -0500 Received: by mail-yb1-xb35.google.com with SMTP id 3f1490d57ef6-d9fe0a598d8so1620418276.2 for <66801@debbugs.gnu.org>; Thu, 16 Nov 2023 23:36:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=deeplinks-com.20230601.gappssmtp.com; s=20230601; t=1700206594; x=1700811394; darn=debbugs.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=uvpQq7eJrJgNpMlv5/+XtlTwIPy+bN4liSzhhJZrVpI=; b=t91aLzHib0AxOkeMPhISIJ9GKcnbo72HNHPT+oWXtFcl/GT0opey283Kv+LY0Xjygk kkRbAmyLdUp3w9hQrJARyWkgdidWyiLKrGAbBR8aaw5BcghTmhy6M63i+5aT2fMvjUr1 GXSiEYM2vvZdPFGW9mM4JPfxTd9OvXl+wh2huQX83BJZIHOEfgzL1K+qzgJH7rPndyGw Pc7HUuG5JFndYXeAm77oU7tmQyqusgRm6cdjNb34am8+ziqKiULjB9tOia0rLUul1X4j E2d+nklw3rTM7zBFYce0KmKa9IRn+M2ouVil0LxElREku7f0CfniaG4NxFGp1OxPfIMg nO5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700206594; x=1700811394; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uvpQq7eJrJgNpMlv5/+XtlTwIPy+bN4liSzhhJZrVpI=; b=Od6RpT77w5oZ8DFAc41A97yJuwh8Ps+j+EiF30yZkYXIwyk+wbP+m+mOJB7Xdf+2kF O0+2MKpnpIgblPMPsjnS5USzGX6bDGCp22N32VYnfrcSwktabTX213QcJxaj6E7w4cv3 eVuMzVGmLy1mql4Xr+O+wytjOJxiKxsy6AkNgSodMcMws1IE05VTSWjb3snutR8Og1de er2pEIYrXElCC3vPSXvskB4x9OMzi7LRHGWN3EedR36gMO7hWdeKpYxN2Mj1abnmnkDB R0kxt9GJHR2jSLTPRiTStEYvEwDNRA7/OmWt9YxV11NCGB+yg89x7K9I28GCUfnY2asN ix1A== X-Gm-Message-State: AOJu0YzC3EitMZ0DNos1Jzc+nqiTdf6GQhf7yIikPgp5jYfJy5oZbXmb 9Fv5VqBUQyKAXetp9D7Y6WMQCQO6WmUDzwmzRAcO/w== X-Google-Smtp-Source: AGHT+IHCl8wIeK2v4IL4ao7uQD8KTyFYmSnJfsWcUFKPTgK959t4oRdZnU0x8tn11j86q1KxOXF/CL/szp8CwP0expk= X-Received: by 2002:a25:6983:0:b0:d9a:ea20:7eb6 with SMTP id e125-20020a256983000000b00d9aea207eb6mr19672933ybc.38.1700206594212; Thu, 16 Nov 2023 23:36:34 -0800 (PST) MIME-Version: 1.0 References: <67c324d191a9698aed8d9887260cb0ef2bc031df.1700088189.git.phfrohring@deeplinks.com> <66cf8aa37e2c9696af3a8895059b9293669f485c.camel@gmail.com> <6aca9ebb2099f6d8ee3bbe39199bc42c364c40f0.camel@gmail.com> In-Reply-To: <6aca9ebb2099f6d8ee3bbe39199bc42c364c40f0.camel@gmail.com> From: Pierre-Henry =?UTF-8?Q?Fr=C3=B6hring?= Date: Fri, 17 Nov 2023 08:36:22 +0100 Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -2.09 X-Spam-Score: -2.09 X-Migadu-Queue-Id: A9020581BC X-Migadu-Scanner: mx10.migadu.com X-TUID: XhrJJ469Q8HY * WAITING Comment ** lilyp > +(define* (install-hex #:key name inputs outputs #:allow-other-keys) > + "Install Hex." > + (let ((hex-archive-path (string-append (getenv "MIX_ARCHIVES") > "/hex"))) > + (mkdir-p hex-archive-path) > + (symlink (car (list-directories (elixir-libdir (assoc-ref inputs > "elixir") > + (assoc-ref inputs > "elixir-hex")))) > + (string-append hex-archive-path "/hex")))) Why do we need this? It looks like we'll be pasting the same (native?) input all over the store, which imho would be bad. ** phf Without ~Hex~, you get this error: #+begin_example starting phase `build' Could not find Hex, which is needed to build dependency :ex_doc Shall I install Hex? (if running non-interactively, use "mix local.hex --force") [Yn] #+end_example This message is called from ~handle_rebar_not_found~ in ~lib/mix/lib/mix/tasks/deps.compile.ex~ ; which is called from ~rebar_cmd~ because a ~manager~ (~Hex~) could not be found. Hex must be present =3D> install-hex must be executed. I thought that this should not be a problem since Hex is needed at build ti= me, and just symlinked. Maybe should it be copied instead? ** lilyp Is hex not an (implicit) native-input in your build system? Anything that keeps it from functioning is a packaging bug imho. ** phf If ~mix~ finds ~Hex~ under ~$MIX_ARCHIVES/hex/hex~, then ~mix compile~ does not emit the message above. I'm not sure how could this be changed. I've tried to se= t ~MIX_PATH~ to ~/gnu/store/=E2=80=A6-elixir-hex-2.0.5/lib/elixir/1.14~ witho= ut success. So, this is the reason why ~install-hex~ phase installs Hex like it does. ** lilyp Look into mix and how it invokes hex. There's hardcoding potential, I'm su= re of it. ** phf :PROPERTIES: :ID: d9ffbbfa-3cd3-4a44-9acb-04a0627b90d7 :END: - Observation: ~mix compile~ =E2=87=92 "Could not find Hex, which is needed= to build dependency :ex_doc". - We have the source - guix build --source - We found from where this message is emitted. - lib/mix/lib/mix/hex.ex - We now why the message was emitted. - Code.ensure_loaded?(Hex) failed. - https://hexdocs.pm/elixir/1.14.0/Code.html#ensure_loaded/1 Well, adding the store path of Hex to ~ERL_LIBS~ solves the problem. The ~install-hex~ phase can be deleted in favor of manipulating ~ERL_LIBS~ as suggested with propagated inputs [[id:d7cd6e3d-9802-499f-a157-7698aca942d4][below]]. * WAITING Comment :PROPERTIES: :ID: d7cd6e3d-9802-499f-a157-7698aca942d4 :END: ** lilyp > + (define (install-inputs mix-env) > + (for-each (cut install-input mix-env <>) > + (append inputs native-inputs))) Installing native inputs: probably a bad idea (inhibits cross- compilation). ** phf A slight variant that does not link things when it should not: #+begin_src scheme (define (install-inputs mix-env) (for-each (cut install-input mix-env <>) (cond ((string=3D? mix-env "prod") inputs) ((member mix-env '("shared" "test")) (append inputs native-inputs)) (else (error (format #f "Unexpected Mix env: ~a~%" mix-env)= ))))) #+end_src I did not consider cross-compilation yet. The following might be wrong be h= ere we go. As far as I understand, Erlang applications are largely platform independent. Once the code is compiled to BEAM bytecode, it can run on any platform that has the Erlang VM installed. If an Erlang library uses native extensions, then cross-compilation might be required. For a build to succee= d in a given environment (one of "prod", "test", "shared"), the BEAM files of all dependencies should be present on the build machine. So, all dependenci= es must be installed ** lilyp Not an expert on elixir, but that sounds borked. ** phf Yes. Did not have time to look into it as of now. ** lilyp You might get around this with propagated-inputs maybe? That being said, native-inputs shouldn't blow up a build if missing. ** phf If ~native-inputs~ are missing, it's fine. But wait, maybe there is a misunderstanding here. Please, check the reasoning: in a cross-compilation context, we have two machines A and B, and: - native-inputs: dependencies that must be built on A for A ; - inputs: dependencies that must be built on A for B ; - propagated-inputs: like inputs but installed in the profile. If installing Elixir (like Python) gathers all libraries in the profile wit= h a variable like ~GUIX_ERL_LIBS~, then, it would be enough to list dependencie= s (in packages) in ~propagated-inputs~ instead of ~inputs~ and make them availabl= e to Elixir through ~ERL_LIBS~ (like ~GUIX_PYTHONPATH~ and ~PYTHONPATH~). As a consequence, the ~install-dependencies~ phase would be reduced to ~ERL_LIBS=3D$GUIX_ERL_LIBS~. Is this an answer to: "You might get around this with propagated-inputs maybe?" ** lilyp If environment variables work, that is clearly to be preferred over any oth= er magic we've considered so far. My hunch is that rebar-build- system was written this way because environment variables didn't work, however. Same with Rust and Node, which are kinda yucky in this regard. Other than that, yes, (ab)using propagated-inputs like that is the way to g= o when there's no smarter alternative available. ** phf As hinted [[id:d9ffbbfa-3cd3-4a44-9acb-04a0627b90d7][above]], abusing propagated inputs works up to the ~elixir-machete~ package. Essentially, it is enough to add the code below to the ~elixir~ package: #+begin_src scheme (native-search-paths (list (search-path-specification (variable "GUIX_ERL_LIBS") (files (list "lib/erlang/lib" (string-append "lib/elixir/" (version-major+minor version))))))) #+end_src ~install-dependencies~ has been deleted altogether for a single line added = to ~set-mix-env~. #+begin_src scheme (setenv "ERL_LIBS" (getenv "GUIX_ERL_LIBS")) #+end_src So, if Erlang and Elixir dependencies are added to propagated inputs, then = it seems to work as expected.