From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id +NCPKbcX1mXu1QAAe85BDQ:P1 (envelope-from ) for ; Wed, 21 Feb 2024 16:33:11 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id +NCPKbcX1mXu1QAAe85BDQ (envelope-from ) for ; Wed, 21 Feb 2024 16:33:11 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=futurile.net header.s=selector1 header.b=WLi9C9cw; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1708529502; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=//+zJDb4puy87v0i+VOAHHQF7FNv+hDr8FfNAOXrKEg=; b=elqqYdz+gnB6/nbcH6D+XlbRJmU7sZH8gGy52JWiMqTNseh9VUgvNvoVmaKQmpvus6kjC+ kUnhk5oSByhZF8AhMciqRmn/5ifaeJDBXHjPMGEcvl4PhtdciRls2DwwQIzDRA1CnmmBja AhLgKiuY7uNEuSdIYdmBdHFuH2F8MYve6BAgRcNlcmjO4FpvCPuzeKLCmDJB353Jylabt9 XixFwaWm1fs1fsQOrdUAAz/6wA1gjZTDjK5KkTlmI8UbJ1qZWOq1o1LpRvDgJCu71PIW2T gddOC/aTJir+0f1ONL1VF+nmoiLPMP+dfiS7QzCU20hp/O6kzVmRsA3h9kd03g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=futurile.net header.s=selector1 header.b=WLi9C9cw; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Seal: i=1; s=key1; d=yhetil.org; t=1708529502; a=rsa-sha256; cv=none; b=mneglWI32c+Jq1cNtOW1c8RmmGBTKfoPue590WIwTUMqnCDz0fULa73AJI3GN0GTyIOZbX 0z4r79gyXm/nt22jNIbfsY2ZNt0H8XmVQHI7HFrkhY2ot1KXWWlBT7LavGZovPCHi7S6Yx 0/mXWtCVm62YTkMaltHrxcBGL+v3VZ7r1m06hQMq9MoSdFPOPY6OgS/TF0PShFX0dfT8i+ ioCu19EJBl7k4VAfM0clRh2bEclJWCk5kjkvEhGGOzed5kV7avGnG5ZBrbPkhfPL0XUaL9 U/40wWWJCKgbasynWDJmJ4G+yWqu4Br56JRt+Y9YH3Mwm2bZa7FG3/94enTNcA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2435A7A74F for ; Wed, 21 Feb 2024 16:31:42 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rcoMq-0004yU-7k; Wed, 21 Feb 2024 10:18:40 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcoJl-0000nB-ST for guix-devel@gnu.org; Wed, 21 Feb 2024 10:15:30 -0500 Received: from mailtransmit05.runbox.com ([2a0c:5a00:149::26]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rcknl-0005Q0-Pf for guix-devel@gnu.org; Wed, 21 Feb 2024 06:30:23 -0500 Received: from mailtransmit02.runbox ([10.9.9.162] helo=aibo.runbox.com) by mailtransmit05.runbox.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1rcknc-0045zh-M9; Wed, 21 Feb 2024 12:30:04 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=futurile.net; s=selector1; h=In-Reply-To:Content-Transfer-Encoding: Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=//+zJDb4puy87v0i+VOAHHQF7FNv+hDr8FfNAOXrKEg=; b=WLi9C9cwKP1qv4l3E0Baa+5knS 4oEIiWaqcGyF73RqnzZHyBfPKE8DmyNuWkOabN3i+rKDdYkxPaCYqp61MEVsgq7q9Su2xHiJcN9Ut 7YChCIiEMDE6nlxColdPjDq5DE6PzoqDSPO+dvIfG6CX57IMQrlFwvd9NJorpW2vgv7PrqBiuuBQU ARwiPH03P6us08Mc8DSlvImLMAJqrUTgYzicKFZJKsw8G8gKs+dNh6KITdimm53d9v7VSmwp6BaRA bFrM+qxFhlSYFFAtkwr9NuNVt7bRTUpljbLrxsGnhkKmGqC5RRwvVexNdpS7XwCvDhA/bPdJ09dw5 iGnHBBsw==; Received: from [10.9.9.72] (helo=submission01.runbox) by mailtransmit02.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1rcknb-0001vM-2Y; Wed, 21 Feb 2024 12:30:03 +0100 Received: by submission01.runbox with esmtpsa [Authenticated ID (641962)] (TLS1.2:ECDHE_SECP256R1__RSA_SHA256__AES_256_GCM:256) (Exim 4.93) id 1rcknT-00F1NK-Un; Wed, 21 Feb 2024 12:29:56 +0100 Date: Wed, 21 Feb 2024 11:29:54 +0000 From: Steve George To: Maxime Devos Cc: "guix-devel@gnu.org" , r0man , Reilly Siegel Subject: Re: Proposal to turn off AOT in clojure-build-system Message-ID: References: <20240219154444.p2kj2B0094wMGJ4012kk71@baptiste.telenet-ops.be> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20240219154444.p2kj2B0094wMGJ4012kk71@baptiste.telenet-ops.be> Received-SPF: permerror client-ip=2a0c:5a00:149::26; envelope-from=steve@futurile.net; helo=mailtransmit05.runbox.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_PERMERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -6.89 X-Spam-Score: -6.89 X-Migadu-Queue-Id: 2435A7A74F X-TUID: Uw/7YFugUxO9 Hi Maxime, On 19 Feb, Maxime Devos wrote: (...) > > Consequently, there is no specific statement saying 'Distributors should not AOT libraries' that I can point to. > > In this bit about differences in perspective, I haven’t seen any mention of AOT, hence the “Consequently” does not follow. The part that’s missing here is that (IIUC) in Clojure, it is somewhat conventional to stuff the compiled .class files in a superior Aryan JAR instead – the inferior UnderJARs you get from the “guix install clj-whatever” equivalent would only contain non-compiled .clj (and data files, whatever). > > > But, I would like to draw attention to this thread on Clojureverse as the best source I could find: > >Alex Miller is the main community manager for Clojure, and is a maintainer of the core libraries, so his perspective is key. He notes that, AOT code is tied to *specific versions of Clojure*: > > > > "AOT'ed code is that it is inherently the product of a particular version of tthe Clojure compiler ... I would recommend NOT AOT compiling libraries" [4] > > This reasoning does not follow – yes, it is tied to the Clojure version, so what? Guix automatically rebuilds dependents when the dependency (in this case, the Clojure compiler) changes. (...) I think this preceding sentence is the heart of different assumptions between us. The Clojure packages we haave are for developers writing applications (libraries and tools). The ecosystem has very few end-user applications [0]. As a Clojure developer I can use the Clojure tools from Guix, and a few libraries. We (and all the other distributions) have a miniscule portion of the Clojure/Java library universe [1]. This leads to the following usage scenarios: 1. A developer installs Clojure from Guix, and uses libraries from outside Guix. They can install the JVM/Clojure and some common tools (like clj-tools-cli). They will use libraries from elsewhere, including their own. AOT compilation is a problem because of the issue of mixed AOT and non-AOT. 2. A developer installs a Clojure from outside Guix, uses libraries from inside Guix This will cause problems because the Guix Clojure libraries will have been AOT'd by a different version of the compiler. It's also fairly common to install/use parallel versions of Clojure/jvm due to different deployment needs, this is likely to cause difficult to find bugs. I can see the sense of compiling to byte code if it's an end-user application. In that case we'd want to make the start-up as fast as possible. Your comments seem to have this use-case in mind. But, today there aren't any such end-user Clojure applications in Guix. > >I believe this means that with AOT code on, any user who installs a different version of Clojure from the one that we used to AOT the libraries *may* have problems. > > Unlike, say, Maven, this situation simply does not happen in Guix, because we don’t just download binaries and call it a day (except for some bootstrapping stuff, but that’s not relevant for Clojure AOT), because we have functioning recompilation of dependents, because of shebang patching, because binaries that are to be invoked should not rely on the ambient CLASSPATH / LD_LIBRARY_PATH / etc. and, if, the underlying binaries do rely on that, they are wrapped (see wrap-program) to set them (or, at least, they should be, you might find some bugs in this department if you go looking). > > Even if they aren’t wrapped, then in that case the dependencies are propagated-inputs, and you can only have a single version of a propagated package in the same environment (barring stacking environment shenanigans, but then you are looking for it and/or you can just report a bug about how the binaries should be wrapped/classpath should be patched in/...). > In this paragraph you're assumption is that a Guix user is only using libraries from within Guix. Hopefully, I've shown why this assumption is unlikely above. You also mentioned Debian, and @e.hashiman [2] said that Clojure libraries are not AOT'd on Debian, while applications are. From what I can find there are 130 packages in Debian with the word Clojure in them [3]. I looked at a selection and it seems true that Debian does not AOT libraries (and I can't find any Clojure 'apps'). For completeness I also checked what Clojars, the main distribution archive for Clojure developers, does: - https://sources.debian.org/src/core-match-clojure/1.0.0-1/project.clj/ - Lein-based project - has to have :aot keyword - distributes as source files - Clojure source files (.clj) in Debian - Clojure source files in Clojars - Byte compiled files in GUIX - installed and inspected with jar -tvf - https://sources.debian.org/src/data-csv-clojure/1.0.0-1/deps.edn/ - tools.deps project - has to have a specific aot alias - distributes as source files - Clojure source files (.clj) in the Debian provided jar - Clojure source files in the Clojars provided jar - Byte compiled files in Guix - installed and inspected with jar -tvf I tried looking at both Arch and Nix but neither of them package any Clojure libraries at all. Does this help clarify why I'm asking to change the default? Thanks, Steve [0] https://fr.surveymonkey.com/stories/SM-_2BH3b49f_2FXEkUlrb_2BJSThxg_3D_3D/ , there are tools for programmers like clj and babashka [1] We have 12 Clojure packages, Debian has 180 and according to Maven central (https://mvnrepository.com/repos) there are ~2,000 repos, Clojars shows 3105 projects. [2] https://mastodon.social/@ehashman@cloudisland.nz/111960714219364737 [3] https://sources.debian.org/search/clojure/ [4] https://www.debian.org/doc/packaging-manuals/java-policy/ch02.html#policy-libraries