From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id +FENHP2eBmd7rAAA62LTzQ:P1 (envelope-from ) for ; Wed, 09 Oct 2024 15:19:25 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id +FENHP2eBmd7rAAA62LTzQ (envelope-from ) for ; Wed, 09 Oct 2024 17:19:25 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b="p7T/WfWb"; dkim=fail ("headers rsa verify failed") header.d=runbox.eu header.s=selector1 header.b=GwVz2nat; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=runbox.eu (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1728487165; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=linbsRDj2UYRzL2K8HId+b/n3ba7Vnq7fjGXuW82GEg=; b=KHURJJnFP/Yw27DBCjz/uGRfYDWic4DAQdTIN6UHadwZq0yBg3jdoQ0TFuTSZeTlBD1aR7 JE0iSC7glknN5oKvV1hrcIJJf/jW0E4tNDTocm3R9DmeCwUpb+w3HFUwq9K0Swo3dFP5nG EA8EeD1uk0/QIQfksS0jhRZnFR9CXsWj/u+FhhJSCBNG0gR3Cyg3D156LXAd5z18oBUHRc kfvGC3xGQJbNsNM7tHAd9R0QEw1oTOvWTKUPPU/cKOtTxmiXVPotMfzwSXOM0EVUqxGwu5 jCS554xfU1SdbuWYCgFg3N8qH0FEds9flZ7I5GAzkijMcJRhQ8Mqq/tw7jyumg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b="p7T/WfWb"; dkim=fail ("headers rsa verify failed") header.d=runbox.eu header.s=selector1 header.b=GwVz2nat; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=runbox.eu (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1728487165; a=rsa-sha256; cv=none; b=Nflc60gc1n7EM7TJtGbrYhhCso+Q4g/fOAvPEZOKI/r+qWw8I11L5sy6GjQS+kSKYzHh+A 1EGasqMw8OAftrv96xzpN441121WDqHDUC3xQWm3hUf+CBIxt1Qa8H1+7OTMlI7Afj6q47 vreKsmXgBGWIhWfFiZCm1AWt4Ke3urz4EEerWj2xbHq8fEFss/V2fJYk2YYOBn5NU8xrdS haUV0nsgnTNNuZjGqAWWKAB6Q28B72TEGi9Dt1NvZRcqnc7SvAYLv2zqltY4b/ID2aD/CD 4yr/VGyTDhaHjJb1RX7e8QAQTd1jLQ4rdbgzJa7dRITlRW7xrJ3ZqaUFhzr5Kw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 331B0400F9 for ; Wed, 09 Oct 2024 17:19:25 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1syYSn-0007Pc-8H; Wed, 09 Oct 2024 11:18:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1syYSi-0007PS-Gd for bug-guix@gnu.org; Wed, 09 Oct 2024 11:18:53 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1syYSi-0000fJ-9M for bug-guix@gnu.org; Wed, 09 Oct 2024 11:18:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=In-Reply-To:References:Date:From:MIME-Version:To:Subject; bh=linbsRDj2UYRzL2K8HId+b/n3ba7Vnq7fjGXuW82GEg=; b=p7T/WfWbmtv24xw3nrLpSwSDA2X0dnLOueVyvf1/PMlyZDbmsL/fsB4s0hY+zH05J0e0LA1brQ3tTXxzSt0IF6S37Q21/weTP3X317FNEFqfAqQzvHu1M4ctMsO29Wf7Ubac1Re81qK+P1Aw4tdKkufmDAXsr1trYFPh+tR+L3K47n7D7nQBmkCud6Cbtlv1d84FLMKKAh0ZSbt0bwinJsfFmVL+bxcDCDu3xx8v3broUI4nR5ZRKlHRf9J3EbWhztnPkHQPIsBYLBIIkjL24afurOXDHsZhcJw3roWQUsClIQBIw6Crw8j1/HqtHkZH+MUAqLXfu768w/orkeOEaw==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1syYSs-0005oh-4Q for bug-guix@gnu.org; Wed, 09 Oct 2024 11:19:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#73681: Maybe partly undo the patch on Elisp comp-el-to-eln-filename Resent-From: "Martin =?UTF-8?Q?Edstr=C3=B6m?=" Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 09 Oct 2024 15:19:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 73681 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: "Liliana Marie Prikler" Cc: 73681 <73681@debbugs.gnu.org> Received: via spool by 73681-submit@debbugs.gnu.org id=B73681.172848710522312 (code B ref 73681); Wed, 09 Oct 2024 15:19:02 +0000 Received: (at 73681) by debbugs.gnu.org; 9 Oct 2024 15:18:25 +0000 Received: from localhost ([127.0.0.1]:57382 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1syYSG-0005no-4c for submit@debbugs.gnu.org; Wed, 09 Oct 2024 11:18:24 -0400 Received: from mailtransmit05.runbox.com ([185.226.149.38]:40402) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1syYSD-0005nX-Gv for 73681@debbugs.gnu.org; Wed, 09 Oct 2024 11:18:22 -0400 Received: from mailtransmit02.runbox ([10.9.9.162] helo=aibo.runbox.com) by mailtransmit05.runbox.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1syYPq-000tZ4-OS; Wed, 09 Oct 2024 17:15:54 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=runbox.eu; s=selector1; h=Message-Id:In-Reply-To:References:Date:Subject:CC:To:From: MIME-Version:Content-Transfer-Encoding:Content-Type; bh=linbsRDj2UYRzL2K8HId+b/n3ba7Vnq7fjGXuW82GEg=; b=GwVz2natrDPDaZudyB1FhG0Kpu 6SV/248rtVGACYNVXGktY1zp7IJTaoII0nlSe27FHYDETKWSXbB/VxJZ38nA/EwU0AcTJSysGT/IR c3BQ1g8IZ7uAODW5fK6k/0GoUDH1l4W1WByjpMoqpl4chhe5Yg6c8h+SvzPpbg7yc1QtBtu7Vazbt uBMk3959xAHbWdyC04qAA8Hu9f9sC5yzfYQmK7nZZ1JSqnrlFV2GAxNWztjiqj3/GGbOuvNG/LPUr YaepJAr9Qj7udsuZRnw0/HPSPDMpfY+yfvJKEU0zYDQFzucMX5YH70+b/XQdOhku+bOpArebatOTy T4KXGZSA==; Received: from [10.9.9.127] (helo=rmmprod05.runbox) by mailtransmit02.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1syYPq-0001yu-C7; Wed, 09 Oct 2024 17:15:54 +0200 Received: from mail by rmmprod05.runbox with local (Exim 4.86_2) (envelope-from ) id 1syYPq-0005iN-A6; Wed, 09 Oct 2024 17:15:54 +0200 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Received: from [Authenticated alias (1196375)] by runbox.com with http (RMM6); Wed, 09 Oct 2024 15:15:54 GMT From: "Martin =?UTF-8?Q?Edstr=C3=B6m?=" Date: Wed, 09 Oct 2024 17:15:54 +0200 (CEST) X-RMM-Aliasid: 1196375 X-Mailer: RMM6 References: <58598114857dce8a25e3b4d0477d212467a0173f.camel@gmail.com> In-Reply-To: Message-Id: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx11.migadu.com X-Migadu-Spam-Score: 3.05 X-Spam-Score: 3.05 X-Migadu-Queue-Id: 331B0400F9 X-TUID: FXJypgCpQ9aX Hi, thanks for info! This email will be a bit long because I'm quite confus= ed and thinking aloud, so no need to reply to all of it. But I appreciate = any attempt to shed clarity! On Tue, 08 Oct 2024 19:33:06 +0200, Liliana Marie Prikler wrote: > > It comes as part of the package. I don't want to assume that it has > > been compiled, since it's fairly performance-sensitive. That's why > > I'll either use a previously existing compiled object or make a new > > one. > Could you leave that decision to the user? I'm considering it, but ran into essentially the same problem. I need a wa= y to tell which one was loaded, of the .elc, .eln or .el. This takes the thread a bit off-topic but ... Do you know how? Currently I'm leaning towards an algorithm like #+begin_src elisp (defun guess-loaded-lib (basename) (let* ((el (locate-library (concat basename ".el"))) (elc (locate-library (concat basename ".elc"))) (eln (comp-el-to-eln-filename el))) (if load-prefer-newer (car (sort (delq 'nil (list el elc eln)) #'file-newer-than-file-p)) (if (and (file-newer-than-file-p elc el) (file-newer-than-file-p eln el)) ;; If elc is newer than el, assume it was compiled from el, and ;; therefore is safe to replace with eln eln elc)))) #+end_src I don't know how to "leave it to the user" any better than that.=20=20 On Guix, even in the future when we re-enable JIT compilation, this algorit= hm could never return an .eln, since =3Dfile-newer-than-file-p=3D returns n= il when both paths have identical timestamps from 1970. I found a pretty neat built-in:=20 (symbol-file 'SOME-FUNCTION-FROM-THE-LIB nil t) but its internals also use =3Dfile-newer-than-file-p=3D and =3Dcomp-el-to-e= ln-rel-filename=3D, so not sure it is any more reliable than what I have ab= ove. > There is a separate load path for natively compiled files, called > `native-comp-eln-load-path'. Good to know! I don't know how Emacs consults it though. I can't e.g. loca= te an .eln of a given library by calling something like #+begin_src elisp (locate-eln-file "org-node-parser") #+end_src so it seems I must simply do: #+begin_src elisp (let ((el (locate-library "org-node-parser.el"))) (when el (locate-eln-file (comp-el-to-eln-rel-filename el)))) #+end_src which is functionally equivalent to: #+begin_src elisp (let ((eln (comp-el-to-eln-filename (locate-library "org-node-parser.el")))) (when (file-exists-p eln) eln)) #+end_src > > I infer that Emacs starts with finding a library in load-path, then > > converts the path with `comp-el-to-eln-filename`, and checks if that > > file exists, then loads it. > > > > And crucially, it is not just about the filepath, the function hashes > > the file contents as well. That ensures that the output path is > > always different if the source file changes. > I think relying on such implementation details is perhaps permitted if > it's inside of Emacs itself, but even then it clashes with our > expectation that Emacs be graftable. OK, so if passing "file.el" to =3Dcomp-el-to-eln-filename=3D produces a pat= h with a hash of the content, like: .../eln-cache/29.4-HASH/module/file-HASH-BASED-ON-CONTENT.eln this would make Emacs non-graftable. Because the hash would change after f= ile.el is upgraded. I suppose that makes sense, thanks! Maybe, as a hack, the graft process could symlink the pre-graft filename to= the post-graft filename... so we don't have to alter the behavior of =3Dco= mp-el-to-eln-filename=3D. But yea... it's less dev effort to just not pre-compile any .eln files. Out of curiosity: if Guix does no JIT compilation at all anyway, why does i= t not let Emacs do it the usual way into ~/.emacs.d/eln-cache, post-install= ation? (Aside from the fact it would necessitate reverting the behavior of= =3Dcomp-el-to-eln-filename=3D.) Actually... (with reservation that I may be wasting your time) I'm starting= to wonder why the filename needs to stay the same for Emacs to be graftabl= e? Realistically, if all Emacs packages get upgraded, and if =3Dcomp-el-to= -eln-filename=3D works like upstream, we have some paths like /gnu/store/emacs-29.4/bin/emacs /gnu/store/emacs-magit-OLD/lisp/magit.el /gnu/store/emacs-magit-NEW/lisp/magit.el .../wherever/magit-OLDHASH.eln .../wherever/magit-NEWHASH.eln Then we start Emacs, and run its =3Dcomp-el-to-eln-filename=3D on the new m= agit.el, it would correctly return .../wherever/magit-NEWHASH.eln. No need = for static filenames to swap out. Is graftability just about the compiled objects inside of /gnu/store/emacs= -29.4/share/emacs/site-lisp/ that might be symlinked in from other Guix pac= kages? If so, I guess it ties back to my confusion about how Emacs uses the native= -comp-eln-load-path, because it seems that IF it just converted an .el to f= ind an .eln as I thought, then grafting would work automatically because th= e new .el is also symlinked in along with the new .eln, which would result = in =3Dcomp-el-to-eln-filename=3D returning the new .eln correctly. No, grafting is about avoiding a world re-compile, right? So like if packa= ge-A was built with an old dependency package-B, it does not get re-built w= hen package-B is upgraded. But does grafting actually ever change anything = in Emacs? When you start emacs and it loads package-A which in turn loads = package-B because there's a =3Drequire=3D statement for package-B, Emacs wi= ll actually load the fresh package-B from /gnu/store/emacs-package-B, won't= it? So grafting would seem to be meaningless. Not sure how to feel if it's like this instead: a compiled function from pa= ckage A, containing a call to some function "package-B-frobnicate", will ac= tually call some bytecode originating from /gnu/store/emacs-package-A/share= /emacs/site-lisp/package-B.elc? That's... not what happens, right? OK, pro= bably yes if it was a defsubst or defmacro. So we can have package A using a macro that was defined in an old version o= f package-B. I like that not, but the reason I ask is I'm trying to find o= ut where it is that the file path would influence the graft. > The way our load paths are set up, it is actually the opposite (which > still is a bug, just not the one reported). While `guix upgrade` or a > command to the similar effect will swap out the .eln under the hood, > the `.el` and `.elc` files stay stable =E2=80=93 remember what I wrote in= the > previous message about that having caused issues with byte compilation? I confess I'm puzzled. Does this just apply to a case like the following? - Emacs itself is upgraded - A package emacs-magit is NOT upgraded so that - magit.eln is rebuilt - magit.el and magit.elc stay the same That would make sense. Do you mean it's a bug because .elc files are also = not portable between different Emacsen? Just checking: is it correct to expect that if you actually upgrade the ema= cs-magit package, then its .el, .elc and .eln are all upgraded?=