From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id UH3+JNFE4l+UTAAA0tVLHw (envelope-from ) for ; Tue, 22 Dec 2020 19:11:13 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id YN3OINFE4l9pEgAA1q6Kng (envelope-from ) for ; Tue, 22 Dec 2020 19:11:13 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 92B38940367 for ; Tue, 22 Dec 2020 19:11:12 +0000 (UTC) Received: from localhost ([::1]:53750 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1krn3v-0004R5-Gt for larch@yhetil.org; Tue, 22 Dec 2020 14:11:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:57074) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1krn3m-0004Qw-O3 for bug-guix@gnu.org; Tue, 22 Dec 2020 14:11:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:39379) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1krn3m-0006DX-GC for bug-guix@gnu.org; Tue, 22 Dec 2020 14:11:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1krn3m-0007VN-Bd for bug-guix@gnu.org; Tue, 22 Dec 2020 14:11:02 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#45316: [PATCH]: Re-introduce Emacs packages specific installation prefix. Resent-From: Leo Prikler Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 22 Dec 2020 19:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45316 X-GNU-PR-Package: guix X-GNU-PR-Keywords: patch To: Maxim Cournoyer Received: via spool by 45316-submit@debbugs.gnu.org id=B45316.160866424728824 (code B ref 45316); Tue, 22 Dec 2020 19:11:02 +0000 Received: (at 45316) by debbugs.gnu.org; 22 Dec 2020 19:10:47 +0000 Received: from localhost ([127.0.0.1]:50925 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1krn3X-0007Uq-6y for submit@debbugs.gnu.org; Tue, 22 Dec 2020 14:10:47 -0500 Received: from mailrelay.tugraz.at ([129.27.2.202]:4140) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1krn3V-0007Ue-0P for 45316@debbugs.gnu.org; Tue, 22 Dec 2020 14:10:46 -0500 Received: from nijino.local (217-149-174-13.nat.highway.telekom.at [217.149.174.13]) by mailrelay.tugraz.at (Postfix) with ESMTPSA id 4D0mBn18VYz3xRV; Tue, 22 Dec 2020 20:10:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tugraz.at; s=mailrelay; t=1608664241; bh=42/OqDa9gmC486axDWnf9abCwq91RHJTI3YfnJ00iGQ=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=TIxftcZy2Y7WJLB+ZIMYi4+SCzEXfx1URvINkWMRs73QSS85WewzQ/tcX7Imq6rUp grxtfwWSTVPcF0nL87YU7ZcZyfvDbjhPWTyY7x/MCPfUhYwu26+p0Ay38h1BRdUWfU wSWnHkVxOQI3znSMIrooiHNTD3SM/0/rZCAEDGug= Message-ID: <4f40e3fb175201b21e40718022e93eb27efab3fb.camel@student.tugraz.at> From: Leo Prikler Date: Tue, 22 Dec 2020 20:10:40 +0100 In-Reply-To: <875z4tyapq.fsf@gmail.com> References: <87ft3ysen7.fsf@gmail.com> <878dbbdf4883d509865a1049c11fbe9ae792cace.camel@student.tugraz.at> <875z4tyapq.fsf@gmail.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.34.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUG-Backscatter-control: bt4lQm5Tva3SBgCuw0EnZw X-Spam-Scanner: SpamAssassin 3.003001 X-Spam-Score-relay: -1.9 X-Scanned-By: MIMEDefang 2.74 on 129.27.10.117 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 45316@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.22 Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=tugraz.at header.s=mailrelay header.b=TIxftcZy; dmarc=fail reason="SPF not aligned (relaxed)" header.from=student.tugraz.at (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 92B38940367 X-Spam-Score: -1.22 X-Migadu-Scanner: scn1.migadu.com X-TUID: iGkM3re9ZvO+ Hi Maxim, Am Dienstag, den 22.12.2020, 13:09 -0500 schrieb Maxim Cournoyer: > Hi Leo, > > Leo Prikler writes: > > > Hello Maxim, > > > > As someone, who lobbied for the current status quo, I have some > > thoughts to share. > > I'm happy you're commenting :-). > > > Am Montag, den 21.12.2020, 22:28 -0500 schrieb Maxim Cournoyer: > > > The Emacs packages built with the Emacs built system used to be > > > installed in a sub-directory under the share/emacs/guix.d/ > > > directory, > > > but this was changed in commit > > > 65a7dd2950ca13a8b942b2836260a2192351b271 > > > shortly after having accommodated the site-start.el machinery to > > > enable > > > loading packages from any profile (via the EMACSLOADPATH search > > > path > > > specification). > > Won't this reintroduce then? > > This bug was caused by the EMACSLOADPATH environment variable being > too > long. This was because the original search path specification was > recursively adding any directories under site-lisp/ so that package > installed under a guix.d sub-directory could were directly added to > the > load path. > > In the current situation, the EMACSLOADPATH contains only the site- > lisp/ > directory of any profile, plus the versioned lisp/ directory of the > installed Emacs package. This patch series doesn't change that, so > no, > that bug wouldn't be reintroduced. Which is also the reason why our startup code changes and people have to accommodate to that, right? > > > While this change allowed to expose simply and directly the > > > packages > > > found in EMACSLOADPATH, it does introduce the risk of file name > > > collisions when multiple Emacs packages are joined in the same > > > profile, > > > especially with Emacs packages increasing in complexity (e.g., > > > using > > > more than a single .el file!) and expecting to have both their > > > sources > > > and resources extracted under their own nested directory rather > > > than > > > as > > > a flat collection (ELPA, MELPA). > > > One recent example I stumbled on was attempting to use the > > > emacs-yasnippet-snippets package along with emacs-elpy; both > > > wanted > > > to > > > install a 'snippets' directory to share/emacs/site-lisp/snippets, > > > collided and resulted in problems that prove difficult to > > > understand. > > I believe that to be a problem in those packages. Data should not > > be > > installed into share/emacs/site-lisp, but share/emacs/etc. See for > > instance also emacs-telega, which – while not quite adhering to the > > above – installs its data in share/emacs/{telega-contrib,telega- > > data}. > > > > Regardless of what you intend to do with site-lisp otherwise, data > > files should *not*, I repeat *not* be installed there. I do > > believe > > standardizing share/emacs/etc is the way to go, however. > > While I agree that it would be more elegant the way you propose, > that's > not the way Emacs packages have been standardized. So going against > the > single "content directory" (c.f. info "(elisp) Packaging Basics") > would > break many Elisp library assumptions about where there files are, > causing more friction (thus work) to adapt them in Guix. The content > directory of an Elisp package can contain any kind of files (code, > images, etc.), according to info "(elisp) Multi-file Packages". I suppose said manual is perhaps slightly outdated. > A package is either a “simple package” or a “multi-file package”. A > simple package is stored in a package archive as a single Emacs Lisp > file, while a multi-file package is stored as a tar file (containing > multiple Lisp files, and possibly non-Lisp files such as a manual). When was the last time, you've installed a tar to site-lisp? I also feel as though the expected contents are limited very much to README, COPYING and the like: --8<---------------cut here--------------begin-------------->8--- $ find ~/.guix-profile/share/emacs/27.1/lisp/ -type f \ -not -name '*.el' \ -and -not -name '*.elc' \ -and -not -name '*.el.gz' ~/.guix-profile/share/emacs/27.1/lisp/term/README ~/.guix-profile/share/emacs/27.1/lisp/COPYING ~/.guix-profile/share/emacs/27.1/lisp/README --8<---------------cut here---------------end--------------->8--- Perhaps we should ask Emacs folks how they feel about separating code and data, but bugs have already been caused here and elsewhere by stuffing them together. (I believe yasnippet was once already the culprit at some point before it got reorganized.) In the meantime, yes, doing so might cause extra work, but 1) it only affects a subset of packages, and 2) of this subset, we can prioritize those, that do exhibit this behaviour. > > > This is what motivated this patch series, where the site-start.el > > > auxiliary code used for package discovery is extended to support > > > packages installed in their own directory under a > > > 'share/emacs/guix' > > > installation prefix, via Emacs' own package library! > > > > > > The emacs-build-system is updated for this new installation > > > prefix, > > > as > > > well as existing packages and documentation. Parting with a > > > directly > > > usable EMACSLOADPATH means that site-start.el *must* run for > > > packages > > > to > > > appear in the load-path; that means for running a test suite, the > > > -Q > > > or > > > --quick Emacs options cannot be used, since it implies --no-site- > > > file. > > Having no usable EMACSLOADPATH sounds like a dealbreaker to > > me. Could > > one be restored by using direct subdirectories to share/emacs/site- > > lisp > > or would that merely be a cosmetic change? > > We have two schemes to accommodate for our Emacs packages: > > 1. Those installed via their own mean, e.g. make install, using the > gnu-build-system for example. These would still typically install > their > packages directly under site-lisp, possibly multiple files (that > could > still collide). Why? Seems inconsistent, does it not? > 2. Those installed via the emacs-build-system. With the proposed > changes, those now go to site-lisp/guix/. The 'guix' sub-directory > makes it unambiguous that anything found under is to be loaded by > package.el; the `package-directory-list' variable can be pointed to > it > to have the Emacs' package library discover these self-contained > packages. I think you've lost me here. Basically, instead of being able to `(require 'my-package)` you first need to load my-package through package.el and then can `(require 'my-package)`? I am not sure, what the benefit of that would be, if any. > Currently if you use -Q, the Elisp libraries are in the load-path, > but > not loaded (you need manually do M-x load-library before you can use > it), or call M-x guix-emacs-autoload-packages to load their > autoloads. > For programs, this mean (require 'some-library) works, but M-x > some-library-autoloaded-function doesn't. > > After this change, if you use -Q the new style Emacs packages are not > visible at all until you activate them with 'M-x load-library > guix-emacs' then 'M-x guix-package-initialize', or (require 'guix- > emacs) > then (guix-package-initialize), programmatically. I do think, that there are legitimate reasons to not require a full initialization here, particularly for the use in batch scripts, but I'm not quite sure how much work we currently put into making sure that everything is available. > I don't see this as a problem. -Q is simply an alias for > "--no-init-file" "--no-site-lisp" "--no-splash" "--no-x-resources" > "--no-site-file". But not "--no-load-path" "--no-nothing" ;) Being able to load the same libraries as without is vital for debugging and scripts. (Granted, some distros probably break with --no-site- lisp, but that's nothing to strive for imo.) > [...] > When put in perspective, that's 3 seconds to refresh about 900 > packages. I'm guessing most users must have less than 100 Emacs > packages in their collections (I have about 35). I think the current > performance is good enough to not worry optimizing it at this stage. Point taken, I don't have that many either. I was merely interested, given that I rarely dive that deep into runtime statistics of Elisp code. Regards, Leo