From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id fDx3KVkR6F+6YAAA0tVLHw (envelope-from ) for ; Sun, 27 Dec 2020 04:45:13 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id eHjbJFkR6F8MagAAB5/wlQ (envelope-from ) for ; Sun, 27 Dec 2020 04:45:13 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 27EED9403E6 for ; Sun, 27 Dec 2020 04:45:12 +0000 (UTC) Received: from localhost ([::1]:49822 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktNvZ-0005zy-Do for larch@yhetil.org; Sat, 26 Dec 2020 23:45:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50826) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktNvS-0005zp-8O for bug-guix@gnu.org; Sat, 26 Dec 2020 23:45:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:48621) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ktNvS-0003K2-0I for bug-guix@gnu.org; Sat, 26 Dec 2020 23:45:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ktNvR-0002CD-Sj for bug-guix@gnu.org; Sat, 26 Dec 2020 23:45:01 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#45316: [PATCH]: Re-introduce Emacs packages specific installation prefix. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 27 Dec 2020 04:45:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45316 X-GNU-PR-Package: guix X-GNU-PR-Keywords: patch To: Leo Prikler Received: via spool by 45316-submit@debbugs.gnu.org id=B45316.16090442828398 (code B ref 45316); Sun, 27 Dec 2020 04:45:01 +0000 Received: (at 45316) by debbugs.gnu.org; 27 Dec 2020 04:44:42 +0000 Received: from localhost ([127.0.0.1]:60167 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktNv7-0002BN-D3 for submit@debbugs.gnu.org; Sat, 26 Dec 2020 23:44:42 -0500 Received: from mail-qk1-f182.google.com ([209.85.222.182]:39380) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktNv5-0002BA-NL for 45316@debbugs.gnu.org; Sat, 26 Dec 2020 23:44:40 -0500 Received: by mail-qk1-f182.google.com with SMTP id p14so6574177qke.6 for <45316@debbugs.gnu.org>; Sat, 26 Dec 2020 20:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=5IhA8/vR3Wzw5YCRVYEnhVIBBb0xPpd64g4FwHO8dek=; b=M1DpFQmVolYtDufIuznQ03wKJxut2dq8xpTnbIZATb0MUiSOcYK9cilQWKX1mVHaxI 1Yo9fSPg73OoSnVrY64dUwKP6S1QmVv5U8WLRYO9TlpQYwL3BOqPCNt8gNTUKuVqoTIp /dSnScFVsH7kEV+BRw5+xwZbKgni4zW6xMwQc4mtxOltkinFgY6wUSaoUv6rCdGwuCS1 saHN0s6zJ10tz0f8niAFKo/TneL4NhrgLvWweNLB6WvnkkJdebbDoNl/1dKcwFmUEj9Y zLoq0O592tSbaVVVktyOw1nKV2Choj+26U1UHXcuAzMRFaOANTx8Ebxi1dtaxFq4iK5+ vTDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=5IhA8/vR3Wzw5YCRVYEnhVIBBb0xPpd64g4FwHO8dek=; b=h5YiJ2whHwhBEXW6n/a+6oHahAhYHcFtnqakV0zg2PZfKROdixnMaEUjCC1FPRLpRe B9jsPG7+d0eCV4ulte/IJDkekJfkmqmWftxNK9Hlftcaco5V1cWWZxCgOk6K9LTh7Zt7 RR6pgv8qxjozgUi6IXk8W6NT7KkrAGWsPAZM+DkG6KJPlfGMkHXEb7C/tB0apkdGcp9f +fsxXj6ntIi17KtZBrc3c8qKGxqntH/Y3ZBn0+9UfrNGj2D4vbtzTezcvUH6dWnygTme +m+y7FfZpxLx5PYnTEL6wIrZFmAUW87m/Tt7h3PG0lbwr6/iDarlfZqZuFynreCcTEIP WYFg== X-Gm-Message-State: AOAM5318g5ybBhvRr9Uuhe1xgrqNNoayqFTnUAVpZbxBegT/nKlQJCBa GZg+276iV7XgTZRto69vpaqiWHu9ROo= X-Google-Smtp-Source: ABdhPJxDWTF3sTkBdf4lvsWqZNlDJ0CrxmX4jYW8fxyjvpeZVzb8ghmhR1FGvq1uBkxFOMH6skFhkg== X-Received: by 2002:a37:c442:: with SMTP id h2mr40095964qkm.283.1609044273730; Sat, 26 Dec 2020 20:44:33 -0800 (PST) Received: from hurd (dsl-10-149-45.b2b2c.ca. [72.10.149.45]) by smtp.gmail.com with ESMTPSA id l1sm21273691qtb.42.2020.12.26.20.44.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 26 Dec 2020 20:44:33 -0800 (PST) From: Maxim Cournoyer References: <87ft3ysen7.fsf@gmail.com> <878dbbdf4883d509865a1049c11fbe9ae792cace.camel@student.tugraz.at> <875z4tyapq.fsf@gmail.com> <4f40e3fb175201b21e40718022e93eb27efab3fb.camel@student.tugraz.at> <87pn2xw47q.fsf@gmail.com> Date: Sat, 26 Dec 2020 23:44:32 -0500 In-Reply-To: (Leo Prikler's message of "Sat, 26 Dec 2020 11:56:27 +0100") Message-ID: <878s9jx3gv.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 45316@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: 0.28 Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=M1DpFQmV; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 27EED9403E6 X-Spam-Score: 0.28 X-Migadu-Scanner: scn1.migadu.com X-TUID: iuPVCO1/2csQ Hi Leo, Leo Prikler writes: [...] >> If you want to have a clearer idea of how packages from ELPA and the >> likes are installed, you can have a peek into the 'package.el' file >> shipped with Emacs (spoiler: it's basically just extracting the >> contents >> of the package archive to a single directory -- see the `package- >> unpack' >> procedure). > If we follow that to the letter, should we not install those packages > directly to site-lisp/$PACKAGE-$VERSION? Why the guix directory? I used to wonder the same, but after getting familiar with the 'package' Emacs library, it defaults (per the default value of the `package-directory-list' variable) to such a layout: directory-on-load-path/prefix/name-version/ where 'prefix' defaults to 'elpa'. It seems a good idea to have the ELPA-style packages live by themselves, separated from the 'classic' packages that are installed directly to the load path, to avoid the package.el loader getting confused or scanning entries it wouldn't know how to load anyway. That said, it's shouldn't be strictly required, as old style packages or resource directories would be lacking a *-pkg.el file and would be disregarded, and it should be possible to install the package directories directly to the load path if we wanted to. > Also, if we do go that route, I think we should install a subdirs.el > through a profile hook, that adds all these package directories to the > load path. See also '(elisp) Startup Summary'. Alternatively, we can > let subdirs.el defer to guix-emacs.el, but that'd be very cheeky. > Either way, we should definitely make -Q work. If we use a subdirs.el file, it could add the ELPA-style packages more simply if they are under the guix sub-directory (guix/*). Otherwise we'd have to filter the resource directories that potentially find their ways to the load path for old style packages. Our custom subdirs.el file could be shipped with our Emacs packages, installed in their output at share/emacs/site-lisp/subdirs.el. When a profile union would be created this subdirs.el would automatically be made available. >> Here's an experiment I've done with a profile containing the whole of >> our current emacs-build-system based packages, with this current >> change, >> that checks for collision at the top level of a package installation >> directory: >> >> --8<---------------cut here---------------end--------------->8--- >> $ ./pre-inst-env guix package -p /tmp/nnew-emacs -m emacs-packages- >> manifest.scm >> >> $ find -L /tmp/nnew-emacs/share/emacs/site-lisp/guix -maxdepth 2 >> -mindepth 2 -not -regex '.*\.elc?$' \ >> | awk -F '/' '{ if ($9 in packages) \ >> {printf("%s directory of %s package collides with that of >> package %s\n", $9, $8, packages[$9])} \ >> else {packages[$9] =3D $8} }' >> >> doc directory of modus-operandi-theme-1.0.2 package collides with >> that of package racket-mode-0.0.2-6.5eb31a2 >> tools directory of unidecode-0.2-1.5502ada package collides with that >> of package company-cabal-0.3.0-1.62112a7 >> snippets directory of minitest-0.8.0-1.1aadb78 package collides with >> that of package elpy-1.35.0 >> test directory of realgud-1.5.1 package collides with that of package >> flycheck-haskell-0.8-2.32ddff8 >> snippets directory of yasnippet-snippets-0.23 package collides with >> that of package elpy-1.35.0 >> snippets directory of haskell-snippets-0.1.0-0.07b0f46 package >> collides with that of package elpy-1.35.0 >> snippets directory of rspec-1.11-1.66ea7cc package collides with that >> of package elpy-1.35.0 >> doc directory of modus-themes-1.0.2 package collides with that of >> package racket-mode-0.0.2-6.5eb31a2 >> data directory of emojify-1.2 package collides with that of package >> unidecode-0.2-1.5502ada >> test directory of systemd-mode-1.6 package collides with that of >> package flycheck-haskell-0.8-2.32ddff8 >> lib directory of slime-2.26.1 package collides with that of package >> robe-0.8.2 >> contrib directory of sly-1.0.0-7.68561f1 package collides with that >> of package slime-2.26.1 >> lib directory of sly-1.0.0-7.68561f1 package collides with that of >> package robe-0.8.2 >> doc directory of modus-vivendi-theme-1.0.2 package collides with that >> of package racket-mode-0.0.2-6.5eb31a2 >> doc directory of evil-1.14.0 package collides with that of package >> racket-mode-0.0.2-6.5eb31a2 >> data directory of all-the-icons-4.0.1 package collides with that of >> package unidecode-0.2-1.5502ada >> snippets directory of feature-mode-20190801-1.11ae167 package >> collides with that of package elpy-1.35.0 >> --8<---------------cut here---------------end--------------->8--- >> >> So 17 Emacs packages in Guix currently conflict, and Guix seems to be >> silent about it when building a profile with them via 'guix package >> -p' >> (a bug?). > It's a bug in those packages; not in Guix. I used to think the same, but after reading the Elisp reference manual mentioning how Emacs packages to be installed with package.el are expected to live in their own distinct directory with their own resources; we can't really say that's it's a bug in the packages: they just weren't designed to be merged in a flat directory with other Elisp packages. Users installing these packages manually can simply add their top level directory to their load path, else use package.el to install them. > The whole idea of Guix is, that such directories are merged. Now > collisions within files in those directories are a different topic =E2=80= =93 > only one can be chosen there and IIRC there should also be a way to > make them errors. I'll investigate this separately. [...] >> > > We have two schemes to accommodate for our Emacs packages: >> > > >> > > 1. Those installed via their own mean, e.g. make install, using >> > > the >> > > gnu-build-system for example. These would still typically >> > > install >> > > their >> > > packages directly under site-lisp, possibly multiple files (that >> > > could >> > > still collide). >> > Why? Seems inconsistent, does it not? >> >> They can be adapted in time; until then it's easy to accommodate >> both. > I think we should aim for consistency here, but let's adapt them in > follow-up commits. Agreed. >> > > 2. Those installed via the emacs-build-system. With the proposed >> > > changes, those now go to site-lisp/guix/. The 'guix' sub- >> > > directory >> > > makes it unambiguous that anything found under is to be loaded by >> > > package.el; the `package-directory-list' variable can be pointed >> > > to >> > > it >> > > to have the Emacs' package library discover these self-contained >> > > packages. >> > I think you've lost me here. Basically, instead of being able to >> > `(require 'my-package)` you first need to load my-package through >> > package.el and then can `(require 'my-package)`? I am not sure, >> > what >> > the benefit of that would be, if any. >> >> Only if you've specified --no-site-start, which would disable >> user-installed package discovery. This is the drawback of the >> trade-off, not the benefit :-). > It wasn't worded like one. The word unambiguous typically has a > positive connotation in my mind. And package discovery should function > regardless of site-start. The word unambiguous was used to describe that using the sub-directory guix/ would mean anything under it would be strictly packages to be loaded with package.el and nothing else. [...] >> > Being able to load the same libraries as without is vital for >> > debugging >> > and scripts. (Granted, some distros probably break with --no-site- >> > lisp, but that's nothing to strive for imo.) >> >> [...] >> >> Thanks for having me think harder about if this is really >> desirable/needed. For me the main plus is that we'd be adhering to >> the >> current standards used for Emacs packaging: everything is unpacked >> into >> a single directory owned by that package, so as long as we include >> any >> needed resources in the #:include argument of the emacs-build-system, >> it >> works as upstream intended it, with no need to patch variables or >> anything. I expect this situation to worsen as Emacs packages tend >> to >> get more complicated, and I don't feel strongly enough about it to go >> arguing about Emacs packaging standards on the Emacs mailing list :- >> ). > I agree, that adhering to the standards sounds nice, but this patch > makes it seem as if we're still cooking our own soup. I believe we'd > need to adhere to them harder than that in order for it to be an > upgrade of the status quo. I understand this may feel a hack; because for all I know, it is ;-). I'll try improving it with the subdirs.el discussed above. [...] > I think I've already laid that out above, but we really ought to have > site-lisp/$PACKAGE-$VERSION and a working subdirs.el. If we do it like > that, I don't think the multi-directory layout should cause us too many > troubles. > > As far as the build system is concerned, we would probably need to set > up an environment similar to what will be found in the end. > Imagine this: > > - environment-variables > - $PACKAGE-$VERSION/ > - $PACKAGE-$VERSION-inputs/ > - subdirs.el > - $INPUT1/ > - $INPUT2/ In the build environment, there is no profile so each package add their individual site-lisp/ to the load path (EMACSLOADPATH). With my proposed idea to add subdirs.el to Emacs itself, there'd be nothing to do here. Thanks again for this discussion; I'll work on a revised patch. Until then, happy new year! Maxim