unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#45359] [PATCH]: Re-introduce Emacs packages specific installation prefix.
@ 2020-12-22  3:28     ` Maxim Cournoyer
       [not found]       ` <handler.45359.D45359.161704236132600.notifdone@debbugs.gnu.org>
  0 siblings, 1 reply; 7+ messages in thread
From: Maxim Cournoyer @ 2020-12-22  3:28 UTC (permalink / raw)
  To: 45359

[-- Attachment #1: Type: text/plain, Size: 455 bytes --]

Hello,

I've goofed and sent this to guix-bugs, apparently, so I'm forwarding it
to guix-patches to make sure everybody possibly interested had a chance
to see it.

Thank you,

Maxim

-------------------- Start of forwarded message --------------------
Subject: bug#45316: [PATCH]: Re-introduce Emacs packages specific installation prefix.
To: 45316@debbugs.gnu.org
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Fri, 18 Dec 2020 17:00:10 -0500


[-- Attachment #2.1: Type: text/plain, Size: 2631 bytes --]

Hello Guix!

tl;dr: The Emacs build system and site-start.el loader are modified so
that Emacs packages are installed in their own distinct installation
directory.

The Emacs packages built with the Emacs built system used to be
installed in a sub-directory under the share/emacs/guix.d/ directory,
but this was changed in commit 65a7dd2950ca13a8b942b2836260a2192351b271
shortly after having accommodated the site-start.el machinery to enable
loading packages from any profile (via the EMACSLOADPATH search path
specification).

While this change allowed to expose simply and directly the packages
found in EMACSLOADPATH, it does introduce the risk of file name
collisions when multiple Emacs packages are joined in the same profile,
especially with Emacs packages increasing in complexity (e.g., using
more than a single .el file!) and expecting to have both their sources
and resources extracted under their own nested directory rather than as
a flat collection (ELPA, MELPA).

One recent example I stumbled on was attempting to use the
emacs-yasnippet-snippets package along with emacs-elpy; both wanted to
install a 'snippets' directory to share/emacs/site-lisp/snippets,
collided and resulted in problems that prove difficult to understand.

This is what motivated this patch series, where the site-start.el
auxiliary code used for package discovery is extended to support
packages installed in their own directory under a 'share/emacs/guix'
installation prefix, via Emacs' own package library!

The emacs-build-system is updated for this new installation prefix, as
well as existing packages and documentation.  Parting with a directly
usable EMACSLOADPATH means that site-start.el *must* run for packages to
appear in the load-path; that means for running a test suite, the -Q or
--quick Emacs options cannot be used, since it implies --no-site-file.

Benefits of using this approach:

+ Avoid inter-package file name collisions.

+ Better integration with user installed packages via M-x
package-install.  The Guix-installed packages are listed in M-x
package-list as 'external'.

Cons include:

- Slightly more complex loader (although much of it is offloaded to
  package.el), thus slightly slower (see the comparison below).

- Requires to ensure every package's test suite doesn't make use of -Q.

In my opinion the benefits outweighs the cons by a comfortable margin,
especially with the boring work of adapting the package collection
already done.

To test the performance of the new approach, the following manifest file
was used to test the rebuild of the ~900 Emacs packages making use of
the Emacs build system:


[-- Attachment #2.2: emacs-packages-manifest.scm --]
[-- Type: text/plain, Size: 806 bytes --]

(use-modules (gnu packages)
             (guix build-system)
             (guix packages)
             (srfi srfi-1))

(define %broken-emacs-packages
  (map specification->package
       '("emacs-picpocket"              ;tests fail
         "emacs-twittering-mode"        ;build fails

         ;; Broken only on current master, without new changes.
         "emacs-md4rd"
         "emacs-el-patch"
         "emacs-flymake-shellcheck"
         )))

(define %emacs-packages
  (fold-packages (lambda (package lst)
                   (if (eq? (build-system-name (package-build-system package))
                            'emacs)
                       (cons package lst)
                       lst))
                 '()))

(packages->manifest
 (lset-difference eqv? %emacs-packages %broken-emacs-packages))

[-- Attachment #2.3: Type: text/plain, Size: 2772 bytes --]


A simple benchmark testing the performance of the activation of the
hundreds of Emacs packages was then run using:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix environment --pure -m emacs-packages-manifest.scm \
                                  --ad-hoc emacs
[env]$ /run/setuid-programs/sudo /bin/sh -c 'echo 3 > /proc/sys/vm/drop_caches'
[env]$ emacs --batch --no-site-file \
    --eval="(progn (require 'guix-emacs) \
                   (require 'benchmark) \
                   (message \"(total gc-count gc-time) = %s\" \
                            (benchmark-run 1 (guix-emacs-autoload-packages))))"
--8<---------------cut here---------------end--------------->8---

On the master branch:

--8<---------------cut here---------------start------------->8---
[...]
Loading /gnu/store/qajc70c7nqycs1301ram8s3x7k9ibg5f-profile/share/emacs/site-lisp/zotxt-autoloads...
Loading /gnu/store/qajc70c7nqycs1301ram8s3x7k9ibg5f-profile/share/emacs/site-lisp/zoutline-autoloads...
Loading /gnu/store/qajc70c7nqycs1301ram8s3x7k9ibg5f-profile/share/emacs/site-lisp/ztree-autoloads...
(total gc-count gc-time) = (25.242400751 13 0.189669369)
--8<---------------cut here---------------end--------------->8---

Or about 0.65 s on a warm cache.

On a branch with these changes:

--8<---------------cut here---------------start------------->8---
Error loading autoloads: (file-missing Cannot open load file No such file or directory kotl/kotl-autoloads)
Error loading autoloads: (file-missing Cannot open load file No such file or directory helm-easymenu)
Error loading autoloads: (file-missing Cannot open load file No such file or directory /gnu/store/ryh0rasi9frm98dkd3kbck6hya6hn2qr-profile/share/emacs/site-lisp/guix/flycheck-cpplint-0.1-1.1d8a090/flycheck-cpplint-autoloads)
Error loading autoloads: (file-missing Cannot open load file No such file or directory /gnu/store/ryh0rasi9frm98dkd3kbck6hya6hn2qr-profile/share/emacs/site-lisp/guix/evil-anzu-0.03/evil-anzu-autoloads)
Error loading autoloads: (file-missing Cannot open load file No such file or directory /gnu/store/ryh0rasi9frm98dkd3kbck6hya6hn2qr-profile/share/emacs/site-lisp/guix/erc-image-0-3.82fb387/erc-image-autoloads)
ad-handle-definition: `ido-completing-read' got redefined
Error loading autoloads: (file-missing Cannot open load file No such file or directory tex-site)
(total gc-count gc-time) = (26.175704339 47 0.783184412)
--8<---------------cut here---------------end--------------->8---

Or about 3 seconds on a warm cache.

There a 6 errors that would need to be looked into, but I these look
like actual packaging problems rather than new issues.  The previously
used way to load the autoloads, '(load f 'noerror)' would have masked
them.

Thanks,

Maxim

[-- Attachment #3: Type: text/plain, Size: 67 bytes --]

-------------------- End of forwarded message --------------------

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [bug#45359] bug#47458: Terrible UX upgrading Emacs in Guix
       [not found] <87o8f23eew.fsf@netris.org>
@ 2021-03-29 15:55 ` Ludovic Courtès
  2021-03-29 15:55 ` Ludovic Courtès
  1 sibling, 0 replies; 7+ messages in thread
From: Ludovic Courtès @ 2021-03-29 15:55 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: 47458, Maxim Cournoyer, 45359

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> Eventually, I realized what the problem was:
>
> (1) My existing emacs session started failing because
>     ~/.guix-profile/share/emacs/27.1 had disappeared out from under it.
>
> (2) My newly launched emacs sessions were failing because my
>     EMACSLOADPATH variable was still set to its old value, pointing at
>     /home/mhw/.guix-profile/share/emacs/27.1/lisp, which no longer
>     existed.
>
> I'm not sure why I've never run into this problem before.  I'm also not
> sure what can be done to make this better, but if anyone has ideas, that
> would be good.  If a 7+ year Guix veteran developer gets bitten badly by
> this, I doubt that less experienced users will be impressed.

Ouch.  “It used to be” (speaking like a veteran :-)) that Emacs in Guix
would not use EMACSLOADPATH.  Then we switched to EMACSLOADPATH, which
had some advantages, but necessarily has this drawback.

IIUC, <https://issues.guix.gnu.org/45359> is about possibly
backtracking.  Maxim, what’s the status of this one?

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 7+ messages in thread

* [bug#45359] bug#47458: Terrible UX upgrading Emacs in Guix
       [not found] <87o8f23eew.fsf@netris.org>
  2021-03-29 15:55 ` [bug#45359] bug#47458: Terrible UX upgrading Emacs in Guix Ludovic Courtès
@ 2021-03-29 15:55 ` Ludovic Courtès
  2021-03-29 18:25   ` bug#45359: " Maxim Cournoyer
  1 sibling, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2021-03-29 15:55 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: 47458, Maxim Cournoyer, 45359

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> Eventually, I realized what the problem was:
>
> (1) My existing emacs session started failing because
>     ~/.guix-profile/share/emacs/27.1 had disappeared out from under it.
>
> (2) My newly launched emacs sessions were failing because my
>     EMACSLOADPATH variable was still set to its old value, pointing at
>     /home/mhw/.guix-profile/share/emacs/27.1/lisp, which no longer
>     existed.
>
> I'm not sure why I've never run into this problem before.  I'm also not
> sure what can be done to make this better, but if anyone has ideas, that
> would be good.  If a 7+ year Guix veteran developer gets bitten badly by
> this, I doubt that less experienced users will be impressed.

Ouch.  “It used to be” (speaking like a veteran :-)) that Emacs in Guix
would not use EMACSLOADPATH.  Then we switched to EMACSLOADPATH, which
had some advantages, but necessarily has this drawback.

IIUC, <https://issues.guix.gnu.org/45359> is about possibly
backtracking.  Maxim, what’s the status of this one?

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 7+ messages in thread

* bug#45359: bug#47458: Terrible UX upgrading Emacs in Guix
  2021-03-29 15:55 ` Ludovic Courtès
@ 2021-03-29 18:25   ` Maxim Cournoyer
  2020-12-22  3:28     ` [bug#45359] [PATCH]: Re-introduce Emacs packages specific installation prefix Maxim Cournoyer
  2021-03-30  8:04     ` [bug#45359] bug#47458: Terrible UX upgrading Emacs in Guix Ludovic Courtès
  0 siblings, 2 replies; 7+ messages in thread
From: Maxim Cournoyer @ 2021-03-29 18:25 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Mark H Weaver, 45359-done, 47458

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Mark,
>
> Mark H Weaver <mhw@netris.org> skribis:
>
>> Eventually, I realized what the problem was:
>>
>> (1) My existing emacs session started failing because
>>     ~/.guix-profile/share/emacs/27.1 had disappeared out from under it.
>>
>> (2) My newly launched emacs sessions were failing because my
>>     EMACSLOADPATH variable was still set to its old value, pointing at
>>     /home/mhw/.guix-profile/share/emacs/27.1/lisp, which no longer
>>     existed.
>>
>> I'm not sure why I've never run into this problem before.  I'm also not
>> sure what can be done to make this better, but if anyone has ideas, that
>> would be good.  If a 7+ year Guix veteran developer gets bitten badly by
>> this, I doubt that less experienced users will be impressed.
>
> Ouch.  “It used to be” (speaking like a veteran :-)) that Emacs in Guix
> would not use EMACSLOADPATH.  Then we switched to EMACSLOADPATH, which
> had some advantages, but necessarily has this drawback.
>
> IIUC, <https://issues.guix.gnu.org/45359> is about possibly
> backtracking.  Maxim, what’s the status of this one?

It's abandoned, The MUMI tracker lacks the responses from Leo Prickler,
but they had good arguments maintaining the status quo rather than going
with the extra complexity.  It also wouldn't change the issue at hand;
it'd merely prevent conflicts of *resources* files of Emacs packages
(and somewhat integrate with the Emacs native package manager, while
making the autoloading a bit slower).  It seems the price to pay is too
high for such a small gain.  I'm closing it now.

On the other hand, this very problem was the motivation for this patch
series here: https://issues.guix.gnu.org/43627, which would solve the
issue ta hand.  You were skeptical of the benefits the last time you
took a look at it; perhaps it's time to take a new look at it :-).

Thanks,

Maxim




^ permalink raw reply	[flat|nested] 7+ messages in thread

* [bug#45359] closed (Re: bug#47458: Terrible UX upgrading Emacs in Guix)
       [not found]       ` <handler.45359.D45359.161704236132600.notifdone@debbugs.gnu.org>
@ 2021-03-29 18:45         ` Maxim Cournoyer
  2021-03-29 18:48         ` bug#45359: [PATCH]: Re-introduce Emacs packages specific installation prefix Maxim Cournoyer
  1 sibling, 0 replies; 7+ messages in thread
From: Maxim Cournoyer @ 2021-03-29 18:45 UTC (permalink / raw)
  To: GNU Debbugs; +Cc: 45359

reopen 45359
thanks

Hi,

It seems I've closed that report by mistake.  Sorry!

Reopening.

Maxim




^ permalink raw reply	[flat|nested] 7+ messages in thread

* bug#45359: [PATCH]: Re-introduce Emacs packages specific installation prefix.
       [not found]       ` <handler.45359.D45359.161704236132600.notifdone@debbugs.gnu.org>
  2021-03-29 18:45         ` [bug#45359] closed (Re: bug#47458: Terrible UX upgrading Emacs in Guix) Maxim Cournoyer
@ 2021-03-29 18:48         ` Maxim Cournoyer
  1 sibling, 0 replies; 7+ messages in thread
From: Maxim Cournoyer @ 2021-03-29 18:48 UTC (permalink / raw)
  Cc: 45359-done

Hi,

help-debbugs@gnu.org (GNU bug Tracking System) writes:

> Your bug report
>
> #45359: [PATCH]: Re-introduce Emacs packages specific installation prefix.
>
> which was filed against the guix-patches package, has been closed.
>
> The explanation is attached below, along with your original report.
> If you require more details, please reply to 45359@debbugs.gnu.org.

Seems the reply title got me confused; this *is* the issue I want to
close.  Doing so again, with the title fixed this time.

Maxim




^ permalink raw reply	[flat|nested] 7+ messages in thread

* [bug#45359] bug#47458: Terrible UX upgrading Emacs in Guix
  2021-03-29 18:25   ` bug#45359: " Maxim Cournoyer
  2020-12-22  3:28     ` [bug#45359] [PATCH]: Re-introduce Emacs packages specific installation prefix Maxim Cournoyer
@ 2021-03-30  8:04     ` Ludovic Courtès
  1 sibling, 0 replies; 7+ messages in thread
From: Ludovic Courtès @ 2021-03-30  8:04 UTC (permalink / raw)
  To: Maxim Cournoyer; +Cc: Mark H Weaver, 45359-done, 47458

Hello,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> Ouch.  “It used to be” (speaking like a veteran :-)) that Emacs in Guix
>> would not use EMACSLOADPATH.  Then we switched to EMACSLOADPATH, which
>> had some advantages, but necessarily has this drawback.
>>
>> IIUC, <https://issues.guix.gnu.org/45359> is about possibly
>> backtracking.  Maxim, what’s the status of this one?
>
> It's abandoned, The MUMI tracker lacks the responses from Leo Prickler,
> but they had good arguments maintaining the status quo rather than going
> with the extra complexity.  It also wouldn't change the issue at hand;
> it'd merely prevent conflicts of *resources* files of Emacs packages
> (and somewhat integrate with the Emacs native package manager, while
> making the autoloading a bit slower).  It seems the price to pay is too
> high for such a small gain.  I'm closing it now.

I see, makes sense!

> On the other hand, this very problem was the motivation for this patch
> series here: https://issues.guix.gnu.org/43627, which would solve the
> issue ta hand.  You were skeptical of the benefits the last time you
> took a look at it; perhaps it's time to take a new look at it :-).

Ah!  Now I may have to revisit it, indeed.

Thanks for explaining!

Ludo’.




^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-03-30  8:05 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <87o8f23eew.fsf@netris.org>
2021-03-29 15:55 ` [bug#45359] bug#47458: Terrible UX upgrading Emacs in Guix Ludovic Courtès
2021-03-29 15:55 ` Ludovic Courtès
2021-03-29 18:25   ` bug#45359: " Maxim Cournoyer
2020-12-22  3:28     ` [bug#45359] [PATCH]: Re-introduce Emacs packages specific installation prefix Maxim Cournoyer
     [not found]       ` <handler.45359.D45359.161704236132600.notifdone@debbugs.gnu.org>
2021-03-29 18:45         ` [bug#45359] closed (Re: bug#47458: Terrible UX upgrading Emacs in Guix) Maxim Cournoyer
2021-03-29 18:48         ` bug#45359: [PATCH]: Re-introduce Emacs packages specific installation prefix Maxim Cournoyer
2021-03-30  8:04     ` [bug#45359] bug#47458: Terrible UX upgrading Emacs in Guix Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).