bug#67292: emacs / emacs-transient collisions and bundling

bug#67292: emacs / emacs-transient collisions and bundling

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1507 bytes --]

Hi,

I recently got the following error message from "guix home build" 
(misleading it says "warning" instead):

> warning: collision encountered:
>   /gnu/store/xjrhyapm3zwgpmq5baz6m9kavz287jjj-emacs-29.1/share/info/transient.info.gz
>   /gnu/store/lg6h2mkg5z15i78dzvzs0s89bgxxwnlf-emacs-transient-0.4.3-0.cc0fa80/share/info/transient.info.gz
> warning: choosing /gnu/store/xjrhyapm3zwgpmq5baz6m9kavz287jjj-emacs-29.1/share/info/transient.info.gz

Unfortunately, the following doesn't work as minimal reproducer, because 
"guix shell" (and likewise for "environment") suppresses collision error 
messages:

$ guix shell emacs emacs-transient

(see: https://issues.guix.gnu.org/54350).

(That's not what this bug report is about.)

Bug 1: emacs bundles emacs-transient.
(Likewise for quite a few other Emacs packages.)

Bug 2: no collision handling.

If I'm guessing correctly, emacs-transient is supposed to be a newer 
version of the transient bundled in emacs, overriding the bundled 
transient in emacs.  But for that to work properly, it needs to actually 
override the stuff, and going by the collision error message, it isn't 
actually overridden but rather the profile building code arbitrarily 
chooses between the two.

I guess (guix profiles) should set #:resolve-collision to something 
where the "[...]/transient.info.gz" of the store item containing 
"[...]/bin/emacs" loses.  (And likewise for the other bundled Emacs stuff.)

Best regards,
Maxime Devos

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#67292: emacs / emacs-transient collisions and bundling

[Simon Tournier]

Hi,

On Sun, 19 Nov 2023 at 23:44, Maxime Devos <maximedevos@telenet.be> wrote:

> Bug 1: emacs bundles emacs-transient.
> (Likewise for quite a few other Emacs packages.)

This is not a bug.  transient.el is part of Emacs since version 28, see
NEWS:

--8<---------------cut here---------------start------------->8---
* New Modes and Packages in Emacs 28.1

** transient.el
This library implements support for powerful keyboard-driven menus.
Such menus can be used as simple visual command dispatchers.  More
complex menus take advantage of infix arguments, which are somewhat
similar to prefix arguments, but are more flexible and discoverable.
--8<---------------cut here---------------end--------------->8---

When an user installs Emacs, they expect to have transient as builtin;
it is not bundled.  I recommend to read the section:

    * New Modes and Packages in Emacs X

in NEWS release files to know which mode or package are included as
builtin.  And yes, these packages are sometimes developed outside the
Emacs tree using a dedicated repository, as transient.el or Org-mode for
instance.


> Bug 2: no collision handling.

This is a bug, IMHO.  Indeed, if an user prefers another version of
builtin modes or packages, they should not collide with the builtin
ones.  Well, they do not collide when using “guix shell”:

    guix shell -C emacs emacs-transient -E TERM -- emacs -q

Therefore, maybe the issue comes from some specific code from “guix
home”.  Hum, I do not know.

Cheers,
simon

bug#67292: emacs / emacs-transient collisions and bundling

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 2796 bytes --]

Op 20-11-2023 om 11:22 schreef Simon Tournier:
> Hi,
> 
> On Sun, 19 Nov 2023 at 23:44, Maxime Devos <maximedevos@telenet.be> wrote:
> 
>> Bug 1: emacs bundles emacs-transient.
>> (Likewise for quite a few other Emacs packages.)
> 
> This is not a bug.  transient.el is part of Emacs since version 28, see
> NEWS:
> 
> --8<---------------cut here---------------start------------->8---
> * New Modes and Packages in Emacs 28.1
> 
> ** transient.el
> This library implements support for powerful keyboard-driven menus.
> Such menus can be used as simple visual command dispatchers.  More
> complex menus take advantage of infix arguments, which are somewhat
> similar to prefix arguments, but are more flexible and discoverable.
> --8<---------------cut here---------------end--------------->8---
> 
> When an user installs Emacs, they expect to have transient as builtin;
> it is not bundled.


This does not answer how the bundling is not a bug.

Unbundling is not incompatible with keeping builtins -- for example, 
emacs-minimal could be the ‘minimalistic Emacs, without the typical 
bundled ‘built-in’ stuff’, and emacs could be 'emacs-minimal + the 
unbundled stuff in propagated-inputs’.

(Well, preferably not propagated, but that's just a matter of wrapping 
or unions etc..)

Another option could be to unbundle things in the 'source' of emacs / 
emacs-minimal and copy over source code from the 'emacs-‘builtin’' 
packages during a build phase.

(IIUC, emacs-minimal is for compiling Emacs packages and for scripts, of 
which most probably have no need for keyboard-driven menus -- and if 
they do have need,)

>  I recommend to read the section:
> 
>      * New Modes and Packages in Emacs X
> 
> in NEWS release files to know which mode or package are included as
> builtin.

I won't -- my package manager is Guix, not Emacs.

> And yes, these packages are sometimes developed outside the
> Emacs tree using a dedicated repository, as transient.el or Org-mode for
> instance.
> 
> 
>> Bug 2: no collision handling.
> 
> This is a bug, IMHO.  Indeed, if an user prefers another version of
> builtin modes or packages, they should not collide with the builtin
> ones.  Well, they do not collide when using “guix shell”:
> 
>      guix shell -C emacs emacs-transient -E TERM -- emacs -q
> 
> Therefore, maybe the issue comes from some specific code from “guix
> home”.  Hum, I do not know.

They do, in fact, collide -- as I mentioned in the e-mail you replied to:

> [...] because 
> "guix shell" (and likewise for "environment") suppresses collision error 
> messages:
> 
> $ guix shell emacs emacs-transient
> 
> (see: https://issues.guix.gnu.org/54350).

Best regards,
Maxime Devos

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#67292: emacs / emacs-transient collisions and bundling

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 790 bytes --]

>>> Bug 2: no collision handling.
>>
>> This is a bug, IMHO.  Indeed, if an user prefers another version of
>> builtin modes or packages, they should not collide with the builtin
>> ones.  Well, they do not collide when using “guix shell”:
>>
>>      guix shell -C emacs emacs-transient -E TERM -- emacs -q
>>
>> Therefore, maybe the issue comes from some specific code from “guix
>> home”.  Hum, I do not know.
> 
> They do, in fact, collide -- as I mentioned in the e-mail you replied to:
> 
>> [...] because "guix shell" (and likewise for "environment") suppresses 
>> collision error messages:
>>
>> $ guix shell emacs emacs-transient
>>
>> (see: https://issues.guix.gnu.org/54350).

More info: (guix profiles) does not seem to handle this anywhere.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#67292: emacs / emacs-transient collisions and bundling

[Simon Tournier]

On Mon, 20 Nov 2023 at 13:39, Maxime Devos <maximedevos@telenet.be> wrote:

> This does not answer how the bundling is not a bug.

It answers.  Nothing is bundled.  It is the official Emacs.


> (IIUC, emacs-minimal is for compiling Emacs packages and for scripts, of
> which most probably have no need for keyboard-driven menus -- and if
> they do have need,)

Here, you are speaking about another thing.  Please open another bug
report for that: is emacs-minimal really minimal?  Or could some
builtin modes and packages be dropped because they are not required
for building Emacs packages?


All the best,
simon

bug#67292: emacs / emacs-transient collisions and bundling

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1135 bytes --]



Op 20-11-2023 om 14:46 schreef Simon Tournier:
> On Mon, 20 Nov 2023 at 13:39, Maxime Devos <maximedevos@telenet.be> wrote:
> 
>> This does not answer how the bundling is not a bug.
> 
> It answers.  Nothing is bundled.  It is the official Emacs.

‘Nothing is bundled’: on the contrary:

   * ‘New Modes and Packages in Emacs 28.1'
   * + a separate emacs-transient exists, and emacs-transient appears to
     be developed outside Emacs:
     <https://github.com/magit/transient>.

Yes, it is official, but the question was how the bundling is not a bug 
(and implicitly, whether it is bundling), not whether the bundling is 
official.
>> (IIUC, emacs-minimal is for compiling Emacs packages and for scripts, of
>> which most probably have no need for keyboard-driven menus -- and if
>> they do have need,)
> 
> Here, you are speaking about another thing.  Please open another bug
> report for that: is emacs-minimal really minimal?  Or could some
> builtin modes and packages be dropped because they are not required
> for building Emacs packages?

Not interested.

Best regards,
Maxime Devos.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#67292: emacs / emacs-transient collisions and bundling

[Simon Tournier]

On mar., 21 nov. 2023 at 19:01, Maxime Devos <maximedevos@telenet.be> wrote:

> Yes, it is official, but the question was how the bundling is not a bug (and
> implicitly, whether it is bundling), not whether the bundling is official.

The bundling is not a bug because it is how Emacs is developed.  The
term “bundle“ – which would potentially imply being unbundled by Guix
packagers – is misleading.

Emacs maintainers grant some packages and make them part of Emacs as
builtin package; whatever where these packages are developed or if these
packages follow another release schedule than the Emacs release
schedule.  The section “New Modes and Packages in Emacs X.Y” of the NEWS
file for each release (NEWS.28, NEWS,27, etc.) lists such promoted
packages.  Please note that Emacs 29 introduces a “New user option
'package-install-upgrade-built-in'”, as mentioned in NEWS.

For instance, the packages widget.el or woman.el or many others were
initially developed outside the Savannah Emacs tree, then integrated
(being promoted builtin), and now the original development location is
gone – which means all the maintenance burden for these builtin packages
is now done by Emacs maintainers.

Once builtin, the code of a package distributed with GNU Emacs is
maintained by Emacs maintainers and fully part of GNU Emacs.

As explicitly commented in the header of transient.el that comes with
Emacs:

--8<---------------cut here---------------start------------->8---
;;; transient.el --- Transient commands  -*- lexical-binding:t -*-

;; Copyright (C) 2018-2023 Free Software Foundation, Inc.

;; Author: Jonas Bernoulli <jonas@bernoul.li>
;; URL: https://github.com/magit/transient

[...]

;; This file is part of GNU Emacs.
--8<---------------cut here---------------end--------------->8---

The collision is a bug.  The report of bundled is not a bug.

Cheers,
simon

bug#67292: emacs / emacs-transient collisions and bundling

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 6645 bytes --]

Op 22-11-2023 om 21:53 schreef Simon Tournier:
> On mar., 21 nov. 2023 at 19:01, Maxime Devos <maximedevos@telenet.be> wrote:
> 
>> Yes, it is official, but the question was how the bundling is not a bug (and
>> implicitly, whether it is bundling), not whether the bundling is official.
> 
> The bundling is not a bug because it is how Emacs is developed.

That's what they all say.  What's do different about Emacs that we 
should accept this from Emacs and not from other bundlers?  Like, sure, 
there are some differences in how Emacs bundles things compared to 
others, but how do these differences matter?

> The
> term “bundle“ – which would potentially imply being unbundled by Guix
> packagers – is misleading.

What is misleading about the "term" bundle / misleading about 
potentially being unbundled by Guix packagers?

> Emacs maintainers grant some packages and make them part of Emacs as
> builtin package; whatever where these packages are developed or if these
> packages follow another release schedule than the Emacs release
> schedule.  The section “New Modes and Packages in Emacs X.Y” of the NEWS
> file for each release (NEWS.28, NEWS,27, etc.) lists such promoted
> packages.  Please note that Emacs 29 introduces a “New user option
> 'package-install-upgrade-built-in'”, as mentioned in NEWS.
> 
> For instance, the packages widget.el or woman.el or many others were
> initially developed outside the Savannah Emacs tree, then integrated
> (being promoted builtin), and now the original development location is
> gone – which means all the maintenance burden for these builtin packages
> is now done by Emacs maintainers.

Yes, Emacs integrates outside things, I've heard it already ...

If we're talking about misleading statements ...

For widget and woman, sure, I'll take your word for it is fully merged 
in Emacs (both the literal code & development (& maintenance)).  But I'm 
talking about emacs-transient (not widget or woman), and note, as I 
pointed out previously, emacs-transient development location appears to 
be <https://github.com/magit/transient> (which is not the Savannah Emacs 
Tree) -- there's even a new commit 8 hours ago, and it doesn't seem to 
be disappearing anytime soon.

If you have examples, please only use non-misleading examples ...

> Once builtin, the code of a package distributed with GNU Emacs is
> maintained by Emacs maintainers and fully part of GNU Emacs.

Yes, and?  How does being fully part of GNU Emacs and being maintained 
by Emacs maintainers make it any less bundling?  There is more to 
development than maintenance.

Even if we were to exclude situations like this from a definition of 
‘bundling’, that just changes the name of the thing, there still remain 
some benefits to what you aren't calling unbundling / downsides to what 
you aren't calling bundling.

> As explicitly commented in the header of transient.el that comes with
> Emacs:
> 
> --8<---------------cut here---------------start------------->8---
> ;;; transient.el --- Transient commands  -*- lexical-binding:t -*-
> 
> ;; Copyright (C) 2018-2023 Free Software Foundation, Inc.
> 
> ;; Author: Jonas Bernoulli <jonas@bernoul.li>
> ;; URL: https://github.com/magit/transient
> 
> [...]
> 
> ;; This file is part of GNU Emacs.
> --8<---------------cut here---------------end--------------->8---

Indeed, bundled dependencies are part of what it is bundled inside. 
And?  How does this extra comment matter w.r.t. whether it is bundling 
and whether the bundling is to be tolerated?

> The collision is a bug.  The report of bundled is not a bug.

If that's your opinion, please give you argument on how the bundling of 
emacs-transient, or whatever you want to call it instead of ‘bundling’, 
is not a bug.

--- (From: Mekeor Milere)

> It might be possible to cut out some parts of Emacs so that emacs-minimal is more minimal. But I think this could become quite complicated because we don't know exactly which parts of Emacs are needed to build Emacs-related packages since they might rely on any Elisp code during compile-time. Also, more generally, it'd be hard to decide which parts are not needed, i.e. where to draw the line etc. All in all, I don't think that it's worth the effort and maintenance. 

If making emacs-mnimal more minimal is too complicated, don't do it 
then, just replace the bundled copy with an up-to-date (source) version, 
as I proposed previously.

Also, I don't think I proposed minimalising Emacs to only what's needed 
to build Emacs-related packages (though I could easily be 
misremembering) -- IIRC, I only proposed minimalising it to the extent 
of excluding the bundled Emacs packages.

> TANGENTIALLY: I'd like to mention that this topic becomes sort of a problem when (1) you have installed Emacs with "guix install emacs-next --with-branch=emacs-next=master" or similar; and (2) you installed some Emacs-related package via Guix, which propagates another Emacs-related package that is also built into Emacs. This would cause a downgrade of that propagated, built-in, Emacs-related package. E.g. this happens with emacs-consult-eglot which propagates emacs-eglot which is also built into Emacs itself. A work-around is to overwrite the input like this: "guix install emacs-next emacs-consult-eglot --with-input=emacs-eglot=emacs-next --with-branch=emacs-next=master". 

No?  Unless you do "--with-branch=emacs-next=something-old" or the like, 
you will never get a downgrade -- you will not get an automatic upgrade 
to what's bundled in Emacs-next, but:

  (a) you won't get anything older than what is currently packaged in
      Guix. (Hence, not a downgrade.)
  (b) and you asked for a latest emacs, not a latest emacs-eglot.

You might even get something newer than in the master branch of Emacs, 
if the Emacs maintainers haven't merged in the latest version yet.

Also, I don't get what "--with-input=emacs-eglot=emacs-next" is supposed 
to do.  Like, good for AOT, but this is not a bug report about AOT and 
AFAIK Emacs has automatic recompilation.

> If you want to know which built-in packages are distributed separately via GNU Elpa, search the following file for ":core". Note that only a subset of those might be packaged separately in Guix.
> 
>     https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/elpa-packages 

I don't want to know.  I use Guix as package manager, not Emacs -- I 
don't really care whether a hypothetical package is distributed via Elpa.

Best regards,
Maxime Devos.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#67292: emacs / emacs-transient collisions and bundling

[Simon Tournier]

On Tue, 28 Nov 2023 at 02:47, Maxime Devos <maximedevos@telenet.be> wrote:

>> Once builtin, the code of a package distributed with GNU Emacs is
>> maintained by Emacs maintainers and fully part of GNU Emacs.
>
> Yes, and?  How does being fully part of GNU Emacs and being maintained 
> by Emacs maintainers make it any less bundling?  There is more to 
> development than maintenance.

Please read GNU Emacs documentation and how GNU Emacs is developed.  I
have tried to do my best for pointing you some links.  If these are not
enough, you should dig by your own.  Maybe ask on emacs-devel mailing
list how the GNU Emacs development process works.

> If making emacs-mnimal more minimal is too complicated, don't do it 
> then, just replace the bundled copy with an up-to-date (source) version, 
> as I proposed previously.

This is *not* the GNU Emacs release model.  GNU Emacs version X.Y *is*
all the exact same files as the ones stored in Savannah.  If you want to
replace the file
/gnu/store/…-emacs-X.Y/share/emacs/X.Y/lisp/transient.el.gz by something
else, you need to package that exact file.  Therefore, your proposal
would imply to have two packages:

 + emacs-transient-next following development of transient.el – this
   development happens outside Savannah.  Please note for other builtin
   packages, this development happens inside the Emacs repository
   located in Savannah.

 + emacs-transient providing the exact same version as the builtin one
   tracked in the GNU Emacs release X.Y branch located in Savannah.

Again, transient.el is not bundled.  It is a builtin package which means
it is fully part of GNU Emacs.  Being developed outside Savannah does
not make it bundled.


Regards,
simon

bug#67292: emacs / emacs-transient collisions and bundling

[Mekeor Melire]

2023-11-28 02:47 maximedevos@telenet.be:
> Op 22-11-2023 om 21:53 schreef Simon Tournier:
> > On mar., 21 nov. 2023 at 19:01, Maxime Devos 
> > <maximedevos@telenet.be> wrote:

> For widget and woman, sure, I'll take your word for it is fully 
> merged
> in Emacs (both the literal code & development (& maintenance)). 
> But
> I'm talking about emacs-transient (not widget or woman), and 
> note, as
> I pointed out previously, emacs-transient development location 
> appears
> to be <https://github.com/magit/transient> (which is not the 
> Savannah
> Emacs Tree) -- there's even a new commit 8 hours ago, and it 
> doesn't
> seem to be disappearing anytime soon.

Note that transient.el from Emacs-master and from github.com/magit/transient actually have diverged. In particular, the following commit is present on Emacs-master while it is not present on magit/transient: https://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/transient.el?id=7705bdfa5b89f78dab049f73f636b9680a3c12bc

The relation between the possibly-existing separate repository of an Emacs-core-package (e.g. github.com/magit/transient) to the Emacs-repository itself seems to depend on the concrete package. As we just saw, for transient.el, that its version in the separate repository and in the Emacs-repository may temporarily diverge. Tarsius, the transient-maintainer, merges changes into Emacs after version bumps.

Another example is eglot.el. It also has a separate repository but that repository is only occasionally synced with the more progressed version that is tracked in the Emacs repository itself.

> --- (From: Mekeor Milere)

It's "Melire". Why did you respond to my mail in a separate thread?

> If making emacs-mnimal more minimal is too complicated, don't do 
> it
> then, just replace the bundled copy with an up-to-date (source)
> version, as I proposed previously.

Replacing the "bundled copy" could erase fixes, that are only present in Emacs, and not present in github.com/magit/transient.

> > TANGENTIALLY: I'd like to mention that this topic becomes sort 
> > of a
> > problem when (1) you have installed Emacs with "guix install
> > emacs-next --with-branch=emacs-next=master" or similar; and 
> > (2) you
> > installed some Emacs-related package via Guix, which 
> > propagates
> > another Emacs-related package that is also built into Emacs. 
> > This
> > would cause a downgrade of that propagated, built-in, 
> > Emacs-related
> > package. E.g. this happens with emacs-consult-eglot which 
> > propagates
> > emacs-eglot which is also built into Emacs itself. A 
> > work-around is
> > to overwrite the input like this: "guix install emacs-next
> > emacs-consult-eglot --with-input=emacs-eglot=emacs-next
> > --with-branch=emacs-next=master".
>
> No?  Unless you do "--with-branch=emacs-next=something-old" or 
> the
> like, you will never get a downgrade -- you will not get an 
> automatic
> upgrade to what's bundled in Emacs-next, but:
>
>  (a) you won't get anything older than what is currently 
>  packaged in
>      Guix. (Hence, not a downgrade.)
>  (b) and you asked for a latest emacs, not a latest emacs-eglot.
>
> You might even get something newer than in the master branch of 
> Emacs,
> if the Emacs maintainers haven't merged in the latest version 
> yet.

As elaborated before, some packages (like Eglot) are primarily developed in the Emacs repository. (Unlike Transient.)

> Also, I don't get what "--with-input=emacs-eglot=emacs-next" is
> supposed to do.

It effectively removes emacs-eglot as input for the installed packages and their dependencies

> > If you want to know which built-in packages are distributed
> > separately via GNU Elpa, search the following file for 
> > ":core". Note
> > that only a subset of those might be packaged separately in 
> > Guix.
> >     https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/elpa-packages
>
> I don't want to know.  I use Guix as package manager, not Emacs 
> -- I
> don't really care whether a hypothetical package is distributed 
> via
> Elpa.

The point is that Guix downloads the source code of (some?) Elpa-distributed packages from Elpa. For example: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/emacs-xyz.scm?h=60c97924e9519361494aaf0686e28eb831a42315#n1011

bug#67292: emacs / emacs-transient collisions and bundling

[Mekeor Melire]

Hello Maxime,

2023-11-20 13:39 maximedevos@telenet.be:

> Op 20-11-2023 om 11:22 schreef Simon Tournier:

> > On Sun, 19 Nov 2023 at 23:44, Maxime Devos 
> > <maximedevos@telenet.be>
> > wrote:

> >> Bug 1: emacs bundles emacs-transient.
> >> (Likewise for quite a few other Emacs packages.)

> > This is not a bug.  transient.el is part of Emacs since 
> > version 28,

> This does not answer how the bundling is not a bug.
>
> Unbundling is not incompatible with keeping builtins -- for 
> example,
> emacs-minimal could be the ‘minimalistic Emacs, without the 
> typical
> bundled ‘built-in’ stuff’, and emacs could be 'emacs-minimal + 
> the
> unbundled stuff in propagated-inputs’.
>
> (Well, preferably not propagated, but that's just a matter of 
> wrapping
> or unions etc..)
>
> Another option could be to unbundle things in the 'source' of 
> emacs /
> emacs-minimal and copy over source code from the 
> 'emacs-‘builtin’'
> packages during a build phase.
>
> (IIUC, emacs-minimal is for compiling Emacs packages and for 
> scripts,
> of which most probably have no need for keyboard-driven menus -- 
> and
> if they do have need,)

It might be possible to cut out some parts of Emacs so that emacs-minimal is more minimal. But I think this could become quite complicated because we don't know exactly which parts of Emacs are needed to build Emacs-related packages since they might rely on any Elisp code during compile-time. Also, more generally, it'd be hard to decide which parts are not needed, i.e. where to draw the line etc. All in all, I don't think that it's worth the effort and maintenance.

> >> Bug 2: no collision handling.

As far as I know, Guix does not provide a facility to prioritize a 
file from one package over the same file from another package.

But also, I don't think this is necessary here, because in a Guix-installed Emacs, the paths to Guix-installed Emacs-related packages appear first in Emacs' load-path variable, whereas the paths to the elisp-directories from Emacs itself come last. Thus, emacs-transient's transient.el is always preferred over Emacs' transient.el.

Finally, here's a list an incomplete list of Emacs-related packages that are both built into Emacs itself, as well as distributed separately over GNU Elpa and thus also as separate packages in Guix:

    emacs-eglot
    emacs-eldoc
    emacs-external-completion
    emacs-jsonrpc
    emacs-let-alist
    emacs-map
    emacs-ntlm
    emacs-project
    emacs-so-long
    emacs-soap-client
    emacs-use-package
    emacs-xref

If you want to know which built-in packages are distributed separately via GNU Elpa, search the following file for ":core". Note that only a subset of those might be packaged separately in Guix.

	https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/elpa-packages

TANGENTIALLY: I'd like to mention that this topic becomes sort of a problem when (1) you have installed Emacs with "guix install emacs-next --with-branch=emacs-next=master" or similar; and (2) you installed some Emacs-related package via Guix, which propagates another Emacs-related package that is also built into Emacs. This would cause a downgrade of that propagated, built-in, Emacs-related package. E.g. this happens with emacs-consult-eglot which propagates emacs-eglot which is also built into Emacs itself. A work-around is to overwrite the input like this: "guix install emacs-next emacs-consult-eglot --with-input=emacs-eglot=emacs-next --with-branch=emacs-next=master".

bug#67292: emacs / emacs-transient collisions and bundling

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 983 bytes --]

Op 23-11-2023 om 15:17 schreef Mekeor Melire:
>> >> Bug 2: no collision handling.
> 
> As far as I know, Guix does not provide a facility to prioritize a 
> file from one package over the same file from another package.

As I mentioned in a previous e-mail in #67292, it does provide a 
facility -- not mentioned: by default, the first package in the list is 
prioritized.

> But also, I don't think this is necessary here, because in a Guix-installed Emacs, the paths to Guix-installed Emacs-related packages appear first in Emacs' load-path variable, whereas the paths to the elisp-directories from Emacs itself come last. Thus, emacs-transient's transient.el is always preferred over Emacs' transient.el.

If the collision is not a problem then Guix shouldn't be warning about 
it -- as such, the collision handling needs to be adjusted to not emit a 
warning in this case.

Best regards,
Maxime Devos

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#67292: emacs / emacs-transient collisions and bundling

[Mekeor Melire]

2023-11-28 01:58 maximedevos@telenet.be:

> [[PGP Signed Part:Undecided]]
> Op 23-11-2023 om 15:17 schreef Mekeor Melire:
> >> >> Bug 2: no collision handling.
> > As far as I know, Guix does not provide a facility to 
> > prioritize a
> > file from one package over the same file from another package.
>
> As I mentioned in a previous e-mail in #67292, it does provide a
> facility -- not mentioned: by default, the first package in the 
> list
> is prioritized.

Sorry for the oversight. Which list are you referring to?

> If the collision is not a problem then Guix shouldn't be warning 
> about
> it -- as such, the collision handling needs to be adjusted to 
> not emit
> a warning in this case.

True. That'd be nice. Should we discuss this here, in this bug-report or in another, new bug-report that is dedicated to the problem of unnecessary collision-warnings?

bug#67292: [PATCH] gnu: emacs: Pin natively compiled packages.

[Liliana Marie Prikler]

* gnu/packages/patches/emacs-native-comp-pin-packages.patch: New patch.
* gnu/local.mk (dist_patch_DATA): Register it here.
* gnu/packages/emacs.scm (emacs)[source]: Use it here.
[#:phases]: Remove ‘disable-native-compilation’.

Fixes: Emacs native-comp collisions <https://issues.guix.gnu.org/67292>
---
Hi Guix

I finally got around to fixing this issue.  The fix is actually quite simple:
it checks that the candidate eln file resides under the same prefix as the elc
file it was generated from.  With this change applied

* `guix shell emacs` will allow emacs to load its own natively-compiled
  transient, assuming a previously clean environment
* `guix shell emacs emacs-transient` will instead load the byte-compiled
  transient from emacs-transient
* `guix shell emacs emacs-transient --with-input=emacs-minimal=emacs` will
  load the native-compiled emacs

We still have to find a better way to enable native compilation for everything,
but this will give us the best performance possible while avoiding similar
issues in the future (side-eyeing org-mode, cc-mode et al. for possible
breakages).

Cheers

 gnu/local.mk                                  |  1 +
 gnu/packages/emacs.scm                        | 11 +-----
 .../emacs-native-comp-pin-packages.patch      | 39 +++++++++++++++++++
 3 files changed, 41 insertions(+), 10 deletions(-)
 create mode 100644 gnu/packages/patches/emacs-native-comp-pin-packages.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index a02745d59c..652a6c1748 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1195,6 +1195,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/emacs-lispy-fix-thread-last-test.patch   \
   %D%/packages/patches/emacs-native-comp-driver-options.patch   \
   %D%/packages/patches/emacs-native-comp-fix-filenames.patch   \
+  %D%/packages/patches/emacs-native-comp-pin-packages.patch   \
   %D%/packages/patches/emacs-next-exec-path.patch   \
   %D%/packages/patches/emacs-next-native-comp-driver-options.patch   \
   %D%/packages/patches/emacs-pasp-mode-quote-file-names.patch  \
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 8140f61882..b0690fc4fb 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -117,6 +117,7 @@ (define-public emacs-minimal
                                        "emacs-fix-scheme-indent-function.patch"
                                        "emacs-native-comp-driver-options.patch"
                                        "emacs-native-comp-fix-filenames.patch"
+                                       "emacs-native-comp-pin-packages.patch"
                                        "emacs-pgtk-super-key-fix.patch"))
               (modules '((guix build utils)))
               (snippet
@@ -233,16 +234,6 @@ (define-public emacs-minimal
                 (("\\(tramp-compat-process-running-p \"(.*)\"\\)" all process)
                  (format #f "(or ~a (tramp-compat-process-running-p ~s))"
                          all (string-append "." process "-real"))))))
-          (add-after 'unpack 'disable-native-compilation
-            (lambda _
-              ;; Temporary workaround to prevent the behaviour discussed in
-              ;; <https://issues.guix.gnu.org/72333>.
-              ;; Please remove once the native-compilation for Emacs packages
-              ;; is fully supported.
-              (substitute* "lisp/transient.el"
-                ((";; End:")
-                 ";; no-native-compile: t
-;; End:"))))
           (add-before 'configure 'fix-/bin/pwd
             (lambda _
               ;; Use `pwd', not `/bin/pwd'.
diff --git a/gnu/packages/patches/emacs-native-comp-pin-packages.patch b/gnu/packages/patches/emacs-native-comp-pin-packages.patch
new file mode 100644
index 0000000000..dc328f03a8
--- /dev/null
+++ b/gnu/packages/patches/emacs-native-comp-pin-packages.patch
@@ -0,0 +1,39 @@
+Index: emacs-29.4/src/lread.c
+===================================================================
+--- emacs-29.4.orig/src/lread.c
++++ emacs-29.4/src/lread.c
+@@ -1668,9 +1668,34 @@ directories, make sure the PREDICATE fun
+ 
+ #ifdef HAVE_NATIVE_COMP
+ static bool
++permit_swap_for_eln (Lisp_Object src_name, Lisp_Object eln_name)
++{
++  char *src = SSDATA (src_name), *eln = SSDATA (eln_name);
++  size_t eln_ln = strlen (eln);
++
++  while (*src && *eln && *src == *eln)
++    {
++      ++src; ++eln; --eln_ln;
++    }
++
++  /* After stripping common prefixes, the first directory should be
++   * "lib/" (inside the Guix store) or "native-lisp" (inside Emacs build).
++   * Alternatively, if eln contains "eln-cache", it's likely the user's
++   * cache, which we will also permit. */
++
++  return
++    (eln_ln > 4 && !strncmp (eln, "lib/", 4)) ||
++    (eln_ln > 12 && !strncmp (eln, "native-lisp/", 12)) ||
++    strstr (eln, "eln-cache") != NULL;
++}
++
++static bool
+ maybe_swap_for_eln1 (Lisp_Object src_name, Lisp_Object eln_name,
+ 		     Lisp_Object *filename, int *fd, struct timespec mtime)
+ {
++  if (!permit_swap_for_eln (src_name, eln_name))
++    return false;
++
+   struct stat eln_st;
+   int eln_fd = emacs_open (SSDATA (ENCODE_FILE (eln_name)), O_RDONLY, 0);
+ 

base-commit: 17d86ac9df841deca4f2f5e173f50bd2461fe680
-- 
2.46.0

bug#67292: [PATCH] gnu: emacs: Pin natively compiled packages.

[Andrew Tropin via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 5731 bytes --]

On 2024-12-08 18:30, Liliana Marie Prikler wrote:

> * gnu/packages/patches/emacs-native-comp-pin-packages.patch: New patch.
> * gnu/local.mk (dist_patch_DATA): Register it here.
> * gnu/packages/emacs.scm (emacs)[source]: Use it here.
> [#:phases]: Remove ‘disable-native-compilation’.
>
> Fixes: Emacs native-comp collisions <https://issues.guix.gnu.org/67292>
> ---
> Hi Guix
>
> I finally got around to fixing this issue.  The fix is actually quite simple:
> it checks that the candidate eln file resides under the same prefix as the elc
> file it was generated from.  With this change applied
>
> * `guix shell emacs` will allow emacs to load its own natively-compiled
>   transient, assuming a previously clean environment
> * `guix shell emacs emacs-transient` will instead load the byte-compiled
>   transient from emacs-transient
> * `guix shell emacs emacs-transient --with-input=emacs-minimal=emacs` will
>   load the native-compiled emacs
>
> We still have to find a better way to enable native compilation for everything,
> but this will give us the best performance possible while avoiding similar
> issues in the future (side-eyeing org-mode, cc-mode et al. for possible
> breakages).
>
> Cheers
>
>  gnu/local.mk                                  |  1 +
>  gnu/packages/emacs.scm                        | 11 +-----
>  .../emacs-native-comp-pin-packages.patch      | 39 +++++++++++++++++++
>  3 files changed, 41 insertions(+), 10 deletions(-)
>  create mode 100644 gnu/packages/patches/emacs-native-comp-pin-packages.patch
>
> diff --git a/gnu/local.mk b/gnu/local.mk
> index a02745d59c..652a6c1748 100644
> --- a/gnu/local.mk
> +++ b/gnu/local.mk
> @@ -1195,6 +1195,7 @@ dist_patch_DATA =						\
>    %D%/packages/patches/emacs-lispy-fix-thread-last-test.patch   \
>    %D%/packages/patches/emacs-native-comp-driver-options.patch   \
>    %D%/packages/patches/emacs-native-comp-fix-filenames.patch   \
> +  %D%/packages/patches/emacs-native-comp-pin-packages.patch   \
>    %D%/packages/patches/emacs-next-exec-path.patch   \
>    %D%/packages/patches/emacs-next-native-comp-driver-options.patch   \
>    %D%/packages/patches/emacs-pasp-mode-quote-file-names.patch  \
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 8140f61882..b0690fc4fb 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -117,6 +117,7 @@ (define-public emacs-minimal
>                                         "emacs-fix-scheme-indent-function.patch"
>                                         "emacs-native-comp-driver-options.patch"
>                                         "emacs-native-comp-fix-filenames.patch"
> +                                       "emacs-native-comp-pin-packages.patch"
>                                         "emacs-pgtk-super-key-fix.patch"))
>                (modules '((guix build utils)))
>                (snippet
> @@ -233,16 +234,6 @@ (define-public emacs-minimal
>                  (("\\(tramp-compat-process-running-p \"(.*)\"\\)" all process)
>                   (format #f "(or ~a (tramp-compat-process-running-p ~s))"
>                           all (string-append "." process "-real"))))))
> -          (add-after 'unpack 'disable-native-compilation
> -            (lambda _
> -              ;; Temporary workaround to prevent the behaviour discussed in
> -              ;; <https://issues.guix.gnu.org/72333>.
> -              ;; Please remove once the native-compilation for Emacs packages
> -              ;; is fully supported.
> -              (substitute* "lisp/transient.el"
> -                ((";; End:")
> -                 ";; no-native-compile: t
> -;; End:"))))
>            (add-before 'configure 'fix-/bin/pwd
>              (lambda _
>                ;; Use `pwd', not `/bin/pwd'.
> diff --git a/gnu/packages/patches/emacs-native-comp-pin-packages.patch b/gnu/packages/patches/emacs-native-comp-pin-packages.patch
> new file mode 100644
> index 0000000000..dc328f03a8
> --- /dev/null
> +++ b/gnu/packages/patches/emacs-native-comp-pin-packages.patch
> @@ -0,0 +1,39 @@
> +Index: emacs-29.4/src/lread.c
> +===================================================================
> +--- emacs-29.4.orig/src/lread.c
> ++++ emacs-29.4/src/lread.c
> +@@ -1668,9 +1668,34 @@ directories, make sure the PREDICATE fun
> + 
> + #ifdef HAVE_NATIVE_COMP
> + static bool
> ++permit_swap_for_eln (Lisp_Object src_name, Lisp_Object eln_name)
> ++{
> ++  char *src = SSDATA (src_name), *eln = SSDATA (eln_name);
> ++  size_t eln_ln = strlen (eln);
> ++
> ++  while (*src && *eln && *src == *eln)
> ++    {
> ++      ++src; ++eln; --eln_ln;
> ++    }
> ++
> ++  /* After stripping common prefixes, the first directory should be
> ++   * "lib/" (inside the Guix store) or "native-lisp" (inside Emacs build).
> ++   * Alternatively, if eln contains "eln-cache", it's likely the user's
> ++   * cache, which we will also permit. */
> ++
> ++  return
> ++    (eln_ln > 4 && !strncmp (eln, "lib/", 4)) ||
> ++    (eln_ln > 12 && !strncmp (eln, "native-lisp/", 12)) ||
> ++    strstr (eln, "eln-cache") != NULL;
> ++}
> ++
> ++static bool
> + maybe_swap_for_eln1 (Lisp_Object src_name, Lisp_Object eln_name,
> + 		     Lisp_Object *filename, int *fd, struct timespec mtime)
> + {
> ++  if (!permit_swap_for_eln (src_name, eln_name))
> ++    return false;
> ++
> +   struct stat eln_st;
> +   int eln_fd = emacs_open (SSDATA (ENCODE_FILE (eln_name)), O_RDONLY, 0);
> + 
>
> base-commit: 17d86ac9df841deca4f2f5e173f50bd2461fe680

Hi Liliana!

I didn't try this patch, but the approach looks reasonable.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#67292: emacs / emacs-transient collisions and bundling

[Suhail Singh]

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> * `guix shell emacs emacs-transient --with-input=emacs-minimal=emacs` will
>   load the native-compiled emacs

Hm there may be a bug somewhere (hopefully not in my testing).

I created a profile using:

#+begin_src sh
  ./pre-inst-env guix package --profile=${HOME}/.guix-extra-profiles/test-67292/test-67292 -i bash coreutils-minimal nss-certs emacs emacs-org emacs-magit emacs-transient --with-input=emacs-minimal=emacs
#+end_src

And then loaded it via:

#+begin_src sh
  guix shell --pure -CPWN -E '.*GTK.*|.*XDG.*|.*DISPLAY.*|TERM|INSIDE_EMACS' -p ${HOME}/.guix-extra-profiles/test-67292/test-67292
#+end_src

After invoking "emacs -nw -q" in the above profile, I evaluated the
below code:

#+begin_src emacs-lisp
  (eval-and-compile
      (require 'help-fns)
      (defmacro utils/report-compilation-status (fun &optional feature)
        "Report on the compilation status of function FUN.
    Optionally load FEATURE before reporting on compilation status."
        `(progn
           (eval-when-compile
             (when ',feature
               (require ',feature)))
           (let ((descstr (substring-no-properties
                           (with-output-to-string
                             (help-fns-function-description-header ',fun)))))
             (cons ',fun
                   (cond
                    ((string-search " native-compiled" descstr) 'native-compiled)
                    ;; ((string-search " autoloaded" descstr) descstr)
                    ((string-search " byte-compiled" descstr) 'byte-compiled)
                    (t descstr))))))

      (defun utils/report-compilation-status/apply (fun &optional feature)
        "Invoke `utils/report-compilation-status' with FUN and FEATURE."
        (eval `(utils/report-compilation-status ,fun ,feature))))
#+end_src

Then, evaluating the below:

#+begin_src emacs-lisp
  (utils/report-compilation-status/apply 'org-agenda 'org-agenda)
#+end_src

Results in '(org-agenda . byte-compiled).

Note, that doing the above without the submitted patch applied results
in '(org-agenda . native-compiled).

-- 
Suhail

bug#67292: emacs / emacs-transient collisions and bundling

[Liliana Marie Prikler]

Am Dienstag, dem 17.12.2024 um 15:10 -0500 schrieb Suhail Singh:
> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> 
> > * `guix shell emacs emacs-transient --with-input=emacs-
> > minimal=emacs` will
> >   load the native-compiled emacs
> 
> Hm there may be a bug somewhere (hopefully not in my testing).
With some printf-debugging, it appears as though profiles aren't
correctly resolved:
  /gnu/store/pm8rjqqkpkiyc7b2hrigw7kgg0z7c8hf-emacs-dash-2.19.1/share/emacs/site-lisp/dash-2.19.1/dash.el → 
  /gnu/store/f5wsraz8qp499g2sc9r5ax5p52w622lm-profile/lib/emacs/native-site-lisp/29.4-a928e9cc/dash.eln

We should sanitize native-comp-load-path like we do for regular load-path.

Cheers