Why not include all ELPA packages in an Emacs release?

Why not include all ELPA packages in an Emacs release?

[Jeremy Bryant]

Would there be a problem in including a snapshot of ELPA packages in an
Emacs release?

There are at least 2 scenarios where there are benefits to
having packages, in a functional albeit perhaps not the most recent version,
in a distribution of Emacs.


1.
"ELPA is Emacs" so new users could have access to the packages given
license requirements are the same.
Example - New LaTeX users can discover and try auctex straightaway.  If they want
to upgrade, they also can through ELPA.
(also e.g. related case of org-mode)

2.
Some sites don't allow downloading of packages (for policy or security
reasons), so all users have access only to whatever is included in their
Emacs distribution (and indeed their system distro).


Perhaps this is a naive view.  Thoughts welcome.

Re: Why not include all ELPA packages in an Emacs release?

[Stefan Kangas]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Would there be a problem in including a snapshot of ELPA packages in an
> Emacs release?

This has been discussed several times in the past.  There are no
principal problems, and AFAIU the consensus is that we would like to
do it, though there are some practical issues to work out.

I believe if you search the list archives, you will find previous
discussions that will shed some light on the issues involved.  We can't
just dump all packages in lisp/ and be done with it, unfortunately.

I honestly can't cite all of the problems off the top of my head.  One
is how we track developments on GNU ELPA, if we mirror packages into
emacs.git, or if we add them as submodules, or what.  At the very least,
we'd probably need to have some record of what we put into the release
tarball.  Do we add _all_ packages or just some subset?  Should they be
registered in `package--builtins´, and how?  Should they be part of the
main release tarball, or released separately?  And so on.

Mostly, we need an interested volunteer to step up and do the work.
This would probably start with reviewing past discussions to find out
what problems they uncovered.  Then propose, in practical terms, how we
could solve them.

So help is both welcome and needed here, I think.

Re: Why not include all ELPA packages in an Emacs release?

[Stefan Monnier]

> Mostly, we need an interested volunteer to step up and do the work.
> This would probably start with reviewing past discussions to find out
> what problems they uncovered.  Then propose, in practical terms, how we
> could solve them.

I don't think it's sufficient, because we've had that.  See

    feature/core-elpa-by-copy
and
    feature/integrated-elpa

AFAICT the problem is that there are some tradeoffs to make, and some
consequences which won't please everyone, so it needs very clear and
explicit support from the head maintainers.  Hell, I think this won't
happen until it's done *by* one of the head maintainers.


        Stefan

Re: Why not include all ELPA packages in an Emacs release?

[Philip Kaludercic]

Stefan Kangas <stefankangas@gmail.com> writes:

> Jeremy Bryant <jb@jeremybryant.net> writes:
>
>> Would there be a problem in including a snapshot of ELPA packages in an
>> Emacs release?
>
> This has been discussed several times in the past.  There are no
> principal problems, and AFAIU the consensus is that we would like to
> do it, though there are some practical issues to work out.
>
> I believe if you search the list archives, you will find previous
> discussions that will shed some light on the issues involved.  We can't
> just dump all packages in lisp/ and be done with it, unfortunately.
>
> I honestly can't cite all of the problems off the top of my head.  

I have bookmarked this message by Eli, where he links to a few threads
with issues that should be dealt with: <83zg27emh8.fsf@gnu.org>

>                                                                    One
> is how we track developments on GNU ELPA, if we mirror packages into
> emacs.git, or if we add them as submodules, or what.  At the very least,
> we'd probably need to have some record of what we put into the release
> tarball.  Do we add _all_ packages or just some subset?  Should they be
> registered in `package--builtins´, and how?  Should they be part of the
> main release tarball, or released separately?  And so on.
>
> Mostly, we need an interested volunteer to step up and do the work.
> This would probably start with reviewing past discussions to find out
> what problems they uncovered.  Then propose, in practical terms, how we
> could solve them.
>
> So help is both welcome and needed here, I think.

-- 
	Philip Kaludercic on peregrine

Re: Why not include all ELPA packages in an Emacs release?

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Tue, 28 May 2024 23:15:49 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>  Philip Kaludercic <philipk@posteo.net>
> 
> Jeremy Bryant <jb@jeremybryant.net> writes:
> 
> > Would there be a problem in including a snapshot of ELPA packages in an
> > Emacs release?
> 
> This has been discussed several times in the past.  There are no
> principal problems, and AFAIU the consensus is that we would like to
> do it, though there are some practical issues to work out.

Actually, this particular suggestion was never on the table.  At least
I cannot remember it being on the table.

What we discussed was the desire to move packages out of core in a way
that a release tarball would include those package.  That is what we
have a consensus about (but not a complete and satisfactory solution
yet).

Back to the suggestion in this thread: I cannot really see why would
we want to add almost 500 packages to Emacs.  Especially since AFAIR
some of them solve the same or very similar problems in different,
sometimes even contradictory, ways.

If there are packages on ELPA which we consider to be a must for users
(I don't think there are, but maybe I'm forgetting something), lets
add them to core instead.

> So help is both welcome and needed here, I think.

That's definitely true.

Re: Why not include all ELPA packages in an Emacs release?

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> If there are packages on ELPA which we consider to be a must for users
> (I don't think there are, but maybe I'm forgetting something), lets
> add them to core instead.

If Emacs considers in-buffer completion an important feature, then I'd
say corfu and cape are must.  vertico and marginalia are also must in my
book since they offer a better experience with vertical minibuffer
completion.

And while we're at it: There are sometimes requests for adding AUCTeX to
core.  Do you have an opinion about that?

Best, Arash

Re: Why not include all ELPA packages in an Emacs release?

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: Stefan Kangas <stefankangas@gmail.com>,  jb@jeremybryant.net,
>  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,  philipk@posteo.net
> Date: Wed, 29 May 2024 17:27:31 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > If there are packages on ELPA which we consider to be a must for users
> > (I don't think there are, but maybe I'm forgetting something), lets
> > add them to core instead.
> 
> If Emacs considers in-buffer completion an important feature, then I'd
> say corfu and cape are must.  vertico and marginalia are also must in my
> book since they offer a better experience with vertical minibuffer
> completion.

If people want them, and their developers agree, we can add them.

> And while we're at it: There are sometimes requests for adding AUCTeX to
> core.  Do you have an opinion about that?

I don't mind.  But let's hear what others think.

Re: Why not include all ELPA packages in an Emacs release?

[Philip Kaludercic]

Arash Esbati <arash@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> If there are packages on ELPA which we consider to be a must for users
>> (I don't think there are, but maybe I'm forgetting something), lets
>> add them to core instead.
>
> If Emacs considers in-buffer completion an important feature, then I'd
> say corfu and cape are must.  

I am not familiar with cape, but IIRC the issue with corfu is that
without any further changes, it doesn't support non-graphical Emacs,
right?

>                               vertico and marginalia are also must in my
> book since they offer a better experience with vertical minibuffer
> completion.

"Better", in this case, than `fido-vertical-mode'?

> And while we're at it: There are sometimes requests for adding AUCTeX to
> core.  Do you have an opinion about that?

I would certainly be a fan of that move.

> Best, Arash

-- 
	Philip Kaludercic on peregrine

Re: Why not include all ELPA packages in an Emacs release?

[Andrea Corallo]

Philip Kaludercic <philipk@posteo.net> writes:

> Arash Esbati <arash@gnu.org> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>> If there are packages on ELPA which we consider to be a must for users
>>> (I don't think there are, but maybe I'm forgetting something), lets
>>> add them to core instead.
>>
>> If Emacs considers in-buffer completion an important feature, then I'd
>> say corfu and cape are must.  
>
> I am not familiar with cape, but IIRC the issue with corfu is that
> without any further changes, it doesn't support non-graphical Emacs,
> right?
>
>>                               vertico and marginalia are also must in my
>> book since they offer a better experience with vertical minibuffer
>> completion.
>
> "Better", in this case, than `fido-vertical-mode'?
>
>> And while we're at it: There are sometimes requests for adding AUCTeX to
>> core.  Do you have an opinion about that?
>
> I would certainly be a fan of that move.

I support AUCTeX inclusion as well.

  Andrea

Re: Why not include all ELPA packages in an Emacs release?

[Tassilo Horn]

Arash Esbati <arash@gnu.org> writes:

>> If there are packages on ELPA which we consider to be a must for
>> users (I don't think there are, but maybe I'm forgetting something),
>> lets add them to core instead.
>
> If Emacs considers in-buffer completion an important feature, then I'd
> say corfu and cape are must.  vertico and marginalia are also must in
> my book since they offer a better experience with vertical minibuffer
> completion.

Nice completion UIs are certainly a very important feature and I have
the same preferences as you.  But I don't see a reason why some specific
one has to be in core.  It has long been ivy and company, now it's
vertico and corfu, and in 2 years the hottest bets might be something
else.

I mean, it's easy to put things into core but hardly possible to remove
them again without annoying someone.

FWIW, I'd rather move more stuff from core to ELPA and add mechanics to
install from ELPA easily.  use-package was one such thing but I imagine
a mechanism that would provide package suggestions, e.g., like asking a
user to install toml-mode when finding a toml file for the first time,
or suggesting to install calc or calculator when typing calc at the M-x
prompt...

> And while we're at it: There are sometimes requests for adding AUCTeX
> to core.  Do you have an opinion about that?

One practical question is what happens with stock tex-mode.el then.
Right now, when you install AUCTeX, it'll take over.  That would annoy
tex-mode users, I guess.

Also, I'm not sure if AUCTeX is not already past its prime time.
Preview-LaTeX is cool but IIRC doesn't work anymore with some popular
styles.  And document parsing in order to highlight user-defined
commands and offer them for completion is probably better done in some
LaTeX language server like texlab and then used with eglot or lsp-mode.

Bye,
  Tassilo

Re: Why not include all ELPA packages in an Emacs release?

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> What we discussed was the desire to move packages out of core in a way
> that a release tarball would include those package.  That is what we
> have a consensus about (but not a complete and satisfactory solution
> yet).

It's possible that we have interpreted the previous discussions a bit
differently, or that I'm filling in too many blanks.

My understanding was that this was always going to be something like
1) make it possible to ship ELPA packages with core Emacs and then
2) decide which packages should be included.

To my mind, step 2 could either be restricted to only include packages
that used to be in core, or it could not be.  The difference is not very
big, at least not in principle, since before we bundle even two
packages, we still have to solve the issue of how to bundle even one.

> Back to the suggestion in this thread: I cannot really see why would
> we want to add almost 500 packages to Emacs.  Especially since AFAIR
> some of them solve the same or very similar problems in different,
> sometimes even contradictory, ways.
>
> If there are packages on ELPA which we consider to be a must for users
> (I don't think there are, but maybe I'm forgetting something), lets
> add them to core instead.

Not sure if there are any that are a must, but I do think a few of them
would be good to have included.  For example, `json-mode` would be nice
to have available OOTB (though we now have `json-ts-mode` too).

Re: Why not include all ELPA packages in an Emacs release?

[Arash Esbati]

Tassilo Horn <tsdh@gnu.org> writes:

> Nice completion UIs are certainly a very important feature and I have
> the same preferences as you.  But I don't see a reason why some specific
> one has to be in core.  It has long been ivy and company, now it's
> vertico and corfu, and in 2 years the hottest bets might be something
> else.

I'd agree in general for almost every other feature, but my vote is to
make an exception for the completion UI since it's an important piece
for the user.  Using completion currently with Emacs built-in tools is
not very appealing, IMHO.

> I mean, it's easy to put things into core but hardly possible to remove
> them again without annoying someone.

Yes, and there is no easy answer, but keeping status quo indefinitely
doesn't feel right either.

> One practical question is what happens with stock tex-mode.el then.
> Right now, when you install AUCTeX, it'll take over.  That would annoy
> tex-mode users, I guess.

I'd say you know best that we can teach AUCTeX to be more defensive in
this regard, so that shouldn't be a showstopper :-)

I think my strongest argument for adding AUCTeX to core is that
currently, GNU/Linux distros offer AUCTeX as an extra package for their
users, at least I know from Debian and openSUSE.  The issue is that
their extra package is often outdated or offers auto-generated styles
which don't work as expected.  Bottom line is that we tell frustrated
users to uninstall what the distro offers and install it from ELPA.
Having AUCTeX in core would make those distro packages vanish
(hopefully) and take the hassle from the users and us providing tarball
releases (which I think we have decided to drop already).  So a dual
approach (core and ELPA) would be my favorite.

> Also, I'm not sure if AUCTeX is not already past its prime time.
> Preview-LaTeX is cool but IIRC doesn't work anymore with some popular
> styles.  And document parsing in order to highlight user-defined
> commands and offer them for completion is probably better done in some
> LaTeX language server like texlab and then used with eglot or
> lsp-mode.

Reg. preview-latex, maybe AUCTeX should have a look how Org creates
previews, IIUC that process was revamped recently(?).  Reg. LSP servers:
I thought many times that LSP would be the savior, but my brief tests
with LaTeX LSP servers were not really satisfying.  Chances are high
that the problem was sitting in front of the monitor.  So yes, there is
a good chance that AUCTeX is about to have its prime time, but I think
not yet because it still offers a lot more than mere completion.

Best, Arash

Re: Why not include all ELPA packages in an Emacs release?

[Arash Esbati]

Philip Kaludercic <philipk@posteo.net> writes:

> I am not familiar with cape,

Cape offers CAPFs, but I think one great feature is that you can switch
the CAPF function during completion with something like this in your
init file:

(use-package cape
  :bind (:map corfu-map
              ("C-c p d" . cape-dabbrev)
              ("C-c p f" . cape-file)
              ("C-c p s" . cape-elisp-symbol)
              ("C-c p w" . ispell-completion-at-point)
              ("C-c p :" . cape-emoji)))

> but IIRC the issue with corfu is that without any further changes, it
> doesn't support non-graphical Emacs, right?

True, there is corfu-terminal.el for that (which I've never used).

> "Better", in this case, than `fido-vertical-mode'?

I admit I've never used `fido-vertical-mode', so I can't offer a serious
comparison, sorry.

Best, Arash

Re: Why not include all ELPA packages in an Emacs release?

[Dmitry Gutov]

On 30/05/2024 01:16, Arash Esbati wrote:
>> but IIRC the issue with corfu is that without any further changes, it
>> doesn't support non-graphical Emacs, right?
> True, there is corfu-terminal.el for that (which I've never used).

Which is in NonGNU because it depends on 'popon' which is also in 
NonGNU. :-/

Re: Why not include all ELPA packages in an Emacs release?

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Wed, 29 May 2024 13:44:37 -0700
> Cc: jb@jeremybryant.net, emacs-devel@gnu.org, monnier@iro.umontreal.ca, 
> 	philipk@posteo.net
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > What we discussed was the desire to move packages out of core in a way
> > that a release tarball would include those package.  That is what we
> > have a consensus about (but not a complete and satisfactory solution
> > yet).
> 
> It's possible that we have interpreted the previous discussions a bit
> differently, or that I'm filling in too many blanks.
> 
> My understanding was that this was always going to be something like
> 1) make it possible to ship ELPA packages with core Emacs and then
> 2) decide which packages should be included.
> 
> To my mind, step 2 could either be restricted to only include packages
> that used to be in core, or it could not be.  The difference is not very
> big, at least not in principle, since before we bundle even two
> packages, we still have to solve the issue of how to bundle even one.

Sure, but we never meant that step 2 will include all of ELPA, I
think.  And at least my understanding of step 2 was indeed that it
should allow to have more core packages on ELPA than to add packages
to the Emacs release tarballs.

Re: Why not include all ELPA packages in an Emacs release?

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Stefan Kangas <stefankangas@gmail.com>,
>   jb@jeremybryant.net,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,
>   philipk@posteo.net
> Date: Wed, 29 May 2024 22:35:36 +0200
> 
> Arash Esbati <arash@gnu.org> writes:
> 
> > If Emacs considers in-buffer completion an important feature, then I'd
> > say corfu and cape are must.  vertico and marginalia are also must in
> > my book since they offer a better experience with vertical minibuffer
> > completion.
> 
> Nice completion UIs are certainly a very important feature and I have
> the same preferences as you.  But I don't see a reason why some specific
> one has to be in core.  It has long been ivy and company, now it's
> vertico and corfu, and in 2 years the hottest bets might be something
> else.

These are valid concerns, thanks.  However, if we consider these
features important enough and add them to core, we take the
responsibility to keep them alive for years to come, even if their
original developers move on.

> I mean, it's easy to put things into core but hardly possible to remove
> them again without annoying someone.

True.  Which is why we should consider the cost of maintaining those
as part of the decision-making.

> FWIW, I'd rather move more stuff from core to ELPA and add mechanics to
> install from ELPA easily.

No one really disagrees.  The problem is how to do that in a way that
doesn't disrupt the users and the Emacs development procedures.  See
the past discussions and the conclusions we have reached (although TBH
not all of the conclusions are agreed-upon by all of us).

> > And while we're at it: There are sometimes requests for adding AUCTeX
> > to core.  Do you have an opinion about that?
> 
> One practical question is what happens with stock tex-mode.el then.
> Right now, when you install AUCTeX, it'll take over.  That would annoy
> tex-mode users, I guess.

We don't have to make AUCTeX the default if we include it.  It could
start being optional, like the various *-ts-mode's are.

Re: Why not include all ELPA packages in an Emacs release?

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Stefan Kangas <stefankangas@gmail.com>,
>   jb@jeremybryant.net,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,
>   philipk@posteo.net
> Date: Thu, 30 May 2024 00:04:49 +0200
> 
> > I mean, it's easy to put things into core but hardly possible to remove
> > them again without annoying someone.
> 
> Yes, and there is no easy answer, but keeping status quo indefinitely
> doesn't feel right either.

Why not?  ELPA was invented for this reason, among others.  There's
nothing wrong, AFAIU, with having some packages on ELPA instead of in
the tarball.  package.el nowadays makes it easy to install and update
such packages, so where's the problem?

Re: Why not include all ELPA packages in an Emacs release?

[Philip Kaludercic]

Arash Esbati <arash@gnu.org> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> I am not familiar with cape,
>
> Cape offers CAPFs, but I think one great featuzre is that you can switch
> the CAPF function during completion with something like this in your
> init file:
>
> (use-package cape
>   :bind (:map corfu-map
>               ("C-c p d" . cape-dabbrev)
>               ("C-c p f" . cape-file)
>               ("C-c p s" . cape-elisp-symbol)
>               ("C-c p w" . ispell-completion-at-point)
>               ("C-c p :" . cape-emoji)))

I tried it out, and it doesn't seem to work that well without something
like corfu or vertico.  Generally it seems like an example where
"completion" is misinterpreted to mean "selection".

>> but IIRC the issue with corfu is that without any further changes, it
>> doesn't support non-graphical Emacs, right?
>
> True, there is corfu-terminal.el for that (which I've never used).
>
>> "Better", in this case, than `fido-vertical-mode'?
>
> I admit I've never used `fido-vertical-mode', so I can't offer a serious
> comparison, sorry.

That's fine.

-- 
	Philip Kaludercic on peregrine

Re: Why not include all ELPA packages in an Emacs release?

[Daniel Mendler via Emacs development discussions.]

Philip Kaludercic <philipk@posteo.net> writes:

> Arash Esbati <arash@gnu.org> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> I am not familiar with cape,
>>
>> Cape offers CAPFs, but I think one great featuzre is that you can switch
>> the CAPF function during completion with something like this in your
>> init file:
>>
>> (use-package cape
>>   :bind (:map corfu-map
>>               ("C-c p d" . cape-dabbrev)
>>               ("C-c p f" . cape-file)
>>               ("C-c p s" . cape-elisp-symbol)
>>               ("C-c p w" . ispell-completion-at-point)
>>               ("C-c p :" . cape-emoji)))
>
> I tried it out, and it doesn't seem to work that well without something
> like corfu or vertico.  Generally it seems like an example where
> "completion" is misinterpreted to mean "selection".

This is not correct. Please stop spreading misinformation like this.
Capfs like the ones from Cape can be used to complete in a stepwise
manner. Simple examples are cape-dict, cape-dabbrev or cape-file.
cape-dabbrev for example predates the builtin dabbrev-capf and works in
the same way.

Daniel

Re: Why not include all ELPA packages in an Emacs release?

[Philip Kaludercic]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Arash Esbati <arash@gnu.org> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> I am not familiar with cape,
>>>
>>> Cape offers CAPFs, but I think one great featuzre is that you can switch
>>> the CAPF function during completion with something like this in your
>>> init file:
>>>
>>> (use-package cape
>>>   :bind (:map corfu-map
>>>               ("C-c p d" . cape-dabbrev)
>>>               ("C-c p f" . cape-file)
>>>               ("C-c p s" . cape-elisp-symbol)
>>>               ("C-c p w" . ispell-completion-at-point)
>>>               ("C-c p :" . cape-emoji)))
>>
>> I tried it out, and it doesn't seem to work that well without something
>> like corfu or vertico.  Generally it seems like an example where
>> "completion" is misinterpreted to mean "selection".
>
> This is not correct. Please stop spreading misinformation like this.
> Capfs like the ones from Cape can be used to complete in a stepwise
> manner. 

I understand that, I am just saying that it doesn't feel that natural
without something like corfu enabled as well.  Or at least with the
above configuration, something like cape-emoji with the default
in-buffer completion is less comfortable than the built-in C-x 8 e.
Same with dabbrev itself vs. cape-dabbrev.

>         Simple examples are cape-dict, cape-dabbrev or cape-file.
> cape-dabbrev for example predates the builtin dabbrev-capf and works in
> the same way.

How?  When I use M-/, the expansion is replaced in place, while
cape-dabbrev behaves more like dabbrev-completion (C-M-/) by prompting
me for input.

> Daniel

-- 
	Philip Kaludercic on peregrine

Re: Why not include all ELPA packages in an Emacs release?

[Po Lu]

Tassilo Horn <tsdh@gnu.org> writes:

> FWIW, I'd rather move more stuff from core to ELPA and add mechanics to
> install from ELPA easily.  use-package was one such thing but I imagine
> a mechanism that would provide package suggestions, e.g., like asking a
> user to install toml-mode when finding a toml file for the first time,
> or suggesting to install calc or calculator when typing calc at the M-x
> prompt...

Definitely not.  calc-mode is one of those packages that are intimately
connected to functionality provided in core, and it would be a
tremendous inconvenience if Emacs developers were required to install
changes in more than one repository when the next core change with
far-reaching consequences, such as bignums had, comes along.  Or, to
give another example, during the development of the Android port, it was
necessary to apply mechanical modifications to plenty of code in core to
adapt it to Android environments, and this task would most probably have
remained unfinished, with respect to to such packages as calc, if they
had been extracted to independent repositories and were only distributed
in release tarballs.

I think it is generally counterproductive to deny or impair our control
over the distribution and maintenance of packages, whether they be also
published on ELPA or not.  As no one has suggested that the quantity of
code being maintained in the repository now has become an intolerable
burden, let us not invent imaginary problems and, with them, paralyzing
solutions.

Re: Why not include all ELPA packages in an Emacs release?

[Philip Kaludercic]

Tassilo Horn <tsdh@gnu.org> writes:

> FWIW, I'd rather move more stuff from core to ELPA and add mechanics to
> install from ELPA easily.

There is a gnu-elpa package, and I have prepared a patch to extend
package.el with auto-suggestion capabilities for major modes that could
be installed from ELPA as well.

-- 
	Philip Kaludercic on peregrine

Re: Why not include all ELPA packages in an Emacs release?

[Daniel Mendler via Emacs development discussions.]

Philip Kaludercic <philipk@posteo.net> writes:

> Daniel Mendler <mail@daniel-mendler.de> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Arash Esbati <arash@gnu.org> writes:
>>>
>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>
>>>>> I am not familiar with cape,
>>>>
>>>> Cape offers CAPFs, but I think one great featuzre is that you can switch
>>>> the CAPF function during completion with something like this in your
>>>> init file:
>>>>
>>>> (use-package cape
>>>>   :bind (:map corfu-map
>>>>               ("C-c p d" . cape-dabbrev)
>>>>               ("C-c p f" . cape-file)
>>>>               ("C-c p s" . cape-elisp-symbol)
>>>>               ("C-c p w" . ispell-completion-at-point)
>>>>               ("C-c p :" . cape-emoji)))
>>>
>>> I tried it out, and it doesn't seem to work that well without something
>>> like corfu or vertico.  Generally it seems like an example where
>>> "completion" is misinterpreted to mean "selection".
>>
>> This is not correct. Please stop spreading misinformation like this.
>> Capfs like the ones from Cape can be used to complete in a stepwise
>> manner. 
>
> I understand that, I am just saying that it doesn't feel that natural
> without something like corfu enabled as well.  Or at least with the
> above configuration, something like cape-emoji with the default
> in-buffer completion is less comfortable than the built-in C-x 8 e.
> Same with dabbrev itself vs. cape-dabbrev.

First, this is not what you said (selection vs completion). Second, the
Capfs provided by Cape work as well or "natural" as other Capfs with the
default completion UI as they do with Corfu. The aforementioned
configuration is meant to trigger completion manually, which will open a
completion UI in any case, but this is not the only way to use these
Cape commands. You can also use the Capfs by adding them to the
`completion-at-point-functions' list. Then you can use them as usual via
`completion-at-point'.

>>         Simple examples are cape-dict, cape-dabbrev or cape-file.
>> cape-dabbrev for example predates the builtin dabbrev-capf and works in
>> the same way.
>
> How?  When I use M-/, the expansion is replaced in place, while
> cape-dabbrev behaves more like dabbrev-completion (C-M-/) by prompting
> me for input.

Yes, dabbrev-expand behaves differently. Note that cape-dabbrev also
replaces the expansion at point if it is unique, like the dabbrev-capf
or dabbrev-completion. But that's not my point here. Cape provides
Capfs, and all I am saying is that the Capfs work as well with the
default completion UI as they do with Corfu. They follow the usual
implementation practices of other Capfs which are already part of Emacs.
You can use them by invoking `completion-at-point' and perform a
step-wise expansion. Selection doesn't has to happen necessarily.

Daniel

Re: Why not include all ELPA packages in an Emacs release?

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arash Esbati <arash@gnu.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  Stefan Kangas <stefankangas@gmail.com>,
>>   jb@jeremybryant.net,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,
>>   philipk@posteo.net
>> Date: Thu, 30 May 2024 00:04:49 +0200
>> 
>> > I mean, it's easy to put things into core but hardly possible to remove
>> > them again without annoying someone.
>> 
>> Yes, and there is no easy answer, but keeping status quo indefinitely
>> doesn't feel right either.
>
> Why not?  ELPA was invented for this reason, among others.  There's
> nothing wrong, AFAIU, with having some packages on ELPA instead of in
> the tarball.  package.el nowadays makes it easy to install and update
> such packages, so where's the problem?

What I meant was: If one finds out that a change is justified and a good
idea now, then one shouldn't hold back the change with the argument "it
is easy to do now and hard to replace/revert in x years when something
else is en vogue".

Making a change just for change's sake is not what I meant.  And I'm a
happy ELPA user.

Best, Arash

Re: Why not include all ELPA packages in an Emacs release?

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Arash Esbati <arash@gnu.org>,  Eli Zaretskii <eliz@gnu.org>,  Stefan
>  Kangas <stefankangas@gmail.com>,  jb@jeremybryant.net,
>   emacs-devel@gnu.org,  monnier@iro.umontreal.ca,  philipk@posteo.net
> Date: Thu, 30 May 2024 15:55:56 +0800
> 
> Tassilo Horn <tsdh@gnu.org> writes:
> 
> > FWIW, I'd rather move more stuff from core to ELPA and add mechanics to
> > install from ELPA easily.  use-package was one such thing but I imagine
> > a mechanism that would provide package suggestions, e.g., like asking a
> > user to install toml-mode when finding a toml file for the first time,
> > or suggesting to install calc or calculator when typing calc at the M-x
> > prompt...
> 
> Definitely not.  calc-mode is one of those packages that are intimately
> connected to functionality provided in core, and it would be a
> tremendous inconvenience if Emacs developers were required to install
> changes in more than one repository when the next core change with
> far-reaching consequences, such as bignums had, comes along.

Packages that will be moved to ELPA (when that happens) should have
their dedicated maintainers, and it will be the job of those
maintainers to adapt to any changes in Emacs that affect each package.

I see no show-stoppers there, FWIW.

> I think it is generally counterproductive to deny or impair our control
> over the distribution and maintenance of packages, whether they be also
> published on ELPA or not.  As no one has suggested that the quantity of
> code being maintained in the repository now has become an intolerable
> burden, let us not invent imaginary problems and, with them, paralyzing
> solutions.

The issues which led us to seriously consider moving some core
packages to ELPA are not imaginary, they are very real.

Re: Why not include all ELPA packages in an Emacs release?

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> Packages that will be moved to ELPA (when that happens) should have
> their dedicated maintainers, and it will be the job of those
> maintainers to adapt to any changes in Emacs that affect each package.
>
> I see no show-stoppers there, FWIW.

Does Calc, or its ilk?  And will such maintainers notice and be willing
to adapt their packages to minor changes that don't affect themselves
and their existing users, as,

2023-06-26  Po Lu  <luangruo@yahoo.com>

	* lisp/calc/calc.el (calc-mode, calc): Make sure the on-screen
	keyboard is not hidden when a Calc buffer is created or a Calc
	Trail window is being created for the first time.

	* lisp/touch-screen.el (touch-screen-window-selection-changed):
	Take touch-screen-display-keyboard in to account.

during the development cycle of a new feature in a new Emacs release, as
was done here?  Had Calc been transferred into an independent
repository, with its own maintainers, these changes would have been
considerably delayed at best, depriving Android users of a functioning
scientific calculator in the interim.  Alternatively, we would have been
obliged to install this ourselves, in the ELPA repository, which you'll
agree, if you recall installing changesets as individual changes to each
modified file, is essentially no different from that, just with the
additional user burden of updating.  Another example:

2023-01-24  Po Lu  <luangruo@yahoo.com>

	* lisp/cedet/semantic/db-ebrowse.el
	(semanticdb-create-ebrowse-database):
	* lisp/gnus/mail-source.el (mail-source-movemail-program):
	* lisp/hexl.el (hexl-program):
	* lisp/htmlfontify.el (hfy-etags-bin):
	* lisp/ielm.el (inferior-emacs-lisp-mode):
	* lisp/mail/rmail.el (rmail-autodetect):
	(rmail-insert-inbox-text):
	* lisp/org/org-ctags.el (org-ctags-path-to-ctags):
	* lisp/progmodes/cperl-mode.el (cperl-etags):
	* lisp/speedbar.el (speedbar-fetch-etags-command):
	* lisp/textmodes/reftex-global.el (reftex-create-tags-file): Use
	new variables.

and indeed the modification to org-ctags.el was, until two days ago,
absent from the version of Org maintained by its dedicated maintainers.

> The issues which led us to seriously consider moving some core
> packages to ELPA are not imaginary, they are very real.

Which issues and packages are you speaking of?

Re: Why not include all ELPA packages in an Emacs release?

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: tsdh@gnu.org,  arash@gnu.org,  stefankangas@gmail.com,
>   jb@jeremybryant.net,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca,
>   philipk@posteo.net
> Date: Thu, 30 May 2024 19:53:54 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Packages that will be moved to ELPA (when that happens) should have
> > their dedicated maintainers, and it will be the job of those
> > maintainers to adapt to any changes in Emacs that affect each package.
> >
> > I see no show-stoppers there, FWIW.
> 
> Does Calc, or its ilk?

Calc is not on ELPA yet, so the question's answer is "undefined".

> And will such maintainers notice and be willing
> to adapt their packages to minor changes that don't affect themselves
> and their existing users, as,

If they don't, someone will ping them, sooner or later.  usually via a
bug report.

> 2023-06-26  Po Lu  <luangruo@yahoo.com>
> 
> 	* lisp/calc/calc.el (calc-mode, calc): Make sure the on-screen
> 	keyboard is not hidden when a Calc buffer is created or a Calc
> 	Trail window is being created for the first time.
> 
> 	* lisp/touch-screen.el (touch-screen-window-selection-changed):
> 	Take touch-screen-display-keyboard in to account.
> 
> during the development cycle of a new feature in a new Emacs release, as
> was done here?  Had Calc been transferred into an independent
> repository, with its own maintainers, these changes would have been
> considerably delayed at best, depriving Android users of a functioning
> scientific calculator in the interim.

Welcome to the real world.  We have such problems in Emacs all the
time, even without moving stuff to ELPA.  Emacs is immense, and in
many cases even identifying the places where such adaptations are
needed takes a long time.  How long do you think it took me to find
all the places where bidi support needed changes, even though I was
here all the time?

There's nothing new here, and we shouldn't reject the idea of moving
stuff to ELPA because of that.

> > The issues which led us to seriously consider moving some core
> > packages to ELPA are not imaginary, they are very real.
> 
> Which issues and packages are you speaking of?

Please re-read the discussions, I see no need to reiterate all that
here again.

Re: Why not include all ELPA packages in an Emacs release?

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> If they don't, someone will ping them, sooner or later.  usually via a
> bug report.

In a perfect world, perhaps, but in this, they are frequently
unreachable or unresponsive, as in the case of the popular "evil", which
a one-liner would enable to operate on Android, but whose maintainers
cannot be contacted and whose users cannot take a hint and would sooner
repeat the same few questions, at my inconvenience, than submit this
code to its developers:

  (put 'evil-mouse-drag-region 'ignored-mouse-command t)

(If Tom Dalziel is reading this, hint, hint.)

> Welcome to the real world.  We have such problems in Emacs all the
> time, even without moving stuff to ELPA.  Emacs is immense, and in
> many cases even identifying the places where such adaptations are
> needed takes a long time.  How long do you think it took me to find
> all the places where bidi support needed changes, even though I was
> here all the time?
>
> There's nothing new here, and we shouldn't reject the idea of moving
> stuff to ELPA because of that.

Bidi was quite an extensive update, and it's little wonder that plenty
of effort went into updating Emacs's motley components for it.  It
should also have been clear from my examples that my concerns are not
founded on such changes, but the far more numerous trivial changes that
require adjustments hither and yon, which are so much less taxing when
all concerned components are centralized in one repository.

Re: Why not include all ELPA packages in an Emacs release?

[Stefan Monnier]

> I think it is generally counterproductive to deny or impair our control
> over the distribution and maintenance of packages, whether they be also
> published on ELPA or not.

Assuming we have the ability to include GNU ELPA packages in the
tarball, the choice between keeping a package in core or in GNU ELPA
boils down to maintenance/administration.

From that point of view, if the package is mainly developed&maintained
by core Emacs developers it makes sense for it to live in core, but if
it's mostly developed by people who do it elsewhere (e.g. Org or
Idlwave) it should rather live in GNU ELPA to avoid the pain
of synchronization.


        Stefan

Re: Why not include all ELPA packages in an Emacs release?

[Po Lu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Assuming we have the ability to include GNU ELPA packages in the
> tarball, the choice between keeping a package in core or in GNU ELPA
> boils down to maintenance/administration.
>
> From that point of view, if the package is mainly developed&maintained
> by core Emacs developers it makes sense for it to live in core, but if
> it's mostly developed by people who do it elsewhere (e.g. Org or
> Idlwave) it should rather live in GNU ELPA to avoid the pain
> of synchronization.

That qualification is where I must disagree.  I think the correct
criterion is whether Emacs developers will _ever_ have occasion to edit
or use these packages, for otherwise applying even minuscule changes or
running an Emacs built from the repository will become needlessly
involved.  As you might expect, this immediately excludes Org mode,
which, though not by me, is certainly used by many users who compile
Emacs otherwise than from a release tarball.

Re: Why not include all ELPA packages in an Emacs release?

[Stefan Monnier]

> founded on such changes, but the far more numerous trivial changes that
> require adjustments hither and yon, which are so much less taxing when
> all concerned components are centralized in one repository.

You don't need to convince me of that, but that's only one side of the
coin.  There's also the issue of allowing/encouraging contributions from
people who do not want to contribute to core Emacs (e.g. because they
don't like our low-tech email-based workflow, or they don't like the way
we argue, ...).  Or the fact that in many people's mind once it's in
core Emacs it's in a kind of "long term retirement home" (tho apparently
there is a similar belief about GNU ELPA where some people are reluctant
to contribute a package to it before it's "complete").  Which is why
whether a package should live in core or in GNU ELPA is done on a case
by case basis and it's usually a "lesser evil" kind of choice.

BTW, technically, we *can* make a change to the Evil package without the
maintainers's agreement.  It will mean that the `elpa/evil` branch on
`nongnu.git` will not be in sync with the upstream Evil Git repository
(a "fork") and that we will have to keep *merging* our local changes
with upstream updates in the future (tho that can be automated as long
as it doesn't bump into merge conflicts), so it comes at a cost, but if
the upstream's maintenance goes dead it's an option we should consider.

[ And of course, we take go the hideous hack route of adding
  (put 'evil-mouse-drag-region 'ignored-mouse-command t) in Emacs's own
  code.  ]


        Stefan

Re: Why not include all ELPA packages in an Emacs release?

[Po Lu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> You don't need to convince me of that, but that's only one side of the
> coin.  There's also the issue of allowing/encouraging contributions from
> people who do not want to contribute to core Emacs (e.g. because they
> don't like our low-tech email-based workflow, or they don't like the way
> we argue, ...).

But that surely is a decision for individual package maintainers,
correct?  Org Mode's development procedures and practices are quite
close to ours, for instance, yet it "lives in" both core, ELPA, and
independently just the same.

> Or the fact that in many people's mind once it's in core Emacs it's in
> a kind of "long term retirement home" (tho apparently there is a
> similar belief about GNU ELPA where some people are reluctant to
> contribute a package to it before it's "complete").

In this instance, the problem is the exact inverse, which is to say that
users take prompt action on the part of (NonGNU) ELPA package
maintainers for granted, and it becomes an uphill battle to persuade
them to actively submit these trivial changes upstream, they rightly
perceiving this to be our responsibility.

> Which is why whether a package should live in core or in GNU ELPA is
> done on a case by case basis and it's usually a "lesser evil" kind of
> choice.

The greater evil, in my view, is moving packages from core to the devil
knows where.  Once a package is integrated into Emacs, its disposition
should be accomplished fact, the more so if no singularly compelling
reason (e.g., an uncooperative maintainer) emerges to move it elsewhere.
I think I mentioned why this is so.

> BTW, technically, we *can* make a change to the Evil package without the
> maintainers's agreement.  It will mean that the `elpa/evil` branch on
> `nongnu.git` will not be in sync with the upstream Evil Git repository
> (a "fork") and that we will have to keep *merging* our local changes
> with upstream updates in the future (tho that can be automated as long
> as it doesn't bump into merge conflicts), so it comes at a cost, but if
> the upstream's maintenance goes dead it's an option we should consider.

I think there is life in the old dog yet.  The difficulty is that mail
cannot be delivered to the address listed on its package description.

Re: Why not include all ELPA packages in an Emacs release?

[Stefan Monnier]

> That qualification is where I must disagree.  I think the correct
> criterion is whether Emacs developers will _ever_ have occasion to edit
> or use these packages, for otherwise applying even minuscule changes or
> running an Emacs built from the repository will become needlessly
> involved.

Is the pain of N years' worth of manual synchronization with upstream
development worth the gain N years later when you finally need to make
that minuscule change?

Sometimes it is, sometimes it isn't.


        Stefan

Re: Why not include all ELPA packages in an Emacs release?

[Philip Kaludercic]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Daniel Mendler <mail@daniel-mendler.de> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> Arash Esbati <arash@gnu.org> writes:
>>>>
>>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>>
>>>>>> I am not familiar with cape,
>>>>>
>>>>> Cape offers CAPFs, but I think one great featuzre is that you can switch
>>>>> the CAPF function during completion with something like this in your
>>>>> init file:
>>>>>
>>>>> (use-package cape
>>>>>   :bind (:map corfu-map
>>>>>               ("C-c p d" . cape-dabbrev)
>>>>>               ("C-c p f" . cape-file)
>>>>>               ("C-c p s" . cape-elisp-symbol)
>>>>>               ("C-c p w" . ispell-completion-at-point)
>>>>>               ("C-c p :" . cape-emoji)))
>>>>
>>>> I tried it out, and it doesn't seem to work that well without something
>>>> like corfu or vertico.  Generally it seems like an example where
>>>> "completion" is misinterpreted to mean "selection".
>>>
>>> This is not correct. Please stop spreading misinformation like this.
>>> Capfs like the ones from Cape can be used to complete in a stepwise
>>> manner. 
>>
>> I understand that, I am just saying that it doesn't feel that natural
>> without something like corfu enabled as well.  Or at least with the
>> above configuration, something like cape-emoji with the default
>> in-buffer completion is less comfortable than the built-in C-x 8 e.
>> Same with dabbrev itself vs. cape-dabbrev.
>
> First, this is not what you said (selection vs completion). 

I do think that this is related. cape-emoji, cape-tex, etc. do not
complete partial input.

>                                                             Second, the
> Capfs provided by Cape work as well or "natural" as other Capfs with the
> default completion UI as they do with Corfu. The aforementioned
> configuration is meant to trigger completion manually, which will open a
> completion UI in any case, but this is not the only way to use these
> Cape commands. You can also use the Capfs by adding them to the
> `completion-at-point-functions' list. Then you can use them as usual via
> `completion-at-point'.

Again, I cannot agree or at least I don't see why.  My impression is
that using the TeX input method remains more natural to me than adding
cape-tex to `completion-at-point-functions'.

>>>         Simple examples are cape-dict, cape-dabbrev or cape-file.
>>> cape-dabbrev for example predates the builtin dabbrev-capf and works in
>>> the same way.
>>
>> How?  When I use M-/, the expansion is replaced in place, while
>> cape-dabbrev behaves more like dabbrev-completion (C-M-/) by prompting
>> me for input.
>
> Yes, dabbrev-expand behaves differently. Note that cape-dabbrev also
> replaces the expansion at point if it is unique, like the dabbrev-capf
> or dabbrev-completion. But that's not my point here. Cape provides
> Capfs, and all I am saying is that the Capfs work as well with the
> default completion UI as they do with Corfu. They follow the usual
> implementation practices of other Capfs which are already part of Emacs.
> You can use them by invoking `completion-at-point' and perform a
> step-wise expansion. Selection doesn't has to happen necessarily.

I can't think of an example OOTB where completion-at-point expands a
string like \alpha results in α.  When playing around with this, it
feels unnatural to me, but if I am the only one who feels like this then
this doesn't matter.  My point is that this feels less unnatural when
used in combination with Corfu, that presents the options as a kind of
selection.

What is relevant for this thread is that there are (debatable) implicit
dependencies between packages that should be kept in mind when
considering to add a package to the core.

> Daniel

-- 
	Philip Kaludercic on peregrine

Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Jeremy Bryant]

Andrea Corallo <acorallo@gnu.org> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Arash Esbati <arash@gnu.org> writes:
>>
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>
>>>> If there are packages on ELPA which we consider to be a must for users
>>>> (I don't think there are, but maybe I'm forgetting something), lets
>>>> add them to core instead.
>>>
>>> If Emacs considers in-buffer completion an important feature, then I'd
>>> say corfu and cape are must.  
>>
>> I am not familiar with cape, but IIRC the issue with corfu is that
>> without any further changes, it doesn't support non-graphical Emacs,
>> right?
>>
>>>                               vertico and marginalia are also must in my
>>> book since they offer a better experience with vertical minibuffer
>>> completion.
>>
>> "Better", in this case, than `fido-vertical-mode'?
>>
>>> And while we're at it: There are sometimes requests for adding AUCTeX to
>>> core.  Do you have an opinion about that?
>>
>> I would certainly be a fan of that move.
>
> I support AUCTeX inclusion as well.
>
>   Andrea

I can volunteer for some of this work under the guidance of the
Emacs maintainers and the AUCTeX maintainers (as I have contributed some
patches already.).  AUCTeX follows the Emacs development guidelines so
it should be easier.

As mentioned by Eli, AUCTeX could be merged in core but not enabled by
default to start.

What would be the most important next step required?

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Philip Kaludercic]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Andrea Corallo <acorallo@gnu.org> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Arash Esbati <arash@gnu.org> writes:
>>>
>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>
>>>>> If there are packages on ELPA which we consider to be a must for users
>>>>> (I don't think there are, but maybe I'm forgetting something), lets
>>>>> add them to core instead.
>>>>
>>>> If Emacs considers in-buffer completion an important feature, then I'd
>>>> say corfu and cape are must.  
>>>
>>> I am not familiar with cape, but IIRC the issue with corfu is that
>>> without any further changes, it doesn't support non-graphical Emacs,
>>> right?
>>>
>>>>                               vertico and marginalia are also must in my
>>>> book since they offer a better experience with vertical minibuffer
>>>> completion.
>>>
>>> "Better", in this case, than `fido-vertical-mode'?
>>>
>>>> And while we're at it: There are sometimes requests for adding AUCTeX to
>>>> core.  Do you have an opinion about that?
>>>
>>> I would certainly be a fan of that move.
>>
>> I support AUCTeX inclusion as well.
>>
>>   Andrea
>
> I can volunteer for some of this work under the guidance of the
> Emacs maintainers and the AUCTeX maintainers (as I have contributed some
> patches already.).  AUCTeX follows the Emacs development guidelines so
> it should be easier.
>
> As mentioned by Eli, AUCTeX could be merged in core but not enabled by
> default to start.
>
> What would be the most important next step required?

Wait, is the plan to add AUCTeX directly to emacs.git, or to include
AUCTeX in the packages that would be bundled with the release-tarball?

-- 
	Philip Kaludercic on peregrine

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Jeremy Bryant]

Philip Kaludercic <philipk@posteo.net> writes:

> Jeremy Bryant <jb@jeremybryant.net> writes:
>
>> Andrea Corallo <acorallo@gnu.org> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> Arash Esbati <arash@gnu.org> writes:
>>>>
>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>>
>>>>>> If there are packages on ELPA which we consider to be a must for users
>>>>>> (I don't think there are, but maybe I'm forgetting something), lets
>>>>>> add them to core instead.
>>>>>
>>>>> If Emacs considers in-buffer completion an important feature, then I'd
>>>>> say corfu and cape are must.  
>>>>
>>>> I am not familiar with cape, but IIRC the issue with corfu is that
>>>> without any further changes, it doesn't support non-graphical Emacs,
>>>> right?
>>>>
>>>>>                               vertico and marginalia are also must in my
>>>>> book since they offer a better experience with vertical minibuffer
>>>>> completion.
>>>>
>>>> "Better", in this case, than `fido-vertical-mode'?
>>>>
>>>>> And while we're at it: There are sometimes requests for adding AUCTeX to
>>>>> core.  Do you have an opinion about that?
>>>>
>>>> I would certainly be a fan of that move.
>>>
>>> I support AUCTeX inclusion as well.
>>>
>>>   Andrea
>>
>> I can volunteer for some of this work under the guidance of the
>> Emacs maintainers and the AUCTeX maintainers (as I have contributed some
>> patches already.).  AUCTeX follows the Emacs development guidelines so
>> it should be easier.
>>
>> As mentioned by Eli, AUCTeX could be merged in core but not enabled by
>> default to start.
>>
>> What would be the most important next step required?
>
> Wait, is the plan to add AUCTeX directly to emacs.git, or to include
> AUCTeX in the packages that would be bundled with the release-tarball?

This suggestion within the thread was to add AUCTeX to Emacs core.

Moving core packages to ELPA [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Jeremy Bryant]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Kangas <stefankangas@gmail.com>
>> Date: Wed, 29 May 2024 13:44:37 -0700
>> Cc: jb@jeremybryant.net, emacs-devel@gnu.org, monnier@iro.umontreal.ca, 
>> 	philipk@posteo.net
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > What we discussed was the desire to move packages out of core in a way
>> > that a release tarball would include those package.  That is what we
>> > have a consensus about (but not a complete and satisfactory solution
>> > yet).
>> 
>> It's possible that we have interpreted the previous discussions a bit
>> differently, or that I'm filling in too many blanks.
>> 
>> My understanding was that this was always going to be something like
>> 1) make it possible to ship ELPA packages with core Emacs and then
>> 2) decide which packages should be included.
>> 
>> To my mind, step 2 could either be restricted to only include packages
>> that used to be in core, or it could not be.  The difference is not very
>> big, at least not in principle, since before we bundle even two
>> packages, we still have to solve the issue of how to bundle even one.
>
> Sure, but we never meant that step 2 will include all of ELPA, I
> think.  And at least my understanding of step 2 was indeed that it
> should allow to have more core packages on ELPA than to add packages
> to the Emacs release tarballs.

What is a good core package to start working on migrating to ELPA?

(In searching previous discussions, it wasn't obvious, apart from IDLWAVE)

Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Jeremy Bryant]

Stefan Kangas <stefankangas@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> What we discussed was the desire to move packages out of core in a way
>> that a release tarball would include those package.  That is what we
>> have a consensus about (but not a complete and satisfactory solution
>> yet).
>
> It's possible that we have interpreted the previous discussions a bit
> differently, or that I'm filling in too many blanks.
>
> My understanding was that this was always going to be something like
> 1) make it possible to ship ELPA packages with core Emacs and then
> 2) decide which packages should be included.
>
> To my mind, step 2 could either be restricted to only include packages
> that used to be in core, or it could not be.  The difference is not very
> big, at least not in principle, since before we bundle even two
> packages, we still have to solve the issue of how to bundle even one.
>
>> Back to the suggestion in this thread: I cannot really see why would
>> we want to add almost 500 packages to Emacs.  Especially since AFAIR
>> some of them solve the same or very similar problems in different,
>> sometimes even contradictory, ways.

Now, understood.

>>
>> If there are packages on ELPA which we consider to be a must for users
>> (I don't think there are, but maybe I'm forgetting something), lets
>> add them to core instead.
>
> Not sure if there are any that are a must, but I do think a few of them
> would be good to have included.  For example, `json-mode` would be nice
> to have available OOTB (though we now have `json-ts-mode` too).

Proposal:
How about selecting 1 package to work on bundling by ease of
implementation.

For example, Philip's auto-header package is on ELPA, and Philip has
ELPA knowledge.  Philip, WDYT?

(Reviewing the feature branches they are constructed in terms of general
examples, it may help to have a concrete single package to work on bundling?)

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Jeremy Bryant]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Jeremy Bryant <jb@jeremybryant.net> writes:
>>
>>> Andrea Corallo <acorallo@gnu.org> writes:
>>>
>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>
>>>>> Arash Esbati <arash@gnu.org> writes:
>>>>>
>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>>>
>>>>>>> If there are packages on ELPA which we consider to be a must for users
>>>>>>> (I don't think there are, but maybe I'm forgetting something), lets
>>>>>>> add them to core instead.
>>>>>>
>>>>>> And while we're at it: There are sometimes requests for adding AUCTeX to
>>>>>> core.  Do you have an opinion about that?
>>>>>
>>>>> I would certainly be a fan of that move.
>>>>
>>>> I support AUCTeX inclusion as well.
>>>>
>>>>   Andrea
>>>
>>> I can volunteer for some of this work under the guidance of the
>>> Emacs maintainers and the AUCTeX maintainers (as I have contributed some
>>> patches already.).  AUCTeX follows the Emacs development guidelines so
>>> it should be easier.
>>>
>>> As mentioned by Eli, AUCTeX could be merged in core but not enabled by
>>> default to start.
>>>
>>> What would be the most important next step required?
>>
>> Wait, is the plan to add AUCTeX directly to emacs.git, or to include
>> AUCTeX in the packages that would be bundled with the release-tarball?
>
> This suggestion within the thread was to add AUCTeX to Emacs core.

Philip, my suggestion was to continue on adapting AUCTeX for emacs.git

Alternatively, I can volunteer to work on a prototype bundling of
AUCTeX, but would need your suggestions on this.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Po Lu]

Jeremy Bryant <jb@jeremybryant.net> writes:

> How about selecting 1 package to work on bundling by ease of
> implementation.
>
> For example, Philip's auto-header package is on ELPA, and Philip has
> ELPA knowledge.  Philip, WDYT?
>
> (Reviewing the feature branches they are constructed in terms of general
> examples, it may help to have a concrete single package to work on bundling?)

And what of us sods who must build Emacs from the source code
repository?  It would be awful if features bundled in source tarballs
were to be absent from, for instance, the Android redistributables:

  https://sourceforge.net/projects/android-ports-for-gnu-emacs

and downloading dozens of ELPA repositories before each build, not to
mention repairing the inevitable incompatibility from time to time, is
not an enviable prospect in any sense.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Dmitry Gutov]

On 08/06/2024 04:44, Po Lu wrote:
> Jeremy Bryant<jb@jeremybryant.net>  writes:
> 
>> How about selecting 1 package to work on bundling by ease of
>> implementation.
>>
>> For example, Philip's auto-header package is on ELPA, and Philip has
>> ELPA knowledge.  Philip, WDYT?
>>
>> (Reviewing the feature branches they are constructed in terms of general
>> examples, it may help to have a concrete single package to work on bundling?)
> And what of us sods who must build Emacs from the source code
> repository?  It would be awful if features bundled in source tarballs
> were to be absent from, for instance, the Android redistributables:
> 
>    https://sourceforge.net/projects/android-ports-for-gnu-emacs
> 
> and downloading dozens of ELPA repositories before each build, not to
> mention repairing the inevitable incompatibility from time to time, is
> not an enviable prospect in any sense.

JFYI, only the ELPA repository should be necessary. The 
internal/external packages only differ in how they are tracked in that 
repository - but they are tracker anyway.

The "external" packages are tracked in separate branches (called 
external/<package-name>) - but you can get all of their contents with 
'git fetch'.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Po Lu]

Dmitry Gutov <dmitry@gutov.dev> writes:

> JFYI, only the ELPA repository should be necessary. The
> internal/external packages only differ in how they are tracked in that
> repository - but they are tracker anyway.
>
> The "external" packages are tracked in separate branches (called
> external/<package-name>) - but you can get all of their contents with
> 'git fetch'.

OK, but that solves none of the other problems I raised.

Re: Moving core packages to ELPA [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: Stefan Kangas <stefankangas@gmail.com>,  emacs-devel@gnu.org,
>   monnier@iro.umontreal.ca,  philipk@posteo.net
> Date: Fri, 07 Jun 2024 22:48:25 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> My understanding was that this was always going to be something like
> >> 1) make it possible to ship ELPA packages with core Emacs and then
> >> 2) decide which packages should be included.
> >> 
> >> To my mind, step 2 could either be restricted to only include packages
> >> that used to be in core, or it could not be.  The difference is not very
> >> big, at least not in principle, since before we bundle even two
> >> packages, we still have to solve the issue of how to bundle even one.
> >
> > Sure, but we never meant that step 2 will include all of ELPA, I
> > think.  And at least my understanding of step 2 was indeed that it
> > should allow to have more core packages on ELPA than to add packages
> > to the Emacs release tarballs.
> 
> What is a good core package to start working on migrating to ELPA?

We never seriously discussed that, since the issues with doing that
are not yet resolved, so any such work would be premature at best.

But the obvious candidates are those that are already on ELPA: Org,
Eglot, Tramp, and a few others.  With those, the only aspects of
"work" to remove them from emacs.git are those which currently prevent
such a move to begin with.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Stefan Kangas <stefankangas@gmail.com>,  Eli Zaretskii <eliz@gnu.org>,
>   emacs-devel@gnu.org,  monnier@iro.umontreal.ca,  philipk@posteo.net
> Date: Sat, 08 Jun 2024 09:44:47 +0800
> 
> And what of us sods who must build Emacs from the source code
> repository?  It would be awful if features bundled in source tarballs
> were to be absent from, for instance, the Android redistributables:
> 
>   https://sourceforge.net/projects/android-ports-for-gnu-emacs
> 
> and downloading dozens of ELPA repositories before each build, not to
> mention repairing the inevitable incompatibility from time to time, is
> not an enviable prospect in any sense.

If you have read the past discussions, this is exactly one of the
important issues that have to be solved before we start moving
packages out of emacs.git to ELPA.  Building Emacs from Git must
include update and build of the core packages that are on ELPA.  So no
need to reiterate those objections: they are already recorded, and
will prevent any such move if not resolved.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Philip Kaludercic]

Po Lu <luangruo@yahoo.com> writes:

> Dmitry Gutov <dmitry@gutov.dev> writes:
>
>> JFYI, only the ELPA repository should be necessary. The
>> internal/external packages only differ in how they are tracked in that
>> repository - but they are tracker anyway.
>>
>> The "external" packages are tracked in separate branches (called
>> external/<package-name>) - but you can get all of their contents with
>> 'git fetch'.
>
> OK, but that solves none of the other problems I raised.

How come?  If we were to copy the branches from elpa.git to emacs.git
and adjust the build system in such a way that the packages are checked
out using worktrees (as elpa-admin does), then the bundled packages
should be made available as a side-effect of trying to build Emacs.

-- 
	Philip Kaludercic on peregrine

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> If you have read the past discussions, this is exactly one of the
> important issues that have to be solved before we start moving
> packages out of emacs.git to ELPA.  Building Emacs from Git must
> include update and build of the core packages that are on ELPA.  So no
> need to reiterate those objections: they are already recorded, and
> will prevent any such move if not resolved.

Good to hear, thanks.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Po Lu]

Philip Kaludercic <philipk@posteo.net> writes:

> How come?  If we were to copy the branches from elpa.git to emacs.git
> and adjust the build system in such a way that the packages are checked
> out using worktrees (as elpa-admin does), then the bundled packages
> should be made available as a side-effect of trying to build Emacs.

But does this not amount to leaving those packages as they stand, just
with a more cumbersome build process?  I doubt very much that the
exponents of moving packages to ELPA had such an arrangement in mind
when they first framed their proposals, and to me this appears to offer
little advantage over the status quo.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Philip Kaludercic]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Proposal:
> How about selecting 1 package to work on bundling by ease of
> implementation.

I don't know if the 0->1 jump will make it easier to add arbitrary
packages.  What I have been considering is re-using package-vc to mirror
the job of elpa-admin inside the Emacs core during build-time.  I'd
imagine we would have some etc/bundled-packages.eld file, with package
specifications just like with ELPA, that would list all the packages
that should bundled with the core.  That way, adding or removing bundled
packages would (hopefully) just be a matter of adjusting that .eld file.

> For example, Philip's auto-header package is on ELPA, and Philip has
> ELPA knowledge.  Philip, WDYT?

The package is rather rudimentary and only works for simple C code where
all the functions have properly formatted man pages, so I don't know if
we really want to add it to Emacs -- though you can consider it as a
running example.

> (Reviewing the feature branches they are constructed in terms of general
> examples, it may help to have a concrete single package to work on bundling?)

-- 
	Philip Kaludercic on peregrine

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Philip Kaludercic]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Jeremy Bryant <jb@jeremybryant.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Jeremy Bryant <jb@jeremybryant.net> writes:
>>>
>>>> Andrea Corallo <acorallo@gnu.org> writes:
>>>>
>>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>>
>>>>>> Arash Esbati <arash@gnu.org> writes:
>>>>>>
>>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>>>>
>>>>>>>> If there are packages on ELPA which we consider to be a must for users
>>>>>>>> (I don't think there are, but maybe I'm forgetting something), lets
>>>>>>>> add them to core instead.
>>>>>>>
>>>>>>> And while we're at it: There are sometimes requests for adding AUCTeX to
>>>>>>> core.  Do you have an opinion about that?
>>>>>>
>>>>>> I would certainly be a fan of that move.
>>>>>
>>>>> I support AUCTeX inclusion as well.
>>>>>
>>>>>   Andrea
>>>>
>>>> I can volunteer for some of this work under the guidance of the
>>>> Emacs maintainers and the AUCTeX maintainers (as I have contributed some
>>>> patches already.).  AUCTeX follows the Emacs development guidelines so
>>>> it should be easier.
>>>>
>>>> As mentioned by Eli, AUCTeX could be merged in core but not enabled by
>>>> default to start.
>>>>
>>>> What would be the most important next step required?
>>>
>>> Wait, is the plan to add AUCTeX directly to emacs.git, or to include
>>> AUCTeX in the packages that would be bundled with the release-tarball?
>>
>> This suggestion within the thread was to add AUCTeX to Emacs core.
>
> Philip, my suggestion was to continue on adapting AUCTeX for emacs.git

I got that; if that is so and the AUCTeX maintainers are on board, I
have no comments or objections.

> Alternatively, I can volunteer to work on a prototype bundling of
> AUCTeX, but would need your suggestions on this.

-- 
	Philip Kaludercic on peregrine

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Philip Kaludercic]

Po Lu <luangruo@yahoo.com> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> How come?  If we were to copy the branches from elpa.git to emacs.git
>> and adjust the build system in such a way that the packages are checked
>> out using worktrees (as elpa-admin does), then the bundled packages
>> should be made available as a side-effect of trying to build Emacs.
>
> But does this not amount to leaving those packages as they stand, just
> with a more cumbersome build process?  I doubt very much that the
> exponents of moving packages to ELPA had such an arrangement in mind
> when they first framed their proposals, and to me this appears to offer
> little advantage over the status quo.

No, I am talking about moving packages from ELPA into the core, not the
other way around.

-- 
	Philip Kaludercic on peregrine

Re: Moving core packages to ELPA [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> But the obvious candidates are those that are already on ELPA: Org,
> Eglot, Tramp, and a few others.  With those, the only aspects of
> "work" to remove them from emacs.git are those which currently prevent
> such a move to begin with.

One aspect are regression tests. Tramp has ca 100 test cases in
tramp-tests.el. I don't know whether they run out-of-the-box with the
Tramp ELPA package. And it would be a big miss if they won't run by the
regular "make check" of Emacs. These regression tests are an infinite
source of indications, what is still missing to be fixed in Tramp.

Best regards, Michael.

Re: Moving core packages to ELPA [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Jeremy Bryant <jb@jeremybryant.net>,  stefankangas@gmail.com,
>   emacs-devel@gnu.org,  monnier@iro.umontreal.ca,  philipk@posteo.net
> Date: Sat, 08 Jun 2024 10:10:31 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > But the obvious candidates are those that are already on ELPA: Org,
> > Eglot, Tramp, and a few others.  With those, the only aspects of
> > "work" to remove them from emacs.git are those which currently prevent
> > such a move to begin with.
> 
> One aspect are regression tests. Tramp has ca 100 test cases in
> tramp-tests.el. I don't know whether they run out-of-the-box with the
> Tramp ELPA package. And it would be a big miss if they won't run by the
> regular "make check" of Emacs. These regression tests are an infinite
> source of indications, what is still missing to be fixed in Tramp.

Does this mean Tramp on the master branch of elpa.git and Tramp on the
master branch of emacs.git are different?  If not, then why don't you
know whether Tramp on ELPA succeeds in the tests that Tramp in Emacs
does?

Anyway, what to do with tests of a core package that is on ELPA is an
interesting question.  I guess the tests should also be moved out of
emacs.git to ELPA, and should be brought/updated into the tree when
the package is updated?

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Arash Esbati]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Philip, my suggestion was to continue on adapting AUCTeX for emacs.git
>
> Alternatively, I can volunteer to work on a prototype bundling of
> AUCTeX, but would need your suggestions on this.

The way I see it Emacs maintainers have signaled they support putting
AUCTeX into core.

I think the natural way to proceed is that AUCTeX maintainers agree on
this step internally, decide what's the best way for the further
development, and come back to Emacs in order to discuss that outcome and
make a final decision together.

I'm saying this because currently, I don't know myself if AUCTeX should
follow the Org approach (Own Git repo, merge with Emacs on a regular
basis) or close the own shop and move into Emacs wholesale.  Let's not
forget the ELPA releases as well.  The bundling is only one step among
many others, I think.  And I don't think we have to rush: This will not
happen for Emacs 30.

Best, Arash

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Stefan Monnier]

> No, I am talking about moving packages from ELPA into the core, not
> the other way around.

As a general rule I'm opposed to *moving* packages from ELPA to
the core.  IIUC we're discussing *bundling* ELPA package with Emacs (or
you can think of it as *adding* ELPA packages to Emacs).


        Stefan

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Stefan Monnier]

> The "external" packages are tracked in separate branches (called
> external/<package-name>) - but you can get all of their contents with
> 'git fetch'.

Actually, they're all kept in separate branches nowadays.

Just some of them *also* have a life outside of `elpa.git` in some
upstream repository (and we poll those upstream repositories to pull
from them on a regular basis to try and keep the two in sync).


        Stefan

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Stefan Monnier]

> and downloading dozens of ELPA repositories before each build, not to

We have only two "ELPA repositories": `elpa.git` (which hosts all GNU
ELPA packages) and `nongnu.git` (which hosts all NonGNU ELPA packages).

So you're talking about downloading just 1 ELPA repository (since AFAIK
noone has yet suggested to bundle packages from NonGNU).

But in any case, as Eli mentioned, the plan was instead to duplicate the
branches from `elpa.git` that we want to bundle with Emacs into sister
branches hosted directly in `emacs.git`.
[ At the cost of a bit of redundant storage and having to keep those in
  sync with the ones in `elpa.git`.  ]


        Stefan


PS: For reference, the previous work in that direction can be found in
`feature/core-elpa-by-copy` (and also `feature/integrated-elpa` which
was an earlier attempt).

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Stefan Monnier]

> Alternatively, I can volunteer to work on a prototype bundling of
> AUCTeX, but would need your suggestions on this.

I really do want to see Emacs growing its build setup so it can bundle
ELPA packages like AUCTeX.  But that should not require any change on
the part of AUCTeX.  The benefit being that end users will get AUCTeX
support without having to download and install the package.

In comparison migrating AUCTeX into Emacs would require significant extra
work on the part of AUCTeX with no extra benefit for the end users.


        Stefan

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Po Lu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> and downloading dozens of ELPA repositories before each build, not to
>
> We have only two "ELPA repositories": `elpa.git` (which hosts all GNU
> ELPA packages) and `nongnu.git` (which hosts all NonGNU ELPA packages).
>
> So you're talking about downloading just 1 ELPA repository (since AFAIK
> noone has yet suggested to bundle packages from NonGNU).

This alone does nothing to reduce the complexity of managing an Emacs
distribution whose source code and development is not centralized but
distributed across many disparate sources with little to no coordination
among themselves (as, apparently, the cohort advocating this find such
coordination too restrictive for their tastes).  In ELPA's case,
branches and repositories are distinct only in name.

What if, for example, two packages should introduce changes that render
themselves incompatible?  Then the build would be broken, for reasons
beyond the immediate control of Emacs developers, and worse yet only for
those with the misfortune to have downloaded a version of elpa.git where
those versions were current.

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Po Lu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I really do want to see Emacs growing its build setup so it can bundle
> ELPA packages like AUCTeX.  But that should not require any change on
> the part of AUCTeX.  The benefit being that end users will get AUCTeX
> support without having to download and install the package.

This comes at the disadvantage of inconveniencing Emacs users who buikd
Emacs from the git repository, unless AUCTeX take pains to coordinate
with us, by, for instance, designating versions that are "safe" for
inclusion in Emacs, and responding to reports of build failures and the
like, which nullifies the stated advantage to them.

> In comparison migrating AUCTeX into Emacs would require significant
> extra work on the part of AUCTeX with no extra benefit for the end
> users.

This extra work is invested once, satisfies everyone once completed, and
preserves a functioning status quo.  AFAIU its developers are also ready
to do so.  Meanwhile bundling ELPA packages in core remains a castle in
the air, and its advantages and deficiencies are not nearly so absolute
or well-understood as it is implied.  So please don't deal in absolutes.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Stefan Monnier]

> What if, for example, two packages should introduce changes that render
> themselves incompatible?  Then the build would be broken, for reasons
> beyond the immediate control of Emacs developers, and worse yet only for
> those with the misfortune to have downloaded a version of elpa.git where
> those versions were current.

The plan was for the Emacs code to have not just a list of GNU ELPA
packages to bundle, but also the specific commit to use: on `master` this
list would presumably be auto-updated every once in a while to point to
the "latest and greatest" of those packages, and then on the release
branch this would presumably be updated only manually.


        Stefan

Re: Moving core packages to ELPA [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> One aspect are regression tests. Tramp has ca 100 test cases in
>> tramp-tests.el. I don't know whether they run out-of-the-box with the
>> Tramp ELPA package. And it would be a big miss if they won't run by the
>> regular "make check" of Emacs. These regression tests are an infinite
>> source of indications, what is still missing to be fixed in Tramp.
>
> Does this mean Tramp on the master branch of elpa.git and Tramp on the
> master branch of emacs.git are different?

Yes.

> If not, then why don't you know whether Tramp on ELPA succeeds in the
> tests that Tramp in Emacs does?

I simply didn't try. At least the subdirectory structure is different,
and the Makefile is not prepared for starting tests. I suppose, these
points are not difficult to solve, but it must be done.

> Anyway, what to do with tests of a core package that is on ELPA is an
> interesting question.  I guess the tests should also be moved out of
> emacs.git to ELPA, and should be brought/updated into the tree when
> the package is updated?

Again, my major concern is, that in this case the tests will be applied
less frequently, if even. Those tests, not triggered by myself, are also
a good indication whether everything is OK on platforms I don't have
access to. See bug#71235 for a recent example.

Best regards, Michael.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> No, I am talking about moving packages from ELPA into the core, not
>> the other way around.
>
> As a general rule I'm opposed to *moving* packages from ELPA to
> the core.  IIUC we're discussing *bundling* ELPA package with Emacs (or
> you can think of it as *adding* ELPA packages to Emacs).

Right, my bad I didn't phrase that correctly.  I agree totally.

>
>         Stefan
>

-- 
	Philip Kaludercic on peregrine

Re: Moving core packages to ELPA [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: jb@jeremybryant.net,  stefankangas@gmail.com,  emacs-devel@gnu.org,
>   monnier@iro.umontreal.ca,  philipk@posteo.net
> Date: Sat, 08 Jun 2024 17:55:56 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Anyway, what to do with tests of a core package that is on ELPA is an
> > interesting question.  I guess the tests should also be moved out of
> > emacs.git to ELPA, and should be brought/updated into the tree when
> > the package is updated?
> 
> Again, my major concern is, that in this case the tests will be applied
> less frequently, if even.

Not if my vision of this is actually implemented.  My idea is that
when one says "git pull", the packages from ELPA get updated in the
tree as well (modulo, perhaps, some more complex Git command), and
that would include their test suites.  Then whenever one runs the test
suite, all the packages will be tested, including those from ELPA.

Re: Moving core packages to ELPA [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> Again, my major concern is, that in this case the tests will be applied
>> less frequently, if even.
>
> Not if my vision of this is actually implemented.  My idea is that
> when one says "git pull", the packages from ELPA get updated in the
> tree as well (modulo, perhaps, some more complex Git command), and
> that would include their test suites.  Then whenever one runs the test
> suite, all the packages will be tested, including those from ELPA.

Great vision, much work. Thanks!

Best regards, Michael.

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Po Lu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The plan was for the Emacs code to have not just a list of GNU ELPA
> packages to bundle, but also the specific commit to use: on `master`
> this list would presumably be auto-updated every once in a while to
> point to the "latest and greatest" of those packages, and then on the
> release branch this would presumably be updated only manually.

What's the substantial difference between this and the status quo?

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Stefan Monnier]

>> I really do want to see Emacs growing its build setup so it can bundle
>> ELPA packages like AUCTeX.  But that should not require any change on
>> the part of AUCTeX.  The benefit being that end users will get AUCTeX
>> support without having to download and install the package.
> This comes at the disadvantage of inconveniencing Emacs users who buikd
> Emacs from the git repository,

I don't see any reason to expect this would be any worse than what we
have currently with all the other packages we have in core.
And if an ELPA package fails to build, you can always re-set it to
a previous commit, or build without ELPA packages.

> unless AUCTeX take pains to coordinate with us, by, for instance,
> designating versions that are "safe" for inclusion in Emacs, and
> responding to reports of build failures and the like, which nullifies
> the stated advantage to them.

Telling us which versions are "safe" is already what maintainers do when
they bump the version number, so it's not necessarily extra work.
And it's much less work than sync'ing changes between two branches with
unrelated history, like what the Org guys do, what the CEDET guys
tried to do (and failed doing), what the Gnus guys did (and stopped
doing at some point), what Alan does with CC-mode, etc...

>> In comparison migrating AUCTeX into Emacs would require significant
>> extra work on the part of AUCTeX with no extra benefit for the end
>> users.
> This extra work is invested once,

Only if the `emacs.git` code becomes the one and only maintained code.
Otherwise there's an on-going requirement to keep thr `emacs.git` code
in-sync with some external branch without being able to use tools like
`git merge`, so it's a rather unexciting manual work.

> Meanwhile bundling ELPA packages in core remains a castle in the air,

Very much so.

> and its advantages and deficiencies are not nearly so absolute
> or well-understood as it is implied.  So please don't deal in absolutes.

If I "dealt in absolutes" it was indeed a mistake.  But I do know about
the pain of keeping separate branches in sync and I want to provide
a way to bundle packages without that extra pain.
Nothing is a pure gain, so it'll come with its own compromises of course.


        Stefan

Re: Candidate packages for ELPA bundling [Was: Re: Why not include all ELPA packages in an Emacs release?]

[Stefan Monnier]

>> The plan was for the Emacs code to have not just a list of GNU ELPA
>> packages to bundle, but also the specific commit to use: on `master`
>> this list would presumably be auto-updated every once in a while to
>> point to the "latest and greatest" of those packages, and then on the
>> release branch this would presumably be updated only manually.
> What's the substantial difference between this and the status quo?

Making it possible for Emacs users to have access to some additional
features without first having to install them.  Potentially even
enabling them by default.


        Stefan

Re: Adding AUCTeX to core Re: Why not include all ELPA packages in an Emacs release?

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Telling us which versions are "safe" is already what maintainers do when
> they bump the version number, so it's not necessarily extra work.
> And it's much less work than sync'ing changes between two branches with
> unrelated history, like what the Org guys do, what the CEDET guys
> tried to do (and failed doing), what the Gnus guys did (and stopped
> doing at some point), what Alan does with CC-mode, etc...

The "etc." there is not to be underestimated either, BTW.

We have several open bugs with packages that are years or even a decade
plus out-of-synch.  Even when everyone is still collaborating -- which
is not necessarily the case given the time scales involved with Emacs --
it still is a pain.

When people stop maintaining packages, for legitimate reasons such as
lack of interest, life, etc., it's more of an uphill battle.

Our experience here tells me that adding in even more opportunities for
that to happen is probably not what we want.