Re: Adding use-package to core

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

From: 	Eli Zaretskii
Subject: 	Re: Adding use-package to core
Date: 	Sun, 13 Nov 2022 09:31:11 +0200
> I don't share Stefan's fears.  If the only issue with moving
> use-package to core is that someone must step forward to take the
> responsibility for maintaining it, I think we can move it into core
> without fear.  I see no reason for making a dedicated maintainer for
> use-package a prerequisite for importing it, given what John says
> about its stability.  It's a non-issue.

Why don't you?  This package has been very popular for a long time and
at least I haven't seen anyone complain about it not being in core.
Whatever gets included seem to freeze in time and becomes very hard to
make non-breaking changes, and their writers probably get frustrated
from that more easily and stop developing it.  There's ever more lines of
code and more packages, I don't think this direction is sustainable and
I hope you will reconsider this approach of adding everything to core
at some point.

Re: Adding use-package to core

[Eli Zaretskii]

> Date: Sun, 13 Nov 2022 17:11:57 +0100 (CET)
> From: xenodasein@tutanota.de
> Cc: emacs-devel@gnu.org
> 
> From: 	Eli Zaretskii
> Subject: 	Re: Adding use-package to core
> Date: 	Sun, 13 Nov 2022 09:31:11 +0200
> > I don't share Stefan's fears.  If the only issue with moving
> > use-package to core is that someone must step forward to take the
> > responsibility for maintaining it, I think we can move it into core
> > without fear.  I see no reason for making a dedicated maintainer for
> > use-package a prerequisite for importing it, given what John says
> > about its stability.  It's a non-issue.
> 
> Why don't you?  This package has been very popular for a long time and
> at least I haven't seen anyone complain about it not being in core.
> Whatever gets included seem to freeze in time and becomes very hard to
> make non-breaking changes, and their writers probably get frustrated
> from that more easily and stop developing it.  There's ever more lines of
> code and more packages, I don't think this direction is sustainable and
> I hope you will reconsider this approach of adding everything to core
> at some point.

If you have read this discussion, you've seen that the package's
author thinks it's best to move use-package into core, and describes
the package as stable without any need for significant development.
So I think your comments are based on some misunderstanding.

Re: Adding use-package to core

[John Wiegley]

>>>>> "EZ" == Eli Zaretskii <eliz@gnu.org> writes:

EZ> If you have read this discussion, you've seen that the package's author
EZ> thinks it's best to move use-package into core, and describes the package
EZ> as stable without any need for significant development.

Correct, I think this is the real moment for "freezing it", as I don't forsee
more functionality being needed. There are quite a number of issues in GitHub,
though, so perhaps it would be nice to see that number significantly reduced
before including it.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Adding use-package to core

[Eli Zaretskii]

> From: "John Wiegley" <johnw@gnu.org>
> Cc: xenodasein@tutanota.de,  emacs-devel@gnu.org
> Date: Sun, 13 Nov 2022 09:53:11 -0800
> 
> >>>>> "EZ" == Eli Zaretskii <eliz@gnu.org> writes:
> 
> EZ> If you have read this discussion, you've seen that the package's author
> EZ> thinks it's best to move use-package into core, and describes the package
> EZ> as stable without any need for significant development.
> 
> Correct, I think this is the real moment for "freezing it", as I don't forsee
> more functionality being needed. There are quite a number of issues in GitHub,
> though, so perhaps it would be nice to see that number significantly reduced
> before including it.

We can do it in parallel, specifically during the pretest of Emacs 29.
Including in core doesn't mean we don't fix bugs or add features,
quite the opposite.

Re: Adding use-package to core

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> We can do it in parallel, specifically during the pretest of Emacs 29.
> Including in core doesn't mean we don't fix bugs or add features, quite the
> opposite.

Oh, sounds good then!

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Adding use-package to core

[Stefan Kangas]

xenodasein--- via "Emacs development discussions." <emacs-devel@gnu.org>
writes:

> I hope you will reconsider this approach of adding everything to core
> at some point.

Historically, yes, everything was added to core.  Not so much in the
last decade or so: what I see is that we are gradually working towards
reducing our scope, and moving inessential things out (see the
lisp/obsolete directory).  I think this is a good thing, and we should
continue that work.

The new things that do get added tend to be key features and libraries
where it *really* makes sense to have them in core.  We must provide
some minimal set of features by default if Emacs is to be useful to
anyone, so this is also a good thing.

But my argument was not about this.  The idea was that keeping
use-package external might make it easier for us to leverage the
community that already exists around that package (see the 150+ issues
in the tracker, open pull requests, etc.).

On the other hand, moving it to core might entice parts of that existing
community to engage with core development.  That would also be a good
thing.

So, at the end of the day, I'm sure we'll be fine either way.

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

Nov 13, 2022, 16:48 by eliz@gnu.org:

> If you have read this discussion, you've seen that the package's
> author thinks it's best to move use-package into core, and describes
> the package as stable without any need for significant development.
> So I think your comments are based on some misunderstanding.
>

How integrated is it with custom, package, etc? Not well enough to be
considered complete IMO.

My understanding is that you will make it yet another member of the
orphanage lisp/ when it could function perfectly from ELPA.

Re: Adding use-package to core

[Stefan Kangas]

xenodasein--- via "Emacs development discussions." <emacs-devel@gnu.org>
writes:

> How integrated is it with custom, package, etc? Not well enough to be
> considered complete IMO.

I have no idea what this refers to.  Please write up the issues
concretely, and we can work on fixing them.  If there are any issues on
GitHub that are important, please point out which they are.

Thanks in advance.

Re: Adding use-package to core

[Eli Zaretskii]

> Date: Sun, 13 Nov 2022 19:12:03 +0100 (CET)
> From: xenodasein@tutanota.de
> Cc: emacs-devel@gnu.org
> 
> Nov 13, 2022, 16:48 by eliz@gnu.org:
> 
> > If you have read this discussion, you've seen that the package's
> > author thinks it's best to move use-package into core, and describes
> > the package as stable without any need for significant development.
> > So I think your comments are based on some misunderstanding.
> >
> 
> How integrated is it with custom, package, etc? Not well enough to be
> considered complete IMO.

If something in it turns out to be incomplete, we will fix it.

> My understanding is that you will make it yet another member of the
> orphanage lisp/ when it could function perfectly from ELPA.

The packages in lisp/ aren't orphans, just look at the Git logs.  (How
can you seriously consider the Lisp packages in core "frozen" when we
average 5000 commits a year for the past ten years??  Where do all
those commits go, in your opinion?)

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

Nov 13, 2022, 18:31 by eliz@gnu.org:

>> Date: Sun, 13 Nov 2022 19:12:03 +0100 (CET)
>> From: xenodasein@tutanota.de
>> Cc: emacs-devel@gnu.org
>>
>> How integrated is it with custom, package, etc? Not well enough to be
>> considered complete IMO.
>>
>
> If something in it turns out to be incomplete, we will fix it.
>
>> My understanding is that you will make it yet another member of the
>> orphanage lisp/ when it could function perfectly from ELPA.
>>
>
> The packages in lisp/ aren't orphans, just look at the Git logs.  (How
> can you seriously consider the Lisp packages in core "frozen" when we
> average 5000 commits a year for the past ten years??  Where do all
> those commits go, in your opinion?)
>

There seems to be many like allout that don't receive a commit for
decades (apart from a typo fix or two). AFAICT most go to new additions.
Stefan K seems to do a lot of thankless housekeeping in that direction,
but more importantly there was the force of nature from Lars before you
ahem frustrated him away, so my criticism mostly applies to from then on;
by all means if we can get Lars back to fixing the million line
interdependent Elisp code, I'll grunt no more.  The way I see it options
are immediately to stop anything else and to hold communions and pray to
get him back, or just start downsizing.

Re: Adding use-package to core

[Eli Zaretskii]

> Date: Sun, 13 Nov 2022 20:19:00 +0100 (CET)
> From: xenodasein@tutanota.de
> Cc: emacs-devel@gnu.org
> 
> > The packages in lisp/ aren't orphans, just look at the Git logs.  (How
> > can you seriously consider the Lisp packages in core "frozen" when we
> > average 5000 commits a year for the past ten years??  Where do all
> > those commits go, in your opinion?)
> >
> 
> There seems to be many like allout that don't receive a commit for
> decades (apart from a typo fix or two).

Which means they are probably not used by many, or don't need any
changes.  That's natural.

> AFAICT most go to new additions.

Not true.

> Stefan K seems to do a lot of thankless housekeeping in that direction,

I guess you only looked at the last year or two.

> but more importantly there was the force of nature from Lars before you
> ahem frustrated him away, so my criticism mostly applies to from then on;
> by all means if we can get Lars back to fixing the million line
> interdependent Elisp code, I'll grunt no more.  The way I see it options
> are immediately to stop anything else and to hold communions and pray to
> get him back, or just start downsizing.

Once again, I'm talking about the last decade, not the last couple of
years.  You are missing the big picture.

And anyway, I don't see what all this has to do with having a package
in core or not.

Re: Adding use-package to core

[John Wiegley]

>>>>> "Edd" == Emacs development discussions <xenodasein---> writes:

Edd> How integrated is it with custom, package, etc?

This is exactly the kind of thinking I was hoping for in suggestion the move
to core. Yes, yes and yes! Integration, documentation, the ability for users
to resolve their own issues (for example, making macro expansion easier to do
for use-package), these do need more thought and care.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

Nov 13, 2022, 18:46 by johnw@gnu.org:

>>>>>> "Edd" == Emacs development discussions <xenodasein---> writes:
>>>>>>
>
> Edd> How integrated is it with custom, package, etc?
>
> This is exactly the kind of thinking I was hoping for in suggestion the move
> to core. Yes, yes and yes! Integration, documentation, the ability for users
> to resolve their own issues (for example, making macro expansion easier to do
> for use-package), these do need more thought and care.
>
> -- 
> John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
> http://newartisans.com>                           60E1 46C4 BD1A 7AC1 4BA2
>

Using package.el and custom through their interfaces currently provides
the Emacs soft-entry into using packages from outside, use-package
provides yet another alternative in between them and manually doing
everything.  My understanding is that most people would actually prefer
solving this without tinkering in init.el but they find custom interface
inadequate so they go for the next alternative which is use-package.
I think we can bridge that gap somehow.  OTOH, use-package also works
great for people who'd like to set things through code, like how JW
uses it in their config on GH; for that way of doing things it is in much
better shape IMO, stable if you will.

Eli:
You said this can be fixed, of course but wouldn't it be much
easier to do without fighting backwards compatibility issues of
core code?

Re: Adding use-package to core

[John Wiegley]

>>>>> xenodasein  <xenodasein@tutanota.de> writes:

> Using package.el and custom through their interfaces currently provides the
> Emacs soft-entry into using packages from outside, use-package provides yet
> another alternative in between them and manually doing everything.

To me, use-package and package.el are mainly orthogonal: Package.el is for
package management (installing, updating, removing), while use-package is for
customization beyond what Customize provides -- or at least allows you to
concentrate changes related to the same package in one place.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

RE: [External] : Re: Adding use-package to core

[Drew Adams]

> To me, use-package and package.el are mainly orthogonal:
> Package.el is for package management (installing, updating,
> removing), while use-package is for customization beyond
> what Customize provides -- or at least allows you to
> concentrate changes related to the same package in one place.

Speaking/asking from ignorance here...

1. "Customization beyond what Customize provides"

What kinds of such customization, besides the
one you call out next (#2)?

2. "allows you to concentrate changes related
to the same package in one place"

Can you be more specific here?  How does what
you have in mind differ from what customize
groups provide?

_____

For #2, a package can even have a group with
subgroups.  And a package has parent groups.
Seems to me that not only do Customize groups
let you concentrate changes in one place, but
they even let you do so in a hierarchical way
(a graph, i.e., hierarchies with sharing),
that is, change your focus of concentration.
This applies for both browsing/discovering
and changing settings.

Examples at different ends of the grouping
spectrum:

`M-x customize-group bookmark-plus' shows 114
options and faces.  Flat: no subgroups.

On the other hand, group `Icicles' has nine
subgroups. `M-x customize-group Icicles' shows
the following, where each parent group and
subgroup name links to its `customize-group'
presentation:
______

Parent groups: Matching Completion Apropos Dabbrev
               Help Recentf Minibuffer Convenience

Icicles group: 
     State : visible group members are all at standard values.

   Minibuffer input completion and cycling of completion candidates.

   See also Doc-Part1, Doc-Part2, Description, Download, Other
   Libraries by Drew, and Send Bug Report.

 hexrgb-canonicalize-defined-colors-flag 
   Non-nil means remove duplicate color names. More

 icicle-completion-style-sets 
   Possible 'completion-styles' values for when 'TAB' completion
   method is 'vanilla'.

Subgroups:
Icicles-Buffers
  Icicles preferences related to buffers.
Icicles-Completions-Display
  Icicles preferences related to display of completion candidates.
Icicles-Files
  Icicles preferences related to files.
Icicles-Key-Bindings
  Icicles preferences related to key bindings.
Icicles-Key-Completion
  Icicles preferences related to key completion
  ('icicle-complete-keys').
Icicles-Matching
  Icicles preferences related to matching input for completion.
Icicles-Minibuffer-Display
  Icicles preferences related to minibuffer display during
  completion.
Icicles-Miscellaneous
  Miscellaneous Icicles preferences.
Icicles-Searching
  Icicles preferences related to searching.
_____

A guess is that you have in mind other _kinds_
of customizations, beyond options and faces.
Is that it?

Customize is limited, but it would be good to
set straight which of its limitations
`use-package' helps overcome.

One guess would be key bindings.  (The Emacs
manual has two completely separate sections,
`Easy Customization Interface' and `Customizing
Key Bindings', with eight and ten subsections,
respectively.)  (`defcustom' now has :type
`key-sequence', but that's of course only for
customizing option values.)
_____

To be clear, I'm not making any statement about
either `use-package' or Customize.  Certainly
the Customize UI could be improved, and there
are user customizations that Customize doesn't
help with at all, OOTB.

It might be good to match some of its limitations
against what `use-package' offers to handle them.
Maybe that's the best solution for them, or maybe
it can serve as food for thought for improvement
to Customize.

Re: [External] : Re: Adding use-package to core

[North Year]

[-- Attachment #1: Type: text/plain, Size: 1571 bytes --]

Drew Adams <drew.adams@oracle.com> writes:

>> To me, use-package and package.el are mainly orthogonal:
>> Package.el is for package management (installing, updating,
>> removing), while use-package is for customization beyond
>> what Customize provides – or at least allows you to
>> concentrate changes related to the same package in one place.
>
> Speaking/asking from ignorance here…
>
> 1. “Customization beyond what Customize provides”
>
> What kinds of such customization, besides the
> one you call out next (#2)?

customize.el offers a handy way for package developers
to specify options that is easier for user to tweak.

use-package has nothing to do with package developer,
it is for users. Users are easy to configure packages
in in one place
(including tweaking stuffs with customize.el by `:config')

use-package brings nothing new, it is just syntax sugar
with builtin functions (like autoload, with-eval-after-load, add-hook) stuffs.

And indeed, when using use-package at the first time, I often
need to macro-expand it to see its behaviors really
do something as expected.

> 2. “allows you to concentrate changes related
> to the same package in one place”

you can wrap all your configurations with a package in one place `(use-package xxx xxxxxxxx)',
where you customize the hooks, stuffs with customized.el,
the things need to do before loading this package, and
the things ned to do after loading this package,
the functions that need to autoloaded,
the keybindings related to this package, etc.

Re: [External] : Re: Adding use-package to core

[Matthew Carter]

Drew Adams <drew.adams@oracle.com> writes:

>> To me, use-package and package.el are mainly orthogonal:
>> Package.el is for package management (installing, updating,
>> removing), while use-package is for customization beyond
>> what Customize provides -- or at least allows you to
>> concentrate changes related to the same package in one place.
>
> Speaking/asking from ignorance here...
>
> 1. "Customization beyond what Customize provides"
>
> What kinds of such customization, besides the
> one you call out next (#2)?
>
> 2. "allows you to concentrate changes related
> to the same package in one place"
>
> Can you be more specific here?  How does what
> you have in mind differ from what customize
> groups provide?
>
> _____
>
> For #2, a package can even have a group with
> subgroups.  And a package has parent groups.
> Seems to me that not only do Customize groups
> let you concentrate changes in one place, but
> they even let you do so in a hierarchical way
> (a graph, i.e., hierarchies with sharing),
> that is, change your focus of concentration.
> This applies for both browsing/discovering
> and changing settings.
>
> Examples at different ends of the grouping
> spectrum:
>
> `M-x customize-group bookmark-plus' shows 114
> options and faces.  Flat: no subgroups.
>
> On the other hand, group `Icicles' has nine
> subgroups. `M-x customize-group Icicles' shows
> the following, where each parent group and
> subgroup name links to its `customize-group'
> presentation:
> ______
>
> Parent groups: Matching Completion Apropos Dabbrev
>                Help Recentf Minibuffer Convenience
>
> Icicles group: 
>      State : visible group members are all at standard values.
>
>    Minibuffer input completion and cycling of completion candidates.
>
>    See also Doc-Part1, Doc-Part2, Description, Download, Other
>    Libraries by Drew, and Send Bug Report.
>
>  hexrgb-canonicalize-defined-colors-flag 
>    Non-nil means remove duplicate color names. More
>
>  icicle-completion-style-sets 
>    Possible 'completion-styles' values for when 'TAB' completion
>    method is 'vanilla'.
>
> Subgroups:
> Icicles-Buffers
>   Icicles preferences related to buffers.
> Icicles-Completions-Display
>   Icicles preferences related to display of completion candidates.
> Icicles-Files
>   Icicles preferences related to files.
> Icicles-Key-Bindings
>   Icicles preferences related to key bindings.
> Icicles-Key-Completion
>   Icicles preferences related to key completion
>   ('icicle-complete-keys').
> Icicles-Matching
>   Icicles preferences related to matching input for completion.
> Icicles-Minibuffer-Display
>   Icicles preferences related to minibuffer display during
>   completion.
> Icicles-Miscellaneous
>   Miscellaneous Icicles preferences.
> Icicles-Searching
>   Icicles preferences related to searching.
> _____
>
> A guess is that you have in mind other _kinds_
> of customizations, beyond options and faces.
> Is that it?
>
> Customize is limited, but it would be good to
> set straight which of its limitations
> `use-package' helps overcome.
>
> One guess would be key bindings.  (The Emacs
> manual has two completely separate sections,
> `Easy Customization Interface' and `Customizing
> Key Bindings', with eight and ten subsections,
> respectively.)  (`defcustom' now has :type
> `key-sequence', but that's of course only for
> customizing option values.)
> _____
>
> To be clear, I'm not making any statement about
> either `use-package' or Customize.  Certainly
> the Customize UI could be improved, and there
> are user customizations that Customize doesn't
> help with at all, OOTB.
>
> It might be good to match some of its limitations
> against what `use-package' offers to handle them.
> Maybe that's the best solution for them, or maybe
> it can serve as food for thought for improvement
> to Customize.
>

I have always found M-x customize lacking, in that it expects to store
my preferences in a machine readable way (albeit, still text, but ill
formatted and automatically appended to my files) and be set up via a
slow manual process (hopping through customize menus is much slower than
just editing elisp).

Conversely, use-package allows me to set up all the options I care about
in one area, share them with others, and tweak them via the plain old
text editing vs clunky menus (I use emacs in TTY mode, maybe in GUI mode
these are less clunky).

If all customs were equivalent to a setq, I could do the old way
(organize on my own, with various (setq some-thing some-value)), however
more and more packages seem to need actual M-x customize assignment (a
simple setq won't make the change take proper effect).

Thankfully use-package accommodates this, and I can set these up in a
plain old lisp format in the :custom block the use-package provides.

In close to 15 years of using Emacs, I've never opted for M-x customize
when other alternatives would do as a user (although I aim to support it
in the non-GNU packages I provide).

-- 
Matthew Carter (m@ahungry.com)
http://ahungry.com

Re: [External] : Re: Adding use-package to core

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 573 bytes --]

On Mon, Nov 14, 2022 at 12:13 AM Matthew Carter <m@ahungry.com> wrote:

> If all customs were equivalent to a setq, I could do the old way
> (organize on my own, with various (setq some-thing some-value)), however
> more and more packages seem to need actual M-x customize assignment (a
> simple setq won't make the change take proper effect).

There's `setopt', now.

> In close to 15 years of using Emacs, I've never opted for M-x customize
> when other alternatives would do as a user

Same here (and just realized this month marks the 25th anniversary of my
emacsing).

[-- Attachment #2: Type: text/html, Size: 854 bytes --]

Re: [External] : Re: Adding use-package to core

[Tim Cross]

Drew Adams <drew.adams@oracle.com> writes:

>> To me, use-package and package.el are mainly orthogonal:
>> Package.el is for package management (installing, updating,
>> removing), while use-package is for customization beyond
>> what Customize provides -- or at least allows you to
>> concentrate changes related to the same package in one place.
>
> Speaking/asking from ignorance here...
>
> 1. "Customization beyond what Customize provides"
>
> What kinds of such customization, besides the
> one you call out next (#2)?
>
> 2. "allows you to concentrate changes related
> to the same package in one place"
>
> Can you be more specific here?  How does what
> you have in mind differ from what customize
> groups provide?
>
Hi Drew,

I'm not sure I agree with the statement from the post you are responding
to and wanted to give a different perspective which might clarify why
some like use-package.

At its most fundamental level, use-package is really just syntactic
sugar. It provides a somewhat more declarative way to manage package
loading, initialisation, configuration and customisation.  It takes
functionality which largely already exists and puts it together in one
declarative 'chunk'. Much of what it does is really just convenience
macros which define a DSL for installing, loading and configuring
packages. For example, it provides convenience features for defining key
bindings, setting interpreters, autoloads, file associations, loading on
demand or loading after something else has been loaded, adding hooks,
setting customize variables, etc. Instead of having

(unless (package-installed-p 'some-package)
  (package-install 'some-package))
(require 'some-package)
(add-hook 'some-package-hook 'my-some-package-setup)
(define-key some-package-map (kbd "C-c f") 'some-mode-do-something)
...
(custom-set-variables
  `(some-package-var t))
  
I might just have

(use-package some-package
  :ensure t
  :hook (some-package . my-some-package-setup)
  :bind ("C-c f" . some-mode-do-something)
  :custom (some-package-var t))
  
which would achieve the same thing.

The 3 main reasons I use it are

1. I don't like the customize interface. This is not a criticism of
customize. I don't like similar interfaces in other editors either. I
find them slow and cumbersome.

2. I like having all the configuration for a package in one
block/stanza. I could also do this with just normal elisp, but
use-package provides some additional syntactic sugar which makes things
less verbose. It also tends to lead to a more declarative configuration.

3. It makes it easy to delay loading of packages, temporarily disabling
or defining when a package is loaded and encourages better habits by
keeping all the configuration associated with a package together so that
if I do temporarily disable it, I don't get other init failures when
other parts of my init file try to reference something which is no
longer being loaded. This isn't so much about what the package does as
much as how it encourages better practice in managing your config.. 

In short, use-package doesn't provide new functionality as much as it
provides an alternative approach to existing functionality. It adds
another choice to how users can manage their configuration. Some people
will like it, some will hate it and probably most won't really care one
way or the other.  After using it for many years, I can say it has
certainly improved the management and maintenance of my init.el

I do think it is worth adding to Emacs and because it can also manage
installing packages from ELPA, I would love it if it was in Emacs core
so I can just use it instead of first ensuring it has been installed and
then using it. 

Re: [External] : Re: Adding use-package to core

[John Wiegley]

>>>>> Drew Adams <drew.adams@oracle.com> writes:

> 1. "Customization beyond what Customize provides"

> What kinds of such customization, besides the one you call out next (#2)?

Defining custom functions, binding keys, advising functions, adding functions
onto hooks (when the hook isn't customizable), using `setq` when necessary,
etc.

use-package was originally created to eliminate repeated boilerplate in my
init.el file, because, not only do I want all of these customizations, but I
want them to load FAST and come into play only if I actually use the package.

> 2. "allows you to concentrate changes related to the same package in one
> place"

> Can you be more specific here? How does what you have in mind differ from
> what customize groups provide?

Customize group is great, but it only works for what is customizable.

I'm a huge fan of customize, btw, and have been using it for many years. The
fact that my init.el is 4.6k lines long should tell you that it doesn't
satisfy all my needs; 463 use-package declarations are needed to achieve that.

  https://github.com/jwiegley/dot-emacs/blob/master/init.el

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Adding use-package to core

[Eli Zaretskii]

> Date: Sun, 13 Nov 2022 20:02:16 +0100 (CET)
> From: xenodasein@tutanota.de
> Cc: "xenodasein--- viaEmacs development discussions." <emacs-devel@gnu.org>,
> 	Eli Zaretskii <eliz@gnu.org>, stefankangas@gmail.com
> 
> You said this can be fixed, of course but wouldn't it be much
> easier to do without fighting backwards compatibility issues of
> core code?

Why would maintaining use-package in core present more
backward-compatibility issues than when it's maintained outside?

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

Nov 13, 2022, 20:04 by eliz@gnu.org:

>> Date: Sun, 13 Nov 2022 20:02:16 +0100 (CET)
>> From: xenodasein@tutanota.de
>> Cc: "xenodasein--- viaEmacs development discussions." <emacs-devel@gnu.org>,
>>  Eli Zaretskii <eliz@gnu.org>, stefankangas@gmail.com
>>
>> You said this can be fixed, of course but wouldn't it be much
>> easier to do without fighting backwards compatibility issues of
>> core code?
>>
>
> Why would maintaining use-package in core present more
> backward-compatibility issues than when it's maintained outside?
>

Same reason emacs-devel is not responsible for every single line of
Elisp code on the Internet?  External packages seem to get more love
from their developers.  If not, something new replaces them, people
migrate, and nobody complains to Emacs bug list about it.

Re: Adding use-package to core

[Eli Zaretskii]

> Date: Mon, 14 Nov 2022 11:32:53 +0100 (CET)
> From: xenodasein@tutanota.de
> Cc: johnw@gnu.org, emacs-devel@gnu.org, stefankangas@gmail.com
> 
> Nov 13, 2022, 20:04 by eliz@gnu.org:
> 
> >> You said this can be fixed, of course but wouldn't it be much
> >> easier to do without fighting backwards compatibility issues of
> >> core code?
> >
> > Why would maintaining use-package in core present more
> > backward-compatibility issues than when it's maintained outside?
> 
> Same reason emacs-devel is not responsible for every single line of
> Elisp code on the Internet?  External packages seem to get more love
> from their developers.

Admitting a package into Emacs doesn't mean its developer goes away
and we take over.  If the developer can make changes in ELPA, he or
she can also make changes in Emacs.  Assuming that the developer wants
to continue developing the package, of course (if not, it means the
package would have died anyway).

IOW, I see no reason to expect that adding a package to Emacs will
slow down, let alone "freeze" its development, not because of it was
added to Emacs anyway.

Re: Adding use-package to core

[Po Lu]

xenodasein--- via "Emacs development discussions." <emacs-devel@gnu.org>
writes:

> Why don't you?  This package has been very popular for a long time and
> at least I haven't seen anyone complain about it not being in core.
> Whatever gets included seem to freeze in time and becomes very hard to
> make non-breaking changes

First of all, this is untrue.  I think we have always let whoever
happens to be maintaining a package in core decide which changes to
make, and that includes breaking ones.

Secondly, not making breaking changes does not cause you to ``freeze in
time''.

> and their writers probably get frustrated from that more easily and
> stop developing it.

Any statistics?

> There's ever more lines of code and more packages, I don't think this
> direction is sustainable and I hope you will reconsider this approach
> of adding everything to core at some point.

Personally, I hope that everything most people find useful will
eventually make its way into Emacs, because doing so is a direct
shortcut to making Emacs more useful for everyone.

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

Nov 14, 2022, 00:27 by luangruo@yahoo.com:
> xenodasein--- via "Emacs development discussions." <emacs-devel@gnu.org>
> writes:
> > Why don't you?  This package has been very popular for a long time and
> > at least I haven't seen anyone complain about it not being in core.
> > Whatever gets included seem to freeze in time and becomes very hard to
> > make non-breaking changes
>
> First of all, this is untrue. I think we have always let whoever
> happens to be maintaining a package in core decide which changes to
> make, and that includes breaking ones.
>
> Secondly, not making breaking changes does not cause you to ``freeze in
> time''.

Something changes or it doesn't, I have no clue what you mean.

> > and their writers probably get frustrated from that more easily and
> > stop developing it.
>
> Any statistics?

I invite you to think what would it be like anytime you mention something
here the answer was any statistics, any statistics?  That this kind of
funny to imagine.

Anyway, 'statistics' in my head formed after looking through lisp files,
how many of them there, and the fact that how few people are maintaining
these, anyone can see this thanks to git.  Problem is over time commits
to core packages keep making assumptions about each other's existence
and that inter-dependency does not seem to encourage anyone to work on
them, even their original writers.  Even you are in your own X corner
and not touching that issue, except for nay-saying on this list.

> > There's ever more lines of code and more packages, I don't think this
> > direction is sustainable and I hope you will reconsider this approach
> > of adding everything to core at some point.
>
> Personally, I hope that everything most people find useful will
> eventually make its way into Emacs, because doing so is a direct
> shortcut to making Emacs more useful for everyone.

Yeah?  Who he is going to put in the cold hard work hours into
maintaining all that?  Furthermore after certain complexity it is
of no help even having numerous developers.  These are well documented
and understood facts of software development and when someone keeps
denying these things without substantial argument it displays blatant
incompetence.  I don't see how bundling millions of lines of code
together when there is already a system to distribute these as external
packages is a shortcut to usefulness for everyone (what does that even
mean?)  Anyone after some kind of shortcut to usefulness seem to simply
download VSCode and be done with it.

Re: Adding use-package to core

[Po Lu]

xenodasein@tutanota.de writes:

> Something changes or it doesn't, I have no clue what you mean.

Most changes break nothing.

> Anyway, 'statistics' in my head formed after looking through lisp files,
> how many of them there, and the fact that how few people are maintaining
> these, anyone can see this thanks to git.

I don't see that.  What I see instead is the same as with any other
mature, well-established software: most of our code is finished and does
not require constant changes.

> Problem is over time commits to core packages keep making assumptions
> about each other's existence and that inter-dependency does not seem
> to encourage anyone to work on them, even their original writers.

Any specific examples?

Anyway, once a package is included with Emacs, and its minimum Emacs
requirement also bumped, it is fine for it to rely on the rest of Emacs
being present.  But if its maintainer decides to support older versions
of Emacs as well, then everyone else does not interfere.  See TRAMP for
an example of one such package.

> Even you are in your own X corner and not touching that issue, except
> for nay-saying on this list.

It might not seem like it, but I have a job to keep me busy, and the
amount of time I can spend on Emacs is quite limited.

Add to that the fact that Emacs 29 is about to be released, and the
major changes to the GUI code in it have inevitably led to regressions
that have to be ironed out, and you will see why most of my changes in
the past two months have been limited to minor refactorings and bug
fixes.  That approach seems to have paid off.  For example, it has led
to bugs that have lain unfound for almost 30 years being fixed (see for
example 25c6bc7a3.)

> Yeah? Who he is going to put in the cold hard work hours into
> maintaining all that?

Presumably whoever wrote the package and has *asked* us to include it
into Emacs.

> Furthermore after certain complexity it is of no help even having
> numerous developers.

[...]

> These are well documented and understood facts of software development
> and when someone keeps denying these things without substantial
> argument it displays blatant incompetence.

So by changing the repository in which some code is placed, other code
is made more complex?  By what, magic?

> I don't see how bundling millions of lines of code together when there
> is already a system to distribute these as external packages is a
> shortcut to usefulness for everyone (what does that even mean?)

You cannot seriously claim it is easier to run several commands to
unpack and install a package in the ELPA directory than to do nothing at
all.

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

Nov 14, 2022, 10:47 by luangruo@yahoo.com:
> xenodasein@tutanota.de writes:
> > Something changes or it doesn't, I have no clue what you mean.
>
> Most changes break nothing.
>
> > Anyway, 'statistics' in my head formed after looking through lisp files,
> > how many of them there, and the fact that how few people are maintaining
> > these, anyone can see this thanks to git.
>
> I don't see that. What I see instead is the same as with any other
> mature, well-established software: most of our code is finished and does
> not require constant changes.

Practically all the programming languages or programs we support keep
evolving yet our code is finished?  Surely you can see how this
contradicts the whole point of software lifecycle wisdom?

> > Problem is over time commits to core packages keep making assumptions
> > about each other's existence and that inter-dependency does not seem
> > to encourage anyone to work on them, even their original writers.
>
> Any specific examples?

The example is the commit history, and the number of people rushing in
to help maintain old Emacs code they didn't originally write.

> Anyway, once a package is included with Emacs, and its minimum Emacs
> requirement also bumped, it is fine for it to rely on the rest of Emacs
> being present. But if its maintainer decides to support older versions
> of Emacs as well, then everyone else does not interfere. See TRAMP for
> an example of one such package.

Crucial point here being Michael A. actually does the hard work and
TRAMP is not one of the orphans.

> > Even you are in your own X corner and not touching that issue, except
> > for nay-saying on this list.
>
> It might not seem like it, but I have a job to keep me busy, and the
> amount of time I can spend on Emacs is quite limited.

That is why keeping the core as simple and easy to maintain as possible
is very important.

> Add to that the fact that Emacs 29 is about to be released, and the
> major changes to the GUI code in it have inevitably led to regressions
> that have to be ironed out, and you will see why most of my changes in
> the past two months have been limited to minor refactorings and bug
> fixes. That approach seems to have paid off. For example, it has led
> to bugs that have lain unfound for almost 30 years being fixed (see for
> example 25c6bc7a3.)

Your fixes are numerous and very helpful, thanks a lot.

> > Yeah? Who he is going to put in the cold hard work hours into
> >  maintaining all that?
>
> Presumably whoever wrote the package and has *asked* us to include it
> into Emacs.

My whole complaint is that this is not happening, so the trend here
should be reducing lines of code, but opposite is happening.  This
not a good direction.

> > Furthermore after certain complexity it is of no help even having
> > numerous developers.
> > [...]
> > These are well documented and understood facts of software development
> > and when someone keeps denying these things without substantial
> > argument it displays blatant incompetence.
>
> So by changing the repository in which some code is placed, other code
> is made more complex? By what, magic?

By their interdependence increasing over time.  All code being in the
same place and the nature of free software without strict rules, seem to
lead to this result, I believe it is easy to observe from Emacs source.

Quoting from some other mail I've sense to list:
"Same reason emacs-devel is not responsible for every single line of
Elisp code on the Internet?  External packages seem to get more love
from their developers.  If not, something new replaces them, people
migrate, and nobody complains to Emacs bug list about it."

> > I don't see how bundling millions of lines of code together when there
> > is already a system to distribute these as external packages is a
> > shortcut to usefulness for everyone (what does that even mean?)
>
> You cannot seriously claim it is easier to run several commands to
> unpack and install a package in the ELPA directory than to do nothing at
> all.

Technically it is not easier but also how much harder it is to install
them is so minuscule that the maintenance burden it causes it is not
worth it.  All that effort is better spent improving Elisp, minibuffer,
simple, cc-mode... i.e. "the good parts" and let convenience projects
get externally maintained.

Re: Adding use-package to core

[Po Lu]

xenodasein@tutanota.de writes:

> Practically all the programming languages or programs we support keep
> evolving yet our code is finished?  Surely you can see how this
> contradicts the whole point of software lifecycle wisdom?

I can maybe see how it contradicts consumerist wisdom, but not any other
wisdom.

> The example is the commit history, and the number of people rushing in
> to help maintain old Emacs code they didn't originally write.

That is not a specific example, or even a useful one.

> Crucial point here being Michael A. actually does the hard work and
> TRAMP is not one of the orphans.

And Michael is the TRAMP maintainer, who asked that TRAMP be included in
Emacs.  Just like John.

> That is why keeping the core as simple and easy to maintain as possible
> is very important.

And why would adding use-package negatively affect that?

I think "core" is basically *.c, and maybe subr.el and the likes of
cl-lib.el, and everything under lisp/emacs-lisp/.  Anyone can feel free
to correct me if that's wrong.

> My whole complaint is that this is not happening, so the trend here
> should be reducing lines of code, but opposite is happening.  This
> not a good direction.

LOC is a terrible metric for almost anything.

> By their interdependence increasing over time.  All code being in the
> same place and the nature of free software without strict rules, seem to
> lead to this result, I believe it is easy to observe from Emacs source.

That only happens when we think it is okay for a specific package.  When
we think it is not, then it does not happen, and as a result TRAMP still
works on Emacs 25.1.

OTOH, it is a Good Thing for other code in Emacs 29 or 30 to be able to
use use-package, as doing so will lead to improvements in both pieces of
code.

> Quoting from some other mail I've sense to list:
> "Same reason emacs-devel is not responsible for every single line of
> Elisp code on the Internet?  External packages seem to get more love
> from their developers.  If not, something new replaces them, people
> migrate

And that is a Bad Thing, which causes trouble to countless numbers of
people.  It is why I gave up on almost all packages from ELPA, after
trying to use and like package management for several years.

> Technically it is not easier but also how much harder it is to install
> them is so minuscule that the maintenance burden it causes it is not
> worth it.

I've never seen the maintenance burden of something increase simply due
to the location of a folder changing.

Re: Adding use-package to core

[Michael Albinus]

Po Lu <luangruo@yahoo.com> writes:

Hi,

>> Crucial point here being Michael A. actually does the hard work and
>> TRAMP is not one of the orphans.
>
> And Michael is the TRAMP maintainer, who asked that TRAMP be included in
> Emacs.  Just like John.

Completely unrelated, and I say it just for the archives: Integration of
Tramp into Emacs 22 was negotiated by Kai Grossjohann, the that-time
maintainer of Tramp. 20+ years ago.

Best regards, Michael.

Re: Adding use-package to core

[Eli Zaretskii]

> Date: Mon, 14 Nov 2022 12:52:50 +0100 (CET)
> From: xenodasein@tutanota.de
> Cc: "xenodasein--- viaEmacs development discussions." <emacs-devel@gnu.org>,
> 	eliz@gnu.org
> 
> My whole complaint is that this is not happening, so the trend here
> should be reducing lines of code, but opposite is happening.  This
> not a good direction.

You are entitled to your opinions, but we disagree, and we have many
years of experience to present as evidence.

Given such stark differences of assumptions, opinions, and experience
in Emacs development, I see no reason to keep arguing about this.

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

Nov 14, 2022, 13:54 by eliz@gnu.org:

>> Date: Mon, 14 Nov 2022 12:52:50 +0100 (CET)
>> From: xenodasein@tutanota.de
>> Cc: "xenodasein--- viaEmacs development discussions." <emacs-devel@gnu.org>,
>>  eliz@gnu.org
>>
>> My whole complaint is that this is not happening, so the trend here
>> should be reducing lines of code, but opposite is happening.  This
>> not a good direction.
>>
>
> You are entitled to your opinions, but we disagree, and we have many
> years of experience to present as evidence.
>
> Given such stark differences of assumptions, opinions, and experience
> in Emacs development, I see no reason to keep arguing about this.
>

Experience in accomplishing what exactly?  How can I examine this
evidence?  I don't know what are your goals, is going down from 5%
percent to 4% on StackOverflow's survey this year is one of them?
I really want to see this story of success, but wherever I look
that has numbers tells that you can't compare numbers even with
Notepad++.  Perhaps you can point out what kind of thing can change
your decisions on things like this, but I find it hard to understand
how leaving whoever comes after you an even bigger Emacs despite
having said currently no one understands all parts of it.

Re: Adding use-package to core

[Eli Zaretskii]

> Date: Mon, 14 Nov 2022 15:47:43 +0100 (CET)
> From: xenodasein@tutanota.de
> Cc: luangruo@yahoo.com, emacs-devel@gnu.org
> 
> > You are entitled to your opinions, but we disagree, and we have many
> > years of experience to present as evidence.
> >
> > Given such stark differences of assumptions, opinions, and experience
> > in Emacs development, I see no reason to keep arguing about this.
> 
> Experience in accomplishing what exactly?  How can I examine this
> evidence?  I don't know what are your goals, is going down from 5%
> percent to 4% on StackOverflow's survey this year is one of them?

The overall goal is, and always has been, to make Emacs more powerful,
more capable of supporting text-editing and text-processing
applications in more and more areas where people need such
capabilities and applications.  Examine the NEWS files over the last
20 years and ask yourself if that is not happening with every new
major release.

As for the evidence, it's right before your eyes: show me another
similar platform that lived for so many years, and after all these
years still gets developed at the same, if not higher, rate.  What
other evidence do you need?  5000 commits each year for the past 20
years cannot just come from a small band of like-thinking weirdos who
are detached from their users.

> I really want to see this story of success, but wherever I look
> that has numbers tells that you can't compare numbers even with
> Notepad++.

We develop Emacs for those who want and need to use it.  What matters
to us is not the percentage of our users between editors and IDEs,
what matters is where our users want us to advance, and which new
technologies will allow us to provide them with new and improved
capabilities, infrastructure, and applications.

> I find it hard to understand how leaving whoever comes after you an
> even bigger Emacs despite having said currently no one understands
> all parts of it.

No matter how small we make Emacs, no one will ever be able to
understand all of it.  It spans and uses too many too wide areas of
computer processing technologies for any single person to be able to
understand it.  Apart of the "usual" CS stuff, we have Unicode and
character properties, GUI and TTY display, text shaping, font
selection, image processing and display, Lisp and its compilation,
file I/O, regular expressions, and that's just a sample.  Who could
possible be an expert in all of that?

So this is a red herring.

As for "even bigger Emacs" part -- if you really believe Emacs can be
divided into core and the rest, I think you don't understand the
spirit and the heart of Emacs and its development.  (Which I can
totally feel for, since it takes a lot of time and personal
involvement to really grasp that.)  See, separating the Emacs core
from applications built on that core makes no sense, and if you try,
you will most probably kill Emacs.  If you examine carefully every
significant development in the Emacs core, you will see that each and
every one of them was always about some application or class of
applications.  Take the recent addition of tree-sitter, for example:
it would make no sense to develop that if font-lock for many
programming languages was not in core, because who in their right mind
would do all that hard work if it couldn't be immediately applied to
existing applications that are part of Emacs, and do it Right, using
all the expertise of "doing it Right" we have on board?

You should arrive at the same conclusion if you examine carefully
"core" Lisp packages, like subr.el, simple.el, etc. -- every single
part of them is there because we needed it for some application in
Emacs itself.  How could that happen if the applications were outside
of Emacs?  It is a fact that developers of 3rd-party applications tend
to solve the problem by themselves, almost never asking us to provide
new core features to help them solve those problems.  If most of Emacs
applications were outside the project, we could never have progressed
and extended our basic capabilities and infrastructure as we have them
now.

This intimate bond between the internals and the core infrastructure
on the one hand, and the applications that are built on top of that
OTOH -- this is our main strength, it's what keeps drawing in new core
developers over the years, thus keeping the project alive and kicking,
and developing at a mind-blowing rate.

And yes, this presents a problem: where do we draw the line, how do we
avoid making Emacs "too big" by adding "everything"?  Well, that's
what takes experience and intuition and gut feeling -- and a lot of
arguments like this one.  But by and large, we succeed, as the state
of Emacs today should tell us.

Whatever the difficulties to make the decision in each case, the
opinion that we should generally go in the opposite direction,
i.e. progressively remove more and more application-level code from
Emacs -- that opinion is completely wrong.  IMNSHO, anyway.

Re: Adding use-package to core

[xenodasein--- via Emacs development discussions.]

This is a real answer, I thank you for it.  FWIW, no one can claim
that you are not meticulous in your community management.  It is a
tiresome and hard task rarely done well.


Nov 14, 2022, 17:39 by eliz@gnu.org:

>> Experience in accomplishing what exactly?  How can I examine this
>> evidence?  I don't know what are your goals, is going down from 5%
>> percent to 4% on StackOverflow's survey this year is one of them?
>>
>
> The overall goal is, and always has been, to make Emacs more powerful,
> more capable of supporting text-editing and text-processing
> applications in more and more areas where people need such
> capabilities and applications.  Examine the NEWS files over the last
> 20 years and ask yourself if that is not happening with every new
> major release.
>
> As for the evidence, it's right before your eyes: show me another
> similar platform that lived for so many years, and after all these
> years still gets developed at the same, if not higher, rate.  What
> other evidence do you need?  5000 commits each year for the past 20
> years cannot just come from a small band of like-thinking weirdos who
> are detached from their users.
>
>> I really want to see this story of success, but wherever I look
>> that has numbers tells that you can't compare numbers even with
>> Notepad++.
>>
>
> We develop Emacs for those who want and need to use it.  What matters
> to us is not the percentage of our users between editors and IDEs,
> what matters is where our users want us to advance, and which new
> technologies will allow us to provide them with new and improved
> capabilities, infrastructure, and applications.
>

The problem is, more capable and powerful _compared to what?_
Comparing yourself against only yourself of yesterday is a great maxim
to have, in line with an ideal world and FSF ideals.  The reality is
that almost everyone using a different editor/ide do not actually hate
Emacs and would use it instead of Electron bloatware _if_ it solved
their problems faster, which it does, seen from how things are and my
experience is also supportive of this.  We are a practical species if
nothing else.

I use Emacs because I like it, not because it actually solves some
programming problems faster.  I am aware how it actually does many
things better after investment and when programming is your lifestyle,
but for most it is only a task.  Note that having to invest is not an
advantage or accomplishment, other programmable editors come very
close to Emacs if you program the hell out of them too.

From this perspective Emacs doesn't accomplish to actually be the
better editor overall, if we compare ourselves to others.  For example
Vim's advantages in quick remote editing or availability.  That
availability is also an accomplishment over Emacs, despite it being
the GNU editor.  I agree popularity is not always a great thing to
have and most times does not bring good things to its way.  Ideally
Emacs would be both demonstrably better yet still niche and we could
revel in our l33t productivity, without others rushing in to have some
:).


>> I find it hard to understand how leaving whoever comes after you an
>> even bigger Emacs despite having said currently no one understands
>> all parts of it.
>>
>
> No matter how small we make Emacs, no one will ever be able to
> understand all of it.  It spans and uses too many too wide areas of
> computer processing technologies for any single person to be able to
> understand it.  Apart of the "usual" CS stuff, we have Unicode and
> character properties, GUI and TTY display, text shaping, font
> selection, image processing and display, Lisp and its compilation,
> file I/O, regular expressions, and that's just a sample.  Who could
> possible be an expert in all of that?
>
> So this is a red herring.
>

Is it though?  Vim's Bram Moolenaar seems to manage it, not to mention
how GNU Emacs came to be; there are many more impressive software out
there with this singular dedicated developer.  Aiming for a single
person is indeed too bold, reducing this number to a few on the other
hand almost feels like a low hanging fruit.  Not every contributor has
to be in the top echelon of course, my impression is that you
underestimate the benefits a less complex project will bring and
instead decide it better for the users when there is more stuff all
together.  (I believe it is a correct assumption given you make it in
a time what Internet is not as ubiquitous as today.  Even then there
could be bigger alternative distributions that include optional
packages, mainly for less lucky parts of the Earth.)


> As for "even bigger Emacs" part -- if you really believe Emacs can be
> divided into core and the rest, I think you don't understand the
> spirit and the heart of Emacs and its development.  (Which I can
> totally feel for, since it takes a lot of time and personal
> involvement to really grasp that.)  See, separating the Emacs core
> from applications built on that core makes no sense, and if you try,
> you will most probably kill Emacs.  If you examine carefully every
> significant development in the Emacs core, you will see that each and
> every one of them was always about some application or class of
> applications.  Take the recent addition of tree-sitter, for example:
> it would make no sense to develop that if font-lock for many
> programming languages was not in core, because who in their right mind
> would do all that hard work if it couldn't be immediately applied to
> existing applications that are part of Emacs, and do it Right, using
> all the expertise of "doing it Right" we have on board?
>
> You should arrive at the same conclusion if you examine carefully
> "core" Lisp packages, like subr.el, simple.el, etc. -- every single
> part of them is there because we needed it for some application in
> Emacs itself.  How could that happen if the applications were outside
> of Emacs?  It is a fact that developers of 3rd-party applications tend
> to solve the problem by themselves, almost never asking us to provide
> new core features to help them solve those problems.  If most of Emacs
> applications were outside the project, we could never have progressed
> and extended our basic capabilities and infrastructure as we have them
> now.
>

I actually agree on this mostly, what I try to suggest is to narrowing
the scope of these applications, which must be maintained by
emacs-devel.  Bidirectional editing, advanced support for programming
languages etc. these are all great and must have features.  If
something isn't needed by almost every programmer or writer, package
it; this seems like a good rule of thumb?

(I guess from the timing I sounded like I am strongly against
inclusion of use-package, but this is not the case, this was more of a
general complaint.  I am somewhat against use-package because I think
it is a band-aid over the shortcomings of existing customization
facilities, but maybe Emacs should turn towards use-package style
customization I don't know.)


> This intimate bond between the internals and the core infrastructure
> on the one hand, and the applications that are built on top of that
> OTOH -- this is our main strength, it's what keeps drawing in new core
> developers over the years, thus keeping the project alive and kicking,
> and developing at a mind-blowing rate.
>
> And yes, this presents a problem: where do we draw the line, how do we
> avoid making Emacs "too big" by adding "everything"?  Well, that's
> what takes experience and intuition and gut feeling -- and a lot of
> arguments like this one.  But by and large, we succeed, as the state
> of Emacs today should tell us.
>
> Whatever the difficulties to make the decision in each case, the
> opinion that we should generally go in the opposite direction,
> i.e. progressively remove more and more application-level code from
> Emacs -- that opinion is completely wrong.  IMNSHO, anyway.
>

I do not understand what live-ness or dead-ness mean for Emacs' case,
as long as FSF and its support lives.  Because no matter what a lot of
people including me will keep using it because of its aesthetics,
community, the way it works etc.  This is indeed a great
accomplishment by itself, it is nightmarish to imagine where Emacs was
not an existing (viable) alternative.

My overall argument here is to demonstrates why I think emacs-devel's
methods are not as effective as they think, and should be more open to
different approaches.  Being in this state of being immortal (or
arguably already dead) gives you unique opportunities to be more
experimental I think.

IIUC the vision is having as many features (that mostly work) as
possible, instead of few superb ones.  Doing the opposite granted Vim
"the editor wars," giving an order of magnitude larger user base and
widespread availability.

Configuration helpers (wad: Adding use-package to core)

[Stefan Monnier]

BTW,

While I'm happy to see `use-package` just reached GNU ELPA and I'm sure
even more people would be happy if it made it into Emacs itself,
`use-package` is solving an old problem and I hope someone will improve
it (or some replacement) to solve new problems.

Some of the problems I think need solving in users's config files are:

- flymake: currently, flymake tends to flag a lot of configuration code
  because that code will be riddled with `setq` on variables that don't
  exist (yet) and calls to functions that don't exist (yet) either.
  A configuration system like use-package/setup/leaf provides some of
  the info needed to resolve this problem, but those packages have yet
  to be tweaked to explain to flymake what's going on.

- idempotent: a good configuration system should make it "hard" to
  write non-idempotent code, and should aim to make it so re-reading
  your init file will give you almost the same result as
  restarting Emacs.
  E.g. if you change a hook function, the result should not be to have
  both the old hook function and the new hook function on that hook.
  Or if you remove the "add-hook" code, re-reading the init file should
  end up removing the hook that was previously added.


        Stefan

Re: Adding use-package to core

[Eli Zaretskii]

> Date: Tue, 15 Nov 2022 16:38:10 +0100 (CET)
> From: xenodasein@tutanota.de
> Cc: emacs-devel@gnu.org
> 
> > We develop Emacs for those who want and need to use it.  What matters
> > to us is not the percentage of our users between editors and IDEs,
> > what matters is where our users want us to advance, and which new
> > technologies will allow us to provide them with new and improved
> > capabilities, infrastructure, and applications.
> 
> The problem is, more capable and powerful _compared to what?_

Compared to what we did yesterday and to what our users want or may
want us to do tomorrow.  For the latter, we keep tabs on other similar
editors and on technological developments, and consider their use in
Emacs.

> I use Emacs because I like it, not because it actually solves some
> programming problems faster.  I am aware how it actually does many
> things better after investment and when programming is your lifestyle,
> but for most it is only a task.  Note that having to invest is not an
> advantage or accomplishment, other programmable editors come very
> close to Emacs if you program the hell out of them too.

We develop Emacs for those who like it and want to use it, regardless
of their reasons.  Why people like Emacs and use it is not relevant to
this discussion, because no matter what we do, we cannot change the
basic facets and look-and-feel of Emacs and its usage.  Whoever likes
it, for whatever reasons, will keep liking it, and those who don't
will only rarely change their minds.

> > No matter how small we make Emacs, no one will ever be able to
> > understand all of it.  It spans and uses too many too wide areas of
> > computer processing technologies for any single person to be able to
> > understand it.  Apart of the "usual" CS stuff, we have Unicode and
> > character properties, GUI and TTY display, text shaping, font
> > selection, image processing and display, Lisp and its compilation,
> > file I/O, regular expressions, and that's just a sample.  Who could
> > possible be an expert in all of that?
> >
> > So this is a red herring.
> 
> Is it though?  Vim's Bram Moolenaar seems to manage it, not to mention
> how GNU Emacs came to be; there are many more impressive software out
> there with this singular dedicated developer.

I don't know how Vim compares to Emacs from the technological POV, I
don't have time to study Vim that thoroughly.  Maybe Vim is less
complex and implements fewer technologies by itself, maybe the Vim
developer is much more talented and/or has more time than we do, maybe
something else.  One thing is sure: with Emacs, with the kind of core
developers we get for the past 30 years, this didn't happen even once,
and so I consider it impractical to hope that it can be done.  And
rational project management doesn't build on hopes for miracles.

> Aiming for a single person is indeed too bold, reducing this number
> to a few on the other hand almost feels like a low hanging fruit.

Once you have 2 or three people, it doesn't matter how many they are:
you need to discuss changes that span more than one area (and almost
all of them do), you need to be able to do forensics on code for which
you have no experts on board, etc.  This is where we are, and no
reasonable downscaling will ever be able to change that.

> my impression is that you underestimate the benefits a less complex
> project will bring

On what is that impression based?

It goes without saying that a significantly smaller project could be
easier to manage, but the important question is: how much smaller can
we make Emacs before it ceases being Emacs?  And my answer to that is
"not much".  Just review all the Lisp packages that currently see a
lot of commits, and you will arrive at the same conclusion: they are
all important.  (Packages that don't see changes aren't contributing
to difficulties in maintenance.)

> I actually agree on this mostly, what I try to suggest is to narrowing
> the scope of these applications, which must be maintained by
> emacs-devel.

It cannot be done.  If you narrow the scope, you lose the motivation
for development.  More importantly, you will begin losing
contributors, most of which are only interested in application levels,
and that is the slippery slope to death, because contributors are the
source of future Emacs developers and maintainers.

> Bidirectional editing, advanced support for programming languages
> etc. these are all great and must have features.  If something isn't
> needed by almost every programmer or writer, package it; this seems
> like a good rule of thumb?

We already make decisions based on our gut feeling what is and what
isn't important.  So if you agree with that in general, what is left
is differences in judgment calls, and those can never be usefully
argued about, because they are basically subjective.

And our experience is that the decision "what is needed by every
programmer or writer" is very non-trivial.  Emacs is so vast and
people's interests and needs are so diverse that one person's main
package in Emacs could be almost useless to someone else.  E.g., there
are people who will say that they use Emacs only because it has Org,
and others who don't use Org at all, but cannot imagine their lives
without VC or Tramp or Dired.

> I do not understand what live-ness or dead-ness mean for Emacs' case,
> as long as FSF and its support lives.

Death means slowdown of development rate, stagnation, loss of users
due to lack of maintenance, and then the project goes dark.  Cf
XEmacs.

> My overall argument here is to demonstrates why I think emacs-devel's
> methods are not as effective as they think, and should be more open to
> different approaches.  Being in this state of being immortal (or
> arguably already dead) gives you unique opportunities to be more
> experimental I think.

We do experiment, only somewhat cautiously.  This isn't an educational
student project; we have a lot of users who don't want us to make
fatal mistakes.  That's a huge responsibility.

> IIUC the vision is having as many features (that mostly work) as
> possible, instead of few superb ones.

But that's not the vision.  The vision is to have as many features we
_need_to_have_in_core_ as possible.  That's a tautological definition,
of course, like every good definition is ("GNU's not Unix" and all
that), which again brings us to the crucial point: it's a judgment
call what to include and what not to include, which technology to
adopt and which not.  That's a very large portion of what Emacs
maintenance is all about, day in and day out.

Re: Adding use-package to core

[Sean Whitton]

Hello,

On Mon 14 Nov 2022 at 08:27AM +08, Po Lu wrote:

> Personally, I hope that everything most people find useful will
> eventually make its way into Emacs, because doing so is a direct
> shortcut to making Emacs more useful for everyone.

We are still aiming to bundle all of GNU ELPA with releases of Emacs at
some point, right?

-- 
Sean Whitton

Re: Adding use-package to core

[Philip Kaludercic]

Sean Whitton <spwhitton@spwhitton.name> writes:

> Hello,
>
> On Mon 14 Nov 2022 at 08:27AM +08, Po Lu wrote:
>
>> Personally, I hope that everything most people find useful will
>> eventually make its way into Emacs, because doing so is a direct
>> shortcut to making Emacs more useful for everyone.
>
> We are still aiming to bundle all of GNU ELPA with releases of Emacs at
> some point, right?

Was that the plan, or to have a separate release of Emacs (fat Emacs)
with all GNU ELPA packages installed by default?

Either way, I wonder how well that would work.

RE: [External] : Re: Adding use-package to core

[Drew Adams]

> > We are still aiming to bundle all of GNU ELPA with releases of Emacs at
> > some point, right?
> 
> Was that the plan, or to have a separate release of Emacs (fat Emacs)
> with all GNU ELPA packages installed by default?
> 
> Either way, I wonder how well that would work.

IIRC, there once was a CONTRIB directory delivered
as part of Emacs.  But that was long, long ago.

And maybe it wasn't delivered with GNU Emacs per se
(dunno) but was instead set up here and there in
various labs, universities, etc.  I think it was
provided with GNU Emacs, but I may be wrong.

Re: Adding use-package to core

[Eli Zaretskii]

> From: Sean Whitton <spwhitton@spwhitton.name>
> Cc: xenodasein--- via "Emacs development discussions."
>  <emacs-devel@gnu.org>,  eliz@gnu.org,  xenodasein@tutanota.de
> Date: Fri, 18 Nov 2022 12:29:25 -0700
> 
> Hello,
> 
> On Mon 14 Nov 2022 at 08:27AM +08, Po Lu wrote:
> 
> > Personally, I hope that everything most people find useful will
> > eventually make its way into Emacs, because doing so is a direct
> > shortcut to making Emacs more useful for everyone.
> 
> We are still aiming to bundle all of GNU ELPA with releases of Emacs at
> some point, right?

Not all of it, no.  Only some of it.

Re: Adding use-package to core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Sean Whitton <spwhitton@spwhitton.name>
>> Cc: xenodasein--- via "Emacs development discussions."
>>  <emacs-devel@gnu.org>,  eliz@gnu.org,  xenodasein@tutanota.de
>> Date: Fri, 18 Nov 2022 12:29:25 -0700
>> 
>> Hello,
>> 
>> On Mon 14 Nov 2022 at 08:27AM +08, Po Lu wrote:
>> 
>> > Personally, I hope that everything most people find useful will
>> > eventually make its way into Emacs, because doing so is a direct
>> > shortcut to making Emacs more useful for everyone.
>> 
>> We are still aiming to bundle all of GNU ELPA with releases of Emacs at
>> some point, right?
>
> Not all of it, no.  Only some of it.

What would the criteria for inclusion be like?

Re: Adding use-package to core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Sean Whitton <spwhitton@spwhitton.name>,  luangruo@yahoo.com,
>   emacs-devel@gnu.org,  xenodasein@tutanota.de
> Date: Fri, 18 Nov 2022 20:43:33 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Sean Whitton <spwhitton@spwhitton.name>
> >> Cc: xenodasein--- via "Emacs development discussions."
> >>  <emacs-devel@gnu.org>,  eliz@gnu.org,  xenodasein@tutanota.de
> >> Date: Fri, 18 Nov 2022 12:29:25 -0700
> >> 
> >> Hello,
> >> 
> >> On Mon 14 Nov 2022 at 08:27AM +08, Po Lu wrote:
> >> 
> >> > Personally, I hope that everything most people find useful will
> >> > eventually make its way into Emacs, because doing so is a direct
> >> > shortcut to making Emacs more useful for everyone.
> >> 
> >> We are still aiming to bundle all of GNU ELPA with releases of Emacs at
> >> some point, right?
> >
> > Not all of it, no.  Only some of it.
> 
> What would the criteria for inclusion be like?

Packages that we'd like to have in Emacs, but for some reason are on
ELPA instead.  This would allow packages like Magit, Org, project.el,
and maybe others to stay only on ELPA.  Which is why including in a
release packages from ELPA is something we'd like to be able to do in
the first place.

ELPA admits packages that do the same job as other packages, and also
packages that do some jobs that are extremely niche jobs.  So
including everything makes very little sense to me.

Re: Adding use-package to core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Sean Whitton <spwhitton@spwhitton.name>,  luangruo@yahoo.com,
>>   emacs-devel@gnu.org,  xenodasein@tutanota.de
>> Date: Fri, 18 Nov 2022 20:43:33 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Sean Whitton <spwhitton@spwhitton.name>
>> >> Cc: xenodasein--- via "Emacs development discussions."
>> >>  <emacs-devel@gnu.org>,  eliz@gnu.org,  xenodasein@tutanota.de
>> >> Date: Fri, 18 Nov 2022 12:29:25 -0700
>> >> 
>> >> Hello,
>> >> 
>> >> On Mon 14 Nov 2022 at 08:27AM +08, Po Lu wrote:
>> >> 
>> >> > Personally, I hope that everything most people find useful will
>> >> > eventually make its way into Emacs, because doing so is a direct
>> >> > shortcut to making Emacs more useful for everyone.
>> >> 
>> >> We are still aiming to bundle all of GNU ELPA with releases of Emacs at
>> >> some point, right?
>> >
>> > Not all of it, no.  Only some of it.
>> 
>> What would the criteria for inclusion be like?
>
> Packages that we'd like to have in Emacs, but for some reason are on
> ELPA instead.  This would allow packages like Magit, Org, project.el,
> and maybe others to stay only on ELPA.  

1. project.el and Org is already included, even developed in Emacs?

2. Are we talking about GNU ELPA or both NonGNU ELPA and GNU ELPA.
   Because fro what I understand, Magit would require a copyright
   exception to be added.

Perhaps we can take a look at the results of the Emacs Survey, when that
comes out later this month and collect a list of popular contenders?

>                                         Which is why including in a
> release packages from ELPA is something we'd like to be able to do in
> the first place.
>
> ELPA admits packages that do the same job as other packages, and also
> packages that do some jobs that are extremely niche jobs.  So
> including everything makes very little sense to me.

I agree.

Re: Adding use-package to core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: spwhitton@spwhitton.name,  luangruo@yahoo.com,  emacs-devel@gnu.org,
>   xenodasein@tutanota.de
> Date: Sat, 19 Nov 2022 07:33:30 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> What would the criteria for inclusion be like?
> >
> > Packages that we'd like to have in Emacs, but for some reason are on
> > ELPA instead.  This would allow packages like Magit, Org, project.el,
> > and maybe others to stay only on ELPA.  
> 
> 1. project.el and Org is already included, even developed in Emacs?

The intent was to leave them only on ELPA when the way of including
ELPA packages in a release is figured out.  (And Org is definitely NOT
developed in Emacs.)

> 2. Are we talking about GNU ELPA or both NonGNU ELPA and GNU ELPA.

GNU ELPA only.

> Perhaps we can take a look at the results of the Emacs Survey, when that
> comes out later this month and collect a list of popular contenders?

We could do that, but until we have a reliable way of including ELPA
packages in a release (which should include the solution for how to
upgrade such packages after the released Emacs is installed on the
user's machine), doing these secondary jobs is IMO just a waste of
time and energy.  For example, if the solutions are far away in the
future, the list of contenders you collect now will be outdated by
then, and will need to be redone anew.

The issues we are touching here were all discussed in the past, and
the difficulties that need to be resolved were described and also
discussed.  It's nothing new, and I don't think anything's changed
since those discussions, we are still where we were back then wrt our
ability to include ELPA packages.

Re: Adding use-package to core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: spwhitton@spwhitton.name,  luangruo@yahoo.com,  emacs-devel@gnu.org,
>>   xenodasein@tutanota.de
>> Date: Sat, 19 Nov 2022 07:33:30 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> What would the criteria for inclusion be like?
>> >
>> > Packages that we'd like to have in Emacs, but for some reason are on
>> > ELPA instead.  This would allow packages like Magit, Org, project.el,
>> > and maybe others to stay only on ELPA.  
>> 
>> 1. project.el and Org is already included, even developed in Emacs?
>
> The intent was to leave them only on ELPA when the way of including
> ELPA packages in a release is figured out.  (And Org is definitely NOT
> developed in Emacs.)

Of course.

>> 2. Are we talking about GNU ELPA or both NonGNU ELPA and GNU ELPA.
>
> GNU ELPA only.

OK, but then again, what to do about Magit being on NonGNU ELPA.

>> Perhaps we can take a look at the results of the Emacs Survey, when that
>> comes out later this month and collect a list of popular contenders?
>
> We could do that, but until we have a reliable way of including ELPA
> packages in a release (which should include the solution for how to
> upgrade such packages after the released Emacs is installed on the
> user's machine), doing these secondary jobs is IMO just a waste of
> time and energy.  For example, if the solutions are far away in the
> future, the list of contenders you collect now will be outdated by
> then, and will need to be redone anew.
>
> The issues we are touching here were all discussed in the past, and
> the difficulties that need to be resolved were described and also
> discussed.  It's nothing new, and I don't think anything's changed
> since those discussions, we are still where we were back then wrt our
> ability to include ELPA packages.

What exactly is the complication here?  Wouldn't it be possible to have
a "contrib"/"elpa"/... directory under lisp with ELPA packages that are
prepared before packaging?  Or should these packages be moved into the
core?  Would using fancy tricks like git worktrees, as done by
elpa-admin be a possible approach to tackle the issue?

Re: Adding use-package to core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: spwhitton@spwhitton.name,  luangruo@yahoo.com,  emacs-devel@gnu.org,
>   xenodasein@tutanota.de
> Date: Sat, 19 Nov 2022 10:09:37 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> 2. Are we talking about GNU ELPA or both NonGNU ELPA and GNU ELPA.
> >
> > GNU ELPA only.
> 
> OK, but then again, what to do about Magit being on NonGNU ELPA.

AFAIU, an effort to collect all the assignments for Magit is under
way, with the purpose of adding Magit to ELPA and/or Emacs.

> > We could do that, but until we have a reliable way of including ELPA
> > packages in a release (which should include the solution for how to
> > upgrade such packages after the released Emacs is installed on the
> > user's machine), doing these secondary jobs is IMO just a waste of
> > time and energy.  For example, if the solutions are far away in the
> > future, the list of contenders you collect now will be outdated by
> > then, and will need to be redone anew.
> >
> > The issues we are touching here were all discussed in the past, and
> > the difficulties that need to be resolved were described and also
> > discussed.  It's nothing new, and I don't think anything's changed
> > since those discussions, we are still where we were back then wrt our
> > ability to include ELPA packages.
> 
> What exactly is the complication here?  Wouldn't it be possible to have
> a "contrib"/"elpa"/... directory under lisp with ELPA packages that are
> prepared before packaging?  Or should these packages be moved into the
> core?  Would using fancy tricks like git worktrees, as done by
> elpa-admin be a possible approach to tackle the issue?

The basic issue is: when I install a new Emacs version, and later want
to upgrade to a newer version of a package maintained on ELPA, how
does Emacs do that, and what are user-level implications, given the
existing methods of installing Emacs, both by building it locally and
installing a distro?

Again, the details of this were described and discussed, so I suggest
to look up those past discussions and read there.

Re: Adding use-package to core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: spwhitton@spwhitton.name,  luangruo@yahoo.com,  emacs-devel@gnu.org,
>>   xenodasein@tutanota.de
>> Date: Sat, 19 Nov 2022 10:09:37 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> 2. Are we talking about GNU ELPA or both NonGNU ELPA and GNU ELPA.
>> >
>> > GNU ELPA only.
>> 
>> OK, but then again, what to do about Magit being on NonGNU ELPA.
>
> AFAIU, an effort to collect all the assignments for Magit is under
> way, with the purpose of adding Magit to ELPA and/or Emacs.

That I did not know, but I can't find anything on the topic online either.

>> > We could do that, but until we have a reliable way of including ELPA
>> > packages in a release (which should include the solution for how to
>> > upgrade such packages after the released Emacs is installed on the
>> > user's machine), doing these secondary jobs is IMO just a waste of
>> > time and energy.  For example, if the solutions are far away in the
>> > future, the list of contenders you collect now will be outdated by
>> > then, and will need to be redone anew.
>> >
>> > The issues we are touching here were all discussed in the past, and
>> > the difficulties that need to be resolved were described and also
>> > discussed.  It's nothing new, and I don't think anything's changed
>> > since those discussions, we are still where we were back then wrt our
>> > ability to include ELPA packages.
>> 
>> What exactly is the complication here?  Wouldn't it be possible to have
>> a "contrib"/"elpa"/... directory under lisp with ELPA packages that are
>> prepared before packaging?  Or should these packages be moved into the
>> core?  Would using fancy tricks like git worktrees, as done by
>> elpa-admin be a possible approach to tackle the issue?
>
> The basic issue is: when I install a new Emacs version, and later want
> to upgrade to a newer version of a package maintained on ELPA, how
> does Emacs do that, and what are user-level implications, given the
> existing methods of installing Emacs, both by building it locally and
> installing a distro?

Ok, I get that.

> Again, the details of this were described and discussed, so I suggest
> to look up those past discussions and read there.

I'll try that too.  Do you have any specific discussions in mind.
Searching the archive can be a bit difficult when looking for general
discussions.

Re: Adding use-package to core

[Stefan Kangas]

Philip Kaludercic <philipk@posteo.net> writes:

>> AFAIU, an effort to collect all the assignments for Magit is under
>> way, with the purpose of adding Magit to ELPA and/or Emacs.
>
> That I did not know, but I can't find anything on the topic online either.

I don't think anyone is working on it.  I found this in the archives:

https://lists.gnu.org/archive/html/emacs-devel/2017-07/msg00437.html

Re: Adding use-package to core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: spwhitton@spwhitton.name,  luangruo@yahoo.com,  emacs-devel@gnu.org,
>   xenodasein@tutanota.de
> Date: Sat, 19 Nov 2022 11:02:28 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> OK, but then again, what to do about Magit being on NonGNU ELPA.
> >
> > AFAIU, an effort to collect all the assignments for Magit is under
> > way, with the purpose of adding Magit to ELPA and/or Emacs.
> 
> That I did not know, but I can't find anything on the topic online either.

Here's the last time this was discussed:

  https://lists.gnu.org/archive/html/emacs-devel/2022-09/msg00125.html

> > Again, the details of this were described and discussed, so I suggest
> > to look up those past discussions and read there.
> 
> I'll try that too.  Do you have any specific discussions in mind.
> Searching the archive can be a bit difficult when looking for general
> discussions.

I think the last time this was discussed is here:

  https://lists.gnu.org/archive/html/emacs-devel/2020-12/msg00729.html

In particular, I tried to describe the issues that bother me here:

  https://lists.gnu.org/archive/html/emacs-devel/2020-12/msg00917.html

Re: Adding use-package to core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: spwhitton@spwhitton.name,  luangruo@yahoo.com,  emacs-devel@gnu.org,
>>   xenodasein@tutanota.de
>> Date: Sat, 19 Nov 2022 11:02:28 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> OK, but then again, what to do about Magit being on NonGNU ELPA.
>> >
>> > AFAIU, an effort to collect all the assignments for Magit is under
>> > way, with the purpose of adding Magit to ELPA and/or Emacs.
>> 
>> That I did not know, but I can't find anything on the topic online either.
>
> Here's the last time this was discussed:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2022-09/msg00125.html
>
>> > Again, the details of this were described and discussed, so I suggest
>> > to look up those past discussions and read there.
>> 
>> I'll try that too.  Do you have any specific discussions in mind.
>> Searching the archive can be a bit difficult when looking for general
>> discussions.
>
> I think the last time this was discussed is here:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2020-12/msg00729.html
>
> In particular, I tried to describe the issues that bother me here:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2020-12/msg00917.html

Thanks!

Re: Adding use-package to core

[Stefan Monnier]

> packages in a release (which should include the solution for how to
> upgrade such packages after the released Emacs is installed on the
> user's machine),

FWIW, this has been a solved problem already in Emacs-24: package.el has
always been able to handle the presence of both system-wide packages and
user-installed packages and to pick the most recent version.


        Stefan

Re: Adding use-package to core

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> packages in a release (which should include the solution for how to
>> upgrade such packages after the released Emacs is installed on the
>> user's machine),
>
> FWIW, this has been a solved problem already in Emacs-24: package.el has
> always been able to handle the presence of both system-wide packages and
> user-installed packages and to pick the most recent version.

This is what has been confusing me about Eli's comments.  Debian does
this all the time, there are a great number of packages that can be
installed on a system-wide basis:

--8<---------------cut here---------------start------------->8---
$ apt search elpa-

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Sorting...
Full Text Search...
elpa-a/testing 1.0.0-2 all
  functions for dealing with associative structures

elpa-ac-rtags/testing 2.38-6 all
  auto-complete back-end for RTags

elpa-ace-link/testing 0.5.0-3 all
  selecting a link to jump to

elpa-ace-popup-menu/testing 0.2.1-3 all
  replace GUI popup menu with something more efficient

elpa-ace-window/testing 0.10.0-1 all
  selecting a window to switch to
--8<---------------cut here---------------end--------------->8---

That is also why, in my eyes the issue is reduced to the matter of
deciding if the selected ELPA packages are bundled in Git (e.g. as
worktrees) or when the tarballs are generated (that distributions use to
build their packages).

But I am not done with reading through the archives, so I might just be
repeating stuff pointlessly.

Re: Adding use-package to core

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Philip Kaludercic <philipk@posteo.net>,  spwhitton@spwhitton.name,
>   luangruo@yahoo.com,  emacs-devel@gnu.org,  xenodasein@tutanota.de
> Date: Sat, 19 Nov 2022 10:28:56 -0500
> 
> > packages in a release (which should include the solution for how to
> > upgrade such packages after the released Emacs is installed on the
> > user's machine),
> 
> FWIW, this has been a solved problem already in Emacs-24: package.el has
> always been able to handle the presence of both system-wide packages and
> user-installed packages and to pick the most recent version.

It might be _almost_ solved, but I'm not sure it is 100% solved.  We
need a clear description of:

  . how are such packages installed when a release tarball is built
    and installed ("how" here means where and with which artifacts)
  . how will this work in a built but uninstalled source tree
  . how are such packages installed from a distro
  . how can they be upgraded from ELPA and downgraded back to the
    version that came with the tarball

I asked this question in the Dec 2020 discussion, and you responded
here:

  https://lists.gnu.org/archive/html/emacs-devel/2020-12/msg00923.html

The response triggered a few followup questions, and I don't think we
have satisfactory, let alone final, answers for all of those
questions.

I'm not saying this part of the issue is rocket science, but it does
have to be figured out.

Re: Adding use-package to core

[Stefan Monnier]

> Packages that we'd like to have in Emacs, but for some reason are on
> ELPA instead.

Yup.  I think it would grow a bit in the same kind of way that we've
added packages to Emacs in the past with similar tradeoffs between
offering a better "out of the box" experience and increasing the size of
the Emacs tarball too much.

> ELPA admits packages that do the same job as other packages, and also
> packages that do some jobs that are extremely niche jobs.
> So including everything makes very little sense to me.

Agreed.  Also there's the question of code quality.

Think of it a bit like NonGNU ELPA: in theory we could add almost all
"Melpa packages to NonGNU ELPA" (aka "GNU ELPA packages to Emacs"), but
instead we add them one by one.


        Stefan