* Proposals for fixing package-initialize
@ 2017-09-04 5:31 Radon Rosborough
2017-09-04 12:47 ` Drew Adams
` (4 more replies)
0 siblings, 5 replies; 9+ messages in thread
From: Radon Rosborough @ 2017-09-04 5:31 UTC (permalink / raw
To: emacs-devel
Hello all,
You may have seen slightly over 150 emails fly by in the last few
weeks talking about package-initialize. Hilariously, we still haven't
managed to reach a conclusion.
This email is an attempt to present all the proposals that have been
discussed, in an even-handed way, so that we can make a decision
without everyone needing to have read the aforementioned 150 emails.
I will *not* be providing any context on the issue; see [1] for that,
but note that the discussion branched out so much that everyone's
positions are now completely different :P
==> Proposal A: Leave it alone
Summary:
- call package-initialize in startup.el after loading the init-file
- when package-initialize is called, also insert a call to it into
the init-file, if one was not found there
- this is the current behavior
Advantages:
- requires no work
- people are used to the current behavior
- backwards compatible
Disadvantages:
- modifying the user's init-file is ugly
- doing it automatically is annoying and possibly dangerous
- if the user has customized package.el, then Emacs will put
package-initialize in the wrong place
- this type of behavior is not common practice
==> Proposal B: Add a second init-file
Summary:
- add a second init-file, ~/.emacs.d/early-init.el
- this file is sourced earlier than init.el, during startup.el
- move the call to package-initialize in startup.el from its current
location to between early-init.el and startup.el
- don't have package.el modify the init-file ever
Advantages:
- does not require modifying the user's init-file
- package.el still works out of the box
- the only people affected by the change are the ones who have
customized package.el
- foolproof, never does the wrong thing (except for advanced
existing configs that customize package.el)
- people can also use early-init.el to disable things like the tool
bar before they are loaded in the first place: two birds with one
stone
- the only people who need to know about early-init.el are advanced
users who want to customize package.el
- Customize works both for customizing package.el and also for
customizing packages
- does not require package-initialize to be put in the user's
init-file, not even manually
- makes it easy to disable package.el permanently
Disadvantages:
- adds another init-file, which is additional complexity
- we must add a mechanism to Customize that allows to specify the
file into which a particular variable should be saved, so
package.el can still be customized
==> Proposal B': Add an early-init special form
Summary:
- same as above, but instead of a separate file, allow init.el to
start with a special (with-early-config ...) form
- if with-early-config form is found in init-file, load it when
early-init.el would be loaded
Advantages:
- we still only have one init-file
Disadvantages:
- the startup sequence is a little weird
- it's not clear how this would work with Customize
==> Proposal C: Add a template init-file
Summary:
- if Emacs is started without an init-file, generate one from a
template
- the template init-file has a call to package-initialize in it, and
some comments explaining where to put package and package.el
configuration
- the template init-file is not generated in 'emacs -Q' mode
- get rid of package.el modifying the init-file
Advantages:
- does not require modifying the user's init-file
- package.el still works out of the box for new users
- we could use the template init-file for encouraging users towards
better defaults without breaking backwards compatibility
Disadvantages:
- having Emacs automatically create files is not ideal
- if we decide against using the template init-file for better
defaults, then it's kind of awkward to just use it for package.el
- doesn't work for users with existing init-files
- still reqiures a call to package-initialize in the user's
init-file
==> Proposal D: Hook into error handlers
Summary:
- call package-initialize whenever a void-function or require error
is encountered
- get rid of package.el modifying the init-file
- possibly get rid of call to package-initialize in startup.el?
Advantages:
- doesn't require modifying the user's init-file
- probably package.el still works out of the box
Disadvantages:
- implications (performance, correctness, implementation details)
are completely unexplored
- not very elegant
- business logic from the package manager is intermixed with a
fundamental, generic part of the Elisp interpreter
- hard to tell when package-initialize gets called, so best practice
would probably still be to have package-initialize in the
init-file
- the fact that a core Lisp error causes lots of magic to happen
will make debugging more "fun"
==> Proposal E: Call package-initialize multiple times
Summary:
- add a second call to package-initialize before init.el is loaded
- if package-initialize is called again, and package-load-list or
other variables have been customized since the last time it was
called, unload any packages that were previously loaded that now
should not be loaded
Advantages:
- does not require modifying the user's init-file
- backwards compatible, at least if it works
Disadvantages:
- impossible to disable package.el without introducing a magic
dotfile (~/.emacs.d/.nopackage) or similar
- impossible to unload packages correctly, in general, so this
approach would be buggy in many edge cases
- severe performance regressions if many packages are disabled using
package-load-list
- "As I said, there's no doubt that do+undo is a poor way to do
nothing." "Again, here we're back to the obvious observation that
do+undo is a poor and complicated way to do nothing." "Can't
disagree that "do+undo" is a poor way to do nothing." -- Stefan
- requires extra work to make multiple calls to package-initialize
faster, not to mention figuring out how unloading is supposed to
work, or if that's even possible in principle
==> Proposal F: Use more declarative configuration
Summary:
- make it so that packages can be configured in a way that causes
package-initialize to be called automatically
Advantages:
- would solve all the problems
Disadvantages:
- nobody has any idea how to do this
- and it would probably take several years anyway
==> Proposal G: Parse the init-file
Summary:
- call package-initialize in startup.el before loading init.el
- parse the init-file in order to find out the values of
package-load-list, etc. before running package-initialize
Advantages:
- in the cases that it works, it would allow package.el to work out
of the box without the need for package-initialize in the user's
init-file
- Emacs would not need to modify the user's init-file
Disadvantages:
- would require solving the halting problem to work in general
- is a horrifying hack
==> Proposal H: Make the user do it manually
Summary:
- make it so that package.el doesn't modify the init-file, but don't
change anything else
- this is how it worked before the first "solution" to this problem
was implemented
Advantages:
- Emacs does not modify the user's init-file
Disadvantages:
- package.el does not work out of the box, since package
customizations will fail when put into the init-file
- consequently we get lots of superfluous bug reports
==> My position
I think that the above advantages and disadvantages speak for
themselves, but to be clear, I think Proposal B is the best. It has
basically no practical disadvantages, solves every problem that is
tangled up in this issue with zero migration necessary except for very
advanced users, is trivial to implement, and simultaneously solves a
large class of other problems related to disabling things that are
enabled early during startup. Every other proposal has significant
practical disadvantages.
==> Postscript
I'd like to say thank you to Eli for pushing the question of whether
it would be practical to call package-initialize only in startup.el,
Timur for questioning why a second init-file would be so bad, and
Clément for explaining to me what was wrong with my original proposal.
Their arguments resulted in the discussion of what I now think is a
better proposal than my original one.
And yes, I really did review all 150 of those emails to gather
information for this summary :o
Best regards,
Radon Rosborough
[1]: https://lists.gnu.org/archive/html/emacs-devel/2017-08/msg00154.html
^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: Proposals for fixing package-initialize
2017-09-04 5:31 Proposals for fixing package-initialize Radon Rosborough
@ 2017-09-04 12:47 ` Drew Adams
2017-09-04 16:28 ` Thien-Thi Nguyen
` (3 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Drew Adams @ 2017-09-04 12:47 UTC (permalink / raw
To: Radon Rosborough, emacs-devel
Great work summarizing and pushing the question. Thx.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Proposals for fixing package-initialize
2017-09-04 5:31 Proposals for fixing package-initialize Radon Rosborough
2017-09-04 12:47 ` Drew Adams
@ 2017-09-04 16:28 ` Thien-Thi Nguyen
2017-09-04 16:36 ` Radon Rosborough
2017-09-05 16:41 ` Howard Melman
` (2 subsequent siblings)
4 siblings, 1 reply; 9+ messages in thread
From: Thien-Thi Nguyen @ 2017-09-04 16:28 UTC (permalink / raw
To: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 2205 bytes --]
() Radon Rosborough <radon.neon@gmail.com>
() Sun, 3 Sep 2017 22:31:57 -0700
This email is an attempt to present all the proposals that
have been discussed, in an even-handed way, so that we can
make a decision without everyone needing to have read the
aforementioned 150 emails.
Cool, thanks! I now feel less guilty skimming through that
backlog. I appreciate the clear presentation style, too.
==> Proposal F: Use more declarative configuration
Summary:
- make it so that packages can be configured in a way that
causes package-initialize to be called automatically
Advantages:
- would solve all the problems
Disadvantages:
- nobody has any idea how to do this
- and it would probably take several years anyway
This strikes me as the best long-term fix (i.e., defining a data
format, thus giving space to various package manglers to bloom,
wither and die around it, improving it as time goes on).
It's hyperbole to say nobody has any idea, however -- there are
several general text-oriented serialization formats that people
have already defined (e.g., YAML, JSON, sexps). Any of these
can be easily adapted to the package-metadata domain. Re time,
granted, settling on a suitable filename extension might take
years (i'll throw out ".epdef" for "Emacs package definition"),
but i think a few weeks (months) of nailing down .epdef by one
person (serial) would inspire very rapid (re-)development of
programs afterwards (parallel) resulting in overall wall-time of
much less than several years.
Sez Perlis:
We kid ourselves if we think that the ratio of procedure to
data in an active data-base system can be made arbitrarily
small or even kept small.
so the trick for Someone will be to define ONLY a data format,
and refrain from stepping into "an active data-base system" (as
much as possible -- w/ Emacs, there be hooks a-lurking 'neath).
--
Thien-Thi Nguyen -----------------------------------------------
(defun responsep (query)
(pcase (context query)
(`(technical ,ml) (correctp ml))
...)) 748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Proposals for fixing package-initialize
2017-09-04 16:28 ` Thien-Thi Nguyen
@ 2017-09-04 16:36 ` Radon Rosborough
0 siblings, 0 replies; 9+ messages in thread
From: Radon Rosborough @ 2017-09-04 16:36 UTC (permalink / raw
To: Thien-Thi Nguyen; +Cc: emacs-devel
> It's hyperbole to say nobody has any idea, however -- there are
> several general text-oriented serialization formats that people have
> already defined (e.g., YAML, JSON, sexps). Any of these can be
> easily adapted to the package-metadata domain.
Oh, I certainly agree. The difficulty I see is that many people want
to do their package (and package manager) configuration dynamically.
For example, set package-load-list so that only packages are loaded
which aren't in some folder used for active development. Or only
interface with the package manager at arbitrary times to effect
deferred installation.
So a purely declarative format will never work, because of all the
interesting use cases it would exclude; it has to be an imperative
format that nevertheless allows Emacs some more control than the
present. Or a declarative format that allows for as much
customizability as an imperative one.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Proposals for fixing package-initialize
2017-09-04 5:31 Proposals for fixing package-initialize Radon Rosborough
2017-09-04 12:47 ` Drew Adams
2017-09-04 16:28 ` Thien-Thi Nguyen
@ 2017-09-05 16:41 ` Howard Melman
2017-09-07 15:55 ` Clément Pit-Claudel
2017-09-14 19:55 ` Ted Zlatanov
4 siblings, 0 replies; 9+ messages in thread
From: Howard Melman @ 2017-09-05 16:41 UTC (permalink / raw
To: emacs-devel
FWIW, I'm a 30+ year user of emacs. I'm comfortable writing
elisp and reading emacs elisp source but have never dived
into the C source and am a little behind on the newer
trendier packages.
When I started using package.el I ran into the common
problem and had to figure out to add (package-initialize) to
my init and thought it was clunky. When I read the reason
for me to do this I immediately wondered why it didn't work
like Proposal B. I've never felt the need to customize
package.el so I haven't had to change anything since.
But I wanted to point out another advantage of Proposal B,
it's obvious to semi-advanced users.
--
Howard
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Proposals for fixing package-initialize
2017-09-04 5:31 Proposals for fixing package-initialize Radon Rosborough
` (2 preceding siblings ...)
2017-09-05 16:41 ` Howard Melman
@ 2017-09-07 15:55 ` Clément Pit-Claudel
2017-09-14 19:55 ` Ted Zlatanov
4 siblings, 0 replies; 9+ messages in thread
From: Clément Pit-Claudel @ 2017-09-07 15:55 UTC (permalink / raw
To: emacs-devel
Incredible work. Thanks a lot!
On 2017-09-04 07:31, Radon Rosborough wrote:
> Hello all,
>
> You may have seen slightly over 150 emails fly by in the last few
> weeks talking about package-initialize. Hilariously, we still haven't
> managed to reach a conclusion.
>
> This email is an attempt to present all the proposals that have been
> discussed, in an even-handed way, so that we can make a decision
> without everyone needing to have read the aforementioned 150 emails.
>
> I will *not* be providing any context on the issue; see [1] for that,
> but note that the discussion branched out so much that everyone's
> positions are now completely different :P
>
> ==> Proposal A: Leave it alone
>
> Summary:
> - call package-initialize in startup.el after loading the init-file
> - when package-initialize is called, also insert a call to it into
> the init-file, if one was not found there
> - this is the current behavior
>
> Advantages:
> - requires no work
> - people are used to the current behavior
> - backwards compatible
>
> Disadvantages:
> - modifying the user's init-file is ugly
> - doing it automatically is annoying and possibly dangerous
> - if the user has customized package.el, then Emacs will put
> package-initialize in the wrong place
> - this type of behavior is not common practice
>
> ==> Proposal B: Add a second init-file
>
> Summary:
> - add a second init-file, ~/.emacs.d/early-init.el
> - this file is sourced earlier than init.el, during startup.el
> - move the call to package-initialize in startup.el from its current
> location to between early-init.el and startup.el
> - don't have package.el modify the init-file ever
>
> Advantages:
> - does not require modifying the user's init-file
> - package.el still works out of the box
> - the only people affected by the change are the ones who have
> customized package.el
> - foolproof, never does the wrong thing (except for advanced
> existing configs that customize package.el)
> - people can also use early-init.el to disable things like the tool
> bar before they are loaded in the first place: two birds with one
> stone
> - the only people who need to know about early-init.el are advanced
> users who want to customize package.el
> - Customize works both for customizing package.el and also for
> customizing packages
> - does not require package-initialize to be put in the user's
> init-file, not even manually
> - makes it easy to disable package.el permanently
>
> Disadvantages:
> - adds another init-file, which is additional complexity
> - we must add a mechanism to Customize that allows to specify the
> file into which a particular variable should be saved, so
> package.el can still be customized
>
> ==> Proposal B': Add an early-init special form
>
> Summary:
> - same as above, but instead of a separate file, allow init.el to
> start with a special (with-early-config ...) form
> - if with-early-config form is found in init-file, load it when
> early-init.el would be loaded
>
> Advantages:
> - we still only have one init-file
>
> Disadvantages:
> - the startup sequence is a little weird
> - it's not clear how this would work with Customize
>
> ==> Proposal C: Add a template init-file
>
> Summary:
> - if Emacs is started without an init-file, generate one from a
> template
> - the template init-file has a call to package-initialize in it, and
> some comments explaining where to put package and package.el
> configuration
> - the template init-file is not generated in 'emacs -Q' mode
> - get rid of package.el modifying the init-file
>
> Advantages:
> - does not require modifying the user's init-file
> - package.el still works out of the box for new users
> - we could use the template init-file for encouraging users towards
> better defaults without breaking backwards compatibility
>
> Disadvantages:
> - having Emacs automatically create files is not ideal
> - if we decide against using the template init-file for better
> defaults, then it's kind of awkward to just use it for package.el
> - doesn't work for users with existing init-files
> - still reqiures a call to package-initialize in the user's
> init-file
>
> ==> Proposal D: Hook into error handlers
>
> Summary:
> - call package-initialize whenever a void-function or require error
> is encountered
> - get rid of package.el modifying the init-file
> - possibly get rid of call to package-initialize in startup.el?
>
> Advantages:
> - doesn't require modifying the user's init-file
> - probably package.el still works out of the box
>
> Disadvantages:
> - implications (performance, correctness, implementation details)
> are completely unexplored
> - not very elegant
> - business logic from the package manager is intermixed with a
> fundamental, generic part of the Elisp interpreter
> - hard to tell when package-initialize gets called, so best practice
> would probably still be to have package-initialize in the
> init-file
> - the fact that a core Lisp error causes lots of magic to happen
> will make debugging more "fun"
>
> ==> Proposal E: Call package-initialize multiple times
>
> Summary:
> - add a second call to package-initialize before init.el is loaded
> - if package-initialize is called again, and package-load-list or
> other variables have been customized since the last time it was
> called, unload any packages that were previously loaded that now
> should not be loaded
>
> Advantages:
> - does not require modifying the user's init-file
> - backwards compatible, at least if it works
>
> Disadvantages:
> - impossible to disable package.el without introducing a magic
> dotfile (~/.emacs.d/.nopackage) or similar
> - impossible to unload packages correctly, in general, so this
> approach would be buggy in many edge cases
> - severe performance regressions if many packages are disabled using
> package-load-list
> - "As I said, there's no doubt that do+undo is a poor way to do
> nothing." "Again, here we're back to the obvious observation that
> do+undo is a poor and complicated way to do nothing." "Can't
> disagree that "do+undo" is a poor way to do nothing." -- Stefan
> - requires extra work to make multiple calls to package-initialize
> faster, not to mention figuring out how unloading is supposed to
> work, or if that's even possible in principle
>
> ==> Proposal F: Use more declarative configuration
>
> Summary:
> - make it so that packages can be configured in a way that causes
> package-initialize to be called automatically
>
> Advantages:
> - would solve all the problems
>
> Disadvantages:
> - nobody has any idea how to do this
> - and it would probably take several years anyway
>
> ==> Proposal G: Parse the init-file
>
> Summary:
> - call package-initialize in startup.el before loading init.el
> - parse the init-file in order to find out the values of
> package-load-list, etc. before running package-initialize
>
> Advantages:
> - in the cases that it works, it would allow package.el to work out
> of the box without the need for package-initialize in the user's
> init-file
> - Emacs would not need to modify the user's init-file
>
> Disadvantages:
> - would require solving the halting problem to work in general
> - is a horrifying hack
>
> ==> Proposal H: Make the user do it manually
>
> Summary:
> - make it so that package.el doesn't modify the init-file, but don't
> change anything else
> - this is how it worked before the first "solution" to this problem
> was implemented
>
> Advantages:
> - Emacs does not modify the user's init-file
>
> Disadvantages:
> - package.el does not work out of the box, since package
> customizations will fail when put into the init-file
> - consequently we get lots of superfluous bug reports
>
> ==> My position
>
> I think that the above advantages and disadvantages speak for
> themselves, but to be clear, I think Proposal B is the best. It has
> basically no practical disadvantages, solves every problem that is
> tangled up in this issue with zero migration necessary except for very
> advanced users, is trivial to implement, and simultaneously solves a
> large class of other problems related to disabling things that are
> enabled early during startup. Every other proposal has significant
> practical disadvantages.
>
> ==> Postscript
>
> I'd like to say thank you to Eli for pushing the question of whether
> it would be practical to call package-initialize only in startup.el,
> Timur for questioning why a second init-file would be so bad, and
> Clément for explaining to me what was wrong with my original proposal.
> Their arguments resulted in the discussion of what I now think is a
> better proposal than my original one.
>
> And yes, I really did review all 150 of those emails to gather
> information for this summary :o
>
> Best regards,
> Radon Rosborough
>
> [1]: https://lists.gnu.org/archive/html/emacs-devel/2017-08/msg00154.html
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Proposals for fixing package-initialize
2017-09-04 5:31 Proposals for fixing package-initialize Radon Rosborough
` (3 preceding siblings ...)
2017-09-07 15:55 ` Clément Pit-Claudel
@ 2017-09-14 19:55 ` Ted Zlatanov
2017-09-15 1:30 ` Radon Rosborough
4 siblings, 1 reply; 9+ messages in thread
From: Ted Zlatanov @ 2017-09-14 19:55 UTC (permalink / raw
To: emacs-devel
On Sun, 3 Sep 2017 22:31:57 -0700 Radon Rosborough <radon.neon@gmail.com> wrote:
RR> You may have seen slightly over 150 emails fly by in the last few
RR> weeks talking about package-initialize. Hilariously, we still haven't
RR> managed to reach a conclusion.
RR> This email is an attempt to present all the proposals that have been
RR> discussed, in an even-handed way, so that we can make a decision
RR> without everyone needing to have read the aforementioned 150 emails.
Thank you!
FWIW I'm in favor of Proposal F: Use more declarative configuration
Ted
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Proposals for fixing package-initialize
2017-09-14 19:55 ` Ted Zlatanov
@ 2017-09-15 1:30 ` Radon Rosborough
2017-09-15 12:35 ` Stefan Monnier
0 siblings, 1 reply; 9+ messages in thread
From: Radon Rosborough @ 2017-09-15 1:30 UTC (permalink / raw
To: emacs-devel
I'm currently in the process of working on a patch to implement
Proposal B, since it is clear how to do that whereas Proposal F is
probably a fair ways off still.
The main complexities are:
- updating Custom to allow saving variables to different files,
because package.el config now needs to go in the early init file
- moving some things around in startup.el (e.g. username validation),
because the early init file needs to be loaded early of course!
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Proposals for fixing package-initialize
2017-09-15 1:30 ` Radon Rosborough
@ 2017-09-15 12:35 ` Stefan Monnier
0 siblings, 0 replies; 9+ messages in thread
From: Stefan Monnier @ 2017-09-15 12:35 UTC (permalink / raw
To: emacs-devel
> The main complexities are:
> A- updating Custom to allow saving variables to different files,
> because package.el config now needs to go in the early init file
> B- moving some things around in startup.el (e.g. username validation),
> because the early init file needs to be loaded early of course!
Note that the A part can be done as a second step (it's not a new
problem, so the step B doesn't make the step A more urgent).
Stefan
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-09-15 12:35 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-09-04 5:31 Proposals for fixing package-initialize Radon Rosborough
2017-09-04 12:47 ` Drew Adams
2017-09-04 16:28 ` Thien-Thi Nguyen
2017-09-04 16:36 ` Radon Rosborough
2017-09-05 16:41 ` Howard Melman
2017-09-07 15:55 ` Clément Pit-Claudel
2017-09-14 19:55 ` Ted Zlatanov
2017-09-15 1:30 ` Radon Rosborough
2017-09-15 12:35 ` Stefan Monnier
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).