RE: Summary and next steps for (package-initialize)

[Drew Adams <drew.adams@oracle.com>]

> Let me clarify. I see the following spectrum in this debate:
> 
>     People who want package.el to be enabled by default in spite of
>     the resulting UX problems (too much magic, automatic modification
>     of init-file).
> 
>     ^
>     | (a) Have Emacs write (package-initialize) into the existing
>     |     init-file at startup.
>     |
>     | (b) Have Emacs create a template init-file with
>     |     (package-initialize) at startup, if one doesn't exist
>     |     already.
>     |
>     | (c) Don't create (package-initialize) calls anywhere, but still
>     |     call package-initialize in startup.el.
>     |
>     | (d) Don't call (package-initialize) anywhere; the user must call
>     |     it manually in their init-file.
>     v
> 
>     People who want package.el to be disabled by default in spite of
>     the resulting UX problems (package management system doesn't work
>     out of the box).
> 
> You and I are both at point (d). However, this is a rather extreme
> position, so I am not arguing for point (d) or even point (c), but
> rather point (b), because I view point (b) as an improvement over
> point (a), which is the current situation.
> 
> Most people on this thread view any solution that doesn't involve
> package.el working out of the box as a non-starter. That is why I am
> not proposing such a solution. Instead, I am proposing a solution
> which has much less potential for annoyance, while still having
> package.el work out of the box.
> 
> Does this clear up the confusion?

I guess so, and I appreciate the time you spent spelling that out.

What I would actually prefer, IIUC, is both of these at the same time:

* package.el to be enabled  by default
* package.el to be disabled by default

;-)

Rather, as a user I don't want to have to prescribe whether
package.el should be enabled or disabled, by default or
otherwise.  I don't want to have to care about package.el.

I would like accessing packages to just happen, when/if I
ask to access them - e.g. if I do `M-x list-packages'.

And if I have updated whatever persistent info is needed for
it, I would like Emacs to automatically enable this or that
package when I start Emacs.  It should be easy for me to
tell Emacs that I prefer to automatically enable this or that
package on startup.

I would want something like what exists for bookmark.el.
If/when you try to access bookmarks then your bookmarks file
is loaded automatically.  It's on-demand, but your bookmarks
file is separate from your init file.

> > > I agree wholeheartedly with everything you have said. However, a
> > > significant number of people on this list disagree,
> >
> > Hm.  I haven't read this thread very carefully.  But I also
> > haven't gotten the impression that a significant number of
> > people here have argued for generating init files.
> 
> You're right. I meant that you're arguing that Emacs shouldn't
> go through contortions to make package.el work out of the box,

Not sure what you mean.  I don't care whether Emacs goes
through such contorsions.  I don't think users should go
through contorsions.

If by Emacs going through contortions you mean Emacs adding
stuff to user init files or creating new init files, then yes,
I don't think Emacs should do that.

If by Emacs going through contortions you mean that it does
some work under the covers then no, I don't care whether it
does that.

> and people disagree with that. Meanwhile I'm just trying
> to make the contortions less horrifying, as an immediate
> improvement to the current situation.
> 
> > So I can't say whether your proposal is better or is a
> > compromise of some sort.
> 
> The part of my proposal that is relevant to you is the part where
> Emacs doesn't try to write (package-initialize) into your init-file
> automatically. Do you consider that an improvement?

I've already said that I don't think Emacs should write to
init files.  (And I don't think it should have to do that.
Why is that considered a hard requirement?)

But I've also said that if the package system needs to
record some info persistently for a user then it should use
a different file from the init file.  That doesn't mean that
I disagree with Emacs writing such info locally.  It just
means hands-off init files, please.

> > I don't like the idea of Emacs generating an init file if there is
> > none OR of Emacs messing with an existing init file.
> 
> Same here. Unfortunately, the technical challenges in making
> package.el work out of the box pretty much guarantee we have
> to do one of these.

I don't see why.  But I don't pretend to be familiar with the
technical challenges.  If it can do what it needs to do by
writing to an init file then why can't it do what it needs to
do by using a separate file?

Is the problem the difficulty for a user to determine where
in the init file to load the separate file?  If so, then
Emacs can perhaps detect bad placement - not by examining
the init file but by the effects of this or that inappropriate
placement.  E.g., if it is loaded before XYZ has been defined
or initialized then tell the user to move it etc.

> Now, whether or not it's worth all this trouble to make
> package.el work out of the box is another debate, but that's a much
> harder debate than this one, which is why I'm not arguing it (at least
> for now).
> 
> > I don't see your proposal as a "middle way", for users.
> 
> It's a middle way because creating a default init-file is less
> intrusive than modifying an existing one, but still more intrusive
> than doing nothing at all.

Debatable whether it is less bothersome/intrusive for a user.
In any case, I don't argue that Emacs must do nothing at all.
I argue (admittedly naively) that Emacs should keep its hands
off init files; that's all.
 
> > Naively, I'd suggest that package.el should do what custom.el
> > and bookmark.el and lots of other libraries do: Use a separate
> > Lisp file for persisting info the library needs - a file whose
> > name/location the user can set using an option.
> >
> > If that would mean that a user might need to load that file
> > at an appropriate place in an init file, so be it.  The
> > package system might well require some understanding of it
> > before using it.
> 
> Again, agreed -- but emacs-devel does not want the user to have to
> have any understanding of the package system before using it, so I'm
       ^^^
> not making that argument.

I'll make the argument.  If Emacs cannot make it easy to use
packages without it writing to init files then it sounds like
a design flaw and time to get back to the drawing board.

> (By the way, you're spot on with the separate Lisp file idea -- in
> fact, package.el already does this, and the separate Lisp file
> [directory] is ~/.emacs.d/elpa. This debate is over how to force users
> to use package.el by activating it automatically. Again, while neither
> of us thinks such a thing should be done, I think it's more realistic
> to try to get it done in the least bad way possible, at least for
> now.)

I see.  Too bad.

> > FWIW, I am also not in favor of Customize writing to your
> > init file.
>
> Same.
> 
> > If it is difficult for users to find an appropriate place in
> > their init file to load whatever state/settings file package.el
> > might require, or to call `package-initialize' or whatever,
> > then perhaps the package system could be improved to help with
> > that somehow.
> 
> The solution here is improving the documentation of package.el,
> which I've already suggested.

Even without being familiar with that doc, I can pretty much
blindly guess "+1" to that suggestion.

But as someone will no doubt point out, lots of users
(especially nowadays) do not read doc.  Create a good video
presenting the same relevant information and that might be
all that's needed.

That plus making the package system perform some checks to
recognize problematic situations resulting from inappropriate
use - and then point to relevant videos/doc etc.

IOW, if this is about pilot error then (1) improve the pilot
doc and (2) add some indicator lights or announcements that
pop up telling a pilot about a mistake s?he made and how to
correct it.

> > I say "naively" because I know next to nothing about
> > package.el, and I'm not trying to design the package system.
> 
> As someone who has in fact already written an Emacs package manager
> from scratch (straight.el), I think all your points are spot on.
> Thanks for your comments.

Thank you for your explanations.