Re: Friendly discussion about (package-initialize)

[Radon Rosborough <radon.neon@gmail.com>]

> > The problem is that even if I don't use package.el, there may be
> > some stuff left in ~/.emacs.d/elpa from previous times.
>
> Then don't do that:

I'm actually not sure what "that" is referring to here. Sorry if it is
obvious.

> we want to make it as straightforward as possible for users to
> install&use packages from GNU ELPA, so automatic activation of those
> packages that were installed is a prerequisite.

Agreed, but it doesn't have to be done in this way. We could
accomplish the same thing simply by providing a template init-file,
without any of the problems I've mentioned.

Is there any particular reason why providing a template init-file
would be a worse solution than modifying the init-file on the fly? If
so, could you explain to me what it is?

Let me reiterate: providing a template configuration file is an
*extremely common pattern*, and it's even common for the default
behavior of a program (when run with no configuration file) to be
generally considered unacceptable. The principle of least surprise
suggests that Emacs should use the same system as every other piece of
software with this kind of problem uses.

> Inevitably there will be situations where this design goal will
> clash with the end-user who wants to use something else and will
> want to explicitly "disable" package.el.

I'm fine with disabling package.el being an explicit step. Not with it
being an ongoing battle (where package.el strikes back every time I
accidentally use one of its functions without the proper advices
defined). I would be totally satisfied with there being package.el
code in the template init-file and no mention of alternative package
managers, as long as it ends with the template init-file (i.e. it's a
one-time thing).

> Do you mean that it would be worse, or that it would be better but
> still not good enough?

Better but still not good enough. I would consider modifying the
init-file automatically at startup as "atrocious" and doing it at
other times "undesirable but acceptable if it's really the only
solution".

> Based on your earlier message, I'm pretty sure I will not come up
> with a solution which you really like, so I'm only aiming to get
> "better" rather than "good enough".

That's OK. I certainly don't believe it's the responsibility of the
Emacs community to make me happy; sorry if I gave that implication.

> > Would you consider it an acceptable solution to pop up a window or
> > display a message telling the user that they should put
> > `package-initialize' in their init-file, provided that we didn't
> > have it get called during init?

> No,

Yeah, a popup window would be very annoying. It was just an offhand
suggestion. But consider that package.el doesn't currently display any
useful message in the echo area after it's finished (just whatever
byte-compilation message happened to be last). I don't think it would
be a degredation to user experience to have it say

    Finished installing 2 packages

and then if package-initialize wasn't called during init,

    Finished installing 2 packages, but please run
    M-x package-ensure-init-file to finalize installation

or something like that. Perhaps a *Warning* would be appropriate? As
long as the message conveyed a way to find out more details, and a
quick fix (I'm fine with advising to use a function that modifies the
init-file, as long as that's an intentional action).

I'm partially playing devil's advocate here; I really think the
template init-file is the way to go. But even if we do use the
template init-file, some better messaging would still be nice, I
think.

> (all of whom likely already know several different ways to work
> around the current annoyance)

Based on the Reddit thread I linked, most people who encounter this
problem don't know how to fix it. The ones who do spent an hour or two
figuring out how to do so because there's no documentation.

> But if the call is only inserted during package-install it means
> that you'd need both:
> - use package-install
> - in a session where your init failed to call package-initialize
> for the problem to show up. This combination seems a lot less likely
> than the current one.

This happened to me repeatedly when I was testing package managers,
since I obviously wanted to test installing packages in an environment
where they weren't already activated by my primary package manager. I
do agree that this situation is uncommon; it just strikes me that the
mechanism currently in use is rather fragile if it "breaks" in such a
situation.

> Another thing we could consider is to drop the automatic call to
> package-initialize in lisp/startup.el (again, based on the idea that
> this has now been made unnecessary by package--ensure-init-file).

I am strongly in support of this unless it means that we will be more
reluctant to eventually remove `package--ensure-init-file'. Almost all
of the tangible problems I mentioned in my original email stem from
package.el modifying the init-file on the fly; arguing for the
elimination of this behavior is therefore my highest priority.

> So it seems like you're thinking of another kind of "plain Emacs but
> not in `emacs -Q`".

No, I was thinking of the same situation. I know I have been bitten by
this before, but thinking back on it I can't say that any of my use
cases there are actually common in any way. Let's pretend I didn't
make that point; I don't think it is useful.

> >> I'd be very interested to work with maintainers of other package
> >> managers to see how we could make them better interoperate (e.g.
> >> make it possible to install with one tool, but activate&config with
> >> another).
> > Sounds like a great idea. However, do note that this will only be
> > useful for package managers that use the package.el format (e.g.
> > Quelpa, Cask, Pallet) and not for others (e.g. el-get, Borg,
> > straight.el).
>
> Part of "make them interoperate" may involve changing the formats
> accepted&|used by the various tools, indeed (on-disk, on-the-server,
> ...).

Note that source-based package managers such as Borg and straight.el
are fundamentally incompatible with the package.el format (or,
depending on your perspective, the package.el format is fundamentally
incompatible with source-based package managers such as Borg and
straight.el). el-get might benefit from this improved
interoperability, though it is an interesting case as it's the only
package manager I know of that uses both package.el and non-package.el
formats for packages.

Best,
Radon