Package initialization

Package initialization

[Helmut Eller]

What is the official way to initialize an ELPA package?  E.g. if the
package needs add a hook somewhere or change auto-mode-alist.

The manual says this about NAME-autoload.el files:

   They are typically used to autoload the principal user commands defined
   in the package, but they can also perform other tasks, such as adding
   an element to ‘auto-mode-alist’

Which seems to suggest that one should write some
;;;###autoload (progn ...)
somewhere that does the initialization.  Is this the way it should be
done?

And what is supposed to happen if two packages make conflicting changes
to auto-mode-alist?

Helmut

Re: Package initialization

[Artur Malabarba]

> The manual says this about NAME-autoload.el files:
>
>    They are typically used to autoload the principal user commands defined
>    in the package, but they can also perform other tasks, such as adding
>    an element to ‘auto-mode-alist’
>
> Which seems to suggest that one should write some
> ;;;###autoload (progn ...)
> somewhere that does the initialization.  Is this the way it should be
> done?

Yes, that line can be added to any of the package's “.el” files.

Any code that you write there will be run when your package is
activated (not loaded). This means the code is run when Emacs starts,
(either during or after loading the init file, depending on what's in
the actual init file).

Note, however, that the package itself will not be loaded at this
time. And neither should this code load your package (or be slow in
some other way). Adding an entry to auto-mode-alist is fine.

> And what is supposed to happen if two packages make conflicting changes
> to auto-mode-alist?

If you add your entry with add-to-list, nothing unexpected will
happen. If two different packages add entries regarding the same
extension, then both entries will get added. When the user visits a
file, the entry on top (the most recently added) will be the one used,
but I suppose that's what they get for installing two major-modes for
the same file type.

Re: Package initialization

[Helmut Eller]

On Sat, Jul 18 2015, Artur Malabarba wrote:

>> Which seems to suggest that one should write some
>> ;;;###autoload (progn ...)
>> somewhere that does the initialization.  Is this the way it should be
>> done?
>
> Yes, that line can be added to any of the package's “.el” files.

The question was not if it works; the question was if this is considered
good style.  Frankly, I think this is very ugly because I can no longer
load the foo-auotload.el file without clobbering global variables.

Helmut

Re: Package initialization

[Artur Malabarba]

>>> Which seems to suggest that one should write some
>>> ;;;###autoload (progn ...)
>>> somewhere that does the initialization.  Is this the way it should be
>>> done?
>>
>> Yes, that line can be added to any of the package's “.el” files.
>
> The question was not if it works; the question was if this is considered
> good style.

As long as these autoload cookies are restricted to autoloading
commands and functions or running some very essential code (like
adding a theme to custom-theme-load-path, or adding a mode entry to
auto-mode-alist), then yes, it's good style.

> Frankly, I think this is very ugly because I can no longer
> load the foo-auotload.el file

Why would you load the file manually? It gets loaded for you.

Or are you complaining about the fact that it gets loaded
automatically? (you can disable that)

> without clobbering global variables.

Autoload cookies should not go around changing any global variable.
For instance, they should not change the fill-column or the
default-input-method. But it makes sense for them to change some
variables, like the auto-mode-alist or the custom-theme-load-path.

Re: Package initialization

[Helmut Eller]

On Sat, Jul 18 2015, Artur Malabarba wrote:

>> Frankly, I think this is very ugly because I can no longer
>> load the foo-auotload.el file
>
> Why would you load the file manually? It gets loaded for you.
>
> Or are you complaining about the fact that it gets loaded
> automatically? (you can disable that)

I'm happy to set things up manually in my .emacs; but other users prefer
that the package stuff does everything automatically.  But if everything
is in the autoload file there's no way to get the autoload definitions
without changing global variables.  Anyway, it looks like the package
stuff has no mechanism to separate the issues.

Helmut

Re: Package initialization

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 217 bytes --]

> But if everything
> is in the autoload file there's no way to get the autoload definitions
> without changing global variables.

I don't understand what you mean by this, could you clarify with an example
scenario?

[-- Attachment #2: Type: text/html, Size: 289 bytes --]

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, Artur Malabarba wrote:

>> But if everything
>> is in the autoload file there's no way to get the autoload definitions
>> without changing global variables.
>
> I don't understand what you mean by this, could you clarify with an example
> scenario?

Well, SLIME has a function slime-setup that adds a hook to
lisp-mode-hook, which among other things changes lisp-indent-function
for that buffer.

Before the package stuff everybody had this in .emacs:

(require 'slime-autoloads)
(slime-setup)

Now with the package stuff the (require 'slime-autoloads) line is
unnecessary.  But additionally somebody claimed that it's idiomatic to
call slime-setup automatically in slime-autoloads.

I quite like to see the explicit call to slime-setup in my .emacs, but
others might not.

Helmut

Re: Package initialization

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

> Now with the package stuff the (require 'slime-autoloads) line is
> unnecessary.  But additionally somebody claimed that it's idiomatic to
> call slime-setup automatically in slime-autoloads.
>
> I quite like to see the explicit call to slime-setup in my .emacs, but
> others might not.

In this case I would not autoload that (though it wouldn't be terrible to
do so). It's not idiomatic for minor modes to turn themselves on as an
autoload.

[-- Attachment #2: Type: text/html, Size: 541 bytes --]

Re: Package initialization

[Stephen J. Turnbull]

Helmut Eller writes:

 > Now with the package stuff the (require 'slime-autoloads) line is
 > unnecessary.  But additionally somebody claimed that it's idiomatic to
 > call slime-setup automatically in slime-autoloads.

FWIW, in XEmacs's package system calls to package initialization
functions in autoloads has always been considered anti-social behavior
(and it's one of the few things we care about enough to override a
maintainer's preference).  In fact, we don't even allow those
functions to be called when a library is loaded.  All too frequently
packages contain useful utility functions in libraries that other
packages require, even though the user doesn't want the require'd
library's primary functionality.

What is considered good style in an XEmacs package:

1.  The package has an idempotent initialization function, typically
    implemented by wrapping the whole function in `(if (not
    <package>-initialized-p) ...)'.
2.  The autoloads include exactly those commands that users are
    expected to invoke in the uninitialized state.  (Users who know
    what they are doing can of course explicitly require or load
    package files and call any function.)
3.  All autoloaded commands should call the package initialization
    function as their first action.
4.  The autoloads may defvar global variables in the package's
    namespace.  However defcustoms (which are automatically
    initialized by the XEmacs package system) are preferred for
    variables the user might customize, even if that is unusual.

XEmacs packages aren't supposed to manipulate load-path etc, but
that's because XEmacs packages are supposed to have a standard layout,
and those standard locations are added to load-path etc by the package
system at startup.  I personally like that system, analogous to FHS
and/or LSB conformance in GNU/Linux systems, but many package
maintainers were distressed by it because they had existing layouts
which were incompatible with XEmacs's preferred layout.  So as Artur
says, it may be desirable to modify search paths and the like in
autoloads, though I prefer to delegate that to the lazily-called
initialization functions.

 > I quite like to see the explicit call to slime-setup in my .emacs, but
 > others might not.

I think the rules 1-4 above would satisfy both you and those others.

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, Stephen J. Turnbull wrote:

> 1.  The package has an idempotent initialization function, typically
>     implemented by wrapping the whole function in `(if (not
>     <package>-initialized-p) ...)'.
> 2.  The autoloads include exactly those commands that users are
>     expected to invoke in the uninitialized state.  (Users who know
>     what they are doing can of course explicitly require or load
>     package files and call any function.)
> 3.  All autoloaded commands should call the package initialization
>     function as their first action.
> 4.  The autoloads may defvar global variables in the package's
>     namespace.  However defcustoms (which are automatically
>     initialized by the XEmacs package system) are preferred for
>     variables the user might customize, even if that is unusual.
[]
>  > I quite like to see the explicit call to slime-setup in my .emacs, but
>  > others might not.
>
> I think the rules 1-4 above would satisfy both you and those others.

I don't see how.  Either (require 'slime-autoloads) calls slime-setup or
not.  Calling it lazily when a SLIME related command is invoked the
first time would kinda work for SLIME, but it would be rather strange
that say the slime-connect command implicitly adds hooks to
lisp-mode-hook.  In general, lazy initialization would also not work
well if those autoloaded commands should be bound to keys because before
the first invocation the keys would not be bound.

Helmut

Re: Package initialization

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

> lisp-mode-hook.  In general, lazy initialization would also not work
> well if those autoloaded commands should be bound to keys because before
> the first invocation the keys would not be bound.

If the user wants to bind an autoloaded command to a key, they can do that
just fine.

[-- Attachment #2: Type: text/html, Size: 347 bytes --]

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, Artur Malabarba wrote:

>> lisp-mode-hook.  In general, lazy initialization would also not work
>> well if those autoloaded commands should be bound to keys because before
>> the first invocation the keys would not be bound.
>
> If the user wants to bind an autoloaded command to a key, they can do that
> just fine.

How without adding something to .emacs?

Helmut

Re: Package initialization

[David Kastrup]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Sun, Jul 19 2015, Artur Malabarba wrote:
>
>>> lisp-mode-hook.  In general, lazy initialization would also not work
>>> well if those autoloaded commands should be bound to keys because before
>>> the first invocation the keys would not be bound.
>>
>> If the user wants to bind an autoloaded command to a key, they can do that
>> just fine.
>
> How without adding something to .emacs?

You can autoload on a prefix keymap as well I think.  So if some
bindings are to be present globally, you can do that without preloading
the package itself.

-- 
David Kastrup

Re: Package initialization

[Artur Malabarba]

>> If the user wants to bind an autoloaded command to a key, they can do that
>> just fine.
>
> How without adding something to .emacs?


;;;###autoload
(defun some-command ()
  ...)

;;;###autoload
(eval-after-load 'lisp-mode
  '(define-key lisp-mode-map "\C-c\C-k" #'some-command))

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, David Kastrup wrote:

>> How without adding something to .emacs?
>
> You can autoload on a prefix keymap as well I think.  So if some
> bindings are to be present globally, you can do that without preloading
> the package itself.

Show me the code and where to put it.

Helmut

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, Artur Malabarba wrote:

>>> If the user wants to bind an autoloaded command to a key, they can do that
>>> just fine.
>>
>> How without adding something to .emacs?
>
>
> ;;;###autoload
> (defun some-command ()
>   ...)
>
> ;;;###autoload
> (eval-after-load 'lisp-mode
>   '(define-key lisp-mode-map "\C-c\C-k" #'some-command))

So you are saying this is good style?  And how is this supposed to offer
the option that people like me don't want to have the keybinding without
setting it explicitly in .emacs.

Helmut

Re: Package initialization

[David Kastrup]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Sun, Jul 19 2015, David Kastrup wrote:
>
>>> How without adding something to .emacs?
>>
>> You can autoload on a prefix keymap as well I think.  So if some
>> bindings are to be present globally, you can do that without preloading
>> the package itself.
>
> Show me the code and where to put it.

Hmmm?

(info "(elisp) autoload")

 -- Function: autoload function filename &optional docstring interactive
          type
     This function defines the function (or macro) named FUNCTION so as
     to load automatically from FILENAME.  The string FILENAME specifies
     the file to load to get the real definition of FUNCTION.

     If FILENAME does not contain either a directory name, or the suffix
     ‘.el’ or ‘.elc’, this function insists on adding one of these
     suffixes, and it will not load from a file whose name is just
     FILENAME with no added suffix.  (The variable ‘load-suffixes’
     specifies the exact required suffixes.)

     The argument DOCSTRING is the documentation string for the
     function.  Specifying the documentation string in the call to
     ‘autoload’ makes it possible to look at the documentation without
     loading the function’s real definition.  Normally, this should be
     identical to the documentation string in the function definition
     itself.  If it isn’t, the function definition’s documentation
     string takes effect when it is loaded.

     If INTERACTIVE is non-‘nil’, that says FUNCTION can be called
     interactively.  This lets completion in ‘M-x’ work without loading
     FUNCTION’s real definition.  The complete interactive specification
     is not given here; it’s not needed unless the user actually calls
     FUNCTION, and when that happens, it’s time to load the real
     definition.

     You can autoload macros and keymaps as well as ordinary functions.
     Specify TYPE as ‘macro’ if FUNCTION is really a macro.  Specify
     TYPE as ‘keymap’ if FUNCTION is really a keymap.  Various parts of
     Emacs need to know this information without loading the real
     definition.

     An autoloaded keymap loads automatically during key lookup when a
     prefix key’s binding is the symbol FUNCTION.  Autoloading does not
     occur for other kinds of access to the keymap.  In particular, it
     does not happen when a Lisp program gets the keymap from the value
     of a variable and calls ‘define-key’; not even if the variable name
     is the same symbol FUNCTION.

[...]

   If you write a function definition with an unusual macro that is not
one of the known and recognized function definition methods, use of an
ordinary magic autoload comment would copy the whole definition into
‘loaddefs.el’.  That is not desirable.  You can put the desired
‘autoload’ call into ‘loaddefs.el’ instead by writing this:

     ;;;###autoload (autoload 'foo "myfile")
     (mydefunmacro foo
       ...)


So basically:

;;;autoload (autoload 'foo-prefix-map "myfile" "Prefix for foo" t 'keymap)
(define-prefix-command 'foo-prefix-map ...

;;;autoload (global-set-key (kbd "C-x c 5") 'foo-prefix-map)


Is there any problem with that?

-- 
David Kastrup

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, David Kastrup wrote:

>;;;autoload (global-set-key (kbd "C-x c 5") 'foo-prefix-map)
> Is there any problem with that?

Yes, some poeple like to set up keybdinings explicitly in .emacs and
don't want that merely installing a package changes global keybindings;
other people don't want to change their .emacs.

Helmut

Re: Package initialization

[bruce.connor.am]

[-- Attachment #1: Type: text/plain, Size: 712 bytes --]

> So you are saying this is good style?  And how is this supposed to offer
> the option that people like me don't want to have the keybinding without
> setting it explicitly in .emacs.

If it's a keybind on a specific mode, and if it doesn't conflict with the
keys for that mode, then it's low-impact enough that it's alright.

It's safe to assume that a user who installs SLIME would like a convenient
way to start SLIME from lisp-mode. The cases where that's not true are
unlikely to be bothered by an extra keybind. The cases that are bothered
(because it conflicts with some personal keybind), will have to add a line
to their init file (either to prevent activation of your package or to fix
the conflict).

[-- Attachment #2: Type: text/html, Size: 814 bytes --]

Re: Package initialization

[David Kastrup]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Sun, Jul 19 2015, David Kastrup wrote:
>
>>;;;autoload (global-set-key (kbd "C-x c 5") 'foo-prefix-map)
>> Is there any problem with that?
>
> Yes, some poeple like to set up keybdinings explicitly in .emacs and
> don't want that merely installing a package changes global keybindings;
> other people don't want to change their .emacs.

Is there a particular point to you making me look up the information in
the manual for you when all you wanted to state is that you want to have
logically incompatible requirements supported?

Are you getting paid for wasting people's time?

-- 
David Kastrup

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, David Kastrup wrote:

> Helmut Eller <eller.helmut@gmail.com> writes:
>
>> On Sun, Jul 19 2015, David Kastrup wrote:
>>
>>>;;;autoload (global-set-key (kbd "C-x c 5") 'foo-prefix-map)
>>> Is there any problem with that?
>>
>> Yes, some poeple like to set up keybdinings explicitly in .emacs and
>> don't want that merely installing a package changes global keybindings;
>> other people don't want to change their .emacs.
>
> Is there a particular point to you making me look up the information in
> the manual for you when all you wanted to state is that you want to have
> logically incompatible requirements supported?
> Are you getting paid for wasting people's time?

If you don't understand the question then don't answer.

Helmut

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, bruce.connor.am@gmail.com wrote:

>> So you are saying this is good style?  And how is this supposed to offer
>> the option that people like me don't want to have the keybinding without
>> setting it explicitly in .emacs.
>
> If it's a keybind on a specific mode, and if it doesn't conflict with the
> keys for that mode, then it's low-impact enough that it's alright.
>
> It's safe to assume that a user who installs SLIME would like a convenient
> way to start SLIME from lisp-mode. The cases where that's not true are
> unlikely to be bothered by an extra keybind. The cases that are bothered
> (because it conflicts with some personal keybind), will have to add a line to
> their init file (either to prevent activation of your package or to fix the
> conflict).
>

In other words: the recommended style is to set up key bindings with
autoload cookies.

Helmut

Re: Package initialization

[David Kastrup]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Sun, Jul 19 2015, David Kastrup wrote:
>
>> Helmut Eller <eller.helmut@gmail.com> writes:
>>
>>> On Sun, Jul 19 2015, David Kastrup wrote:
>>>
>>>>;;;autoload (global-set-key (kbd "C-x c 5") 'foo-prefix-map)
>>>> Is there any problem with that?
>>>
>>> Yes, some poeple like to set up keybdinings explicitly in .emacs and
>>> don't want that merely installing a package changes global keybindings;
>>> other people don't want to change their .emacs.
>>
>> Is there a particular point to you making me look up the information in
>> the manual for you when all you wanted to state is that you want to have
>> logically incompatible requirements supported?
>> Are you getting paid for wasting people's time?
>
> If you don't understand the question then don't answer.

To quote:

    Helmut Eller <eller.helmut@gmail.com> writes:
    >
    >>> On Sun, Jul 19 2015, Artur Malabarba wrote:
    >>
    >>>> lisp-mode-hook.  In general, lazy initialization would also not
    >>>> work well if those autoloaded commands should be bound to keys
    >>>> because before the first invocation the keys would not be
    >>>> bound.
    >>>
    >>> If the user wants to bind an autoloaded command to a key, they
    >>> can do that just fine.
    >>
    >> How without adding something to .emacs?
    >>
    > You can autoload on a prefix keymap as well I think.  So if some
    > bindings are to be present globally, you can do that without
    > preloading the package itself.

    Show me the code and where to put it.

So I would strongly suggest to people they think twice before assuming
anybody but yourself understands your "questions" when bothering to
answer.

-- 
David Kastrup

Re: Package initialization

[David Kastrup]

<bruce.connor.am@gmail.com> writes:

>> So you are saying this is good style?  And how is this supposed to offer
>> the option that people like me don't want to have the keybinding without
>> setting it explicitly in .emacs.
>
> If it's a keybind on a specific mode, and if it doesn't conflict with the
> keys for that mode, then it's low-impact enough that it's alright.
>
> It's safe to assume that a user who installs SLIME would like a convenient
> way to start SLIME from lisp-mode. The cases where that's not true are
> unlikely to be bothered by an extra keybind. The cases that are bothered
> (because it conflicts with some personal keybind), will have to add a line
> to their init file (either to prevent activation of your package or to fix
> the conflict).

Any personal keybinding would override an autoloaded keymap entry anyway
since personal keybindings don't just magically establish themselves
without writing anything in an initialization file.

-- 
David Kastrup

Re: Package initialization

[Artur Malabarba]

> Any personal keybinding would override an autoloaded keymap entry anyway

I think that depends on which code runs first.

Re: Package initialization

[Artur Malabarba]

> In other words: the recommended style is to set up key bindings with
> autoload cookies.

Entry-level keybinds to a specific major-mode, or a prefix-keymap, yes.
Don't clutter the global map with your autoloads.

Re: Package initialization

[David Kastrup]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

>> Any personal keybinding would override an autoloaded keymap entry anyway
>
> I think that depends on which code runs first.

Autoloads don't overwrite already loaded proper code.  They only
overwrite previous autoloads.

So you would have to both
a) define your own bindings with an autoloading keymap (why would you do
that?)
b) load libraries with autoloads _after_ defining your own autoloads
(why would you do that?)

Yes, you can't just load stuff in arbitrary order when coding it in
exactly the same manner and hope that Emacs will guess which stuff is
supposed to be more important to you than other identically coded stuff.

Is that really a surprise to anybody?

-- 
David Kastrup

Re: Package initialization

[Artur Malabarba]

>>> Any personal keybinding would override an autoloaded keymap entry anyway
>>
>> I think that depends on which code runs first.
>
> Autoloads don't overwrite already loaded proper code.  They only
> overwrite previous autoloads.

That depends on what's autoloaded. Autoloads don't overwrite function
or prefix keymap definitions. But if the package developer autoloads
an arbitrary code snippet (like say, `(define-key some-map "k"
#'some-func)'), then that code will be run as is.

> So you would have to both
> a) define your own bindings with an autoloading keymap (why would you do
> that?)
> b) load libraries with autoloads _after_ defining your own autoloads
> (why would you do that?)

If a library autoloads a call to `define-key', that code will be
executed at the end of emacs initialization (by default).
Then, if the user has a call to `define-key' in his init file for the
same key and keymap, that will get overwriten by the autoload (by no
fault of the user).

Re: Package initialization

[Stephen J. Turnbull]

Helmut Eller writes:

 > I don't see how.  Either (require 'slime-autoloads) calls slime-setup or
 > not.

In XEmacs it would be considered pollution, and we'd change the XEmacs
version of the package so that the autoloads do not call slime-setup.
XEmacs (and I assume Emacs ;-) is designed to be usable on a multiuser
system, and I have yet to find a package that absolutely everybody
thinks should be enabled.  For example, you'd think AUCTeX would be a
no-brainer in this day and age, but as far as I know RMS doesn't use
it, and there is at least one long since inactive XEmacs developer who
didn't use AUCTeX and objected vehemently to making it the default
TeX-mode if available.  Ditto cc-mode (although that person did
eventually switch).

Also, any of the schemes that I propose would mean that emacs -u
disables slime except for autoloads (I assume package autoloads can be
defeated some other way, as in XEmacs).  This is an important feature
for debugging.

 > Calling it lazily when a SLIME related command is invoked the
 > first time would kinda work for SLIME, but it would be rather strange
 > that say the slime-connect command implicitly adds hooks to
 > lisp-mode-hook.

Why is that stranger than saying that slime-setup implicitly adds
hooks to lisp-mode-hook?  Or stranger than saying that simply
installing the SLIME package on your system implicitly adds hooks to
lisp-mode-hook?

 > In general, lazy initialization would also not work well if those
 > autoloaded commands should be bound to keys because before the
 > first invocation the keys would not be bound.

Why would you expect them to be bound?  The existence of a package
doesn't necessarily mean that the user wants it invoked at startup.
If the user does want those keys bound, I don't see why it's a problem
to have the user put a call to slime-setup in .emacs, or put it on
lisp-mode-hook itself.

If users really object to that, well, you're right -- the XEmacs
policy would not satisfy their requirements.  But in a setup where
package installation implies package initialization, putting a call to
slime-setup in .emacs looks quite silly.

What I think would be nice for this kind of situation would be to have
a way for SLIME to add slime-setup to a checklist of defcustom options
for lisp-mode-hook.  Then users who do always want slime can use
Customize to enable it "permanently", and it's not in their .emacs.
Since slime-setup should be idempotent, it doesn't hurt to put it in
.emacs even if it would be invoked some other way (including by the
Customized lisp-mode-hook.

Re: Package initialization

[Helmut Eller]

On Mon, Jul 20 2015, Stephen J. Turnbull wrote:

>  > Calling it lazily when a SLIME related command is invoked the
>  > first time would kinda work for SLIME, but it would be rather strange
>  > that say the slime-connect command implicitly adds hooks to
>  > lisp-mode-hook.
>
> Why is that stranger than saying that slime-setup implicitly adds
> hooks to lisp-mode-hook?  Or stranger than saying that simply
> installing the SLIME package on your system implicitly adds hooks to
> lisp-mode-hook?

Because slime-connect is supposed to create a connection to an external
process.  Not mess around in with some mode-hooks and minor-modes.

>  > In general, lazy initialization would also not work well if those
>  > autoloaded commands should be bound to keys because before the
>  > first invocation the keys would not be bound.
>
> Why would you expect them to be bound?  The existence of a package
> doesn't necessarily mean that the user wants it invoked at startup.
> If the user does want those keys bound, I don't see why it's a problem
> to have the user put a call to slime-setup in .emacs, or put it on
> lisp-mode-hook itself.
>
> If users really object to that, well, you're right -- the XEmacs
> policy would not satisfy their requirements.  But in a setup where
> package installation implies package initialization, putting a call to
> slime-setup in .emacs looks quite silly.

package.el seems to be so designed that package installation implies
package initialization.  I don't like that but I can't change it.

Helmut

Re: Package initialization

[Eli Zaretskii]

> From: Helmut Eller <eller.helmut@gmail.com>
> Date: Sun, 19 Jul 2015 09:23:35 +0200
> Cc: Artur Malabarba <bruce.connor.am@gmail.com>,
> 	emacs-devel <emacs-devel@gnu.org>
> 
> lazy initialization would also not work well if those autoloaded
> commands should be bound to keys because before the first invocation
> the keys would not be bound.

We already have that working.  This is how, for example, you can see
what M-$ is bound to, although ispell.el is not loaded until actually
invoked.

Re: Package initialization

[Eli Zaretskii]

> Date: Sun, 19 Jul 2015 11:11:34 +0100
> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Cc: "Stephen J. Turnbull" <stephen@xemacs.org>,
> 	emacs-devel <emacs-devel@gnu.org>
> 
> >> If the user wants to bind an autoloaded command to a key, they can do that
> >> just fine.
> >
> > How without adding something to .emacs?
> 
> 
> ;;;###autoload
> (defun some-command ()
>   ...)
> 
> ;;;###autoload
> (eval-after-load 'lisp-mode
>   '(define-key lisp-mode-map "\C-c\C-k" #'some-command))

Or even just

;;;###autoload (define-key esc-map "$" 'ispell-word)
;;;###autoload
(defun ispell-word (&optional following quietly continue region)
...

Re: Package initialization

[Stefan Monnier]

> Before the package stuff everybody had this in .emacs:

> (require 'slime-autoloads)
> (slime-setup)

> Now with the package stuff the (require 'slime-autoloads) line is
> unnecessary.  But additionally somebody claimed that it's idiomatic to
> call slime-setup automatically in slime-autoloads.

> I quite like to see the explicit call to slime-setup in my .emacs, but
> others might not.

Since it's a configuration choice, it should not be imposed by the
autoloads file.  Instead, the way this is normally handled is by making
a slime-minor-mode, which when enabled does something like
(slime-setup), and when disabled, undoes it.

The important aspect is that it should be possible to install a package
without using it, so the autoloads file should only set things up so
that they *can* be used, but not so that they're automatically used
without being explicitly requested.


        Stefan

Re: Package initialization

[Stephen J. Turnbull]

Helmut Eller writes:
 > On Mon, Jul 20 2015, Stephen J. Turnbull wrote:
 > 
 > >  > Calling it lazily when a SLIME related command is invoked the
 > >  > first time would kinda work for SLIME, but it would be rather strange
 > >  > that say the slime-connect command implicitly adds hooks to
 > >  > lisp-mode-hook.
 > >
 > > Why is that stranger than saying that slime-setup implicitly adds
 > > hooks to lisp-mode-hook?  Or stranger than saying that simply
 > > installing the SLIME package on your system implicitly adds hooks to
 > > lisp-mode-hook?
 > 
 > Because slime-connect is supposed to create a connection to an external
 > process.  Not mess around in with some mode-hooks and minor-modes.

If slime-connect won't work properly without slime-setup, there's no
difference.  If it's useful to do slime-connect without slime-setup,
that would be another story but it sounds unlikely.  I'd have to hear
details to say how XEmacs would prefer to handle it.  (I don't claim
the XEmacs way is the *right* way, only that it has worked very well
for us.  No user has ever complained that installing a package doesn't
change Emacs state in arbitrary ways.  On the contrary, packages where
installation implies initialization were frequently implicated in
mysterious buggy behavior until we started enforcing the policy.)

 > package.el seems to be so designed that package installation
 > implies package initialization.  I don't like that but I can't
 > change it.

I didn't get that impression.  It's one thing for Emacs to
automatically scan for usable packages and add them to load-path, to
set up autoloads for their entry-point commands, and perhaps add their
data directories etc to appropriate paths so they can be found by name
rather than a full filesystem path.  I don't consider that
"initialization" because it's consistent with the "and the kitchen
sink" tradition of Lisp environments (especially GNU Emacs), where
packages that many users would never notice if they weren't installed
are often included with the core distribution.

It's another to change the meaning of user gestures, even in a trivial
and "obviously useful" way like adding to keymaps.

If in fact the majority of lisp-mode users find slime sufficiently
useful, what I would do is negotiate with the lisp-mode maintainer (I
suppose that's actually emacs-devel) to get the slime bindings
"officially" added to the lisp-mode keymaps, and arrange for them to
fail gracefully if the slime package is unavailable, eg, by binding
them to a `lisp-mode-how-to-get-slime' help command.

Re: Package initialization

[Helmut Eller]

On Sun, Jul 19 2015, Stefan Monnier wrote:

> Since it's a configuration choice, it should not be imposed by the
> autoloads file.  Instead, the way this is normally handled is by making
> a slime-minor-mode, which when enabled does something like
> (slime-setup), and when disabled, undoes it.

SLIME is already a minor mode; I think if we can make it work if users
call (add-hook 'lisp-mode-hook 'slime-mode) in their .emacs.

> The important aspect is that it should be possible to install a package
> without using it, so the autoloads file should only set things up so
> that they *can* be used, but not so that they're automatically used
> without being explicitly requested.

I think that's quite an important guideline that should be stated a bit
more prominently in manual.

It also seems to me that there is a certain conflict between what you
say and this from the manual:

  These autoload definitions are saved to a file named
  ‘NAME-autoloads.el’ in the content directory.  They are typically used
  to autoload the principal user commands defined in the package, but
  they can also perform other tasks, such as adding an element to
  ‘auto-mode-alist’.

Helmut

Re: Package initialization

[Helmut Eller]

On Mon, Jul 20 2015, Stephen J. Turnbull wrote:

>  > Because slime-connect is supposed to create a connection to an external
>  > process.  Not mess around in with some mode-hooks and minor-modes.
>
> If slime-connect won't work properly without slime-setup, there's no
> difference.  If it's useful to do slime-connect without slime-setup,
> that would be another story but it sounds unlikely.

It wouldn't work properly without slime-setup.  Hmm... since we can't
call slime-setup from .emacs, I guess we have no other choice than to
call it from slime-connect too; very annoying.

> I'd have to hear
> details to say how XEmacs would prefer to handle it.

The current version of SLIME doesn't work with XEmacs; we stopped
supporting XEmacs about a year ago.

>  > package.el seems to be so designed that package installation
>  > implies package initialization.  I don't like that but I can't
>  > change it.
>
> I didn't get that impression.  It's one thing for Emacs to
> automatically scan for usable packages and add them to load-path, to
> set up autoloads for their entry-point commands, and perhaps add their
> data directories etc to appropriate paths so they can be found by name
> rather than a full filesystem path.  I don't consider that
> "initialization" because it's consistent with the "and the kitchen
> sink" tradition of Lisp environments (especially GNU Emacs), where
> packages that many users would never notice if they weren't installed
> are often included with the core distribution.

I have the impression that the left hand doesn't know what the right
hand wants.  In the manual they say that it's ok to add things to
auto-mode-alist, but then Stefan says that packages should not be
enabled automatically.

> It's another to change the meaning of user gestures, even in a trivial
> and "obviously useful" way like adding to keymaps.
>
> If in fact the majority of lisp-mode users find slime sufficiently
> useful, what I would do is negotiate with the lisp-mode maintainer (I
> suppose that's actually emacs-devel) to get the slime bindings
> "officially" added to the lisp-mode keymaps, and arrange for them to
> fail gracefully if the slime package is unavailable, eg, by binding
> them to a `lisp-mode-how-to-get-slime' help command.

I think we rather keep control of key bindings in our own hands;
especially as we don't always agree with the Emacs maintainers (like the
bindings for M-./M-, until recently etc).

Helmut

Re: Package initialization

[Chad Brown]

I’m making a few guesses here, and I might be wrong, but I believe it 
might help resolve a difference in understanding.

> On 20 Jul 2015, at 10:01, Helmut Eller <eller.helmut@gmail.com> wrote:
> 
> I have the impression that the left hand doesn't know what the right
> hand wants.  In the manual they say that it's ok to add things to
> auto-mode-alist, but then Stefan says that packages should not be
> enabled automatically.


It seems like most of your objections stem from this, and it further
seems like the difference in understanding stems from the difference
between a _guideline_ concerning additions and a _rule_ concerning
additions and replacements.

For example, if I install a package that adds a markdown-mode, its
probably ok to have it add markdown-mode for .md files to auto-mode-list.
If I install AUCTeX, its probably ok for the installation to set
some variables so that M-x preview-latex works, but it probably doesn’t
want to override the bindings for .tex, .ltx, .sty, etc.

It’s not clear to me if you’re aware of this difference and unhappy
about it, or unaware of this difference and unhappy about the
results, or both.

Hope that helps,
~Chad

Re: Package initialization

[Helmut Eller]

On Mon, Jul 20 2015, Chad Brown wrote:

>> I have the impression that the left hand doesn't know what the right
>> hand wants.  In the manual they say that it's ok to add things to
>> auto-mode-alist, but then Stefan says that packages should not be
>> enabled automatically.
>
> It seems like most of your objections stem from this, and it further
> seems like the difference in understanding stems from the difference
> between a _guideline_ concerning additions and a _rule_ concerning
> additions and replacements.

Indeed, I don't understand the difference between guideline and rule.

> For example, if I install a package that adds a markdown-mode, its
> probably ok to have it add markdown-mode for .md files to auto-mode-list.
> If I install AUCTeX, its probably ok for the installation to set
> some variables so that M-x preview-latex works, but it probably doesn’t
> want to override the bindings for .tex, .ltx, .sty, etc.

I also don't understand why phrases like "probably ok" are useful in
this context.  To me that sounds like: "it's ok for amateurs, who have
little knowledge about Emacs conventions, to do it, because the harm
will be quite limited, but real pros wouldn't do it because they are
aware of some obscure problem that might arise even if only with low
probability."

> It’s not clear to me if you’re aware of this difference and unhappy
> about it, or unaware of this difference and unhappy about the
> results, or both.

I'm only asking for a clarification of the guidelines, so that the next
time when some of those amateurs tells me that it's "probably ok" to
enable things automatically, I can point to the manual an tell him that
it violates the usual guidelines.

Helmut

Re: Package initialization

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

> I also don't understand why phrases like "probably ok" are useful in
> this context.  To me that sounds like: "it's ok for amateurs, who have
> little knowledge about Emacs conventions, to do it, because the harm
> will be quite limited, but real pros wouldn't do it because they are
> aware of some obscure problem that might arise even if only with low
> probability."

There are no obscure problems.
In this context, it means that probably nobody will be bothered by it. But
you never know. There may be some user out there who gets enraged at the
fact that they now have to add one line of code to their init file to
prevent a package they explicitly installed from adding an entry to
auto-mode-alist.

[-- Attachment #2: Type: text/html, Size: 831 bytes --]

Re: Package initialization

[Dmitry Gutov]

On 07/20/2015 08:01 PM, Helmut Eller wrote:

> I have the impression that the left hand doesn't know what the right
> hand wants.  In the manual they say that it's ok to add things to
> auto-mode-alist, but then Stefan says that packages should not be
> enabled automatically.

I agree we should have clearer guidelines there. The third-party 
community has had these discussions a couple of years ago 
(https://github.com/skeeto/skewer-mode/issues/22#issuecomment-18301023, 
https://github.com/purcell/elisp-slime-nav/pull/6), and the average 
consensus landed on not even modifying hooks inside autoloads (not 
something I necessarily agree with, but it's an okay guideline). 
Changing auto-mode-alist is usually okay, but even that can lead to 
problems: https://github.com/purcell/mmm-mode/issues/36

Also see 
https://github.com/bbatsov/emacs-lisp-style-guide/#loading-and-autoloading.

Re: Package initialization

[Stefan Monnier]

>> The important aspect is that it should be possible to install a package
>> without using it, so the autoloads file should only set things up so
>> that they *can* be used, but not so that they're automatically used
>> without being explicitly requested.

> I think that's quite an important guideline that should be stated a bit
> more prominently in manual.

> It also seems to me that there is a certain conflict between what you
> say and this from the manual:

>   These autoload definitions are saved to a file named
>   ‘NAME-autoloads.el’ in the content directory.  They are typically used
>   to autoload the principal user commands defined in the package, but
>   they can also perform other tasks, such as adding an element to
>   ‘auto-mode-alist’.

I don't see a conflict there.  Opening a file with the particular
extension counts as an "explicit request".


        Stefan

Re: Package initialization

[Stephen J. Turnbull]

Stefan Monnier writes:

 > >> The important aspect is that it should be possible to install a package
 > >> without using it, so the autoloads file should only set things up so
 > >> that they *can* be used, but not so that they're automatically used
 > >> without being explicitly requested.
[...]
 > > It also seems to me that there is a certain conflict between what you
 > > say and this from the manual:
 > 
 > >   These autoload definitions are saved to a file named
 > >   ‘NAME-autoloads.el’ in the content directory.  They are typically used
 > >   to autoload the principal user commands defined in the package, but
 > >   they can also perform other tasks, such as adding an element to
 > >   ‘auto-mode-alist’.
 > 
 > I don't see a conflict there.  Opening a file with the particular
 > extension counts as an "explicit request".

For a major mode, usually[1], for a minor mode (as slime-mode is),
rarely[2].  IMHO YMMV etc.

Footnotes: 
[1]  Unfortunately overloaded extensions are hardly uncommon.

[2]  Of course, .gz invoking a decompressor is unlikely to bother very
many people.  Nevertheless, in XEmacs we decided it should not be
enabled via autoloads.  FWIW, of course.

Re: Package initialization

[Stephen J. Turnbull]

Helmut Eller writes:

 > I think we rather keep control of key bindings in our own hands;
 > especially as we don't always agree with the Emacs maintainers
 > (like the bindings for M-./M-, until recently etc).

Since I don't know what those do in slime, perhaps you misunderstand
my point about keybindings in lisp-mode: I meant only the keybindings
that would allow slime users to invoke the most common functionality
conveniently.  Then once they do that, slime-setup would be called
implicitly, and all keybindings would become available.

Re: Package initialization

[Helmut Eller]

On Mon, Jul 20 2015, Stefan Monnier wrote:

> Opening a file with the particular
> extension counts as an "explicit request".

Interesting; I've never heard that before.

Helmut

Re: Package initialization

[Helmut Eller]

On Tue, Jul 21 2015, Stephen J. Turnbull wrote:

> Helmut Eller writes:
>
>  > I think we rather keep control of key bindings in our own hands;
>  > especially as we don't always agree with the Emacs maintainers
>  > (like the bindings for M-./M-, until recently etc).
>
> Since I don't know what those do in slime, perhaps you misunderstand
> my point about keybindings in lisp-mode: I meant only the keybindings
> that would allow slime users to invoke the most common functionality
> conveniently.  Then once they do that, slime-setup would be called
> implicitly, and all keybindings would become available.

Most of SLIME's commands don't work before the user creates a connection
with `M-x slime' or 'M-x slime-connect'.  Those two don't have key
bindings; so I guess your idea is not applicable.  I think it's also
more natural to enable all of SLIME's keys in lisp-mode-hook, even if
there's no connection yet, rather than as side effect of creating the
first connection.

Helmut

Re: Package initialization

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Stefan Monnier writes:
>
>  > >> The important aspect is that it should be possible to install a package
>  > >> without using it, so the autoloads file should only set things up so
>  > >> that they *can* be used, but not so that they're automatically used
>  > >> without being explicitly requested.
> [...]
>  > > It also seems to me that there is a certain conflict between what you
>  > > say and this from the manual:
>  > 
>  > >   These autoload definitions are saved to a file named
>  > >   ‘NAME-autoloads.el’ in the content directory.  They are typically used
>  > >   to autoload the principal user commands defined in the package, but
>  > >   they can also perform other tasks, such as adding an element to
>  > >   ‘auto-mode-alist’.
>  > 
>  > I don't see a conflict there.  Opening a file with the particular
>  > extension counts as an "explicit request".
>
> For a major mode, usually[1], for a minor mode (as slime-mode is),
> rarely[2].  IMHO YMMV etc.
>
> Footnotes: 
> [1]  Unfortunately overloaded extensions are hardly uncommon.

Or overloaded modes.  AUCTeX provides major modes different from the
default TeX major modes.  When it is activated system-wide, it provides
an easy customization of modes it should keep its fingers off, and
without further configuration it does not activate its extensive
information parsing and caching so that people without interest in TeX
for which TeX-mode gets triggered when browsing directories do not get
unexpected delays or files lying around.

That's a heavy load of compromises.  I can't remember when we had the
last complaint that AUCTeX was doing something the user did not expect
(in general, plain TeX users don't get reasonable value from AUCTeX and
don't care for "fancy" anyway, but it's been long since the last one
asked how to turn it off).  We do get occasional complaints that AUCTeX
is doing less than the user expects (pointing to the first page of the
manual helps).  So it would seem that we are rather erring on the
conservative side.

-- 
David Kastrup

Re: Package initialization

[Alexis]

Helmut Eller <eller.helmut@gmail.com> writes:

> I think it's also more natural to enable all of SLIME's keys in 
> lisp-mode-hook, even if there's no connection yet, rather than 
> as side effect of creating the first connection.

As the maintainer of one of the modes for PicoLisp:

    https://github.com/flexibeast/picolisp-mode

i would really prefer that SLIME didn't assume that it's the only 
derivative of lisp-mode. This assumption already has to be worked 
around:

    https://github.com/flexibeast/picolisp-mode#note-slime

There would be further problems if SLIME were to decide to impose 
its keybindings on all lisp-mode derivatives.

i think it's perfectly reasonable for a user to want to make use 
of SLIME alongside other lisp-mode derivatives - a user might want 
to use SLIME when programming in CL, but a PicoLisp mode when 
programming in PicoLisp.

Please don't enable all of SLIME's keys in lisp-mode-hook.


Alexis.