eieio persistent, plus autoloads

eieio persistent, plus autoloads

[Eric Abrahamsen]

tl;dr: I'm looking for a safe way of creating autoloads (via
eieio-defclass-autoload) for every defined subclass of a particular
class (even when defined in third-party packages), and then writing
those autoload statements to a custom loaddefs.el file.

longer version:

One of my one-hour-a-month side projects is seeing if I can port BBDB
over to use EIEIO under the hood. There are a lot of potential benefits,
but two of the big ones are automatic file persistence with
eieieo-persistent, and the ability to subclass BBDB records and field
classes, making it much easier to extend and alter the behavior of a
BBDB database.

Those two features together create an immediate problem. Say a
third-party package defines a new field type. I instantiate that type in
my current session, make it part of my database, and then write it to
file. The next time I restart emacs, there's a danger that the database
file will be read before the class type has been defined, leading to a
type error.

eieio-persistent provides some tools for this in the form of autoloads:
eieio-defclass-autoload creates class definition autoload objects, and
the file-reading function looks out for them while reading a persistence
file. I'm just stuck on making it work right.

Working backwards, I envision it happening like this: at the top of the
BBDB code, in a spot that gets loaded before any databases are read,
there's a line like this:

(load "bbdb-loaddefs.el" t t t)

That file would be full of autoload statements, created by
eieio-defclass-autoload, for any custom subclass that's been created.
The question is, how do I get those statements in there?

eieio-hook is only run the first time eieio-defclass is called, so
that's no good.

My other idea was to provide a custom macro that wraps defclass, but I
can't even get that to work: it seems that update-file-autoloads only
works for autoload cookies in comments, not for regular autoload or
eieio-defclass-autoload calls. You can't put that comment cookie in a
macro, so I still can't find a way to programmatically write these
autoload statements to a single file...

I hope that made sense. Any pointers very appreciated!

Eric

Re: eieio persistent, plus autoloads

[David Engster]

Eric Abrahamsen writes:
> Working backwards, I envision it happening like this: at the top of the
> BBDB code, in a spot that gets loaded before any databases are read,
> there's a line like this:
>
> (load "bbdb-loaddefs.el" t t t)
>
> That file would be full of autoload statements, created by
> eieio-defclass-autoload, for any custom subclass that's been created.
> The question is, how do I get those statements in there?

It might very well be that I don't understand what you're trying to do,
but `eieio-defclass-autoload' is used the same way as the normal
`autoload': you either call it manually, or you use a magic autoload
comment in front of `defclass' and run the files through
`update-file-autoloads', so that those statements get generated for
you. Why doesn't that work in your case?

-David

Re: eieio persistent, plus autoloads

[Eric Abrahamsen]

David Engster <deng@randomsample.de> writes:

> Eric Abrahamsen writes:
>> Working backwards, I envision it happening like this: at the top of the
>> BBDB code, in a spot that gets loaded before any databases are read,
>> there's a line like this:
>>
>> (load "bbdb-loaddefs.el" t t t)
>>
>> That file would be full of autoload statements, created by
>> eieio-defclass-autoload, for any custom subclass that's been created.
>> The question is, how do I get those statements in there?
>
> It might very well be that I don't understand what you're trying to do,
> but `eieio-defclass-autoload' is used the same way as the normal
> `autoload': you either call it manually, or you use a magic autoload
> comment in front of `defclass' and run the files through
> `update-file-autoloads', so that those statements get generated for
> you. Why doesn't that work in your case?

Yeah, I didn't do a terribly good job of expressing the problem...

That doesn't work because when users or third-party package managers
want to subclass, say, the bbdb-field class, they would have to remember
to do this:

;;;###autoload (eieio-defclass-autoload 'my-field-class "thisfile.el")
(defclass my-field-class (bbdb-field) etc etc

It just seems a lot safer to have something automatic. And even the
above would not result in the autoload statement getting written to the
custom loaddefs file.

So I was thinking either hot-wire `defclass' (probably with advice), or
provide a special macro for subclassing these classes. But both of those
solutions have the same problem: as far as I can tell, calling
`eieio-defclass-autoload' in a program, followed by
`update-file-autoloads', does *not* write anything to a loaddef file,
that only happens when compiling a file with autoload cookies in it.

So... now I'm thinking that, if I really want to insist on this, I'll
still go the macro route, but just use some of the bits and pieces from
autoload.el and cobble together something semi-manual.

I don't know, maybe I've answered my own question...

Eric

Re: eieio persistent, plus autoloads

[David Engster]

Eric Abrahamsen writes:
> That doesn't work because when users or third-party package managers
> want to subclass, say, the bbdb-field class, they would have to remember
> to do this:
>
> ;;;###autoload (eieio-defclass-autoload 'my-field-class "thisfile.el")

A plain ;;;###autoload is enough; the autoload-updating mechanism will
see that it is for a defclass.

Otherwise: Yes, the package maintainer should put an autoload comment on
functions which are likely to be used before the package is loaded, and
the same best practice applies to classes.

I don't see this being specific to EIEIO. In general, how do you make
sure a package is loaded before you use it? In Emacs, you usually
require it in your .emacs; for big packages, like Gnus or CEDET, that
require simply loads a file filled with autoloads, which were generated
during their compilation. For small packages, it is usually tolerable to
simply load the whole thing on startup. (I happen to create an autoload
file for all those little packages, but I think that's pretty unusual,
and I also wouldn't recommend it to people unfamiliar with Emacs
autoloads.)

> It just seems a lot safer to have something automatic.

No, on the contrary. One package fiddling with autoloads from another
package is asking for trouble. What if I happen to load a package and it
creates a class which inherits from bbdb-field, and I don't even know
that? If would put a permanent autoload into my BBDB setup, that would
annoy me immensely. What happens if I remove that package?

> So I was thinking either hot-wire `defclass' (probably with advice)
>, or provide a special macro for subclassing these classes. But both of
>those solutions have the same problem: as far as I can tell, calling
>`eieio-defclass-autoload' in a program, followed by
>`update-file-autoloads', does *not* write anything to a loaddef file,
>that only happens when compiling a file with autoload cookies in it.

Yes, it's the same with plain `autoload'. `update-file-autoloads' is
just a mechanism so that you don't have to manually write and maintain a
file containing all the autoloads of your package.

> So... now I'm thinking that, if I really want to insist on this, I'll
> still go the macro route, but just use some of the bits and pieces from
> autoload.el and cobble together something semi-manual.

IMHO it's really simple: If you're using a third-party package which
extends BBDB, it has to be loaded before you load your bbbdb file. The
user should take care of that, and I don't think that's asking too
much. :-)

-David

Re: eieio persistent, plus autoloads

[Eric Abrahamsen]

David Engster <deng@randomsample.de> writes:

> Eric Abrahamsen writes:
>> That doesn't work because when users or third-party package managers
>> want to subclass, say, the bbdb-field class, they would have to remember
>> to do this:
>>
>> ;;;###autoload (eieio-defclass-autoload 'my-field-class "thisfile.el")
>
> A plain ;;;###autoload is enough; the autoload-updating mechanism will
> see that it is for a defclass.
>
> Otherwise: Yes, the package maintainer should put an autoload comment on
> functions which are likely to be used before the package is loaded, and
> the same best practice applies to classes.
>
> I don't see this being specific to EIEIO. In general, how do you make
> sure a package is loaded before you use it? In Emacs, you usually
> require it in your .emacs; for big packages, like Gnus or CEDET, that
> require simply loads a file filled with autoloads, which were generated
> during their compilation. For small packages, it is usually tolerable to
> simply load the whole thing on startup. (I happen to create an autoload
> file for all those little packages, but I think that's pretty unusual,
> and I also wouldn't recommend it to people unfamiliar with Emacs
> autoloads.)

Okay, I guess I get it. I was thinking of eieio-persistent as a slightly
unusual case just because object constructors are being written to a
file, and the potential for breakage seems a little more dramatic. But
perhaps some nice error handling and reporting is all that's necessary.

Just out of curiosity, how do you create an autoload file for your
smaller packages?

>> It just seems a lot safer to have something automatic.
>
> No, on the contrary. One package fiddling with autoloads from another
> package is asking for trouble. What if I happen to load a package and it
> creates a class which inherits from bbdb-field, and I don't even know
> that? If would put a permanent autoload into my BBDB setup, that would
> annoy me immensely. What happens if I remove that package?

Good point!

>> So I was thinking either hot-wire `defclass' (probably with advice)
>>, or provide a special macro for subclassing these classes. But both of
>>those solutions have the same problem: as far as I can tell, calling
>>`eieio-defclass-autoload' in a program, followed by
>>`update-file-autoloads', does *not* write anything to a loaddef file,
>>that only happens when compiling a file with autoload cookies in it.
>
> Yes, it's the same with plain `autoload'. `update-file-autoloads' is
> just a mechanism so that you don't have to manually write and maintain a
> file containing all the autoloads of your package.
>
>> So... now I'm thinking that, if I really want to insist on this, I'll
>> still go the macro route, but just use some of the bits and pieces from
>> autoload.el and cobble together something semi-manual.
>
> IMHO it's really simple: If you're using a third-party package which
> extends BBDB, it has to be loaded before you load your bbbdb file. The
> user should take care of that, and I don't think that's asking too
> much. :-)

Cool, this makes sense. I guess I half-expected, when I posted this, to
be told I was barking up the wrong tree entirely, and I'm pleased to get
another opinion on best practices.

Thanks!
Eric

Re: eieio persistent, plus autoloads

[David Engster]

Eric Abrahamsen writes:
> Okay, I guess I get it. I was thinking of eieio-persistent as a slightly
> unusual case just because object constructors are being written to a
> file, and the potential for breakage seems a little more dramatic. But
> perhaps some nice error handling and reporting is all that's necessary.

`eieio-defclass-autoload' only creates a mock class; this is necessary
so that EIEIO already knows about it, but that class is not yet
functional. It then creates a plain `autoload' for the actual
constructor function. So far this has worked very well, at least for
CEDET, which autoloads a lot of classes.

> Just out of curiosity, how do you create an autoload file for your
> smaller packages?

With an emacs script like this:

#!emacs --script

(setq my-base-path (expand-file-name "~/emacs-packages/"))
(setq dirs '("smallstuff" "speck" "minimap" "magit"))
(let ((generated-autoload-file (concat my-base-path "loaddefs.el")))
  (cd my-base-path)
  (dolist (dir dirs)
    (update-directory-autoloads dir)))

-David

Re: eieio persistent, plus autoloads

[Eric Abrahamsen]

David Engster <deng@randomsample.de> writes:

> Eric Abrahamsen writes:
>> Okay, I guess I get it. I was thinking of eieio-persistent as a slightly
>> unusual case just because object constructors are being written to a
>> file, and the potential for breakage seems a little more dramatic. But
>> perhaps some nice error handling and reporting is all that's necessary.
>
> `eieio-defclass-autoload' only creates a mock class; this is necessary
> so that EIEIO already knows about it, but that class is not yet
> functional. It then creates a plain `autoload' for the actual
> constructor function. So far this has worked very well, at least for
> CEDET, which autoloads a lot of classes.
>
>> Just out of curiosity, how do you create an autoload file for your
>> smaller packages?
>
> With an emacs script like this:
>
> #!emacs --script
>
> (setq my-base-path (expand-file-name "~/emacs-packages/"))
> (setq dirs '("smallstuff" "speck" "minimap" "magit"))
> (let ((generated-autoload-file (concat my-base-path "loaddefs.el")))
>   (cd my-base-path)
>   (dolist (dir dirs)
>     (update-directory-autoloads dir)))
>
> -David

Thanks!
Eric