Treatise on require

Treatise on require

[Bill Wohler]

Is there a treatise on loading that shows best practices in the use of
require, autoload, defvar, eval-when-compile, eval-and-compile? I've
read the manual (repeatedly) and sort of understand each piece, but
don't have a good big picture.

Over time, nasty circular dependencies have reared their ugly head in
MH-E. For example, mh-e requires just about other file and just about
every other file requires mh-e. Just for fun there are mini-loops with
mh-customize and and mh-utils. This is currently handled by putting the
call to provides (in mh-e.el) before the various calls to require (whose
files require mh-e).

But it's very fragile.

I'm sort of thinking that it might be better to replace the require
calls with something like:

  (eval-when-compile
    (autoload 'foo "foo")
    (defvar foo))

This would placate the compiler without creating require loops.
Thoughts?

Also, has anyone written anything to build a call graph so that I might
be able to reorganize the files as another way to eliminate the loops.

Anyway, any feedback on this topic or pointers to existing discussions
would be extremely welcome. Thanks!

p.s. The elisp manual for eval-and-compile says this:

     If a macro has a helper function to build its result, and that
     macro is used both locally and outside the package, then
     `eval-and-compile' should be used to get the helper both when
     compiling and then later when running.

     If functions are defined programmatically (with `fset' say), then
     `eval-and-compile' can be used to have that done at compile-time
     as well as run-time, so calls to those functions are checked (and
     warnings about "not known to be defined" suppressed).

However, it doesn't say *how* eval-and-compile should be used. I have no
idea. Examples for both paragraphs would help.

-- 
Bill Wohler <wohler@newt.com>  http://www.newt.com/wohler/  GnuPG ID:610BD9AD
Maintainer of comp.mail.mh FAQ and MH-E. Vote Libertarian!
If you're passed on the right, you're in the wrong lane.

Re: Treatise on require

[Thien-Thi Nguyen]

Bill Wohler <wohler@newt.com> writes:

> feedback [...] welcome

keeping things in separate files is nice for maintenance,
but if they are meant to be used together you might want to
investigate an alternate build procedure: cat all the "core"
files together and compile that instead of each one separately.
this is the approach i take w/ EDB, which is at:

  http://www.glug.org/people/ttn/software/edb/

see file bfuncs for the concatenation (and compilation).
note that not all files are considered core, just those
that are mutually (or inextricably ;-) useful.

also, in the subdir skram/, there is machinery for collecting
information on various elements (kind of like etags i suppose)
of the EDB source.  this info can be massaged into a dependency
graph (not worth it for EDB, but you get the idea).  see file
make-skram.el and try "make skram.data" to see it in action.

of course, EDB is a separate package, so its build methods
can be as weird as necessary.  i'm not sure if this approach
would be acceptable for Emacs proper, where each file is
traditionally compiled on its own...

thi

Re: Treatise on require

[Andreas Schwab]

Bill Wohler <wohler@newt.com> writes:

> Over time, nasty circular dependencies have reared their ugly head in
> MH-E. For example, mh-e requires just about other file and just about
> every other file requires mh-e. Just for fun there are mini-loops with
> mh-customize and and mh-utils. This is currently handled by putting the
> call to provides (in mh-e.el) before the various calls to require (whose
> files require mh-e).

IMHO in the long run it would be better to reorganize the files to remove
the circular dependencies.  For example the interdependency between
mh-customize and mh-utils appears to be gratuitous judging just from the
file names.  Maybe the common functions should be factored out to
different/new file.

Btw, looking at mh-utils.el, I don't think it should use `load' for
optional dependencies.  The libraries may already be loaded and should not
be reloaded just by loading mh-utils.  For example, tool-bar is already
preloaded in most configurations.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Treatise on require

[Richard M. Stallman]

    Over time, nasty circular dependencies have reared their ugly head in
    MH-E. For example, mh-e requires just about other file and just about
    every other file requires mh-e.

You really should get rid of that.  If you put the specific things
that are depended on into one file, then that file need not depend
on any others.

    Also, has anyone written anything to build a call graph so that I might
    be able to reorganize the files as another way to eliminate the loops.

I don't recall that there is one in Emacs itself.  I agree it would
be useful to have.

I suggest this method:

1. Move all macros and defsubsts into the base file.

2. Compile another file and see what it might depend on.

3. Move that stuff into the base file.

Etc.

Re: Treatise on require

[Stefan Monnier]

> Is there a treatise on loading that shows best practices in the use of
> require, autoload, defvar, eval-when-compile, eval-and-compile?

I think `defvar' in an outlier in your list.  It's unrelated.
`eval-and-compile' is only useful for self-dependencies (e.g. when a macro
uses the value of a variable defined in the same file: so the variable
definition needs to be evaluated both at compile time (when you use the
macros) and at run time) so it's not related either.

> Over time, nasty circular dependencies have reared their ugly head in
> MH-E.

Get rid of them by moving things around, including into new files.
Sometimes merging small files into one big one makes more sense.

> For example, mh-e requires just about other file

That's to be expected: mh-e.el probably uses most/all of the mh-e code.

> and just about every other file requires mh-e.

That can't be right.

> This is currently handled by putting the call to provides (in mh-e.el)
> before the various calls to require (whose files require mh-e).

Bad idea.

> I'm sort of thinking that it might be better to replace the require
> calls with something like:

>   (eval-when-compile
>     (autoload 'foo "foo")
>     (defvar foo))

Better not.  Especially the `defvar' inside `eval-when-compile' is
basically meaningless, it should be moved out.

> This would placate the compiler without creating require loops.

If you like to fix symptoms rather than causes, go for it.  Or better yet:
use `with-no-warnings'.  But I'd advise you against it.


        Stefan