Semantics of autoload cookies on defcustoms

Semantics of autoload cookies on defcustoms

[Stefan Monnier]

What is the intended effect of adding an autoload cookie on a defcustom?

I ask this question because I recently noticed that setting variables such
as diary-file via custom causes calendar.el to be loaded at startup, even
tho I can't see any reason why such a setting would justify eagerly loading
calendar.el.

So should such variable not have an autoload cookie, or should autoload.el
be adjusted so that it doesn't add calls to `custom-autoload' for them?


        Stefan "who thinks both should be used, since changing autoload.el
                is the only sane way to fix it once and for all, but who
                also thinks that there should be a very good reason
                (explained in a comment) to justify an autoload cookie on
                a variable."

Re: Semantics of autoload cookies on defcustoms

[Richard Stallman]

    What is the intended effect of adding an autoload cookie on a defcustom?

It puts a defvar and the custom-autoload call into loaddefs.

    I ask this question because I recently noticed that setting variables such
    as diary-file via custom causes calendar.el to be loaded at startup, even
    tho I can't see any reason why such a setting would justify eagerly loading
    calendar.el.

There was a reason for this, and I used to know it, but I have
forgotten.

In order to process the specified value, custom needs the defcustom
info.  So there are two options:

1. Save the value away and process it if/when the defcustom is loaded.
2. Load the defcustom now, and process the value right away.

#1 is the usual method.  That is fine for variables that aren't really
defined at all until the package is loaded.

But when a variable is autoloaded, that means its value is meaningful
already, and it could be used at any time.  So there is no correct
alternative except #2.

Putting the actual defcustom into loaddefs would also work.  We would
not want to do that for all autoloaded defcustoms.  For one thing,
there are defcustoms for which this won't work.  For others, it would
just waste space inside the dumped Emacs.  But there could be some
(perhaps diary-file is one) for which this would be better.

It would not be hard to implement this option.  How could we
specify whether to do #2 or this?

Re: Semantics of autoload cookies on defcustoms

[T. V. Raman]

The other negative consequence of putting an autoload cookie on a
defcustom is the following:

So let's say you have package foo.el and with option foo.
Let's further assume you've customized foo and saved your
setting.

If package  foo gets autloaded sometime after startup --- the
option foo ends up with its default value -- not your saved value
--- and you end up needing to do a "reset to saved".

>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:
    Richard>     What is the intended effect of adding an
    Richard> autoload cookie on a defcustom?  It puts a defvar
    Richard> and the custom-autoload call into loaddefs.
    Richard> 
    Richard>     I ask this question because I recently noticed
    Richard> that setting variables such as diary-file via custom
    Richard> causes calendar.el to be loaded at startup, even tho
    Richard> I can't see any reason why such a setting would
    Richard> justify eagerly loading calendar.el.
    Richard> 
    Richard> There was a reason for this, and I used to know it,
    Richard> but I have forgotten.
    Richard> 
    Richard> In order to process the specified value, custom
    Richard> needs the defcustom info.  So there are two options:
    Richard> 
    Richard> 1. Save the value away and process it if/when the
    Richard> defcustom is loaded.  2. Load the defcustom now, and
    Richard> process the value right away.
    Richard> 
    Richard> #1 is the usual method.  That is fine for variables
    Richard> that aren't really defined at all until the package
    Richard> is loaded.
    Richard> 
    Richard> But when a variable is autoloaded, that means its
    Richard> value is meaningful already, and it could be used at
    Richard> any time.  So there is no correct alternative except
    Richard> #2.
    Richard> 
    Richard> Putting the actual defcustom into loaddefs would
    Richard> also work.  We would not want to do that for all
    Richard> autoloaded defcustoms.  For one thing, there are
    Richard> defcustoms for which this won't work.  For others,
    Richard> it would just waste space inside the dumped Emacs.
    Richard> But there could be some (perhaps diary-file is one)
    Richard> for which this would be better.
    Richard> 
    Richard> It would not be hard to implement this option.  How
    Richard> could we specify whether to do #2 or this?
    Richard> 
    Richard> 
    Richard> _______________________________________________
    Richard> Emacs-devel mailing list Emacs-devel@gnu.org
    Richard> http://lists.gnu.org/mailman/listinfo/emacs-devel

-- 
Best Regards,
--raman

      
Email:  raman@users.sf.net
WWW:    http://emacspeak.sf.net/raman/
AIM:    emacspeak       GTalk: tv.raman.tv@gmail.com
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc
Google: tv+raman 
IRC:    irc://irc.freenode.net/#emacs

Re: Semantics of autoload cookies on defcustoms

[Stefan Monnier]

>     What is the intended effect of adding an autoload cookie on a defcustom?
> It puts a defvar and the custom-autoload call into loaddefs.

Yes, I saw that this is the actual behavior in autoload.el.
I was expecting a description of the intention behind this actual behavior.

>     I ask this question because I recently noticed that setting variables such
>     as diary-file via custom causes calendar.el to be loaded at startup, even
>     tho I can't see any reason why such a setting would justify eagerly loading
>     calendar.el.

> There was a reason for this, and I used to know it, but I have
> forgotten.

> In order to process the specified value, custom needs the defcustom
> info.

That's sometimes the case, but I can't see why it would be the case for
diary-file.  E.g. I understand the need for custom-autoload if the defcustom
has a :setter but for a plain defcustom such as diary-file it doesn't seem
to make much sense.

> So there are two options:

> 1. Save the value away and process it if/when the defcustom is loaded.
> 2. Load the defcustom now, and process the value right away.

> #1 is the usual method.  That is fine for variables that aren't really
> defined at all until the package is loaded.

> But when a variable is autoloaded, that means its value is meaningful
> already, and it could be used at any time.  So there is no correct
> alternative except #2.

So if I understand correctly, you're saying that diary-file should simply
not have an autoload cookie?


        Stefan

Re: Semantics of autoload cookies on defcustoms

[Stefan Monnier]

> The other negative consequence of putting an autoload cookie on a
> defcustom is the following:

> So let's say you have package foo.el and with option foo.
> Let's further assume you've customized foo and saved your
> setting.

> If package  foo gets autloaded sometime after startup --- the
> option foo ends up with its default value -- not your saved value
> --- and you end up needing to do a "reset to saved".

Please use another thread because I don't think it's related other
than circumstantially.  And please give a precise test case.


        Stefan


>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:
Richard> What is the intended effect of adding an
Richard> autoload cookie on a defcustom?  It puts a defvar
Richard> and the custom-autoload call into loaddefs.
Richard> 
Richard> I ask this question because I recently noticed
Richard> that setting variables such as diary-file via custom
Richard> causes calendar.el to be loaded at startup, even tho
Richard> I can't see any reason why such a setting would
Richard> justify eagerly loading calendar.el.
Richard> 
Richard> There was a reason for this, and I used to know it,
Richard> but I have forgotten.
Richard> 
Richard> In order to process the specified value, custom
Richard> needs the defcustom info.  So there are two options:
Richard> 
Richard> 1. Save the value away and process it if/when the
Richard> defcustom is loaded.  2. Load the defcustom now, and
Richard> process the value right away.
Richard> 
Richard> #1 is the usual method.  That is fine for variables
Richard> that aren't really defined at all until the package
Richard> is loaded.
Richard> 
Richard> But when a variable is autoloaded, that means its
Richard> value is meaningful already, and it could be used at
Richard> any time.  So there is no correct alternative except
Richard> #2.
Richard> 
Richard> Putting the actual defcustom into loaddefs would
Richard> also work.  We would not want to do that for all
Richard> autoloaded defcustoms.  For one thing, there are
Richard> defcustoms for which this won't work.  For others,
Richard> it would just waste space inside the dumped Emacs.
Richard> But there could be some (perhaps diary-file is one)
Richard> for which this would be better.
Richard> 
Richard> It would not be hard to implement this option.  How
Richard> could we specify whether to do #2 or this?
Richard> 
Richard> 
Richard> _______________________________________________
Richard> Emacs-devel mailing list Emacs-devel@gnu.org
Richard> http://lists.gnu.org/mailman/listinfo/emacs-devel

> -- 
> Best Regards,
> --raman

      
> Email:  raman@users.sf.net
> WWW:    http://emacspeak.sf.net/raman/
> AIM:    emacspeak       GTalk: tv.raman.tv@gmail.com
> PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc
> Google: tv+raman 
> IRC:    irc://irc.freenode.net/#emacs


> _______________________________________________
> Emacs-devel mailing list
> Emacs-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Semantics of autoload cookies on defcustoms

[Richard Stallman]

    If package  foo gets autloaded sometime after startup --- the
    option foo ends up with its default value -- not your saved value
    --- and you end up needing to do a "reset to saved".

I don't understand the scenario you describe here.
Could you be more specific?

Re: Semantics of autoload cookies on defcustoms

[Richard Stallman]

    > In order to process the specified value, custom needs the defcustom
    > info.

    That's sometimes the case, but I can't see why it would be the case for
    diary-file.  E.g. I understand the need for custom-autoload if the defcustom
    has a :setter but for a plain defcustom such as diary-file it doesn't seem
    to make much sense.

That may be true, but Custom has to load the defcustom in order to see
what it does or does not have.

    So if I understand correctly, you're saying that diary-file should simply
    not have an autoload cookie?

No, what I said does not go that far.  If it did not have an autoload
cookie, that would be better in this one way.  But there may be some
good reasons for it to have an autoload cookie.  I don't know.

Anyway, I proposed another approach:

    Putting the actual defcustom into loaddefs would also work.  We would
    not want to do that for all autoloaded defcustoms.  For one thing,
    there are defcustoms for which this won't work.  For others, it would
    just waste space inside the dumped Emacs.  But there could be some
    (perhaps diary-file is one) for which this would be better.

    It would not be hard to implement this option.  How could we
    specify whether to do #2 or this?

Any ideas?

Re: Semantics of autoload cookies on defcustoms

[Stefan Monnier]

>> In order to process the specified value, custom needs the defcustom
>> info.

>     That's sometimes the case, but I can't see why it would be the case
>     for diary-file.  E.g. I understand the need for custom-autoload if the
>     defcustom has a :setter but for a plain defcustom such as diary-file
>     it doesn't seem to make much sense.

> That may be true, but Custom has to load the defcustom in order to see
> what it does or does not have.

But autoload does know whether there is a :setter, so it can decide whether
to put a custom-autoload, thus informing Custom opf whether or not the
package needs to be eagerly loaded.

>     So if I understand correctly, you're saying that diary-file should simply
>     not have an autoload cookie?

> No, what I said does not go that far.  If it did not have an autoload
> cookie, that would be better in this one way.  But there may be some
> good reasons for it to have an autoload cookie.  I don't know.

What good reason could there be?
[ The same question applies to most other autoloaded defcustoms and
  defvars, I guess. ]

> Anyway, I proposed another approach:

>     Putting the actual defcustom into loaddefs would also work.  We would
>     not want to do that for all autoloaded defcustoms.  For one thing,
>     there are defcustoms for which this won't work.  For others, it would
>     just waste space inside the dumped Emacs.  But there could be some
>     (perhaps diary-file is one) for which this would be better.

>     It would not be hard to implement this option.  How could we
>     specify whether to do #2 or this?

AFAICT the answer depends on what is the intention behaind
the ;;;###autoload cookie used for those vars that don't need to preload the
package (and that don't have :setters either).


        Stefan

Re: Semantics of autoload cookies on defcustoms

[Richard Stallman]

    > That may be true, but Custom has to load the defcustom in order to see
    > what it does or does not have.

    But autoload does know whether there is a :setter, so it can decide whether
    to put a custom-autoload, thus informing Custom opf whether or not the
    package needs to be eagerly loaded.

That is true.  In the absence of a :set function and (perhaps) certain
other specific options, there is no need to load the file in order
to set the variable.

But it IS necessary to set the variable immediately, and not wait till
the file that defines it is loaded, in case the variable is used
elsewhere.  (This is not in fact the case for diary-file, but it could
be the case for other autoloaded defcustoms.)  This is what I proposed
as behavior #3.

Perhaps a good way to implement that is to write another argument into
the custom-autoload call.  That optional argument, if non-nil, would
specify this new combination of behaviors that I called #3.
The decision could be based on the presence of :set.

    > No, what I said does not go that far.  If it did not have an autoload
    > cookie, that would be better in this one way.  But there may be some
    > good reasons for it to have an autoload cookie.  I don't know.

    What good reason could there be?

1. To cause the variable to be visible for help commands
even if the file has not been loaded.

2. If other files use the variable.

Re: Semantics of autoload cookies on defcustoms

[Stefan Monnier]

> Perhaps a good way to implement that is to write another argument into
> the custom-autoload call.  That optional argument, if non-nil, would
> specify this new combination of behaviors that I called #3.
> The decision could be based on the presence of :set.

Well, there's no need for an extra arg: we just then skip the
custom-autoload and that should be all there is to it.

How 'bout the patch below?
What it does is the following:
1 - remove the minor-mode stuff which is unnecessary anyway (since the
    custom-autoload line added just above already ensures that the package
    gets loaded (which adds the setter) before the variable is custom-set).
2 - only add the custom-autoload line for those defcustom that have a setter
    since it seems to be the only ones that need it.

I guess this isn't quite right tho: the var should still cause autoloading
if we want customize it (since it needs to go fetch the :type at the very
least).  Is that what you meant by an additional arg to custom-autoload?


        Stefan


--- autoload.el	28 mai 2006 22:48:22 -0400	1.115
+++ autoload.el	09 jui 2006 01:27:08 -0400	
@@ -124,17 +124,11 @@
 	    )
 	`(progn
 	   (defvar ,varname ,init ,doc)
-	   (custom-autoload ',varname ,file)
-           ;; The use of :require in a defcustom can be annoying, especially
-           ;; when defcustoms are moved from one file to another between
-           ;; releases because the :require arg gets placed in the user's
-           ;; .emacs.  In order for autoloaded minor modes not to need the
-           ;; use of :require, we arrange to store their :setter.
            ,(let ((setter (condition-case nil
                               (cadr (memq :set form))
                             (error nil))))
-              (if (equal setter ''custom-set-minor-mode)
-                  `(put ',varname 'custom-set 'custom-set-minor-mode))))))
+              (when setter
+                `(custom-autoload ',varname ,file))))))
 
      ((eq car 'defgroup)
       ;; In Emacs this is normally handled separately by cus-dep.el, but for

Re: Semantics of autoload cookies on defcustoms

[Richard Stallman]

    -              (if (equal setter ''custom-set-minor-mode)
    -                  `(put ',varname 'custom-set 'custom-set-minor-mode))))))

I think it would be incorrect to delete those two lines.  In the
existing code, that `put' call is output in addition to the
`custom-autoload' call.  With your change, only the `custom-autoload'
call would be output in that case.

I am pretty sure we added that `put' call a few months ago to fix a bug.
Deleting it would bring back the bug.

Re: Semantics of autoload cookies on defcustoms

[Stefan Monnier]

>     -              (if (equal setter ''custom-set-minor-mode)
>     -                  `(put ',varname 'custom-set 'custom-set-minor-mode))))))

> I think it would be incorrect to delete those two lines.  In the
> existing code, that `put' call is output in addition to the
> `custom-autoload' call.  With your change, only the `custom-autoload'
> call would be output in that case.

> I am pretty sure we added that `put' call a few months ago to fix a bug.
> Deleting it would bring back the bug.

Actually, I don't think so.  I added that put because I thought it was
necessary when we removed the use of the :require property from
define-minor-mode.  But I didn't know about this custom-autoload back then.
AFAICT the custom-autoload already solves the problem and this `put' was
redundant from the start.


        Stefan

Re: Semantics of autoload cookies on defcustoms

[Richard Stallman]

    AFAICT the custom-autoload already solves the problem and this `put' was
    redundant from the start.

Ok.

Re: Semantics of autoload cookies on defcustoms

[Stefan Monnier]

>> Perhaps a good way to implement that is to write another argument into
>> the custom-autoload call.  That optional argument, if non-nil, would
>> specify this new combination of behaviors that I called #3.
>> The decision could be based on the presence of :set.

> Well, there's no need for an extra arg: we just then skip the
> custom-autoload and that should be all there is to it.

> How 'bout the patch below?
> What it does is the following:
> 1 - remove the minor-mode stuff which is unnecessary anyway (since the
>     custom-autoload line added just above already ensures that the package
>     gets loaded (which adds the setter) before the variable is custom-set).
> 2 - only add the custom-autoload line for those defcustom that have a setter
>     since it seems to be the only ones that need it.

> I guess this isn't quite right tho: the var should still cause autoloading
> if we want customize it (since it needs to go fetch the :type at the very
> least).  Is that what you meant by an additional arg to custom-autoload?

Here is a patch which I believe does it better:

- Add an arg to custom-autoload to mean that the variable should only be
  autoloaded for operations other than just `set'.
- Use this arg in autoload.el so that `set' only autoloads those variables
  that use a setter function.
- change custom.el so as to obey this new autoload alternative.
- change cus-edit.el to deal with the fact that those variables that have
  an autoload cookie but that are not autoloaded (because they don't have
  a setter) are not mistakenly marked as "changed outside customize".

With this patch, my Emacs starts up noticeably faster.
Should I install it?  Could some custom-theme specialist sanity-check it?


        Stefan


Index: lisp/emacs-lisp/autoload.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/emacs-lisp/autoload.el,v
retrieving revision 1.116
diff -u -r1.116 autoload.el
--- lisp/emacs-lisp/autoload.el	13 Jul 2006 18:13:06 -0000	1.116
+++ lisp/emacs-lisp/autoload.el	13 Jul 2006 19:29:16 -0000
@@ -124,7 +124,10 @@
 	    )
 	`(progn
 	   (defvar ,varname ,init ,doc)
-	   (custom-autoload ',varname ,file))))
+	   (custom-autoload ',varname ,file
+                            ,(condition-case nil
+                                 (null (cadr (memq :set form)))
+                               (error nil))))))
 
      ((eq car 'defgroup)
       ;; In Emacs this is normally handled separately by cus-dep.el, but for
Index: lisp/custom.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/custom.el,v
retrieving revision 1.127
diff -u -r1.127 custom.el
--- lisp/custom.el	13 May 2006 16:16:43 -0000	1.127
+++ lisp/custom.el	13 Jul 2006 19:29:16 -0000
@@ -558,9 +558,10 @@
     (unless (member load loads)
       (put symbol 'custom-loads (cons (purecopy load) loads)))))
 
-(defun custom-autoload (symbol load)
-  "Mark SYMBOL as autoloaded custom variable and add dependency LOAD."
-  (put symbol 'custom-autoload t)
+(defun custom-autoload (symbol load &optional noset)
+  "Mark SYMBOL as autoloaded custom variable and add dependency LOAD.
+If NOSET is non-nil, don't bother autoloading LOAD when setting the variable."
+  (put symbol 'custom-autoload (if noset 'noset t))
   (custom-add-load symbol load))
 
 ;; This test is also in the C code of `user-variable-p'.
@@ -827,9 +828,6 @@
 	    (if (and (eq prop 'theme-value)
 		     (boundp symbol))
 		(let ((sv (get symbol 'standard-value)))
-		  (when (and (null sv) (custom-variable-p symbol))
-		    (custom-load-symbol symbol)
-		    (setq sv (get symbol 'standard-value)))
 		  (unless (and sv
                                (equal (eval (car sv)) (symbol-value symbol)))
                     (setq old (list (list 'changed (symbol-value symbol))))))
@@ -907,6 +904,10 @@
 	    (when requests
 	      (put symbol 'custom-requests requests)
 	      (mapc 'require requests))
+            (unless (or (get symbol 'standard-value)
+                        (memq (get symbol 'custom-autoload) '(nil noset)))
+              ;; This symbol needs to be autoloaded, even just for a `set'.
+              (custom-load-symbol symbol))
 	    (setq set (or (get symbol 'custom-set) 'custom-set-default))
 	    (put symbol 'saved-value (list value))
 	    (put symbol 'saved-variable-comment comment)
Index: lisp/cus-edit.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/cus-edit.el,v
retrieving revision 1.296
diff -u -r1.296 cus-edit.el
--- lisp/cus-edit.el	12 Jul 2006 15:56:33 -0000	1.296
+++ lisp/cus-edit.el	13 Jul 2006 19:29:16 -0000
@@ -2668,7 +2668,18 @@
 			     (error nil))
 			   (cond
 			    ((eq (caar tmp) 'user) 'saved)
-			    ((eq (caar tmp) 'changed) 'changed)
+			    ((eq (caar tmp) 'changed)
+                             (if (condition-case nil
+                                     (and (null comment)
+                                          (equal value
+                                                 (eval
+                                                  (car (get symbol 'standard-value)))))
+                                   (error nil))
+                                 ;; The value was originally set outside
+                                 ;; custom, but it was set to the standard
+                                 ;; value (probably an autoloaded defcustom).
+                                 'standard
+                               'changed))
 			    (t 'themed))
 			 'changed))
 		      ((setq tmp (get symbol 'standard-value))

Re: Semantics of autoload cookies on defcustoms

[Richard Stallman]

    With this patch, my Emacs starts up noticeably faster.
    Should I install it?  Could some custom-theme specialist sanity-check it?

Please do install it.  If it has a problem, we will find out.

Thanks.

Re: Semantics of autoload cookies on defcustoms

[Stefan Monnier]

>     With this patch, my Emacs starts up noticeably faster.
>     Should I install it?  Could some custom-theme specialist sanity-check it?
> Please do install it.  If it has a problem, we will find out.

Done,


        Stefan