always put Customizations in `custom-file', never in `user-init-file'

always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

Having customizations go into `custom-file' has a great advantage over
having them stuck in `user-init-file' (.emacs): The entire `custom-file' is
generally not intended to be edited by a user, whereas `user-init-file' is
specifically intended to be edited.

I think it was probably a mistake ever to have put Customize customizations
into a file intended for editing. That can confuse users, and it's just
asking for trouble. Now that we have a good alternative (`custom-file'), I
think that should be used exclusively.

nil should then not be allowed as the value of `custom-file'. The default
value could be something like `~/.emacs-custom-file'.

You might say that this would be too disruptive, because many people already
have Customizations in `user-init-file'.

To handle that, Emacs would move any existing Customizations from
`user-init-file' to `custom-file' (replacing them at that place by a load of
`custom-file'). This would be done when `user-init-file' is loaded if
Customizations are encountered there. The user would be informed when this
happens. Customize could even be opened to `custom-file' at that time, to
let the user specify the `custom-file' to use.

A user would not be able to set `custom-file' to the value of
`user-init-file' or vice versa: an error would be raised whenever either is
attempted.

WDOT?

Re: always put Customizations in `custom-file', never in `user-init-file'

[Eric Hanchrow]

>>>>> "Drew" == Drew Adams <drew.adams@oracle.com> writes:

    Drew> nil should then not be allowed as the value of
    Drew> `custom-file'.  The default value could be something like
    Drew> `~/.emacs-custom-file'.

    Drew> WDOT?

Fine idea.  The custom stuff in my .emacs has always made me very
slightly uneasy.

-- 
A novice of the temple once approached the Master Programmer with a question.
"Master, does Emacs have the Buddha nature?" the novice asked.
The Master Programmer had been in the temple for many years and
could be relied upon to know these things.  He thought for
several minutes before replying.  "I don't see why not.  It's
bloody well got everything else."

Re: always put Customizations in `custom-file', never in `user-init-file'

[Lennart Borgman (gmail)]

Drew Adams wrote:
> I think it was probably a mistake ever to have put Customize customizations
> into a file intended for editing. That can confuse users, and it's just
> asking for trouble. Now that we have a good alternative (`custom-file'), I
> think that should be used exclusively.
...
> WDOT?


Great. And I guess custom-file should be run after .emacs, since that 
makes it possible to chose another custom-file.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

Lennart Borgman (gmail) writes:

 > Drew Adams wrote:
 > > I think it was probably a mistake ever to have put Customize customizations
 > > into a file intended for editing. That can confuse users, and it's just
 > > asking for trouble. Now that we have a good alternative (`custom-file'), I
 > > think that should be used exclusively.
 > ...
 > > WDOT?

Be very careful.  In particular, the natural idea of migrating by
loading the init file and using Custom to save its internal state to a
custom-file caused no end of pain to users when it was implemented in
XEmacs.  If there is any error in the init file, you can lose all
customizations.

 > Great. And I guess custom-file should be run after .emacs, since that 
 > makes it possible to chose another custom-file.

Experience in XEmacs showed that users often want to use information
in the custom-file in their init files.  There were issues (maybe
XEmacs-specific) with initial values for faces, too.  After the init
file was run was too late.  It's easy enough to load the custom-file
late for those users for whom that matters: don't use the default
name.  Something like

(defvar custom-file "custom.el")
(defvar default-custom-file-loaded-p nil)
(progn
  (do-emacs-early-initializations)
  (when (file-readable-p custom-file)
    (load custom-file))
  (load user-init-file)
  (when (and (not default-custom-file-loaded-p)
             (file-readable-p custom-file))
    (load custom-file)))

Re: always put Customizations in `custom-file', never in `user-init-file'

[Lennart Borgman (gmail)]

Stephen J. Turnbull wrote:
> Lennart Borgman (gmail) writes:
> 
>  > Drew Adams wrote:
>  > > I think it was probably a mistake ever to have put Customize customizations
>  > > into a file intended for editing. That can confuse users, and it's just
>  > > asking for trouble. Now that we have a good alternative (`custom-file'), I
>  > > think that should be used exclusively.
>  > ...
>  > > WDOT?
> 
> Be very careful.  In particular, the natural idea of migrating by
> loading the init file and using Custom to save its internal state to a
> custom-file caused no end of pain to users when it was implemented in
> XEmacs.  If there is any error in the init file, you can lose all
> customizations.

Then it might be best to avoid automatic migration, but to give a 
command to migrate custom data to custom-file.

>  > Great. And I guess custom-file should be run after .emacs, since that 
>  > makes it possible to chose another custom-file.
> 
> Experience in XEmacs showed that users often want to use information
> in the custom-file in their init files.  There were issues (maybe
> XEmacs-specific) with initial values for faces, too.  After the init
> file was run was too late.  It's easy enough to load the custom-file
> late for those users for whom that matters: don't use the default
> name.  Something like
> 
> (defvar custom-file "custom.el")
> (defvar default-custom-file-loaded-p nil)
> (progn
>   (do-emacs-early-initializations)
>   (when (file-readable-p custom-file)
>     (load custom-file))
>   (load user-init-file)
>   (when (and (not default-custom-file-loaded-p)
>              (file-readable-p custom-file))
>     (load custom-file)))

Yes, you are of course right.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> > Be very careful.  In particular, the natural idea of migrating by
> > loading the init file and using Custom to save its internal state to a
> > custom-file caused no end of pain to users when it was implemented in
> > XEmacs.  If there is any error in the init file, you can lose all
> > customizations.
>
> Then it might be best to avoid automatic migration, but to give a
> command to migrate custom data to custom-file.

I don't see the problem, if it's understood that `custom-file' would not be
loaded automatically, but only when (where) the user requests via (load-file
custom-file).

Re: always put Customizations in `custom-file', never in `user-init-file'

[Lennart Borgman (gmail)]

Drew Adams wrote:
>>> Be very careful.  In particular, the natural idea of migrating by
>>> loading the init file and using Custom to save its internal state to a
>>> custom-file caused no end of pain to users when it was implemented in
>>> XEmacs.  If there is any error in the init file, you can lose all
>>> customizations.
>> Then it might be best to avoid automatic migration, but to give a
>> command to migrate custom data to custom-file.
> 
> I don't see the problem, if it's understood that `custom-file' would not be
> loaded automatically, but only when (where) the user requests via (load-file
> custom-file).

If you do not mean that custom-file should be loaded automatically then 
I must have misunderstood you. I thought it would be a good idea to by 
default have custom-file separate, but as Jason points out there was 
probably a good reason why this was not done from the beginning. Though 
I can't remember that reason now. Maybe it had something to do with the 
order of loading? Maybe it was some problem with custom-set-variables 
from the beginning?

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> > `custom-file' would not be loaded automatically, but only
> > when (where) the user requests via (load-file custom-file).
>
> If you do not mean that custom-file should be loaded automatically then
> I must have misunderstood you.

Yes, I think you did. I said that `custom-file' would be loaded explicitly:
(load-file custom-file). Just as it is now. All I proposed was that any
Customize stuff currently in `user-init-file' should be moved to
`custom-file'.

> I thought it would be a good idea to by
> default have custom-file separate,

You were right. And not just by default.

> but as Jason points out there was
> probably a good reason why this was not done from the beginning.

So what you thought would be a good idea turns out in fact to be a bad idea
simply because someone designed the status quo? Sheesh! You say you don't
know what the arguments behind the design were, but, although unknown, the
fact that they "probably" existed is enough to convince you to change your
mind?

We need never think again: our world is a product of decisions and other
actions, many of which were "probably" made consciously. Let it be.

What kind of an argument is that? Of course there was "probably a good
reason". That doesn't mean that the best decision was made then or, a
fortiori, that that decision is also the best decision now.

If you want to make specific arguments, whether repeated from a previous
discussion or not, please do. But let's not argue only from authority,
saying that whatever is, is right.

> Though I can't remember that reason now. Maybe it had something
> to do with the order of loading? Maybe it was some problem
> with custom-set-variables from the beginning?

Maybe those maybe's don't help. What about the order of loading? With an
explicit (load-file custom-file), you can have any order you like. What
problem with custom-set-variables from the beginning?

It doesn't help here to grasp at straws or appeal to authority. Let's
discuss the pros and cons and forget about imagining what someone might have
thought about this long ago, if it was in fact thought about in this way.

This is nuts. We can't have a discussion if as soon as someone says, in
effect, "that's the way it is because that's the way it was decided" we stop
thinking about it.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Lennart Borgman (gmail)]

Drew Adams wrote:
>>> `custom-file' would not be loaded automatically, but only
>>> when (where) the user requests via (load-file custom-file).
>> If you do not mean that custom-file should be loaded automatically then
>> I must have misunderstood you.
> 
> Yes, I think you did. I said that `custom-file' would be loaded explicitly:
> (load-file custom-file). Just as it is now. All I proposed was that any
> Customize stuff currently in `user-init-file' should be moved to
> `custom-file'.

If custom-file is not loaded by default then it makes it even more 
difficult for users so I think it have to be loaded by default.

If something has to be done before custom-file I would prefer a separate 
file, .pre-custom-emacs or something. The only thing I believe would 
have to go there is actually the name of custom-file (if the user wants 
to change it).

>> Though I can't remember that reason now. Maybe it had something
>> to do with the order of loading? Maybe it was some problem
>> with custom-set-variables from the beginning?
> 
> Maybe those maybe's don't help. What about the order of loading? With an
> explicit (load-file custom-file), you can have any order you like. What
> problem with custom-set-variables from the beginning?

It was probably me misunderstanding custom-set-variables from the 
beginning. I just do not remember at the moment.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> > `custom-file' would be loaded explicitly:
> > (load-file custom-file). Just as it is now. All I
> > proposed was that any Customize stuff currently in
> > `user-init-file' should be moved to `custom-file'.
>
> If custom-file is not loaded by default then it makes it even more
> difficult for users

Makes what more difficult for users? More difficult than what?

`custom-file' has never been loaded automatically. Why propose that? What
problem would that solve?

> so I think it have to be loaded by default.

Why?

When would you load it, wrt loading `user-init-file'? Before? After? In the
middle somewhere?

What would you do if the `user-init-file' or some file it loads has an
explicit (load-file `custom-file')? What if the automatic load had already
happened?

Users should be able to control when Customize customizations take place.
Loading `custom-file' automatically would reduce their control. That would
be a drastic, and negative, change from the current behavior.

> If something has to be done before custom-file I would prefer a separate
> file, .pre-custom-emacs or something. The only thing I believe would
> have to go there is actually the name of custom-file (if the user wants
> to change it).

That way lies madness...

How can you decide (for all users?) what "would have to go there" or when
`custom-file' should be loaded?

We need only two files: `user-init-file' and `custom-file'. The former
should be for users to edit (possibly to load other files). Editing the
latter should be discouraged: normally, users should not need to open
`custom-file' in an editor.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Jason Rumney]

Drew Adams wrote:
> `custom-file' has never been loaded automatically. Why propose that? What
> problem would that solve?
>   

It would solve a problem that does not exist until the default value for
custom-file is changed. That problem is that a new user has no way of
knowing, and nor should they, that they have to manually add a line to
their .emacs to get their customizations loaded.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Lennart Borgman (gmail)]

Drew Adams wrote:
> We need only two files: `user-init-file' and `custom-file'. The former
> should be for users to edit (possibly to load other files). Editing the
> latter should be discouraged: normally, users should not need to open
> `custom-file' in an editor.


Yes, see my reply to Stephen.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

Lennart Borgman (gmail) writes:

 > Then it might be best to avoid automatic migration, but to give a 
 > command to migrate custom data to custom-file.

Well, it's embarrassing to admit, but we really screwed up.  Keeping a
backup of the original .emacs should have been enough.  We didn't.
The thing to be aware of, is that after aborting due to an error,
there are no customizations known to customize, but users will often
save a few customizations so that they can get some work done.  This
will then destroy the customization file (at least in our version of
customize) since a custom-save simply writes out the whole state.
Custom has some protection against doing this automatically, I think,
but the migration code wasn't aware of it, and neither were users.
Ooops!  Tricky, very tricky.

The other thing to note is that this was typically reported by people
using Emacs and trying XEmacs.  Normally you don't have errors in your
init file. :-)

I think separate custom-file and automigration are both good ideas.
Just be aware that (at least in our implementation of custom, but I
think it's probably true of Emacs's, too) there are some ways to go
badly wrong, and be careful to protect the user's init file and
existing customizations.

 > Yes, you are of course right.

In hindsight, yes, but we had to learn that from experience.  The
experience is there, I'd like you to benefit from it.

Re: always put Customizations in `custom-file', never in `user-init-file'

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> The other thing to note is that this was typically reported by people
> using Emacs and trying XEmacs.  Normally you don't have errors in your
> init file. :-)
>
> I think separate custom-file and automigration are both good ideas.

I don't think automigration is a good idea.  The one point of time where
it happens is when you try out a new version of Emacs/XEmacs the very
first time, and that is exactly when you don't want to have it change
your setup or nag you about it while you are still in test drive mode.

> In hindsight, yes, but we had to learn that from experience.  The
> experience is there, I'd like you to benefit from it.

And that is appreciated.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

David Kastrup writes:

 > I don't think automigration is a good idea.  The one point of time where
 > it happens is when you try out a new version of Emacs/XEmacs the very
 > first time, and that is exactly when you don't want to have it change
 > your setup or nag you about it while you are still in test drive mode.

That's the user's point of view.  At the time of introducing
automigration, the number of people who decided to edit custom-set-*
forms, or introduce multiple instances, and got into trouble was quite
annoying.  We don't hear about that kind of bug at all any more,
except from a few retro "everything in .emacs" types.  Who are
properly embarrassed about failing to follow good advice, usually. :-)

Automigration is good for the maintainers at (some, small) expense to
the users.  This isn't as important an issue as the "read you your
rights" splash screen, but the fairly minor annoyance you describe can
be handled in the same way, with equal success in our experience.
Heck, we got a *lot* less pushback on automigration than you guys did
with the splash screen.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> Well, it's embarrassing to admit, but we really screwed up.  Keeping a
> backup of the original .emacs should have been enough.

I don't know if it's sufficient, but it's necessary.

> We didn't.
> The thing to be aware of, is that after aborting due to an error,
> there are no customizations known to customize, but users will often
> save a few customizations so that they can get some work done.

I'm not sure I follow completely. Could you run down such a scenario,
including when the error occurred? During which Emacs session: before,
during, or after migration, and when during the session?

> This will then destroy the customization file (at least in our version
> of customize) since a custom-save simply writes out the whole state.
> Custom has some protection against doing this automatically, I think,
> but the migration code wasn't aware of it, and neither were users.
> Ooops!  Tricky, very tricky.
>
> The other thing to note is that this was typically reported by people
> using Emacs and trying XEmacs.  Normally you don't have errors in your
> init file. :-)
>
> I think separate custom-file and automigration are both good ideas.

Let's try to find a safe way to do it, then.

> Just be aware that (at least in our implementation of custom, but I
> think it's probably true of Emacs's, too) there are some ways to go
> badly wrong, and be careful to protect the user's init file and
> existing customizations.
>
>  > Yes, you are of course right.
>
> In hindsight, yes, but we had to learn that from experience.  The
> experience is there, I'd like you to benefit from it.

Thanks for reporting the experience. It sounds to me like the potential
gotchas could be handled OK, but it would be good for someone more expert in
this area to comment. And yes, we should proceed with any such change
carefully.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

Drew Adams writes:

 > I'm not sure I follow completely. Could you run down such a scenario,
 > including when the error occurred? During which Emacs session: before,
 > during, or after migration, and when during the session?

No, I can't be precise because I'm not familiar with the Emacs startup
process.  In the XEmacs cases I recall, the error occured during
migration when the init file is being loaded but before the
customizations were initialized (or perhaps partially initialized).
Then the user would customize some options and custom-save after
return to top-level, which would omit any customizations that had not
been initialized.

 > Let's try to find a safe way to do it, then.

It's not terribly unsafe in terms of odds; thousands of people have
migrated init files with only a few having problems.  Thing is, a
large collection of Customizations is extremely costly to reproduce,
and will be a nagging annoyance for weeks or months.

So the most important thing is to keep a backup copy of the user init
file during migration.  It's also important to be aware that there are
potential pitfalls to using custom-save-all to write the new
customizations file, because it writes the whole database in one shot
based on current state, rather than merging changes into the on-disk
version.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> Be very careful.  In particular, the natural idea of migrating by
> loading the init file and using Custom to save its internal state to a
> custom-file caused no end of pain to users when it was implemented in
> XEmacs.  If there is any error in the init file, you can lose all
> customizations.

Why? And in what circumstance?

Do you mean an error that pre-exists in the init file or one that is
introduced during migration?

If the former, then why would anything be different after the migration?

If there is not already an error in the init file, then I guess you mean
that an error is introduced by substituting (load-file custom-file) for the
custom-set-* sexps in the init file. What kinds of errors were there? There
could be a disk write error, for instance, but I'm curious what kinds of
errors you are speaking of.

A copy should be made of the original init file, of course, so that in case
of error on next startup, the user can recuperate.

Another possibility would be to copy the Customization code from
`user-init-file' to `custom-file' and then comment-out that code in
`user-init-file' (instead of removing it altogether).

>  > Great. And I guess custom-file should be run after .emacs, since that
>  > makes it possible to chose another custom-file.

I am against that - see my separate response to Lennart.

> Experience in XEmacs showed that users often want to use information
> in the custom-file in their init files.  There were issues (maybe
> XEmacs-specific) with initial values for faces, too.  After the init
> file was run was too late.  It's easy enough to load the custom-file
> late for those users for whom that matters: don't use the default
> name.  Something like
>
> (defvar custom-file "custom.el")
> (defvar default-custom-file-loaded-p nil)
> (progn
>   (do-emacs-early-initializations)
>   (when (file-readable-p custom-file)
>     (load custom-file))
>   (load user-init-file)
>   (when (and (not default-custom-file-loaded-p)
>              (file-readable-p custom-file))
>     (load custom-file)))

Where is such code? I assume it's not in `user-init-file', since this code
loads `user-init-file'.

In any case, IIUYC, users should be able to do whatever they like in the new
situation also, because it would only load `custom-file' upon explicit
request, never automatically.

Users need to be able to control when Customizations get loaded. I, for
example, load `custom-file' last, at the end of my .emacs. Others load it
first. Still others load it somewhere in the middle. There should be no
automatic loading of `custom-file'.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

Drew Adams writes:

 > Do you mean an error that pre-exists in the init file or one that is
 > introduced during migration?

Most init files have pre-existing errors if you are trying multiple
Emacsen, all of which load .emacs if it's present.  That was the
common case.

 > If the former, then why would anything be different after the migration?

Our experience was that most people would first encounter migration
around the time of a major release, which is also the time they
typically think about other Emacsen to see what it offers.  This
happened in late beta, where backward incompatibilities (not fboundp,
void-variable, etc) still arise frequently.  The users were aware of
the beta status, but they were still quite shocked that their
customizations were destroyed.

 > Where is [the code that manages loading of `custom-file']? I assume
 > it's not in `user-init-file', since this code loads `user-init-file'.

It requires changing the Emacs startup code.  It needs to be changed
if Emacs wants to do automigration anyway.

 > In any case, IIUYC, users should be able to do whatever they like in the new
 > situation also, because it would only load `custom-file' upon explicit
 > request, never automatically.

Our experience is that there are a lot of users who do fine with *no*
*init file* *at all*.  Use of Customization should not depend on the
presence of an init file, and even one pair of parentheses is enough
to scare some users.  (Cf. "A Brief History of Time".  I don't think
that "(load custom-file)" deserves anywhere near the consideration
that "E = mc^2" got from Prof. Hawking.)

 > Users need to be able to control when Customizations get loaded. I, for
 > example, load `custom-file' last, at the end of my .emacs. Others load it
 > first. Still others load it somewhere in the middle. There should be no
 > automatic loading of `custom-file'.

Our experience says that automatic "after" loading is fine for 99% of
users (that's all we actually provide in a released version), and that
"before" is satisfactory for the rest.  (There may be users we haven't
encountered who *need* finer control, but the fact that in the 7 years
since December 2000 we haven't encountered *one* suggests there are
very few.)  The reason I suggest providing the "before" option is that
I know there are use cases for it.  I have never heard of anybody who
*needed* it loaded in the middle of the init file.  I *have* helped
debug problems that happened because customizations were loaded in the
middle of the init file rather than at the appropriate end, though.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Lennart Borgman (gmail)]

Stephen J. Turnbull wrote:

> Our experience says that automatic "after" loading is fine for 99% of
> users (that's all we actually provide in a released version), and that
> "before" is satisfactory for the rest.  (There may be users we haven't
> encountered who *need* finer control, but the fact that in the 7 years
> since December 2000 we haven't encountered *one* suggests there are
> very few.)  The reason I suggest providing the "before" option is that
> I know there are use cases for it.  I have never heard of anybody who
> *needed* it loaded in the middle of the init file.  I *have* helped
> debug problems that happened because customizations were loaded in the
> middle of the init file rather than at the appropriate end, though.

Ah, now I think I remember. A good way to handle the load sequence could be:

   1) load .emacs
   2) if .emacs did not load custom-file then do that
      immediately after .emacs.

That solves the problem with the loading sequence with very little 
trouble (I believe). Those users which want to use the values from 
custom-set-variables in their .emacs will have to learn to insert

   (load custom-file)

in their .emacs but I would expect that to be easy for those users.

But, Stephen, maybe you see some problem with this?

Re: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

Lennart Borgman (gmail) writes:

 > Ah, now I think I remember. A good way to handle the load sequence could be:
 > 
 >    1) load .emacs
 >    2) if .emacs did not load custom-file then do that
 >       immediately after .emacs.
 > 
 > That solves the problem with the loading sequence with very little 
 > trouble (I believe). Those users which want to use the values from 
 > custom-set-variables in their .emacs will have to learn to insert
 > 
 >    (load custom-file)
 > 
 > in their .emacs but I would expect that to be easy for those users.
 > 
 > But, Stephen, maybe you see some problem with this?

There was some objection from users who wanted custom-file loaded
before user-init-file, but I don't remember anything concrete.  I
think the problem was mainly esthetic: custom-file is managed behind
the scenes by XEmacs, and it should stay there unless there's strong
reason.

Emacs's esthetics are different though, and that approach certainly
works.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> > Our experience says that automatic "after" loading is fine for 99% of
> > users (that's all we actually provide in a released version), and that
> > "before" is satisfactory for the rest.  (There may be users we haven't
> > encountered who *need* finer control, but the fact that in the 7 years
> > since December 2000 we haven't encountered *one* suggests there are
> > very few.)  The reason I suggest providing the "before" option is that
> > I know there are use cases for it.  I have never heard of anybody who
> > *needed* it loaded in the middle of the init file.  I *have* helped
> > debug problems that happened because customizations were loaded in the
> > middle of the init file rather than at the appropriate end, though.
>
> Ah, now I think I remember. A good way to handle the load
> sequence could be:
>
>    1) load .emacs
>    2) if .emacs did not load custom-file then do that
>       immediately after .emacs.

Yes, I mentioned that also. I agree that that is workable. (After loading
.emacs includes, of course, after loading any files .emacs might load.)

> That solves the problem with the loading sequence with very little
> trouble (I believe). Those users which want to use the values from
> custom-set-variables in their .emacs will have to learn to insert
>
>    (load custom-file)
>
> in their .emacs but I would expect that to be easy for those users.

Yes, that is what I would suggest too. If loading the custom file "is fine
for 99% of users", then the rest (which, according to Stephen, is nearly the
other 1%) can add (load custom-file) where they want it.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

>  > In any case, IIUYC, users should be able to do whatever they
>  > like in the new situation also, because it would only load
>  > `custom-file' upon explicit request, never automatically.
>
> Our experience is that there are a lot of users who do fine with *no*
> *init file* *at all*.  Use of Customization should not depend on the
> presence of an init file, and even one pair of parentheses is enough
> to scare some users.

Not sure I understand.

Currently, if a user doesn't have an init file, then s?he can use Customize
but s?he cannot save Customize customizations. That would not change with
what I propose.

And if s?he doesn't try to save Customize customizations, then she still
need not have an init file.

If a user wants to save Customize customizations, then s?he will need both
files in my proposal, whereas s?he currently needs only an init file.

We should, in any case, make it easy for a user, after using Customize (but
not being able to save, because no files), to create an init file that
contains (load-file custom-file) - but see also below. And we should give
the user a chance to specify the files to use for `user-init-file' and
`custom-file'.

>  > Users need to be able to control when Customizations get loaded. I, for
>  > example, load `custom-file' last, at the end of my .emacs.
>  > Others load it first. Still others load it somewhere in the middle.
>  > There should be no automatic loading of `custom-file'.
>
> Our experience says that automatic "after" loading is fine
> for 99% of users

I suppose that we could adopt the convention that if `custom-file' was not
loaded by the init file (or by any files it loaded), but variable
`custom-file' is defined at the end of this loading, then `custom-file'
would be loaded at that time.

Another approach would be to display a warning at the end of init file
loading that `custom-file' was defined but never loaded. That's perhaps a
bit of a cop-out.

Customize should in any case not add (load-file custom-file) to the init
file automatically, except possibly when the init file is empty. The user
might instead have (load-file custom-file) in some other file (that is
loaded) - Customize should not assume anything about where/when
`custom-file' is loaded.

> (that's all we actually provide in a released version), and that
> "before" is satisfactory for the rest.  (There may be users we haven't
> encountered who *need* finer control, but the fact that in the 7 years
> since December 2000 we haven't encountered *one* suggests there are
> very few.)

That sounds about right to me. But there is no reason not to give users the
flexibility to control when `custom-file' is loaded, is there?

> The reason I suggest providing the "before" option is that
> I know there are use cases for it.  I have never heard of anybody who
> *needed* it loaded in the middle of the init file.  I *have* helped
> debug problems that happened because customizations were loaded in the
> middle of the init file rather than at the appropriate end, though.

Again, that all sounds right to me.

Are you suggesting having an explicit option for this, with `before' and
`after' values? That has the advantage that it obviates needing to put
(load-file custom-file) in the init file.

On the other hand, it loses some flexibility - you might want to load
`custom-file' after some event, or in some situation but not in others, or
after loading some feature, or...  Granted, it might be that few users would
need or want that flexibility, but why not allow it?

A better possibility would be to have option values `before', `after', and
nil, with the latter meaning no automatic loading - the user could still use
an explicit (load-file custom-file).

Of course, there might be a problem for Emacs to try to know the value of
that option without having yet loaded `custom-file'... ;-) Back to editing
the init file, no? Or to having Customize put something in the init file to
say when `custom-file' should be loaded... Neither of which would help those
users you mention who want saved customizations but no init file.

Overall, I think (so far) the best solution is for users to put (load-file
custom-file) in their init files, and to help them do that when they first
try to save Customize settings, and to help them migrate from an existing
init file that contains Customize settings.

At the end of the day, in my view, a user would have either no init file and
no custom file or both. And the init file would contain (load-file
custom-file) somewhere (with a comment), or else the custom file would not
get loaded (or it could be loaded at init end, as mentioned above).

BTW, it might also be worth putting something at the beginning of the custom
file to detect its being loaded more than once - perhaps raise an error in
that case.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

Drew Adams writes:

 > Currently, if a user doesn't have an init file, then s?he can use Customize
 > but s?he cannot save Customize customizations. That would not change with
 > what I propose.

IIRC, in XEmacs, that was false: if you had no .emacs and tried to
save, Customize would create it for you.  (Presumably after prompting.)

 > If a user wants to save Customize customizations, then s?he will need both
 > files in my proposal, whereas s?he currently needs only an init file.

Inelegant, and confusing to the user.

 > And we should give the user a chance to specify the files to use
 > for `user-init-file' and `custom-file'.

The kind of user who benefits from total automation doesn't *want*
that, and the kind of user who doesn't benefit doesn't *need* that.

 > I suppose that we could adopt the convention that if `custom-file' was not
 > loaded by the init file (or by any files it loaded), but variable
 > `custom-file' is defined at the end of this loading, then `custom-file'
 > would be loaded at that time.

What do you have against customizations, that you're willing to go to
such lengths that nobody will ever accidently have their
customizations loaded?  IMO, the unusual case is where custom-file
exists but the user doesn't want it loaded at normal startup.  I think
it's perfectly reasonable to have such users set custom-file to a
non-nil symbol and have Emacs compute

    (progn
      (setq custom-file (or custom-file user-init-file))
      (when (and (stringp custom-file)
                 (file-readable-p custom-file))
        (load custom-file)))

where it loads the custom-file.  This is backwards compatible with the
current meaning of nil, and allows users to suppress loading of the
Customizations.

 > Customize should not assume anything about where/when
 > `custom-file' is loaded.

I don't understand.  Customize is about providing a hood-welded-shut
facility for customizations.  Warranty is void if this seal is broken.
ALL FACES YOURS NOW OURS ARE.

This is not a problem.  setq works on defcustoms, you know!

 > > (that's all we actually provide in a released version), and that
 > > "before" is satisfactory for the rest.  (There may be users we haven't
 > > encountered who *need* finer control, but the fact that in the 7 years
 > > since December 2000 we haven't encountered *one* suggests there are
 > > very few.)
 > 
 > That sounds about right to me. But there is no reason not to give users the
 > flexibility to control when `custom-file' is loaded, is there?

*shrug* It's theirs for the taking.  The same after-init-hook hack
that currently allows the "before" users to put (load custom-file) at
the very top of their user-init-file allows them to put it anywhere
else they want, too.

;;; load custom.el here, not after the init file
(load custom-file)
(setq custom-file-saved custom-file
      custom-file nil)
(add-hook 'after-init-hook (lambda () (setq custom-file custom-file-saved)))

Of course the hacker who writes the support for automatic "after"
loading needs to ensure that after-init-hook is called *after*
custom-file would be loaded.

 > Are you suggesting having an explicit option for this, with `before' and
 > `after' values? That has the advantage that it obviates needing to put
 > (load-file custom-file) in the init file.

No.  As you note, such an option would need to be a command-line
option.  Rather, people who insist on "after" would rename their
custom-file---I used "~/.xemacs/after-init.el" when I was
experimenting---and put something like

(setq custom-file (expand-file-name "after-init.el" "~/.xemacs"))

in their user-init-file.  This is preferable to automatic "after"
loading primarily because using after-init-hook looks more fragile,
and this has more robust failure modes (the custom.el-was-loaded flag
is managed by Emacs transparently to the user, so custom.el won't be
loaded twice unless the user does it explicitly).

 > On the other hand, it loses some flexibility - you might want to load
 > `custom-file' after some event, or in some situation but not in others, or
 > after loading some feature, or...  Granted, it might be that few users would
 > need or want that flexibility, but why not allow it?

"Option overload" is not a good thing.  Users who want to take the
flexibility can do so using standard functionality.  Other users just
don't want to know, and I don't blame them. ;-)

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

>  > Currently, if a user doesn't have an init file, then s?he can
>  > use Customize but s?he cannot save Customize customizations.
>  > That would not change with what I propose.
>
> IIRC, in XEmacs, that was false: if you had no .emacs and tried to
> save, Customize would create it for you.  (Presumably after prompting.)
>
>  > If a user wants to save Customize customizations, then s?he
>  > will need both files in my proposal, whereas s?he currently
>  > needs only an init file.
>
> Inelegant, and confusing to the user.

How do you do it in XEmacs (after the migration)? I think you said that
users can have a custom file without an init file. Where do they let Emacs
know what file is their custom file? If they don't set `custom-file' in the
init file, where is it set and read at startup?

The most inelegant and most confusing is what we have now in GNU Emacs, IMO:
mixing user hand customizations with Customize-generated code in a single
file.

We should move to a system of separate files for user editing and Customize.
That's the first thing to define, including handling these cases:

- a user who starts from scratch and doesn't know or care about Lisp
- users who want to specify the location of their custom file
- users who want to load the custom file at different times
  during startup, via an explicit (load-file custom-file)

The question of migration is a separate question. We should first define the
desired end state.

>  > And we should give the user a chance to specify the files to use
>  > for `user-init-file' and `custom-file'.

I should have mentioned only `custom-file'. Variable `user-init-file' is not
a user option, but you can specify the init file using a command-line
option.

> The kind of user who benefits from total automation doesn't *want*
> that, and the kind of user who doesn't benefit doesn't *need* that.

Please describe the "total automation". But see below; perhaps I've guessed
it sufficiently.

I'm one user who might not benefit from total automation (depending on what
you mean), and I do want to specify my custom file location. (The Emacs
manual, node Saving Customizations, gives an example of why a user might
want to specify it.)

>  > I suppose that we could adopt the convention that if
>  > `custom-file' was not loaded by the init file (or by any
>  > files it loaded), but variable `custom-file' is defined
>  > at the end of this loading, then `custom-file'
>  > would be loaded at that time.
>
> What do you have against customizations, that you're willing to go to
> such lengths that nobody will ever accidently have their
> customizations loaded?

Huh? I don't have anything against customizations. What lengths do you think
I'm going to? I don't understand your point here. What is the problem you
see with what I suggested? I think it's the same thing Lennart suggested,
which you said "certainly works":

>>    1) load .emacs
>>    2) if .emacs did not load custom-file then do that
>>       immediately after .emacs.

> IMO, the unusual case is where custom-file
> exists but the user doesn't want it loaded at normal startup.

I don't disagree with that.

> I think it's perfectly reasonable to have such users set
> custom-file to a non-nil symbol and have Emacs compute
>
>     (progn
>       (setq custom-file (or custom-file user-init-file))
>       (when (and (stringp custom-file)
>                  (file-readable-p custom-file))
>         (load custom-file)))
>
> where it loads the custom-file.  This is backwards compatible with the
> current meaning of nil, and allows users to suppress loading of the
> Customizations.

I guess that is some proposed standard startup code (and not user code). I
think you're saying that a user who for some reason doesn't want to load an
existing custom file would set `custom-file' to a non-nil symbol, say
`foobar', to avoid loading it. Is that right?

I would have said that s?he would just set `custom-file' to nil: if the
value doesn't name an existing file, then no custom file gets loaded (in GNU
Emacs, but perhaps not in XEmacs).

But why are we talking about this unusual case? Did you think I was
referring to it? When/where would this code be executed? Would it run in
addition to loading the init file or instead of that? Sorry, I don't really
see your point.

>  > Customize should in any case not add (load-file custom-file)
>  > to the init file automatically, except possibly when the init
>  > file is empty. The user might instead have (load-file
>  > custom-file) in some other file (that is loaded) - Customize
>  > should not assume anything about where/when `custom-file' is
>  > loaded.
>
> I don't understand.  Customize is about providing a hood-welded-shut
> facility for customizations.  Warranty is void if this seal is broken.
> ALL FACES YOURS NOW OURS ARE. This is not a problem.  setq works on
> defcustoms, you know!

I have no idea what you are talking about; sorry. My point was that
Customize should not write (load-file custom-file) to the init file unless
the latter is empty. The reason I gave is that the user might load it from
another file (which is loaded ultimately from the init file).

In any case, I think (?) we agree now that Customize need never write
(load-file custom-file) to the init file.

I guess maybe you're saying that the custom file should always be loaded
automatically, after the init file, but I can't tell.

Again, I personally have no problem with it being loaded automatically at
the end, if it has not already been loaded.

To prevent it being loaded more than once, the custom file could have
(provide 'custom-file), and the startup sequence could do an implicit
(require 'custom-file custom-file) at the end. That way, if it was already
loaded (e.g. if a user put (load-file custom-file) at the beginning of
.emacs) then it would not be reloaded.

>  > > (that's all we actually provide in a released version), and that
>  > > "before" is satisfactory for the rest.  (There may be users
>  > > we haven't encountered who *need* finer control, but the fact
>  > > that in the 7 years since December 2000 we haven't encountered
>  > > *one* suggests there are very few.)
>  >
>  > That sounds about right to me. But there is no reason not to
>  > give users the flexibility to control when `custom-file' is
>  > loaded, is there?
>
> *shrug* It's theirs for the taking.  The same after-init-hook hack
> that currently allows the "before" users to put (load custom-file) at
> the very top of their user-init-file allows them to put it anywhere
> else they want, too.

Then it sounds like we agree. You didn't mention such a hook hack before or
say that the "before" treatment was in fact an "anywhere" treatment. You
mentioned "the `before' option", so I thought the XEmacs startup did
something specifically limited to loading custom file "before" init file.

> ;;; load custom.el here, not after the init file
> (load custom-file)
> (setq custom-file-saved custom-file
>       custom-file nil)
> (add-hook 'after-init-hook (lambda () (setq custom-file
> custom-file-saved)))

I guess you're saying that a XEmacs user does that to prevent multiple loads
of the custom file (instead of using provide/require or some other way) - is
that right? I have no real problem with that way or another way.

> Of course the hacker who writes the support for automatic "after"
> loading needs to ensure that after-init-hook is called *after*
> custom-file would be loaded.

I don't see a problem with using `require' and putting a `provide' in the
custom file. But whatever works.

> people who insist on "after" would rename their
> custom-file---I used "~/.xemacs/after-init.el" when I was
> experimenting---and put something like
> (setq custom-file (expand-file-name "after-init.el" "~/.xemacs"))

Sorry, but I'm really having trouble following you. That just sets
`custom-file'. How does that change the before/after behavior? And which do
you mean here by "insist on `after'"? Do you mean load custom file after
init file or load init file after custom file?

I guess you mean the former, but I can't tell. I thought that the "after"
case was loading custom file after init, and that that was the default
behavior ("99%") - so there would be nothing to "insist on". I thought that
you didn't need to do anything to have the custom file loaded after the init
file, and you needed to explicitly load the custom file in your init file to
get it loaded before the (rest of the) init file.

> in their user-init-file.  This is preferable to automatic "after"
> loading primarily because using after-init-hook looks more fragile,
> and this has more robust failure modes (the custom.el-was-loaded flag
> is managed by Emacs transparently to the user, so custom.el won't be
> loaded twice unless the user does it explicitly).
>
> "Option overload" is not a good thing.  Users who want to take the
> flexibility can do so using standard functionality.  Other users just
> don't want to know, and I don't blame them. ;-)

I think we agree, but I'm not sure.

I think we're both saying that the custom file would be loaded after the
init file automatically, unless it had already been loaded.

I think you said that XEmacs users could, if they want, have only a custom
file and not an init file, which is something I hadn't considered (it's not
possible in GNU Emacs AFAIK). I assume that that means a custom file at the
default location (or perhaps there is a command-line option also?).

I don't have a problem with that. And I agree that it would be desirable for
a user who doesn't care or want to edit Lisp to be able to simply do nothing
and use only Customize.

Is this something you would agree with:

- Customize saves to the location specified by `custom-file' or to a default
location if that option is nil.

- Customize never writes to the init file. The custom file and the init file
cannot be the same file. If the value of `custom-file' is .emacs (or the
value of the init file from the command line), then it is ignored (and a
warning is shown?).

- A user can prevent loading an existing custom file by either setting
`custom-file' to a non-nil symbol (your suggestion) or by using a
command-line option.

- At startup, .emacs is loaded, if it exists.

- After loading .emacs (or instead of loading it, if it does not exist), the
custom file is loaded, unless it has already been loaded. If the value of
`custom-file' names an existing file (other than the init file), then that
is loaded; otherwise, if the default custom file exists, then it is loaded.
If the user has no init file, then the custom file must be at the default
location.

If you disagree but are otherwise in favor of getting rid of Customize stuff
in the init file, please make a concrete proposal. You have valuable
experience in this area; please contribute a proposal describing the end
result and how to get there.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

Drew Adams writes:

 > > Inelegant, and confusing to the user.
 > 
 > How do you do it in XEmacs (after the migration)? I think you said that
 > users can have a custom file without an init file. Where do they let Emacs
 > know what file is their custom file? If they don't set `custom-file' in the
 > init file, where is it set and read at startup?

(defvar user-init-directory (expand-filename "~/.xemacs"))
(defvar user-init-file-basename "init.el")
(defvar custom-file-basename "custom.el")

or similar happen very early in the startup process, even before
options processing.  There's a bunch of compatibility cruft which
checks for ~/.emacs and so on, but the basic idea is that there are
standard places for these.

There are options --user-init-directory (so I can do
--user-init-directory=~drew/.xemacs and get your whole setup) and
--user-init-file (for those with multiple personalities).  Since our
current custom.el policy is load-after-init-file, we don't have a
command option to set that; you have to use a setq on custom-file in
the init file.

 > - a user who starts from scratch and doesn't know or care about Lisp
 > - users who want to specify the location of their custom file
 > - users who want to load the custom file at different times
 >   during startup, via an explicit (load-file custom-file)

All of these are supported by the XEmacs scheme, although the support
for the third could be better.

 > Huh? I don't have anything against customizations. What lengths do you think
 > I'm going to? I don't understand your point here. What is the problem you
 > see with what I suggested? I think it's the same thing Lennart suggested,
 > which you said "certainly works":
 > 
 > >>    1) load .emacs
 > >>    2) if .emacs did not load custom-file then do that
 > >>       immediately after .emacs.

The difference is that you keep focusing on the case where the user
wants to do something complicated, and optimizing for it.  The
presumption throughout has been "Emacs should not load customizations
unless the user says to do so."  This doesn't make a lot of sense IMO.

The strategy that has been implemented in XEmacs is

    1) if there is an init file, load it.
    2) if there is a customizations file, load it.

If you don't want XEmacs to do that for you (and that is quite
reasonable), simply hide those files by giving them names that XEmacs
doesn't know about.  The only tricky part is getting `custom-file' set
up so that Customize knows where to save its state.

However, for the customizations file, your suggestion of using `require'
to load it makes a lot of sense; to suppress automatic loading, just do
`(provide 'custom-file)'.  This can be done explicitly to suppress the
customizations entirely, or implicitly in the customizations file as
you suggest.

 > I guess that is some proposed standard startup code (and not user code). I
 > think you're saying that a user who for some reason doesn't want to load an
 > existing custom file would set `custom-file' to a non-nil symbol, say
 > `foobar', to avoid loading it. Is that right?

Yes.

 > I would have said that s?he would just set `custom-file' to nil: if the
 > value doesn't name an existing file, then no custom file gets loaded (in GNU
 > Emacs, but perhaps not in XEmacs).

AIUI, that is not backward compatible: if custom-file is nil,
Customizations currently get saved to the user's init file.  If that's
not true, then setting it to nil is fine.

 > In any case, I think (?) we agree now that Customize need never write
 > (load-file custom-file) to the init file.

It should never do that (except on explicit user request).

 > I guess maybe you're saying that the custom file should always be loaded
 > automatically, after the init file, but I can't tell.

Yes.  As I say, there are use cases for loading it before the init
file, but that can be handled with an explicit load.

 > Again, I personally have no problem with it being loaded automatically at
 > the end, if it has not already been loaded.
 > 
 > To prevent it being loaded more than once, the custom file could have
 > (provide 'custom-file), and the startup sequence could do an implicit
 > (require 'custom-file custom-file) at the end. That way, if it was already
 > loaded (e.g. if a user put (load-file custom-file) at the beginning of
 > .emacs) then it would not be reloaded.

That would work.

 > Sorry, but I'm really having trouble following you. That just sets
 > `custom-file'. How does that change the before/after behavior? And which do
 > you mean here by "insist on `after'"? Do you mean load custom file after
 > init file or load init file after custom file?

This is in the context of the code which will try to load custom-file
before the init file, and if not loaded by then, try to load it
afterward.  The idea is that the "before" customization file must have
the default name, or Emacs can't find it.  Then you tell Emacs what
its name is by setq'ing custom-file, and it finds it at the after
stage.

 > Is this something you would agree with:
 > 
 > - Customize saves to the location specified by `custom-file' or to a default
 > location if that option is nil.

Sort of.  `custom-file' should be initialized to the default.  The
user can still explicitly set it to nil, and behavior should be
backward compatible.

 > - Customize never writes to the init file. The custom file and the
 > init file cannot be the same file. If the value of `custom-file' is
 > .emacs (or the value of the init file from the command line), then
 > it is ignored (and a warning is shown?).

No.  Having the init file be the customizations file is a very bad
idea IMO, but code that tries to enforce it is a waste of space.
"ln -s .emacs .emacs.customizations", for starters.

 > - A user can prevent loading an existing custom file by either setting
 > `custom-file' to a non-nil symbol (your suggestion) or by using a
 > command-line option.

Yes.  (We've never had a request for such a command-line option.)

 > - At startup, .emacs is loaded, if it exists.

Yes.

 > - After loading .emacs (or instead of loading it, if it does not exist), the
 > custom file is loaded, unless it has already been loaded. If the value of
 > `custom-file' names an existing file (other than the init file), then that
 > is loaded; otherwise, if the default custom file exists, then it is loaded.
 > If the user has no init file, then the custom file must be at the default
 > location.

Yes, except that as above I would not bother with the check for the
init file being different from the customizations file.  Also, I would
initialize `custom-file' to the default, making the "otherwise" clause
unnecessary.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

>  > How do you do it in XEmacs (after the migration)? I think you said
>  > that users can have a custom file without an init file. Where do 
>  > they let Emacs know what file is their custom file? If they don't
>  > set `custom-file' in the init file, where is it set and read at
>  > startup?
> 
> (defvar user-init-directory (expand-filename "~/.xemacs"))
> (defvar user-init-file-basename "init.el")
> (defvar custom-file-basename "custom.el")
> 
> or similar happen very early in the startup process, even before
> options processing.  There's a bunch of compatibility cruft which
> checks for ~/.emacs and so on, but the basic idea is that there are
> standard places for these.
> 
> There are options --user-init-directory (so I can do
> --user-init-directory=~drew/.xemacs and get your whole setup) and
> --user-init-file (for those with multiple personalities).  Since our
> current custom.el policy is load-after-init-file, we don't have a
> command option to set that; you have to use a setq on custom-file in
> the init file.

OK, so to have a different custom file from `custom.el' (which is in
the same directory as the init file), the user can, in the init file,
set `custom-file' to something else - any file, anywhere. Correct?

>  > - a user who starts from scratch and doesn't know or care about Lisp
>  > - users who want to specify the location of their custom file
>  > - users who want to load the custom file at different times
>  >   during startup, via an explicit (load-file custom-file)
> 
> All of these are supported by the XEmacs scheme, although the support
> for the third could be better.
>
>  > Huh? I don't have anything against customizations. What 
>  > lengths do you think I'm going to? I don't understand your
>  > point here. What is the problem you see with what I
>  > suggested? I think it's the same thing Lennart suggested,
>  > which you said "certainly works":
>  > 
>  > >>    1) load .emacs
>  > >>    2) if .emacs did not load custom-file then do that
>  > >>       immediately after .emacs.
> 
> The difference is that you keep focusing on the case where the user
> wants to do something complicated, and optimizing for it.  The
> presumption throughout has been "Emacs should not load customizations
> unless the user says to do so."  This doesn't make a lot of sense IMO.

No, you're the one who brought up not loading the custom file, not
I.  I agreed with you that that is a corner case.

I used almost the same words as Lennart. I did not use the words you
(mis)quote at all. I said:

  "we could adopt the convention that if `custom-file' was not
   loaded by the init file (or by any files it loaded), but
   variable `custom-file' is defined at the end of this loading,
   then `custom-file' would be loaded at that time."

It's true that initially my idea was not to load custom-file
automatically but only when the user explicitly loads it. But I then
agreed that it could be loaded automatically after the emacs
file. Provided it has not already been loaded.

It's that last proviso that I insisted on, to prevent reloading.
That's all.  I never mentioned the case that you brought up and that
"you keep focusing on", which is a user who exceptionally wants to
prevent loading custom-file for some reason.

> The strategy that has been implemented in XEmacs is
> 
>     1) if there is an init file, load it.
>     2) if there is a customizations file, load it.

That sounds good to me.

> If you don't want XEmacs to do that for you (and that is quite
> reasonable), simply hide those files by giving them names that XEmacs
> doesn't know about.  The only tricky part is getting `custom-file' set
> up so that Customize knows where to save its state.

As long as a user who wants to can set `custom-file' to any file name
in the init file, I think #1 and #2 should be sufficient.

> However, for the customizations file, your suggestion of using `require'
> to load it makes a lot of sense; to suppress automatic loading, just do
> `(provide 'custom-file)'.  This can be done explicitly to suppress the
> customizations entirely, or implicitly in the customizations file as
> you suggest.
> 
>  > I would have said that s?he would just set `custom-file' to nil:
>  > if the value doesn't name an existing file, then no custom file
>  > gets loaded (in GNU Emacs, but perhaps not in XEmacs).
> 
> AIUI, that is not backward compatible: if custom-file is nil,
> Customizations currently get saved to the user's init file.  If that's
> not true, then setting it to nil is fine.

Right, it's not the same behavior as now. That was one of the first
things I said: nil would mean there is no custom file - nothing to
load and no place to save customizations. (You could define
`custom-file' to non-nil during a session, to be able to save.)

But the XEmacs approach for this sounds OK to me. What happens in
XEmacs for saving customizations when the `custom-file' value is a
symbol (so no custom file was loaded)? Are settings nevertheless saved
to the default custom file?

If so, I think (until I understand better ;-)) that it would be better
to prevent saving customizations until the user redefines
`custom-file' to a file name (possibly the default file).

Again, this is about the unusual case where some user wants to prevent
loading an existing custom file for some reason. I think it would be
bad to go ahead and save to the default custom file - better to let
the user know that s?he needs to specify the custom file. Otherwise,
if the user's custom file is in a different location and s?he decides
not to load it, and then forgets that, then settings get saved to a
different custom file (the default), so s?he now has two custom files.
 
>  > In any case, I think (?) we agree now that Customize need never write
>  > (load-file custom-file) to the init file.
>
> It should never do that (except on explicit user request).
>
>  > I guess maybe you're saying that the custom file should always 
>  > be loaded automatically, after the init file, but I can't tell.
> 
> Yes.  As I say, there are use cases for loading it before the init
> file, but that can be handled with an explicit load.
> 
>  > Again, I personally have no problem with it being loaded 
>  > automatically at the end, if it has not already been loaded.
>  > 
>  > To prevent it being loaded more than once, the custom file could have
>  > (provide 'custom-file), and the startup sequence could do an implicit
>  > (require 'custom-file custom-file) at the end. That way, if it 
>  > was already loaded (e.g. if a user put (load-file custom-file)
>  > at the beginning of .emacs) then it would not be reloaded.
> 
> That would work.

I think we basically agree on what's needed.

> This is in the context of the code which will try to load custom-file
> before the init file, and if not loaded by then, try to load it
> afterward.  The idea is that the "before" customization file must have
> the default name, or Emacs can't find it.  Then you tell Emacs what
> its name is by setq'ing custom-file, and it finds it at the after
> stage.

Sorry, now I'm confused again. Are there two different custom files at
the same time in this scenario, both a before file and an after file?
I don't follow you at all here.

I was thinking this:

To load the custom file before the (rest of the) init file, you would
put (load-file custom-file) at the beginning of the init file. You
could first set `custom-file' to any file name you wanted.

To load the custom file after the init file, you would do nothing. You
could, if you wanted, set `custom-file' in the init file to any file
name.

>  > Is this something you would agree with:
>  > 
>  > - Customize saves to the location specified by `custom-file' 
>  >   or to a default location if that option is nil.
> 
> Sort of.  `custom-file' should be initialized to the default.  The
> user can still explicitly set it to nil, and behavior should be
> backward compatible.

Initially set to the default, which is a file name, not nil: OK.

If a user sets it to nil, I was saying that should prevent use of a
custom file. But you suggested using a non-nil symbol to do that
instead, which is fine.

You're saying that nil would give the current GNU Emacs behavior: save
to .emacs. Probably a good idea. Any "retro `everything in .emacs'
types" would appreciate that.

There would still be the migration to discuss: how to help users move
from a .emacs that has Customize stuff to using a separate custom
file.

>  > - Customize never writes to the init file. The custom file and the
>  > init file cannot be the same file. If the value of `custom-file' is
>  > .emacs (or the value of the init file from the command line), then
>  > it is ignored (and a warning is shown?).
> 
> No.  Having the init file be the customizations file is a very bad
> idea IMO, but code that tries to enforce it is a waste of space.
> "ln -s .emacs .emacs.customizations", for starters.

The idea is not to try to prevent someone who is bent on saving
Customize stuff to .emacs from doing it. Your nil `custom-file'
suggestion explicitly allows that. The idea is to prevent someone who
doesn't understand well from mistakenly setting `custom-file' to the
init file or vice versa. It's about helping users not mistakenly shoot
themselves in the foot; it's not about preventing them from aiming at
their foot and pulling the trigger.

>  > - A user can prevent loading an existing custom file by either setting
>  > `custom-file' to a non-nil symbol (your suggestion) or by using a
>  > command-line option.
> 
> Yes.  (We've never had a request for such a command-line option.)
> 
>  > - At startup, .emacs is loaded, if it exists.
> 
> Yes.
> 
>  > - After loading .emacs (or instead of loading it, if it does 
>  > not exist), the custom file is loaded, unless it has already
>  > been loaded. If the value of `custom-file' names an existing
>  > file (other than the init file), then that is loaded;
>  > otherwise, if the default custom file exists, then it is loaded.
>  > If the user has no init file, then the custom file must be at 
>  > the default location.
> 
> Yes, except that as above I would not bother with the check for the
> init file being different from the customizations file.  Also, I would
> initialize `custom-file' to the default, making the "otherwise" clause
> unnecessary.

Good. I don't see any important differences between what you and I
have each suggested.

A couple of others have disagreed, without much in the way of argument
against. Stefan doesn't see the need for a change. Bob sometimes wants
to edit Customize stuff in a single file and doesn't want to remember
two file names.

Let's see what others think. If we can agree about what is TRT, then
we can think about how to handle migration. That could range from (1)
just informing people of new behavior to (2) helping them make some
init-file changes.

For those users who have no custom file now, the minimum for #2 could
be to tell them they can set `custom-file' to nil in .emacs if they
really want to keep Customize settings there.

However, we will need to think about the common case where a user will
do nothing, including not read the doc. We don't want a user to end up
with old Customize stuff in .emacs and new Customize stuff in a
custom-file.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Stephen J. Turnbull]

Drew Adams writes:

[XEmacs specific setup omitted]

 > OK, so to have a different custom file from `custom.el' (which is in
 > the same directory as the init file), the user can, in the init file,
 > set `custom-file' to something else - any file, anywhere. Correct?

That is the intent.

 > > The strategy that has been implemented in XEmacs is
 > > 
 > >     1) if there is an init file, load it.
 > >     2) if there is a customizations file, load it.
 > 
 > That sounds good to me.
 > 
 > > If you don't want XEmacs to do that for you (and that is quite
 > > reasonable), simply hide those files by giving them names that XEmacs
 > > doesn't know about.  The only tricky part is getting `custom-file' set
 > > up so that Customize knows where to save its state.
 > 
 > As long as a user who wants to can set `custom-file' to any file name
 > in the init file, I think #1 and #2 should be sufficient.

Plus a mechanism so that the user can control timing if she wants to.
Using `require' to load the custom-file gives that, since an explicit
(provide 'custom-file) will inhibit Emacs, and the user can use load-file.

 > Right, it's not the same behavior as now. That was one of the first
 > things I said: nil would mean there is no custom file - nothing to
 > load and no place to save customizations.

Well, I don't see a good reason for that.  Some people consider
"customizations in .emacs" a feature, and the traditional way to say
that is nil.  Since in the schemes we're discussing now the default
will be non-nil anyway, there's no reason I can see, except to oppress
those folks, to not use a different non-string value to mean "don't
ask, don't tell [the custom-file]".

 > But the XEmacs approach for this sounds OK to me. What happens in
 > XEmacs for saving customizations when the `custom-file' value is a
 > symbol (so no custom file was loaded)? Are settings nevertheless saved
 > to the default custom file?

I don't know; I imagine you get an invalid-state or invalid-argument
error.  The XEmacs design assumes you want to load and save
customizations, and assumes that `custom-file' will be a string;
AFAICS it does not deal gracefully with anything else.  I'm not
advocating the XEmacs design here; I'm just saying it works for us.

 > If so, I think (until I understand better ;-)) that it would be better
 > to prevent saving customizations until the user redefines
 > `custom-file' to a file name (possibly the default file).

Well, an error wil do that.  ;-)

 > if the user's custom file is in a different location and s?he decides
 > not to load it, and then forgets that, then settings get saved to a
 > different custom file (the default), so s?he now has two custom files.

Seems like a YAGNI to me.

> > This is in the context of the code which will try to load custom-file
 > > before the init file, and if not loaded by then, try to load it
 > > afterward.  The idea is that the "before" customization file must have
 > > the default name, or Emacs can't find it.  Then you tell Emacs what
 > > its name is by setq'ing custom-file, and it finds it at the after
 > > stage.
 > 
 > Sorry, now I'm confused again. Are there two different custom files at
 > the same time in this scenario, both a before file and an after file?
 > I don't follow you at all here.

No.  There are use cases for loading before the init file and for
loading after the init file.  This scenario attempts to support both,
without having "(load-file custom-file)" in the init file.  This is a
matter of style, and nobody in Emacs seems to care about it.  So let's
forget about it, and assume that customizations will be loaded after
the init file, unless explicitly loaded or inhibited by the init file.

 > I was thinking this:
 > 
 > To load the custom file before the (rest of the) init file, you would
 > put (load-file custom-file) at the beginning of the init file. You
 > could first set `custom-file' to any file name you wanted.
 > 
 > To load the custom file after the init file, you would do nothing. You
 > could, if you wanted, set `custom-file' in the init file to any file
 > name.

This will work; it's the current state of affairs in released XEmacsen.

 > There would still be the migration to discuss: how to help users move
 > from a .emacs that has Customize stuff to using a separate custom
 > file.

It's not that hard: you find the init file, walk over the top-level
sexps, moving custom-set forms to a new file.  Probably you should
warn but not move any such forms not at top-level.

 > The idea is to prevent someone who doesn't understand well from
 > mistakenly setting `custom-file' to the init file or vice
 > versa. It's about helping users not mistakenly shoot themselves in
 > the foot; it's not about preventing them from aiming at their foot
 > and pulling the trigger.

Naive users are not going to do such things.  It wouldn't occur to
them that it is possible.  Most "modern" applications hide all that
stuff from the user.  Have you ever tried to find your Firefox cache
or bookmarks file?

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

>  > What happens in XEmacs for saving customizations when the
>  > `custom-file' value is a symbol (so no custom file was
>  > loaded)? Are settings nevertheless saved to the default
>  > custom file?
>
> I don't know; I imagine you get an invalid-state or invalid-argument
> error.  The XEmacs design assumes you want to load and save
> customizations, and assumes that `custom-file' will be a string;
> AFAICS it does not deal gracefully with anything else.  I'm not
> advocating the XEmacs design here; I'm just saying it works for us.
>
>  > If so, I think (until I understand better ;-)) that it would be better
>  > to prevent saving customizations until the user redefines
>  > `custom-file' to a file name (possibly the default file).
>
> Well, an error wil do that.  ;-)

Yes, that's what I was suggesting.

>  > if the user's custom file is in a different location and s?he decides
>  > not to load it, and then forgets that, then settings get saved to a
>  > different custom file (the default), so s?he now has two custom files.
>
> Seems like a YAGNI to me.

Worse than a YAGNI. I wasn't promoting it; I was pointing it out as a
potential problem if you can bypass loading your custom file but then saving
saves silently to the default custom file.

If an error is raised when `custom-file' is a non-nil symbol and you try to
save customizations, then there is no problem.

At any rate, the discussion is moot, now that RMS has decided for the status
quo.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Richard Stallman]

    Currently, if a user doesn't have an init file, then s?he can use Customize
    but s?he cannot save Customize customizations.

If that's true, it is a bug.  Can you provide a precise test case?

I think Customize will create the .emacs file if there is none.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

>     Currently, if a user doesn't have an init file, then s?he can
>     use Customize but s?he cannot save Customize customizations.
>
> If that's true, it is a bug.  Can you provide a precise test case?
> I think Customize will create the .emacs file if there is none.

My bad. I was thinking that it had the same behavior as for emacs -Q. With
emacs -Q, you cannot save customizations.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> Great. And I guess custom-file should be run after .emacs, since that
> makes it possible to chose another custom-file.

No, I think `custom-file' should continue to be loaded only upon explicit
request, never automatically. You should be able to put (load-file
custom-file) wherever you like - the order can sometimes be important.

If an old `user-init-file' already has Customizations, then the explicit
(load-file custom-file) should replace them at the same location.

If it happens (would it?) that an old `user-init-file' has Customizations,
but other code is intermingled with them (e.g. custom-set-variables, then
some other, non-customize stuff, then customize-set-faces), then the user
should be informed and have to decide what to do to remedy the situation.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Stefan Monnier]

> If an old `user-init-file' already has Customizations, then the explicit
> (load-file custom-file) should replace them at the same location.

My explicit (load custom-file) is not at top-level.


        Stefan

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> > If an old `user-init-file' already has Customizations, then the explicit
> > (load-file custom-file) should replace them at the same location.
>
> My explicit (load custom-file) is not at top-level.

Does your init file have Customize settings (`custom-set-*')? If not, then
nothing should need to be done to it.

In any case, I agree that migration - getting Customize stuff out of user
init files - might not be 100% automatic, without some user intervention.

Let's first see if we can agree on what the end result should be. Then we
can see how we might get there.

There are degrees of possible agreement. At the lowest level, I hope we can
all agree that the doc should recommend using a separate custom file.

I hope we can agree to more than that - I hope that we can agree to stop
letting Customize save to the init file. And I hope we can agree on a safe
and relatively painless way to get to a situation where users will have
separate custom and init files (if they have both).

Re: always put Customizations in `custom-file', never in `user-init-file'

[Stefan Monnier]

> Let's first see if we can agree on what the end result should be.
> Then we can see how we might get there.

Honestly, I see no indication that the current situation is a problem.

The rare few people who have expressed the desire to have the custom
settings in a seaprate file are people who can very easily do
it themselves.


        Stefan

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> > Let's first see if we can agree on what the end result should be.
> > Then we can see how we might get there.
>
> Honestly, I see no indication that the current situation is a problem.
>
> The rare few people who have expressed the desire to have the custom
> settings in a seaprate file are people who can very easily do
> it themselves.

OK, so you:

1. See no reason to change to preventing a mix of user code and Customize
automatic insertions.

2. Think that keeping customizations separate is not even something to be
recommended. It is something for only a few rare people who in any case are
not newbies.

Clear enough.

Re: always put Customizations in `custom-file', never in `user-init-file'

[David Kastrup]

"Drew Adams" <drew.adams@oracle.com> writes:

>> > Let's first see if we can agree on what the end result should be.
>> > Then we can see how we might get there.
>>
>> Honestly, I see no indication that the current situation is a problem.
>>
>> The rare few people who have expressed the desire to have the custom
>> settings in a seaprate file are people who can very easily do
>> it themselves.
>
> OK, so you:
>
> 1. See no reason to change to preventing a mix of user code and Customize
> automatic insertions.
>
> 2. Think that keeping customizations separate is not even something to be
> recommended. It is something for only a few rare people who in any case are
> not newbies.
>
> Clear enough.

Drew, what problem are you trying to fix?  "I don't like how others
organize their Emacs customizations" is nothing that warrants action.

What problem do you have with the current state of affairs?  What
user-visible advantages do you expect?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: always put Customizations in `custom-file', never in `user-init-file'

[Jason Rumney]

Drew Adams wrote:
> Now that we have a good alternative (`custom-file'), I
> think that should be used exclusively.
>   

The alternative has always been there, in what cases is it not used now?

When custom was first introduced, the default name for this file was
.custom.el, so the decision to put everything in .emacs where the user
can find it was a concious one, probably for good reasons. I certainly
disagree with your proposal to restrict the value of custom file to
actively prevent users from putting customizations in their
user-init-file, and with having Emacs move customizations that are
already there.

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

> > Now that we have a good alternative (`custom-file'), I
> > think that should be used exclusively.
>
> The alternative has always been there, in what cases is it not used now?

If `custom-file' is nil, then Customizations are added to `user-init-file'.
They are not added to a separate custom file.

> When custom was first introduced, the default name for this file was
> .custom.el, so the decision to put everything in .emacs where the user
> can find it was a concious one, probably for good reasons.

I assumed it was a conscious one.

The good reasons at that time (can anyone state them?) might no longer be so
good. They might have included not wanting to disturb existing habits, such
as expecting all customization code to be in .emacs. Or perhaps the
potential problems were not sufficiently considered at that time. Who knows?

And that decision could not benefit from hindsight. Encouraging users to
edit their own customizations in a file that also contains Customize
customizations (which generally should not be edited) is just asking for
trouble.

I am sure (but I have no proof) that it has caused some users trouble. I am
convinced (but you can change my mind) that such trouble can reasonably be
avoided by keeping Customize away from `user-init-file'.

You are welcome to restate any pertinent arguments from that discussion of
so long ago. Without seeing the arguments, however,...

I think there is a problem. It's generally unclean to mix automatically
generated code with user-edited code in the same file. Let's hear arguments
about the problem (really a problem or not?) or the solution (what I
suggested or another).

> I certainly disagree with your proposal to restrict the value
> of custom file to actively prevent users from putting
> customizations in their user-init-file,

What restriction are you referring to? No one suggested preventing users
from putting customizations in `user-init-file'. That's what
`user-init-file' is for, after all!

What I suggested was preventing Customize from putting Customize
customizations (custom-set-*) in `user-init-file'. I explicitly said that:

>>   I think it was probably a mistake ever to have put
>>   Customize customizations into a file intended for editing.

> and [I certainly disagree] with having Emacs move
> customizations that are already there.

And your reason is?  ("I certainly disagree" is not a reason.)

Re: always put Customizations in `custom-file', never in `user-init-file'

[Robert J. Chassell]

    The entire `custom-file' is generally not intended to be edited by
    a user ...

Over the past 30 years or so, one of the great advantages of Emacs has
been its easy and built-in programming.  Whether an expert user or new
user, everyone not only has been able to learn but has been encouraged
to learn.  As far as I am concerned -- especially since I am not an
expert programmer -- that makes Emacs better than other window
managers with editing libraries.

I agree, faces and fonts are confusing.  At least, I am confused by
them.  For me, it is worth making them simple to set.  Thus, regarding
faces, it is easy to see and change, and change again (and perhaps
again and again) by modifying

    (custom-set-faces
     '(bold ((t (:background "DodgerBlue4" :foreground "coral1"))))
    ...

Moreover, it is easier for me to remember the name of one
customization file, a .emacs file, than two.

-- 
    Robert J. Chassell                          GnuPG Key ID: 004B4AC8
    bob@rattlesnake.com                         bob@gnu.org
    http://www.rattlesnake.com                  http://www.teak.cc

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

>     The entire `custom-file' is generally not intended to be edited by
>     a user ...
>
> ... For me, it is worth making [faces] simple to set.  Thus, regarding
> faces, it is easy to see and change, and change again (and perhaps
> again and again) by modifying
>
>     (custom-set-faces
>      '(bold ((t (:background "DodgerBlue4" :foreground "coral1"))))
>     ...
>
> Moreover, it is easier for me to remember the name of one
> customization file, a .emacs file, than two.

Are you saying that you edit the custom-set-faces sexp in your .emacs
yourself?

If so, then presumably you don't need Customize or use Customize to save
anything. In that case, you can keep on editing everything, including
custom-set-faces, in your .emacs. A proposal that Customize not save stuff
to .emacs will not affect you if you don't use Customize.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Robert J. Chassell]

   ... you can keep on editing everything, including custom-set-faces,
   in your .emacs.

Except I don't always edit custom-set-faces in my .emacs.  Sometimes I
use a customize interface.

As I said, I find fonts and faces confusing.  When I first set a face
-- even after putting more than a hundred in my custom-set-faces
expression in my .emacs -- I often use the less efficient interface
that reminds me what to do.  That is a great advantage of that
non-programming interface.  It is especially important since I seldom
make changes often enough to remember all the options.  However, when
I change the face (or many faces all at once), I generally edit my
.emacs file.

   A proposal that Customize not save stuff to .emacs will not affect
   you if you don't use Customize.

... which does not help me because I sometimes use the non-programming
customize interface rather than the programming Customize interface.

-- 
    Robert J. Chassell                          GnuPG Key ID: 004B4AC8
    bob@rattlesnake.com                         bob@gnu.org
    http://www.rattlesnake.com                  http://www.teak.cc

RE: always put Customizations in `custom-file', never in `user-init-file'

[Drew Adams]

>    ... you can keep on editing everything, including custom-set-faces,
>    in your .emacs.
>
> Except I don't always edit custom-set-faces in my .emacs.  Sometimes I
> use a customize interface.

So you sometimes use Customize to save faces, and then you edit those saved
settings in .emacs.

And you said your objections are that

1. "it is easy to see and change, and change again (and perhaps
again and again)" `custom-set-faces' in your .emacs.

2. "it is easier for me to remember the name of one
customization file, a .emacs file, than two."

Clear enough.

Re: always put Customizations in `custom-file', never in `user-init-file'

[Richard Stallman]

I do not want to consider any change in this.
It works ok now; changing it is risky.
My decision is to keep it the way it is.

There are several bugs to be fixed in Emacs 22;
would people please try to fix them?