RE: [External] : Re: Default custom file was: Re: Propose to add setup-wizard.el to ELPA

[Drew Adams <drew.adams@oracle.com>]

> > I have a suspicion that what you're wary of, or
> > objecting to, is perhaps really the automatic
> > loading of `custom-file' if it isn't already
> > loaded.  That's the only "complexity" I can
> > imagine you think you see.  (I don't see that
> > just giving `custom-file' a default value adds
> > complexity.)
> 
> Yes, my main concern was certainly related to various suggestions
> regarding automated detection and other proposals which appear to be
> focused on minimising change impact on existing users etc.

Thanks for the clarification.  I too disagree
with, or am wary of, those proposals with an
emphasis on not impacting existing users.

I think the impact on them of what I proposed
is quite minimal: just set `custom-file' to
init file (or even perhaps just to nil, after
its default change to some standard file name.

But I didn't focus on minimizing impact on
existing users.  A main aim is to help new
users.
 
> However, having said that, I'm still not convinced this is a real
> problem. I would agree that if I was implementing this feature now, I
> don't think I would have the custom data written to the init file and
> would probably write it to a custom specific file. However, we are not
> implementing it now and it has been implemented in this way for over 20
> years and I've seen few problem reports. I have seen a few people
> comment that they think it was a bad decision and/or they don't like it,
> but few actual problems.

Thanks for clarifying that too.  Dunno whether
or how much the problem has actually manifested.

But I think the downside is minimal.  Only those
users who _really_ need to use their init file
for Customize are inconvenienced, and only by
a one-time setting of `custom-file' to the init
file (or to nil).

And the upside is big: everyone else (including,
but not limited to novices) gets clean separation,
which we apparently agree is beneficial in itself.

> > If not - if you have a problem also with the
> > aim of getting more users to use `custom-file',
> > so as to separate Custom automatic editing of
> > init files (when it saves) - then what is your
> > critique of that aim?
> 
> I guess my main critique is two-fold. Firstly, it feels like work for
> works sake which is largely being justified by an argument that
> essentially boils down to "it isn't best practice".

What's the "work"?  (1) setting up the default,
and (2) a tiny number of users needing to set
`custom-file' to their init file.  (Or perhaps
a larger but still small number of users who for
some reason (what?) prefer that Customize use
their init file.)

> Such an argument
> might have some weight if there was nothing the user could do, but that
> isn't the case. This is my second critique - for those who don't like
> having their custom variables in their init file, they can easily
> change it.

They need to realize the advantage, to take it.
And, so far at least, Emacs isn't even telling
them about it - not even recommending it.  And
if Emacs were to recommend it then the obvious
question would be that if it's recommended, why
isn't that the default behavior?

> I'm sure some will argue that it isn't easy 
> to change for novices,

It's easy enough - it's just as easy for everyone
to set `custom-file' to a file name as it is for
a tiny number of people to set it to their init
file name. ;-)

It's not that it's difficult for anyone to set
up a `custom-file'.  It's that the awareness of
it isn't there.

> but I find such an argument weak as novices are unlikely
> to even know/care that the custom variable section is
> written to their init file

If they never edit their init file then there's
no problem - then it's essentially a file only
for Customize.  But many (most?) users, even
novices, copy stuff from here and there and
stick it in their init files.  And it goes on
from there.

The virgin state you refer to isn't problematic,
right, but few remain virgin long. ;-)

> and those that do will likely find the standard
> solution pretty quickly.

What's the standard solution?

I suspect that relatively few Emacs users use
`custom-file', and even relatively few are aware
of it.  Based on what do I say that?  Not a lot,
but Q & A here & there.

Again, it would be better if Emacs at least
recommended using `custom-file' - in the manual,
and maybe even in a tutorial that does some minor
customizing (a face or a common option).

If most Emacs users were aware of it, and had an
idea of why they might want to use a separate
file, then I'd maybe agree that "those who do
will likely find the standard solution pretty
quickly", assuming using `custom-file' is the
standard solution you had in mind.

> > (If you agree with that aim, but not with any
> > of the proposals to move toward it, please
> > make that clear too.)
> 
> If the *only* change we are talking about is setting a default value for
> custom-file, I probably wouldn't have any concerns. However, I think it
> is a mis-characterisation to claim that is all that will change. There
> has also been mention of adding new command line switches (like -Q and
> -q) to turn off loading of custom file, questions around when the custom
> file will be loaded and ability of the user to control that timing,
> automated prompting of custom-file name/location etc.

Mea culpa.  I suggested a number of things, yes.
But I also ranked them in priority, and I made
clear that what really matters (to me) is that
we get more/most users to use a separate file
for Customize.  How that happens, and anything
else related that might happen - to me that's
secondary.

Wrt -q and -Q turning off loading of `custom-file':
No, not at all.

What I suggested was having them turn off the
(proposed) _automatic_ loading of it after the
init file.

If the init file isn't loaded (-q and Q), I
think it makes sense not to load `custom-file'
(which by the proposal would be at a default
location, and which might well contain some
customizations).

This was a detail.  I perhaps shouldn't have
covered such details till we get past the main
aim and proposal.)

> > To be clear, the aim is not at all "to make
> > the Emacs startup process 'smarter'", and not
> > at all to provide some kind of "optimization"
> > (premature or otherwise).  And I don't think
> > anything proposed so far does that.
> 
> Well, I guess we will disagree on that point. My response in this thread
> was specifically triggered by proposals relating to auto-detection of
> whether users have set a custom-file name, 

I haven't see that proposal.  How is that even
possible?

> discussions regarding adding additional command
> line switches

That was me - the detail I tried to clarify above.

> and various other suggestions relating
> to contgrol of when the custom-file is loaded (or not).

OK.

> > The problem to try to solve - particularly
> > for new users or users unsure of Emacs Lisp
> > - is to prevent users (accidentally or with
> > good intentions) from messing with generated
> > code, and (less likely) to prevent Customize
> > from messing with user code.  That's all.
> >
> > Is that a purely "theoretical" problem?
> 
> OK, theoretical was probably a bad choice of words.

No, I don't have a problem with that word here.
I really meant to pose the question.  Maybe it
is a solution looking for a problem.  I don't
think so, but maybe it is.

I addressed this above: we may not have evidence
of it having been manifested; I grant you that.

But I've seen plenty of questions about this or
that happening because of things people do to
their init files.

> Perhaps contrived
> problem would be a better description. In 25 years of using Emacs, I
> have never seen Emacs customize messing with user code.

Nor have I.  It's users messing with generated
code that I think can be problematic.  And
maybe that's just hypothetical; dunno.

As I said, it doesn't make sense to mix the
two - nothing is gained by that design (as the
default at least, and probably in general).

> I think that one
> is purely theoretical. With respect to users, especially new users,
> intentionally or unintentionally messing with generated code, I think
> that is just part of the normal learning process.

OK.  They can certainly learn some things that
way.  Whether that's the normal, or a necessary,
way to learn, I doubt.  That sounds like trying
to make a virtue out of necessity (status quo).

> There is a very clear warning

Hm.  An obvious question is that if we need to
warn about that then why is it in the init file?

> and if they choose to disregard it, that is their choice.

Or accident.  People who go through red lights
don't always choose to do so.

> If we are going to go down the protect users
> from themselves road, 

There's no reason to exaggerate.  There's no
baked-in choice of going down some road, just
because we might do some little thing to avoid
dumb mistakes.  We ask you to confirm deletion
of files flagged for deletion in Dired?  That
doesn't imply any obligation to convert Emacs
to a "nanny-state" editor.

> then we should remove the init file completely
> and only allow them to configure
> the system using customize.

That doesn't follow.

> A better question is whether this is a significant enough problem to
> justify the change, change management that will be required and
> additional complexity (even if only small) it will add.

We seemingly have greatly different views of
the change and complexity required.

Even if our little warning were to tell users
about `custom-file' and recommend using it, that
in itself would be an improvement.  Is that a
big change or complex?  And yet it hasn't been
done.

This is not the first time I've suggested this
kind of thing (but it's the first time I've
said so much about it).  And I don't think I'm
the only one who's brought it up before.

[And XEmacs apparently did it, without the world
coming to an end.  (XEmacs itself came to an end,
but not because it defaulted to using a custom
file.)]

> I don't see this
> as a significant issue and personally, prefer having the control to set
> a custom file and load it when I see fit for my own configuration.

Now I have a doubt that you read what I proposed.
I've said several times that you'd lose zero such
control.  Users should and would have 100% control
over whether to use a `custom-file', where it is,
when and where and whether to load it, and so on.

> In a nutshell, the potential negatives seem to outweigh the benefits.

I hear you.  But I think, especially given what
you just wrote, that you might have a mistaken
notion of the negatives.

> If we
> were seeing large numbers of reports from users having errors or
> problems because of the current behaviour, my position would likely be
> different.

That I understand.  I expected someone would
say that at some point, and I have no rebuttal.

On the other hand, I don't see the downside.
To me this is a no-lose no-brainer for Emacs
users - all of them.

> There is also a positive side to having the custom variables stored in
> your init file - one which is probably even more relevant for new users.
> The benefit is that ALL configuration related settings are in the one
> file.

First time that's been pointed out.  And that
too I expected someone would say at some point.
And there too I have no rebuttal.  Except to
say that customization settings are in a world
of their own - Customize is particular.

But yes, there's an argument to be made that a
hand-coded `setq' of option `foobar' being in
the same file as a `custom-set-variables' that
includes `foobar' might make some sense.

On the other hand, ask yourself what happens
if there's a hand-coded `custom-set-variables'
along with a Customize-inserted one.

> The new user does not need to know that in addition to their init
> file, there is this other 'custom' file which will also impact on their
> configuration.

Maybe, maybe not.  Interaction of user code
with Customize code can be...interesting.
Face changes, for instance.

> Right now, if, for example, I was having issues getting
> some configuration relating to 'x' to work, I can search for 'x' in the
> init file and there is a good chance I will find it even if it is being
> set via a custom setting I had forgotten about.

Yes; agreed.  Provided your init file isn't
a monster.  In my case, my init file loads
a boatload of other code files.  And my
`custom-file' is pretty small.

> > As I asked earlier:
> >
> >  The default behavior of Emacs now is to have
> >  a single file that mixes user coding and
> >  Custom coding.  Why is that a great idea?
> >                  ^^^^^^^^^^^^^^^^^^^^^^^^
> 
> I think that is an irrelevant question. Perhaps it wasn't a great idea
> or perhaps it was a great idea at the time it was made. However, that is
> irrelevant. The better question is whether it was a bad enough idea to
> need change and I don't think it is.

I don't think it's irrelevant at all.  I think
it's worth thinking about.  But I agree that
whether to change also involves the question
of the cost vs benefit of the change.

> It is probably also worth noting that the extremely popular VS Code
> editor actually does a very similar thing. That editor is configured via
> JSON and has two interfaces - one which provides a sort of graphical
> interface similar in flavor to customize and one where the user
> writes/edits JSON directly. It is all stored in the same file (though
> you can also have additional config files). The Emacs approach is IMO a
> better solution because the two are clearly separated while in VS Code,
> the user can screw up the manual code which will also screw up the GUI
> interface to the config.

OK.

> > No one has answered that.  Forget, for a
> > second (and no, I'm not saying this isn't
> > important), that there is habit & legacy.
> >
> > Just imagine that you are designing from
> > scratch.  Would you really want Customize
> > (or any other code generation) to write
> > to a file that users code also?
> 
> Well, if I was designing from scratch, I probably wouldn't have
> customize in its current form either.

So much for VS Code's brilliance. ;-)

> But if we are going down that
> road, I also would do font locking differently, use different key
> bindings, use different terminology for many Emacs features, have real
> namespaces, etc, etc, ...

Please, let's not throw in the kitchen sink.
There's no committal to any road.  There's
no marriage or contract.

> > We don't do that for any other generated
> > Elisp code.  We write all such code to
> > dedicated, separate files - not files
> > that users code in.  (We don't _prevent_
> > users from editing such files, of course.)
> >
> > Why do we do that?  Premature optimization?
> > Paranoid theoretical problem-worrying?
> >
> > ___
> >
> > In order of decreasing priority (to me):
> >
> > 1. Let's agree on the problem, potential at
> >    least.
> 
> Sorry, but no I cannot. I would characterise it as a contrived rather
> than potential problem. This solution has been there for a long time -
> if the problem had real potential, it would have been realised in
> significant numbers by now.

> > On the road from 1 to 4, how far do you get?
> 
> I guess zero :-)

Right.  I should have said from 0 to 4.

> > Personally, I get as far as #4.  I proposed
> > some concrete changes as solution, but #4 is
> > an open question.
> >
> > Possible solutions include:
> >
> > a. At a minimum (I think) explicitly encourage
> >    users - particularly new users - to define
> >    `custom-file', and load it to get their
> >    saved settings.
> 
> This seems to be in conflict with one of the justifications underpinning
> this argument i.e. dealing with elisp code is hard for new users.

There's no such hard-and-fast assumption
underpinning anything I wrote.  In fact, I
_don't_ think dealing with Elisp code is hard
for new users.

I think separating hand-coded code from generated
code makes sense, by default, for old as well as
new users.  I use a separate `custom-file' myself.

> > b. What I proposed, which I won't repeat but I
> >    can summarize as `custom-file'-default-value.
> >
> > Please note that even poor (a) has never been
> > done: we don't encourage use of `custom-file'.
> >
> > I don't think doing nothing is good (see 1-3).
> > And I don't think that (a) alone (recommending
> > use of `custom-file') is sufficient.
> 
> I guess that is the big stumbling point. I'm still unconvinced that we
> need to do anything or that it is potentially as simple as just setting
> a default value for custom file.
> 
> This isn't an issue I will die in the gutter over. If sufficient numbers
> agree the change is needed, then it will happen.

My expectation is that it won't happen.  Eli's
said about as much already.  I've suggested it
before, to no avail - but not recently.

> My only real request is
> that whatever change is put in place, ensure that I can still set the
> name and location of my custom settings file and have full control over
> when in the init process it is loaded. This includes setting and loading
> the custom file in files outside of the main user init file (i.e. in
> files loaded by my init file).

I've said it several times now, and I'll say it
again: _absolutely_.  Nothing changes in that
regard.  (I myself use a separate `custom-file',
and I load it twice in my init file.)

Thank you for taking the time to express your
point of view clearly and in detail.