Changed outside --> set, in Customize UI

["Drew Adams" <drew.adams@oracle.com>]

A lot has been written on the question of eliminating the distinction,
in the Customize UI, between "set" and "changed outside".

To summarize a bit:

I argued for removing this distinction.

Per and others said that keeping a distinction can be useful for
debugging. I replied that if so, then an internal distinction should
be enough for that purpose; we should still eliminate the distinction
in the UI.

Stefan and others pointed out that if a saved user option is changed
afterward by Lisp code (including code in .emacs), this can be
confusing.

Along the same lines, Luc and others pointed out non-trivial problems
with libraries that modify user-option lists such as hooks. One of
these problems, for example, is that libraries sometimes make changes
that essentially implement `standard' values, but are not recognized
by Customize as doing that. 

A user cannot then reset subsequent changes, to get back to the
`standard' state the library provided. And there is no easy way for a
user to add hook functions to pre-supplied (=`standard') functions in
a hook and later be able to take advantage of bug fixes that might
change what the `standard' component is.  Instead of including
explicit standard functions in the hook, the intention is really to
have the equivalent of ,@(current-std-hook-fns) in the hook.

Some of the problems raised are serious, and their investigation and
possible ways to fix them have opened an interesting discussion.
However: 1) all of the problems raised exist already today, and 2)
they are unrelated to the UI proposal.

Some people feel that the "changed outside" indication in the UI
serves a purpose by in some way warning users of these potential
problems. I maintain that it does _not_ act as such a warning and does
_not_ help in any way with these problems.

One problem mentioned is that trying to save a "changed-outside"
option might not "work": some library you load might change the saved
value to something different after you restart. Well, _setting_ an
option in Customize and then saving it might not work either - for
exactly the same reason. If users think they can rely on _not_ seeing
a changed-outside "warning", then they are misled. If you see a
changed-outside warning, beware; if you don't see such a warning,
beware. This is a distinction without a distinction.

There can also be lots of user-option changes made outside Customize
that are _not_ problematic. _Every_ change to a user option, besides a
user clicking Set in Customize, currently shows up with the
changed-outside "warning" - even benign changes. Instead of seeing a
warning in such cases, the user should simply see that the option has
been _set_ - because it has. A library might well set some user
options as an intended effect, the library being used in effect as an
extension to Customize, without any unpleasant side effects
necessarily occurring.

The same motivation for changing `set-variable' (post 21.4) so that
its effect is reflected in Customize as "set" instead of
"changed-outside" (ignoring the :set complication) is my motivation
for treating all other ways to change options the same way: make
things clearer and simpler for the Customize user. Customize should
play well with `set-variable'; it should do likewise with all other
user-option changes.

After all, what is special about `set-variable'? What about your new
command `my-super-face-morpher-and-tweaker', which changes face
parameters?  Why shouldn't it benefit from the same treatment in
Customize as `set-variable'? Your command effectively extends the
Customize UI, letting users change face parameters in a new, powerful,
and easy-to-use way. So, let's add your command to the Customize club.

But, where should we draw the line on admission to the club?
Interactive changes only, perhaps?

    When changing a defcustomed variable from a function, that
    function should be interactively called by the user

Why? And what is "interactive", anyway, in this context - does it mean
only `interactive-p'? A Lisp function or a library can define a
different `standard' value for any user option. Why shouldn't a Lisp
function or a library "set" an option value - or even a bunch of
values? If your _purpose_ in using the program or library is the same
as your purpose in using `set-variable' - to set some values for the
current session, then why should Customize consider such settings as
"rogue" (changed-outside) and throw up its hands?

The idea behind limiting Customize admission to only interactive
functions I think hints at the proper criterion, but it does not hit
it square on. The idea of user Helen "setting" an option really comes
down to this: user _intention_. That is not a criterion we can test by
program, but it is the only reasonable criterion, IMO.

If something changes an option in a way that Helen is not aware of or
doesn't want, then that is _not_ Helen, the user, setting the option,
and that behavior will confuse her. If she uses <WHATEVER means> to
set an option _on purpose_, then that _is_ Helen setting the
option. Helen does not care, in principle, whether she uses Customize
or `my-super-face-morpher-and-tweaker'; she just wants to change her
settings and then (possibly) save them.

Only Helen can know her intentions. We cannot reasonably separate
user-option changes into changes that are "inside" and changes that
are "outside" her intentions - only Helen can do that. What we can do
is try to limit the number of functions and libraries that change user
options and make their behavior clear and obvious.

Claiming that any change to a user option should be reflected as "set"
in Customize does not of course mean that any arbitrary change is a
good change. It is neither necessarily bad nor necessarily good (nor
even benign) for Lisp code to modify a user option. It is bad for Lisp
code to diddle Helen's options without her knowledge, and it is very
bad for Lisp code to change her options in a way that prevents her
from returning to saved or standard values.

A user option is a user option, and it should be totally under _user_
control. Proper functioning of a user hook, for instance, should not
depend on any properties of the hook value, such as additional hook
functions, that the user does not control or is not aware of. If that
means that we have to split lots of user hooks in two (user; system)
or otherwise bend over backward to avoid stepping on user settings,
then so be it - we'll just have to bite the bullet.

A user option must be under user control. However, as a general rule,
user control _includes_ the ability to use functions and libraries,
and that includes using them to set options.

What about the use of the "changed outside" flag for those outside
changes that do represent real problems? Even there it is
inappropriate, for "changed outside" is insufficient to let anyone
know what the problem might be or what to do about it. Combining the
benign and helpful with the seriously problematic, "changed outside"
really means nothing useful to users as a "warning".

We can try to distinguish between 1) outside changes that can be seen
to represent bugs and lead to problematic behavior and 2) other
outside changes. We can then try to fix the problems. 

Until fixes are implemented, we could, as Luc suggested, display a
proper warning for the cases we can identify as problematic. We should
not, however, paint all outside changes with the same broad brush.
Bugs are bugs. It does not help to treat each outside change as a
possible bug by flagging it with a warning - quite the contrary, if we
want people to pay attention to such warnings.

Let's separate 1) the question of the _desirability of changing the
UI_ ("changed outside" -> "set") from 2) the how-to questions of
_problems that need fixing_.

It would be useful to separate these two issues even if the problems
raised did not exist already and were not, therefore, unrelated to the
proposed UI change. The distinction here is user interface vs
implementation: Is "changed outside" really helpful in the UI? It is
all the more useful to separate these two discussions, since the
problems do exist independently of the UI question.

If we nevertheless suppose, for argument's sake, that "changed
outside" does in some way help users avoid some of the problems
raised, then a weaker form of the UI-change question can be
considered: _If_ problems like those discussed can be solved, is it
then a good idea to use only "set" for today's "set" plus today's
"changed outside"?

As to the desirability of the UI change (replacing "changed outside"
by "set"):

      > It is clearer for users without such a distinction.
    
    I rely heavily on the distinction. I cannot see what is "clear"
    about identifying two things that have not the slightest thing to
    do with each other.

Changing user options inside customize and changing them outside have
this to do with each other: they are both about _changing user
options_ (which is what users think customizing is all about). If a
distinction doesn't serve, then things are clearer without it.

It would be good to hear arguments as to why the UI distinction should
be kept. So far, all of the arguments have involved reference to
existing problems that are independent of the proposed UI change.

But even if there are no good arguments not to change, some people
might just prefer the UI as it is. At a minimum, this preference
could, well, itself be treated as a user preference, that is, a user
option. That might satisfy everyone. After all, if we maintain the
distinction internally anyway, then it should be trivial to optionally
maintain it or remove it in the UI. Of course, we might then argue
about which presentation should be the default one, but at least both
preferences would find a home.

At a minimum, then, I would argue for a user option to let users not
be bothered with the difference between changes inside and outside
Customize. I think that will be clearer for users without this
distinction; others think not. Why don't we let users choose
individually?