bug#63891: 29.0.91; customize-save-variable should not save all variables if a custom file exists

[Mauro Aranda <maurooaranda@gmail.com>]

On 28/10/23 15:22, Drew Adams wrote:
 >> if packages are going to be changing this 2 options
 >> without asking the user about it, why do the packages
 >> need to lie to Custom saying that the user asked for
 >> that?  Why don't just setq, add-to-list or modify it
 >> some other way? At least that way Custom would know
 >> the truth, the setting was changed outside of Customize.
 >>
 >> That's why I don't understand what is the expectation
 >> about Custom here (apart from being less naive when
 >> saving the custom-file).  The code is modifying a user
 >> option and tells Custom that it was upon the user
 >> request, when in fact it hasn't.
 >>
 >> Finally, have you considered the approach of having
 >> the user option plus another variable which packages
 >> should modify when desired? Then the code could merge
 >> the user settings with the package settings.
 >
 > Hi Mauro,
 >
 > My 2 cents about such things -
 >
 > 1. Libraries can usefully modify user options.  They
 > just need to make users aware of this happening, so
 > that taking advantage of this is a user choice.
 >
 > In particular, libraries can define commands whose
 > purpose includes changing option values - sometimes
 > even saving such changes immediately.  To me, this
 > isn't a no-no.  What is a no-no is changing an
 > option value behind a user's back, and a fortiori
 > saving such a change - big NO-NO.

Of course, and we agree.  I sent a message yesterday asking why didn't
the code just used customize-set-variable, which is one of the functions
that Custom gives for libraries when they want to change the value of
some user option for the session, upon user request.  That's why I'm
raising the point that this change is happening behind the user's back.

But it seems to be the way that these particular user options,
connection-local-profile-alist and connection-local-criteria-alist, are
supposed to be used by packages.  IOW, it seems to me that this is not
specific to Tramp.  For example:

emacs -Q
(progn
   (require 'files-x)
   (setq old-connection-local-profile-alist connection-local-profile-alist)
   (require 'eshell)
   (equal old-connection-local-profile-alist 
connection-local-profile-alist))

That evaluates to nil.  This is not happening via some command. Of
course, the libraries might need to change the value to work correctly,
but that's why I'm asking what's the expectation about Custom after the
change, because telling Custom this happened upon user request is not
true.  Another bad consequence is this:

M-x customize-option RET connection-local-profile-alist

State shown is: SAVED and set

But of course, if you visit custom-file, you'll see no such reference to
connection-local-profile-alist.  This should show that this is not the
way custom-set-variables is supposed to be used.

 > Wrt your last paragraph above: That's the approach
 > that Emacs generally takes.  It's OK, though it's a
 > bit of a kludge.  More importantly, users can _want_
 > to modify an option value on the fly, as opposed to
 > modifying its non-option shadow/stand-in variable.

OK, that's something I haven't considered.

 > 2. Emacs should make available features I noted in
 > previous posts, such as a function to consider a
 > change to an option value (by program) to not be a
 > change.  This lets code change a value but not
 > have Customize consider that a change has been
 > made - so the change won't be saved automatically
 > or reported as having occurred.

I wonder what does this mean in terms of the possible states for a user
option.  AFAICT, none of the current possible states (STANDARD, SAVED,
CHANGED, SET) fit in your description.
STANDARD: No, standard value and current value don't match.
SAVED: No, that setting wasn't saved.
CHANGED: No, because you're saying that Custom shouldn't consider the
change.
SET: No, because the code needs to use customize-set-variable, but that
wouldn't be Custom ignoring the change.

 > 3. Emacs should also make available the ability for
 > `defvar' (or a new macro) to use the features of
 > `defcustom'.  In particular, this includes :set and
 > :type, and it includes the ability to persist the
 > value.
 >
 > The difference between this and `defcustom' is that
 > uch variables aren't recognized by Emacs as user
 > options.  Users can't use the Customize UI or
 > `set-variable' with such vars.  `C-h v' provides no
 > `customize' link, etc.

This sounds interesting.

 > I proposed #3 to emacs-devel@gnu.org back in 2009:
 >
 > https://lists.gnu.org/archive/html/emacs-devel/2009-10/msg00668.html
 >
 > I sent a patch for #3 as bug #27348 (closed by Lars,
 > saying that no one would want such a feature):
 >
 > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27348
 >
 > That patch, with a couple trivial tweaks, is still
 > valid against Emacs 29, I believe.
 >
 > It adds keyword `:not-custom-var' to `defcustom'.
 > If its value is non-`nil' then the variable doesn't
 > satisfy `custom-variable-p', which means it's _not
 > available for interactive use_ (completion,
 > `set-variable', `apropos-user-option' output, etc.).
 > IOW, such a variable is for code more than for user
 > configuration.
 >
 > The patch also defines macro `defvarc', which is
 > just `defmacro' with `:not-custom-var' set to `t'.
 >
 > The `defcustom' keywords etc. could have just been
 > added to `defvar' for optional use.  I defined a
 > separate macro just to not interfere with any
 > existing uses of `defvar'.
 >
 > In sum, this uncouples interactive customization
 > from the other features that Customize offers, in
 > particular, type-checking and persistence, and it
 > provides those features for non-option variables.
 >
 > The ability to type-check, provide `:set' and
 > `:initialize' trigger functions, automatically
 > `:require' libraries, add links to doc, associate
 > with one or more `:groups', etc.  These are useful
 > things to be able to do with at least some defvars,
 > not just with defcustoms.
 >
 > Similarly, the ability to persist non-option vars
 > in a user's custom file can be useful.  This alone
 > is a frequent question, to which the answer has
 > been `savehist-additional-variables', `desktop.el',
 > or Bookmark+ variable-list bookmarks.
 >
 > The patch also includes a macro `with-user-vars',
 > which temporarily lets a set of variables be
 > customizable.  That is, it lets you treat a
 > `defvarc' variable as if it were a `defcustom'
 > option.  So if you want, you can use the Customize
 > UI to change a defvarc's value, or define commands
 > that use (e.g. complete) `defvarc' variable names.

Thank you for the links and the explanation.  I'll take the time to read
them.

Just by reading your explanation, I don't think I understand if just by
using something like your description of 'defvarc' would solve the issue
of having to combine user's customizations with a package setting by
using two different variables.