A fundamental problem with defcustoms that are lists

A fundamental problem with defcustoms that are lists

[Lennart Borgman (gmail)]

If you have a defcustom which uses variable length lists

  (defcustom my-option
    :type '(repeat ...)
    )

then you may want the default values from the list but also add some of
your own.

If you customize such an option and the default values are changed you
will not notice because when your setting is used Emacs never sees the
changes in default.

This mean that any enhancements or bug corrections to the default will
never be used.

Wouldn't it be worthwhile to give some way to add entries at the
begining or end of the list but add those to the default value instead
of just replacing the default value?

RE: A fundamental problem with defcustoms that are lists

[Drew Adams]

> If you have a defcustom which uses variable length lists
>   (defcustom my-option
>     :type '(repeat ...))
> 
> then you may want the default values from the list but also 
> add some of your own.
> 
> If you customize such an option and the default values are changed you
> will not notice because when your setting is used Emacs never sees the
> changes in default.
> 
> This mean that any enhancements or bug corrections to the default will
> never be used.
> 
> Wouldn't it be worthwhile to give some way to add entries at the
> begining or end of the list but add those to the default value instead
> of just replacing the default value?

A clear recipe would have helped, but I think I know what you mean.

I'd say no. 

The default value of a list option is typically not just a starting point to add
to but is a default value to replace. What you raise is the difference between a
default value that is a list and a list that includes a minimum set of values by
default, that is, a list of default values.

In the latter case, a library can do something like this:

(defvar foo-defaults '(a b c) "List of default values")
(defcustom foo-extras () "Your added values"
  :type '(repeat ...))
(setq foo (append foo-defaults foo-extras))

You would then customize only `foo-extras' in your scenario, and any update to
`foo-defaults' would be accommodated automatically. This is the way to express
that a user's changes are limited to additions to a list of default values, not
replacement of that list (as a single default value).

And `foo-defaults' could itself be a defcustom if we want to let users define
the list of default values for `foo'. The difference between the (purposes/uses
of the) two options should be made clear to users.

In short, if you don't want to lose something, start with nil. "When you ain't
got nothin, you got nothin to lose." ;-)

I think what you're asking for is that all list options `foo' automatically
provide a way to customize only the "extras" part. I think such a possibility
should be explicitly provided by the library itself, and only when appropriate.
It is only the library that really knows what "default" means and whether it
makes sense for users to typically include the defaults and only add to them.

The problem you raise is not fundamentally different if a non-list defcustom is
used. Any existing user customizations will not take into account a change in
the default value for a new version of the library. And that default-value
change might well mean that a user would want to change (or remove) an existing
customized value.

I think there is no magic solution to this potential gotcha - it's what default
values are all about. Library changes that are likely to make users want to
rethink their customizations are similar to incompatible changes. Even when not
truly incompatible, users should be made aware of them. This can be pointed out
in the library's doc or even by a runtime or customize-time message. Only the
particular library can know whether a particular option or particular code
change needs to be treated this way.

Re: A fundamental problem with defcustoms that are lists

[Lennart Borgman (gmail)]

Drew Adams wrote:
>> If you have a defcustom which uses variable length lists
>>   (defcustom my-option
>>     :type '(repeat ...))
>>
>> then you may want the default values from the list but also 
>> add some of your own.
>>
>> If you customize such an option and the default values are changed you
>> will not notice because when your setting is used Emacs never sees the
>> changes in default.
>>
>> This mean that any enhancements or bug corrections to the default will
>> never be used.
>>
>> Wouldn't it be worthwhile to give some way to add entries at the
>> begining or end of the list but add those to the default value instead
>> of just replacing the default value?

> The default value of a list option is typically not just a starting point to add
> to but is a default value to replace.

Why do you think so?

There are certainly situation when this is the case, but I can without
doubt point to other where instead the default value is useful and it is
natural to complement it, not replace it.

> What you raise is the difference between a
> default value that is a list and a list that includes a minimum set of values by
> default, that is, a list of default values.
> 
> In the latter case, a library can do something like this:
> 
> (defvar foo-defaults '(a b c) "List of default values")
> (defcustom foo-extras () "Your added values"
>   :type '(repeat ...))
> (setq foo (append foo-defaults foo-extras))

I can see your idea (but it should not be handled the way you propose
above, use :set instead).

But that would not give the same flexibility as what I propose.

Here is my proposal a bit more concretely expressed:

- In the values stored for a list (ie type 'repeat) for a defcustom
allow a value 'custom-insert-default to insert the default value for the
'repeat list.

Is that clear? Of course the UI have to provide functionality for this too.

RE: A fundamental problem with defcustoms that are lists

[Drew Adams]

> >> If you have a defcustom which uses variable length lists
> >>   (defcustom my-option
> >>     :type '(repeat ...))
> >>
> >> then you may want the default values from the list but also 
> >> add some of your own.
> >>
> >> If you customize such an option and the default values are 
> >> changed you will not notice because when your setting is used Emacs 
> >> never sees the changes in default.
> >>
> >> This mean that any enhancements or bug corrections to the 
> >> default will never be used.
> >>
> >> Wouldn't it be worthwhile to give some way to add entries at the
> >> begining or end of the list but add those to the default 
> >> value instead of just replacing the default value?
> 
> > The default value of a list option is typically not just a 
> > starting point to add to but is a default value to replace.
> 
> Why do you think so?

That's what a default value is: something you can replace. The question is
whether your non-nil value serves as the default value of a list (hence, can be
replaced) or is a list of default values to serve as the minimal starting point
of some other list.

Perhaps two different informal senses of the word "default value" are leading to
confusion. If a list always starts out with elements (1 2 3) regardless of your
customizations, you might want to call that starting point its "default value",
but that is not the sense of "default value" that is implemented by defvar or
defcustom. They use "default value" in the sense of the value that is used if
you don't replace it with something else.

Another way to say it is that you want to customize the list in a way that only
adds to what was there, not replace it. That is a different customization
mechanism. There are ways for a given library to let you do that, as I pointed
out.

> There are certainly situation when this is the case, but I can without
> doubt point to other where instead the default value is 
> useful and it is natural to complement it, not replace it.

Those sound like cases where a list of default values is called for, not a
default value that is a list. We can only judge that on a case-by-case basis -
be specific.

The question is what you (a library) want/expect users to replace: the whole
list  or just part of it. If you want some part of it to always be there, then
include that part automatically, outside of the user's customization. In that
case, start the user-replaceable part off with a default value of nil.

But I'm repeating myself.

> > What you raise is the difference between a default value that
> > is a list and a list that includes a minimum set of values by
> > default, that is, a list of default values.
> > 
> > In the latter case, a library can do something like this:
> > 
> > (defvar foo-defaults '(a b c) "List of default values")
> > (defcustom foo-extras () "Your added values"
> >   :type '(repeat ...))
> > (setq foo (append foo-defaults foo-extras))
> 
> I can see your idea (but it should not be handled the way you propose
> above, use :set instead).

Fine. The exact mechanism used is not important here.

I was trying to point out the difference between a list of default values and a
list that serves as a default value, and that it is the library and its use of
the list that decides which it needs (possibly both).

> But that would not give the same flexibility as what I propose.

Be specific.

> Here is my proposal a bit more concretely expressed:
> 
> - In the values stored for a list (ie type 'repeat) for a defcustom
> allow a value 'custom-insert-default to insert the default 
> value for the 'repeat list.

How is that better than having a separate variable (or function) that provides
the list of default values, and having the code splice that in where needed? Is
it that you are trying to give the user greater control over where the
default-values list is spliced in?

I really don't see a useful use case for this. Why don't you start by explaining
your concrete use case or a problematic situation? Either you will convince
people that something "fundamental" is in fact missing, or someone will show you
that you can already do what you want.

> Is that clear? Of course the UI have to provide functionality 
> for this too.

I don't see a need for such a thing, so far. Maybe someone else does. AFAICT,
all that is needed is a single mechanism to provide for a default value. It's up
to library developers to use that appropriately.

But I don't want to discourage you. I was just trying to help, thinking that you
were perhaps a bit confused. Please detail your use case and your proposed
solution. I'll keep quiet and let others help.

Re: A fundamental problem with defcustoms that are lists

[Lennart Borgman (gmail)]

Drew Adams wrote:
> The question is what you (a library) want/expect users to replace: the whole
> list  or just part of it. If you want some part of it to always be there, then
> include that part automatically, outside of the user's customization. In that
> case, start the user-replaceable part off with a default value of nil.

I clearly want both possibilities. (And you are again saying that they
should be different cases. I said before I do not agree with that.)

> I was trying to point out the difference between a list of default values and a
> list that serves as a default value, and that it is the library and its use of
> the list that decides which it needs (possibly both).

Yes, both. That is the normal case in my opinion.

>> But that would not give the same flexibility as what I propose.
> 
> Be specific.

It is about giving the both the possibility to replace the default value
and add things to it.

I took lists as an example. There might be other cases too, but this is
the most important case I believe.

>> Here is my proposal a bit more concretely expressed:
>>
>> - In the values stored for a list (ie type 'repeat) for a defcustom
>> allow a value 'custom-insert-default to insert the default 
>> value for the 'repeat list.
> 
> How is that better than having a separate variable (or function) that provides
> the list of default values, and having the code splice that in where needed? Is
> it that you are trying to give the user greater control over where the
> default-values list is spliced in?

Yes. Plus that it gives the user the freedom to replace all values.

> I really don't see a useful use case for this. Why don't you start by explaining
> your concrete use case or a problematic situation? Either you will convince
> people that something "fundamental" is in fact missing, or someone will show you
> that you can already do what you want.

I can give that later if it is needed.

Re: A fundamental problem with defcustoms that are lists

[Stefan Monnier]

> If you customize such an option and the default values are changed you
> will not notice because when your setting is used Emacs never sees the
> changes in default.

This is a long standing problem, yes.


        Stefan

Re: A fundamental problem with defcustoms that are lists

[martin rudalics]

 > If you customize such an option and the default values are changed you
 > will not notice because when your setting is used Emacs never sees the
 > changes in default.
 >
 > This mean that any enhancements or bug corrections to the default will
 > never be used.

This problem is comparable to what happens when you change a file under
version control and subsequently update from the repository.  You have
to decide whether and how to merge your changes with those from the
repository.  A similar merge mechanism would be needed for customize.

martin

Re: A fundamental problem with defcustoms that are lists

[Lennart Borgman (gmail)]

martin rudalics wrote:
>> If you customize such an option and the default values are changed you
>> will not notice because when your setting is used Emacs never sees the
>> changes in default.
>>
>> This mean that any enhancements or bug corrections to the default will
>> never be used.
> 
> This problem is comparable to what happens when you change a file under
> version control and subsequently update from the repository.  You have
> to decide whether and how to merge your changes with those from the
> repository.  A similar merge mechanism would be needed for customize.


Thanks, that is a nice idea. I think it would be good, but not
necessary. Maybe some intermediate thing, something that reminded of
"merge conflicts" could be used instead?

Re: A fundamental problem with defcustoms that are lists

[Stefan Monnier]

> Thanks, that is a nice idea. I think it would be good, but not
> necessary. Maybe some intermediate thing, something that reminded of
> "merge conflicts" could be used instead?

That's a very good point.  I think Custom should always store not just
the value you set, but also the corresponding original value.
Some variables often change, so we may not want to warn the user for
each and every change to every variable, but most variables's default
value never changes, so it would make sense to warn users when the
default has changed, even if that variable only holds some symbol or
number rather than a list.


        Stefan