:file keyword for Customize

:file keyword for Customize

[Drew Adams]

How about adding a :file keyword for `defcustom' and `defface'?

(defcustom foo-bar-flag t
  "Non-nil means toto."
  :type 'boolean :group 'foo
  :file (if (eq system-type 'windows-nt)
            "C:\\foo-stuff\\foo-cus.el"
          "/toto/foo-stuff/foo-cus.el"))

Its argument would be evaluated to a file name. That particular `defcustom' or
`defface' setting would be saved in that file, instead of in `custom-file' or
`init-file'. A nil or "" value for :file would mean the same thing as the
absence of a :file keyword: as now, use `custom-file' or, if no `custom-file',
`init-file'.

This could provide a little more modularity for packages (and for any other
groupings of options & faces). It could help deal with things like platform
differences (for a package) and initialization order of settings. It could allow
users a little more flexibility in terms of when some groups of Customize
settings are loaded (currently, all Customize settings are loaded at once). It
could simplify communication with package authors about bugs (e.g. automatically
include the package's Customize settings in a bug report).

I don't have a particular use-case in mind; it's just something that occurred to
me. There is nothing special in this, but I think it might help organize things
a bit. A user's `custom-file' or `init-file' can become a monolithic blob, and
this could help cut down on that.

Re: :file keyword for Customize

[Jason Rumney]

Drew Adams wrote:
> How about adding a :file keyword for `defcustom' and `defface'?
>   
What do existing packages like Gnus and JDEE do to save their options in 
a separate file? If what they do is hacky, then that would support such 
a need, but if they can do it cleanly already, then it might be suitable 
for your needs too.

RE: :file keyword for Customize

[Drew Adams]

> > How about adding a :file keyword for `defcustom' and `defface'?
>   
> What do existing packages like Gnus and JDEE do to save their 
> options in a separate file?

No idea. I don't have or use either.

> If what they do is hacky, then that would support such 
> a need, but if they can do it cleanly already, then it might 
> be suitable for your needs too.

To be clear, I don't have any particular needs in this regard. I just thought
this might make sense generally.

The granularity for Customize customizations, in terms of persistence and
loading is currently all or none: everything is in one file (`custom-file' or
`init-file'). It just occurred to me that it might be better to at least let
Emacs-Lisp programmers group customizations into smaller persistence/load units,
if they see fit.

Re: :file keyword for Customize

[Jason Rumney]

Drew Adams wrote:
> The granularity for Customize customizations, in terms of persistence and
> loading is currently all or none: everything is in one file (`custom-file' or
> `init-file').
JDEE at least, saves its own customizations in project files in the 
local directory. I think Gnus also lets you save your Gnus related 
customizations in .gnus. That is why I suggested looking at what those 
packages do.

>  It just occurred to me that it might be better to at least let
> Emacs-Lisp programmers group customizations into smaller persistence/load units,
> if they see fit.
>   

It might be better to do it at the group level than each individual 
customization then, if grouping is the desired outcome.

Re: :file keyword for Customize

[Ted Zlatanov]

On Thu, 08 May 2008 18:08:36 +0100 Jason Rumney <jasonr@gnu.org> wrote: 

JR> Drew Adams wrote:
>> The granularity for Customize customizations, in terms of persistence and
>> loading is currently all or none: everything is in one file (`custom-file' or
>> `init-file').
JR> JDEE at least, saves its own customizations in project files in the
JR> local directory. I think Gnus also lets you save your Gnus related
JR> customizations in .gnus. That is why I suggested looking at what those
JR> packages do.

Gnus has two kinds of settings: normal ELisp variables, customized and
used as usual, and newsrc settings, which go into .newsrc.eld.  There
are a few others, like the Gnus registry and score files, but generally
the two groups above are the major ones.  There's some magic with the
group/topic parameters that I can explain but it wouldn't matter much
for what Drew proposed.

.gnus.el is just ELisp code, nothing special.  Gnus doesn't set anything
special for the ELisp variables it defines, so if they are customized,
they will be saved in the usual places.

FWIW, I would make custom-file accept an alist of defgroup/defcustom
symbol name regex, associated with a file name or a function.  That puts
the power in the user's hands, yet allows package authors to add to the
alist.  The big win is that no special :file keyword is needed, and all
existing packages will work with or without this special settings.

Ted

RE: :file keyword for Customize

[Drew Adams]

> <jasonr@gnu.org> wrote: 
> JR> Drew Adams wrote:
> >> The granularity for Customize customizations, in terms of 
> >> persistence and loading is currently all or none: everything
> >> is in one file (`custom-file' or `init-file').
> >>
> JR> JDEE at least, saves its own customizations in project 
> JR> files in the local directory. I think Gnus also lets you
> JR> save your Gnus related customizations in .gnus. That is
> JR> why I suggested looking at what those packages do.
> 
> Gnus has two kinds of settings: normal ELisp variables, customized and
> used as usual, and newsrc settings, which go into .newsrc.eld.  There
> are a few others, like the Gnus registry and score files, but 
> generally the two groups above are the major ones.  There's some magic
> with the group/topic parameters that I can explain but it wouldn't
> matter much for what Drew proposed.
> 
> .gnus.el is just ELisp code, nothing special.  Gnus doesn't 
> set anything special for the ELisp variables it defines, so if they
> are customized, they will be saved in the usual places.
> 
> FWIW, I would make custom-file accept an alist of defgroup/defcustom
> symbol name regex, associated with a file name or a function. 

I don't think that's the place to specify such organization, and I don't think a
regexp offers the right kind of flexibility.

> That puts the power in the user's hands, yet allows package
> authors to add to the alist.  The big win is that no special
> :file keyword is needed, and all existing packages will work
> with or without this special settings.

I don't see any such big win. It's no big deal to add a :file keyword. And all
existing packages would continue to work with no changes - :file would be
optional, of course.

This is about letting an individual option or face (or perhaps group) decide
where it is stored (and consequently let users decide when it is loaded). The
place to do that is in `defcustom' and `defface' (and perhaps `defgroup').

Re: :file keyword for Customize

[Ted Zlatanov]

On Thu, 8 May 2008 11:00:40 -0700 "Drew Adams" <drew.adams@oracle.com> wrote: 

>Ted Z wrote:
>> FWIW, I would make custom-file accept an alist of defgroup/defcustom
>> symbol name regex, associated with a file name or a function. 

DA> I don't think that's the place to specify such organization, 

We disagree then.

DA> and I don't think a regexp offers the right kind of flexibility.

That may be, the selection mechanism is not important.  It can be a cond
form.  The point is to put the choice in the user's hands.

>> That puts the power in the user's hands, yet allows package authors
>> to add to the alist.  The big win is that no special :file keyword is
>> needed, and all existing packages will work with or without this
>> special settings.

DA> I don't see any such big win. It's no big deal to add a :file
DA> keyword. And all existing packages would continue to work with no
DA> changes - :file would be optional, of course.

All existing packages can work with the new mechanism I proposed without
modifications, whereas your proposal requires special *new* code
everywhere a different custom file is desired, and leaves the choice up
to whoever writes the source code.  Thus the win is that less code is
written all around, users have more choice, and everything works as it
did before.

DA> This is about letting an individual option or face (or perhaps
DA> group) decide where it is stored (and consequently let users decide
DA> when it is loaded). The place to do that is in `defcustom' and
DA> `defface' (and perhaps `defgroup').

I think I understand your proposal, and I think it puts the option in
the place where it's hardest to change it if the package author or the
user should need to do it.  It also makes it hard to organize
configurations in a way that the *user* likes.  Perhaps I misunderstand
your proposal.  Will the user be able to change the :file parameter on
their own without modifying code, for example?

What happens if the file name changes with a new package?  Do you add
extra code to handle the possible values of :file?

What if the user wants to save the configuration for two packages in one
custom file, but all other packages in the default place?  This really
is not an unusual need.  I like to keep my Gnus and BBDB variables
separate from the rest of Emacs but in one file, for example.

The default path the package author picks may not be appropriate.  For
example, in Gnus a lot of work had to be done to remove dependencies on
~/News and ~/Mail that were put in place over time.  Many users needed
those paths relocated to somewhere that made sense to them.

Ted

RE: :file keyword for Customize

[Drew Adams]

> >> FWIW, I would make custom-file accept an alist of 
> >> defgroup/defcustom symbol name regex, associated with
> >> a file name or a function. 
> 
> DA> I don't think that's the place to specify such organization, 
> 
> We disagree then.

Nothing wrong with that. ;-)

> DA> and I don't think a regexp offers the right kind of flexibility.
> 
> That may be, the selection mechanism is not important.  It 
> can be a cond form.  The point is to put the choice in the user's hands.

The ultimate choice must be the user's. But the programmer can write code that
decides _how_ users make the choices and _which_ choices they can make. The
programmer knows about code dependencies and other internal needs, and can DTRT
wrt both the code and the user (UI design) - giving the user the choices s?he
needs but guiding how those choices are made.

> >> That puts the power in the user's hands, yet allows package authors
> >> to add to the alist.  The big win is that no special :file 
> >> keyword is needed, and all existing packages will work with or
> >> without this special settings.
> 
> DA> I don't see any such big win. It's no big deal to add a :file
> DA> keyword. And all existing packages would continue to work with no
> DA> changes - :file would be optional, of course.
> 
> All existing packages can work with the new mechanism I 
> proposed without modifications, whereas your proposal requires
> special *new* code everywhere a different custom file is desired,
> and leaves the choice up to whoever writes the source code.
> Thus the win is that less code is written all around, users have
> more choice, and everything works as it did before.
> 
> DA> This is about letting an individual option or face (or perhaps
> DA> group) decide where it is stored (and consequently let 
> DA> users decide when it is loaded). The place to do that is in
> DA> `defcustom' and `defface' (and perhaps `defgroup').
> 
> I think I understand your proposal, and I think it puts the option in
> the place where it's hardest to change it if the package author or the
> user should need to do it.  It also makes it hard to organize
> configurations in a way that the *user* likes.  Perhaps I 
> misunderstand your proposal.  Will the user be able to change the
> :file parameter on their own without modifying code, for example?

What I was thinking was that library writers would typically provide one or more
user options or functions as the :file sexps - it is those sexps that would then
let users, in effect, determine the :file values used. IOW, end users would not
directly determine the sexps used for :file, but they would ultimately determine
the values of those sexps in some way (defined by the programmer). 

There would be nothing in the nature of :file itself that would force
programmers to do that, but they would do it naturally - it would make no sense,
for instance, for a programmer to hard-code an absolute file name as a :file
sexp or it's value. (In that, the example I gave was misleading, but I wanted to
get the general idea across, showing that arbitrary code could be used for
:file.)

The idea would be to let programmers define the choices that an end user has,
just as they do with `defcustom' in general (not every variable is a user
option). For instance, the code used for several :file args could be made to
return the same value in some context if that is appropriate, effectively
ensuring that those options would all be stored in the same file. That's
different from letting users specify the files to use for individual options
arbitrarily.

(Of course, any user can always resort to changing the code, as well, but that's
not your point nor mine.)

> What happens if the file name changes with a new package?  Do you add
> extra code to handle the possible values of :file?

Again, :file would not have a literal absolute file name as argument - no one
would code that way. 

> What if the user wants to save the configuration for two 
> packages in one custom file, but all other packages in the
> default place?

That would certainly be possible if each package provided the user a choice wrt
the location, as I imagine. The question is how to provide the user with such
choices - where to do that. 

In what I imagine, programmers can do that as they like, using whatever code
they think is appropriate as the various :file sexps. They can choose to make
these 4 options use the same sexp and those 3 options use a different sexp
(regardless of the whether the sexp values differ). They can choose to specify
:file at the `defgroup' level and define :group's that fit the persistencs and
loading needs. They can do just about anything they think is appropriate for
their users.

In designing this, the choice is not programmer vs user. The idea is to come up
with something for programmers that helps them help their users. Each has some
say in defining the persistence and load behavior. It's `and', not
`either...or'.

> This really is not an unusual need.  I like to keep my Gnus and BBDB
> variables separate from the rest of Emacs but in one file, for example.
> 
> The default path the package author picks may not be appropriate.
> For example, in Gnus a lot of work had to be done to remove 
> dependencies on ~/News and ~/Mail that were put in place over time.
> Many users needed those paths relocated to somewhere that made sense
> to them.

I don't disagree at all about any of that. The question is which tools to
provide programmers so they can meet user needs flexibly.

Re: :file keyword for Customize

[Ted Zlatanov]

On Thu, 8 May 2008 14:27:10 -0700 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> What I was thinking was that library writers would typically provide
DA> one or more user options or functions as the :file sexps - it is
DA> those sexps that would then let users, in effect, determine the
DA> :file values used. IOW, end users would not directly determine the
DA> sexps used for :file, but they would ultimately determine the values
DA> of those sexps in some way (defined by the programmer).
...
DA> In what I imagine, programmers can do that as they like, using
DA> whatever code they think is appropriate as the various :file
DA> sexps. [...] They can do just about anything they think is
DA> appropriate for their users.

That seems really complicated, and still requires every package to do it
explicitly.

It seems like you're trying to set up a :filegroup option, really, which
tells the Customize interface that some options should be logically
grouped together when they are saved.  I think defgroup does this
logical grouping already for the vast majority of options, so it's a
matter of saying "save this defgroup's variables in file X."  That may
be possible as an optional add-on package that doesn't change the way
Customize works normally.  The approach you proposed would, I think,
require changes to the core Customize behavior instead so it's less
desirable.

Ted

RE: :file keyword for Customize

[Stephen J. Turnbull]

Drew Adams writes:

 > This is about letting an individual option or face (or perhaps group) decide
 > where it is stored (and consequently let users decide when it is loaded). The
 > place to do that is in `defcustom' and `defface' (and perhaps `defgroup').

I think this is a YAGNI, and likely to lead to a plethora of confusing
custom files which appear at random times (ie, when you first save a
customization for a variable that targets that file).

RE: :file keyword for Customize

[Drew Adams]

>  > This is about letting an individual option or face (or 
>  > perhaps group) decide where it is stored (and consequently
>  > let users decide when it is loaded). The place to do that
>  > is in `defcustom' and `defface' (and perhaps `defgroup').
> 
> I think this is a YAGNI, and likely to lead to a plethora of confusing
> custom files which appear at random times (ie, when you first save a
> customization for a variable that targets that file).

OK, so you think that. Care to give a reason or two?

Why is throwing everything into the same sack better? I grant you that at least
one knows which sack everything is in, if there is only one sack. But beyond
that?

Re: :file keyword for Customize

[Jason Rumney]

Drew Adams wrote:
> Why is throwing everything into the same sack better? I grant you that at least
> one knows which sack everything is in, if there is only one sack. But beyond
> that?
>   

There are fewer ways to break things if you throw them all in the same sack.

RE: :file keyword for Customize

[Stephen J. Turnbull]

Drew Adams writes:
 > >  > This is about letting an individual option or face (or 
 > >  > perhaps group) decide where it is stored
 > > 
 > > I think this is a YAGNI, and likely to lead to a plethora of confusing
 > > custom files which appear at random times (ie, when you first save a
 > > customization for a variable that targets that file).
 > 
 > OK, so you think that. Care to give a reason or two?

YAGNI?  The proponent says he doesn't really have a use case in
<004101c8b129$788cd490$0ab32382@us.oracle.com>.  That's a slam dunk.

 > Why is throwing everything into the same sack better?

That's not what I said.  I said that the files appear at random times,
implying that that is what would be confusing.

 > I grant you that at least one knows which sack everything is in, if
 > there is only one sack.

That is another good reason.

 > But beyond that?

Sorry, you only get three wishes per magic lamp.  I'm not going to play.

Re: :file keyword for Customize

[Bastien]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Drew Adams writes:
>
>  > This is about letting an individual option or face (or perhaps group) decide
>  > where it is stored (and consequently let users decide when it is loaded). The
>  > place to do that is in `defcustom' and `defface' (and perhaps `defgroup').
>
> I think this is a YAGNI, and likely to lead to a plethora of confusing
> custom files which appear at random times (ie, when you first save a
> customization for a variable that targets that file).

I also think so. 

For me, using both a ~/.emacs.el and a ~/.emacs-custom.el is precisely
useful only because you have to edit the first one by hand and you can
leave the second one behind the curtain, you can even forget about it.

Having to forget about more than one file will make it harder to
remember the location of these files for the few occasions where
you'll have to edit them manually...

-- 
Bastien

RE: :file keyword for Customize

[Drew Adams]

> > The granularity for Customize customizations, in terms of 
> > persistence and loading is currently all or none: everything
> > is in one file (`custom-file' or `init-file').
>
> JDEE at least, saves its own customizations in project files in the 
> local directory. I think Gnus also lets you save your Gnus related 
> customizations in .gnus. That is why I suggested looking at 
> what those packages do.

I'm thinking of this for ordinary options and faces. I know that libraries can
write their own persistent data to whatever files they want. This is about
ordinary user options and faces.

> > It just occurred to me that it might be better to at least let
> > Emacs-Lisp programmers group customizations into smaller 
> > persistence/load units, if they see fit.
> 
> It might be better to do it at the group level than each individual 
> customization then, if grouping is the desired outcome.

I considered that, but I don't think so. This is for maximum flexibility.
Grouping via `defgroup' tends to be about user-apparent similarities, e.g.
similarity of area of application. This is a different kind of grouping, having
to do with where data is stored and when it is loaded.

We could of course also allow :file for `defgroup', to provide an easy way to
specify the same file for all options & faces in a particular group. There is no
reason, however, to hard-couple the two kinds of grouping. Also, it is common
for the same `defcustom' or `defface' to use multiple :group's.

Re: :file keyword for Customize

[David Kastrup]

"Drew Adams" <drew.adams@oracle.com> writes:

> How about adding a :file keyword for `defcustom' and `defface'?
>
> (defcustom foo-bar-flag t
>   "Non-nil means toto."
>   :type 'boolean :group 'foo
>   :file (if (eq system-type 'windows-nt)
>             "C:\\foo-stuff\\foo-cus.el"
>           "/toto/foo-stuff/foo-cus.el"))
>
> Its argument would be evaluated to a file name. That particular `defcustom' or
> `defface' setting would be saved in that file, instead of in `custom-file' or
> `init-file'. A nil or "" value for :file would mean the same thing as the
> absence of a :file keyword: as now, use `custom-file' or, if no `custom-file',
> `init-file'.
>
> This could provide a little more modularity for packages (and for any
> other groupings of options & faces).

I don't see what scattering the saved options across different files has
to do with modularity.

> It could help deal with things like platform differences (for a
> package) and initialization order of settings.

Uh no.  It actually _curbs_ being able to deal with platform differences
since you can no longer have one options file per platform.

> It could allow users a little more flexibility in terms of when some
> groups of Customize settings are loaded (currently, all Customize
> settings are loaded at once).

So what?  What is the problem you are trying to fix here?

> It could simplify communication with package authors about bugs
> (e.g. automatically include the package's Customize settings in a bug
> report).

Nonsense.  report.el already provides a way to report relevant settings.
And if you wanted to have a special way to report everything customized
for a package, you'd do it by listing the customization group rather
than some file.  We _never_ include a file in bug reports but rather
settings.  And since people could put their settings anywhere, and since
variables can be set without customizing them or even overriden, a file
would be utterly unreliable, anyway, as a source of information.

> I don't have a particular use-case in mind; it's just something that
> occurred to me.  There is nothing special in this, but I think it
> might help organize things a bit.  A user's `custom-file' or
> `init-file' can become a monolithic blob, and this could help cut down
> on that.

So what?  It is a "monolithic blob" not intended for human consumption,
and one can't really delay loading it, anyway.  So all this buys us is
added complication and longer load times.

Don't fix what is not broken.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

RE: :file keyword for Customize

[Drew Adams]

> > This could provide a little more modularity for packages 
> > (and for any other groupings of options & faces).
> 
> I don't see what scattering the saved options across 
> different files has to do with modularity.

Scattering is not about modularity, but no one suggested scattering. Grouping is
about modularity, and this is about grouping: deciding which options belong
together wrt persistence and loading.

The choice is not a binary one: (1) accept one giant lump or (2) explode and
scatter everything into elementary particles. It's about providing constructs
that let programmers and users decide how to group things for persistence and
loading: which things to save in the same file.

> > It could help deal with things like platform differences (for a
> > package) and initialization order of settings.
> 
> Uh no.  It actually _curbs_ being able to deal with platform 
> differences

How so? 

> since you can no longer have one options file per platform.

Why not? How would that be prevented?

How can adding choices prevent you from doing what you do today?

> > It could allow users a little more flexibility in terms of when some
> > groups of Customize settings are loaded (currently, all Customize
> > settings are loaded at once).
> 
> So what?  What is the problem you are trying to fix here?

I already said that I don't have a particular problem I am trying to fix.

Currently, all user settings are stored persistently in the same sack (modulo
special programming to store data in particular files). That's not very modular,
and it gives both the programmer and the user little choice wrt what is stored
where and when things are loaded - all Customize settings are stored together
and loaded together.

> > It could simplify communication with package authors about bugs
> > (e.g. automatically include the package's Customize 
> > settings in a bug report).
> 
> Nonsense.  report.el already provides a way to report 
> relevant settings.

That report.el exists does not imply that the proposed feature is nonsense. Such
reasoning is, however, nonsense.

> And if you wanted to have a special way to report everything 
> customized for a package, you'd do it by listing the customization
> group rather than some file.

Not necessarily. Customization groups can have many different meanings and
different grouping criteria. Those don't necessarily map directly to appropriate
persistence and loading models.

> We _never_ include a file in bug reports but rather settings.

I'll let you speak for what "we _never_" do, but I, for one, have sometimes been
interested to see what a user's .emacs or `custom-file' looked like in
connection with a bug report. Seeing dynamic settings and seeing persistent
values (and knowing when those are loaded) both can sometimes provide useful
information. They are not exclusive options. You seem to be bent on imposing
non-existent binary choices.

There is, in any case, nothing in what I proposed that requires you ("we") to
use the feature to communicate persistent settings in bug reports.  

> And since people could put their settings anywhere, and since
> variables can be set without customizing them or even 
> overriden, a file would be utterly unreliable, anyway, as a
> source of information.

Again, nothing forces you to "rely" on persistent settings. But there is more
than one request in the help lists to see what might be in a reporting user's
.emacs - sometimes that's useful.

If such information is not useful in some context, or if you think it is
"_never_" interesting to you, then, uh, just don't use it; don't ask for it.

I would like to be able to (be _able_ to, not be forced to) suggest to a user to
load the customizations for package foo after, not before, those for package
bar. That is not possible today, when it comes to Customize settings - they're
simply all lumped together.

> > I don't have a particular use-case in mind; it's just something that
> > occurred to me.  There is nothing special in this, but I think it
> > might help organize things a bit.  A user's `custom-file' or
> > `init-file' can become a monolithic blob, and this could 
> > help cut down on that.
> 
> So what?  It is a "monolithic blob" not intended for human 
> consumption,

It's certainly intended for programmatic use. And ultimately for human use, in
terms of its effects.

Such an argument could be applied to all software engineering (and sometimes
was, about 40 years ago): just throw everything in one sack; it's all internal,
so it doesn't matter how messy it is.

> and one can't really delay loading it, anyway.  So all this buys us is
> added complication and longer load times.

No idea what you mean by that. Users today can decide when to load
`custom-file'. They could do the same with any such files of Customize options,
the same way.

> Don't fix what is not broken.

Not every improvement is a bug fix.

Re: :file keyword for Customize

[Jason Rumney]

Drew Adams wrote:
> No idea what you mean by that. Users today can decide when to load
> `custom-file'. They could do the same with any such files of Customize options,
> the same way.
>   

I'm having trouble seeing what use this feature is. The user doesn't 
have control over which file different options are saved in, and the 
package authors don't have control over when or even if the various 
custom files they write to are loaded relative to each other. Who is 
this for, and what problem is it trying to solve?

RE: :file keyword for Customize

[Drew Adams]

> > No idea what you mean by that. Users today can decide when to load
> > `custom-file'. They could do the same with any such files 
> > of Customize options, the same way.
> 
> I'm having trouble seeing what use this feature is. The user doesn't 
> have control over which file different options are saved in,

Yes, s?he would, typically. The programmer would use as a :file sexp a user
option or a function that in some way allowed the user such a choice.

But the choice would not necessarily be arbitrary and separate for each option.
See my previous response about this.

> and the package authors don't have control over when or even if the various 
> custom files they write to are loaded relative to each other.

In general, that is correct (and a good thing). But here too, package authors
could defined the context to some extent, within which a user makes such a
choice. But, yes, a user can decide when to load a given Customize settings
file. That is one of the reasons for this feature.

> Who is this for, and what problem is it trying to solve?

See previous, wrt load decisions. In general, this would give both programmers
and users more control over where related options are stored and when they are
loaded. That's all.

Re: :file keyword for Customize

[David Kastrup]

"Drew Adams" <drew.adams@oracle.com> writes:

>> Who is this for, and what problem is it trying to solve?
>
> See previous, wrt load decisions. In general, this would give both
> programmers and users more control over where related options are
> stored and when they are loaded. That's all.

What problem is it trying to solve, Drew?  What problem?  We don't have
a way to specify which memory addresses Emacs is supposed to load
particular libraries to, either, because it is pointless.  It is
nonsensical to give people more control over things that have no
discernable advantage.

So please let us focus on solving existing problems instead of
_creating_ new ones.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: :file keyword for Customize

[David Kastrup]

"Drew Adams" <drew.adams@oracle.com> writes:

>> So what?  What is the problem you are trying to fix here?
>
> I already said that I don't have a particular problem I am trying to
> fix.

Then let's leave this alone until there is a problem.

> Currently, all user settings are stored persistently in the same sack
> (modulo special programming to store data in particular files). That's
> not very modular, and it gives both the programmer and the user little
> choice wrt what is stored where and when things are loaded - all
> Customize settings are stored together and loaded together.

Yes.  And the whole "Emacs" executable is stored together and loaded
together.  There are even files preloaded into it in order to make it
load faster.  What is bad about that?  Why give the user "choice" about
something which is utterly irrelevant?  So that there are more things
that can go wrong or be misconfigured without any advantage?

> I would like to be able to (be _able_ to, not be forced to) suggest to
> a user to load the customizations for package foo after, not before,
> those for package bar. That is not possible today, when it comes to
> Customize settings - they're simply all lumped together.

C-h f defcustom RET

[...]

:set-after VARIABLES
	Specifies that symbol should be set after the list of variables
        VARIABLES when both have been customized.

>> > I don't have a particular use-case in mind; it's just something
>> > that occurred to me.  There is nothing special in this, but I think
>> > it might help organize things a bit.  A user's `custom-file' or
>> > `init-file' can become a monolithic blob, and this could help cut
>> > down on that.
>> 
>> So what?  It is a "monolithic blob" not intended for human
>> consumption,
>
> It's certainly intended for programmatic use. And ultimately for human
> use, in terms of its effects.

But splitting it causes no different effect whatsoever except making the
whole less transparent.

> Such an argument could be applied to all software engineering (and
> sometimes was, about 40 years ago): just throw everything in one sack;
> it's all internal, so it doesn't matter how messy it is.

Lisp mostly works with a single obarray.  This makes things more
transparent rather than more opaque.  All one sack.  There is nothing
messy about that.

>> and one can't really delay loading it, anyway.  So all this buys us
>> is added complication and longer load times.
>
> No idea what you mean by that. Users today can decide when to load
> `custom-file'. They could do the same with any such files of Customize
> options, the same way.

Why?

>> Don't fix what is not broken.
>
> Not every improvement is a bug fix.

Not every complication is an improvement.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: :file keyword for Customize

[Ted Zlatanov]

On Thu, 08 May 2008 22:48:47 +0200 David Kastrup <dak@gnu.org> wrote: 

DK> So what?  [the user's custom file] is a "monolithic blob" not
DK> intended for human consumption, and one can't really delay loading
DK> it, anyway.  So all this buys us is added complication and longer
DK> load times.

I think monolithic blobs have historically been proven unsuitable for
human consumption.  In the case of a binary, that's fine.  For
configuration data, that's dangerous.

For example, the /etc/shadow file is intended to make the /etc/passwd
file less dangerous when compromised.  /etc/passwd can do the job by
itself, but it makes sense for security to take the dangerous bits out.

As another example, the Windows registry is a monolithic blob.  A
special tool is needed to edit the blob and its capabilities are limited
by the lowest common denominator (key=value, plain text data).

In the same way a user's customizations for authentication are more
sensitive than the default customizations, and should probably be stored
in a separate file that's encrypted with some mechanism.  The default
customizations are then available to fellow users without danger of
compromising security.

In the case of Gnus, it makes sense to separate Gnus-specific settings.
I do this manually in my gnus.el, but I can't customize the Gnus
variables without a manual resync between my custom file and gnus.el.

I think Drew's proposal is worth considering as an add-on package, but I
wouldn't push it on every user by default.

Ted

Re: :file keyword for Customize

[David Kastrup]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Thu, 08 May 2008 22:48:47 +0200 David Kastrup <dak@gnu.org> wrote: 
>
> DK> So what?  [the user's custom file] is a "monolithic blob" not
> DK> intended for human consumption, and one can't really delay loading
> DK> it, anyway.  So all this buys us is added complication and longer
> DK> load times.
>
> I think monolithic blobs have historically been proven unsuitable for
> human consumption.

What about "not intended for human consumption" did you not understand?

> In the same way a user's customizations for authentication are more
> sensitive than the default customizations, and should probably be
> stored in a separate file that's encrypted with some mechanism.

Put them into ~/.netrc

> The default customizations are then available to fellow users without
> danger of compromising security.

There is nothing worse than people indiscriminately copying .emacs
files, anyway.

-- 
David Kastrup

Re: :file keyword for Customize

[Ted Zlatanov]

On Fri, 09 May 2008 16:34:49 +0200 David Kastrup <dak@gnu.org> wrote: 

DK> Ted Zlatanov <tzz@lifelogs.com> writes:
>> In the same way a user's customizations for authentication are more
>> sensitive than the default customizations, and should probably be
>> stored in a separate file that's encrypted with some mechanism.

DK> Put them into ~/.netrc

That's not always possible.  User names and passwords are not the only
possible sensitive information.  The netrc file is not as flexible as a
custom file.  Also, not every Emacs package supports netrc files (I'm
actually working on auth-source.el to remedy that somewhat, but it's not
ready).

>> The default customizations are then available to fellow users without
>> danger of compromising security.

DK> There is nothing worse than people indiscriminately copying .emacs
DK> files, anyway.

That's not what I'm talking about, and there are many things worse than
copying .emacs files.  Please consider my actual use cases that
substantiate the need for separate custom files.  If you think they are
not generally applicable, fine, but don't use hyperbole to derail the
discussion.

Ted

Re: :file keyword for Customize

[Stephen J. Turnbull]

Ted Zlatanov writes:

 > Please consider my actual use cases that substantiate the need for
 > separate custom files.  If you think they are not generally
 > applicable, fine, but don't use hyperbole to derail the discussion.

Your security use case clearly substantiates the need for separate
files, but also pretty much rules out the use of customize in my
opinion.  For example, use of customize to customize authentification
information almost certainly will leave that information in memory,
for root, somebody who has compromised your account, or a core dump to
access in clear.

Certainly you might want to use *some* of the widget machinery for
formatting input screens, but the actual input handling should be
special-cased in the same way functions that handle passwords are.

:password customize type (was: :file keyword for Customize)

[Ted Zlatanov]

On Sat, 10 May 2008 06:17:43 +0900 "Stephen J. Turnbull" <stephen@xemacs.org> wrote: 

SJT> Ted Zlatanov writes:
>> Please consider my actual use cases that substantiate the need for
>> separate custom files.  If you think they are not generally
>> applicable, fine, but don't use hyperbole to derail the discussion.

SJT> Your security use case clearly substantiates the need for separate
SJT> files, but also pretty much rules out the use of customize in my
SJT> opinion.  For example, use of customize to customize authentification
SJT> information almost certainly will leave that information in memory,
SJT> for root, somebody who has compromised your account, or a core dump to
SJT> access in clear.

Yes, a :password type is needed for sure, so Emacs can treat it
specially.  In addition to the visuals (don't display it when it's
entered), it could ask for two fields and make sure they are the same.
I couldn't find proposals of a :password type when I searched, and the
ELisp manual doesn't have it.  Has this been discussed before?  It could
also be :secret, indicating that not just passwords qualify.

I know little of the Emacs internals but I'd guess this has to be
implemented in the C core.

IMO worrying about root compromises is futile.  If an attacker has root
on your machine, scrambling memory contents is useless.  The :password
custom type should maybe scramble things with a simple symmetric cipher
(to protect against memory dumps), but anything more is
overkill.

OTOH, file contents are permanent and much more susceptible to
compromise, so encryption is always a good idea if the information is
sensitive.  Also, if the file name is well-known, it's much easier to be
compromised.  Mozilla/Firefox for instance save profiles in a
hard-to-guess directory, so casual attackers will have a harder time
finding secret data.  Similarly in the case of Emacs, it would be nice
if the user could decide on a hard-to-guess location for sensitive data
while keeping the main custom file accessible, which comes back to
Drew's :file proposal.  Maybe it's enough to have just two custom files:
one for regular data, one for secret data?

Ted

Re: :file keyword for Customize

[Richard M Stallman]

To split up the Custom info into separate files seems undesirable to
me.  And it looks like your proposal is both (1) taking up lots of
time and (2) not really useful for anything.

I suggest that people drop the subject.

Re: :file keyword for Customize

[Paul R]

"Drew Adams" <drew.adams@oracle.com> writes:

> How about adding a :file keyword for `defcustom' and `defface'?
>
> (defcustom foo-bar-flag t
>   "Non-nil means toto."
>   :type 'boolean :group 'foo
>   :file (if (eq system-type 'windows-nt)
>             "C:\\foo-stuff\\foo-cus.el"
>           "/toto/foo-stuff/foo-cus.el"))
>
> (...)
>
> I don't have a particular use-case in mind; it's just something that occurred to
> me. There is nothing special in this, but I think it might help organize things
> a bit. A user's `custom-file' or `init-file' can become a monolithic blob, and
> this could help cut down on that.
>

Well, I see a clear use case : providing tools to manage emacs
configuration in multiple files, instead of in a single monolithic
.emacs. I've been using a system for several months now, which aim is
the same : have clear boundaries between different chunks of
configurations, namely putting them into different files.
As an exemple, my gnus-related configuration is loaded just before
gnus is activated, not earlier. Same for org-mode, or for just any
emacs tool I use. Emacs is for most of users more than a file-content
editor, it is a platforme with many *different* and *independant*
tools on it. We do not want a single configuration file for all
different tools we use, do we ? At least, it is not the way it works
generally. I think we should make a plan for modular emacs
configuration and packages managements (external lisp packages), and
this proposal should be part of this plan.

More info on configuration managers at http://emacs.kekerekex.net/ and
http://www.emacswiki.org/cgi-bin/wiki/Grail


-- 
      Paul

Re: :file keyword for Customize

[Stefan Monnier]

>> How about adding a :file keyword for `defcustom' and `defface'?
>> 
>> (defcustom foo-bar-flag t
>> "Non-nil means toto."
>> :type 'boolean :group 'foo
>> :file (if (eq system-type 'windows-nt)
>> "C:\\foo-stuff\\foo-cus.el"
>> "/toto/foo-stuff/foo-cus.el"))

OK guys, can we stop this?
The above is an absolute non-starter.


        Stefan

Re: :file keyword for Customize

[Miles Bader]

Paul R <paul.r.ml@gmail.com> writes:
> Well, I see a clear use case : providing tools to manage emacs
> configuration in multiple files, instead of in a single monolithic
> .emacs.

I suspect it would be much better to offer more indirect methods of
categorization.

[E.g., allow the user to specify "settings for custom group X go in file Y".]

-Miles

-- 
`Life is a boundless sea of bitterness'