Feature Request: Per-package custom save files?

Feature Request: Per-package custom save files?

[T. V. Raman]

As a long-term emacs user, I switched to custom about 11 years
ago.

I let custom save its settings to a .customize-emacs -- rather
than let it add to my .emacs so I can continue to edit my .emacs
by hand.

This means that I've let custom manage .customize-emacs -- that
is nice; however looking at it now, I see that that file has
become huge.  What's more  -- there are a lot of saved settings
in there that are likely now obsolete/irrelevant -- also some
settings that needed to be set explicitly in early versions of
packages is now achieved by not doing anything since the defaults
do the right thing. Finally,  custom ends up loading
customizations for *every* package you've ever seen -- including
ones that you use at present, ones that you autoload and ones
that you dont have around any more.

So feature request:

Could we arrange for customize to save package specific settings
in a package-specific customizations file?  The present one
custom file that binds them all feels like the Windows Registry:-)
-- 

-- 

Re: Feature Request: Per-package custom save files?

[David Kastrup]

"T. V. Raman" <raman@google.com> writes:

> This means that I've let custom manage .customize-emacs -- that
> is nice; however looking at it now, I see that that file has
> become huge.  What's more  -- there are a lot of saved settings
> in there that are likely now obsolete/irrelevant -- also some
> settings that needed to be set explicitly in early versions of
> packages is now achieved by not doing anything since the defaults
> do the right thing. Finally,  custom ends up loading
> customizations for *every* package you've ever seen -- including
> ones that you use at present, ones that you autoload and ones
> that you dont have around any more.
>
> So feature request:
>
> Could we arrange for customize to save package specific settings
> in a package-specific customizations file?  The present one
> custom file that binds them all feels like the Windows Registry:-)

That would affect load time quite a bit.  Doing a sorted cleanup should
be reasonably possible using

M-x customize-saved RET

-- 
David Kastrup

Re: Feature Request: Per-package custom save files?

[John Yates]

On Sat, Jun 21, 2014 at 11:55 AM, David Kastrup <dak@gnu.org> wrote:
>
> ...  Doing a sorted cleanup should be reasonably possible using
>
> M-x customize-saved RET

Yuck.  That is a horrible experience, in some ways better but also in
some ways worse than auditing one's custom file manually.

I agree with some of T. V. Raman's sentiments.  I like to use the
customize interface for the purpose of exploring the options offered
by a package.  Still long ago I discovered that the customize-file
provides no help for understanding one's configuration.  My approach
is to document all of my customizations in my .emacs file.  I still
use custom to load those customizations but I automatically compare
the contents of my two file to confirm that they are in sync.  You can
see this in my .emacs on github
(https://github.com/jsyjr/MyConfiguration/blob/master/.emacs, most
easily viewed with folding mode).  See in particular the sections near
the top of the file labeled:
  ;;{{{ Customization auditing framework
  ;;{{{ Load customizations
And the section near the bottom of the file labeled:
  ;;{{{ Audit customizations

T. V. Raman points out the phenomenon of having explicit
customizations recorded that merely duplicate defaults.  Currently my
scheme does not address this issue.

/john

Re: Feature Request: Per-package custom save files?

[Stefan Monnier]

> Could we arrange for customize to save package specific settings
> in a package-specific customizations file?  The present one
> custom file that binds them all feels like the Windows Registry:-)

Other than the feeling, what benefits are you looking for?
I'm not just asking rhetorically: the best solution depends on the answer.

One useful thing would be a custom-cleanup command which could show you
all your settings that are equal to the default.  Showing the settings
for packages you don't use would be a more difficult.  I guess we could
search for those settings in your load-path and if it's not found, it's
clearly an "obsolete" setting.  But if the setting still exists (you
just don't use its package), I'm not sure what we could do.  Maybe we
could move all your settings to an "unused" list of settings and then
move the settings out of that list whenever they're used.  So after
a few days/weeks you can look at the settings remaining in the "unused
list" to see which ones qualify.


        Stefan

Re: Feature Request: Per-package custom save files?

[T.V Raman]

Hi Stefan,

Here are some "features" I'm looking for:
1. custom-clean-up would be nice.

2. My thought around factoring out custom settings into a
canonical per-package custom-settings file was to achieve the
second goal -- not loading settings for packages you dont use --

so: Today I have 
(setq custom-file (expand-file-name "~/.customize-emacs"))
In my .emacs.

What if instead -- for each add-on elisp package <foo>    we
used custom-<foo>-settings.el  -- then as an example,  I could
load custom-ecb-settings.el only when I load the <ecb> package.


Stefan Monnier writes:
 > > Could we arrange for customize to save package specific settings
 > > in a package-specific customizations file?  The present one
 > > custom file that binds them all feels like the Windows Registry:-)
 > 
 > Other than the feeling, what benefits are you looking for?
 > I'm not just asking rhetorically: the best solution depends on the answer.
 > 
 > One useful thing would be a custom-cleanup command which could show you
 > all your settings that are equal to the default.  Showing the settings
 > for packages you don't use would be a more difficult.  I guess we could
 > search for those settings in your load-path and if it's not found, it's
 > clearly an "obsolete" setting.  But if the setting still exists (you
 > just don't use its package), I'm not sure what we could do.  Maybe we
 > could move all your settings to an "unused" list of settings and then
 > move the settings out of that list whenever they're used.  So after
 > a few days/weeks you can look at the settings remaining in the "unused
 > list" to see which ones qualify.
 > 
 > 
 >         Stefan

Re: Feature Request: Per-package custom save files?

[chad]

On 23 Jun 2014, at 09:52, T.V Raman <raman@google.com> wrote:

> 2. My thought around factoring out custom settings into a
> canonical per-package custom-settings file was to achieve the
> second goal -- not loading settings for packages you dont use --
> 
> so: Today I have 
> (setq custom-file (expand-file-name "~/.customize-emacs"))
> In my .emacs.
> 
> What if instead -- for each add-on elisp package <foo>    we
> used custom-<foo>-settings.el  -- then as an example,  I could
> load custom-ecb-settings.el only when I load the <ecb> package.


Take a look at jwiegley's use-package package for some cool ideas in this space:

	https://github.com/jwiegley/use-package
	http://melpa.milkbox.net/#/use-package

~Chad

Re: Feature Request: Per-package custom save files?

[T.V Raman]

Pulled use-package --it's a nice idea!  Would be really nice to
bring package-specific custom settings into the scope of a
use-package call.

chad writes:
 > On 23 Jun 2014, at 09:52, T.V Raman <raman@google.com> wrote:
 > 
 > > 2. My thought around factoring out custom settings into a
 > > canonical per-package custom-settings file was to achieve the
 > > second goal -- not loading settings for packages you dont use --
 > > 
 > > so: Today I have 
 > > (setq custom-file (expand-file-name "~/.customize-emacs"))
 > > In my .emacs.
 > > 
 > > What if instead -- for each add-on elisp package <foo>    we
 > > used custom-<foo>-settings.el  -- then as an example,  I could
 > > load custom-ecb-settings.el only when I load the <ecb> package.
 > 
 > 
 > Take a look at jwiegley's use-package package for some cool ideas in this space:
 > 
 > 	https://github.com/jwiegley/use-package
 > 	http://melpa.milkbox.net/#/use-package
 > 
 > ~Chad

Re: Feature Request: Per-package custom save files?

[Stefan Monnier]

> 2. My thought around factoring out custom settings into a
> canonical per-package custom-settings file was to achieve the
> second goal -- not loading settings for packages you dont use --

That doesn't really sound like a goal to me.  More specifically, I'd
like to know why you care about loading settings you don't use.

IOW, let's imagine we change it so that the settings are only loaded
once they're needed.  How will you tell the difference?


        Stefan

Re: Feature Request: Per-package custom save files?

[T.V Raman]

HMM, good point 

What I'm mostly looking for is  to have a modular collection of
custom files -- note -- here I'm explicitly talking about
customizations for add-on packages -- not for built-ins.  

Stefan Monnier writes:
 > > 2. My thought around factoring out custom settings into a
 > > canonical per-package custom-settings file was to achieve the
 > > second goal -- not loading settings for packages you dont use --
 > 
 > That doesn't really sound like a goal to me.  More specifically, I'd
 > like to know why you care about loading settings you don't use.
 > 
 > IOW, let's imagine we change it so that the settings are only loaded
 > once they're needed.  How will you tell the difference?
 > 
 > 
 >         Stefan

Re: Feature Request: Per-package custom save files?

[Stefan Monnier]

> What I'm mostly looking for is  to have a modular collection of
> custom files -- note -- here I'm explicitly talking about
> customizations for add-on packages -- not for built-ins.

That's still not very clear.  For example, we could split the
customization "by package" (more or less) while still loading all the
customizations at startup.  So it would satisfy your above requirement,
but not the requirement you sent earlier (quoted below).

And it's still not clear how that would change your life, as an Emacs user.

Again, I'm not arguing against it, really, but there are many ways to
"do it" with very different results, and some answers may correspond to
one of your descriptions yet might fail to really solve your problem.
So, if you could give us an example of a concrete problem you face, that
would help.
E.g. "loading my customization file takes too long".
Or "I get lost in my customize-changed".
Or "I share my customization between different systems but some
customizations are system-specific so I don't want to share those".
...


        Stefan


> Stefan Monnier writes:
>> > 2. My thought around factoring out custom settings into a
>> > canonical per-package custom-settings file was to achieve the
>> > second goal -- not loading settings for packages you dont use --
>> 
>> That doesn't really sound like a goal to me.  More specifically, I'd
>> like to know why you care about loading settings you don't use.
>> 
>> IOW, let's imagine we change it so that the settings are only loaded
>> once they're needed.  How will you tell the difference?
>> 
>> 
>> Stefan

Re: Feature Request: Per-package custom save files?

[raman]

So:

Once the custom settings are split out:

1. Load on demand 

2. Smaller, more manageable custom files.

Between 1 and 2 I care more about 2 -- computers are plenty fast today
to make 1 unnoticeable.

The reason this bubbled up for me is that in the past few months I've
had to go back in time to retrieve a custom file from my local git repo
because the giant custom file got corrupted during an emacs  custom-save
at some point -- I've not chased down the culprit -- but when that
happens, you're left with a giant file that is impossible to fix by
hand. I now regularly git commit my .custom file to a local repo just
for this reason 

Re: Feature Request: Per-package custom save files?

[Stefan Monnier]

> Once the custom settings are split out:
> 1. Load on demand

I don't see any need for it.  The time to load a single customization file
should be negligible unless it's *really* large or causes loading other
files (e.g. if some of the customizations enable minor modes).

And doing it "on-demand" might be very difficult.  E.g. you can't delay
"activate global-reveal-mode" to when reveal.el is loaded, since it's the
activation of global-reveal-mode which would cause loading reveal.el.
And it's this loading which takes the time, so delaying the other
reveal-related settings would be pointless since these take
a ridiculously small amount of time.

If we want to speed up startup, there are several things we can do, but
I don't think any of them would be helped by splitting the
customization file.

> 2. Smaller, more manageable custom files.

We can cut the file in chunks, but that doesn't mean it's necessarily
more manageable.

> The reason this bubbled up for me is that in the past few months I've
> had to go back in time to retrieve a custom file from my local git repo
> because the giant custom file got corrupted during an emacs  custom-save
> at some point -- I've not chased down the culprit -- but when that
> happens, you're left with a giant file that is impossible to fix by
> hand.

Can you give details of the difficulties you encountered?
Would they have been different if the customizations were split
among several files, each of them similarly corrupted?
Or would it only be different under the assumption that the corruption
would have affected a single file, thus reducing the amount of corruption?

The customization file should contain 2 parts: the face settings and the
var settings.  Each part should be sorted alphabetically which (due to
the prefix-convention we use for names) should largely group settings
"by package".

So splitting the "one big file" into several smaller files would not
really change any ordering.

> I now regularly git commit my .custom file to a local repo just
> for this reason

I do that as well, tho not for this reason.


        Stefan

Re: Feature Request: Per-package custom save files?

[T.V Raman]

Like I said -- (1) is not relevant to me -- not sure why you
spent a whole paragraph beating that dead horse;-)

Where my custom file has broken:


when custom settings that are large s-expressions break during
save -- say the partial sexp gets saved, the whole file breaks.
Examples of large customization settings:

1. Newsticker -- list of feeds

2. keybinding lists from various packages.

Anyway as I said, I have a soution that works for  me -- personal
git repo and cron joab that backs up my custom settings. 

Stefan Monnier writes:
 > > Once the custom settings are split out:
 > > 1. Load on demand
 > 
 > I don't see any need for it.  The time to load a single customization file
 > should be negligible unless it's *really* large or causes loading other
 > files (e.g. if some of the customizations enable minor modes).
 > 
 > And doing it "on-demand" might be very difficult.  E.g. you can't delay
 > "activate global-reveal-mode" to when reveal.el is loaded, since it's the
 > activation of global-reveal-mode which would cause loading reveal.el.
 > And it's this loading which takes the time, so delaying the other
 > reveal-related settings would be pointless since these take
 > a ridiculously small amount of time.
 > 
 > If we want to speed up startup, there are several things we can do, but
 > I don't think any of them would be helped by splitting the
 > customization file.
 > 
 > > 2. Smaller, more manageable custom files.
 > 
 > We can cut the file in chunks, but that doesn't mean it's necessarily
 > more manageable.
 > 
 > > The reason this bubbled up for me is that in the past few months I've
 > > had to go back in time to retrieve a custom file from my local git repo
 > > because the giant custom file got corrupted during an emacs  custom-save
 > > at some point -- I've not chased down the culprit -- but when that
 > > happens, you're left with a giant file that is impossible to fix by
 > > hand.
 > 
 > Can you give details of the difficulties you encountered?
 > Would they have been different if the customizations were split
 > among several files, each of them similarly corrupted?
 > Or would it only be different under the assumption that the corruption
 > would have affected a single file, thus reducing the amount of corruption?
 > 
 > The customization file should contain 2 parts: the face settings and the
 > var settings.  Each part should be sorted alphabetically which (due to
 > the prefix-convention we use for names) should largely group settings
 > "by package".
 > 
 > So splitting the "one big file" into several smaller files would not
 > really change any ordering.
 > 
 > > I now regularly git commit my .custom file to a local repo just
 > > for this reason
 > 
 > I do that as well, tho not for this reason.
 > 
 > 
 >         Stefan

Re: Feature Request: Per-package custom save files?

[Stefan Monnier]

> when custom settings that are large s-expressions break during
> save --

If you still have such a broken file, please M-x report-emacs-bug to see
what we could do to avoid these problems.

Maybe for such important files that aren't saved very often, it would be
worthwhile to double-check that what we're about to write is what we
intended to write (i.e. try to read it back before saving).

> say the partial sexp gets saved, the whole file breaks.

Right, in some cases, indeed, having several files could confine
the breakage.  In that case, the specific way the list is broken up
doesn't matter that much, which is good.

Maybe we could simply allow custom-file to be a function which takes the
list of settings to save and splits them into a list of (FILE
. SETTINGS).


        Stefan

Re: Feature Request: Per-package custom save files?

[T.V Raman]

Having custom-save split the custom file into a set of subfiles
would satisfy a lot of these requirements.