Settings

Settings

[Lars Ingebrigtsen]

Reading some bug reports about problems with saving Customize settings
(for instance, running two Emacs instances and customising different
variables and saving them will overwrite each other; they're not written
to the .emacs file atomically so an error will make the .emacs file
invalid) reminded me of something that was discussed in the NSM context
years ago, but never acted upon, which is:

Emacs should have one way to save settings.

Currently, there way settings are saved are in three classes:

1) Older stuff:

(put 'narrow-to-region 'disabled nil) saved to ~/.emacs (or whatever
the init file is called).

2) Post-Customize stuff saved to ~/.emacs: 

(custom-set-variables
 ;; custom-set-variables was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 '(load-home-init-file t t)
 ...)

3) Ad-hoc methods for saving key/values in different files
(~/url/cookies, ~/.emacs.d/network-security.data)

I think Emacs can do better.

As a first principle, I think Emacs should not mess, at all, with the
~/.emacs file.  That's for users to put their lovingly hand-crafted
setqs in.

Second of all, Emacs should allow listing, with one command, all
settings that Emacs has saved on your behalf, and that allows you to
remove anything you don't want to have saved.

And finally, Emacs should save some metadata about the setting, like
when it was done and from what Emacs version it was done.

This is where somebody will say "let's use sqlite3!", and sure, a real
database would be nice.  If someone were to come up with a fast,
query-based solution, that'd be great.

We could also just use an append-mostly data-base-log like approach,
which would go something like this:

There's a file ~/.emacs.d/settings, and whenever we want to register a
command as not disabled, or save a Customize setting, or store an NSM
exception, we just append a form to that file.  It could be, like,

[:type variable :variable load-home-init-file :value t :time
"2019-08-15T14:43:37" :version "27.1"]
[:type plist :symbol narrow-to-region :property disabled :value nil :time
"2019-08-15T14:43:37" :version "27.1"]
[:type key/value :realm "NSM" :key "sha1:5b77fdefcb11074b38d23d182a7fb414b0d8be24"
:value "sha1:a7:e2:85:c8:59:2a:e9:06:98:ad:86:ee:ff:ea:39:34:04:6f:e0:c6"
:time "2019-08-15T14:43:37" :version "27.1"]

and when Emacs starts up, then it reads the file and the final value
wins.  (We'd have a compacting function to get rid of redundant
entries.)

Of course the syntax should be a whole lot less verbose -- the file
doesn't have to be very human-readable.

There's a compat issue, though -- if you start, say, Emacs 28, and it
converts your old-timey settings into new-style settings, then if you
shift back to Emacs 27, you may not have settings any more.  But the
converter could leave the forms in place in .emacs and add helpful
comments about them not actually being used any more in Emacs 28, so I
don't think that's an insurmountable problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

RE: Settings

[Drew Adams]

> Reading some bug reports about problems with saving Customize settings...

For example?  What kinds of problems saving
Customize settings?

But you go on to talk about non-Customize
settings.  What kinds of problems can you
report about saving those?

Just what's the problem you're trying to solve?

> Emacs should have one way to save settings.

Why only one?  What's the problem?

> Currently, there way settings are saved are in three classes:
> 1) Older stuff:
> (put 'narrow-to-region 'disabled nil) saved to ~/.emacs (or whatever
> the init file is called).

Are you referring to the use of symbol property
lists?  Or just to the fact that such info is
being saved in the init file?  Or are you
referring only to the use of property `disabled'
to control command disabling?

(Property lists are about as old as Lisp, but
there's nothing outmoded or "older" about them.)

> 2) Post-Customize stuff saved to ~/.emacs:
> 
> (custom-set-variables
>  ;; custom-set-variables was added by Custom.
>  ;; If you edit it by hand, you could mess it up, so be careful.
>  ;; Your init file should contain only one such instance.
>  ;; If there is more than one, they won't work right.
>  '(load-home-init-file t t)
>  ...)

Similarly, is this about Customize settings or
the fact of saving them in the init file?
 
> 3) Ad-hoc methods for saving key/values in different files
> (~/url/cookies, ~/.emacs.d/network-security.data)
> 
> I think Emacs can do better.

"Ad hoc" and "better' makes it sound like there
is no reason for saving different things in
different files.

The devil's in the details, for all such
considerations.  It can make sense for a given
function to save some info in a specific file
(e.g. chosen by the user).  Depends on the info
and its use.

And it can make sense for some info to be
handled by Customize, as options and faces, and
so saved in the single file that's handled by
Customize.

> As a first principle, I think Emacs should not mess, at all, with the
> ~/.emacs file.  That's for users to put their lovingly hand-crafted
> setqs in.

On that, I agree 100%.  I've long advocated
that users use `custom-file' and not let
Customize fiddle with their init file.

> Second of all, Emacs should allow listing, with one command, all
> settings that Emacs has saved on your behalf, and that allows you to
> remove anything you don't want to have saved.

Define "Emacs", here.

If library synonyms.el has user options for a
cache file and a thesaurus file, then that
library - not "Emacs" - should be responsible
for the functions that update/save such info.

Why would "Emacs" have any need to know about
or include such files in any listing, which
arguably "Emacs" (by way of that library)
saved on the user's behalf?

Likewise, a bookmark library is responsible
for a user's bookmarks file(s).  That's how
it should be.

A user can easily transport just her bookmarks,
or whatever.  Why would we want to save
everything that a user might want to persist
in the same giant file/database?

Customize already lets a user "list" options
and their values, and it does so on a library
basis.

Customize also allows ad hoc (yes) tags, called
"groups" for categorizing its options and faces.
And you can "list" the entries for any group.

> And finally, Emacs should save some metadata about the setting, like
> when it was done and from what Emacs version it was done.

Again, this should be up to (the user and) the
facility that's managing the persistent thingy:
Customize or a library.  And only if it (its
author) thinks that's helpful/appropriate for
that thingy.

> There's a file ~/.emacs.d/settings, and whenever we want to register a
> command as not disabled, or save a Customize setting, or store an NSM
> exception, we just append a form to that file...
> and when Emacs starts up, then it reads the file and the final value
> wins.  (We'd have a compacting function to get rid of redundant
> entries.)

No one-size-fits-all solution looking for a
problem to solve, please.

The following should be decided based on the
particular thingy and its purpose:

* Whether and when and where to save it
* Whether and when and where to restore it

And who better knows those conditions than the
relevant library (together with the user)?

Even when to load `custom-file' is problematic.
Ultimately only the user knows what's right
for that.

> if you start, say, Emacs 28, and it converts
> your old-timey settings into new-style settings,

I'll try to be sure to make sure that never
happens. ;-)

Doesn't sound great so far, to me.  Sounds
like a one-size-fits-all solution in search
of a copasetic problem.

But if you report about specific problems
you're looking to solve then perhaps things
will be clearer.

Before inventing a high-level solution, can
we please see some specific problems that
you think call for such a solution?

Re: Settings

[Stefan Monnier]

> As a first principle, I think Emacs should not mess, at all, with the
> ~/.emacs file.  That's for users to put their lovingly hand-crafted
> setqs in.

I don't have a strong opinion on that, but I think it's important to
realize that depending on the circumstances, the custom-set-variables
call may have to come before some of the user's config as well as it may
also have to come after some of the user's config.

IOW if Emacs just loads ~/.emacs.d/custom.el before ~/.emacs.d/emacs.el
some things will break, and same thing if it swaps the order.


        Stefan

RE: Settings

[Drew Adams]

> > As a first principle, I think Emacs should not mess, at all, with the
> > ~/.emacs file.  That's for users to put their lovingly hand-crafted
> > setqs in.
> 
> I don't have a strong opinion on that, but I think it's important to
> realize that depending on the circumstances, the custom-set-variables
> call may have to come before some of the user's config as well as it
> may
> also have to come after some of the user's config.
> 
> IOW if Emacs just loads ~/.emacs.d/custom.el before ~/.emacs.d/emacs.el
> some things will break, and same thing if it swaps the order.

Definitely.  The doc should tell users that they
need to decide when/where in their init files to
load `custom-file'.

Sometimes it can even be preferable to load it
more than once, e.g., to pick up stuff from some
loaded libraries and then to let some later-loaded
libraries pick up stuff from `custom-file' (or
vice versa).

The most important point for users is that by
using `custom-file' they can prevent Customize
from fiddling with their init file, and they
can prevent accidental fiddling with Customize
settings when they edit their init file.

But a secondarily important point is that when
`custom-file' gets loaded can be important.

Re: Settings

[Stefan Monnier]

>> IOW if Emacs just loads ~/.emacs.d/custom.el before ~/.emacs.d/emacs.el
>> some things will break, and same thing if it swaps the order.
> Definitely.  The doc should tell users that they
> need to decide when/where in their init files to
> load `custom-file'.

Maybe another option is to provide a command

    M-x keep-your-fingers-off-my-.emacs

which moves the custom-set-variables to a new file and replaces it with
a load and a setq of custom-file.


        Stefan

Re: Settings

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I don't have a strong opinion on that, but I think it's important to
> realize that depending on the circumstances, the custom-set-variables
> call may have to come before some of the user's config as well as it may
> also have to come after some of the user's config.
>
> IOW if Emacs just loads ~/.emacs.d/custom.el before ~/.emacs.d/emacs.el
> some things will break, and same thing if it swaps the order.

Yes, the currently kinda-not-defined evaluation order of the custom
stuff is also painful to deal with.  By having it outside of .emacs, we
could provide a well-defined evaluation order.  (That could be
controlled by the user, or course.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Settings

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > IOW if Emacs just loads ~/.emacs.d/custom.el before ~/.emacs.d/emacs.el
  > some things will break, and same thing if it swaps the order.

I wonder if we can do something more intelligent about this.
For instance, can we figure out that certain variables need to be set early
and others late?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

RE: Settings

[Drew Adams]

>   > IOW if Emacs just loads ~/.emacs.d/custom.el before
> ~/.emacs.d/emacs.el
>   > some things will break, and same thing if it swaps the order.
> 
> I wonder if we can do something more intelligent about this.
> For instance, can we figure out that certain variables need to be set
> early and others late?

Whatever might be figured out, IMO it needs to be possible
for a user to control when `custom-file' (it's value) gets
loaded.

IOW, maybe some intelligent default handling can be
contrived.  But a user who needs to load `custom-file'
at one (or more!) specific points in her init file should
always be able to do so.

Re: Settings

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Whatever might be figured out, IMO it needs to be possible
  > for a user to control when `custom-file' (its value) gets
  > loaded.

My point is that maybe we don't want to process all custom
definitions in one single batch.  If we can identify specific
needs with regard to particular custom variables, we could
do something more sophisticated, and MAYBE we could make that
work right for every user without requiring that the user
figure out a solution that works right for all of them.


-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Settings

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> My point is that maybe we don't want to process all custom
> definitions in one single batch.  If we can identify specific
> needs with regard to particular custom variables, we could
> do something more sophisticated, and MAYBE we could make that
> work right for every user without requiring that the user
> figure out a solution that works right for all of them.

It's possible, but I think that sounds...  very ambitious.  But I could
see the case for saying something general like "customising hook
variables should happen before loading .emacs, because that will make
add-hook work", but even that would be controversial.  (Customising a
hook sets the variable to a specific value; it doesn't add functions.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Settings

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It's possible, but I think that sounds...  very ambitious.  But I could
  > see the case for saying something general like "customising hook
  > variables should happen before loading .emacs, because that will make
  > add-hook work", but even that would be controversial.  (Customising a
  > hook sets the variable to a specific value; it doesn't add functions.)'

I think that single change would help some users.

A collection of customizations can be overconstrained so that no time
is a good time to install them all.  If we can move some variables
automatically to the right time for them, that may make it possible to
find a good point to load the rest of the customizations.

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Settings

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Mon, 26 Aug 2019 19:13:30 -0400
> Cc: larsi@gnus.org, emacs-devel@gnu.org
> 
>   > IOW if Emacs just loads ~/.emacs.d/custom.el before ~/.emacs.d/emacs.el
>   > some things will break, and same thing if it swaps the order.
> 
> I wonder if we can do something more intelligent about this.
> For instance, can we figure out that certain variables need to be set early
> and others late?

What is "early" and what is "late" depends on the order of user
customizations in .emacs, which is entirely out of our control, and
can be almost arbitrary.

We introduced the early-init file in Emacs 27 to let users themselves
control which customizations are done early and which later, because
some customizations should be in place before we read .emacs.