bug#583: Use XDG basedir spec for configuration files?

bug#583: Use XDG basedir spec for configuration files?

[Fernando]

[-- Attachment #1: Type: text/plain, Size: 1886 bytes --]

The XDG Base Directory
Specification<http://standards.freedesktop.org/basedir-spec/>provides
some common paths in user's home to store personal application
configurations, or, using their own words "*defines where these files should
be looked for by defining one or more base directories relative to which
files should be located*".

XDG Base Directory Specification allows efficient backup, in which you can
easily choose to backup your data and/or your configuration files for
instance. An application could easily propose such backup for the whole
system if all applications were matching these specification.

Therefore, although I think that this bug/enhancement is not vital, it would
be great for Emacs to be "FreeDesktop XDG Base Directory Specification"
compliant.

Currently Emacs is using "$HOME/.emacs.d/". This is what XDG basedir spec
defines:

   - $XDG_DATA_HOME (usually $HOME/.local/share/) as "*the base directory
   relative to which user specific data files should be stored*"
   - $XDG_CONFIG_HOME (usually $HOME/.config/) as "*the base directory
   relative to which user specific configuration files should be stored*"
   - $XDG_CACHE_HOME (usually $HOME/.cache/) as "*the base directory
   relative to which user specific non-essential data files should be stored
   *"

(http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html)

In order to make Emacs XDG basedir compliant, I think that it would be nice
to use:

   - $XDG_CONFIG_HOME/emacs  ;;for "init.el" and storing customize
   configuration


   - $XDG_DATA_HOME/emacs  ;;for sessions and backups


   - $XDG_CACHE_HOME/emacs  ;;for cache files

Or maybe some other distribution. What do you think about it?

There's a small C library that may be useful check it out, written for the
sole purpose of xdg basedir spec compliance:
https://n.ethz.ch/student/nevillm/download/libxdg-basedir/

[-- Attachment #2: Type: text/html, Size: 3214 bytes --]

bug#583: XDG basedir specification

[Eric Heintzmann]

What is planned now about FreeDesktop XDG basedir specification for Emacs?


http://ploum.net/post/207-modify-your-application-to-use-xdg-folders
https://live.gnome.org/GnomeGoals/XDGConfigFolders
http://standards.freedesktop.org/basedir-spec/latest/

bug#583: XDG basedir specification

[Jan Djärv]

8 dec 2012 kl. 13:18 skrev Eric Heintzmann <Heintzmann.Eric@free.fr>:

> What is planned now about FreeDesktop XDG basedir specification for Emacs?
> 
> 
> http://ploum.net/post/207-modify-your-application-to-use-xdg-folders
> https://live.gnome.org/GnomeGoals/XDGConfigFolders
> http://standards.freedesktop.org/basedir-spec/latest/

Hopefully nothing, as the free desktop specs in this area keeps changing and never reach a stable state.

	Jan D.

bug#583: XDG basedir specification

[Eli Zaretskii]

> From: Jan Djärv <jan.h.d@swipnet.se>
> Date: Sat, 8 Dec 2012 20:06:34 +0100
> Cc: 583@debbugs.gnu.org
> 
> 8 dec 2012 kl. 13:18 skrev Eric Heintzmann <Heintzmann.Eric@free.fr>:
> 
> > What is planned now about FreeDesktop XDG basedir specification for Emacs?
> > 
> > 
> > http://ploum.net/post/207-modify-your-application-to-use-xdg-folders
> > https://live.gnome.org/GnomeGoals/XDGConfigFolders
> > http://standards.freedesktop.org/basedir-spec/latest/
> 
> Hopefully nothing, as the free desktop specs in this area keeps changing and never reach a stable state.

I agree, FWIW.

bug#583: Use XDG basedir spec for configuration files?

[Paul Eggert]

After discussion we added something along the suggested lines to Emacs master here:

https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=4118297ae2fab4886b20d193ba511a229637aea3

so I am closing bug report number 583.

bug#583: Use XDG basedir spec for configuration files?

[Glenn Morris]

It's great to finally see some progress in this area, but IIUC this is
still not really following the XDG spec. For example, I think a proper
solution would see user elpa packages stored undo XDG_DATA_HOME, etc.
I think the OP points this out.
It looks like what we have at the moment is basically
~/.emacs.d/ -> ~/.config/emacs/?

bug#583: Use XDG basedir spec for configuration files?

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Wed, 28 Aug 2019 12:11:21 -0400
> Cc: ferkiwi+a@gmail.com, eggert@cs.ucla.edu
> 
> It's great to finally see some progress in this area, but IIUC this is
> still not really following the XDG spec. For example, I think a proper
> solution would see user elpa packages stored undo XDG_DATA_HOME, etc.

Yes, because we don't set user-emacs-directory from XDG_DATA_HOME.  We
only load the init files from there.

bug#583: Use XDG basedir spec for configuration files?

[Glenn Morris]

Eli Zaretskii wrote:

>> It's great to finally see some progress in this area, but IIUC this is
>> still not really following the XDG spec. For example, I think a proper
>> solution would see user elpa packages stored undo XDG_DATA_HOME, etc.
>
> Yes, because we don't set user-emacs-directory from XDG_DATA_HOME.  We
> only load the init files from there.

I'm not sure I've made my point.
With a proper following of the XDG spec, the concept of (a single)
"user-emacs-directory" would no longer be present. Obviously this
requires more radical changes.

bug#583: Use XDG basedir spec for configuration files?

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 583@debbugs.gnu.org,  ferkiwi+a@gmail.com,  eggert@cs.ucla.edu
> Date: Wed, 28 Aug 2019 12:50:46 -0400
> 
> With a proper following of the XDG spec, the concept of (a single)
> "user-emacs-directory" would no longer be present.

I'm not sure I understand why.  Can you tell more?  (I know very
little about XDG.)

bug#583: Use XDG basedir spec for configuration files?

[Glenn Morris]

Eli Zaretskii wrote:

>> From: Glenn Morris <rgm@gnu.org>
>> Cc: 583@debbugs.gnu.org,  ferkiwi+a@gmail.com,  eggert@cs.ucla.edu
>> Date: Wed, 28 Aug 2019 12:50:46 -0400
>> 
>> With a proper following of the XDG spec, the concept of (a single)
>> "user-emacs-directory" would no longer be present.
>
> I'm not sure I understand why.  Can you tell more?  (I know very
> little about XDG.)

It's well documented, the link in the OP still works.
See also eg https://wiki.archlinux.org/index.php/XDG_Base_Directory

IMO a true conversion of Emacs to the XDG spec would involve the
addition of new variables (user-config-directory, user-data-directory,
user-cache-directory, etc). Each use of user-emacs-directory would have
to be reviewed to see where it actually belongs. For some, it would
probably be ambiguous.

bug#583: Use XDG basedir spec for configuration files?

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 583@debbugs.gnu.org,  ferkiwi+a@gmail.com,  eggert@cs.ucla.edu
> Date: Wed, 28 Aug 2019 22:17:55 -0400
> 
> IMO a true conversion of Emacs to the XDG spec would involve the
> addition of new variables (user-config-directory, user-data-directory,
> user-cache-directory, etc). Each use of user-emacs-directory would have
> to be reviewed to see where it actually belongs. For some, it would
> probably be ambiguous.

I'm afraid almost all of them will be ambiguous.  Moreover, defining a
clear enough set of rules for where to put files that will be added in
the future is a non-trivial job, and enforcing it will not be easy.  I
predict confusion and arguments.

My gut feeling is that Emacs should have one, at most 2 XDG-derived
directories, certainly not 3.  And if we want to support that
correctly, these environment variables should be accessed much earlier
in the startup process, somewhere in init_callproc and friends,
because setting user-emacs-directory in startup is too late and
creates problems, as we saw already.

We've made a small step in that direction; if we want to continue that
way, someone should volunteer to do the job of defining what files
will go were.

bug#583: Use XDG basedir spec for configuration files?

[Paul Eggert]

Glenn Morris wrote:
> It's great to finally see some progress in this area, but IIUC this is
> still not really following the XDG spec. For example, I think a proper
> solution would see user elpa packages stored undo XDG_DATA_HOME, etc.
> I think the OP points this out.
> It looks like what we have at the moment is basically
> ~/.emacs.d/ -> ~/.config/emacs/?

Yes, that's what Emacs basically has in master now. I don't see where Fernando 
mentioned ELPA, though. And it's not clear that an ELPA package's files should 
be considered "user specific data files" (XDG_DATA_HOME) as opposed to "user 
specific configuration files" (XDG_CONFIG_HOME).

Looking at my own home directory for what other XDG-ish programs do, I see data 
files under ~/.local/share (XDG_DATA_FILES default): things like 
xorg/Xorg.0.log, gnome-shell/application_state, gvfs-metadata/home, and 
evolution/addressbook/system/contacts.db. Programs (if any) are few and far 
between. (Not that I use many plugins in any programs, so perhaps I'm not the 
best person to check this.)

For what it's worth, I found an old directory named '~/.gimp-2.8' and a newer 
one named '~/.config/GIMP/2.10' with similar structure. It appears that when 
GIMP made the transition to XDG style in GIMP version 2.9.2, it did what Emacs 
just did in master now: that is, GIMP moved all its stuff into ~/config/GIMP. 
Although I don't use GIMP plug-ins, the GIMP plug-ins directory moved too.

bug#583: Use XDG basedir spec for configuration files?

[Glenn Morris]

Paul Eggert wrote:

>> ~/.emacs.d/ -> ~/.config/emacs/?
>
> Yes, that's what Emacs basically has in master now. I don't see where
> Fernando mentioned ELPA, though.

He didn't. Elpa may not even have existed when this report was opened.
I was using it as an example.

> And it's not clear that an ELPA package's files should be considered
> "user specific data files" (XDG_DATA_HOME) as opposed to "user
> specific configuration files" (XDG_CONFIG_HOME).

It seems clear to me. They are data, not configuration. You would not
install them under /etc, but somewhere like /usr/share.

> For what it's worth, I found an old directory named '~/.gimp-2.8' and
> a newer one named '~/.config/GIMP/2.10' with similar structure. It
> appears that when GIMP made the transition to XDG style in GIMP
> version 2.9.2, it did what Emacs just did in master now: that is, GIMP
> moved all its stuff into ~/config/GIMP. Although I don't use GIMP
> plug-ins, the GIMP plug-ins directory moved too.

IMO "dump everything under ~/.config/emacs" isn't a proper
implementation, even if some other packages have done this.

bug#583: Use XDG basedir spec for configuration files?

[Paul Eggert]

Glenn Morris wrote:
>> And it's not clear that an ELPA package's files should be considered
>> "user specific data files" (XDG_DATA_HOME) as opposed to "user
>> specific configuration files" (XDG_CONFIG_HOME).

> It seems clear to me. They are data, not configuration. You would not
> install them under /etc, but somewhere like /usr/share.

If I understand correctly, ELPA packages are not really either "data" or 
"configuration": they're software packages. And the XDG scheme doesn't appear to 
be designed for installing software packages: it's designed only for user 
preferences (aka configuration), user data, and information cached for the user.

If ELPA packages are just local copies from a server somewhere, it seems the 
most plausible place for them is the XDG cache (XDG_CACHE_HOME, which is 
~/.cache by default) rather than either in "data" or "configuration"; only the 
list of downloaded packages should be placed in XDG_CONFIG_HOME. Presumably the 
ELPA package manager could arrange for this.

Not being an expert in these matters, I looked at another popular packaging 
scheme: Flatpak. It appears to put everything under ~/.local/share, i.e., under 
XDG_DATA_HOME. This includes configuration. See 
<https://github.com/flatpak/flatpak/wiki/Filesystem>.

It's quite a mess, huh?

bug#583: Use XDG basedir spec for configuration files?

[Štěpán Němec]

On Wed, 28 Aug 2019 23:22:35 -0700
Paul Eggert wrote:

> If ELPA packages are just local copies from a server somewhere, it
> seems the most plausible place for them is the XDG cache
> (XDG_CACHE_HOME, which is ~/.cache by default) rather than either in
> "data" or "configuration"; only the list of downloaded packages should
> be placed in XDG_CONFIG_HOME. Presumably the ELPA package manager
> could arrange for this.

XDG_CACHE_HOME is for "non-essential data files" (quoting the spec).
Some people put it on tmpfs, so it is deleted on system restart.

> Not being an expert in these matters, I looked at another popular
> packaging scheme: Flatpak. It appears to put everything under
> ~/.local/share, i.e., under XDG_DATA_HOME. This includes
> configuration. See
> <https://github.com/flatpak/flatpak/wiki/Filesystem>.

Some programs (notably Python[1]) use ~/.local/lib/, which makes sense,
but is not part of the XDG spec.

> It's quite a mess, huh?

It is.

[1] https://www.python.org/dev/peps/pep-0370/

-- 
Štěpán

bug#583: Use XDG basedir spec for configuration files?

[Glenn Morris]

BTW, I think the handling of user-emacs-directory is buggy:

rm -rf /tmp/foo
mkdir -p /tmp/foo/.config/emacs
HOME=/tmp/foo emacs

Now:
user-init-file         -> /tmp/foo/.config/emacs/init
user-emacs-directory   -> ~/.emacs.d/

I guess user-emacs-directory is being set at build time, not run time.

bug#583: Use XDG basedir spec for configuration files?

[Glenn Morris]

PS I should have prefaced this recipe with "having built Emacs in an
environment where ~/.config/emacs does not exist".

bug#583: Use XDG basedir spec for configuration files?

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Thu, 29 Aug 2019 14:30:54 -0400
> Cc: ferkiwi@gmail.com, 583@debbugs.gnu.org
> 
> rm -rf /tmp/foo
> mkdir -p /tmp/foo/.config/emacs
> HOME=/tmp/foo emacs
> 
> Now:
> user-init-file         -> /tmp/foo/.config/emacs/init
> user-emacs-directory   -> ~/.emacs.d/
> 
> I guess user-emacs-directory is being set at build time, not run time.

It's a defconst in subr.el, and I believe one of my review comments
for the initial version of the patch was that I didn't think using
defconst for this was correct.

bug#583: Use XDG basedir spec for configuration files?

[Paul Eggert]

Glenn Morris wrote:
> BTW, I think the handling of user-emacs-directory is buggy:

Yes, it is. I'll have to take a look at that. I guess we'll have to change 
user-home-directory to a variable, since we can't easily determine it until 
command-line runs.

Also, a good many places in the documentaiton refer to ~/.emacs.d and these need 
to be updated.

bug#583: Use XDG basedir spec for configuration files?

[Glenn Morris]

Paul Eggert wrote:

>> BTW, I think the handling of user-emacs-directory is buggy:
>
> Yes, it is. I'll have to take a look at that. I guess we'll have to
> change user-home-directory to a variable, since we can't easily
> determine it until command-line runs.
>
> Also, a good many places in the documentaiton refer to ~/.emacs.d and
> these need to be updated.

IMO this means one should allow a user-defined user-emacs-directory,
which means un-wontfix'ing https://debbugs.gnu.org/15539 .
Maybe the patch in https://debbugs.gnu.org/15539#40 contains some useful ideas.
Although in hindsight an environment variable doesn't seem like the way
to go.

bug#583: Use XDG basedir spec for configuration files?

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Fri, 30 Aug 2019 12:18:40 -0400
> Cc: ferkiwi@gmail.com, 583@debbugs.gnu.org
> 
> Although in hindsight an environment variable doesn't seem like the way
> to go.

Actually, I think it does, because otherwise the user-defined value
will be noticed too late, after defcustom's that depend on it were
already recomputed.

bug#15539: [PATCH] Setting user-emacs-directory

[Paul Eggert]

Glenn Morris mentioned Bug#15539's user-emacs-directory issue here:

https://bugs.gnu.org/583#68

... so this email is giving you a heads-up that I recently installed a patch 
into GNU Emacs master that should allow something approximating Bug#15539's 
requested behavior. For example, one can run Emacs like this:

XDG_CONFIG_HOME=/home/rms/.config emacs

to employ user rms's Emacs configuration, instead of your configuration.

The patch I installed did not do anything special to fix bug#15539, as the new 
Emacs behavior is a natural fallout of supporting the XDG spec's requirements 
for XDG_CONFIG_HOME.

This all assumes people are using the XDG-conforming locations for config files, 
which I hope is good enough since older Emacs obviously wasn't meeting the 
Bug#15539 need anyway.

bug#15539: [PATCH] Setting user-emacs-directory

[Glenn Morris]

Paul Eggert wrote:

> ... so this email is giving you a heads-up that I recently installed a
> patch into GNU Emacs master that should allow something approximating
> Bug#15539's requested behavior. For example, one can run Emacs like
> this:
>
> XDG_CONFIG_HOME=/home/rms/.config emacs
>
> to employ user rms's Emacs configuration, instead of your configuration.

This is an improvement, but it is not a proper solution to #15539, for
some of the same reasons that "just change HOME" wasn't. XDG_CONFIG_HOME
is not an Emacs-specific variable, so changing it will impact other
things that may be called from within Emacs.

bug#15539: [PATCH] Setting user-emacs-directory

[Paul Eggert]

Glenn Morris wrote:
> XDG_CONFIG_HOME
> is not an Emacs-specific variable, so changing it will impact other
> things that may be called from within Emacs.

True, but it should suffice for many use cases mentioned in Bug#15539. Of the 
cases mentioned here:

https://bugs.gnu.org/15539#66

XDG_HOME should suffice for most of issues mentioned. Not all: for example, 
overriding XDG_CONFIG_HOME overrides Gtk configuration, font configuration, and 
other settings covered by the XDG convention, whereas one might want to override 
just ~/.config/emacs. But it strikes me that this is often just as much of a 
feature as a drawback, since Gtk etc. configurations influence Emacs so much 
that they really ought to be saved/restored when one is thinking of 
saving/restoring Emacs settings.

One might have a situation where one wants to vary (or save) files traditionally 
kept in ~/.emacs.d/, and no other files. But it's also quite plausible that one 
will want some of those files and not others, just as one might want 
~/.config/emacs and ~/.config/fontconfig but not ~/.config/gtk-3.0. And it's 
unlikely that a single option will have all the fine-grained control that one 
would need for all these situations. Instead, it's probably better just to 
suggest to people that they set up $HOME (or perhaps $XDG_CONFIG_HOME) to point 
to a directory full of configurations that are just they way they like it.

bug#15539: [PATCH] Setting user-emacs-directory

[Mike Carifio]

As the OP approximately six years ago, I neglected to add the "use-case" 
I wanted --user-emacs-directory for, namely to experiment with 
~/.emacs.d/init.el including installing new packages via elpa. I still 
think there's a need, especially as the number of useful packages have 
grown and they can interact. I currently use the "symlink" approach when 
I want to start adding new packages and configurations. It's useful to 
return to a "known good state" if my init.el hacking careens off the 
tracks. Since there was already a _variable_ user-emacs-directory, I 
thought I was just asking to set it early at the command line. It seemed 
to be analogous to --load for an emacs file or --user for another user's 
init file. I didn't realize it had a "read-only" flavor.

Yes, this switch adds another knob, but I happen to think its a useful 
knob and is consistent with --load and --user, both of which allow the 
user to designate a different init file at the command line. Redefining 
HOME at the command line and then "setting it back" inside init.el seems 
convoluted. It could also potentially break site-start.el if some code 
there relied on the right binding of HOME. Admittedly that's a 
farfetched scenario, but not impossible either. Sure would be confusing 
to debug if you didn't know what to look for. All the "do it yourself" 
strategies (other than symlink) also force the user to deeply understand 
the details of the init process, e.g. what switches to throw to override 
various features. So if the criticism is "yet another knob" I would say 
you are pushing people to construct homegrown solutions ... repeatedly.

The XDG patch will let emacs adhere to the XDG desktop conventions and 
you can designate the user-emacs-directory implicitly as well, a 
two-for-one special. Not every platform follows the XDG conventions, but 
I personally mostly use linux, so I'm less concerned with those.

bug#15539: [PATCH] Setting user-emacs-directory

[Lars Ingebrigtsen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> One might have a situation where one wants to vary (or save) files
> traditionally kept in ~/.emacs.d/, and no other files. But it's also
> quite plausible that one will want some of those files and not others,
> just as one might want ~/.config/emacs and ~/.config/fontconfig but
> not ~/.config/gtk-3.0. And it's unlikely that a single option will
> have all the fine-grained control that one would need for all these
> situations. Instead, it's probably better just to suggest to people
> that they set up $HOME (or perhaps $XDG_CONFIG_HOME) to point to a
> directory full of configurations that are just they way they like it.

So I think the conclusion to this long thread was that we don't want to
add a specific switch for this, and instead people can say
"XDG_CONFIG_HOME=/whatever emacs" when they want to change these paths.
So I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#583: Use XDG basedir spec for configuration files?

[Paul Eggert]

Glenn Morris wrote:
> IMO this means one should allow a user-defined user-emacs-directory,
> which means un-wontfix'ing https://debbugs.gnu.org/15539  .

Thanks for mentioning that. I followed up in that bug report, here:

https://bugs.gnu.org/15539#109

By the way, making user-emacs-directory a variable instead of a constant has its 
downsides, because if a user changes user-emacs-directory, then a lot of other 
variables will keep their old values derived from the previous value of 
user-emacs-directory, leading to a confusing mish-mosh of old and new behavior. 
Changing user-emacs-directory works coherently only if it's done very early in 
startup (which is all that Bug#15539 was asking for).

bug#583: Use XDG basedir spec for configuration files?

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sat, 31 Aug 2019 19:02:01 -0700
> Cc: 583@debbugs.gnu.org
> 
> Changing user-emacs-directory works coherently only if it's done very early in 
> startup (which is all that Bug#15539 was asking for).

We should document what that "very early" means in practice.  Does it
mean in the early-init file? or can this be done reliably in the
normal init file as well?

The problem here is that some defcustom's refer to
user-emacs-directory, and so changing the value of
user-emacs-directory after those defcustoms were computed during
startup will leave those defcustoms pointing at the old place.

bug#583: Use XDG basedir spec for configuration files?

[Paul Eggert]

Eli Zaretskii wrote:
> We should document what that "very early" means in practice.  Does it
> mean in the early-init file? or can this be done reliably in the
> normal init file as well?

Sorry, I don't know. That's why I didn't document it.

> The problem here is that some defcustom's refer to
> user-emacs-directory, and so changing the value of
> user-emacs-directory after those defcustoms were computed during
> startup will leave those defcustoms pointing at the old place.

Yes, it's quite a mess.

bug#583: Use XDG basedir spec for configuration files?

[Eli Zaretskii]

> Cc: rgm@gnu.org, 583@debbugs.gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sun, 1 Sep 2019 11:40:21 -0700
> 
> Eli Zaretskii wrote:
> > We should document what that "very early" means in practice.  Does it
> > mean in the early-init file? or can this be done reliably in the
> > normal init file as well?
> 
> Sorry, I don't know. That's why I didn't document it.
> 
> > The problem here is that some defcustom's refer to
> > user-emacs-directory, and so changing the value of
> > user-emacs-directory after those defcustoms were computed during
> > startup will leave those defcustoms pointing at the old place.
> 
> Yes, it's quite a mess.

If doing this in early-init works, I'd rather we documented that as
the canonical place that saves users from the mess.

bug#583: Use XDG basedir spec for configuration files?

[Paul Eggert]

Eli Zaretskii wrote:
>> Yes, it's quite a mess.
> If doing this in early-init works, I'd rather we documented that as
> the canonical place that saves users from the mess.

I doubt whether it even works in early-init. Not that I've tested it; I don't 
use early-init and would not be a good person to delve deeper into that part of 
the mess.

bug#583: Use XDG basedir spec for configuration files?

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

Glenn Morris wrote:
> I guess user-emacs-directory is being set at build time, not run time.

Thanks for reporting that. I installed the attached to fix it. I plan to follow 
up soon on the other issues raised here.

[-- Attachment #2: 0001-Calculate-user-emacs-directory-on-startup.patch --]
[-- Type: text/x-patch, Size: 4737 bytes --]

From 2befb4f0a1494f699f56215d5f28ba055663d881 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sat, 31 Aug 2019 14:47:04 -0700
Subject: [PATCH] Calculate user-emacs-directory on startup

Problem reported by Glenn Morris (Bug#583#56).
* lisp/startup.el (startup--xdg-config-default): New constant.
(startup--xdg-config-home-emacs): New var.
(startup--xdg-or-homedot): New function.
(normal-top-level): Use it to set user-emacs-directory early on.
(command-line): Also use it to determine the startup init directory.
* lisp/subr.el (user-emacs-directory): Just initialize to nil.
---
 lisp/startup.el | 51 +++++++++++++++++++++++++++++++++++++------------
 lisp/subr.el    | 14 ++------------
 2 files changed, 41 insertions(+), 24 deletions(-)

diff --git a/lisp/startup.el b/lisp/startup.el
index c1e429b8db..a16db242da 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -490,6 +490,27 @@ normal-top-level-add-to-load-path
     (when tail
       (setcdr tail (append (mapcar 'expand-file-name dirs) (cdr tail))))))
 
+;; The default location for XDG-convention Emacs init files.
+(defconst startup--xdg-config-default "~/.config/emacs/")
+;; The location for XDG-convention Emacs init files.
+(defvar startup--xdg-config-home-emacs)
+
+;; Return the name of the init file directory for Emacs, assuming
+;; XDG-DIR is the XDG location and USER-NAME is the user name.
+;; If USER-NAME is nil or "", use the current user.
+;; Prefer the XDG location unless it does does not exist and the
+;; .emacs.d location does exist.
+(defun startup--xdg-or-homedot (xdg-dir user-name)
+  (if (file-exists-p xdg-dir)
+      xdg-dir
+    (let ((emacs-d-dir (concat "~" user-name
+			       (if (eq system-type 'ms-dos)
+				   "/_emacs.d/"
+				 "/.emacs.d/"))))
+      (if (file-exists-p emacs-d-dir)
+	  emacs-d-dir
+	xdg-dir))))
+
 (defun normal-top-level ()
   "Emacs calls this function when it first starts up.
 It sets `command-line-processed', processes the command-line,
@@ -499,6 +520,14 @@ normal-top-level
       (message internal--top-level-message)
     (setq command-line-processed t)
 
+    (setq startup--xdg-config-home-emacs
+	  (let ((xdg-config-home (getenv-internal "XDG_CONFIG_HOME")))
+	    (if xdg-config-home
+		(concat xdg-config-home "/emacs/")
+	      startup--xdg-config-default)))
+    (setq user-emacs-directory
+	  (startup--xdg-or-homedot startup--xdg-config-home-emacs nil))
+
     ;; Look in each dir in load-path for a subdirs.el file.  If we
     ;; find one, load it, which will add the appropriate subdirs of
     ;; that dir into load-path.  This needs to be done before setting
@@ -1167,19 +1196,17 @@ command-line
                          :error))))
 
   ;; Calculate the name of the Emacs init directory.
-  ;; This is typically equivalent to ~/.config/emacs if the user is
-  ;; following the XDG convention, and is ~INIT-FILE-USER/.emacs.d
-  ;; on other systems.
-  (setq xdg-dir (concat (or (getenv "XDG_CONFIG_HOME")
-			    (concat "~" init-file-user "/.config"))
-			"/emacs/"))
+  ;; This is typically ~INIT-FILE-USER/.config/emacs unless the user
+  ;; is following the ~INIT-FILE-USER/.emacs.d convention.
+  (setq xdg-dir startup--xdg-config-home-emacs)
   (setq startup-init-directory
-	(if (file-exists-p xdg-dir)
-	    xdg-dir
-	  (let ((emacs-d-dir (concat "~" init-file-user "/.emacs.d/")))
-	    (if (file-exists-p emacs-d-dir)
-		emacs-d-dir
-	      xdg-dir))))
+	(if (or (zerop (length init-file-user))
+		(and (eq xdg-dir user-emacs-directory)
+		     (not (eq xdg-dir startup--xdg-config-default))))
+	    user-emacs-directory
+	  ;; The name is not obvious, so access more directories to calculate it.
+	  (setq xdg-dir (concat "~" init-file-user "/.config/emacs/"))
+	  (startup--xdg-or-homedot xdg-dir init-file-user)))
 
   ;; Load the early init file, if found.
   (startup--load-user-init-file
diff --git a/lisp/subr.el b/lisp/subr.el
index 566a3fc758..cf6fb108e9 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -2938,18 +2938,8 @@ temp-buffer-setup-hook
 mode.")
 
 (defconst user-emacs-directory
-  (let ((config-dir (concat (or (getenv-internal "XDG_CONFIG_HOME")
-				"~/.config")
-			    "/emacs/")))
-    (if (file-exists-p config-dir)
-	config-dir
-      (let ((emacs-d-dir (if (eq system-type 'ms-dos)
-			     ;; MS-DOS cannot have initial dot.
-			     "~/_emacs.d/"
-			   "~/.emacs.d/")))
-	(if (file-exists-p emacs-d-dir)
-	    emacs-d-dir
-	  config-dir))))
+  ;; The value does not matter since Emacs sets this at startup.
+  nil
   "Directory beneath which additional per-user Emacs-specific files are placed.
 Various programs in Emacs store information in this directory.
 Note that this should end with a directory separator.
-- 
2.17.1

bug#583: Use XDG basedir spec for configuration files?

[Sven Joachim]

On 2019-08-31 14:51 -0700, Paul Eggert wrote:

> Glenn Morris wrote:
>> I guess user-emacs-directory is being set at build time, not run time.
>
> Thanks for reporting that. I installed the attached to fix it. I plan
> to follow up soon on the other issues raised here.

FWIW, this patch broke auto-save-list-file-prefix, at least when
user-emacs-directory is "~/.emacs.d/".  See bug #37354.

,----
| auto-save-list-file-prefix is a variable defined in ‘startup.el’.
| Its value is "auto-save-list/.saves-"
| Original value was
| "~/.emacs.d/auto-save-list/.saves-"
`----

Cheers,
       Sven


> From 2befb4f0a1494f699f56215d5f28ba055663d881 Mon Sep 17 00:00:00 2001
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sat, 31 Aug 2019 14:47:04 -0700
> Subject: [PATCH] Calculate user-emacs-directory on startup
>
> Problem reported by Glenn Morris (Bug#583#56).
> * lisp/startup.el (startup--xdg-config-default): New constant.
> (startup--xdg-config-home-emacs): New var.
> (startup--xdg-or-homedot): New function.
> (normal-top-level): Use it to set user-emacs-directory early on.
> (command-line): Also use it to determine the startup init directory.
> * lisp/subr.el (user-emacs-directory): Just initialize to nil.
> ---
>  lisp/startup.el | 51 +++++++++++++++++++++++++++++++++++++------------
>  lisp/subr.el    | 14 ++------------
>  2 files changed, 41 insertions(+), 24 deletions(-)
>
> diff --git a/lisp/startup.el b/lisp/startup.el
> index c1e429b8db..a16db242da 100644
> --- a/lisp/startup.el
> +++ b/lisp/startup.el
> @@ -490,6 +490,27 @@ normal-top-level-add-to-load-path
>      (when tail
>        (setcdr tail (append (mapcar 'expand-file-name dirs) (cdr tail))))))
>  
> +;; The default location for XDG-convention Emacs init files.
> +(defconst startup--xdg-config-default "~/.config/emacs/")
> +;; The location for XDG-convention Emacs init files.
> +(defvar startup--xdg-config-home-emacs)
> +
> +;; Return the name of the init file directory for Emacs, assuming
> +;; XDG-DIR is the XDG location and USER-NAME is the user name.
> +;; If USER-NAME is nil or "", use the current user.
> +;; Prefer the XDG location unless it does does not exist and the
> +;; .emacs.d location does exist.
> +(defun startup--xdg-or-homedot (xdg-dir user-name)
> +  (if (file-exists-p xdg-dir)
> +      xdg-dir
> +    (let ((emacs-d-dir (concat "~" user-name
> +			       (if (eq system-type 'ms-dos)
> +				   "/_emacs.d/"
> +				 "/.emacs.d/"))))
> +      (if (file-exists-p emacs-d-dir)
> +	  emacs-d-dir
> +	xdg-dir))))
> +
>  (defun normal-top-level ()
>    "Emacs calls this function when it first starts up.
>  It sets `command-line-processed', processes the command-line,
> @@ -499,6 +520,14 @@ normal-top-level
>        (message internal--top-level-message)
>      (setq command-line-processed t)
>  
> +    (setq startup--xdg-config-home-emacs
> +	  (let ((xdg-config-home (getenv-internal "XDG_CONFIG_HOME")))
> +	    (if xdg-config-home
> +		(concat xdg-config-home "/emacs/")
> +	      startup--xdg-config-default)))
> +    (setq user-emacs-directory
> +	  (startup--xdg-or-homedot startup--xdg-config-home-emacs nil))
> +
>      ;; Look in each dir in load-path for a subdirs.el file.  If we
>      ;; find one, load it, which will add the appropriate subdirs of
>      ;; that dir into load-path.  This needs to be done before setting
> @@ -1167,19 +1196,17 @@ command-line
>                           :error))))
>  
>    ;; Calculate the name of the Emacs init directory.
> -  ;; This is typically equivalent to ~/.config/emacs if the user is
> -  ;; following the XDG convention, and is ~INIT-FILE-USER/.emacs.d
> -  ;; on other systems.
> -  (setq xdg-dir (concat (or (getenv "XDG_CONFIG_HOME")
> -			    (concat "~" init-file-user "/.config"))
> -			"/emacs/"))
> +  ;; This is typically ~INIT-FILE-USER/.config/emacs unless the user
> +  ;; is following the ~INIT-FILE-USER/.emacs.d convention.
> +  (setq xdg-dir startup--xdg-config-home-emacs)
>    (setq startup-init-directory
> -	(if (file-exists-p xdg-dir)
> -	    xdg-dir
> -	  (let ((emacs-d-dir (concat "~" init-file-user "/.emacs.d/")))
> -	    (if (file-exists-p emacs-d-dir)
> -		emacs-d-dir
> -	      xdg-dir))))
> +	(if (or (zerop (length init-file-user))
> +		(and (eq xdg-dir user-emacs-directory)
> +		     (not (eq xdg-dir startup--xdg-config-default))))
> +	    user-emacs-directory
> +	  ;; The name is not obvious, so access more directories to calculate it.
> +	  (setq xdg-dir (concat "~" init-file-user "/.config/emacs/"))
> +	  (startup--xdg-or-homedot xdg-dir init-file-user)))
>  
>    ;; Load the early init file, if found.
>    (startup--load-user-init-file
> diff --git a/lisp/subr.el b/lisp/subr.el
> index 566a3fc758..cf6fb108e9 100644
> --- a/lisp/subr.el
> +++ b/lisp/subr.el
> @@ -2938,18 +2938,8 @@ temp-buffer-setup-hook
>  mode.")
>  
>  (defconst user-emacs-directory
> -  (let ((config-dir (concat (or (getenv-internal "XDG_CONFIG_HOME")
> -				"~/.config")
> -			    "/emacs/")))
> -    (if (file-exists-p config-dir)
> -	config-dir
> -      (let ((emacs-d-dir (if (eq system-type 'ms-dos)
> -			     ;; MS-DOS cannot have initial dot.
> -			     "~/_emacs.d/"
> -			   "~/.emacs.d/")))
> -	(if (file-exists-p emacs-d-dir)
> -	    emacs-d-dir
> -	  config-dir))))
> +  ;; The value does not matter since Emacs sets this at startup.
> +  nil
>    "Directory beneath which additional per-user Emacs-specific files are placed.
>  Various programs in Emacs store information in this directory.
>  Note that this should end with a directory separator.
> -- 
> 2.17.1