bug#37116: 26.1; User choice user-emacs-directory is not honored

bug#37116: 26.1; User choice user-emacs-directory is not honored

[bruno.vasselle--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1862 bytes --]

Although I start my init.el by setting `user-emacs-directory' to something that suits me, emacs keeps on creating an unwanted "~/.emacs.d/". 

How to reproduce: 

Create an init file, say "~/.config/emacs/please-do-it.el": 
(setq user-emacs-directory "~/.cache/emacs/") 


;; (setq auto-save-list-file-prefix ;; I shall not have to do this 
;; (concat user-emacs-directory "saves")) 


Do the following in a shell session: 
$ cd 
$ rm -r .emacs.d # actually there was one 
$ ls .emacs.d 
ls: cannot access '.emacs.d': No such file or directory 
$ emacs -l ~/.config/emacs/please-do-it.el & 
$ ls -d .emacs.d 
.emacs.d 
$ ls .emacs.d 
$ 

There's nothing in .emacs.d, but the directory is there. It should not, as this is not my choice as an emacs user, and the only way I've got to state this is setting `user-emacs-directory' in my init file. 

Let's edit some file with emacs and wait about 2 minutes: 
$ ls .emacs.d 
auto-save-list 
$ 

In emacs, describe variable `user-emacs-directory': this is actually what I want, namely "~/.cache/emacs/". 
Quit emacs 

Is the auto-save guilty ? If I uncomment the part related to auto-save in please-to-it.el: 

$ rm -r .emacs.d 
$ emacs -l ~/.config/emacs/please-do-it.el & 
$ ls -d .emacs.d 
.emacs.d 
$ 

Edit some file and wait about 2 minutes; then: 
$ ls .emacs.d 
$ 

Nothing in .emacs.d. Let's see what I asked for auto-save: 

$ ls .cache/emacs 
saves14634-xanadu~ 
$ 

Alright... but it's not legitimate: as I've set `user-emacs-directory', I should not have to furthermore tell auto-save where to put its material. If I need to, this means I need to also for every possible package that uses user-emacs-directory. And emacs still trashes my home directory. 

I did not find bug reports on the subject, except for this one, which is precisely on topic: 
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=15539 



[-- Attachment #2: Type: text/html, Size: 4861 bytes --]

bug#37116: 26.1; User choice user-emacs-directory is not honored

[Eli Zaretskii]

> Date: Tue, 20 Aug 2019 16:33:46 +0200 (CEST)
> From: bruno.vasselle--- via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Alright... but it's not legitimate: as I've set `user-emacs-directory', I should not have to furthermore tell
> auto-save where to put its material. If I need to, this means I need to also for every possible package that uses
> user-emacs-directory. And emacs still trashes my home directory.

Does the patch below solve your problem?

diff --git a/lisp/startup.el b/lisp/startup.el
index 32051c2..d753e6a 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -405,6 +405,7 @@ auto-save-list-file-prefix
 Directories in the prefix will be created if necessary.
 Set this to nil if you want to prevent `auto-save-list-file-name'
 from being initialized."
+  :initialize 'custom-initialize-delay
   :type '(choice (const :tag "Don't record a session's auto save list" nil)
 		 string)
   :group 'auto-save)

bug#37116: 26.1; User choice user-emacs-directory is not honored

[bruno.vasselle--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1883 bytes --]

Added (manually) to commit tagged emacs-26.1.92, in defcustom auto-save-list-file-prefix': 
:initialize 'custom-initialize-delay 

Done same tests with auto-save configuration commented out in "please-do-it.el". 
- The directory .emacs.d is created 
- It is populated with a directory auto-save-list-file-prefix 

Checked that emacs is actually the one I've just build: it is. 
Also checked that /usr/share/emacs/26.1.92/lisp/startup.el* are up to date: they are, though sibling .el are not affected except subdirs.el. 

Thus, no, it does not seem to work. 

Shall I try from another commit ? 

----- Mail original -----

De: "Eli Zaretskii" <eliz@gnu.org> 
À: "bruno vasselle" <bruno.vasselle@laposte.net> 
Cc: 37116@debbugs.gnu.org 
Envoyé: Mardi 20 Août 2019 18:55:50 
Objet: Re: bug#37116: 26.1; User choice user-emacs-directory is not honored 

> Date: Tue, 20 Aug 2019 16:33:46 +0200 (CEST) 
> From: bruno.vasselle--- via "Bug reports for GNU Emacs, 
> the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> 
> 
> Alright... but it's not legitimate: as I've set `user-emacs-directory', I should not have to furthermore tell 
> auto-save where to put its material. If I need to, this means I need to also for every possible package that uses 
> user-emacs-directory. And emacs still trashes my home directory. 

Does the patch below solve your problem? 

diff --git a/lisp/startup.el b/lisp/startup.el 
index 32051c2..d753e6a 100644 
--- a/lisp/startup.el 
+++ b/lisp/startup.el 
@@ -405,6 +405,7 @@ auto-save-list-file-prefix 
Directories in the prefix will be created if necessary. 
Set this to nil if you want to prevent `auto-save-list-file-name' 
from being initialized." 
+ :initialize 'custom-initialize-delay 
:type '(choice (const :tag "Don't record a session's auto save list" nil) 
string) 
:group 'auto-save) 


[-- Attachment #2: Type: text/html, Size: 2766 bytes --]

bug#37116: 26.1; User choice user-emacs-directory is not honored

[bruno.vasselle--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2465 bytes --]

Note: commit 32051c2 is not in my git tree... 
Note: tried from commit emacs-26.2: same result 
Note: tried from last commit in master (68f086): same result 



Typo: you've certainly corrected by yourself, but it is actually auto-save-list that is created under .emacs.d, not auto-save-list-file-prefix 

----- Mail original -----

De: "bruno vasselle" <bruno.vasselle@laposte.net> 
À: "Eli Zaretskii" <eliz@gnu.org> 
Cc: 37116@debbugs.gnu.org 
Envoyé: Mardi 20 Août 2019 20:50:47 
Objet: Re: bug#37116: 26.1; User choice user-emacs-directory is not honored 

Added (manually) to commit tagged emacs-26.1.92, in defcustom auto-save-list-file-prefix': 
:initialize 'custom-initialize-delay 

Done same tests with auto-save configuration commented out in "please-do-it.el". 
- The directory .emacs.d is created 
- It is populated with a directory auto-save-list-file-prefix 

Checked that emacs is actually the one I've just build: it is. 
Also checked that /usr/share/emacs/26.1.92/lisp/startup.el* are up to date: they are, though sibling .el are not affected except subdirs.el. 

Thus, no, it does not seem to work. 

Shall I try from another commit ? 

----- Mail original -----

De: "Eli Zaretskii" <eliz@gnu.org> 
À: "bruno vasselle" <bruno.vasselle@laposte.net> 
Cc: 37116@debbugs.gnu.org 
Envoyé: Mardi 20 Août 2019 18:55:50 
Objet: Re: bug#37116: 26.1; User choice user-emacs-directory is not honored 

> Date: Tue, 20 Aug 2019 16:33:46 +0200 (CEST) 
> From: bruno.vasselle--- via "Bug reports for GNU Emacs, 
> the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> 
> 
> Alright... but it's not legitimate: as I've set `user-emacs-directory', I should not have to furthermore tell 
> auto-save where to put its material. If I need to, this means I need to also for every possible package that uses 
> user-emacs-directory. And emacs still trashes my home directory. 

Does the patch below solve your problem? 

diff --git a/lisp/startup.el b/lisp/startup.el 
index 32051c2..d753e6a 100644 
--- a/lisp/startup.el 
+++ b/lisp/startup.el 
@@ -405,6 +405,7 @@ auto-save-list-file-prefix 
Directories in the prefix will be created if necessary. 
Set this to nil if you want to prevent `auto-save-list-file-name' 
from being initialized." 
+ :initialize 'custom-initialize-delay 
:type '(choice (const :tag "Don't record a session's auto save list" nil) 
string) 
:group 'auto-save) 



[-- Attachment #2: Type: text/html, Size: 3677 bytes --]

bug#37116: 26.1; User choice user-emacs-directory is not honored

[Eli Zaretskii]

> Date: Tue, 20 Aug 2019 22:11:32 +0200 (CEST)
> From: bruno.vasselle@laposte.net
> Cc: 37116@debbugs.gnu.org
> 
> Typo: you've certainly corrected by yourself, but it is actually auto-save-list that is created under .emacs.d,
> not auto-save-list-file-prefix

It's not a typo: auto-save-list-file-prefix is a defcustom from whose
value Emacs creates the name of the directory where it will save the
auto-save files.  The change was supposed to delay computing the value
of auto-save-list-file-prefix until after your init file is read, so
that it catches the new value of user-emacs-directory.

bug#37116: 26.1; User choice user-emacs-directory is not honored

[bruno.vasselle--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 354 bytes --]


> > Typo: you've certainly corrected by yourself, but it is actually auto-save-list that is created under .emacs.d, 
> > not auto-save-list-file-prefix 
> 
> It's not a typo: auto-save-list-file-prefix is a defcustom from whose 

I mean: the directory that is created under ".emacs.d/' is not named " auto-save-list-file-prefix", but " auto-save-list" 

[-- Attachment #2: Type: text/html, Size: 1734 bytes --]

bug#37116: 26.1; User choice user-emacs-directory is not honored

[Eli Zaretskii]

> Date: Wed, 21 Aug 2019 10:24:50 +0200 (CEST)
> From: bruno.vasselle@laposte.net
> Cc: 37116@debbugs.gnu.org
> 
> > > Typo: you've certainly corrected by yourself, but it is actually auto-save-list that is created under
> .emacs.d,
> > > not auto-save-list-file-prefix
> > 
> > It's not a typo: auto-save-list-file-prefix is a defcustom from whose
> 
> I mean: the directory that is created under ".emacs.d/' is not named "auto-save-list-file-prefix", but
> "auto-save-list"

Yes, auto-save-list-file-prefix is the variable whose value includes
the string "auto-save-list".  The problem is that the value of
auto-save-list-file-prefix is computed before user-emacs-directory is
updated by your init file.

Please try the following patch instead of the one I sent earlier:

diff --git a/lisp/startup.el b/lisp/startup.el
index 5644285..ff90646 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -406,6 +406,7 @@ auto-save-list-file-prefix
 Directories in the prefix will be created if necessary.
 Set this to nil if you want to prevent `auto-save-list-file-name'
 from being initialized."
+  :initialize 'custom-initialize-delay
   :type '(choice (const :tag "Don't record a session's auto save list" nil)
 		 string)
   :group 'auto-save)
@@ -1282,8 +1283,7 @@ command-line
   ;; depends on the runtime context, in case some of them depend on
   ;; the window-system features.  Example: blink-cursor-mode.
   (let (current-load-list) ; c-r-s may call defvar, and hence LOADHIST_ATTACH
-    (mapc 'custom-reevaluate-setting custom-delayed-init-variables)
-    (setq custom-delayed-init-variables nil))
+    (mapc 'custom-reevaluate-setting custom-delayed-init-variables))
 
   (normal-erase-is-backspace-setup-frame)
 
@@ -1377,6 +1377,14 @@ command-line
 		 (eq face-ignored-fonts old-face-ignored-fonts))
       (clear-face-cache)))
 
+  ;; Re-evaluate again the predefined variables whose initial value
+  ;; depends on the runtime context, in case the user init file
+  ;; modified user-emacs-directory.  Examples: abbrev-file-name,
+  ;; auto-save-list-file-prefix.
+  (let (current-load-list) ; c-r-s may call defvar, and hence LOADHIST_ATTACH
+    (mapc 'custom-reevaluate-setting custom-delayed-init-variables)
+    (setq custom-delayed-init-variables nil))
+
   (setq after-init-time (current-time))
   ;; Display any accumulated warnings after all functions in
   ;; `after-init-hook' like `desktop-read' have finalized possible

bug#37116: 26.1; User choice user-emacs-directory is not honored

[Eli Zaretskii]

[Please use Reply All so that the bug tracker is CC'ed and records the
discussion.]

> Date: Thu, 22 Aug 2019 18:18:17 +0200 (CEST)
> From: bruno.vasselle@laposte.net
> 
> > Please try the following patch instead of the one I sent earlier:
> Patched against master (ef8531d2): details below in 'git diff master'
> 
> Emacs still creates .emacs.d in my home dir.

This is expected, and cannot be helped, at least not easily, because
various variables are computed during the startup, when your init file
has not yet been processed.

But with the patch I sent the auto-save-history files should be in
your customized user-emacs-directory.  Did you check that?  If not,
please do.

> Furthermore, if I feed it with my actual 'init.el'  - which defines a user-init-file emacs "custom.el" can play with
> under user-emacs-directory - I come up with:
> 
> Cannot open file file: No such file or directory, ~/.emacs.d/custom.el
> 
> in the minibuffer.

I cannot look into this without seeing a minimal init.el file that
causes this.  In particular, I need to understand how come your
init.el file uses ~/.emacs.d when your user-emacs-directory is
different.

> I very much appreciate your help, especially in this case where anybody would ask "who cares?".
> 
> But I mind. I very mind, because I like emacs. But I don't like it omitting respect for its users.

My latest patch should do that, please try it.

> I secretly hoped you would come to the conclusion that the solution was already given in the bug report
> (actually more in wishlist) I've mentioned earlier (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=15539).

I don't think the solution proposed there would help in your case
because the value of user-emacs-directory is used to compute
auto-save-list-file-prefix before the command-line arguments are
processed.  IOW, by the time we look at the command-line arguments,
it's too late for auto-save-list-file-prefix: its value was already
computed using the default value of "~/.emacs.d".

> We're mutually wasting out times. Don't misunderstand me: you've got all my respect, but I won't follow you
> furthermore in the direction you take.

Does this mean you are giving up on this issue, and don't want to
continue looking for its solution?

bug#37116: 26.1; User choice user-emacs-directory is not honored

[bruno.vasselle--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

> Does this mean you are giving up on this issue, and don't want to 
> continue looking for its solution? 

this is the end, beautiful friend, the end of everything that stands. I'm probably of another generation. 


[-- Attachment #2: Type: text/html, Size: 891 bytes --]

bug#37116: 26.1; User choice user-emacs-directory is not honored

[Eli Zaretskii]

> Date: Thu, 22 Aug 2019 19:27:33 +0200 (CEST)
> From: bruno.vasselle@laposte.net
> Cc: 37116@debbugs.gnu.org
> 
> > Does this mean you are giving up on this issue, and don't want to
> > continue looking for its solution?
> 
> this is the end, beautiful friend, the end of everything that stands. I'm probably of another generation.

Patch pushed to the master branch.

bug#37116: 26.1; User choice user-emacs-directory is not honored

[Eli Zaretskii]

> Date: Thu, 22 Aug 2019 20:47:29 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 37116-done@debbugs.gnu.org
> 
> Patch pushed to the master branch.

It turns out that patch was wrong and had bad consequences.  So it was
reverted.

I think the right solution for this kind of situation is to set
user-emacs-directory in the early-init file, instead of the normal
init file.

bug#37116: 26.1; User choice user-emacs-directory is not honored

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> I think the right solution for this kind of situation is to set
> user-emacs-directory in the early-init file, instead of the normal
> init file.

Personally I'd hope that we can get the XDG code sorted out so that we
could use the XDG_CONFIG_HOME environment variable for this sort of
thing.

In any case, perhaps it's worth documenting how to properly set
user-emacs-directory in the doc string.

Best regards,
Stefan Kangas

bug#37116: 26.1; User choice user-emacs-directory is not honored

[bruno.vasselle--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 754 bytes --]


> > Furthermore, if I feed it with my actual 'init.el' - which defines a user-init-file emacs "custom.el" can play with 
> > under user-emacs-directory - I come up with: 
> > 
> > Cannot open file file: No such file or directory, ~/.emacs.d/custom.el 
> > 
> > in the minibuffer. 
> 
> I cannot look into this without seeing a minimal init.el file that 
> causes this. In particular, I need to understand how come your 
> init.el file uses ~/.emacs.d when your user-emacs-directory is 
> different. 

This is entirely my fault. My real init.el won't touch user-emacs-directory unless some environment variable is not set, which I removed during the test. Please accept my apologies. 

However, this does not affect the primary test "please-do-it.el". 


[-- Attachment #2: Type: text/html, Size: 1442 bytes --]