A cache directory?

A cache directory?

[Yuan Fu]

Maybe this has been discussed but I can’t find any threads in the archive. .emacs.d is pretty crowded: every package puts their temporary files under it. I wonder if there could be a standard cache directory? I know there are packages that's solely for this purpose: resetting every single xxx-file to be under a cache directory. However, it would be better if there is just a standard one and everybody uses it.

Even if we don’t appoint a standard cache directory, we could provide a built-in function that searches for possible cache directories, and ask every one to use that function. Specifically, something like:

(defun cache-directory ()
  "Return a possible cache directory.
Return nil if none found."
  (catch 'found
    (dolist (candidate '(".cache" "cache"
                         ".tmp" "tmp"
                         ".temp" "temp"))
      (let ((path (expand-file-name candidate user-emacs-directory)))
        (when (and (file-exists-p path)
                   (file-writable-p path)
                   (file-directory-p path))
          (throw 'found path))))))

Yuan

Re: A cache directory?

[Yuri Khan]

On Mon, 17 Aug 2020 at 02:30, Yuan Fu <casouri@gmail.com> wrote:
>
> Maybe this has been discussed but I can’t find any threads in the archive. .emacs.d is pretty crowded: every package puts their temporary files under it. I wonder if there could be a standard cache directory? I know there are packages that's solely for this purpose: resetting every single xxx-file to be under a cache directory. However, it would be better if there is just a standard one and everybody uses it.

There is a package called [no-littering][1] that modifies many
packages’ defaults to use .emacs.d/var/ for their persistent data, of
which caches are a subset. The convention (simplified; see README for
full details) is, if a package has a single persistent file, then it
goes right there; otherwise, they go in a subdirectory named after the
package responsible.

[1]: https://github.com/emacscollective/no-littering

Cleaning up and structuring user-emacs-directory (was: A cache directory?)

[Amin Bandali]

[-- Attachment #1: Type: text/plain, Size: 2289 bytes --]

Yuan Fu writes:

> Maybe this has been discussed but I can’t find any threads in the
> archive. .emacs.d is pretty crowded: every package puts their
> temporary files under it. I wonder if there could be a standard cache
> directory? I know there are packages that's solely for this purpose:
> resetting every single xxx-file to be under a cache
> directory. However, it would be better if there is just a standard one
> and everybody uses it.
>
[...]

I too have found myself generally displeased with how various packages
create arbitrary files in my `user-emacs-directory'.  I would personally
like to see this issue tackled more generally, for different sorts of
files rather than just cache/temporary files.

Of the packages that aim to do this, I liked the approach of the
"no-littering" package like the most.  Essentially, it designates two
directories, one for config files and another for persistent data files,
defaulting to "etc/" and "var/" under the `user-emacs-directory'
respectively.  It then changes the directory/file location settings for
many packages, dividing up each package's files into one of the two
general categories.  If a package stores multiple files of the same
category, no-littering creates a directory under that category with the
package name to keep those files together.

If we were to tackle this in Emacs itself, maybe we could have two
defcustoms `user-config-directory' and `user-data-directory', defaulting
to "etc/" and "var/" in `user-emacs-directory'.  If a lot of packages
use temporary/cache files, having a third `user-cache-directory' that
defaults to "var/cache/" in `user-emacs-directory' would make sense.
We could then ask packages to use one of these variables when defining
their file locations, rather than `user-emacs-directory' directly.
To avoid disruptions for users with existing files, we could take a
similar approach as the recently added XDG support, using the current
location for existing files, but defaulting to the new location for
each file that doesn't exist.

Perhaps an FHS-like [0] directory structure in `user-emacs-directory'
was always inevitable, since Emacs is an operating system after all. ;-)

[0]: https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: A cache directory?

[Yuan Fu]

> On Aug 16, 2020, at 3:45 PM, Yuri Khan <yuri.v.khan@gmail.com> wrote:
> 
> On Mon, 17 Aug 2020 at 02:30, Yuan Fu <casouri@gmail.com> wrote:
>> 
>> Maybe this has been discussed but I can’t find any threads in the archive. .emacs.d is pretty crowded: every package puts their temporary files under it. I wonder if there could be a standard cache directory? I know there are packages that's solely for this purpose: resetting every single xxx-file to be under a cache directory. However, it would be better if there is just a standard one and everybody uses it.
> 
> There is a package called [no-littering][1] that modifies many
> packages’ defaults to use .emacs.d/var/ for their persistent data, of
> which caches are a subset. The convention (simplified; see README for
> full details) is, if a package has a single persistent file, then it
> goes right there; otherwise, they go in a subdirectory named after the
> package responsible.
> 
> [1]: https://github.com/emacscollective/no-littering

That’s good, too. But even just a single cache directory that cleans up .emacs.d is welcome.

Yuan

Re: Cleaning up and structuring user-emacs-directory (was: A cache directory?)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If we were to tackle this in Emacs itself, maybe we could have two
  > defcustoms `user-config-directory' and `user-data-directory', defaulting
  > to "etc/" and "var/" in `user-emacs-directory'.  If a lot of packages
  > use temporary/cache files, having a third `user-cache-directory' that
  > defaults to "var/cache/" in `user-emacs-directory' would make sense.

LGTM.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Cleaning up and structuring user-emacs-directory

[Gunnar Horrigmo]

Amin Bandali <bandali@gnu.org> writes:

> Of the packages that aim to do this, I liked the approach of the
> "no-littering" package like the most.  Essentially, it designates two
> directories, one for config files and another for persistent data files,
> defaulting to "etc/" and "var/" under the `user-emacs-directory'
> respectively.  It then changes the directory/file location settings for
> many packages, dividing up each package's files into one of the two
> general categories.  If a package stores multiple files of the same
> category, no-littering creates a directory under that category with the
> package name to keep those files together.
>
> If we were to tackle this in Emacs itself, maybe we could have two
> defcustoms `user-config-directory' and `user-data-directory', defaulting
> to "etc/" and "var/" in `user-emacs-directory'.

Would it not be better to follow the freedesktop spec?
https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.6.html

-- 
Gunnar

Re: Cleaning up and structuring user-emacs-directory

[Gunnar Horrigmo]

Amin Bandali <bandali@gnu.org> writes:

> Of the packages that aim to do this, I liked the approach of the
> "no-littering" package like the most.  Essentially, it designates two
> directories, one for config files and another for persistent data files,
> defaulting to "etc/" and "var/" under the `user-emacs-directory'
> respectively.  It then changes the directory/file location settings for
> many packages, dividing up each package's files into one of the two
> general categories.  If a package stores multiple files of the same
> category, no-littering creates a directory under that category with the
> package name to keep those files together.
>
> If we were to tackle this in Emacs itself, maybe we could have two
> defcustoms `user-config-directory' and `user-data-directory', defaulting
> to "etc/" and "var/" in `user-emacs-directory'.

Would it not be better to follow the freedesktop spec?
https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.6.html

-- 
Gunnar

Re: Cleaning up and structuring user-emacs-directory

[Paul Eggert]

On 8/18/20 12:51 AM, Gunnar Horrigmo wrote:

> Would it not be better to follow the freedesktop spec?
> https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.6.html

Yes, I don't see why Emacs should do things differently from other applications.

Re: Cleaning up and structuring user-emacs-directory

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Would it not be better to follow the freedesktop spec?
  > https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.6.html

Concretely, what change would that mean in Emacs?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Cleaning up and structuring user-emacs-directory

[Amin Bandali]

[-- Attachment #1: Type: text/plain, Size: 1875 bytes --]

Richard Stallman writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Would it not be better to follow the freedesktop spec?
>   > https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.6.html
>
> Concretely, what change would that mean in Emacs?

In short, the approach that I proposed divides up `user-emacs-directory'
into multiple subdirectories and keeps `user-emacs-directory' as the
all-in-one source of a user's Emacs files, while the XDG Base Directory
approach as suggested by Gunnar would imply keeping only configuration
files in `user-emacs-directory', and moving data and cache files to
"$XDG_DATA_HOME/emacs/" and "$XDG_CACHE_HOME/emacs/" respectively, where
'XDG_DATA_HOME' defaults to "~/.local/share" and 'XDG_CACHE_HOME' to
"~/.cache".

The pros of the approach I proposed include `user-emacs-directory'
remaining self-contained like before, and it being divided up into
subdirectories for specific categories of files for each package.  The
cons of the approach include it not following a particular spec such as
the XDG Base Directory spec.

The pros of the XDG Base Directory approach suggested by Gunnar include
the proposed behaviour by Emacs conforming to the XDG Base Directory
spec, which has been seeing adoption among GNU/Linux applications.  Its
cons would include `user-emacs-directory' no longer being the one-stop
shop for the user's Emacs files, a more significant departure from
Emacs's traditional behaviour.

I'm personally leaning towards the former, somewhat more conservative
approach, but could see the arguments for the latter as well.  I'm
interested in hearing other folks' thoughts and opinions, and other
pros/cons they can think of.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: Cleaning up and structuring user-emacs-directory

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

On Wed, Aug 19, 2020 at 01:03:51AM -0400, Amin Bandali wrote:
> Richard Stallman writes:
> 
> > [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> > [[[ whether defending the US Constitution against all enemies,     ]]]
> > [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> >
> >   > Would it not be better to follow the freedesktop spec?
> >   > https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.6.html
> >
> > Concretely, what change would that mean in Emacs?
> 
> In short, the approach that I proposed divides up `user-emacs-directory'
> into multiple subdirectories and keeps `user-emacs-directory' as the
> all-in-one source of a user's Emacs files, while the XDG Base Directory
> approach as suggested by Gunnar would imply keeping only configuration
> files in `user-emacs-directory', and moving data and cache files to
> "$XDG_DATA_HOME/emacs/" and "$XDG_CACHE_HOME/emacs/" respectively, where
> 'XDG_DATA_HOME' defaults to "~/.local/share" and 'XDG_CACHE_HOME' to
> "~/.cache".

I've now read the freedesktop thing. If you ask me: please make that
optional. I have none of those $XDG_* env vars set, and I have nothing
of relevance in .local.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Cleaning up and structuring user-emacs-directory

[Stefan Monnier]

> The pros of the XDG Base Directory approach suggested by Gunnar include
> the proposed behaviour by Emacs conforming to the XDG Base Directory
> spec, which has been seeing adoption among GNU/Linux applications.  Its
> cons would include `user-emacs-directory' no longer being the one-stop
> shop for the user's Emacs files, a more significant departure from
> Emacs's traditional behaviour.

Another important downside of the XDG thingy is that AFAIK it's not used
under w32 (where some other system is used instead).

I think it'd makes sense to make it *possible* to follow either the
"system-specific" approach (e.g. XDG under GNU/Linux) or the
"Emacs-specific" approach (i.e. all under ~/.emacs.d).

Note also the the OP didn't suggest using XDG but only suggested to add
some structure underneath ~/.emacs.d, which seems quite reasonable and
useful regardless of the use of XDG.


        Stefan

Re: Cleaning up and structuring user-emacs-directory

[Amin Bandali]

[-- Attachment #1: Type: text/plain, Size: 2157 bytes --]

tomas@tuxteam.de writes:

> On Wed, Aug 19, 2020 at 01:03:51AM -0400, Amin Bandali wrote:
>> Richard Stallman writes:
>> 
>> > [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>> > [[[ whether defending the US Constitution against all enemies,     ]]]
>> > [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>> >
>> >   > Would it not be better to follow the freedesktop spec?
>> >   > https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.6.html
>> >
>> > Concretely, what change would that mean in Emacs?
>> 
>> In short, the approach that I proposed divides up `user-emacs-directory'
>> into multiple subdirectories and keeps `user-emacs-directory' as the
>> all-in-one source of a user's Emacs files, while the XDG Base Directory
>> approach as suggested by Gunnar would imply keeping only configuration
>> files in `user-emacs-directory', and moving data and cache files to
>> "$XDG_DATA_HOME/emacs/" and "$XDG_CACHE_HOME/emacs/" respectively, where
>> 'XDG_DATA_HOME' defaults to "~/.local/share" and 'XDG_CACHE_HOME' to
>> "~/.cache".
>
> I've now read the freedesktop thing. If you ask me: please make that
> optional. I have none of those $XDG_* env vars set, and I have nothing
> of relevance in .local.
>
> Cheers
>  - t

Reading this, another con of the XDG Base Directory approach just
occurred to me:

As of now, the XDG support added in Emacs 27 for the location of
`user-emacs-directory' is completely optional: if "~/.emacs.d/" exists,
"$XDG_CONFIG_HOME/emacs/" won't be used at all.  And some people seem to
quite dislike the XDG approach.  So, basing this change upon that, users
who choose not to use "$XDG_CONFIG_HOME/emacs/" will inevitably be opted
out of this cleaning up of their Emacs directory.  I'd personally like
to have my `user-emacs-directory' be clean, even if I choose to use
"~/.emacs.d/" and not "$XDG_CONFIG_HOME/emacs/".  So, another reason for
me to prefer my original proposal of diving `user-emacs-directory' up
into subdirectories, regardless of whether or not the user uses
"$XDG_CONFIG_HOME/emacs/".

Thoughts?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: Cleaning up and structuring user-emacs-directory

[tomas]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

On Wed, Aug 19, 2020 at 09:33:51AM -0400, Stefan Monnier wrote:

[...]

> I think it'd makes sense to make it *possible* to follow either the
> "system-specific" approach (e.g. XDG under GNU/Linux) or the
> "Emacs-specific" approach (i.e. all under ~/.emacs.d).

That sounds far more attractive to me.

> Note also the the OP didn't suggest using XDG but only suggested to add
> some structure underneath ~/.emacs.d, which seems quite reasonable and
> useful regardless of the use of XDG.

Agreed.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Cleaning up and structuring user-emacs-directory

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

On Wed, Aug 19, 2020 at 09:59:41AM -0400, Amin Bandali wrote:

[...]

> Reading this, another con of the XDG Base Directory approach just
> occurred to me:
> 
> As of now, the XDG support added in Emacs 27 for the location of
> `user-emacs-directory' is completely optional: if "~/.emacs.d/" exists,
> "$XDG_CONFIG_HOME/emacs/" won't be used at all.  And some people seem to
> quite dislike the XDG approach.  So, basing this change upon that, users
> who choose not to use "$XDG_CONFIG_HOME/emacs/" will inevitably be opted
> out of this cleaning up of their Emacs directory.  I'd personally like
> to have my `user-emacs-directory' be clean, even if I choose to use
> "~/.emacs.d/" and not "$XDG_CONFIG_HOME/emacs/".  So, another reason for
> me to prefer my original proposal of diving `user-emacs-directory' up
> into subdirectories, regardless of whether or not the user uses
> "$XDG_CONFIG_HOME/emacs/".
> 
> Thoughts?

That would work for me. I actually structure my ~/.emacs.d anyway, and
could live well with another structuring.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Cleaning up and structuring user-emacs-directory

[David De La Harpe Golden]

On 19/08/2020 09:28, tomas@tuxteam.de wrote:

> I've now read the freedesktop thing. If you ask me: please make that
> optional.

indeed, existing emacs fd.o support in various contexts is generally 
optional.

> I have none of those $XDG_* env vars set, and I have nothing
> of relevance in .local.
> 

They're overrides not mandatory, the normal circumstance for an app that 
follows the spec is falling back to the standard paths.  When emacs uses 
them, which it may already do in various circumstances, you can see it 
does that:

https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/xdg.el#n52

At startup, emacs apparently already uses  ~/.config/emacs/ if present 
and .emacs and ~/.emacs.d aren't:

https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/startup.el#n494

However, at time of writing a user using the existing emacs xdg 
~/.config/emacs/ dir  support will AFAICS actually find various arguably 
"data" and "cache" files also plonked into the "config" directory 
~/.config/emacs/ dir by emacs, as it's basically treated as an 
alternatively spelled ~/.emacs.d/

That may be consided at least vaguely suspect under the xdg spec 
conventions, user config, cache and data files are supposed to have 
those split locations.  To concretize, testing with emacs 27.1 
(personally I actually still use ~/.emacs and ~/.emacs.d/ normally!):


# keep for restoration!

$ mkdir -p .config/emacs/
$ cp .emacs .config/emacs/init.el
$ mv .emacs .emacs.off
$ mv .emacs.d .emacs.d.off


Run emacs

$ emacs

# intra-emacs do something arguably "cache" and "data" like getting the
# latest elpa package list and installing a package

M-x list-packages
<install something, trie for argument's sake>
C-x C-c


#  let's look inside:

ls ~/.config/emacs/


$ ls -1 ~/.config/emacs/
auto-save-list
elpa
init.el

$ ls -1 ~/.config/emacs/elpa/
archives
gnupg
heap-0.5
heap-0.5.signed
queue-0.2
queue-0.2.signed
tNFA-0.1.1
tNFA-0.1.1.signed
trie-0.4
trie-0.4.signed


# -> user ends up with emacs init.el in config (fine), but also
# elpa package lists stored in xdg config subdir not cache
# elpa packages installed in xdg config subdir not data

This contrasts to e.g. python's "pip install --user", as that will 
install python packages  ~/.local/lib/python3 etc. as "data", and keep 
its various transient metadata things in ~/.cache/pip/ as "cache"


















> Cheers
>   - t
> 

Re: Cleaning up and structuring user-emacs-directory

[noah swainland]

Seeing as there are already behavioural checks depending on whether
"~/.emacs.d"" exists or not, could not these proposed
`user-emacs-directory' subdirectory definitions change depending on
whether or not "~/.emacs.d" exists?  I'm not sure what exactly you're
proposing each one to be, but they'd probably match up at least partly
to the XDG directories.

I personally like "$XDG_CONFIG_HOME/emacs", it makes it easier for me to
copy a single folder of configurations between computers.

--
noah ~ distinctly.pink
  never stop stopping to smile at the flowers

-- Email domain proudly hosted at https://migadu.com

Re: Cleaning up and structuring user-emacs-directory

[tomas]

[-- Attachment #1: Type: text/plain, Size: 899 bytes --]

On Wed, Aug 19, 2020 at 03:42:25PM +0100, David De La Harpe Golden wrote:
> On 19/08/2020 09:28, tomas@tuxteam.de wrote:
> 
> >I've now read the freedesktop thing. If you ask me: please make that
> >optional.
> 
> indeed, existing emacs fd.o support in various contexts is generally
> optional.

Thanks

> However, at time of writing a user using the existing emacs xdg
> ~/.config/emacs/ dir  support will AFAICS actually find various
> arguably "data" and "cache" files also plonked into the "config"
> directory ~/.config/emacs/ dir by emacs, as it's basically treated
> as an alternatively spelled ~/.emacs.d/
> 
> That may be consided at least vaguely suspect [...]

Absolutely agreed. From my POV, I think even Emacs oldtimers
wouldn't object to see data and cache in separate locations
(possibly all under ~/.emacs.d). But let other oldtimers chime
in.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Cleaning up and structuring user-emacs-directory

[Yuan Fu]

I think one concern could be that if we change the cache directory, users would need to manually move their cache files, or lost their data. That doesn’t has to be the case, we can make this new feature opt-in:

(defvar user-cache-directory nil)

(defun expand-cache-name (name)
  (if user-cache-directory
      (expand-file-name name user-cache-directory)
    (expand-file-name user-emacs-directory)))

Then,

(defvar xxx-file (expand-file-name "xxx-file" user-emacs-directory))

Would become

(defvar xxx-file (expand-cache-name "xxx-file”))

This way, existing cache files in .Emacs.d are still valid, and people who want a clean .Emacs.d can define user-cache-directory to what ever they want.

Yuan

Re: Cleaning up and structuring user-emacs-directory

[Stefan Monnier]

>   (if user-cache-directory
>       (expand-file-name name user-cache-directory)
>     (expand-file-name user-emacs-directory)))

AKA (expand-file-name name (or user-cache-directory user-emacs-directory))


        Stefan

Re: Cleaning up and structuring user-emacs-directory

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 19 Aug 2020 13:33:18 -0400
> Cc: tomas@tuxteam.de, emacs-devel@gnu.org
> 
> >   (if user-cache-directory
> >       (expand-file-name name user-cache-directory)
> >     (expand-file-name user-emacs-directory)))
> 
> AKA (expand-file-name name (or user-cache-directory user-emacs-directory))

Are you sure?

Re: Cleaning up and structuring user-emacs-directory

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > In short, the approach that I proposed divides up `user-emacs-directory'
  > into multiple subdirectories and keeps `user-emacs-directory' as the
  > all-in-one source of a user's Emacs files, while the XDG Base Directory
  > approach as suggested by Gunnar would imply keeping only configuration
  > files in `user-emacs-directory', and moving data and cache files to
  > "$XDG_DATA_HOME/emacs/" and "$XDG_CACHE_HOME/emacs/" respectively, where
  > 'XDG_DATA_HOME' defaults to "~/.local/share" and 'XDG_CACHE_HOME' to
  > "~/.cache".

There is a benefit to putting all cache files in ~/.cache.  For
instance, you know where to find all the files you can safely delete,
regardless of which program made them.

What concretely are "data" files, for Emacs?  I don't know where in
the sources to look for that info.  Can you mention a few examples and
what data they contain?  With specifics, we can see the concrete
advantage of one choice or the other.  With only the abstraction "data
files", we can see only at the level of abstractions, and that is a sort
of blindness that can lead to bad decisions.



-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Cleaning up and structuring user-emacs-directory

[Stefan Monnier]

>> >   (if user-cache-directory
>> >       (expand-file-name name user-cache-directory)
>> >     (expand-file-name user-emacs-directory)))
>> 
>> AKA (expand-file-name name (or user-cache-directory user-emacs-directory))
>
> Are you sure?

Yup

    (if user-cache-directory
        (expand-file-name name user-cache-directory)
      (expand-file-name user-emacs-directory)))
=>
    (expand-file-name name
                      (if user-cache-directory
                          user-cache-directory
                        user-emacs-directory))
=>
    (expand-file-name name (or user-cache-directory user-emacs-directory))


-- Stefan

Re: Cleaning up and structuring user-emacs-directory

[Andreas Schwab]

On Aug 19 2020, Stefan Monnier wrote:

> Yup
>
>     (if user-cache-directory
>         (expand-file-name name user-cache-directory)
>       (expand-file-name user-emacs-directory)))
> =>
>     (expand-file-name name
>                       (if user-cache-directory
>                           user-cache-directory
>                         user-emacs-directory))

That conversion isn't bug-compatible. :-)

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Cleaning up and structuring user-emacs-directory

[Stefan Monnier]

>> Yup
>>
>>     (if user-cache-directory
>>         (expand-file-name name user-cache-directory)
>>       (expand-file-name user-emacs-directory)))
>> =>
>>     (expand-file-name name
>>                       (if user-cache-directory
>>                           user-cache-directory
>>                         user-emacs-directory))
>
> That conversion isn't bug-compatible. :-)

Indeed.


        Stefan

Re: Cleaning up and structuring user-emacs-directory

[Yuan Fu]

> On Aug 19, 2020, at 11:09 PM, Richard Stallman <rms@gnu.org> wrote:
> 
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>> In short, the approach that I proposed divides up `user-emacs-directory'
>> into multiple subdirectories and keeps `user-emacs-directory' as the
>> all-in-one source of a user's Emacs files, while the XDG Base Directory
>> approach as suggested by Gunnar would imply keeping only configuration
>> files in `user-emacs-directory', and moving data and cache files to
>> "$XDG_DATA_HOME/emacs/" and "$XDG_CACHE_HOME/emacs/" respectively, where
>> 'XDG_DATA_HOME' defaults to "~/.local/share" and 'XDG_CACHE_HOME' to
>> "~/.cache".
> 
> There is a benefit to putting all cache files in ~/.cache.  For
> instance, you know where to find all the files you can safely delete,
> regardless of which program made them.
> 
> What concretely are "data" files, for Emacs?  I don't know where in
> the sources to look for that info.  Can you mention a few examples and
> what data they contain?  With specifics, we can see the concrete
> advantage of one choice or the other.  With only the abstraction "data
> files", we can see only at the level of abstractions, and that is a sort
> of blindness that can lead to bad decisions.
> 

I don’t know about the concrete definition, but some built-in examples includes:

recentf-save-file
bookmark-default-file
savehist-file
project-list-file

The main benefit for me is to clean up my .emacs.d, similar to XDG’s goal to clean up the home directory.

Yuan

Re: Cleaning up and structuring user-emacs-directory

[Stefan Monnier]

Yuan Fu [2020-08-20 10:56:28] wrote:

>> On Aug 19, 2020, at 11:09 PM, Richard Stallman <rms@gnu.org> wrote:
>> 
>> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>> [[[ whether defending the US Constitution against all enemies,     ]]]
>> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>> 
>>> In short, the approach that I proposed divides up `user-emacs-directory'
>>> into multiple subdirectories and keeps `user-emacs-directory' as the
>>> all-in-one source of a user's Emacs files, while the XDG Base Directory
>>> approach as suggested by Gunnar would imply keeping only configuration
>>> files in `user-emacs-directory', and moving data and cache files to
>>> "$XDG_DATA_HOME/emacs/" and "$XDG_CACHE_HOME/emacs/" respectively, where
>>> 'XDG_DATA_HOME' defaults to "~/.local/share" and 'XDG_CACHE_HOME' to
>>> "~/.cache".
>> 
>> There is a benefit to putting all cache files in ~/.cache.  For
>> instance, you know where to find all the files you can safely delete,
>> regardless of which program made them.
>> 
>> What concretely are "data" files, for Emacs?  I don't know where in
>> the sources to look for that info.  Can you mention a few examples and
>> what data they contain?  With specifics, we can see the concrete
>> advantage of one choice or the other.  With only the abstraction "data
>> files", we can see only at the level of abstractions, and that is a sort
>> of blindness that can lead to bad decisions.
>> 
>
> I don’t know about the concrete definition, but some built-in examples includes:
>
> recentf-save-file
> bookmark-default-file
> savehist-file
> project-list-file

ecomplete-database-file


        Stefan

Re: Cleaning up and structuring user-emacs-directory

[T.V Raman]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=gb18030, Size: 1956 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:


also auto-insert-directory could profit fromusing
locate-user-emacs-file
> Yuan Fu [2020-08-20 10:56:28] wrote:
>
>>> On Aug 19, 2020, at 11:09 PM, Richard Stallman <rms@gnu.org> wrote:
>>> 
>>> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>>> [[[ whether defending the US Constitution against all enemies,     ]]]
>>> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>>> 
>>>> In short, the approach that I proposed divides up `user-emacs-directory'
>>>> into multiple subdirectories and keeps `user-emacs-directory' as the
>>>> all-in-one source of a user's Emacs files, while the XDG Base Directory
>>>> approach as suggested by Gunnar would imply keeping only configuration
>>>> files in `user-emacs-directory', and moving data and cache files to
>>>> "$XDG_DATA_HOME/emacs/" and "$XDG_CACHE_HOME/emacs/" respectively, where
>>>> 'XDG_DATA_HOME' defaults to "~/.local/share" and 'XDG_CACHE_HOME' to
>>>> "~/.cache".
>>> 
>>> There is a benefit to putting all cache files in ~/.cache.  For
>>> instance, you know where to find all the files you can safely delete,
>>> regardless of which program made them.
>>> 
>>> What concretely are "data" files, for Emacs?  I don't know where in
>>> the sources to look for that info.  Can you mention a few examples and
>>> what data they contain?  With specifics, we can see the concrete
>>> advantage of one choice or the other.  With only the abstraction "data
>>> files", we can see only at the level of abstractions, and that is a sort
>>> of blindness that can lead to bad decisions.
>>> 
>>
>> I don¡¯t know about the concrete definition, but some built-in examples includes:
>>
>> recentf-save-file
>> bookmark-default-file
>> savehist-file
>> project-list-file
>
> ecomplete-database-file
>
>
>         Stefan
>
>

-- 
7©4Id: kg:/m/0285kf1  •0Ü87©4

Re: Cleaning up and structuring user-emacs-directory

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The doc string of ecomplete-database-file suggests that the file is
really a kind of cache.  Is that so?  Would deleting the file be
harmless?

I have a feeling that recentf-save-file is also a cache file.  Is that so?

On the other hand, it looks like bookmark-default-file is a file of
user data, and deleting the file would lose the user's data.  Is that
so?

It is not unreasonable to store the data files in a separate directory
from configuration files.  However, if there are few of them it is not
worth the trouble to separate them.

We should take care to distinguish which files are cache files
and put them into ~/.cache.


It makes sense to put data files like that one into a separate place


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Cleaning up and structuring user-emacs-directory

[Gunnar Horrigmo]

Amin Bandali <bandali@gnu.org> writes:

> The pros of the XDG Base Directory approach suggested by Gunnar include
> the proposed behaviour by Emacs conforming to the XDG Base Directory
> spec, which has been seeing adoption among GNU/Linux applications.  Its
> cons would include `user-emacs-directory' no longer being the one-stop
> shop for the user's Emacs files, a more significant departure from
> Emacs's traditional behaviour.

I don't see that as a con, since I want all my configuration files under
version control. Having my configurations seperate from all of emacs'
own configurations is less hassle (even if it's easily dealt with).


> I'm personally leaning towards the former, somewhat more conservative
> approach, but could see the arguments for the latter as well.  I'm
> interested in hearing other folks' thoughts and opinions, and other
> pros/cons they can think of.

I prefer the XDG way, as that makes it neater and easier to simply put 
~/.config under version control, and pretty much be done with it.

Would it be out of the question supporting both (via a
--old-school-config or somesuch (or new-school for that matter :)))?

-- 
Gunnar

Re: Cleaning up and structuring user-emacs-directory

[Stefan Monnier]

> The doc string of ecomplete-database-file suggests that the file is
> really a kind of cache.  Is that so?

Not exactly, it's more like a history (of email addresses we've bumped into).

> Would deleting the file be harmless?

It doesn't contain data the user painstakingly created, but deleting it
would cause the suggested completions to be much less useful for a while.

So it's somewhat like a cache, except that a cache is expected to affect
only the performance but not the actual behavior.

> I have a feeling that recentf-save-file is also a cache file.  Is that so?

Same: it's a history (of files we visited).

> We should take care to distinguish which files are cache files
> and put them into ~/.cache.

Agreed.  I think package.el's `archive-contents` file do qualify as
caches, as do the <pkg>-readme.txt files.


        Stefan

Re: Cleaning up and structuring user-emacs-directory

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> We should take care to distinguish which files are cache files
>> and put them into ~/.cache.
>
> Agreed.  I think package.el's `archive-contents` file do qualify as
> caches, as do the <pkg>-readme.txt files.

How about something like:

(locate-user-cache-file "foo")
=> ~/.cache/emacs/foo

(locate-user-cache-file "archive-contents" 'package)
=> ~/.cache/emacs/package/archive-contents

(We already have `xdg-cache-home', BTW.)

Best regards,
Stefan Kangas

Re: Cleaning up and structuring user-emacs-directory

[John Yates]

On Sun, Aug 23, 2020 at 11:16 AM Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
. . .
> So it's somewhat like a cache, except that a cache is expected to affect
> only the performance but not the actual behavior.
. . .
> Same: it's a history (of files we visited).

Most web browsers explicitly call out the notion of history.
Perhaps emacs should do the same.

/john

Re: Cleaning up and structuring user-emacs-directory

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So it's somewhat like a cache, except that a cache is expected to affect
  > only the performance but not the actual behavior.

That means it is not really a cache.  So it is a data file.
Thanks.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)