xdg-directories.el

xdg-directories.el

[francisco.colaco]

[-- Attachment #1.1: Type: text/plain, Size: 2775 bytes --]

  Friends,

I have written several years ago a package called xdg-paths.el.  I have
rewritten it as xdg-directories.el.

* WHAT IS xdg-directories.el

xdg-directories.el allows the package writers to locate a file in
Emacs user directories for different domains: data, configuration,
cache and runtime.  It also locates files in the domains defined by the
utility xdg-user-dir, which is executed when finding the domain.

Using this package, an emacs-lisp package writer can put different
files under different directories, according to the domain thereof.  A
cache file, which is normally not wanted in backups, can be named
under ~/.cache/emacs/<filename>, whereas a configuration file would be
under a different directory, in ~/.config/emacs/.  Passwords and
security sensible files can also made to be in a directory where, as
the user logs out, is normally erased.

* BRIEF USAGE (see README.md at the Github repository)

Files form several domains can be located.  Located means here, at the
lack of a better term, named, since the file name will be returned,
regardless of the file existence.

- User documents:

> (locate-user-document-file "org/agenda.txt")
> "/home/francisco.colaco/Documentos/org/agenda.txt"

- Configuration files:

> (locate-user-emacs-config-file "init.el")
> "/home/francisco.colaco/.config/emacs/emacs/init.el"

- Data files:

> (locate-user-emacs-data-file "recentf")
> "/home/francisco.colaco/.local/share/emacs/emacs/recentf"

- Cache files:

> (locate-user-emacs-cache-file "elfeed/index")
> "/home/francisco.colaco/.cache/emacs/emacs/elfeed/index"

- Runtime files:

> (locate-user-emacs-runtime-file "credentials.txt")
> "/run/user/1000/emacs/emacs/credentials.txt"

For example, init.el could be marked as a configuration file (and reside
under ~/.config/emacs) and recentf as data file (being under
~/.local/share/emacs).

There is more to the library.  I have grated a Github repository, at

  https://github.com/francisco-colaco/xdg-directories-el/

*  ABOUT user-init-file

I would request that this package (after revision and a possible API
change) becomes part of GNU Emacs.  I would also suggest that
~/.config/emacs/init.el (the result of "(locate-user-emacs-config-file
"init.el") becomes in vanilla emacs part of the chain to determine
user-init-file.

As an advantage, one could weed out cache files from user backups or line
transmissions.  This package breaks no existing functions, and so, as the
package writers adopt it, the separation of domains is assured.  I see no
real advantage in keeping almost everything under ~/.emacs.d/, the present
situation, but I am open to other arguments.

  Thanks for your attention.

    Francisco Colaço

[-- Attachment #1.2: Type: text/html, Size: 3286 bytes --]

[-- Attachment #2: xdg-directories.el --]
[-- Type: text/x-emacs-lisp, Size: 7780 bytes --]

;;; xdg-directories.el --- XDG directory specification

;; Copyright (C) 2016  Francisco Miguel Colaço

;; Author: Francisco Miguel Colaço <francisco.colaco@gmail.com>
;; Keywords: XDG, directory, cache, config

;;; Commentary:
;;;
;;; XDG-DIRECTORIES contains functions to locate user Emacs files, in
;;; accordance to the XDG Base Directory Specification.
;;;
;;; There are different domains to the files, as they are broken into
;;; config, data, cache and runtime.  We advise the users of this
;;; package to look at the aforementioned specification to learn the
;;; differences of the domains.
;;;
;;; The locate-* functions are to be used by package writers.  User
;;; Emacs configuration, data and cache files can then be segregated
;;; into their own directories, making it simple to migrate
;;; configurations among several machines --- since the files that are
;;; unnecessary, being cached files, could easily not be transmitted.
;;;
;;; This package is released under the GNU General Public License,
;;; version 3.0 or, at your choice, above.  One may read the GNU
;;; General Public License at the GNU Web site, at http://www.gnu.org
;;;

;;; Code:

(require 's)


(defgroup xdg-directories
    ()
  "XDG Directory Specification"
  :group 'environment
  :link '(emacs-library-link "xdg-directories"))


(defun xdg-get-path (domain &optional path)
  "Return the directory at DOMAIN, with an optional PATH relative to it."
  (let ((dir (s-chomp (shell-command-to-string (concat "xdg-user-dir " domain)))))
    (if path
        (expand-file-name path dir)
        dir)))


(defcustom xdg-data-home
  (or (getenv "XDG_DATA_HOME") (expand-file-name "~/.local/share"))
  "The base directory relative to which user specific data files
  should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom xdg-config-home
  (or (getenv "XDG_CONFIG_HOME") (expand-file-name "~/.config"))
  "The base directory relative to which user specific
  configuration files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom xdg-cache-home
  (or (getenv "XDG_CACHE_HOME") (expand-file-name "~/.cache"))
  "The base directory relative to which user specific
  non-essential data files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom xdg-runtime-dir
  (getenv "XDG_RUNTIME_DIR")
  "The directory where files that pertain only to this session
  are stored.  These files can be erased when the application quits
  or the user logs out."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-emacs-data-home
  (expand-file-name "emacs" xdg-data-home)
  "The base directory relative to which user Emacs specific data
  files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-emacs-config-home
  (expand-file-name "emacs" xdg-config-home)
  "The base directory relative to which user Emacs specific
  configuration files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-emacs-cache-home
  (expand-file-name "emacs" xdg-cache-home)
  "The base directory relative to which user Emacs specific
  non-essential data files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-emacs-runtime-dir
  (expand-file-name "emacs" xdg-runtime-dir)
  "The directory where Emacs files that pertain only to this session
  are stored.  These files can be erased when the application quits
  or the user logs out."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-desktop-directory
  (xdg-get-path "DESKTOP")
  "The desktop directory."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-download-directory
  (xdg-get-path "DOWNLOAD")
  "The directory where the user stores his downloaded files by default."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-templates-directory
  (xdg-get-path "TEMPLATES")
  "The directory where the user stores his templates."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-public-share-directory
  (xdg-get-path "PUBLICSHARE")
  "The directory where the user stores his shared files."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-documents-directory
  (xdg-get-path "DOCUMENTS")
  "The directory where the user stores his documents."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-music-directory
  (xdg-get-path "MUSIC")
  "The directory where the user stores his music."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-pictures-directory
  (xdg-get-path "PICTURES")
  "The directory where the user stores his pictures."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-videos-directory
  (xdg-get-path "VIDEOS")
  "The directory where the user stores his videos."
  :group 'xdg-directories
  :type 'directory)


(defun xdg-get-user-file (filename parent-directory &optional create-parent-dirs)
  "Locate FILENAME under PARENT-DIRECTORY/emacs, wherther it exists or not.

If CREATE-PARENT-DIRS is T, the parent directories are created."
  (let* ((base-dir (expand-file-name "emacs" parent-directory))
         (file (expand-file-name filename base-dir))
         (dir (file-name-directory file)))
    ;; Create any parent directories, if so requested.
    (if create-parent-dirs
        (unless (file-exists-p dir)
          (make-directory dir t)))
    ;; Return the file name.
    file))


(defun locate-user-emacs-cache-file (filename &optional create-parent-dirs)
  "Locate a cache file named FILENAME.

The file may exist or not under the user Emacs cache dir (under
XDG-CACHE-HOME/emacs).  The path of the file is returned
regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (xdg-get-user-file filename user-emacs-cache-home create-parent-dirs))


(defun locate-user-emacs-config-file (filename &optional create-parent-dirs)
  "Locate a config file named FILENAME.

The file may exist or not under the user Emacs config dir (under
XDG-CONFIG-HOME/emacs).  The path of the file is returned
regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (xdg-get-user-file filename user-emacs-config-home create-parent-dirs))


(defun locate-user-emacs-data-file (filename &optional create-parent-dirs)
  "Locate a data file named FILENAME.

The file may exist or not under the user Emacs data dir (under
XDG-DATA-HOME/emacs).  The path of the file is returned
regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (xdg-get-user-file filename user-emacs-data-home create-parent-dirs))


(defun locate-user-emacs-runtime-file (filename &optional create-parent-dirs)
  "Locate a runtime file named FILENAME.

The file may exist or not under the user Emacs runtime dir (under
XDG-RUNTIME-DIR/emacs).  The path of the file is returned
regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (xdg-get-user-file filename user-emacs-runtime-dir create-parent-dirs))


(defun locate-user-document-file (filename)
  "Locate a document file named FILENAME.

The file may exist or not under the user Emacs documents
dir (which is predetermined by this package using operating
system tools or, if found lacking, sensible defaults).  The path
of the file is returned regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (expand-file-name filename user-documents-directory))


(provide 'xdg-directories)
;;; xdg-directories.el ends here

Re: xdg-directories.el

[Noam Postavsky]

Last time this was proposed
(https://debbugs.gnu.org/cgi/bugreport.cgi?bug=583) there was some
skepticism about the stability of such standards.

On Tue, Sep 6, 2016 at 11:24 AM, francisco.colaco@gmail.com
<francisco.colaco@gmail.com> wrote:
>> (locate-user-emacs-config-file "init.el")
>> "/home/francisco.colaco/.config/emacs/emacs/init.el"
[...]
> ~/.config/emacs/init.el (the result of "(locate-user-emacs-config-file
> "init.el")

Is the double emacs/emacs/ in the first case a typo?

Re: xdg-directories.el

[francisco.colaco]

[-- Attachment #1.1: Type: text/plain, Size: 1864 bytes --]

Noam,

You were right about the bug.  When I created the user-emacs-<domain>-dir
libraries, which contained by default the subdirectory "emacs", I forgot to
take it from the xdg-get-user-file.  Needless to say, I have only
remembered of creating the user-emacs-<domain>-dir variables right before I
commited.  I have been using the library for months without a flaw.

So the bug is now corrected.  Thanks for the heads up.

About the stability of the specification: according to the XDG Web site,
the last version of the specification dates from 2010.  That is six years
of no modifications.  Seems to be pretty stable now.  Maybe it is time for
Emacs to start the adoption of the XDG Base Dir Spec and separate the files.

A .emacs.d/init.el could be a fallback for user-init-file, as .emacs is
today.  Also, maybe in Windows the xdg-<domain>-dir variables could take
other values and not the defaults I hardwired.  As Windows is really not
what I use, the package is pending suggestions of improvement.

As to the API, should the locate-user-emacs-<domain>-file be renamed?
locate-user-emacs-file has the NEW-NAME OLD-NAME parameters, and creates
unconditionally the directory when OLD-NAME is not passed.

  Francisco Colaço

2016-09-06 22:49 GMT+01:00 Noam Postavsky <npostavs@users.sourceforge.net>:

> Last time this was proposed
> (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=583) there was some
> skepticism about the stability of such standards.
>
> On Tue, Sep 6, 2016 at 11:24 AM, francisco.colaco@gmail.com
> <francisco.colaco@gmail.com> wrote:
> >> (locate-user-emacs-config-file "init.el")
> >> "/home/francisco.colaco/.config/emacs/emacs/init.el"
> [...]
> > ~/.config/emacs/init.el (the result of "(locate-user-emacs-config-file
> > "init.el")
>
> Is the double emacs/emacs/ in the first case a typo?
>

[-- Attachment #1.2: Type: text/html, Size: 2640 bytes --]

[-- Attachment #2: xdg-directories.el --]
[-- Type: text/x-emacs-lisp, Size: 7744 bytes --]

;;; xdg-directories.el --- XDG directory specification

;; Copyright (C) 2016  Francisco Miguel Colaço

;; Author: Francisco Miguel Colaço <francisco.colaco@gmail.com>
;; Keywords: XDG, directory, cache, config

;;; Commentary:
;;;
;;; XDG-DIRECTORIES contains functions to locate user Emacs files, in
;;; accordance to the XDG Base Directory Specification.
;;;
;;; There are different domains to the files, as they are broken into
;;; config, data, cache and runtime.  We advise the users of this
;;; package to look at the aforementioned specification to learn the
;;; differences of the domains.
;;;
;;; The locate-* functions are to be used by package writers.  User
;;; Emacs configuration, data and cache files can then be segregated
;;; into their own directories, making it simple to migrate
;;; configurations among several machines --- since the files that are
;;; unnecessary, being cached files, could easily not be transmitted.
;;;
;;; This package is released under the GNU General Public License,
;;; version 3.0 or, at your choice, above.  One may read the GNU
;;; General Public License at the GNU Web site, at http://www.gnu.org
;;;

;;; Code:

(require 's)


(defgroup xdg-directories
    ()
  "XDG Directory Specification"
  :group 'environment
  :link '(emacs-library-link "xdg-directories"))


(defun xdg-get-path (domain &optional path)
  "Return the directory at DOMAIN, with an optional PATH relative to it."
  (let ((dir (s-chomp (shell-command-to-string (concat "xdg-user-dir " domain)))))
    (if path
        (expand-file-name path dir)
        dir)))


(defcustom xdg-data-home
  (or (getenv "XDG_DATA_HOME") (expand-file-name "~/.local/share"))
  "The base directory relative to which user specific data files
  should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom xdg-config-home
  (or (getenv "XDG_CONFIG_HOME") (expand-file-name "~/.config"))
  "The base directory relative to which user specific
  configuration files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom xdg-cache-home
  (or (getenv "XDG_CACHE_HOME") (expand-file-name "~/.cache"))
  "The base directory relative to which user specific
  non-essential data files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom xdg-runtime-dir
  (getenv "XDG_RUNTIME_DIR")
  "The directory where files that pertain only to this session
  are stored.  These files can be erased when the application quits
  or the user logs out."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-emacs-data-home
  (expand-file-name "emacs" xdg-data-home)
  "The base directory relative to which user Emacs specific data
  files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-emacs-config-home
  (expand-file-name "emacs" xdg-config-home)
  "The base directory relative to which user Emacs specific
  configuration files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-emacs-cache-home
  (expand-file-name "emacs" xdg-cache-home)
  "The base directory relative to which user Emacs specific
  non-essential data files should be stored."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-emacs-runtime-dir
  (expand-file-name "emacs" xdg-runtime-dir)
  "The directory where Emacs files that pertain only to this session
  are stored.  These files can be erased when the application quits
  or the user logs out."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-desktop-directory
  (xdg-get-path "DESKTOP")
  "The desktop directory."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-download-directory
  (xdg-get-path "DOWNLOAD")
  "The directory where the user stores his downloaded files by default."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-templates-directory
  (xdg-get-path "TEMPLATES")
  "The directory where the user stores his templates."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-public-share-directory
  (xdg-get-path "PUBLICSHARE")
  "The directory where the user stores his shared files."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-documents-directory
  (xdg-get-path "DOCUMENTS")
  "The directory where the user stores his documents."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-music-directory
  (xdg-get-path "MUSIC")
  "The directory where the user stores his music."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-pictures-directory
  (xdg-get-path "PICTURES")
  "The directory where the user stores his pictures."
  :group 'xdg-directories
  :type 'directory)


(defcustom user-videos-directory
  (xdg-get-path "VIDEOS")
  "The directory where the user stores his videos."
  :group 'xdg-directories
  :type 'directory)


(defun xdg-get-user-file (filename directory &optional create-parent-dirs)
  "Locate FILENAME under DIRECTORY, wherther it exists or not.

If CREATE-PARENT-DIRS is T, the parent directories are created.

Returns the complete file path, so it can be chained."
  (let ((file (expand-file-name filename directory)))
    ;; Create any parent directories, if so requested.
    (if create-parent-dirs
      (unless (file-exists-p directory)
        (make-directory directory t)))
    ;; Return the name of the file with the given path.
    file))


(defun locate-user-emacs-cache-file (filename &optional create-parent-dirs)
  "Locate a cache file named FILENAME.

The file may exist or not under the user Emacs cache dir (under
XDG-CACHE-HOME/emacs).  The path of the file is returned
regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (xdg-get-user-file filename user-emacs-cache-home create-parent-dirs))


(defun locate-user-emacs-config-file (filename &optional create-parent-dirs)
  "Locate a config file named FILENAME.

The file may exist or not under the user Emacs config dir (under
XDG-CONFIG-HOME/emacs).  The path of the file is returned
regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (xdg-get-user-file filename user-emacs-config-home create-parent-dirs))


(defun locate-user-emacs-data-file (filename &optional create-parent-dirs)
  "Locate a data file named FILENAME.

The file may exist or not under the user Emacs data dir (under
XDG-DATA-HOME/emacs).  The path of the file is returned
regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (xdg-get-user-file filename user-emacs-data-home create-parent-dirs))


(defun locate-user-emacs-runtime-file (filename &optional create-parent-dirs)
  "Locate a runtime file named FILENAME.

The file may exist or not under the user Emacs runtime dir (under
XDG-RUNTIME-DIR/emacs).  The path of the file is returned
regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (xdg-get-user-file filename user-emacs-runtime-dir create-parent-dirs))


(defun locate-user-document-file (filename)
  "Locate a document file named FILENAME.

The file may exist or not under the user Emacs documents
dir (which is predetermined by this package using operating
system tools or, if found lacking, sensible defaults).  The path
of the file is returned regardless of it's existence.

If CREATE-PARENT-DIRS is t, then the directory and its parent
dirs will be created in case they are not found."
  (expand-file-name filename user-documents-directory))


(provide 'xdg-directories)
;;; xdg-directories.el ends here

Re: xdg-directories.el

[Stefan Monnier]

> About the stability of the specification: according to the XDG Web site,
> the last version of the specification dates from 2010.  That is six years
> of no modifications.  Seems to be pretty stable now.  Maybe it is time for
> Emacs to start the adoption of the XDG Base Dir Spec and separate the files.

My experience with those kinds of standards is not that they evolve, but
that they get replaced.  6 years is pretty short for Emacs's time scales.

I'm not dead set against it, tho.  I like the idea of standardizing
those things, but I wonder exactly what would be the benefits beyond
following some standard.

Some of the down sides for me is that ~/.emacs.d is a directory where
users are expected to sometimes manually edit files (e.g. the ../init.el,
../gnus.el, ...), whereas my impression is that ~/.config is usually
meant for "hidden configs".  It's much easier to fetch C-x C-f ~/.emacs RET
C-x C-f ~/.config/emacs/init.el RET

But there might be some upsides to following the standard, e.g. some
generic "config management" tools might be able to provide useful
support, for example to sync configs between different hosts.  Tho my
impression is that the XDG standard is much too lax about the actual
format of the files to make such a tool possible, but I might be proved
wrong about that.


        Stefan

Re: xdg-directories.el

[Eli Zaretskii]

> From: "francisco.colaco@gmail.com" <francisco.colaco@gmail.com>
> Date: Tue, 6 Sep 2016 16:24:56 +0100
> 
> I would request that this package (after revision and a possible API change) becomes part of GNU Emacs.  I would also suggest that ~/.config/emacs/init.el (the result of "(locate-user-emacs-config-file "init.el") becomes in vanilla emacs part of the chain to determine user-init-file.

Thanks.  Some comments below.

 . IMO, we need to figure out where this stuff fits into Emacs.  Do
   the XDG places override the traditional Emacs places, or the other
   way around?  Do we even want this, and if so, why?

 . If the XDG places override the traditional ones, an important
   aspect to consider is the transition period: the first Emacs
   version that turns on this feature will need to help users migrate
   from the old places to the new ones.  This requires support code
   that I don't see in your package.

 . The package in its present form "needs work" before it can be
   admitted into Emacs:

   . The few settings that must be preloaded and the minimal support
     code should go to files.el; the rest of the package doesn't have
     to be preloaded, AFAICT.

   . The package currently depends on s.el for a single function; I
     think that dependency should be removed, and standard facilities
     used instead.

   . The usage of shell commands, such as xdg-user-dir, is
     problematic, at least on non-Posix platforms.  I wonder if that
     script is really needed, or how important it is for the overall
     functionality.  AFAICS, the script just accesses some environment
     variables and reads a file, something we could do from Lisp.

   . Symbols (functions, variables) defined by the package should have
     a unique package-specific prefix, in this case probably "xdg-".

   . Maybe it's just me, but I find some of the terminology, and the
     respective variable/function names, confusing, because they clash
     with the long tradition of the Emacs terminology.  For example,
     "user file" has a precise meaning in Emacs, so the purpose of
     xdg-get-user-file is a surprise.  Likewise with "emacs data
     file".  More generally, the naming convention doesn't sound
     consistent: some functions that return file names are called
     locate-SOME-file, but other similar functions are
     xdg-get-SOME-file.

   . The doc strings need various minor fixes, as they include typos
     and copy/paste errors.

   . A minor nit: GNU coding standards frown on using "path" to refer
     to anything but PATH-style directory lists; use "file name" or
     "directory name" instead.

Thanks for working on this.

Re: xdg-directories.el

[francisco.colaco]

[-- Attachment #1: Type: text/plain, Size: 5113 bytes --]

Eli,

I have not created a perfect package, I have just shared something I
personally use, and that works for my purposes (to separate files I want to
backup from those I do not want; and configuration files which I share
among machines from data files which are generated in the machine, like
recentf or Emacs session files).  I emphasize the idea that Emacs must
adhere to the XDG protocol, as almost all other applications are.  Emacs
does so much more now than twenty-five years ago, when I started using it,
and the configuration files have also grown concomitantly.

About the adoption, I think it is a non-issue.  At the very least,
user-emacs-directory should be changed to ~/.local/share/emacs, if it
exists, and keep on being ~/.emacs.d if not.  As far as I know, no package
has "~/.emacs.d" hardwritten, so the transition should be painless.  In the
same spirit, user-emacs-init should look first for (locate-user-config-file
"init.el") then ~/.emacs.d/init.el and then the usual choices, as detailed
in the Emacs manual.

(At this moment, ~/.emacs takes precedence over ~/.emacs.d/init.el, which
is annoying.  Debian and Fedora tend to install a .emacs of their own, and
a pretty useless one.  I have been fooled once, surprised my configuration
in ~/.emacs.d/init.el was not called.)

About some of the useful suggestions you made:

-  s-chomp does replace a longer elisp call chain.  That is why a few
months ago I made the choice to use s, which I can tell is a good library.
I have never made the change to f, though, not because the lack of want,
but because expand-file-name worked well enough.

- I am aware that xdg-user-dir reads the environment variables, which
actually are in the standard.  I guessed, maybe wrongly, at the time, that
xdg-user-dir would be the future-proof standard way of getting the values.
Thinking better, I will change that.

- About the API: I am open to all suggestions.  Change at will, and if you
want, I will provide you with full access to the Github repository!

- About the doc strings: I have to revise those once again.

Thanks for your input.  I will try do address most of the changes today.

  Francisco Colaço


2016-09-07 15:18 GMT+01:00 Eli Zaretskii <eliz@gnu.org>:

> > From: "francisco.colaco@gmail.com" <francisco.colaco@gmail.com>
> > Date: Tue, 6 Sep 2016 16:24:56 +0100
> >
> > I would request that this package (after revision and a possible API
> change) becomes part of GNU Emacs.  I would also suggest that
> ~/.config/emacs/init.el (the result of "(locate-user-emacs-config-file
> "init.el") becomes in vanilla emacs part of the chain to determine
> user-init-file.
>
> Thanks.  Some comments below.
>
>  . IMO, we need to figure out where this stuff fits into Emacs.  Do
>    the XDG places override the traditional Emacs places, or the other
>    way around?  Do we even want this, and if so, why?
>
>  . If the XDG places override the traditional ones, an important
>    aspect to consider is the transition period: the first Emacs
>    version that turns on this feature will need to help users migrate
>    from the old places to the new ones.  This requires support code
>    that I don't see in your package.
>
>  . The package in its present form "needs work" before it can be
>    admitted into Emacs:
>
>    . The few settings that must be preloaded and the minimal support
>      code should go to files.el; the rest of the package doesn't have
>      to be preloaded, AFAICT.
>
>    . The package currently depends on s.el for a single function; I
>      think that dependency should be removed, and standard facilities
>      used instead.
>
>    . The usage of shell commands, such as xdg-user-dir, is
>      problematic, at least on non-Posix platforms.  I wonder if that
>      script is really needed, or how important it is for the overall
>      functionality.  AFAICS, the script just accesses some environment
>      variables and reads a file, something we could do from Lisp.
>
>    . Symbols (functions, variables) defined by the package should have
>      a unique package-specific prefix, in this case probably "xdg-".
>
>    . Maybe it's just me, but I find some of the terminology, and the
>      respective variable/function names, confusing, because they clash
>      with the long tradition of the Emacs terminology.  For example,
>      "user file" has a precise meaning in Emacs, so the purpose of
>      xdg-get-user-file is a surprise.  Likewise with "emacs data
>      file".  More generally, the naming convention doesn't sound
>      consistent: some functions that return file names are called
>      locate-SOME-file, but other similar functions are
>      xdg-get-SOME-file.
>
>    . The doc strings need various minor fixes, as they include typos
>      and copy/paste errors.
>
>    . A minor nit: GNU coding standards frown on using "path" to refer
>      to anything but PATH-style directory lists; use "file name" or
>      "directory name" instead.
>
> Thanks for working on this.
>

[-- Attachment #2: Type: text/html, Size: 6081 bytes --]

Re: xdg-directories.el

[Stefan Monnier]

> -  s-chomp does replace a longer elisp call chain.

It's probably (replace-regexp-in-string "\n\\'" "" str) or something
like that.  I expect you need it because you're using a sub-process
script and need to get rid of its final newline.  Once you replace this
script with Elisp code the need should disappear since the data you'll
get will already be properly "chomp'd".

BTW, rather than shell-command* you're usually better off using
`call-process' when the command you run doesn't need any of shell's
features (e.g. doesn't use pipes, etc...).  The benefit is that you
won't have to worry about the need to shell-quote-argument.

> That is why a few months ago I made the choice to use s, which I can
> tell is a good library.

Yes, we'd like to have it in GNU ELPA.  Help would be welcome for that.


        Stefan

Re: xdg-directories.el

[francisco.colaco]

[-- Attachment #1: Type: text/plain, Size: 6707 bytes --]

  Eli,

I have seen once again the source code for xdg-user-dir, and there is no
environment variables until one runs ~/.config/user-dirs.dirs (or the file
in $XDG_CONFIG_HOME).  That is how, _in the script_, the environment
variables for user directories are set.  They aren't normally set, only get
set for specific packages.

We would either have to read the lines of user-dirs.dirs and get the values
after the = sign and inside the quotes; or execute the file through a
shell, as intended, and read the directory names from the environment
values.  I would not bet on a simple file readout, for one day someone will
end up replacing a directory name by a backticked shell script.

Now, I remember why I have used xdg-user-dirs in the first place for the
user directories: not using it would be reimplementing it in Emacs Lisp, a
needless effort.

If xdg-user-dirs does not exist then there is no point in having a PICTURES
or DOCUMENTS user directory.  In MS Windows the directories are determined
in a totally different method.  In OSX almost all goes to ~/Library,
according to an answer to
http://stackoverflow.com/questions/3373948/equivalents-of-xdg-config-home-and-xdg-data-home-on-mac-os-x

So, to perfect the package in order to have a solid candidate, can you
suggest new function names?

   Francisco Colaço


2016-09-07 15:59 GMT+01:00 francisco.colaco@gmail.com <
francisco.colaco@gmail.com>:

> Eli,
>
> I have not created a perfect package, I have just shared something I
> personally use, and that works for my purposes (to separate files I want to
> backup from those I do not want; and configuration files which I share
> among machines from data files which are generated in the machine, like
> recentf or Emacs session files).  I emphasize the idea that Emacs must
> adhere to the XDG protocol, as almost all other applications are.  Emacs
> does so much more now than twenty-five years ago, when I started using it,
> and the configuration files have also grown concomitantly.
>
> About the adoption, I think it is a non-issue.  At the very least,
> user-emacs-directory should be changed to ~/.local/share/emacs, if it
> exists, and keep on being ~/.emacs.d if not.  As far as I know, no package
> has "~/.emacs.d" hardwritten, so the transition should be painless.  In the
> same spirit, user-emacs-init should look first for (locate-user-config-file
> "init.el") then ~/.emacs.d/init.el and then the usual choices, as detailed
> in the Emacs manual.
>
> (At this moment, ~/.emacs takes precedence over ~/.emacs.d/init.el, which
> is annoying.  Debian and Fedora tend to install a .emacs of their own, and
> a pretty useless one.  I have been fooled once, surprised my configuration
> in ~/.emacs.d/init.el was not called.)
>
> About some of the useful suggestions you made:
>
> -  s-chomp does replace a longer elisp call chain.  That is why a few
> months ago I made the choice to use s, which I can tell is a good library.
> I have never made the change to f, though, not because the lack of want,
> but because expand-file-name worked well enough.
>
> - I am aware that xdg-user-dir reads the environment variables, which
> actually are in the standard.  I guessed, maybe wrongly, at the time, that
> xdg-user-dir would be the future-proof standard way of getting the values.
> Thinking better, I will change that.
>
> - About the API: I am open to all suggestions.  Change at will, and if you
> want, I will provide you with full access to the Github repository!
>
> - About the doc strings: I have to revise those once again.
>
> Thanks for your input.  I will try do address most of the changes today.
>
>   Francisco Colaço
>
>
> 2016-09-07 15:18 GMT+01:00 Eli Zaretskii <eliz@gnu.org>:
>
>> > From: "francisco.colaco@gmail.com" <francisco.colaco@gmail.com>
>> > Date: Tue, 6 Sep 2016 16:24:56 +0100
>> >
>> > I would request that this package (after revision and a possible API
>> change) becomes part of GNU Emacs.  I would also suggest that
>> ~/.config/emacs/init.el (the result of "(locate-user-emacs-config-file
>> "init.el") becomes in vanilla emacs part of the chain to determine
>> user-init-file.
>>
>> Thanks.  Some comments below.
>>
>>  . IMO, we need to figure out where this stuff fits into Emacs.  Do
>>    the XDG places override the traditional Emacs places, or the other
>>    way around?  Do we even want this, and if so, why?
>>
>>  . If the XDG places override the traditional ones, an important
>>    aspect to consider is the transition period: the first Emacs
>>    version that turns on this feature will need to help users migrate
>>    from the old places to the new ones.  This requires support code
>>    that I don't see in your package.
>>
>>  . The package in its present form "needs work" before it can be
>>    admitted into Emacs:
>>
>>    . The few settings that must be preloaded and the minimal support
>>      code should go to files.el; the rest of the package doesn't have
>>      to be preloaded, AFAICT.
>>
>>    . The package currently depends on s.el for a single function; I
>>      think that dependency should be removed, and standard facilities
>>      used instead.
>>
>>    . The usage of shell commands, such as xdg-user-dir, is
>>      problematic, at least on non-Posix platforms.  I wonder if that
>>      script is really needed, or how important it is for the overall
>>      functionality.  AFAICS, the script just accesses some environment
>>      variables and reads a file, something we could do from Lisp.
>>
>>    . Symbols (functions, variables) defined by the package should have
>>      a unique package-specific prefix, in this case probably "xdg-".
>>
>>    . Maybe it's just me, but I find some of the terminology, and the
>>      respective variable/function names, confusing, because they clash
>>      with the long tradition of the Emacs terminology.  For example,
>>      "user file" has a precise meaning in Emacs, so the purpose of
>>      xdg-get-user-file is a surprise.  Likewise with "emacs data
>>      file".  More generally, the naming convention doesn't sound
>>      consistent: some functions that return file names are called
>>      locate-SOME-file, but other similar functions are
>>      xdg-get-SOME-file.
>>
>>    . The doc strings need various minor fixes, as they include typos
>>      and copy/paste errors.
>>
>>    . A minor nit: GNU coding standards frown on using "path" to refer
>>      to anything but PATH-style directory lists; use "file name" or
>>      "directory name" instead.
>>
>> Thanks for working on this.
>>
>
>

[-- Attachment #2: Type: text/html, Size: 8159 bytes --]

Re: xdg-directories.el

[Eli Zaretskii]

> From: "francisco.colaco@gmail.com" <francisco.colaco@gmail.com>
> Date: Wed, 7 Sep 2016 16:31:18 +0100
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
> I have seen once again the source code for xdg-user-dir, and there is no environment variables until one runs
> ~/.config/user-dirs.dirs (or the file in $XDG_CONFIG_HOME). That is how, _in the script_, the environment
> variables for user directories are set. They aren't normally set, only get set for specific packages.
> 
> We would either have to read the lines of user-dirs.dirs and get the values after the = sign and inside the
> quotes; or execute the file through a shell, as intended, and read the directory names from the environment
> values. I would not bet on a simple file readout, for one day someone will end up replacing a directory name by
> a backticked shell script.
> 
> Now, I remember why I have used xdg-user-dirs in the first place for the user directories: not using it would be
> reimplementing it in Emacs Lisp, a needless effort. 
> 
> If xdg-user-dirs does not exist then there is no point in having a PICTURES or DOCUMENTS user directory. In
> MS Windows the directories are determined in a totally different method. In OSX almost all goes to ~/Library,
> according to an answer to
> http://stackoverflow.com/questions/3373948/equivalents-of-xdg-config-home-and-xdg-data-home-on-mac-os-x

If the script is only run to find the PICTURES and DOCUMENTS
directories, then at least on Windows those could be found by
alternative means that don't use any scripts.  (We don't currently
have a Lisp-visible API for that, but it'd be easy to add.)

On OS X, it sounds like we should simply put different default values,
and be done.

> So, to perfect the package in order to have a solid candidate, can you suggest new function names?

For which functions?

Re: xdg-directories.el

[francisco.colaco]

[-- Attachment #1: Type: text/plain, Size: 1422 bytes --]

Eli,

I can implement the functions for Windows and OSX.  I believe all Unices
have implemented the XDG Base Directory Specification.

But we do have a problem: once this is implemented for all operating
systems, XDG will become a meaningless package name and prefix.  Maybe we
should not have a prefix at all and find sensible names not used by Emacs
at the moment.

I propose that xdg-config-dir becomes instead user-emacs-config-directory
and gets to be initialised according to the operating system's best
practices.  Defaulting to ~/.emacs.d/config.  user-emacs-cache-directory
defaults to ~/.emacs.d/cache and user-emacs-data-directory to
~/.emacs.d/data.  The defaults would only be used iff the operating system
determination functions return nil.

user-emacs-directory could either be an alias of user-emacs-data-directory
or default to it.

locate-user-emacs-<domain>-file would then make sense, and no names should
be changed.  I think I should make these functions with the same interface
of locate-user-emacs-file, and give it analog behaviour.

In my opinion, the chain of user-init-file should be (in order of
precedence):

  1. (locate-user-emacs-config-file "init.el")
  2. (locate-user-emacs-file "init.el")
  3. ~/.emacs.d/init.el
  4. ~/.emacs (and the rest of the present order, like terminal specific
files).

Would that seem sensible to you?

  Francisco Colaço

[-- Attachment #2: Type: text/html, Size: 1912 bytes --]

Re: xdg-directories.el

[francisco.colaco]

[-- Attachment #1: Type: text/plain, Size: 150 bytes --]

The dependency on s was removed and I have corrected some docstrings.  The
Github repo is in

https://github.com/francisco-colaco/xdg-directories-el/

[-- Attachment #2: Type: text/html, Size: 289 bytes --]

Re: xdg-directories.el

[Stefan Monnier]

>> I have seen once again the source code for xdg-user-dir, and there is no
>> environment variables until one runs
>> ~/.config/user-dirs.dirs (or the file in $XDG_CONFIG_HOME).

Oh, right.  It looks like a shell is unavoidable, indeed.  So we might
as well run xdg-user-dir instead of re-implementing it (but let's do it
via call-process rather than shell-command*).

> If the script is only run to find the PICTURES and DOCUMENTS
> directories, then at least on Windows those could be found by
> alternative means that don't use any scripts.  (We don't currently
> have a Lisp-visible API for that, but it'd be easy to add.)

Should we use XDG under Windows?
[my gut feeling is "hell, no", but I really have no idea. ]


        Stefan

Re: xdg-directories.el

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 07 Sep 2016 12:46:54 -0400
> 
> > If the script is only run to find the PICTURES and DOCUMENTS
> > directories, then at least on Windows those could be found by
> > alternative means that don't use any scripts.  (We don't currently
> > have a Lisp-visible API for that, but it'd be easy to add.)
> 
> Should we use XDG under Windows?
> [my gut feeling is "hell, no", but I really have no idea. ]

What do you mean by "use XDG"?

We can honor the XDG environment variables, if they are defined (we
already do in a couple of places), and if not, fall back to standard
Windows places.  (Actually, as you might know, Windows _invented_ all
those places in the first place, and freedesktop just copycat that.
Windows had "My Documents", "My Pictures", "My Music", etc. since much
more than the last 6 years.)

My point was that the "standard places" on Windows don't need us to
run a shell script, as on GNU/Linux.

Re: xdg-directories.el

[Eli Zaretskii]

> From: "francisco.colaco@gmail.com" <francisco.colaco@gmail.com>
> Date: Wed, 7 Sep 2016 17:13:43 +0100
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
> I can implement the functions for Windows and OSX.

Thanks.  However, I don't see your copyright assignment on file, so
unless it's in the works, perhaps you shouldn't do that yet.

> But we do have a problem: once this is implemented for all operating systems, XDG will become a
> meaningless package name and prefix. Maybe we should not have a prefix at all and find sensible names not
> used by Emacs at the moment.

I don't follow this logic.  Being able to locate the "My Documents"
directory on Windows has nothing to do with XDG.  The package will
simply use that as fallback on Windows, if XDG_HOME etc. is not found
in the environment.  So the package will still have its place, I
think.

Beyond that, as I wrote earlier, places that are used by core Emacs
features, like user-emacs-directory, will have to be defined in
preloaded files, outside of the XDG package.

> 1. (locate-user-emacs-config-file "init.el")
> 2. (locate-user-emacs-file "init.el")
> 3. ~/.emacs.d/init.el
> 4. ~/.emacs (and the rest of the present order, like terminal specific files).
> 
> Would that seem sensible to you?

I think so, but I'm not an expert on XDG related stuff.

We should also think whether we need to do anything if we find more
than one init.el in these places.

Re: xdg-directories.el

[Eli Zaretskii]

> From: "francisco.colaco@gmail.com" <francisco.colaco@gmail.com>
> Date: Wed, 7 Sep 2016 18:56:45 +0100
> 
> In Windows, it is not so simple. User application data should live in ~/AppData. I Think
> AppData/emacs/<data,config,cache,runtime>/ is a good starting point. Documents and pictures directories,
> however, have their names localised. Thus, we have to dwelve in the evil faded registry to get their names in
> the system, if any other method is not available.

There are existing APIs to retrieve these without looking in the
Registry, see, for example

 https://msdn.microsoft.com/en-us/library/windows/desktop/bb762181(v=vs.85).aspx

>  We should also think whether we need to do anything if we find more
>  than one init.el in these places.
> 
> The first to be found takes precedence and is read, to the exclusion of the postcedent.

I'm not sure I agree.  We might want at least warn the user.