Inclusion of XDG Base Directory library

Inclusion of XDG Base Directory library

[Francisco Miguel Colaço]

[-- Attachment #1: Type: text/plain, Size: 4221 bytes --]

  Friends,

  I have one again written a library to segregate user configuration
files among different concerns --- data, config, runtime and cache ---
following the XDG user directory specification.  I enclose the library.

  From the commentary in user-directories/user-directories.el

;; Following the XDG Base Directory Specification
;; (https://wiki.archlinux.org/index.php/XDG_Base_Directory_support),
;; user-directories strives to allow a package writer or an Emacs user
;; to segregate the various Emacs configuration and runtime files into
;; appropriate directories.
;;
;; Thus, following the specification, a configuration file (normally a
;; file that is written by the user or in behalf of him) will reside
;; in the configuration directory (normally ~/.config/emacs).  It ca
;; be located with:
;;
;;   (locate-user-config-file "init.el")
;;   "/home/fhc/.config/emacs/init.el"
;;
;; Similarly, there are locator functions for third party files ---
;; `locate-user-data-file' --- and for cache files ---
;; `locate-user-cache-file'.  Run time files, locatable with
;; `locate-user-runtime-files' will be erased at the last logout,
;; according to the specification, and are thus appropriate for
;; security-sensitive data.
;;
;; There are also directories defined for :documents, :music, :videos,
;; :pictures and :downloads.  In Linux, also other that the command
;; xdg-user-dir can reference, like :templates or :publicshare.
;; Locator functions are concomitantly created.  For instance, in one
;; of my machines (Linux, portuguese):
;;
;;   (locate-user-documents-file "org/index.txt")
;;   "/home/fhc/Documentos/org/index.txt"
;;
;;   (locate-user-pictures-file "image.png")
;;   "/home/fhc/Imagens/image.png"
;;
;; Emacs Lisp package writers may use the former functions to
;; segregate files among different concerns.  For instance:
;;
;;   (setq abbrev-file-name (locate-user-config-file "abbrev_defs"))
;;   (setq projectile-known-projects-file (locate-user-data-file
"projectile-bookmarks.eld"))
;;   (setq projectile-cache-file (locate-user-cache-file
"projectile.cache"))
;;

* HOW DO I INSTALL IT?

  Just including the two directories in a site-lisp directory is enough
on my machine to recognize ~/.config/emacs/init.el and run it at the
site start.

  Unfortunately, user-init-file will not be set, because I think it is
hardcoded in C from an array of possibilities.  However,
user-emacs-directory is correctly set, and thus all the files that would
be in ~/.emacs.d will be located by default at the :data directory ---
and elisp package writers can start to put all created files to the
correct directories, according to their concerns.

* WHAT ABOUT FREEBSD, OSX, ETC?

  Discovery is provided on Windows and Linux.  The value at system-type
will determine which discovery provider is loaded.  A value of gnu/linux
will load user-directories-linux.el and a value of windows-nt will load
user-directories-windows-nt.el.

  If one wants to add a file named user-directories-darwin.el or
user-directories-kfreebsd.el, patterned after user-directories-linux.el
or user-directories-windows-nt.el, then the directories for the two
systems would be correctly discovered.  As a cavaet emptor, I haven't
tested any windows-nt discovery, since I do not use Emacs on Windows. 
If the value system-type has no file to provide discoveries, all the
:data, :cache, :runtime and :config directories will be created, but
this time inside user-emacs-directory.

  I have assigned the copyright to the Free Software Foundation.

* WHAT CAN STILL BE DONE?

  - user-directories/user-directories-darwin.el;

  - user-directories/user-directories-kfreebsd.el;

  - improve setup-user-directories-default, if the layout specified at
that function is not sensible

  - make user-init-file writable during initialisation --- while one
loads the XDG file, and tries to set user-init-file at
user-directories-init.el, the value reverts to ~/.emacs for no reason. 
Help needed here.


  Thanks for your time.

    Best wishes,

      Francisco Colaço

 


[-- Attachment #2: user-directories.tar.bz2 --]
[-- Type: application/x-bzip, Size: 5883 bytes --]

[-- Attachment #3: 0xDB83911F.asc --]
[-- Type: application/pgp-keys, Size: 12199 bytes --]

Re: Inclusion of XDG Base Directory library

[Francisco Miguel Colaço]

  Friends,

  I have set a public github repository at:

    https://github.com/francisco-colaco/user-directories

  Please feel free to contribute there or to raise issues.

  Best regards,

    Francisco Colaço

Re: Inclusion of XDG Base Directory library

[Stefan Monnier]

>   I have set a public github repository at:
>     https://github.com/francisco-colaco/user-directories
>   Please feel free to contribute there or to raise issues.

I think it'd be nice to have this as a GNU ELPA package.  But I guess
there's some questions as to how it should be installed in order to
avoid needing a ~/.emacs.d directory at all.

The "HOW DO I INSTALL IT?" section wasn't clear enough for me to know
how it's currently meant to be installed.  It says:

      Just including the two directories in a site-lisp directory is enough
    on my machine to recognize ~/.config/emacs/init.el and run it at the
    site start.

I'm glad it works for you, but I don't know what "a site-lisp
directory" is.  [ I can guess what you mean by "the two directories",
tho it is a bit vague.  ]

I wonder how/why it works for you, tho, because I don't see any code in
Emacs which will look at something named "site-start.d" by default.

Could it be that your site-start.d gets used by some code provided by
your distribution (e.g. I see some site-start.d directories used by
Debian's Emacs package)?

      Unfortunately, user-init-file will not be set, because I think it is
    hardcoded in C from an array of possibilities.

No, it's set in lisp/startup.el.


        Stefan

Re: Inclusion of XDG Base Directory library

[Eli Zaretskii]

> From: Francisco Miguel Colaço <francisco.colaco@gmail.com>
> Date: Mon, 7 May 2018 09:26:36 +0100
> 
>   Friends,
> 
>   I have set a public github repository at:
> 
>     https://github.com/francisco-colaco/user-directories
> 
>   Please feel free to contribute there or to raise issues.

The code calls windows-shell-folder, but I cannot find that function
anywhere.  What did I miss?

The environment variable LOCALAPPDATA is not defined on my system.
Why do we need it?

I also think we need not override the code which already computes
where to place the temporary directory.

On GNU/Linux, this calls an external program; can we avoid that?

Thanks.

Re: Inclusion of XDG Base Directory library

[Francisco Miguel Colaço]

  Eli,

  Thanks for the heads up.  The windows-shell-folder and
windows-read-registry-value, which I thought were part of Windows Emacs,
were added to user-directories-windows-nt.el.

  I have not tested Windows NT code because I do not currently own any
Windows computer and the virtual machines on which I run it do not have
Emacs.  So, I am dependent on the goodwill and patches from the many
that use Emacs on Microsoft platforms.

  According to http://environmentvariables.org/LocalAppData, the
variable was introduced in Windows Vista and defaults to
C:\Users\{username}\AppData\Local.  I have used it on some applications
I wrote a long time ago and seem to remember that the purpose of that
directory is to store temporary files, which should not be shared among
machines and pertain only to the local session.  If there is no value in
having that variable evaluated, then there is no obstacles to redefine
the code and drop it --- or even get the temp files code from the
existing code.

  About Linux, the canonical way to get an XDG user folder is by the use
of the command xdg-user-dir.  The alternative is to parse
~/.config/user-dirs.dirs, something that is already done by
xdg-user-dir.  Here are two lines of mine (European Portuguese):

  XDG_DESKTOP_DIR="$HOME/Área de Trabalho"
  XDG_DOWNLOAD_DIR="$HOME/Transferências"

  Although it does not pose a gruesome task to parse it (blessed be
regexps), as far as I can understand the XDG Base Directory Standard the
recommended way to find directories is not to parse the file, but to use
the command.  Every distribution since near 2010 includes
xdg-user-dirs.  I can write parsing code, but it seems inadequate,
considering there is a blessed way to perform the search.  It is akin to
use 'movemail' to send and receive mail or 'git' by Magit: there is
little value added in the effort of reimplementing that which is already
functional, something that would leave a dangerous point of stagnation
or even breakage at the future.

  Thanks for your comments.

  Best regards,

    Francisco Colaço


Às 13:26 de 11-05-2018, Eli Zaretskii escreveu:
>> From: Francisco Miguel Colaço <francisco.colaco@gmail.com>
>> Date: Mon, 7 May 2018 09:26:36 +0100
>>
>>   Friends,
>>
>>   I have set a public github repository at:
>>
>>     https://github.com/francisco-colaco/user-directories
>>
>>   Please feel free to contribute there or to raise issues.
> The code calls windows-shell-folder, but I cannot find that function
> anywhere.  What did I miss?
>
> The environment variable LOCALAPPDATA is not defined on my system.
> Why do we need it?
>
> I also think we need not override the code which already computes
> where to place the temporary directory.
>
> On GNU/Linux, this calls an external program; can we avoid that?
>
> Thanks.

Re: Inclusion of XDG Base Directory library

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Francisco Miguel Colaço <francisco.colaco@gmail.com>
> Date: Fri, 11 May 2018 21:22:37 +0100
> 
>   Thanks for the heads up.  The windows-shell-folder and
> windows-read-registry-value, which I thought were part of Windows Emacs,
> were added to user-directories-windows-nt.el.

Thanks.

>   I have not tested Windows NT code because I do not currently own any
> Windows computer and the virtual machines on which I run it do not have
> Emacs.  So, I am dependent on the goodwill and patches from the many
> that use Emacs on Microsoft platforms.

I understand.  Therefore, I raised an issue with the package, listing
there the problems I see in the code, and I hope the person(s) who
contributed this code will take notice and fix the problems I
mentioned there.

(Btw, I suggest to keep track of all your contributors, otherwise we
will have difficulties admitting the code into ELPA later.)

>   According to http://environmentvariables.org/LocalAppData, the
> variable was introduced in Windows Vista and defaults to
> C:\Users\{username}\AppData\Local.

That's exactly my point: Emacs on Windows still tries to support
versions of Windows that predate Vista.  The code should not fail for
them, but should provide fallbacks for those directories that were
introduced in more recent versions.

In addition, I think it's gross to use 'reg' and 'echo' as external
commands to produce the Registry values; in particular, this will fail
if the values include non-ASCII characters not from the system locale.
We should have primitives for that.  Volunteers are welcome to add
such capabilities to Emacs.

>   About Linux, the canonical way to get an XDG user folder is by the use
> of the command xdg-user-dir.  The alternative is to parse
> ~/.config/user-dirs.dirs, something that is already done by
> xdg-user-dir.  Here are two lines of mine (European Portuguese):
> 
>   XDG_DESKTOP_DIR="$HOME/Área de Trabalho"
>   XDG_DOWNLOAD_DIR="$HOME/Transferências"
> 
>   Although it does not pose a gruesome task to parse it (blessed be
> regexps), as far as I can understand the XDG Base Directory Standard the
> recommended way to find directories is not to parse the file, but to use
> the command.  Every distribution since near 2010 includes
> xdg-user-dirs.  I can write parsing code, but it seems inadequate,
> considering there is a blessed way to perform the search.

IMO, the advantage of parsing files in Emacs is that we can be more
sure we deal properly with non-ASCII file names.  When you invoke a
command to do the job for you, you rely on guesswork for decoding what
the program outputs, which could be tricky, especially when you do
that remotely from another system, with a different locale.  By
contrast, I expect the file to be encoded in UTF-8 always, is that
right?

Re: Inclusion of XDG Base Directory library

[Francisco Miguel Colaço]

  Eli,

  Thanks for your reply.  I did my best to address your concerns and
have commited code at Github.

    Francisco


Às 07:52 de 12-05-2018, Eli Zaretskii escreveu
> I understand.  Therefore, I raised an issue with the package, listing
> there the problems I see in the code, and I hope the person(s) who
> contributed this code will take notice and fix the problems I
> mentioned there.

Years ago, I have slightly adapted code I have seen on the Internet to
write the two functions (windows-read-registry-value and
windows-shell-folders).  I will have to locate where the code was shown
(maybe on Stackoverflow, I sincerely do not remember where).

Actually, I do not think those two should be in this library, since
there is a broader utility and scope to be considered.  Maybe
windows-read-registry-value should be a C primitive or a loadable module
--- now that we can have them in Emacs.

What do you think of factoring these two functions away from this
library, creating an also dynamically configurable os.el library not
unlike this one?That's exactly my point: Emacs on Windows still tries to
support
> versions of Windows that predate Vista.  The code should not fail for
> them, but should provide fallbacks for those directories that were
> introduced in more recent versions.
Done.  if not found, the value of LOCALAPPDATA will not be used, but
that of APPDATA, which, as far as I know, is even present in Windows XP.

> In addition, I think it's gross to use 'reg' and 'echo' as external
> commands to produce the Registry values; in particular, this will fail
> if the values include non-ASCII characters not from the system locale.
> We should have primitives for that.  Volunteers are welcome to add
> such capabilities to Emacs.
Can any Windows user tackle the subject?  Anyone can contribute, and the
more, the merrier! :-)

I really think, as I have said above, that these particular functions
should be taken from this library and to a library of their own.
 
>>  
>> IMO, the advantage of parsing files in Emacs is that we can be more
>> sure we deal properly with non-ASCII file names.  When you invoke a
>> command to do the job for you, you rely on guesswork for decoding what
>> the program outputs, which could be tricky, especially when you do
>> that remotely from another system, with a different locale.  By
>> contrast, I expect the file to be encoded in UTF-8 always, is that
>> right?
Actually, Portuguese does have diacritics, like "Transferências" and
"Área de Trabalho" and "Vídeos" and no problems have arisen to me so
far.  Maybe our users Arabic, Hebrew, Cyrillic. Indic and CJK scripts
can see if it works on their machine.  XDG does mandate UTF-8, so I
reckon that we will have no problems by using the file.

On another record, if the blessed way is to use the command
'xdg-user-dir', how can one be sure they will not alter the file format
to suit some future expansion desiderata?  The regexp way is quite
simple, but we are on the very dangerous assumption that the line format
is set in stone and will never change.  Nevertheless, if we get XDG
people to recommend us the direct parsing of ~/.config/user-dirs.dirs,
then we should definitely implement it that way.

Re: Inclusion of XDG Base Directory library

[Stefan Monnier]

>   About Linux, the canonical way to get an XDG user folder is by the use
> of the command xdg-user-dir.  The alternative is to parse
> ~/.config/user-dirs.dirs, something that is already done by
> xdg-user-dir.  Here are two lines of mine (European Portuguese):

I think it makes more sense to use the command (and just skip the whole
XDG thing if it's not found).


        Stefan