[PATCH] checking eww download directory

[PATCH] checking eww download directory

[Mark Oteiza]

Two things:

- it would be nice if eww looked a little harder for an existing
  downloads directory.  Perhaps what's below is too much, comments
  welcome
- currently if eww-download-directory isn't accessible, emacs will
  download a file and the sentinel will fail, and one is left with a
  hidden buffer with the downloaded contents and no saved file.  Easy
  solution: fail early by checking with access-file

diff --git a/lisp/net/eww.el b/lisp/net/eww.el
index 0282fe68e6..6c62729849 100644
--- a/lisp/net/eww.el
+++ b/lisp/net/eww.el
@@ -55,9 +55,20 @@ eww-search-prefix
   :group 'eww
   :type 'string)
 
-(defcustom eww-download-directory "~/Downloads/"
+(defcustom eww-download-directory
+  (cond
+   ((memq system-type '(ms-dos windows-nt cygwin))
+    (expand-file-name "Downloads" (getenv "USERPROFILE")))
+   ((cl-some
+     (lambda (str)
+       (let ((f (expand-file-name str "~")))
+         (and (file-directory-p f)
+              (string-match-p "downloads?" (downcase f))
+              (abbreviate-file-name f))))
+     (directory-files "~")))
+   (t "~/Downloads/"))
   "Directory where files will downloaded."
-  :version "24.4"
+  :version "26.1"
   :group 'eww
   :type 'directory)
 
@@ -1501,6 +1512,7 @@ eww-copy-page-url
 (defun eww-download ()
   "Download URL under point to `eww-download-directory'."
   (interactive)
+  (access-file eww-download-directory "Download failed")
   (let ((url (get-text-property (point) 'shr-url)))
     (if (not url)
         (message "No URL under point")

Re: [PATCH] checking eww download directory

[Lars Ingebrigtsen]

Mark Oteiza <mvoteiza@udel.edu> writes:

> - it would be nice if eww looked a little harder for an existing
>   downloads directory.  Perhaps what's below is too much, comments
>   welcome

[...]

> -(defcustom eww-download-directory "~/Downloads/"
> +(defcustom eww-download-directory
> +  (cond
> +   ((memq system-type '(ms-dos windows-nt cygwin))
> +    (expand-file-name "Downloads" (getenv "USERPROFILE")))
> +   ((cl-some
> +     (lambda (str)
> +       (let ((f (expand-file-name str "~")))
> +         (and (file-directory-p f)
> +              (string-match-p "downloads?" (downcase f))
> +              (abbreviate-file-name f))))
> +     (directory-files "~")))
> +   (t "~/Downloads/"))

Yikes.  There must be a gazillion places directories are defined under
the home director.  Do they all do this dance?  If they do, perhaps it
should be made into a library function?

> +  (access-file eww-download-directory "Download failed")

Seem OK, except that the error message should say what the error is
instead of this.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH] checking eww download directory

[Lars Ingebrigtsen]

Mark Oteiza <mvoteiza@udel.edu> writes:

>   and the sentinel will fail, and one is left with a
>   hidden buffer with the downloaded contents and no saved file.

(This has been fixed on the with-url branch.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH] checking eww download directory

[Clément Pit-Claudel]

On 2017-01-27 14:10, Mark Oteiza wrote:
> - it would be nice if eww looked a little harder for an existing
>   downloads directory.  Perhaps what's below is too much, comments
>   welcome

This sounds like a great idea!  And I'd say what's below is not enough, actually :)  For example, it fails in locales that are not English (on French versions of Ubuntu, the Downloads folder is "Téléchargements"), or if users manually customized the name of that folder (I use ~/dls).

How hard would it be to integrate with the xdg-user-dirs standard instead? On many GNU/Linux systems, running "xdg-user-dir DOWNLOAD" is enough (http://stackoverflow.com/questions/13161226/).  This would make eww consistent with Firefox, too (they take the slightly more painful route of parsing ~/.config/user-dirs.dirs: https://dxr.mozilla.org/mozilla-release/source/xpcom/io/SpecialSystemDirectory.cpp#256)

Cheers,
Clément.

Re: [PATCH] checking eww download directory

[Mark Oteiza]

On 27/01/17 at 08:21pm, Lars Ingebrigtsen wrote:
> Mark Oteiza <mvoteiza@udel.edu> writes:
> 
> > - it would be nice if eww looked a little harder for an existing
> >   downloads directory.  Perhaps what's below is too much, comments
> >   welcome
> 
> [...]
> 
> > -(defcustom eww-download-directory "~/Downloads/"
> > +(defcustom eww-download-directory
> > +  (cond
> > +   ((memq system-type '(ms-dos windows-nt cygwin))
> > +    (expand-file-name "Downloads" (getenv "USERPROFILE")))
> > +   ((cl-some
> > +     (lambda (str)
> > +       (let ((f (expand-file-name str "~")))
> > +         (and (file-directory-p f)
> > +              (string-match-p "downloads?" (downcase f))
> > +              (abbreviate-file-name f))))
> > +     (directory-files "~")))
> > +   (t "~/Downloads/"))
> 
> Yikes.  There must be a gazillion places directories are defined under
> the home director.  Do they all do this dance?  If they do, perhaps it
> should be made into a library function?

I just went off of knowing it's called Downloads in windows, and trying
to consider plural/nonplural and case insensitivity on Linux.  Mine, for
instance is just lower cased.  So... the cl-some is probably overkill.

This is something I've thought about and I know I've seen johnw kick the
idea around somewhere, but it would take some
work to generalize it.  Just one example of what such a library might
read is user-dirs.dirs for XDG thingies:

https://www.freedesktop.org/wiki/Software/xdg-user-dirs/

It's a shell-looking file with lines like

  XDG_DOWNLOAD_DIR="$HOME/Downloads"

> > +  (access-file eww-download-directory "Download failed")
> 
> Seem OK, except that the error message should say what the error is
> instead of this.

It does, access-file appends the error:

  Download failed: No such file or directory, ~/NOWOAIWJD

Re: [PATCH] checking eww download directory

[Eli Zaretskii]

> Date: Fri, 27 Jan 2017 14:48:16 -0500
> From: Mark Oteiza <mvoteiza@udel.edu>
> Cc: emacs-devel@gnu.org
> 
> I just went off of knowing it's called Downloads in windows

The Downloads directory exists on Windows only since Windows 7, it's
not there on older versions.

Re: [PATCH] checking eww download directory

[Mark Oteiza]

Clément Pit-Claudel <cpitclaudel@gmail.com> writes:

> On 2017-01-27 14:10, Mark Oteiza wrote:
>> - it would be nice if eww looked a little harder for an existing
>>   downloads directory.  Perhaps what's below is too much, comments
>>   welcome
>
> This sounds like a great idea!  And I'd say what's below is not
> enough, actually :) For example, it fails in locales that are not
> English (on French versions of Ubuntu, the Downloads folder is
> "Téléchargements"), or if users manually customized the name of that
> folder (I use ~/dls).
>
> How hard would it be to integrate with the xdg-user-dirs standard
> instead? On many GNU/Linux systems, running "xdg-user-dir DOWNLOAD" is
> enough (http://stackoverflow.com/questions/13161226/).  This would
> make eww consistent with Firefox, too (they take the slightly more
> painful route of parsing ~/.config/user-dirs.dirs:
> https://dxr.mozilla.org/mozilla-release/source/xpcom/io/SpecialSystemDirectory.cpp#256)

An xdg.el would be nice to have for elisp that interfaces
with XDG-following things.  mpc.el and image-dired.el are two files that
come to mind: mpd follows [0], image-dired
supports [1].  Actually, it looks like those are the only two files with
XDG bits, and I put them there.  How about that.

Anyways, these aren't difficult to implement.  If it would be a welcome
addition I'd have a go at writing it.

[0] https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
[1] https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html

P.S. Please Cc:

Re: [PATCH] checking eww download directory

[Clément Pit-Claudel]

On 2017-01-27 15:17, Mark Oteiza wrote:
> Anyways, these aren't difficult to implement.  If it would be a welcome
> addition I'd have a go at writing it.

I think that would be very nice :)

Re: [PATCH] checking eww download directory

[Eli Zaretskii]

> From: Mark Oteiza <mvoteiza@udel.edu>
> Date: Fri, 27 Jan 2017 15:17:10 -0500
> Cc: emacs-devel@gnu.org
> 
> > How hard would it be to integrate with the xdg-user-dirs standard
> > instead? On many GNU/Linux systems, running "xdg-user-dir DOWNLOAD" is
> > enough (http://stackoverflow.com/questions/13161226/).  This would
> > make eww consistent with Firefox, too (they take the slightly more
> > painful route of parsing ~/.config/user-dirs.dirs:
> > https://dxr.mozilla.org/mozilla-release/source/xpcom/io/SpecialSystemDirectory.cpp#256)
> 
> An xdg.el would be nice to have for elisp that interfaces
> with XDG-following things.  mpc.el and image-dired.el are two files that
> come to mind: mpd follows [0], image-dired
> supports [1].  Actually, it looks like those are the only two files with
> XDG bits, and I put them there.  How about that.
> 
> Anyways, these aren't difficult to implement.  If it would be a welcome
> addition I'd have a go at writing it.
> 
> [0] https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
> [1] https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html

This has come up before.  I think this would be welcome, but one issue
that didn't get resolved is how much of that should be automatic
(i.e. should Emacs automatically search certain directories for
certain kinds of files), and if so, how to split the imaginary xdg.el
into two parts, with the automatic part living in some preloaded file
(probably files.el).  There's also the issue of whether users and/or
Lisp programs should be able to disable this search (e.g., by binding
some variable).

Also, IMO supporting this on Windows will need a new primitive written
in C, but that's an aside, and can be omitted in the initial
implementation.

Thanks.

Re: [PATCH] checking eww download directory

[Yuri Khan]

On Sat, Jan 28, 2017 at 3:10 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> The Downloads directory exists on Windows only since Windows 7, it's
> not there on older versions.

The SHGetKnownFolderPath API function is supported since Vista,
though. The politically correct way on Windows would probably be to
try to call that with FOLDERID_Downloads, although since Windows 7
Microsoft dropped the dubious practice of localizing directory names
on the file system, replacing it with the equally dubious practice of
replacing actual directory names with localized strings in the shell
UI instead.

Re: [PATCH] checking eww download directory

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sat, 28 Jan 2017 15:01:23 +0700
> Cc: Mark Oteiza <mvoteiza@udel.edu>, Lars Magne Ingebrigtsen <larsi@gnus.org>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> On Sat, Jan 28, 2017 at 3:10 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > The Downloads directory exists on Windows only since Windows 7, it's
> > not there on older versions.
> 
> The SHGetKnownFolderPath API function is supported since Vista,
> though.

I'd prefer using SHGetFolderPath (which Emacs already uses), as its
availability is almost universal on the supported versions of the OS.
Probably in conjunction with CSIDL_PROFILE.

> The politically correct way on Windows would probably be to
> try to call that with FOLDERID_Downloads, although since Windows 7
> Microsoft dropped the dubious practice of localizing directory names
> on the file system, replacing it with the equally dubious practice of
> replacing actual directory names with localized strings in the shell
> UI instead.

My point was where do you put stuff if Downloads doesn't exist.  That
is separate from the issue of finding the directory when it does
exist.

Re: [PATCH] checking eww download directory

[Lars Ingebrigtsen]

Mark Oteiza <mvoteiza@udel.edu> writes:

> I just went off of knowing it's called Downloads in windows, and trying
> to consider plural/nonplural and case insensitivity on Linux.  Mine, for
> instance is just lower cased.  So... the cl-some is probably overkill.

I think it's a good idea to try to find the "downloads" directory on the
host system, but I think this should be factored out into its own
function, so that we can use it in similar circumstances elsewhere.  

Or perhaps there aren't any other directories like that?  Hm...  well,
"documents", perhaps, and "desktop"...

>> > +  (access-file eww-download-directory "Download failed")
>> 
>> Seem OK, except that the error message should say what the error is
>> instead of this.
>
> It does, access-file appends the error:
>
>   Download failed: No such file or directory, ~/NOWOAIWJD

Oh, cool.  Perhaps this line from the doc string could be clearer:

"The second argument STRING is used in the error message."

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [PATCH] checking eww download directory

[Mark Oteiza]

On 28/01/17 at 03:14pm, Lars Ingebrigtsen wrote:
> Mark Oteiza <mvoteiza@udel.edu> writes:
> 
> > I just went off of knowing it's called Downloads in windows, and trying
> > to consider plural/nonplural and case insensitivity on Linux.  Mine, for
> > instance is just lower cased.  So... the cl-some is probably overkill.
> 
> I think it's a good idea to try to find the "downloads" directory on the
> host system, but I think this should be factored out into its own
> function, so that we can use it in similar circumstances elsewhere.  

Fair, I'll leave it.

> Or perhaps there aren't any other directories like that?  Hm...  well,
> "documents", perhaps, and "desktop"...
> 
> >> > +  (access-file eww-download-directory "Download failed")
> >> 
> >> Seem OK, except that the error message should say what the error is
> >> instead of this.
> >
> > It does, access-file appends the error:
> >
> >   Download failed: No such file or directory, ~/NOWOAIWJD
> 
> Oh, cool.  Perhaps this line from the doc string could be clearer:

Yep, useful function.  I changed the docstring words to "prepended to"

Library for XDG things (was Re: [PATCH] checking eww download directory)

[Mark Oteiza]

On 28/01/17 at 09:53am, Eli Zaretskii wrote:
> > From: Mark Oteiza <mvoteiza@udel.edu>
> > Date: Fri, 27 Jan 2017 15:17:10 -0500
> > Cc: emacs-devel@gnu.org
> > 
> > > How hard would it be to integrate with the xdg-user-dirs standard
> > > instead? On many GNU/Linux systems, running "xdg-user-dir DOWNLOAD" is
> > > enough (http://stackoverflow.com/questions/13161226/).  This would
> > > make eww consistent with Firefox, too (they take the slightly more
> > > painful route of parsing ~/.config/user-dirs.dirs:
> > > https://dxr.mozilla.org/mozilla-release/source/xpcom/io/SpecialSystemDirectory.cpp#256)
> > 
> > An xdg.el would be nice to have for elisp that interfaces
> > with XDG-following things.  mpc.el and image-dired.el are two files that
> > come to mind: mpd follows [0], image-dired
> > supports [1].  Actually, it looks like those are the only two files with
> > XDG bits, and I put them there.  How about that.
> > 
> > Anyways, these aren't difficult to implement.  If it would be a welcome
> > addition I'd have a go at writing it.
> > 
> > [0] https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
> > [1] https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html
> 
> This has come up before.  I think this would be welcome, but one issue
> that didn't get resolved is how much of that should be automatic
> (i.e. should Emacs automatically search certain directories for
> certain kinds of files), and if so, how to split the imaginary xdg.el
> into two parts, with the automatic part living in some preloaded file
> (probably files.el).  There's also the issue of whether users and/or
> Lisp programs should be able to disable this search (e.g., by binding
> some variable).

Sounds like a daunting goal.  I imagine having XDG bits preloaded would
only really be necessary if Emacs decided it was going to follow the
base directory spec (and probably only on GNU/Linux).

In the mean time, the following is a small library of some useful
functions. At least for the base directory spec it remains rather low
level as the standard gives a lot of freedom to how applications want to
handle multiple locations of config/data:

;;; xdg.el --- XDG specification and standard library -*- lexical-binding: t -*-

;; Copyright (C) 2017  Mark Oteiza <mvoteiza@udel.edu>

;; Author: Mark Oteiza <mvoteiza@udel.edu>
;; Keywords: files, data

;; This program is free software; you can redistribute it and/or
;; modify it under the terms of the GNU General Public License
;; as published by the Free Software Foundation; either version 3
;; of the License, or (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program. If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;; Library providing some convenience functions for the following XDG
;; standards and specifications
;;
;; - XDG Base Directory Specification
;; - Thumbnail Managing Standard
;; - xdg-user-dirs configuration

;;; Code:

\f
;; XDG Base Directory Specification
;; https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

(defmacro xdg--dir-home (environ default-path)
  (declare (debug (stringp stringp)))
  (let ((env (make-symbol "env")))
    `(let ((,env (getenv ,environ)))
       (if (or (null ,env) (not (file-name-absolute-p ,env)))
           (expand-file-name ,default-path)
         ,env))))

(defun xdg-config-home ()
  "Return the base directory for user specific configuration files."
  (xdg--dir-home "XDG_CONFIG_HOME" "~/.config"))

(defun xdg-cache-home ()
  "Return the base directory for user specific cache files."
  (xdg--dir-home "XDG_CACHE_HOME" "~/.cache"))

(defun xdg-data-home ()
  "Return the base directory for user specific data files."
  (xdg--dir-home "XDG_DATA_HOME" "~/.local/share"))

(defun xdg-runtime-dir ()
  "Return the value of $XDG_RUNTIME_DIR."
  (getenv "XDG_RUNTIME_DIR"))

(defun xdg-config-dirs ()
  "Return the config directory search path as a list."
  (let ((env (getenv "XDG_CONFIG_DIRS")))
    (if (or (null env) (string= env ""))
        '("/etc/xdg")
      (parse-colon-path env))))

(defun xdg-data-dirs ()
  "Return the data directory search path as a list."
  (let ((env (getenv "XDG_CONFIG_DIRS")))
    (if (or (null env) (string= env ""))
        '("/usr/local/share/" "/usr/share/")
      (parse-colon-path env))))

\f
;; Thumbnail Managing Standard
;; https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html

(defun xdg-thumb-uri (filename)
  "Return the canonical URI for FILENAME.
If FILENAME has absolute path /foo/bar.jpg, its canonical URI is
file:///foo/bar.jpg"
  (concat "file://" (expand-file-name filename)))

(defun xdg-thumb-name (filename)
  "Return the appropriate thumbnail filename for FILENAME."
  (concat (md5 (xdg-thumb-uri filename)) ".png"))

(defun xdg-thumb-mtime (filename)
  "Return modification time of FILENAME as integral seconds from the epoch."
  (floor (float-time (nth 5 (file-attributes filename)))))

\f
;; XDG User Directories
;; https://www.freedesktop.org/wiki/Software/xdg-user-dirs/

(defconst xdg-line-regexp
  (eval-when-compile
    (rx "XDG_"
        (group-n 1 (or "DESKTOP" "DOWNLOAD" "TEMPLATES" "PUBLICSHARE"
                       "DOCUMENTS" "MUSIC" "PICTURES" "VIDEOS"))
        "_DIR=\""
        (group-n 2 (or "/" "$HOME/") (*? (or (not (any "\"")) "\\\"")))
        "\""))
  "Regexp matching non-comment lines in xdg-user-dirs config files.")

(defvar xdg-user-dirs nil
  "Alist of directory keys and values.")

(defun xdg--user-dirs-parse-line ()
  "Return pair of user-dirs key to directory value in LINE, otherwise nil.
This should be called at the beginning of a line."
  (skip-chars-forward "[:blank:]")
  (when (and (/= (following-char) ?#)
             (looking-at xdg-line-regexp))
    (let ((k (match-string 1))
          (v (match-string 2)))
      (when (and k v) (cons k v)))))

(defun xdg--user-dirs-parse-file (filename)
  "Return alist of xdg-user-dirs from FILENAME."
  (let (elt res)
    (with-temp-buffer
      (insert-file-contents filename)
      (goto-char (point-min))
      (while (not (eobp))
        (setq elt (xdg--user-dirs-parse-line))
        (when (consp elt) (push elt res))
        (forward-line)))
    res))

(defun xdg-user-dir (name)
  "Return the path of user directory referred to by NAME."
  (when (null xdg-user-dirs)
    (setq xdg-user-dirs
          (xdg--user-dirs-parse-file
           (expand-file-name "user-dirs.dirs" (xdg-config-home)))))
  (cdr (assoc name xdg-user-dirs)))

(provide 'xdg)

;;; xdg.el ends here

Re: Library for XDG things (was Re: [PATCH] checking eww download directory)

[Ted Zlatanov]

On Sat, 28 Jan 2017 20:01:50 -0500 Mark Oteiza <mvoteiza@udel.edu> wrote: 

MO> In the mean time, the following is a small library of some useful
MO> functions. At least for the base directory spec it remains rather low
MO> level as the standard gives a lot of freedom to how applications want to
MO> handle multiple locations of config/data:

I think this would be an excellent addition to Emacs.

Ted