bug#21702: shell-quote-argument semantics and safety

[taylanbayirli@gmail.com (Taylan Ulrich Bayırlı/Kammer)]

[-- Attachment #1: Type: text/plain, Size: 3446 bytes --]

The documentation of shell-quote-argument only says

    Quote ARGUMENT for passing as argument to an inferior shell.

It's unclear for which shells this is supposed to work.  In a recent
thread in emacs-devel, it has been demonstrated that if the result is
passed to csh, it can allow an attacker to execute an arbitrary shell
command, although without arguments:

    (let ((argument (read untrusted-source)))
      (assert (stringp argument))
      (call-process "csh" nil t nil "-c"
                    (concat "echo " (shell-quote-argument argument))))

    ;; If untrusted-source gives us "\nevil-command\n", we get:
    ;; evil-command: Command not found.

The function should clearly document

    1) for which shells will the quoting work absolutely, i.e. lead to
    the given string to appear *verbatim* in an element of the ARGV of
    the called command,

    2) optionally, for which shells will the quoting at least prevent
    code injection,

    3) optionally, for which shells and character sets for ARGUMENT will
    the quoting work absolutely,

    4) optionally, for which shells and character sets for ARGUMENT will
    the quoting at least prevent code injection,

    5) optionally, for which shells will the quoting work at all even if
    it provides no clear semantics, such that one can at least use it
    with data coming from trusted sources (e.g. other parts of Emacs's
    source code, or the user sitting in front of Emacs), where it's the
    user's/programmer's responsibility to stick to values for ARGUMENT
    that are intuitively known to be unproblematic even if the character
    set isn't well-defined.

Currently #5 seems to be implied for all shells, for lack of further
documentation.  Possibly, the function was never meant to be used with
untrusted data, but there's no warning against doing so either.

I stress-tested the strategy it uses for POSIX shells with the following
horrible hack; the results are positive, i.e. the strategy seems to meet
the criteria #1 above for POSIX shells.

for i in {0..999}
do
    dd if=/dev/urandom of=/dev/stdout bs=1K count=1 2>/dev/null |
        tr -d '\000' > randomfile  # NULL bytes in ARGV are impossible

    emacs -q --batch --eval \
          "(with-temp-buffer 
             (insert-file-contents-literally \"randomfile\")
             (let ((data
                    (replace-regexp-in-string
                      \"\\n\" \"'\\n'\"
                      (replace-regexp-in-string 
                        \"[^-0-9a-zA-Z_./\\n]\" 
                        \"\\\\\\\\\\\\&\" 
                        (buffer-substring (point-min) (point-max))))))
               (erase-buffer)
               (insert \"printf %s \")
               (insert data)
               (write-region (point-min) (point-max) \"commandfile\")))"

    sh - < commandfile > output  # tested with bash, dash, and ksh

    diff randomfile output || exit
done

There's also wording in POSIX which seems to guarantee the safety of the
strategy:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_02_01

"A <backslash> that is not quoted shall preserve the literal value of
the following character, with the exception of a <newline>. [...]"


For now, here's a trivial patch improving the docstring.  If anyone is
confident in the safety of the function for shells other than those
conforming to POSIX sh, feel free to change the docstring accordingly.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-lisp-subr.el-shell-quote-argument-Improve-documentat.patch --]
[-- Type: text/x-diff, Size: 1094 bytes --]

From dedcb603da981dcab8f576dea2f36d58fd2ddcfa Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?=
 <taylanbayirli@gmail.com>
Date: Sun, 18 Oct 2015 14:23:35 +0200
Subject: [PATCH] * lisp/subr.el (shell-quote-argument): Improve documentation.

---
 lisp/subr.el | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lisp/subr.el b/lisp/subr.el
index e176907..940ebe6 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -2711,7 +2711,11 @@ Note: :data and :device are currently not supported on Windows."
 (declare-function w32-shell-dos-semantics "w32-fns" nil)
 
 (defun shell-quote-argument (argument)
-  "Quote ARGUMENT for passing as argument to an inferior shell."
+  "Quote ARGUMENT for passing as argument to an inferior shell.
+
+This is safe for shells conforming to POSIX sh.  No guarantees
+regarding code injection are made for other shells, but csh,
+MS-DOS and Windows NT are supported for simple cases as well."
   (cond
    ((eq system-type 'ms-dos)
     ;; Quote using double quotes, but escape any existing quotes in
-- 
2.5.0