bug#54227: 29.0.50; [PATCH] Inconsistencies with Eshell variable interpolation

[Jim Porter <jporterbugs@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2504 bytes --]

On 3/3/2022 10:43 AM, Eli Zaretskii wrote:
>> Cc: 54227@debbugs.gnu.org
>> From: Jim Porter <jporterbugs@gmail.com>
>> Date: Thu, 3 Mar 2022 09:56:14 -0800
>>
>> If you have any ideas about how to improve the wording, I'm happy to
>> update it though. I'll try to keep thinking as well.
> 
> Something like the below:
> 
>    (defmacro eshell-with-temp-command (region &rest body)
>      "Narrow the buffer to REGION and execute the forms in BODY.
> 
>    REGION is a cons cell (START . END) that specifies the region
>    to which to narrow the buffer.  REGION can also be a string,
>    in which case the macro temporarily inserts it into the
>    buffer at point, and narrows the buffer to the inserted string.
>    Before executing BODY, point is set to the beginning of the
>    narrowed REGION.

Thanks, updated to use that docstring.

>> diff --git a/doc/misc/eshell.texi b/doc/misc/eshell.texi
>> index 5581e5cd9e..47f8902d5a 100644
>> --- a/doc/misc/eshell.texi
>> +++ b/doc/misc/eshell.texi
>> @@ -1043,15 +1043,16 @@ Dollars Expansion
[snip]
>>   
>> -Multiple sets of indices can also be specified. For example, if
>> -@var{var} is a list of lists, @samp{$@var{var}[0][0]} is equivalent to
>> -@samp{(caar @var{var})}.
>> +Multiple sets of indices can also be specified.  For example, if
>> +@var{var} is @samp{((1 2) (3 4))}, then @samp{$@var{var}[0][1]} will
>> +expand to @code{2}.
> 
> I would add to the last sentence: ", i.e.@: the second element of the
> first list member (all indices are zero-based)."

Ok, added.

> Also, it sounds like you just dropped the ball on the alist use case?

I think we just had different ideas of how much detail was necessary. 
Given your above comment, I think I have a better idea of the level of 
detail, so I've expanded this section into a table. The single paragraph 
was a little too dense, so breaking it out into separate blocks for each 
data type makes it easier to provide a more thorough explanation.

>> -(defun eshell-parse-inner-double-quote (bound)
>> -  "Parse the inner part of a double quoted string.
>> +(defun eshell-unescape-inner-double-quote (bound)
>> +  "Unescape the inner part of a double quoted string.
> 
> "Unescape escaped characters of a double-quoted string."

Done, though I worded it as, "Unescape escaped characters inside a 
double-quoted string." I wanted to be extra-clear that this only 
operates on the bits *between* the double-quotes, but doesn't do 
anything with the surrounding double-quotes themselves.

[-- Attachment #2: 0001-Improve-wording-of-Eshell-variable-interpolation-cod.patch --]
[-- Type: text/plain, Size: 8351 bytes --]

From ea4c9b0b7770b73c1320cf4a99ad2ed36638c4ae Mon Sep 17 00:00:00 2001
From: Jim Porter <jporterbugs@gmail.com>
Date: Thu, 3 Mar 2022 09:37:25 -0800
Subject: [PATCH] Improve wording of Eshell variable interpolation
 code/documentation

* lisp/eshell/esh-arg.el (eshell-unescape-inner-double-quote): Rename
from 'eshell-parse-inner-double-quote'.

* lisp/eshell/esh-cmd.el (eshell-with-temp-command): Improve
docstring.

* lisp/eshell/esh-var.el (eshell-parse-variable-ref): Use
'eshell-unescape-inner-double-quote' and improve robustness of quoted
variable name matching.
(eshell-parse-indices): Use 'eshell-unescape-inner-double-quote'.

* doc/misc/eshell.texi (Dollars Expansion): Improve wording of
subscript notation.
---
 doc/misc/eshell.texi   | 41 ++++++++++++++++++++++++++++++-----------
 lisp/eshell/esh-arg.el |  4 ++--
 lisp/eshell/esh-cmd.el | 25 ++++++++++++++-----------
 lisp/eshell/esh-var.el | 14 ++++++++------
 4 files changed, 54 insertions(+), 30 deletions(-)

diff --git a/doc/misc/eshell.texi b/doc/misc/eshell.texi
index 5581e5cd9e..2df4de1cef 100644
--- a/doc/misc/eshell.texi
+++ b/doc/misc/eshell.texi
@@ -1040,18 +1040,37 @@ Dollars Expansion
 Expands to the @var{i}th element of the result of @var{expr}, an
 expression in one of the above forms listed here.  If multiple indices
 are supplied, this will return a list containing the elements for each
-index.  If @var{expr}'s value is a string, it will first be split at
-whitespace to make it a list.  If @var{expr}'s value is an alist
-(@pxref{Association List Type, Association Lists, , elisp, The Emacs
-Lisp Reference Manual}), this will call @code{assoc} on the result of
-@var{expr}, returning the @code{cdr} of the element of the result
-whose car is equal to @code{"i"}.  Raises an error if the value is not
-a sequence (@pxref{Sequences Arrays Vectors, Sequences, , elisp, The
+index.  The exact behavior depends on the type of @var{expr}'s value:
+
+@table @asis
+
+@item a sequence
+Expands to the element at the (zero-based) index @var{i} of the
+sequence (@pxref{Sequences Arrays Vectors, Sequences, , elisp, The
 Emacs Lisp Reference Manual}).
 
-Multiple sets of indices can also be specified. For example, if
-@var{var} is a list of lists, @samp{$@var{var}[0][0]} is equivalent to
-@samp{(caar @var{var})}.
+@item a string
+Split the string at whitespace, and then expand to the @var{i}th
+element of the resulting sequence.
+
+@item an alist
+If @var{i} is a non-numeric value, expand to the value associated with
+the key @code{"i"}. For example, if @var{var} is @samp{(("dog"
+. "fido") ("cat" . "felix"))}, then @samp{$@var{var}[dog]} expands to
+@code{"fido"}.  Otherwise, this behaves as with sequences; e.g.,
+@samp{$@var{var}[0]} expands to @code{("dog" . "fido")}.
+@xref{Association List Type, Association Lists, , elisp, The Emacs
+Lisp Reference Manual}.
+
+@item anything else
+Raises an error.
+
+@end table
+
+Multiple sets of indices can also be specified.  For example, if
+@var{var} is @samp{((1 2) (3 4))}, then @samp{$@var{var}[0][1]} will
+expand to @code{2}, i.e.@: the second element of the first list member
+(all indices are zero-based).
 
 @item $@var{expr}[@var{regexp} @var{i...}]
 As above (when @var{expr} expands to a string), but use @var{regexp}
@@ -1064,7 +1083,7 @@ Dollars Expansion
 one of the above forms.  For example, @samp{$#@var{var}} returns the
 length of the variable @var{var} and @samp{$#@var{var}[0]} returns the
 length of the first element of @var{var}.  Again, raises an error if
-the result of @var{expr} is not a sequence.
+the result of @var{expr} is not a string or a sequence.
 
 @end table
 
diff --git a/lisp/eshell/esh-arg.el b/lisp/eshell/esh-arg.el
index e19481c4ba..ee3f907f85 100644
--- a/lisp/eshell/esh-arg.el
+++ b/lisp/eshell/esh-arg.el
@@ -354,8 +354,8 @@ eshell-parse-double-quote
 		  (list 'eshell-escape-arg arg))))
 	  (goto-char (1+ end)))))))
 
-(defun eshell-parse-inner-double-quote (bound)
-  "Parse the inner part of a double quoted string.
+(defun eshell-unescape-inner-double-quote (bound)
+  "Unescape escaped characters inside a double-quoted string.
 The string to parse starts at point and ends at BOUND.
 
 If Eshell is currently parsing a quoted string and there are any
diff --git a/lisp/eshell/esh-cmd.el b/lisp/eshell/esh-cmd.el
index 04b54d9d79..8be1136e31 100644
--- a/lisp/eshell/esh-cmd.el
+++ b/lisp/eshell/esh-cmd.el
@@ -350,24 +350,27 @@ eshell-complete-lisp-symbols
 
 (defvar eshell--sep-terms)
 
-(defmacro eshell-with-temp-command (command &rest body)
-  "Narrow the buffer to COMMAND and execute the forms in BODY.
-COMMAND can either be a string, or a cons cell demarcating a
-buffer region.  If COMMAND is a string, temporarily insert it
-into the buffer before narrowing.  Point will be set to the
-beginning of the narrowed region.
+(defmacro eshell-with-temp-command (region &rest body)
+  "Narrow the buffer to REGION and execute the forms in BODY.
+
+REGION is a cons cell (START . END) that specifies the region to
+which to narrow the buffer.  REGION can also be a string, in
+which case the macro temporarily inserts it into the buffer at
+point, and narrows the buffer to the inserted string.  Before
+executing BODY, point is set to the beginning of the narrowed
+REGION.
 
 The value returned is the last form in BODY."
   (declare (indent 1))
-  `(let ((cmd ,command))
-     (if (stringp cmd)
+  `(let ((reg ,region))
+     (if (stringp reg)
          ;; Since parsing relies partly on buffer-local state
          ;; (e.g. that of `eshell-parse-argument-hook'), we need to
          ;; perform the parsing in the Eshell buffer.
          (let ((begin (point)) end
 	       (inhibit-point-motion-hooks t))
            (with-silent-modifications
-             (insert cmd)
+             (insert reg)
              (setq end (point))
              (unwind-protect
                  (save-restriction
@@ -376,8 +379,8 @@ eshell-with-temp-command
                    ,@body)
                (delete-region begin end))))
        (save-restriction
-         (narrow-to-region (car cmd) (cdr cmd))
-         (goto-char (car cmd))
+         (narrow-to-region (car reg) (cdr reg))
+         (goto-char (car reg))
          ,@body))))
 
 (defun eshell-parse-command (command &optional args toplevel)
diff --git a/lisp/eshell/esh-var.el b/lisp/eshell/esh-var.el
index af89e35f55..8746f2bb93 100644
--- a/lisp/eshell/esh-var.el
+++ b/lisp/eshell/esh-var.el
@@ -437,7 +437,7 @@ eshell-parse-variable-ref
             `(eshell-convert
               (eshell-command-to-value
                (eshell-as-subcommand
-                ,(let ((subcmd (or (eshell-parse-inner-double-quote end)
+                ,(let ((subcmd (or (eshell-unescape-inner-double-quote end)
                                    (cons (point) end)))
                        (eshell-current-quoted nil))
                    (eshell-parse-command subcmd)))))
@@ -470,13 +470,15 @@ eshell-parse-variable-ref
     (condition-case nil
         `(eshell-command-to-value
           (eshell-lisp-command
-           ',(read (or (eshell-parse-inner-double-quote (point-max))
+           ',(read (or (eshell-unescape-inner-double-quote (point-max))
                        (current-buffer)))))
       (end-of-file
        (throw 'eshell-incomplete ?\())))
-   ((looking-at (rx (or "'" "\"" "\\\"")))
-    (eshell-with-temp-command (or (eshell-parse-inner-double-quote (point-max))
-                                  (cons (point) (point-max)))
+   ((looking-at (rx-to-string
+                 `(or "'" ,(if eshell-current-quoted "\\\"" "\""))))
+    (eshell-with-temp-command
+        (or (eshell-unescape-inner-double-quote (point-max))
+            (cons (point) (point-max)))
       (let ((name (if (eq (char-after) ?\')
                       (eshell-parse-literal-quote)
                     (eshell-parse-double-quote))))
@@ -506,7 +508,7 @@ eshell-parse-indices
 	(if (not end)
 	    (throw 'eshell-incomplete ?\[)
 	  (forward-char)
-          (eshell-with-temp-command (or (eshell-parse-inner-double-quote end)
+          (eshell-with-temp-command (or (eshell-unescape-inner-double-quote end)
                                         (cons (point) end))
 	    (let (eshell-glob-function (eshell-current-quoted nil))
 	      (setq indices (cons (eshell-parse-arguments
-- 
2.25.1