bug#54603: 29.0.50; [PATCH] Eshell's external pipe module interferes with other argument parsing hooks

[Sean Whitton <spwhitton@spwhitton.name>]

Hello,

On Thu 31 Mar 2022 at 01:58PM -07, Jim Porter wrote:

> I'm not so sure. That would mean "'foo" in "echo 'foo" is treated as a
> Lisp form, but Eshell expects you to use "#'" (or "(" or "`") to
> introduce a Lisp form. As I understand it, that's so that typos don't do
> such surprising things. There's a good chance that a user typing "echo
> 'foo" actually meant "echo 'foo'".

Right okay.  We can just skip over entire Lisp forms when we find them.
I don't think there could be a non-highly esoteric shell command for
standard POSIX shells -- which is what this feature is for -- which that
would break.  Like this:

>> diff --git a/lisp/eshell/em-extpipe.el b/lisp/eshell/em-extpipe.el
>> index eb5b3bfe1d..0787ab791b 100644
>> --- a/lisp/eshell/em-extpipe.el
>> +++ b/lisp/eshell/em-extpipe.el
>> @@ -99,7 +99,7 @@ eshell-parse-external-pipeline
>>                       (let* ((found
>>                               (save-excursion
>>                                 (re-search-forward
>> -                                "\\(?:#?'\\|\"\\|\\\\\\)" bound t)))
>> +                                "\\(?:(\\|#?'\\|\"\\|\\\\\\)" bound t)))
>>                              (next (or (and found (match-beginning 0))
>>                                        bound)))
>>                         (if (re-search-forward pat next t)
>>

Something in my init.el is breaking the extpipe tests atm, but I ad hoc
tested one of your cases for this bug and it works.  Could you confirm?

> That said, this isn't the only situation where "unbalanced" single
> quotes can occur in an Eshell command. For example, see this command:
>
>    echo $(list "one" "two")(:s'o'x')
>
> This creates a list of two strings, and then performs a regexp
> substitution from "o" to "x", so the output is:
>
>    ("xne" "twx")
>
> Under Emacs 27/28, this works correctly, but it fails in 29 for the
> same reason as the original issue. The extpipe module could account
> for this and try to parse argument predicates/modifiers so that it
> knows when to stay out of things, but then what about configurations
> where that module is disabled? (And for that matter, a third-party
> Eshell module could cause conflicts in a similar manner.)

I think the correct thing is for extpipe to do something sensible with
all the Eshell syntax that is enabled by default.  We should call the
relevant parsing function to skip over these predicates/modifiers too.

If you're changing the list of modules then you might have to drop
extpipe as part of that process -- that's just inherent in how Eshell
modules work.  There's always going to be a limit to how much syntax you
can have enabled at once.

-- 
Sean Whitton