bug#8132: 23.1; comint shell replaces \ in paths with /

bug#8132: 23.1; comint shell replaces \ in paths with /

[Zeljko Vrba]

[-- Attachment #1: Type: text/plain, Size: 3761 bytes --]

*** E-Mail body has been placed on clipboard, please paste them here! ***

On Windows 7 (64-bit mod), run M-x shell, and try to do any filename 
expansion.  All backslashes (even
manually typed!) are converted to forward slashes upon tab-expansion.



In GNU Emacs 23.1.1 (i386-mingw-nt6.1.7600)
  of 2009-07-30 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 6.1.7600
configured using `configure --with-gcc (4.4)'

Important settings:
   value of $LC_ALL: nil
   value of $LC_COLLATE: nil
   value of $LC_CTYPE: nil
   value of $LC_MESSAGES: nil
   value of $LC_MONETARY: nil
   value of $LC_NUMERIC: nil
   value of $LC_TIME: nil
   value of $LANG: ENU
   value of $XMODIFIERS: nil
   locale-coding-system: cp1252
   default-enable-multibyte-characters: t

Major mode: Buffer Menu

Minor modes in effect:
   shell-dirtrack-mode: t
   tooltip-mode: t
   tool-bar-mode: t
   mouse-wheel-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   global-auto-composition-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   column-number-mode: t
   line-number-mode: t
   transient-mark-mode: t

Recent input:
<return> <help-echo> <help-echo> <down-mouse-1> <help-echo>
<down-mouse-1> <mouse-1> C-x k <return> <help-echo>
<down-mouse-1> <mouse-1> C-x 1 <help-echo> M-x c u
s t SPC - g r o SPC <return> c o m i t <backspace>
SPC <return> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <help-echo> <help-echo>
<down-mouse-1> <mouse-2> <help-echo> <down-mouse-1>
<mouse-1> <down-mouse-1> <mouse-1> <help-echo> <help-echo>
<help-echo> <down-mouse-1> <mouse-1> <help-echo> C-z
C-x k <return> C-x C-b C-x o <down> <down> <return>
<help-echo> C-x 1 <down-mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1>
<help-echo> C-x k C-g <return> <down-mouse-1> <mouse-2>
<help-echo> <help-echo> C-x k <return> C-x k <return>
C-x C-b <up> <up> <return> c : \ u <tab> s <tab> \
z v <tab> <backspace> <backspace> <backspace> <backspace>
' z <backspace> <backspace> \ z <tab> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> > <return>
<return> C-x 1 c : \ u s e r s \ z v r b a \ l u a
j <tab> <C-left> <C-left> <C-left> <C-left> <C-left>
<C-left> <C-left> C-e l <backspace> s r c / l u a j
i t . e x e <return> <return> <return> C-g C-g <help-echo>
<down-mouse-1> <mouse-1> M-p <return> <return> <return>
= SPC 2 + 3 <return> C-x k <return> <down-mouse-1>
<M-mouse-1> M-x r e p o SPC r t SPC e m a SPC b u SPC
<return>

Recent messages:
Completed
Completing file name...
No completions of /zv
Completing file name...
Completed
Completing file name...
Completed
Quit [2 times]
History item: 1
Making completion list...


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3666 bytes --]

bug#8132: 23.1; comint shell replaces \ in paths with /

[Stefan Monnier]

> On Windows 7 (64-bit mod), run M-x shell, and try to do any filename
> expansion.  All backslashes (even
> manually typed!) are converted to forward slashes upon tab-expansion.

Yes, that's the expected behavior.  Could you explain why it's
a problem?


        Stefan

bug#8132: 23.1; comint shell replaces \ in paths with /

[Zeljko Vrba]

[-- Attachment #1: Type: text/plain, Size: 686 bytes --]

On 2011-02-28 4:32 AM, Stefan Monnier wrote:
>> On Windows 7 (64-bit mod), run M-x shell, and try to do any filename
>> expansion.  All backslashes (even
>> manually typed!) are converted to forward slashes upon tab-expansion.
>
> Yes, that's the expected behavior.  Could you explain why it's
> a problem?
>
It is a problem because commands built in to cmd.exe (and also external 
commands) interpret slash as a command-line switch character.  So you 
end up with the following situation:

--> comint expanded \ to / : del ../lpeg-0.10.2/re.html
Invalid switch - "lpeg-0.10.2".

In short, it breaks all native windows command-line tools.

Best regards,
   Zeljko.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3666 bytes --]

bug#8132: 23.1; comint shell replaces \ in paths with /

[Lennart Borgman]

On Mon, Feb 28, 2011 at 3:59 PM, Zeljko Vrba <zvrba.external@zvrba.net> wrote:
> On 2011-02-28 4:32 AM, Stefan Monnier wrote:
>>>
>>> On Windows 7 (64-bit mod), run M-x shell, and try to do any filename
>>> expansion.  All backslashes (even
>>> manually typed!) are converted to forward slashes upon tab-expansion.
>>
>> Yes, that's the expected behavior.  Could you explain why it's
>> a problem?
>>
> It is a problem because commands built in to cmd.exe (and also external
> commands) interpret slash as a command-line switch character.  So you end up
> with the following situation:
>
> --> comint expanded \ to / : del ../lpeg-0.10.2/re.html
> Invalid switch - "lpeg-0.10.2".
>
> In short, it breaks all native windows command-line tools.
>
> Best regards,
>  Zeljko.

Does not most external commands recognize paths with / in them? Which
external commands does not do that?

bug#8132: 23.1; comint shell replaces \ in paths with /

[Zeljko Vrba]

[-- Attachment #1: Type: text/plain, Size: 783 bytes --]

On 2011-02-28 4:47 PM, Lennart Borgman wrote:
>
> Does not most external commands recognize paths with / in them? Which
> external commands does not do that?
 >
None of the most often used file-manipulation commands do. (See below 
for a transcript). Yes, I know that NTFS in itself allows both / and \ 
as path separators, but, AFAIK, all native (i.e., not ported from unix) 
command-line tools use / as switch.

C:\Users\zvrba>rmdir ../zvrba
Invalid switch - "zvrba".

C:\Users\zvrba>mkdir ../q
The syntax of the command is incorrect.

C:\Users\zvrba>del ../zvrba
Invalid switch - "zvrba".

C:\Users\zvrba>dir ../zvrba
Invalid switch - "zvrba".

C:\Users\zvrba>copy ../zvrba/.emacs.d/init.el .
The syntax of the command is incorrect.

C:\Users\zvrba>



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3666 bytes --]

bug#8132: 23.1; comint shell replaces \ in paths with /

[Lennart Borgman]

On Mon, Feb 28, 2011 at 6:47 PM, Zeljko Vrba <zvrba.external@zvrba.net> wrote:
> On 2011-02-28 4:47 PM, Lennart Borgman wrote:
>>
>> Does not most external commands recognize paths with / in them? Which
>> external commands does not do that?
>
>>
> None of the most often used file-manipulation commands do. (See below for a
> transcript). Yes, I know that NTFS in itself allows both / and \ as path
> separators, but, AFAIK, all native (i.e., not ported from unix) command-line
> tools use / as switch.
>
> C:\Users\zvrba>rmdir ../zvrba
> Invalid switch - "zvrba".
>
> C:\Users\zvrba>mkdir ../q
> The syntax of the command is incorrect.
>
> C:\Users\zvrba>del ../zvrba
> Invalid switch - "zvrba".
>
> C:\Users\zvrba>dir ../zvrba
> Invalid switch - "zvrba".
>
> C:\Users\zvrba>copy ../zvrba/.emacs.d/init.el .
> The syntax of the command is incorrect.
>
> C:\Users\zvrba>

Are not these all built in to cmd.exe?

bug#8132: 23.1; comint shell replaces \ in paths with /

[Zeljko Vrba]

[-- Attachment #1: Type: text/plain, Size: 260 bytes --]

On 2011-02-28 6:57 PM, Lennart Borgman wrote:
> On Mon, Feb 28, 2011 at 6:47 PM, Zeljko Vrba<zvrba.external@zvrba.net>  wrote:

>
> Are not these all built in to cmd.exe?
 >
That may be so, but I haven't found external equivalents for these commands.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3666 bytes --]

bug#8132: 23.1; comint shell replaces \ in paths with /

[Lennart Borgman]

On Mon, Feb 28, 2011 at 11:05 PM, Zeljko Vrba <zvrba.external@zvrba.net> wrote:
> On 2011-02-28 6:57 PM, Lennart Borgman wrote:
>>
>> On Mon, Feb 28, 2011 at 6:47 PM, Zeljko Vrba<zvrba.external@zvrba.net>
>>  wrote:
>
>>
>> Are not these all built in to cmd.exe?
>
>>
> That may be so, but I haven't found external equivalents for these commands.

In eshell (which Eli mentioned) you have built in replacement for
those commands. Please see the eshell manual. You find this by opening
the Emacs manual (with for example C-h m, or from the help menu) and
then type "u" (for up).

bug#8132: 23.1; comint shell replaces \ in paths with /

[Eli Zaretskii]

> Date: Mon, 28 Feb 2011 15:59:07 +0100
> From: Zeljko Vrba <zvrba.external@zvrba.net>
> Cc: 8132@debbugs.gnu.org
> 
> On 2011-02-28 4:32 AM, Stefan Monnier wrote:
> >> On Windows 7 (64-bit mod), run M-x shell, and try to do any filename
> >> expansion.  All backslashes (even
> >> manually typed!) are converted to forward slashes upon tab-expansion.
> >
> > Yes, that's the expected behavior.  Could you explain why it's
> > a problem?
> >
> It is a problem because commands built in to cmd.exe (and also external 
> commands) interpret slash as a command-line switch character.  So you 
> end up with the following situation:
> 
> --> comint expanded \ to / : del ../lpeg-0.10.2/re.html
> Invalid switch - "lpeg-0.10.2".
> 
> In short, it breaks all native windows command-line tools.

Do you really need cmd.exe as your shell inside Emacs?  What cmd
features do you depend on?

If you just need an interactive shell session within Emacs, I suggest
to try eshell instead.  Its command work just fine with forward
slashes.

bug#8132: 23.1; comint shell replaces \ in paths with /

[Zeljko Vrba]

[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

On 2011-02-28 7:34 PM, Eli Zaretskii wrote:
>
> Do you really need cmd.exe as your shell inside Emacs?  What cmd
> features do you depend on?
>
I do not *need* cmd.exe, but I do not want to run cygwin shell either.

 >
> If you just need an interactive shell session within Emacs, I suggest
> to try eshell instead.  Its command work just fine with forward
> slashes.
>
OK, I will try that.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3666 bytes --]

bug#8132: 23.1; comint shell replaces \ in paths with /

[Stefan Monnier]

>> In short, it breaks all native windows command-line tools.
> Do you really need cmd.exe as your shell inside Emacs?  What cmd
> features do you depend on?

If / doesn't work when the shell is cmd.exe and cmd.exe is the only
(external) shell available by default under Windows (or at least the
most commonly used), then shell.el should try to accomodate it somehow.
Maybe the best way is for shell.el to burp or warn the user that she
would be better served by eshell.  But maybe file completion in shell.el
should be tweaked to not turn \ into / in that case.


        Stefan

bug#8132: 23.1; comint shell replaces \ in paths with /

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Zeljko Vrba <zvrba.external@zvrba.net>,  8132@debbugs.gnu.org
> Date: Mon, 28 Feb 2011 18:04:19 -0500
> 
> Maybe the best way is for shell.el to burp or warn the user that she
> would be better served by eshell.  But maybe file completion in shell.el
> should be tweaked to not turn \ into / in that case.

It could be easier to make a wrapper around the file completion, which
would simply convert all / into \ when the result is inserted into the
shell buffer.  That's because working with / is very basic in
file-name completion, and reaches deep into the code and the
primitives it uses.

bug#8132: 23.1; comint shell replaces \ in paths with /

[Stefan Monnier]

>> Maybe the best way is for shell.el to burp or warn the user that she
>> would be better served by eshell.  But maybe file completion in shell.el
>> should be tweaked to not turn \ into / in that case.

> It could be easier to make a wrapper around the file completion, which
> would simply convert all / into \ when the result is inserted into the
> shell buffer.

Yes, that's more or less what I meant.

> That's because working with / is very basic in file-name completion,
> and reaches deep into the code and the primitives it uses.

Yes, I know ;-)


        Stefan

bug#8132: 23.1; comint shell replaces \ in paths with /

[zvrba]

On Tue, Mar 01, 2011 at 05:54:50AM +0200, Eli Zaretskii wrote:
> 
> It could be easier to make a wrapper around the file completion, which
> would simply convert all / into \ when the result is inserted into the
> shell buffer.  That's because working with / is very basic in
> file-name completion, and reaches deep into the code and the
> primitives it uses.
>
I suspect that that would break switches to commands.  I.e., it would
require quite involved logic to find out that in

  dir /a ../blah

/a should not be converted to \a.  So that code would have to keep
track which strings have been autocompleted and change / to \ *only*
in those strings.

(I did not quite understand your proposal, i.e., when and which text
would the conversion be applied to, so I apologize if that's what you
suggested.)

PS: The eshell manual is mostly empty, and when I write "help", I get the
output of the cmd.exe's help.  Needless to say, eshell does not recognize
commands that are built-in to cmd.exe (copy, for example).  I had to guess
myself towards using ls and cp, but how do I then get a list of all *eshell*
builtins?

Furthermore, neither comint with cmd.exe nor eshell like interactive commands.
I have just run "sc" which prints some usage info and a prompt like this:

Would you like to see help for the QUERY and QUERYEX commands? [ y | n ]: 

In raw cmd.exe (own window, outside of emacs), when I press 'n', the programm
immediately exits (no need to press enter after n).  In eshell, nothing
happens.  The shell never returns to the prompt; pressing up/down arrows
cycles through the history, but no input is accepted until I press C-c twice. 

Incidentally, the same thing happens in M-x shell, just that after pressing
C-c twice, it becomes visible that all input had been given to the shell,
just that the buffer hadn't been flushed.  This is the output after pressing
'n ENTER' to the above prompt:

--
Would you like to see help for the QUERY and QUERYEX commands? [ y | n ]: 
n

  C-c C-c

  c:\Users\Hue\Desktop>n
  'n' is not recognized as an internal or external command,
  operable program or batch file.
--

Pressing 'n' and C-c C-c is OK, i.e., I get the prompt and no extraneous input
has been sent to the shell.

--

I have also tried to run interactive Lua interpreter through M-x shell and
I had the same issue with interactivity.. basically no input/output to/from
the interpreter has been visible in the shell window.

bug#8132: 23.1; comint shell replaces \ in paths with /

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> It could be easier to make a wrapper around the file completion, which
> would simply convert all / into \ when the result is inserted into the
> shell buffer.  That's because working with / is very basic in
> file-name completion, and reaches deep into the code and the
> primitives it uses.

It should respect remote file names. A conversion to \ is not helpful
for them.

Likely, it is sufficient to suppress the conversion if default-directory
is remote.

Best regards, Michael.

bug#8132: 23.1; comint shell replaces \ in paths with /

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  zvrba.external@zvrba.net,  8132@debbugs.gnu.org
> Date: Tue, 01 Mar 2011 10:02:23 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It could be easier to make a wrapper around the file completion, which
> > would simply convert all / into \ when the result is inserted into the
> > shell buffer.  That's because working with / is very basic in
> > file-name completion, and reaches deep into the code and the
> > primitives it uses.
> 
> It should respect remote file names. A conversion to \ is not helpful
> for them.

Sorry, I'm not following: what remote file names?  We are talking
about "M-x shell" with the stock Windows shell.  That shell doesn't
support remote file names at all.  What am I missing?

> Likely, it is sufficient to suppress the conversion if default-directory
> is remote.

How can it be, in "M-x shell"?

bug#8132: 23.1; comint shell replaces \ in paths with /

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>> zvrba.external@zvrba.net, 8132@debbugs.gnu.org
>> Date: Tue, 01 Mar 2011 10:02:23 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > It could be easier to make a wrapper around the file completion, which
>> > would simply convert all / into \ when the result is inserted into the
>> > shell buffer.  That's because working with / is very basic in
>> > file-name completion, and reaches deep into the code and the
>> > primitives it uses.
>> 
>> It should respect remote file names. A conversion to \ is not helpful
>> for them.
>
> Sorry, I'm not following: what remote file names?  We are talking
> about "M-x shell" with the stock Windows shell.  That shell doesn't
> support remote file names at all.  What am I missing?

You have the following call sequence:

shell -> make-comint-in-buffer -> comint-exec -> comint-exec-1 -> start-file-process

If default-directory is remote, start-file-process invokes Tramp.

Best regards, Michael.

bug#8132: 23.1; comint shell replaces \ in paths with /

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: monnier@iro.umontreal.ca,  zvrba.external@zvrba.net,  8132@debbugs.gnu.org
> Date: Wed, 02 Mar 2011 09:42:47 +0100
> 
> shell -> make-comint-in-buffer -> comint-exec -> comint-exec-1 -> start-file-process
> 
> If default-directory is remote, start-file-process invokes Tramp.

If someone uses remote default-directory with cmd.exe, they deserve
this complication.

That said, if we wrap the file completion with cmd-sensitive code, we
can easily make it cater to remote default-directories.

bug#8132: 23.1; comint shell replaces \ in paths with /

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> If someone uses remote default-directory with cmd.exe, they deserve
> this complication.

But maybe it is desired? Interactively, if `default-directory' is
remote, `shell' could ask to change `explicit-shell-file-name' if it is
nil. WDYT?

> That said, if we wrap the file completion with cmd-sensitive code, we
> can easily make it cater to remote default-directories.

Yep.

Best regards, Michael.