bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Ken Brown]

This is another long-standing problem, but there's an easy fix, which
I'll send in a second message.  (I want to wait until I have a bug
number so that I can make a proper ChangeLog entry.)

The test for the file name " foo\tbar baz\t" hangs on Cygwin, because
Cygwin interprets the backslash as a Windows path separator; see the
section "Forbidden characters in filenames" in
https://cygwin.com/cygwin-ug-net/using-specialnames.html.  Here's an
example outside of emacs:

$ touch "foo\tbar"
touch: cannot touch =E2=80=98foo\\tbar=E2=80=99: No such file or directory

I'll send a patch shortly.

Ken



In GNU Emacs 25.0.50.5 (x86_64-unknown-cygwin, GTK+ Version 3.14.9)
  of 2015-03-16 on moufang
Repository revision: 5d9b1e100aa4ddb79471f7ec2347fdb65d6a9a70
Windowing system distributor `The Cygwin/X Project', version 11.0.11701000
Configured using:
  `configure --without-all --cache-file=3D/tmp/config.cache'

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 337 bytes --]

On 3/16/2015 12:21 PM, Ken Brown wrote:
> This is another long-standing problem, but there's an easy fix, which
> I'll send in a second message.

The patch is attached.  I can't actually test it on the current HEAD, 
because Bug#20117 causes the test to abort immediately.  But I tested an 
equivalent patch on revision 5d9b1e100.

Ken


[-- Attachment #2: 0001-Don-t-test-t-in-file-names-on-Cygwin.patch --]
[-- Type: text/plain, Size: 1405 bytes --]

From 87a5498e9224695b391c89c949314314db123521 Mon Sep 17 00:00:00 2001
From: Ken Brown <kbrown@cornell.edu>
Date: Mon, 16 Mar 2015 12:25:42 -0400
Subject: [PATCH] Don't test "\t" in file names on Cygwin.

Fixes: debbugs:20119

* test/automated/tramp-tests.el (tramp--test-special-characters): Don't
test "\t" in file names on Cygwin.
---
 test/ChangeLog                | 5 +++++
 test/automated/tramp-tests.el | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/test/ChangeLog b/test/ChangeLog
index 6a474e1..a7d1dfd 100644
--- a/test/ChangeLog
+++ b/test/ChangeLog
@@ -1,3 +1,8 @@
+2015-03-16  Ken Brown  <kbrown@cornell.edu>
+
+	* automated/tramp-tests.el (tramp--test-special-characters): Don't
+	test "\t" in file names on Cygwin.  (Bug#20119)
+
 2015-03-10  Jackson Ray Hamilton  <jackson@jacksonrayhamilton.com>
 
 	* indent/js-indent-init-dynamic.js: Fix spelling error.
diff --git a/test/automated/tramp-tests.el b/test/automated/tramp-tests.el
index 020f31f..25b11ff 100644
--- a/test/automated/tramp-tests.el
+++ b/test/automated/tramp-tests.el
@@ -1626,7 +1626,7 @@ This requires restrictions of file name syntax."
   (tramp--test-check-files
    (if (tramp--test-smb-or-windows-nt-p)
        "foo bar baz"
-     (if (tramp--test-adb-p)
+     (if (or (tramp--test-adb-p) (eq system-type 'cygwin))
 	 " foo bar baz "
        " foo\tbar baz\t"))
    "$foo$bar$$baz$"
-- 
2.1.4

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Michael Albinus]

Ken Brown <kbrown@cornell.edu> writes:

> The patch is attached.  I can't actually test it on the current HEAD,
> because Bug#20117 causes the test to abort immediately.  But I tested
> an equivalent patch on revision 5d9b1e100.

Looks good to me, pls install. Maybe you could also add a comment that
"\t" is not expanded to <TAB> on cygwin and windows-nt systems. And
maybe it is the same for Android, I'll check. 

> Ken

Best regards, Michael.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Ken Brown]

On 3/16/2015 3:05 PM, Michael Albinus wrote:
> Ken Brown <kbrown@cornell.edu> writes:
>
>> The patch is attached.  I can't actually test it on the current HEAD,
>> because Bug#20117 causes the test to abort immediately.  But I tested
>> an equivalent patch on revision 5d9b1e100.
>
> Looks good to me, pls install. Maybe you could also add a comment that
> "\t" is not expanded to <TAB> on cygwin and windows-nt systems. And
> maybe it is the same for Android, I'll check.

Done as commit a961dce; closing.

Ken

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Michael Albinus]

Ken Brown <kbrown@cornell.edu> writes:

Hi Ken,

> This is another long-standing problem, but there's an easy fix, which
> I'll send in a second message.  (I want to wait until I have a bug
> number so that I can make a proper ChangeLog entry.)
>
> The test for the file name " foo\tbar baz\t" hangs on Cygwin, because
> Cygwin interprets the backslash as a Windows path separator; see the
> section "Forbidden characters in filenames" in
> https://cygwin.com/cygwin-ug-net/using-specialnames.html.  Here's an
> example outside of emacs:
>
> $ touch "foo\tbar"
> touch: cannot touch =E2=80=98foo\\tbar=E2=80=99: No such file or directory

"\t" shall be expanded as <TAB>. And leading (or trailing) spaces are
also not allowed in file names on w32; likely it is the same for cygwin.

I bet it is sufficient to add cygwin in the check of
tramp--test-smb-or-windows-nt-p. Could you, pls check?

> I'll send a patch shortly.
>
> Ken

Best rewgards, Michael.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Ken Brown]

On 3/16/2015 12:39 PM, Michael Albinus wrote:

Hi Michael,

> "\t" shall be expanded as <TAB>. And leading (or trailing) spaces are
> also not allowed in file names on w32; likely it is the same for cygwin.

In the context of file names on Cygwin, "\t" is not expanded to <TAB> 
because of the special treatment of the backslash.  But leading and 
trailing spaces are fine; see my patch.
>
> I bet it is sufficient to add cygwin in the check of
> tramp--test-smb-or-windows-nt-p. Could you, pls check?

It's sufficient but not necessary.

Ken

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Michael Albinus]

Ken Brown <kbrown@cornell.edu> writes:

> Hi Michael,

Hi Ken,

>> "\t" shall be expanded as <TAB>. And leading (or trailing) spaces are
>> also not allowed in file names on w32; likely it is the same for cygwin.
>
> In the context of file names on Cygwin, "\t" is not expanded to <TAB>
> because of the special treatment of the backslash.  But leading and
> trailing spaces are fine; see my patch.

Your patch works fine, but I'm still curious: what happens with this
test, if we don't pass "\t" but the <TAB> character in the file name?
This shall work, shouldn't it?

Maybe you have time to test the follwing patch:

--8<---------------cut here---------------start------------->8---
*** /home/albinus/src/tramp/test/tramp-tests.el.~master~	2015-03-18 20:50:29.399389219 +0100
--- /home/albinus/src/tramp/test/tramp-tests.el	2015-03-18 16:43:56.073936927 +0100
***************
*** 1522,1528 ****
    "Run a simple but comprehensive test over every file in FILES."
    (let ((tmp-name1 (tramp--test-make-temp-name))
  	(tmp-name2 (tramp--test-make-temp-name 'local))
! 	(files (delq nil files)))
      (unwind-protect
  	(progn
  	  (make-directory tmp-name1)
--- 1522,1529 ----
    "Run a simple but comprehensive test over every file in FILES."
    (let ((tmp-name1 (tramp--test-make-temp-name))
  	(tmp-name2 (tramp--test-make-temp-name 'local))
! 	(files (mapcar (lambda (x) (with-output-to-string (princ x)))
! 		       (delq nil files))))
      (unwind-protect
  	(progn
  	  (make-directory tmp-name1)
***************
*** 1629,1635 ****
    (tramp--test-check-files
     (if (tramp--test-smb-or-windows-nt-p)
         "foo bar baz"
!      (if (or (tramp--test-adb-p) (eq system-type 'cygwin))
  	 " foo bar baz "
         " foo\tbar baz\t"))
     "$foo$bar$$baz$"
--- 1630,1636 ----
    (tramp--test-check-files
     (if (tramp--test-smb-or-windows-nt-p)
         "foo bar baz"
!      (if (tramp--test-adb-p)
  	 " foo bar baz "
         " foo\tbar baz\t"))
     "$foo$bar$$baz$"
--8<---------------cut here---------------end--------------->8---

> Ken

Best regards, Michael.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Wed, 18 Mar 2015 20:53:16 +0100
> Cc: 20119@debbugs.gnu.org
> 
> Your patch works fine, but I'm still curious: what happens with this
> test, if we don't pass "\t" but the <TAB> character in the file name?
> This shall work, shouldn't it?

In general, TAB characters are not allowed in file names on Windows,
but maybe Cygwin makes that possible.

P.S. I wrote in the manual, per your request, which characters are not
allowed in Windows file names, but for some reason you don't consult
that text :-(

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> In general, TAB characters are not allowed in file names on Windows,
> but maybe Cygwin makes that possible.
>
> P.S. I wrote in the manual, per your request, which characters are not
> allowed in Windows file names, but for some reason you don't consult
> that text :-(

I read when you had written that text, but (acc to my wife) I'm
forgetting everything. Sorry for that!

Cygwin seems to be different, for example trailing spaces are not a
problem. And, as you might see in tramp--test-special-characters,
several other special characters in file names work for Cygwin, which
are not valid for Windows.

Best regards, Michael.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Ken Brown]

On 3/18/2015 3:53 PM, Michael Albinus wrote:
> Your patch works fine, but I'm still curious: what happens with this
> test, if we don't pass "\t" but the <TAB> character in the file name?
> This shall work, shouldn't it?

No, the test hangs as it did before.  I tried instrumenting the test, 
but the only output before the hang was the following:

Tramp: Opening connection for moufang using mock...
Opening connection for moufang using mock... \
Tramp: Sending command `exec sh -i'
Tramp: Waiting for prompts from remote shell...
Waiting for prompts from remote shell... \
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on `moufang'
Tramp: Opening connection for moufang using mock...done

Ken

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Michael Albinus]

Ken Brown <kbrown@cornell.edu> writes:

>> Your patch works fine, but I'm still curious: what happens with this
>> test, if we don't pass "\t" but the <TAB> character in the file name?
>> This shall work, shouldn't it?
>
> No, the test hangs as it did before.  I tried instrumenting the test,
> but the only output before the hang was the following:

OK, so let it be. I was just curious ...

> Ken

Thanks for checking, and best regards, Michael.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Mon, 16 Mar 2015 17:39:07 +0100
> Cc: 20119@debbugs.gnu.org
> 
> And leading (or trailing) spaces are also not allowed in file names
> on w32

They are allowed, just not convenient in some situations.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Date: Mon, 16 Mar 2015 17:39:07 +0100
>> Cc: 20119@debbugs.gnu.org
>> 
>> And leading (or trailing) spaces are also not allowed in file names
>> on w32
>
> They are allowed, just not convenient in some situations.

IIRC I got errors when I have tested tramp-test30-special-characters
with leading spaces in file names on w32. Maybe it is just a Tramp
error, but I haven't been motivated enough to debug.

It could be also the case that it is an error in file name primitives on
w32. I haven't checked either (w32 systems are terra incognita for me).

Best regards, Michael.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: kbrown@cornell.edu,  20119@debbugs.gnu.org
> Date: Mon, 16 Mar 2015 19:56:25 +0100
> 
> It could be also the case that it is an error in file name primitives on
> w32.

Could be, but at least "C-x C-f" will happily create such files.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Cc: kbrown@cornell.edu,  20119@debbugs.gnu.org
>> Date: Mon, 16 Mar 2015 19:56:25 +0100
>> 
>> It could be also the case that it is an error in file name primitives on
>> w32.
>
> Could be, but at least "C-x C-f" will happily create such files.

One case on w32 I could reproduce is the following: There is just one
special file "~/ file name with spaces ". The first test is OK, the
other two tests not (wrong handling of trailing spaces)

(file-exists-p "~/ file name with spaces ")
=> t

(file-exists-p "~/ file name with spaces")
=> t

(directory-files "~/")
=> (" file name with spaces" ...)

Best regards, Michael.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: kbrown@cornell.edu,  20119@debbugs.gnu.org
> Date: Tue, 17 Mar 2015 12:11:27 +0100
> 
> One case on w32 I could reproduce is the following: There is just one
> special file "~/ file name with spaces ". The first test is OK, the
> other two tests not (wrong handling of trailing spaces)
> 
> (file-exists-p "~/ file name with spaces ")
> => t
> 
> (file-exists-p "~/ file name with spaces")
> => t
> 
> (directory-files "~/")
> => (" file name with spaces" ...)

Where do you see errors in primitives here?  They are just
peculiarities of the underlying filesystem.

Like I said: such file names should be avoided, but they are not
disallowed.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> One case on w32 I could reproduce is the following: There is just one
>> special file "~/ file name with spaces ". The first test is OK, the
>> other two tests not (wrong handling of trailing spaces)
>> 
>> (file-exists-p "~/ file name with spaces ")
>> => t
>> 
>> (file-exists-p "~/ file name with spaces")
>> => t
>> 
>> (directory-files "~/")
>> => (" file name with spaces" ...)
>
> Where do you see errors in primitives here?  They are just
> peculiarities of the underlying filesystem.
>
> Like I said: such file names should be avoided, but they are not
> disallowed.

I do not believe it is important, but in my naive feeling
"~/ file name with spaces " and "~/ file name with spaces" are different.

Especially, since leading spaces in file names are treated:

(file-exists-p "~/file name with spaces ")
=> nil

Feel free to ignore this; in tramp-tests.el such tests are skipped when
running on w32.

Best regards, Michael.

bug#20119: 25.0.50; tramp-test30-special-characters hangs on Cygwin

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: kbrown@cornell.edu,  20119@debbugs.gnu.org
> Date: Tue, 17 Mar 2015 15:46:26 +0100
> 
> >> (file-exists-p "~/ file name with spaces ")
> >> => t
> >> 
> >> (file-exists-p "~/ file name with spaces")
> >> => t
> >> 
> >> (directory-files "~/")
> >> => (" file name with spaces" ...)
> >
> > Where do you see errors in primitives here?  They are just
> > peculiarities of the underlying filesystem.
> >
> > Like I said: such file names should be avoided, but they are not
> > disallowed.
> 
> I do not believe it is important, but in my naive feeling
> "~/ file name with spaces " and "~/ file name with spaces" are different.
> 
> Especially, since leading spaces in file names are treated:
> 
> (file-exists-p "~/file name with spaces ")
> => nil

The Win32 file-name related APIs "normalize" whitespace, and this is
the result.  The only way to work around this is bypass the Win32
layer altogether, and work on lower levels, where many nice features
of Win32 need to be done by hand.  Too much trouble for too small a
gain, IMO.