bug#15058: 24.3; ftp is still broken

bug#15058: 24.3; ftp is still broken

[Devon Sean McCullough]

emacs -Q /ftp@ftp.gnu.org:

Should display the FTP.GNU.ORG: directory but instead bombs -
for details, look in the *ftp ftp@ftp.gnu.org* buffer.

In GNU Emacs 24.3.2 (x86_64-apple-darwin10.8.0, NS apple-appkit-1038.36)
 of 2013-07-09 on Apple-Computers-MacBook-Air.local
Windowing system distributor `Apple', version 10.3.1038
Configured using:
 `configure '--with-ns' 'CC=gcc''

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
M-x r e p o r t SPC e m SPC SPC <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Opening FTP connection to ftp.gnu.org...done
Logging in as user ftp@ftp.gnu.org...done
Getting PWD... [2 times]
Warning! Unable to get home directory
Getting PWD... [2 times]
byte-code: Unable to obtain CWD

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rfc822 mml easymenu mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils ange-ftp
tramp-ftp tramp tramp-compat auth-source eieio byte-opt bytecomp
byte-compile cconv gnus-util mm-util mail-prsvr password-cache
tramp-loaddefs shell pcomplete comint ansi-color ring format-spec advice
help-fns cl-lib advice-preload time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel ns-win tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process ns multi-tty emacs)

bug#15058: 24.3; ftp is still broken

[Eli Zaretskii]

> From: Devon Sean McCullough <Emacs-Hacker2013@jovi.net>
> Date: Fri, 9 Aug 2013 18:10:49 +0800
> 
> emacs -Q /ftp@ftp.gnu.org:
> 
> Should display the FTP.GNU.ORG: directory but instead bombs -
> for details, look in the *ftp ftp@ftp.gnu.org* buffer.

Try this instead:

  emacs -Q /ftp:ftp@ftp.gnu.org:

IOW, you must have the METHOD: part before the URL.

I think this is a cockpit error, since the Emacs User Manual clearly
describes the correct syntax.

bug#15058: 24.3; ftp is still broken

[Eli Zaretskii]

> From: Devon Sean McCullough <Emacs-Hacker2013@jovi.net>
> Date: Fri, 9 Aug 2013 21:03:40 +0800
> 
> Really?  When was /HOST: syntax deprecated?

It wasn't.

Actually, I take back what I wrote earlier: your original example
works fine for me, even without the explicit "ftp:".

Maybe it's a Darwin thing or something specific to your system setup.

bug#15058: 24.3; ftp is still broken

[Stefan Monnier]

>> emacs -Q /ftp@ftp.gnu.org:
> Try this instead:
>   emacs -Q /ftp:ftp@ftp.gnu.org:

I guess Tramp could try and output a useful error message to help the
user fix his file name.


        Stefan

bug#15058: 24.3; ftp is still broken

[Glenn Morris]

>> From: Devon Sean McCullough <Emacs-Hacker2013@jovi.net>
>> Date: Fri, 9 Aug 2013 21:03:40 +0800
>> 
>> Really?  When was /HOST: syntax deprecated?

That's not what you used. You used /METHOD@HOST:, which is not
documented to work.

http://www.gnu.org/software/emacs/manual/html_node/emacs/Remote-Files.html

bug#15058: 24.3; ftp is still broken

[Devon Sean McCullough]

On 2013 Aug 9, at 11:53 PM, Glenn Morris wrote:
> You used /METHOD@HOST:, which is not documented to work.

That's /USER@HOST: where USER is ftp, HOST is ftp.gnu.org
and FILENAME is empty to specify the landing page////dir
configured by the server administrator,
a useful and important capability.

> http://www.gnu.org/software/emacs/manual/html_node/emacs/Remote-Files.html
... /USER@HOST:FILENAME ...
  1. If the host name starts with `ftp.' (with dot), Emacs uses FTP.
  2. If the user name is `ftp' or `anonymous', Emacs uses FTP.

		Peace
			--Devon

bug#15058: 24.3; ftp is still broken

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Fri, 09 Aug 2013 11:53:42 -0400
> 
> 
> >> From: Devon Sean McCullough <Emacs-Hacker2013@jovi.net>
> >> Date: Fri, 9 Aug 2013 21:03:40 +0800
> >> 
> >> Really?  When was /HOST: syntax deprecated?
> 
> That's not what you used. You used /METHOD@HOST:, which is not
> documented to work.
> 
> http://www.gnu.org/software/emacs/manual/html_node/emacs/Remote-Files.html

It's /user@host:, and it works fine for me on fencepost.gnu.org.

bug#15058: 24.3; ftp is still broken

[Devon Sean McCullough]

On 2013 Aug 9, at 9:45 PM, Eli Zaretskii wrote:
> Maybe it's a Darwin thing or something specific to your system setup.

Yes, the bug is OS specific:

CentOS 21.4.1 	ok
Debian 6.0.7 	BUG
MacOS 10.6.8 	BUG
Ubuntu 10.04 	ok

		Peace
			--Devon

bug#15058: 24.3; ftp is still broken

[Glenn Morris]

Eli Zaretskii wrote:

> It's /user@host:, and it works fine for me on fencepost.gnu.org.

Ah, OK (I waa expecting the user to be "anonymous").
Works for me too, on 2 different systems.

bug#15058: 24.3; ftp is still broken

[Stefan Monnier]

>> Really?  When was /HOST: syntax deprecated?
> It wasn't.

We slightly changed the way it works because of the ambiguity between
/HOST: and /METHOD:, so IIRC /HOST: still works fine, but only if HOST
is not also a valid METHOD.  So you need "/ftp:ftp" in order to connect
to a host named "ftp".


        Stefan

bug#15058: 24.3; ftp is still broken

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Devon Sean McCullough <Emacs-Hacker2013@jovi.net>,  15058@debbugs.gnu.org
> Date: Fri, 09 Aug 2013 14:23:22 -0400
> 
> >> Really?  When was /HOST: syntax deprecated?
> > It wasn't.
> 
> We slightly changed the way it works because of the ambiguity between
> /HOST: and /METHOD:, so IIRC /HOST: still works fine, but only if HOST
> is not also a valid METHOD.  So you need "/ftp:ftp" in order to connect
> to a host named "ftp".

Again, on fencepost, I don't need the METHOD.

bug#15058: 24.3; ftp is still broken

[Glenn Morris]

Devon Sean McCullough wrote:

> emacs -Q /ftp@ftp.gnu.org:
>
> Should display the FTP.GNU.ORG: directory but instead bombs -

What do you actually mean by "it bombs"?

> for details, look in the *ftp ftp@ftp.gnu.org* buffer.

Please show us the contents, since no-one else seems to be able to
reproduce this.

> Opening FTP connection to ftp.gnu.org...done
> Logging in as user ftp@ftp.gnu.org...done
> Getting PWD... [2 times]
> Warning! Unable to get home directory

What's going on here?

Seems to resemble
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=1972

Something to do with active/passive ftp?

bug#15058: 24.3; ftp is still broken

[Devon Sean McCullough]

On 2013 Aug 9, at 9:45 PM, Eli Zaretskii wrote:
> Maybe it's a Darwin thing or something specific to your system setup.

The bug seems to depend on whether /usr/bin/ftp
re-writes certain server responses ...

MacOS and Debian BUG: 	(ange-ftp-get-pwd "ftp.gnu.org" "ftp") ==> (nil . "Remote directory: /")
CentOS and Ubuntu OK: 	(ange-ftp-get-pwd "ftp.gnu.org" "ftp") ==> (t . "257 \"/\"")

... apparently emacs is blind to the re-written PWD reply.

		Peace
			--Devon

bug#15058: 24.3; ftp is still broken

[Glenn Morris]

I think we'd have wasted less time on this had you said "bug#8346
still exists" from the start.

bug#15058: 24.3; ftp is still broken

[Glenn Morris]

Maybe this fixes it

*** lisp/net/ange-ftp.el	2013-08-09 21:22:44 +0000
--- lisp/net/ange-ftp.el	2013-08-09 23:44:10 +0000
***************
*** 700,706 ****
    "Regular expression matching the start of a multiline FTP reply.")
  
  (defvar ange-ftp-good-msgs
!   "^220 \\|^230 \\|^226 \\|^25. \\|^221 \\|^200 \\|^[Hh]ash mark"
    "Regular expression matching FTP \"success\" messages.")
  
  ;; CMS and the odd VMS machine say 200 Port rather than 200 PORT.
--- 700,706 ----
    "Regular expression matching the start of a multiline FTP reply.")
  
  (defvar ange-ftp-good-msgs
!   "^220 \\|^230 \\|^226 \\|^25. \\|^221 \\|^200 \\|^[Hh]ash mark\\|Remote directory"
    "Regular expression matching FTP \"success\" messages.")
  
  ;; CMS and the odd VMS machine say 200 Port rather than 200 PORT.
***************
*** 3021,3026 ****
--- 3021,3027 ----
      (if (car result)
  	(save-match-data
  	  (and (or (string-match "\"\\([^\"]*\\)\"" line)
+ 		   (string-match "Remote directory: \\(.*\\)" line)
  		   (string-match " \\([^ ]+\\) " line))	; stone-age VMS servers!
  	       (setq dir (match-string 1 line)))))
      (cons dir line)))

bug#15058: 24.3; ftp is still broken

[Wolfgang Jenkner]

On Fri, Aug 09 2013, Devon Sean McCullough wrote:

> On 2013 Aug 9, at 9:45 PM, Eli Zaretskii wrote:
>> Maybe it's a Darwin thing or something specific to your system setup.
>
> The bug seems to depend on whether /usr/bin/ftp
> re-writes certain server responses ...
>
> MacOS and Debian BUG: 	(ange-ftp-get-pwd "ftp.gnu.org" "ftp") ==> (nil . "Remote directory: /")

If this is tnftp (aka lukemftp[1]) then there's actually no server
response to report here because the client reuses a cached value and
doesn't send anything to the server at this point.  The value comes from
automatically sending a `pwd' after logging in.  You can set the `debug'
flag to observe this.

[1] ftp://ftp.netbsd.org/pub/NetBSD/misc/tnftp/tnftp-20091122.tar.gz

> CentOS and Ubuntu OK: 	(ange-ftp-get-pwd "ftp.gnu.org" "ftp") ==> (t . "257 \"/\"")

Wolfgang

bug#15058: 24.3; ftp is still broken

[Wolfgang Jenkner]

On Sat, Aug 10 2013, Wolfgang Jenkner wrote:

> [1] ftp://ftp.netbsd.org/pub/NetBSD/misc/tnftp/tnftp-20091122.tar.gz

There's actually a newer version of this (which compiles and works out
of the box on a GNU/Linux system here)

ftp://ftp.netbsd.org/pub/NetBSD/misc/tnftp/tnftp-20130505.tar.gz

Wolfgang

bug#15058: 24.3; ftp is still broken

[Devon Sean McCullough]

Glenn,

Thanks for the patch - this nearly identical one works great on MacOS.

		Peace
			--Devon

P.S. I added four characters (two carets, a colon and a space)
--- ange-ftp.el.~0~	2013-02-11 09:13:09.000000000 +0800
+++ ange-ftp.el	2013-08-10 15:50:44.000000000 +0800
@@ -700,7 +700,7 @@
   "Regular expression matching the start of a multiline FTP reply.")
 
 (defvar ange-ftp-good-msgs
-  "^220 \\|^230 \\|^226 \\|^25. \\|^221 \\|^200 \\|^[Hh]ash mark"
+  "^220 \\|^230 \\|^226 \\|^25. \\|^221 \\|^200 \\|^[Hh]ash mark\\|^Remote directory: "
   "Regular expression matching FTP \"success\" messages.")
 
 ;; CMS and the odd VMS machine say 200 Port rather than 200 PORT.
@@ -3021,6 +3021,7 @@
     (if (car result)
 	(save-match-data
 	  (and (or (string-match "\"\\([^\"]*\\)\"" line)
+		   (string-match "^Remote directory: \\(.*\\)" line)
 		   (string-match " \\([^ ]+\\) " line))	; stone-age VMS servers!
 	       (setq dir (match-string 1 line)))))
     (cons dir line)))