unable to visit /ftp:USER@MACHINE.com:~

unable to visit /ftp:USER@MACHINE.com:~

[Gregor Zattler]

Dear emacs users,

I'm unable to visit /ftp:USER@MACHINE.com:~
although there is a line

machine  MACHINE.com port ftp  login USER     password "PASSWORD"

in my ~/.authinfo.gpg

with the correct infos instead of the capitalised parts.

This is no configuration issue, since this:

EMACS -Q -nw --eval "(setq auth-source-debug 'trivia)" --eval '(setq ange-ftp-netrc-filename "~/.authinfo.gpg")'   /ftp:USER@MACHINE.com:~

does not work for emacs24, emacs25 and emacs-snapshot (emacs from
git as of today) as values of EMACS.  I'm asked by gpg-agent for
my passphrase to the key which decrypts ~/.authinfo.gpg and
actually I am able to visit this file and see it's decrypted
content.

Without setting ange-ftp-netrc-filename I'm not asked by
gpg-agent but instead emacs asks me for the password of
USER@MACHINE.com.


but the result is otherwise the same (besides the line regarding
authinfo.gpg):


*Messages*:
For information about GNU Emacs and the GNU system, type C-h C-a.
Decrypting /home/grfz/.authinfo.gpg...done
Opening FTP connection to ha01s001.org-dns.com...
tramp-file-name-handler: FTP Error: OPEN request failed: open ha01s001.org-dns.com



*ftp USER@MACHINE.com*
ftp> open ha01s001.org-dns.com
open ha01s001.org-dns.com
Connected to ha01s001.org-dns.com.
220 ProFTPD 1.3.5b Server (ProFTPD) [212.144.99.144]
234 AUTH TLS successful
[SSL Cipher ECDHE-RSA-AES256-GCM-SHA384]
200 PBSZ 0 successful
200 Protection set to Private
[Encrypted data transfer.]
ftp>

If I then type ls in the *ftp USER@MACHINE.com* (Internal
Ange-ftp:run) buffer I get:

ftp> ls
ls
530 Please login with USER and PASS
ftp: bind: Address already in use
ftp>

This is on debian 9.1 (stretch) with ftp provided by package
ftp-ssl:

~$ type -a ftp
ftp is /usr/bin/ftp
~$ ls -l /usr/bin/ftp
lrwxrwxrwx 1 root root 21 Jul  1 15:38 /usr/bin/ftp -> /etc/alternatives/ftp*
~$ ls -l /etc/alternatives/ftp
lrwxrwxrwx 1 root root 16 Jul  1 15:38 /etc/alternatives/ftp -> /usr/bin/ftp-ssl*
~$ file /usr/bin/ftp-ssl
/usr/bin/ftp-ssl: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=e7426e29e339b8432948fadea38cf0137e6b3e62, stripped
~$





Any ideas how to make emacs/tramp connect to this ftp site?

Thanks for your reading, Gregor

Re: unable to visit /ftp:USER@MACHINE.com:~

[Michael Albinus]

Gregor Zattler <telegraph@gmx.net> writes:

> Dear emacs users,

Hi Gregor,

> I'm unable to visit /ftp:USER@MACHINE.com:~
> although there is a line
>
> machine  MACHINE.com port ftp  login USER     password "PASSWORD"
>
> in my ~/.authinfo.gpg
>
> with the correct infos instead of the capitalised parts.
>
> This is no configuration issue, since this:
>
> EMACS -Q -nw --eval "(setq auth-source-debug 'trivia)" --eval '(setq
> ange-ftp-netrc-filename "~/.authinfo.gpg")' /ftp:USER@MACHINE.com:~
>
> does not work for emacs24, emacs25 and emacs-snapshot (emacs from
> git as of today) as values of EMACS.  I'm asked by gpg-agent for
> my passphrase to the key which decrypts ~/.authinfo.gpg and
> actually I am able to visit this file and see it's decrypted
> content.

I believe ange-ftp.el is not prepared to use auth-source.el. It is still
using good old ~/.netrc, with proper entries. Your entry above shall
seerve in ~/.netrc, without the "port" token.

> Without setting ange-ftp-netrc-filename I'm not asked by
> gpg-agent but instead emacs asks me for the password of
> USER@MACHINE.com.
>
> but the result is otherwise the same (besides the line regarding
> authinfo.gpg):
>
> *Messages*:
> For information about GNU Emacs and the GNU system, type C-h C-a.
> Decrypting /home/grfz/.authinfo.gpg...done
> Opening FTP connection to ha01s001.org-dns.com...
> tramp-file-name-handler: FTP Error: OPEN request failed: open
> ha01s001.org-dns.com
>
> *ftp USER@MACHINE.com*
> ftp> open ha01s001.org-dns.com
> open ha01s001.org-dns.com
> Connected to ha01s001.org-dns.com.
> 220 ProFTPD 1.3.5b Server (ProFTPD) [212.144.99.144]
> 234 AUTH TLS successful
> [SSL Cipher ECDHE-RSA-AES256-GCM-SHA384]
> 200 PBSZ 0 successful
> 200 Protection set to Private
> [Encrypted data transfer.]
> ftp>
>
> If I then type ls in the *ftp USER@MACHINE.com* (Internal
> Ange-ftp:run) buffer I get:
>
> ftp> ls
> ls
> 530 Please login with USER and PASS
> ftp: bind: Address already in use
> ftp>
>
> This is on debian 9.1 (stretch) with ftp provided by package
> ftp-ssl:
>
> ~$ type -a ftp
> ftp is /usr/bin/ftp
> ~$ ls -l /usr/bin/ftp
> lrwxrwxrwx 1 root root 21 Jul  1 15:38 /usr/bin/ftp -> /etc/alternatives/ftp*
> ~$ ls -l /etc/alternatives/ftp
> lrwxrwxrwx 1 root root 16 Jul 1 15:38 /etc/alternatives/ftp ->
> /usr/bin/ftp-ssl*
> ~$ file /usr/bin/ftp-ssl
> /usr/bin/ftp-ssl: ELF 64-bit LSB shared object, x86-64, version 1
> (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
> for GNU/Linux 2.6.32,
> BuildID[sha1]=e7426e29e339b8432948fadea38cf0137e6b3e62, stripped
> ~$
>
> Any ideas how to make emacs/tramp connect to this ftp site?

I've just tried plain ftp in Emacs to "/ftp:anonymous@ftp.gnu.org:",
this works w/o problems on my machine. Could you try it also?

In your case, ftp-ssl is used. No idea whether it makes a difference, on
my machine (Ubuntu 17.04) I have

~/src/emacs> which ftp
/usr/bin/ftp
:~/src/emacs> ls -l /usr/bin/ftp
lrwxrwxrwx 1 root root 21 Mar 12  2012 /usr/bin/ftp -> /etc/alternatives/ftp
~/src/emacs> ls -l /etc/alternatives/ftp
lrwxrwxrwx 1 root root 19 Oct 24  2015 /etc/alternatives/ftp -> /usr/bin/netkit-ftp

> Thanks for your reading, Gregor

Best regards, Michael.

Re: unable to visit /ftp:USER@MACHINE.com:~

[Gregor Zattler]

Hi Michael,
* Michael Albinus <michael.albinus@gmx.de> [2017-08-13; 10:57]:
> Gregor Zattler <telegraph@gmx.net> writes:
>> I'm unable to visit /ftp:USER@MACHINE.com:~
>> although there is a line
>>
>> machine  MACHINE.com port ftp  login USER     password "PASSWORD"
>>
>> in my ~/.authinfo.gpg
>>
>> with the correct infos instead of the capitalised parts.
>>
>> This is no configuration issue, since this:
>>
>> EMACS -Q -nw --eval "(setq auth-source-debug 'trivia)" --eval '(setq
>> ange-ftp-netrc-filename "~/.authinfo.gpg")' /ftp:USER@MACHINE.com:~
>>
>> does not work for emacs24, emacs25 and emacs-snapshot (emacs from
>> git as of today) as values of EMACS.  I'm asked by gpg-agent for
>> my passphrase to the key which decrypts ~/.authinfo.gpg and
>> actually I am able to visit this file and see it's decrypted
>> content.
>
> I believe ange-ftp.el is not prepared to use auth-source.el. It is still
> using good old ~/.netrc, with proper entries. Your entry above shall
> seerve in ~/.netrc, without the "port" token.

I copied the respective line to a new ~./netrc, deleted the "port
ftp" part and redid the test with the above quoted command line:
I was asked for the passphrase for the key to .authinfo.gpg and
the problem was the same.  Without .authinfo.gpg I'm asked for
the ftp password but the problem keeps the same.

When I reuse the ftp process in *ftp USER@MACHINE.com* issuing a
ls command I get a "530 Please login with USER and PASS" which
for me indicates that ange-ftp did not log me in, either did not
try or it failed.  I guess it's the former, since I'm able to log
in manually in the *ftp USER@MACHINE.com* buffer:

ftp> user USER
user USER
331 Password required for USER
Password: PASSWORD

230 User USER logged in
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> dir
dir
200 PORT command successful
425 Unable to build data connection: Connection timed out
ftp>

Therefore ange-ftp should at least be able to log me in.
Actually with argument -p I even see the directory listing.

> I've just tried plain ftp in Emacs to "/ftp:anonymous@ftp.gnu.org:",
> this works w/o problems on my machine. Could you try it also?

Aha!:

ftp> open ftp.gnu.org
open ftp.gnu.org
Connected to ftp.gnu.org.
220 GNU FTP server ready.
530 Please login with USER and PASS.
530 Please login with USER and PASS.
SSL not available
ftp>


I then installed netkit-ftp from package inetutils-ftp.  With
this AND argument -p it works wonderfully, without -p ls won't
finish -- but without ssl encrytion, meaning passwords go
unencrypted over the wires.


It's strange that ange-ftp does not work with ftp/ftp-ssl as in
debian stretch since I'm able to use it manually (with -p):

$ /usr/bin/ftp-ssl -p  ftp.gnu.org
Connected to ftp.gnu.org.
220 GNU FTP server ready.
Name (ftp.gnu.org:grfz): anonymous
530 Please login with USER and PASS.
530 Please login with USER and PASS.
SSL not available
230-Due to U.S. Export Regulations, all cryptographic software on this
230-site is subject to the following legal notice:
230-
230-    This site includes publicly available encryption source code
230-    which, together with object code resulting from the compiling of
230-    publicly available source code, may be exported from the United
230-    States under License Exception "TSU" pursuant to 15 C.F.R. Section
230-    740.13(e).
230-
230-This legal notice applies to cryptographic software only. Please see
230-the Bureau of Industry and Security (www.bxa.doc.gov) for more
230-information about current U.S. regulations.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
227 Entering Passive Mode (208,118,235,20,96,200).
150 Here comes the directory listing.
lrwxrwxrwx    1 0        0               8 Aug 20  2004 CRYPTO.README -> .message
-rw-r--r--    1 0        0           17864 Oct 23  2003 MISSING-FILES
-rw-r--r--    2 0        0            4178 Aug 13  2003 MISSING-FILES.README
-rw-r--r--    1 0        0            1962 Dec 22  2016 README
-rw-r--r--    1 0        0          405121 Oct 23  2003 before-2003-08-01.md5sums.asc
-rw-r--r--    1 0        0          238005 Aug 12 10:25 find.txt.gz
drwxrwxr-x  316 0        3003        12288 May 17 04:15 gnu
drwxrwxr-x    3 0        3003         4096 Mar 10  2011 gnu+linux-distros
-rw-r--r--    1 0        0          444682 Aug 12 10:25 ls-lrRt.txt.gz
drwxr-xr-x    3 0        0            4096 Apr 20  2005 mirrors
lrwxrwxrwx    1 0        0              11 Apr 15  2004 non-gnu -> gnu/non-gnu
drwxr-xr-x   90 0        0            4096 Mar 31  2015 old-gnu
lrwxrwxrwx    1 0        0               1 Aug 05  2003 pub -> .
drwxr-xr-x    2 0        0            4096 Nov 08  2007 savannah
drwxr-xr-x    2 0        0            4096 Aug 02  2003 third-party
drwxr-xr-x    2 0        0            4096 Apr 07  2009 tmp
drwxr-xr-x    2 0        0            4096 May 07  2013 video
-rw-r--r--    1 0        0            1112 Mar 22 17:32 welcome.msg
226 Directory send OK.
ftp>

It also works with the credentials to the ftp account to my
webspace.

So there seems to be no ftp client in debian/stretch compatible
with ange-ftp which supports ssl :-(


Do you think this merits a bug report against emacs?


Thanks for your help, Gregor

P.S.: Thanks also for your super fast action regarding the
documentation bug about defunct ange-ftp-lovers mailing list.

Re: unable to visit /ftp:USER@MACHINE.com:~

[Michael Albinus]

Gregor Zattler <telegraph@gmx.net> writes:

> Hi Michael,

Hi Gregor,

>> I've just tried plain ftp in Emacs to "/ftp:anonymous@ftp.gnu.org:",
>> this works w/o problems on my machine. Could you try it also?
>
> Aha!:
>
> ftp> open ftp.gnu.org
> open ftp.gnu.org
> Connected to ftp.gnu.org.
> 220 GNU FTP server ready.
> 530 Please login with USER and PASS.
> 530 Please login with USER and PASS.
> SSL not available
> ftp>
>
> So there seems to be no ftp client in debian/stretch compatible
> with ange-ftp which supports ssl :-(
>
> Do you think this merits a bug report against emacs?

Too late :-)

I've installed ftp-ssl on my machine, and played with it and
ange-ftp. Same problem accessing "/ftp:anonymous@ftp.gnu.org:". But
finally, I've got it running:

1. ftp-ssl writes an additional line ange-ftp is not aware of:

SSL not available

I've submitted a patch to the Emacs repo adding this message in
ange-ftp.el. I let ange-ftp just skip this line.

2. ftp-ssl has an additional feature to support readline/editline. This
comes in the way of ange-ftp batch processing. You must suppress it.

There is `ange-ftp-ftp-program-args', which contains already useful
arguments. I've added "-e". And voilà, with the resulting setting
'("-e" "-i" "-n" "-g" "-v") I have been able to access
"/ftp:anonymous@ftp.gnu.org:".

> Thanks for your help, Gregor

Best regards, Michael.

Re: unable to visit /ftp:USER@MACHINE.com:~

[Gregor Zattler]

Hi Michael,
* Michael Albinus <michael.albinus@gmx.de> [2017-08-14; 18:21]:
> Gregor Zattler <telegraph@gmx.net> writes:
>> So there seems to be no ftp client in debian/stretch compatible
>> with ange-ftp which supports ssl :-(
>>
>> Do you think this merits a bug report against emacs?
>
> Too late :-)
>
> I've installed ftp-ssl on my machine, and played with it and
> ange-ftp. Same problem accessing "/ftp:anonymous@ftp.gnu.org:". But
> finally, I've got it running:
>
> 1. ftp-ssl writes an additional line ange-ftp is not aware of:
>
> SSL not available
>
> I've submitted a patch to the Emacs repo adding this message in
> ange-ftp.el. I let ange-ftp just skip this line.
>
> 2. ftp-ssl has an additional feature to support readline/editline. This
> comes in the way of ange-ftp batch processing. You must suppress it.
>
> There is `ange-ftp-ftp-program-args', which contains already useful
> arguments. I've added "-e". And voilà, with the resulting setting
> '("-e" "-i" "-n" "-g" "-v") I have been able to access
> "/ftp:anonymous@ftp.gnu.org:".

Great, thanks.  I checked it out.  I had to add "-p" also,
otherwise it would hang before showing the files listing.  Now it
works in case of a connection without SSL.


With regard to my original use case (ftp access to my webspace)
though it still does not work:

*Messages*
For information about GNU Emacs and the GNU system, type C-h C-a.
Opening FTP connection to MACHINE.com...done
Logging in as user USER@MACHINE.com...
tramp-file-name-handler: FTP Error: USER request failed: 234 AUTH TLS successful


*ftp USER@MACHINE.com*
ftp> open MACHINE.com
Connected to MACHINE.com.
220 ProFTPD 1.3.5b Server (ProFTPD) [212.144.99.144]
user "USER" Turtle Power!
234 AUTH TLS successful
[SSL Cipher ECDHE-RSA-AES256-GCM-SHA384]
200 PBSZ 0 successful
200 Protection set to Private
[Encrypted data transfer.]
ftp> 331 Password required for USER
230 User USER logged in
Remote system type is UNIX.
Using binary mode to transfer files.


If I then enter buffer *ftp USER@MACHINE.com* and type "ls" it
works there, but something is still missing for working fully
automated:

ftp> ls
227 Entering Passive Mode (212,144,99,144,223,133).
150 Opening BINARY mode data connection for file list
drwxr-x---   2 USER   psacln       4096 Apr 27  2015 error_docs
drwxr-x---   3 USER   psaserv      4096 Aug 11 15:31 httpdocs
drwx------   2 USER   root         4096 Aug 14 03:40 logs
drwx------   3 USER   psacln       4096 Aug 11 16:20 private
-rw-r--r--   1 USER   psacln         10 Aug 13 09:47 test.txt
226 Transfer complete
ftp>



The culprit is probably the "234 AUTH TLS successful" line, since
in ange-ftp.el, there is no "234" as part of a regex.

This is now perhaps a problem specific to "ProFTPD 1.3.5b Server
(ProFTPD)".  I could provide you with credentials if you are
interested in this.

Regardless of this last issue: Thanks again for your fast
response/fix.

Ciao; Gregor 

Re: unable to visit /ftp:USER@MACHINE.com:~

[Michael Albinus]

Gregor Zattler <telegraph@gmx.net> writes:

> Hi Michael,

Hi Gregor,

> This is now perhaps a problem specific to "ProFTPD 1.3.5b Server
> (ProFTPD)".  I could provide you with credentials if you are
> interested in this.

Pls do; I'll check.

> Ciao; Gregor 

Best regards, Michael.

Re: unable to visit /ftp:USER@MACHINE.com:~

[Michael Albinus]

Gregor Zattler <telegraph@gmx.net> writes:

> Hi Michael,

Hi Gregor,

> The culprit is probably the "234 AUTH TLS successful" line, since
> in ange-ftp.el, there is no "234" as part of a regex.

This and other lines. Should work now, with my latest patch uploaded to
the Emacs repo.

Btw, you don't need to set "-p" explicitely. See `ange-ftp-try-passive-mode'
and `ange-ftp-passive-host-alist'.

> Ciao; Gregor 

Best regards, Michael.

Re: unable to visit /ftp:USER@MACHINE.com:~

[Gregor Zattler]

Hi Michael,
* Michael Albinus <michael.albinus@gmx.de> [2017-08-16; 12:01]:
> Gregor Zattler <telegraph@gmx.net> writes:
>> The culprit is probably the "234 AUTH TLS successful" line, since
>> in ange-ftp.el, there is no "234" as part of a regex.
>
> This and other lines. Should work now, with my latest patch uploaded to
> the Emacs repo.

Yes, I checked: it does.  This is great.  Thank you for your help.

> Btw, you don't need to set "-p" explicitely. See
> `ange-ftp-try-passive-mode' and `ange-ftp-passive-host-alist'.

Also yes, the only needed extra arg with respect to emacs'
default configuration of ange-ftp-ftp-program-args is "-e", as
you posted prior.

And for the record: ange-ftp is able to use login credentials in
~/.authinfo.gpg when told so via ange-ftp-netrc-filename.


I now have to find out, what in my configuration hinders
emacs/ange-ftp to list ftp directory contents.  I assume it's my
dired-listing-switches setting, I'll investigate after having a
sleep.

Thank you very much, Gregor

Re: unable to visit /ftp:USER@MACHINE.com:~

[Gregor Zattler]

Hi emacs lovers,
* Gregor Zattler <telegraph@gmx.net> [2017-08-17; 00:38]:↵
> * Michael Albinus <michael.albinus@gmx.de> [2017-08-16; 12:01]:
[... emacs trunk works now with ftp-ssl ...]
>> Btw, you don't need to set "-p" explicitely. See
>> `ange-ftp-try-passive-mode' and `ange-ftp-passive-host-alist'.
>
> Also yes, the only needed extra arg with respect to emacs'
> default configuration of ange-ftp-ftp-program-args is "-e", as
> you posted prior.
>
> And for the record: ange-ftp is able to use login credentials in
> ~/.authinfo.gpg when told so via ange-ftp-netrc-filename.
>
>
> I now have to find out, what in my configuration hinders
> emacs/ange-ftp to list ftp directory contents.  I assume it's my
> dired-listing-switches setting, I'll investigate after having a
> sleep.

OK, I have dired-listing-switches set to "-altrF
--group-directories-first --block-size='1" and I use package
dired-quick-sort which injects "--sort=version" into the ls
args.  Both do not work with ftp, at least not with the ftp
server I use.

I tried to replace the only occurrence of dired-listing-switches
in ange-ftp.el with "-al" and ditto with dired-actual-switches
but had no luck.

Since I mostly use dired on local file systems I want to use
these customizations.  At the same time I do not know how to
unload the dired-quick-sort package and it's customizations to
emacs variables.  Also I found no pre-ange-ftp-hook where I could
uncustomize these variables.  Therefore I will use an mostly
uncustomized emacs invocation for interacting with the ftp server
to my webspace.

Thanks again to Michael for making ange-ftp work with ftp-ssl.

Ciao; Gregor