Emacs tramp troubles with old Sun

Emacs tramp troubles with old Sun

[Robert J. Chassell]

Today's GNU Emacs CVS snapshot, Tue, 2007 Nov  6    10:52 UTC
GNU Emacs 23.0.50.11 (i686-pc-linux-gnu, GTK+ Version 2.10.13) 
started with

     /usr/local/src/emacs/src/emacs -Q -D

A new 64 bit machine succeeded when I evaluated:

    (find-file "/ssh:bob@fp.gnu.org:~bob" nil) 

but a Sun dated May 2002 SunOS 5.9 sun4u sparc SUNW,Ultra-2
failed when I evaluated:

    (find-file "/ssh:bob@shell.berkshire.net:~bob" nil)

Incidently, on shell.berkshire.net

     $ test -e /tmp
    test: argument expected

     $ which test   
    /usr/ucb/test
     $ echo $PATH
    .:/bin:/usr/games:/usr/etc:/usr/ucb:/usr/local/bin:/usr/bin:/usr/X11/bin

I tried adding /usr/ucb/test to  tramp-find-file-exists-command 
but that failed.

I also put /usr/ucb/test in  tramp-get-test-command
and that failed, too.

The contents of both the *Backtrace* buffer and the 
*debug tramp/ssh bob@shell.berkshire.net* buffer are enclosed.
`tramp-verbose' is set to 8



The *Backtrace* buffer said:

Debugger entered--Lisp error: (file-error "Couldn't find exit status of `test -d /tmp/MAILCHECK\\:\\ cannot\\ unset'")
  signal(file-error ("Couldn't find exit status of `test -d /tmp/MAILCHECK\\:\\ cannot\\ unset'"))
  tramp-send-command-and-check(["ssh" "bob" "shell.berkshire.net" "/tmp/MAILCHECK: cannot unset"] "test -d /tmp/MAILCHECK\\:\\ cannot\\ unset")
  tramp-run-test("-d" "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  tramp-handle-file-directory-p("/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  apply(tramp-handle-file-directory-p "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  tramp-sh-file-name-handler(file-directory-p "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  apply(tramp-sh-file-name-handler file-directory-p "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  tramp-file-name-handler(file-directory-p "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  file-directory-p("/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  find-file-noselect("/ssh:bob@shell.berkshire.net:~bob" nil nil nil)
  find-file("/ssh:bob@shell.berkshire.net:~bob" nil)
  eval((find-file "/ssh:bob@shell.berkshire.net:~bob" nil))
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp)


With    (setq tramp-verbose 8)    the
*debug tramp/ssh bob@shell.berkshire.net* buffer said:

14:08:25 tramp-maybe-open-connection (3) # Opening connection for bob@shell.berkshire.net using ssh...
14:08:25 tramp-get-connection-property (7) # process-name nil
14:08:25 tramp-get-connection-property (7) # process-buffer nil
14:08:25 tramp-maybe-open-connection (6) # /bin/sh
14:08:25 tramp-maybe-open-connection (3) # Waiting 60s for local shell to come up...
14:08:25 tramp-get-connection-property (7) # check-remote-echo nil
14:08:25 tramp-get-connection-property (7) # check-remote-echo nil
14:08:25 tramp-get-connection-property (7) # check-remote-echo nil
14:08:25 tramp-get-connection-property (7) # check-remote-echo nil
14:08:25 tramp-wait-for-regexp (6) # 
$ 
14:08:25 tramp-get-file-property (8) #  gateway nil
14:08:25 tramp-set-connection-property (7) # temp-file /tmp/tramp.9871-iH
14:08:25 tramp-maybe-open-connection (3) # Sending command `ssh shell.berkshire.net -l bob  -e none && exit || exit'
14:08:25 tramp-get-connection-property (7) # process-name nil
14:08:25 tramp-get-connection-property (7) # remote-echo nil
14:08:25 tramp-send-command (6) # ssh shell.berkshire.net -l bob  -e none && exit || exit
14:08:25 tramp-get-connection-property (7) # process-name nil
14:08:25 tramp-get-connection-property (7) # chunksize nil
14:08:25 tramp-set-connection-property (7) # last-cmd-time (18224 30041 100699)
14:08:25 tramp-get-connection-property (7) # process-buffer nil
14:08:25 tramp-process-actions (3) # Waiting for prompts from remote shell
14:08:26 tramp-process-one-action (5) # Looking for regexp ".*ogin\( .*\)?: *\'" from remote shell
14:08:26 tramp-get-connection-property (7) # check-remote-echo nil
14:08:26 tramp-get-connection-property (7) # check-remote-echo nil
14:08:26 tramp-process-one-action (5) # Looking for regexp "^.*\([pP]assword\|[pP]assphrase\).*:

Re: Emacs tramp troubles with old Sun

[Michael Albinus]

"Robert J. Chassell" <bob@rattlesnake.com> writes:

> The contents of both the *Backtrace* buffer and the 
> *debug tramp/ssh bob@shell.berkshire.net* buffer are enclosed.
> `tramp-verbose' is set to 8

Unfortunately, it's not the complete debug buffer. Could you, please,
provide it?

Thanks, and best regards, Michael.

Re: Emacs tramp troubles with old Sun

[Robert J. Chassell]

Today's GNU Emacs CVS snapshot, Wed, 2007 Nov  6  10:52 UTC
GNU Emacs 23.0.50.11 (i686-pc-linux-gnu, GTK+ Version 2.10.13)
started with

     /usr/local/src/emacs/src/emacs -Q -D


    > The contents of ... the
    > *debug tramp/ssh bob@shell.berkshire.net* buffer [is] enclosed.
    > `tramp-verbose' is set to 8

    Unfortunately, it's not the complete debug buffer. Could you, please,
    provide it?

I thought I did provide it.  In any case, here it is again, with
`tramp-verbose' is set to 8.  This is all I got:

01:18:37 tramp-maybe-open-connection (3) # Opening connection for bob@shell.berkshire.net using ssh...
01:18:37 tramp-get-connection-property (7) # process-name nil
01:18:37 tramp-get-connection-property (7) # process-buffer nil
01:18:37 tramp-maybe-open-connection (6) # /bin/sh
01:18:37 tramp-maybe-open-connection (3) # Waiting 60s for local shell to come up...
01:18:37 tramp-get-connection-property (7) # check-remote-echo nil
01:18:37 tramp-get-connection-property (7) # check-remote-echo nil
01:18:37 tramp-get-connection-property (7) # check-remote-echo nil
01:18:37 tramp-get-connection-property (7) # check-remote-echo nil
01:18:37 tramp-wait-for-regexp (6) # 
$ 
01:18:37 tramp-get-file-property (8) #  gateway nil
01:18:37 tramp-set-connection-property (7) # temp-file /tmp/tramp.21835tJR
01:18:37 tramp-maybe-open-connection (3) # Sending command `ssh shell.berkshire.net -l bob  -e none && exit || exit'
01:18:37 tramp-get-connection-property (7) # process-name nil
01:18:37 tramp-get-connection-property (7) # remote-echo nil
01:18:37 tramp-send-command (6) # ssh shell.berkshire.net -l bob  -e none && exit || exit
01:18:37 tramp-get-connection-property (7) # process-name nil
01:18:37 tramp-get-connection-property (7) # chunksize nil
01:18:37 tramp-set-connection-property (7) # last-cmd-time (18225 4717 748014)
01:18:37 tramp-get-connection-property (7) # process-buffer nil
01:18:37 tramp-process-actions (3) # Waiting for prompts from remote shell
01:18:38 tramp-process-one-action (5) # Looking for regexp ".*ogin\( .*\)?: *\'" from remote shell
01:18:38 tramp-get-connection-property (7) # check-remote-echo nil
01:18:38 tramp-get-connection-property (7) # check-remote-echo nil
01:18:38 tramp-process-one-action (5) # Looking for regexp "^.*\([pP]assword\|[pP]assphrase\).*:

Re: Emacs tramp troubles with old Sun

[Michael Albinus]

"Robert J. Chassell" <bob@rattlesnake.com> writes:

>     Unfortunately, it's not the complete debug buffer. Could you, please,
>     provide it?
>
> I thought I did provide it.  In any case, here it is again, with
> `tramp-verbose' is set to 8.  This is all I got:

That's sad: again only 26 lines of the debug buffer, missing the
important part. Could you, please, append the debug buffer as
_enclosure_ of your message, encoded etc? Note that the debug buffer
might contain non ASCII characters, which could confuse mail clients.

Best regards, Michael.

Re: Emacs tramp troubles with old Sun

[Andreas Schwab]

Michael Albinus <michael.albinus@gmx.de> writes:

> "Robert J. Chassell" <bob@rattlesnake.com> writes:
>
>>     Unfortunately, it's not the complete debug buffer. Could you, please,
>>     provide it?
>>
>> I thought I did provide it.  In any case, here it is again, with
>> `tramp-verbose' is set to 8.  This is all I got:
>
> That's sad: again only 26 lines of the debug buffer,

Looks like there is a buggy MTA in your way (Gnus has no problem with
the embedded NUL).  There are 347 more lines in the mail, including
these lines:

01:18:40 tramp-send-command (6) # unset correct autocorrect MAILPATH MAILCHECK MAIL HISTORY CDPATH
01:18:40 tramp-get-connection-property (7) # process-name nil
01:18:40 tramp-get-connection-property (7) # chunksize 0
01:18:40 tramp-set-connection-property (7) # last-cmd-time (18225 4720 353854)
01:18:40 tramp-get-connection-property (7) # process-buffer nil
01:18:40 tramp-get-connection-property (7) # check-remote-echo nil
01:18:40 tramp-get-connection-property (7) # check-remote-echo nil
01:18:40 tramp-get-connection-property (7) # check-remote-echo nil
01:18:40 tramp-get-connection-property (7) # check-remote-echo nil
01:18:40 tramp-wait-for-regexp (6) # 

///8d1e0d285d76513c584ea93c9fec758c
01:18:40 tramp-get-connection-property (7) # process-name nil
01:18:40 tramp-get-connection-property (7) # remote-echo nil
01:18:40 tramp-send-command (6) # test -d /tmp/MAILCHECK\:\ cannot\ unset 2>/dev/null; echo tramp_exit_status $? 

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Emacs tramp troubles with old Sun

[Michael Albinus]

Andreas Schwab <schwab@suse.de> writes:

> Looks like there is a buggy MTA in your way (Gnus has no problem with
> the embedded NUL).

I read emacs-devel via gmane. In its archive, the messages from Bob
are also shortened.

On lists.gnu.org, the messages seem to be complete. Thanks for the hint!

> Andreas.

Best regards, Michael.

Re: Emacs tramp troubles with old Sun

[Michael Albinus]

"Robert J. Chassell" <bob@rattlesnake.com> writes:

> but a Sun dated May 2002 SunOS 5.9 sun4u sparc SUNW,Ultra-2
> failed when I evaluated:

That machine has an echoing shell, which was not compatible with my
recent changes for prompt detection. Should be fixed now in the Emacs
trunk. Could you, please, test the changes?

Thanks, and best regards, Michael.

Re: Emacs tramp troubles with old Sun

[Robert J. Chassell]

Today's GNU Emacs CVS snapshot, Wed, 2007 Nov  7  20:59 UTC
GNU Emacs 23.0.50.2 (i686-pc-linux-gnu, GTK+ Version 2.12.1)
started with

     /usr/local/src/emacs/src/emacs -Q -D


I can reach fp.gnu.org, but the old Sun still fails with

    Couldn't find exit status of `test -e /tmp'


I can reach it  with    ssh -l bob shell.berkshire.net

    Last login: Wed Nov  7 16:07:08 2007 from cpe-69-204-164-
    Sun Microsystems Inc.   SunOS 5.9       Generic May 2002
    tcsh: using dumb terminal settings.


    bob@robin ~ $  test -e /tmp
    test: argument expected

    bob@robin ~ $ which test
    /usr/ucb/test
    bob@robin ~ $  echo $PATH
    .:/bin:/usr/games:/usr/etc:/usr/ucb:/usr/local/bin:/usr/bin:/usr/X11/bin


The contents of both the *Backtrace* buffer and the
*debug tramp/ssh bob@shell.berkshire.net* buffer are enclosed.
`tramp-verbose' is set to 8

The *Backtrace* buffer:
Debugger entered--Lisp error: (file-error "Couldn't find exit status of `test -e /tmp'")
  signal(file-error ("Couldn't find exit status of `test -e /tmp'"))
  tramp-send-command-and-check(["ssh" "bob" "shell.berkshire.net" "/tmp"] "test -e /tmp")
  tramp-handle-file-exists-p("/ssh:bob@shell.berkshire.net:/tmp")
  apply(tramp-handle-file-exists-p "/ssh:bob@shell.berkshire.net:/tmp")
  tramp-sh-file-name-handler(file-exists-p "/ssh:bob@shell.berkshire.net:/tmp")
  apply(tramp-sh-file-name-handler file-exists-p "/ssh:bob@shell.berkshire.net:/tmp")
  tramp-file-name-handler(file-exists-p "/ssh:bob@shell.berkshire.net:/tmp")
  file-exists-p("/ssh:bob@shell.berkshire.net:/tmp")
  tramp-handle-file-attributes("/ssh:bob@shell.berkshire.net:/tmp")
  apply(tramp-handle-file-attributes "/ssh:bob@shell.berkshire.net:/tmp")
  tramp-sh-file-name-handler(file-attributes "/ssh:bob@shell.berkshire.net:/tmp")
  apply(tramp-sh-file-name-handler file-attributes "/ssh:bob@shell.berkshire.net:/tmp")
  tramp-file-name-handler(file-attributes "/ssh:bob@shell.berkshire.net:/tmp")
  file-attributes("/ssh:bob@shell.berkshire.net:/tmp")
  tramp-handle-file-truename("/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  apply(tramp-handle-file-truename "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  tramp-sh-file-name-handler(file-truename "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  apply(tramp-sh-file-name-handler file-truename "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  tramp-file-name-handler(file-truename "/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  file-truename("/ssh:bob@shell.berkshire.net:/tmp/MAILCHECK: cannot unset")
  find-file-noselect("/ssh:bob@shell.berkshire.net:~bob" nil nil nil)
  find-file("/ssh:bob@shell.berkshire.net:~bob" nil)
  eval((find-file "/ssh:bob@shell.berkshire.net:~bob" nil))
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp)



The *debug tramp/ssh bob@shell.berkshire.net* buffer:

21:18:00 tramp-maybe-open-connection (3) # Opening connection for
bob@shell.berkshire.net using ssh...  21:18:00
tramp-get-connection-property (7) # process-name nil 21:18:00
tramp-get-connection-property (7) # process-buffer nil 21:18:00
tramp-maybe-open-connection (6) # /bin/sh 21:18:00
tramp-maybe-open-connection (3) # Waiting 60s for local shell to come
up...  21:18:00 tramp-get-connection-property (7) # check-remote-echo
nil 21:18:00 tramp-get-connection-property (7) # check-remote-echo nil
21:18:00 tramp-get-connection-property (7) # check-remote-echo nil
21:18:00 tramp-get-connection-property (7) # check-remote-echo nil
21:18:00 tramp-wait-for-regexp (6) # $ 21:18:00
tramp-get-file-property (8) # gateway nil 21:18:00
tramp-set-connection-property (7) # temp-file /tmp/tramp.17957v7i
21:18:00 tramp-maybe-open-connection (3) # Sending command `ssh
shell.berkshire.net -l bob -e none && exit || exit' 21:18:00
tramp-get-connection-property (7) # process-name nil 21:18:00
tramp-get-connection-property (7) # remote-echo nil 21:18:00
tramp-send-command (6) # ssh shell.berkshire.net -l bob -e none &&
exit || exit 21:18:00 tramp-get-connection-property (7) # process-name
nil 21:18:00 tramp-get-connection-property (7) # chunksize nil
21:18:00 tramp-set-connection-property (7) # last-cmd-time (18226
11144 550206) 21:18:00 tramp-get-connection-property (7) #
process-buffer nil 21:18:00 tramp-process-actions (3) # Waiting for
prompts from remote shell 21:18:01 tramp-process-one-action (5) #
Looking for regexp ".*ogin\( .*\)?: *\'" from remote shell 21:18:01
tramp-get-connection-property (7) # check-remote-echo nil 21:18:01
tramp-get-connection-property (7) # check-remote-echo nil 21:18:01
tramp-process-one-action (5) # Looking for regexp
"^.*\([pP]assword\|[pP]assphrase\).*:

Re: Emacs tramp troubles with old Sun

[Michael Albinus]

"Robert J. Chassell" <bob@rattlesnake.com> writes:

> but the old Sun still fails with
>
>     Couldn't find exit status of `test -e /tmp'
>
>
> I can reach it  with    ssh -l bob shell.berkshire.net
>
>     Last login: Wed Nov  7 16:07:08 2007 from cpe-69-204-164-
>     Sun Microsystems Inc.   SunOS 5.9       Generic May 2002
>     tcsh: using dumb terminal settings.
>
>
>     bob@robin ~ $  test -e /tmp
>     test: argument expected
>
>     bob@robin ~ $ which test
>     /usr/ucb/test
>     bob@robin ~ $  echo $PATH
>     .:/bin:/usr/games:/usr/etc:/usr/ucb:/usr/local/bin:/usr/bin:/usr/X11/bin

The patch seems to be OK wrt your echoing shell. However, Tramp still
tries to apply "test -e", although it's not applicable. This is likely
because of cached connection properties (see "~/.emacs.d/tramp").

Could you, please, remove this file before you start Emacs? Or,
alternatively, before you're going remote, clear the cache by calling
"M-x tramp-cleanup-connection RET /ssh:bob@shell.berkshire.net: RET"?

Btw, next time you send a debug buffer _please_ append it to your
message as attachment with proper encoding. It contains non ASCII
characters, and email is not 8bit clean by design. I can read your
message via the emacs-devel archive, but there is a delay until a
message is offered there, and (because in HTML format) it doesn't keep
line formatting, which makes reading the debug messages hard.

Thanks, and best regards, Michael.

Re: Emacs tramp troubles with old Sun

[Robert J. Chassell]

Success!

    > but the old Sun still fails with
    >
    >     Couldn't find exit status of `test -e /tmp'

    The patch seems to be OK wrt your echoing shell. However, Tramp
    still tries to apply "test -e", although it's not applicable. This
    is likely because of cached connection properties (see
    "~/.emacs.d/tramp").

    Could you, please, remove this file before you start Emacs? Or,
    alternatively, before you're going remote, clear the cache by
    calling "M-x tramp-cleanup-connection RET
    /ssh:bob@shell.berkshire.net: RET"?

That succeeded!  I ran tramp-cleanup-connection.  That was the final
problem.  Now I am able to connect to that machine in a different and
more complex fashion.

Thank you!

-- 
    Robert J. Chassell                          GnuPG Key ID: 004B4AC8
    bob@rattlesnake.com                         bob@gnu.org
    http://www.rattlesnake.com                  http://www.teak.cc

Re: Emacs tramp troubles with old Sun

[Stefan Monnier]

> Could you, please, remove this file before you start Emacs? Or,
> alternatively, before you're going remote, clear the cache by calling
> "M-x tramp-cleanup-connection RET /ssh:bob@shell.berkshire.net: RET"?

Would it be possible for Tramp to detect the "test: argument expected" error
message and force a "cleanup-connection" at that point?
[ and of course, do similar things wherever applicable: the idea is to try
  and detect when the cached properties become invalid, which will
  necessarily happen over time as machines get upgraded/replaced/... ]


-- Stefan

Re: Emacs tramp troubles with old Sun

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Would it be possible for Tramp to detect the "test: argument expected" error
> message and force a "cleanup-connection" at that point?
> [ and of course, do similar things wherever applicable: the idea is to try
>   and detect when the cached properties become invalid, which will
>   necessarily happen over time as machines get upgraded/replaced/... ]

Exactly the same idea I'm just thinking about. But one must ensure
there's no infloop - when "test" does not work, it could be another
reason but invalid cached values. One shall remember the reason for
calling cleanup-connection, and when the same problem appears again,
it's time to give up.

Another idea is to keep a counter for cached properties. After a while
(100 uses, 100 days, whatever) the values shall be recomputed, just in
case. Think about a changed default $PATH, where it isn't always an
obvious error with the old value, when new directories appear.

At least when "uname -s -r" returns a changed value, it is time to
expire the cached properties for that machine.

> -- Stefan

Best regards, Michael.

Re: Emacs tramp troubles with old Sun

[Michael Albinus]

I wrote:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> Would it be possible for Tramp to detect the "test: argument expected" error
>> message and force a "cleanup-connection" at that point?
>> [ and of course, do similar things wherever applicable: the idea is to try
>>   and detect when the cached properties become invalid, which will
>>   necessarily happen over time as machines get upgraded/replaced/... ]
>
> Exactly the same idea I'm just thinking about. But one must ensure
> there's no infloop - when "test" does not work, it could be another
> reason but invalid cached values. One shall remember the reason for
> calling cleanup-connection, and when the same problem appears again,
> it's time to give up.
>
> Another idea is to keep a counter for cached properties. After a while
> (100 uses, 100 days, whatever) the values shall be recomputed, just in
> case. Think about a changed default $PATH, where it isn't always an
> obvious error with the old value, when new directories appear.
>
> At least when "uname -s -r" returns a changed value, it is time to
> expire the cached properties for that machine.

Finally, I've implemented only the "uname -sr" check. The other checks
for connection properties cleanups I have had in mind are too
dangerous. Tramp sends remote commands on the fly (when it is necessary
only). Therefore, command sequences are not "atomic" such a way one
could just flush the cache, and reapply the command.

Another drawback would be that error analysis might be impossible when
all relevant data are thrown away once an error happened.

The original problem we have spoken about was that Tramp did not
recognize shell echoing on the "old Sun". Wrong cache entries were
likely a result of _this_ error. I expect to reduce the number of errors
in Tramp :-) Such situations should not happen so often in the future.

>> -- Stefan

Best regards, Michael.