bug#38079: On Windows tramp hangs after server-start is called

bug#38079: On Windows tramp hangs after server-start is called

[Wanrong Lin]

On Windows, after M-x server-start, tramp no longer works. When I tried 
to connect to a "pscp" remote Linux host, it just hangs at the message 
"Remote prompt found" (something like that).

Today I finally figured out what the problem is, and put in a work 
around as following:

;; On Windows, after "server-start" is called, somehow (system-name) 
will return lower case host name. If the original
;; (system-name) returns upper case host name now 
tramp-restricted-shell-hosts-alist will be out-of-date and cause
;; tramp not to function properly.
(defadvice server-start (after update-restricted-host-list last activate)
   (setf tramp-restricted-shell-hosts-alist
         (when (memq system-type '(windows-nt))
           (list (concat "\\`" (regexp-quote (system-name)) "\\'")))))

But the proper fix should be to make sure (system-name) always returns 
the same string consistently. I found it is this sexp in "server-start" 
function that changes the return value of (system-name):

       (setq server-process
         (apply #'make-network-process ....))

This bug has bothered me for a long time. Even Emacs 26.3 still has it 
(although I am still using 26.2 for daily work). I hope this can lead to 
a fix that saves some frustration for other fellow Emacs users.

Here is my set up:

In GNU Emacs 26.2 (build 1, x86_64-w64-mingw32)
  of 2019-04-13 built on CIRROCUMULUS
Repository revision: fd1b34bfba8f3f6298df47c8e10b61530426f749

Warren

bug#38079: On Windows tramp hangs after server-start is called

[Eli Zaretskii]

> From: Wanrong Lin <wrglin@gmail.com>
> Date: Tue, 5 Nov 2019 19:52:59 -0500
> 
> ;; On Windows, after "server-start" is called, somehow (system-name) 
> will return lower case host name. If the original
> ;; (system-name) returns upper case host name now 
> tramp-restricted-shell-hosts-alist will be out-of-date and cause
> ;; tramp not to function properly.
> (defadvice server-start (after update-restricted-host-list last activate)
>    (setf tramp-restricted-shell-hosts-alist
>          (when (memq system-type '(windows-nt))
>            (list (concat "\\`" (regexp-quote (system-name)) "\\'")))))
> 
> But the proper fix should be to make sure (system-name) always returns 
> the same string consistently. I found it is this sexp in "server-start" 
> function that changes the return value of (system-name):
> 
>        (setq server-process
>          (apply #'make-network-process ....))

Thanks.  However, your proposed solution is very hard to implement
(for boring technical reasons), so my suggestion is for Trump to
compare system names case insensitively, at least on MS-Windows, with
a proper comment to explain why.

bug#38079: On Windows tramp hangs after server-start is called

[martin rudalics]

 > (for boring technical reasons), so my suggestion is for Trump to
                                                             ^

 > compare system names case insensitively, at least on MS-Windows, with

So Richard was right after all when he suggested to rename this ...

martin

bug#38079: On Windows tramp hangs after server-start is called

[Wanrong Lin]

Yes that can be another way to fix. The downside is we don't know this 
system-name upper/lower case change will cause any surprises somewhere 
else. But I don't understand the underlying difficulty to ensure 
system-name never changes. Thanks for fixing it either way.

Warren

On 11/6/2019 11:06 AM, Eli Zaretskii wrote:
>> From: Wanrong Lin <wrglin@gmail.com>
>> Date: Tue, 5 Nov 2019 19:52:59 -0500
>>
>> ;; On Windows, after "server-start" is called, somehow (system-name)
>> will return lower case host name. If the original
>> ;; (system-name) returns upper case host name now
>> tramp-restricted-shell-hosts-alist will be out-of-date and cause
>> ;; tramp not to function properly.
>> (defadvice server-start (after update-restricted-host-list last activate)
>>     (setf tramp-restricted-shell-hosts-alist
>>           (when (memq system-type '(windows-nt))
>>             (list (concat "\\`" (regexp-quote (system-name)) "\\'")))))
>>
>> But the proper fix should be to make sure (system-name) always returns
>> the same string consistently. I found it is this sexp in "server-start"
>> function that changes the return value of (system-name):
>>
>>         (setq server-process
>>           (apply #'make-network-process ....))
> Thanks.  However, your proposed solution is very hard to implement
> (for boring technical reasons), so my suggestion is for Trump to
> compare system names case insensitively, at least on MS-Windows, with
> a proper comment to explain why.

bug#38079: On Windows tramp hangs after server-start is called

[Michael Albinus]

martin rudalics <rudalics@gmx.at> writes:

>> (for boring technical reasons), so my suggestion is for Trump to
>                                                             ^
>
>> compare system names case insensitively, at least on MS-Windows, with
>
> So Richard was right after all when he suggested to rename this ...

... and I've refused, for good reasons.

Will see what I could do.

> martin

Best regards, Michael.

bug#38079: On Windows tramp hangs after server-start is called

[Eli Zaretskii]

> Cc: 38079@debbugs.gnu.org
> From: Wanrong Lin <wrglin@gmail.com>
> Date: Wed, 6 Nov 2019 11:50:59 -0500
> 
> But I don't understand the underlying difficulty to ensure
> system-name never changes.

There are two system calls that return this information on Windows,
and they treat the letter-case differently.  The second system call
needs the winsock layer to be initialized, and we don't want to do
that at startup because it causes various problems (sorry, I no longer
remember the details of that part).  So we only call the network-aware
system call after some network connection is available.

bug#38079: On Windows tramp hangs after server-start is called

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 420 bytes --]

Wanrong Lin <wrglin@gmail.com> writes:

Hi,

> Yes that can be another way to fix. The downside is we don't know this
> system-name upper/lower case change will cause any surprises somewhere
> else. But I don't understand the underlying difficulty to ensure
> system-name never changes. Thanks for fixing it either way.

Does the appended patch fix the problem for you?

> Warren

Best regards, Michael.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1163 bytes --]

diff --git a/lisp/tramp.el b/lisp/tramp.el
index b1ad1922..7b929255 100644
--- a/lisp/tramp.el
+++ b/lisp/tramp.el
@@ -457,15 +457,20 @@ interpreted as a regular expression which always matches."
   :version "24.3"
   :type 'boolean)

+;; For some obscure technical reasons, `system-name' on w32 returns
+;; either lower case or upper case letters.  See
+;; <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38079#20>.
 (defcustom tramp-restricted-shell-hosts-alist
   (when (memq system-type '(windows-nt))
-    (list (concat "\\`" (regexp-quote (system-name)) "\\'")))
+    (list (format "\\`\\(%s\\|%s\\)\\'"
+		  (regexp-quote (downcase (system-name)))
+		  (regexp-quote (upcase (system-name))))))
   "List of hosts, which run a restricted shell.
 This is a list of regular expressions, which denote hosts running
 a registered shell like \"rbash\".  Those hosts can be used as
 proxies only, see `tramp-default-proxies-alist'.  If the local
 host runs a registered shell, it shall be added to this list, too."
-  :version "24.3"
+  :version "27.1"
   :type '(repeat (regexp :tag "Host regexp")))

 (defcustom tramp-local-host-regexp

bug#38079: On Windows tramp hangs after server-start is called

[Wanrong Lin]

Yes it does. Thanks.

Wanrong

On 11/7/2019 10:23 AM, Michael Albinus wrote:
> Wanrong Lin <wrglin@gmail.com> writes:
>
> Hi,
>
>> Yes that can be another way to fix. The downside is we don't know this
>> system-name upper/lower case change will cause any surprises somewhere
>> else. But I don't understand the underlying difficulty to ensure
>> system-name never changes. Thanks for fixing it either way.
> Does the appended patch fix the problem for you?
>
>> Warren
> Best regards, Michael.
>

bug#38079: On Windows tramp hangs after server-start is called

[Wanrong Lin]

Yes it does. Thanks.

Warren

On 11/7/2019 10:23 AM, Michael Albinus wrote:
> Wanrong Lin <wrglin@gmail.com> writes:
>
> Hi,
>
>> Yes that can be another way to fix. The downside is we don't know this
>> system-name upper/lower case change will cause any surprises somewhere
>> else. But I don't understand the underlying difficulty to ensure
>> system-name never changes. Thanks for fixing it either way.
> Does the appended patch fix the problem for you?
>
>> Warren
> Best regards, Michael.
>

bug#38079: On Windows tramp hangs after server-start is called

[Michael Albinus]

Version: 27.1

Wanrong Lin <wrglin@gmail.com> writes:

> Yes it does. Thanks.

Thanks for the feedback. I've committed the patch to the repositories,
closing the bug.

> Wanrong

Best regards, Michael.

bug#38079: On Windows tramp hangs after server-start is called

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> But I don't understand the underlying difficulty to ensure
>> system-name never changes.
>
> There are two system calls that return this information on Windows,
> and they treat the letter-case differently.  The second system call
> needs the winsock layer to be initialized, and we don't want to do
> that at startup because it causes various problems (sorry, I no longer
> remember the details of that part).  So we only call the network-aware
> system call after some network connection is available.

Shouldn't we document this in etc/PROBLEMS? I can't write this, because
I don't know too much about Emacs on MS Windows.

Best regards, Michael.

bug#38079: On Windows tramp hangs after server-start is called

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Wanrong Lin <wrglin@gmail.com>,  38079@debbugs.gnu.org
> Date: Fri, 08 Nov 2019 08:51:42 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> But I don't understand the underlying difficulty to ensure
> >> system-name never changes.
> >
> > There are two system calls that return this information on Windows,
> > and they treat the letter-case differently.  The second system call
> > needs the winsock layer to be initialized, and we don't want to do
> > that at startup because it causes various problems (sorry, I no longer
> > remember the details of that part).  So we only call the network-aware
> > system call after some network connection is available.
> 
> Shouldn't we document this in etc/PROBLEMS?

What would you suggest to document?  This is an internal detail of the
Emacs implementation, and users have no control on the results.  Maybe
I'm missing something, but what useful workarounds can we suggest?

bug#38079: On Windows tramp hangs after server-start is called

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> > There are two system calls that return this information on Windows,
>> > and they treat the letter-case differently.  The second system call
>> > needs the winsock layer to be initialized, and we don't want to do
>> > that at startup because it causes various problems (sorry, I no longer
>> > remember the details of that part).  So we only call the network-aware
>> > system call after some network connection is available.
>>
>> Shouldn't we document this in etc/PROBLEMS?
>
> What would you suggest to document?  This is an internal detail of the
> Emacs implementation, and users have no control on the results.  Maybe
> I'm missing something, but what useful workarounds can we suggest?

I don't speak about users but developers. At least we shall warn them
that `system-name' can return different values. They might apply counter
measures, like using only (downcase (system-name)) or a alike on w32.

Well, since it is about developers, the Lisp manual might be the better
place to document, in the "Operating System Environment" section.

Best regards, Michael.

bug#38079: On Windows tramp hangs after server-start is called

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: wrglin@gmail.com,  38079@debbugs.gnu.org
> Date: Fri, 08 Nov 2019 11:50:45 +0100
> 
> >> Shouldn't we document this in etc/PROBLEMS?
> >
> > What would you suggest to document?  This is an internal detail of the
> > Emacs implementation, and users have no control on the results.  Maybe
> > I'm missing something, but what useful workarounds can we suggest?
> 
> I don't speak about users but developers.

Then PROBLEMS is not the appropriate place.

> At least we shall warn them that `system-name' can return different
> values. They might apply counter measures, like using only (downcase
> (system-name)) or a alike on w32.
> 
> Well, since it is about developers, the Lisp manual might be the better
> place to document, in the "Operating System Environment" section.

Right.  Or maybe in the doc string.