bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

As mentioned in a recent discussion:

1. M-x eshell
2. type 'cd /'
3. M-: (require 'tramp) ;; New step!
4. M-x pcomplete

I'm getting the error 'tramp-file-name-handler: Host name must not match
method "adb"'.

In GNU Emacs 25.0.92.2 (x86_64-unknown-linux-gnu, GTK+ Version 3.16.7)
 of 2016-03-11 built on axl
Repository revision: facb5e20ce186e47506860bde982e35020fedce5
Windowing system distributor 'The X.Org Foundation', version 11.0.11702000
System Description:	Ubuntu 15.10

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> As mentioned in a recent discussion:
> 1. M-x eshell
> 2. type 'cd /'
> 3. M-: (require 'tramp) ;; New step!
> 4. M-x pcomplete

> I'm getting the error 'tramp-file-name-handler: Host name must not match
> method "adb"'.

I can reproduce it this time (also in 24.5), yes, thank you.
But I can't get a backtrace.  Even debug-on-signal doesn't give me
one :-(

Hard to know where to fix the problem, because it surely needs to be
somewhere along that backtrace.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> As mentioned in a recent discussion:
>> 1. M-x eshell
>> 2. type 'cd /'
>> 3. M-: (require 'tramp) ;; New step!
>> 4. M-x pcomplete
>
>> I'm getting the error 'tramp-file-name-handler: Host name must not match
>> method "adb"'.
>
> I can reproduce it this time (also in 24.5), yes, thank you.
> But I can't get a backtrace.  Even debug-on-signal doesn't give me
> one :-(

Set a breakpoint in `tramp-check-proper-method-and-host'. Then you see

  (if (edebug-after (edebug-before 32) 57 (and (edebug-after (edebug-before 33) 35 (equal (edebug-after 0 34 tramp-syntax) (quote ftp))) (ede$
  (let ((method (edebug-after (edebug-before 1) 3 (tramp-file-name-method (edebug-after 0 2 vec)))) (user (edebug-after (edebug-before 4) 6 ($
  tramp-check-proper-method-and-host([#("scp" 0 3 (tramp-default t)) nil "davs" "" nil])
  tramp-maybe-open-connection([#("scp" 0 3 (tramp-default t)) nil "davs" "" nil])
  tramp-send-command([#("scp" 0 3 (tramp-default t)) nil "davs" "" nil] "cd ~ && pwd")
  tramp-sh-handle-expand-file-name("/davs:" nil)
  apply(tramp-sh-handle-expand-file-name ("/davs:" nil))
  tramp-sh-file-name-handler(expand-file-name "/davs:" nil)
  apply(tramp-sh-file-name-handler expand-file-name ("/davs:" nil))
  tramp-file-name-handler(expand-file-name "/davs:" nil)
  expand-file-name("/davs:" nil)
  apply(expand-file-name ("/davs:" nil))
  tramp-completion-run-real-handler(expand-file-name ("/davs:" nil))
  tramp-completion-file-name-handler(expand-file-name "/davs:" nil)
  file-name-all-completions("" "/davs:")
  completion-file-name-table("/davs:" #[257 "\302\203\n^@\302^A!\205^\^@\301\203^T^@\301^A!\205^\^@\300?\206^\^@\300^A!\207" [file-executable$
  comint-completion-file-name-table("/davs:" #[257 "\302\203\n^@\302^A!\205^\^@\301\203^T^@\301^A!\205^\^@\300?\206^\^@\300^A!\207" [file-exe$
  complete-with-action(t comint-completion-file-name-table "/davs:" #[257 "\302\203\n^@\302^A!\205^\^@\301\203^T^@\301^A!\205^\^@\300?\206^\^$
  completion-table-with-predicate(comint-completion-file-name-table #[257 "\302\203\n^@\302^A!\205^\^@\301\203^T^@\301^A!\205^\^@\300?\206^\^$
  #[771 "\211\305=\203^W^@      \203^W^@\305\306\307B\310^E^E^E#ABB\207\311\n^[^\\312\313\300\314^F^F^F^F^F^F&^F*\207" [#[257 "\302\203\n^@\3$
  complete-with-action(t #[771 "\211\305=\203^W^@       \203^W^@\305\306\307B\310^E^E^E#ABB\207\311\n^[^\\312\313\300\314^F^F^F^F^F^F&^F*\207$
  #[771 "\302^CG\303    ^F^F\"\203)^@^D\304\305\224O^BB\262^B\306\307\310^F^G\"!^BB\262^B^D\305\225\302O\262^E\202^C^@^A\203:^@^B\311=\204O^@$
  all-completions("/davs:" #[771 "\302^CG\303   ^F^F\"\203)^@^D\304\305\224O^BB\262^B\306\307\310^F^G\"!^BB\262^B^D\305\225\302O\262^E\202^C^$
  pcomplete-stub("/davs:" #[771 "\302^CG\303    ^F^F\"\203)^@^D\304\305\224O^BB\262^B\306\307\310^F^G\"!^BB\262^B^D\305\225\302O\262^E\202^C^$
  pcomplete-do-complete("/davs:" #[771 "\302^CG\303     ^F^F\"\203)^@^D\304\305\224O^BB\262^B\306\307\310^F^G\"!^BB\262^B^D\305\225\302O\262^$
  pcomplete(1)
  funcall-interactively(pcomplete 1)
  call-interactively(pcomplete record nil)
  command-execute(pcomplete record)
  execute-extended-command(nil "pcomplete" nil)
  funcall-interactively(execute-extended-command nil "pcomplete" nil)
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

> Hard to know where to fix the problem, because it surely needs to be
> somewhere along that backtrace.
>
>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>   pcomplete-do-complete("/davs:" #[771 "\302^CG\303
>   pcomplete(1)

Hmm... how did we get from "/" to "/davs:"?


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>   pcomplete-do-complete("/davs:" #[771 "\302^CG\303
>>   pcomplete(1)
>
> Hmm... how did we get from "/" to "/davs:"?

Forgot to say: Prior to calling pcomplete, I have typed /davs: in the
eshell buffer.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/15/2016 10:43 AM, Michael Albinus wrote:

>> Hmm... how did we get from "/" to "/davs:"?
>
> Forgot to say: Prior to calling pcomplete, I have typed /davs: in the
> eshell buffer.

I didn't. But I'm getting the same error anyway.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>>> pcomplete-do-complete("/davs:" #[771 "\302^CG\303
>>> pcomplete(1)
>> Hmm... how did we get from "/" to "/davs:"?
> Forgot to say: Prior to calling pcomplete, I have typed /davs: in the
> eshell buffer.

Hmm... looking back at it I'm kind of lost.
I thought this was about adding `non-essential' somewhere, but in the
recipe, I don't see much opportunity for that.  So, I think the question
is really for Micheal:

  Why does "cd / M-x pcomplete RET" signal an error like

    tramp-file-name-handler: Host name must not match method "adb"

  ?

That seems like a bug right there, which doesn't have much to do with
non-essential.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>>> I'm getting the error 'tramp-file-name-handler: Host name must not match
>>> method "adb"'.
[...]
>   tramp-check-proper-method-and-host([#("scp" 0 3 (tramp-default t)) nil "davs" "" nil])
>   tramp-maybe-open-connection([#("scp" 0 3 (tramp-default t)) nil "davs" "" nil])
>   tramp-send-command([#("scp" 0 3 (tramp-default t)) nil "davs" "" nil] "cd ~ && pwd")
>   tramp-sh-handle-expand-file-name("/davs:" nil)
>   apply(tramp-sh-handle-expand-file-name ("/davs:" nil))
>   tramp-sh-file-name-handler(expand-file-name "/davs:" nil)
>   apply(tramp-sh-file-name-handler expand-file-name ("/davs:" nil))
>   tramp-file-name-handler(expand-file-name "/davs:" nil)
>   expand-file-name("/davs:" nil)
>   apply(expand-file-name ("/davs:" nil))
>   tramp-completion-run-real-handler(expand-file-name ("/davs:" nil))
>   tramp-completion-file-name-handler(expand-file-name "/davs:" nil)
>   file-name-all-completions("" "/davs:")
>   completion-file-name-table("/davs:" #[257

completion-file-name-table calls (file-name-all-completions "" "/davs:")
because (file-name-directory "/davs:") returns "/davs:" which seems
wrong if "/davs:" is not a valid directory name.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>>> pcomplete-do-complete("/davs:" #[771 "\302^CG\303
>>>> pcomplete(1)
>>> Hmm... how did we get from "/" to "/davs:"?
>> Forgot to say: Prior to calling pcomplete, I have typed /davs: in the
>> eshell buffer.
>
> Hmm... looking back at it I'm kind of lost.
> I thought this was about adding `non-essential' somewhere, but in the
> recipe, I don't see much opportunity for that.  So, I think the question
> is really for Micheal:
>
>   Why does "cd / M-x pcomplete RET" signal an error like
>
>     tramp-file-name-handler: Host name must not match method "adb"
>
>   ?
>
> That seems like a bug right there, which doesn't have much to do with
> non-essential.

I couldn't provoke the bug with the recipe you ought to use:

emacs -Q
M-x eshell
cd /
M-x pcomplete

So I cannot answer that question. Or do you use another recipe?

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/17/2016 09:43 PM, Michael Albinus wrote:

> I couldn't provoke the bug with the recipe you ought to use:
>
> emacs -Q
> M-x eshell
> cd /
> M-x pcomplete
>
> So I cannot answer that question. Or do you use another recipe?

Like I've described in this bug report, you also need to load Tramp.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>>> I'm getting the error 'tramp-file-name-handler: Host name must not match
>>>> method "adb"'.
> [...]
>>   tramp-check-proper-method-and-host([#("scp" 0 3 (tramp-default t))
>> nil "davs" "" nil])
>>   tramp-maybe-open-connection([#("scp" 0 3 (tramp-default t)) nil
>> "davs" "" nil])
>>   tramp-send-command([#("scp" 0 3 (tramp-default t)) nil "davs" ""
>> nil] "cd ~ && pwd")
>>   tramp-sh-handle-expand-file-name("/davs:" nil)
>>   apply(tramp-sh-handle-expand-file-name ("/davs:" nil))
>>   tramp-sh-file-name-handler(expand-file-name "/davs:" nil)
>>   apply(tramp-sh-file-name-handler expand-file-name ("/davs:" nil))
>>   tramp-file-name-handler(expand-file-name "/davs:" nil)
>>   expand-file-name("/davs:" nil)
>>   apply(expand-file-name ("/davs:" nil))
>>   tramp-completion-run-real-handler(expand-file-name ("/davs:" nil))
>>   tramp-completion-file-name-handler(expand-file-name "/davs:" nil)
>>   file-name-all-completions("" "/davs:")
>>   completion-file-name-table("/davs:" #[257
>
> completion-file-name-table calls (file-name-all-completions "" "/davs:")
> because (file-name-directory "/davs:") returns "/davs:" which seems
> wrong if "/davs:" is not a valid directory name.

What do you expect (file-name-directory "/davs:") to return else?
`file-name-directory' is not expected to check, whether a given argument
is a valid file name. The docstring says "Given a Unix syntax file name,
returns a string ending in slash.".

And yes, in this case no trailing slash is added. Hmm, maybe it shall
return "/davs:./". Would this be better?

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/17/2016 09:43 PM, Michael Albinus wrote:
>
>> I couldn't provoke the bug with the recipe you ought to use:
>>
>> emacs -Q
>> M-x eshell
>> cd /
>> M-x pcomplete
>>
>> So I cannot answer that question. Or do you use another recipe?
>
> Like I've described in this bug report, you also need to load Tramp.

emacs -Q
M-x load-library tramp
M-x eshell
cd /
M-x pcomplete

Still no error. Tests performed in the emacs-25 branch.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/17/2016 09:54 PM, Michael Albinus wrote:

> emacs -Q
> M-x load-library tramp
> M-x eshell
> cd /
> M-x pcomplete
>
> Still no error. Tests performed in the emacs-25 branch.

I do get an error. Or rather, a failure message, like described 
previously. On emacs-25, with -Q, just 'make bootstrap'-ed the latest 
commit to triple-check, and the problem is consistent.

Do you get any completions in the end?

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/17/2016 09:54 PM, Michael Albinus wrote:
>
>> emacs -Q
>> M-x load-library tramp
>> M-x eshell
>> cd /
>> M-x pcomplete
>>
>> Still no error. Tests performed in the emacs-25 branch.
>
> I do get an error. Or rather, a failure message, like described
> previously. On emacs-25, with -Q, just 'make bootstrap'-ed the latest
> commit to triple-check, and the problem is consistent.
>
> Do you get any completions in the end?

Yes. But this aren't the files under "/", but rather all programs I
could start on my Ubuntu machine.

If I do *not* enter RET after "cd /", and call "M-x pcomplete" then, I
get the same error message. OK, that's clear.

Now the backtrace in this scenario:

--8<---------------cut here---------------start------------->8---
  (member (edebug-after 0 54 host) (edebug-after 0 55 methods))
  (and (edebug-after (edebug-before 33) 35 (equal (edebug-after 0 34 tramp-syntax) (quote ftp))) (edebug-after 0 36 host) (edebug-after (edebug-before 37) 44 (or (edebug-after (edebug-before 38) 40 (null (edebug-after 0 39 method))) (edebug-after (edebug-before 41) 43 (get-text-property 0 (quote tramp-default) (edebug-after 0 42 method))))) (edebug-after (edebug-before 45) 52 (or (edebug-after (edebug-before 46) 48 (null (edebug-after 0 47 user))) (edebug-after (edebug-before 49) 51 (get-text-property 0 (quote tramp-default) (edebug-after 0 50 user))))) (edebug-after (edebug-before 53) 56 (member (edebug-after 0 54 host) (edebug-after 0 55 methods))))
  (if (edebug-after (edebug-before 32) 57 (and (edebug-after (edebug-before 33) 35 (equal (edebug-after 0 34 tramp-syntax) (quote ftp))) (edebug-after 0 36 host) (edebug-after (edebug-before 37) 44 (or (edebug-after (edebug-before 38) 40 (null (edebug-after 0 39 method))) (edebug-after (edebug-before 41) 43 (get-text-property 0 (quote tramp-default) (edebug-after 0 42 method))))) (edebug-after (edebug-before 45) 52 (or (edebug-after (edebug-before 46) 48 (null (edebug-after 0 47 user))) (edebug-after (edebug-before 49) 51 (get-text-property 0 (quote tramp-default) (edebug-after 0 50 user))))) (edebug-after (edebug-before 53) 56 (member (edebug-after 0 54 host) (edebug-after 0 55 methods))))) (progn (edebug-after (edebug-before 58) 60 (tramp-cleanup-connection (edebug-after 0 59 vec))) (edebug-after (edebug-before 61) 64 (tramp-user-error (edebug-after 0 62 vec) "Host name must not match method \"%s\"" (edebug-after 0 63 host)))))
  (let ((method (edebug-after (edebug-before 1) 3 (tramp-file-name-method (edebug-after 0 2 vec)))) (user (edebug-after (edebug-before 4) 6 (tramp-file-name-user (edebug-after 0 5 vec)))) (host (edebug-after (edebug-before 7) 9 (tramp-file-name-host (edebug-after 0 8 vec)))) (methods (edebug-after (edebug-before 10) 12 (mapcar (quote car) (edebug-after 0 11 tramp-methods))))) (edebug-after (edebug-before 13) 30 (if (edebug-after (edebug-before 14) 22 (and (edebug-after 0 15 method) (edebug-after (edebug-before 16) 21 (not (edebug-after ... 20 ...))))) (progn (edebug-after (edebug-before 23) 25 (tramp-cleanup-connection (edebug-after 0 24 vec))) (edebug-after (edebug-before 26) 29 (tramp-user-error (edebug-after 0 27 vec) "Unknown method \"%s\"" (edebug-after 0 28 method)))))) (edebug-after (edebug-before 31) 65 (if (edebug-after (edebug-before 32) 57 (and (edebug-after (edebug-before 33) 35 (equal (edebug-after 0 34 tramp-syntax) (quote ftp))) (edebug-after 0 36 host) (edebug-after (edebug-before 37) 44 (or (edebug-after ... 40 ...) (edebug-after ... 43 ...))) (edebug-after (edebug-before 45) 52 (or (edebug-after ... 48 ...) (edebug-after ... 51 ...))) (edebug-after (edebug-before 53) 56 (member (edebug-after 0 54 host) (edebug-after 0 55 methods))))) (progn (edebug-after (edebug-before 58) 60 (tramp-cleanup-connection (edebug-after 0 59 vec))) (edebug-after (edebug-before 61) 64 (tramp-user-error (edebug-after 0 62 vec) "Host name must not match method \"%s\"" (edebug-after 0 63 host)))))))
  tramp-check-proper-method-and-host([#("scp" 0 3 (tramp-default t)) nil "adb" "" nil])
  tramp-maybe-open-connection([#("scp" 0 3 (tramp-default t)) nil "adb" "" nil])
  tramp-send-command([#("scp" 0 3 (tramp-default t)) nil "adb" "" nil] "cd ~ && pwd")
  tramp-sh-handle-expand-file-name("/adb:" "/")
  apply(tramp-sh-handle-expand-file-name ("/adb:" "/"))
  tramp-sh-file-name-handler(expand-file-name "/adb:" "/")
  apply(tramp-sh-file-name-handler expand-file-name ("/adb:" "/"))
  tramp-file-name-handler(expand-file-name "/adb:" "/")
  expand-file-name("adb:" "/")
  apply(expand-file-name ("adb:" "/"))
  tramp-completion-run-real-handler(expand-file-name ("adb:" "/"))
  tramp-completion-file-name-handler(expand-file-name "adb:" "/")
  file-directory-p("adb:")
  #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"]("adb:")
  completion-file-name-table("/" #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] t)
  comint-completion-file-name-table("/" #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] t)
  complete-with-action(t comint-completion-file-name-table "/" #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"])
  completion-table-with-predicate(comint-completion-file-name-table #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] strict "/" nil t)
  #[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"]("/" nil t)
  complete-with-action(t #[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] "/" nil)
  #[771 "\302G\303	\x06\"\203)\x04\304\305\224OB\262\306\307\310\a\"!B\262\x04\305\225\302O\262\202\0\203:\x02\311=\204O\x02\242\312=\204O\313\314\x06B\237\315#\316\300\x06\b$\262\202\307\317\320\321\322\"\"\313\314\aB\237\315#\323\300\b\b\243\206k\315$\211@Y\203\223\x05\311=\203\204\316\x06\300\x06\n$\202\305\312@\x04GZ\\ABB\202\305\x05\311=\203\272\316\x06\300\x06\n$\x02@Z\321\324\325\326\327\330\x06!\331\"\332\333%\"\266\202\202\305\312\x05GZ\\ABB\266\203\207" [#[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] pcomplete--env-regexp nil string-match 0 1 getenv match-string 2 t boundaries mapconcat identity "" complete-with-action apply + mapcar length completion-boundaries make-byte-code 257 "\211\300\301O\207" vconcat vector [nil] 4 "\n\n(fn S)"] 17 "\n\n(fn STRING PRED ACTION)"]("/" nil t)
  all-completions("/" #[771 "\302G\303	\x06\"\203)\x04\304\305\224OB\262\306\307\310\a\"!B\262\x04\305\225\302O\262\202\0\203:\x02\311=\204O\x02\242\312=\204O\313\314\x06B\237\315#\316\300\x06\b$\262\202\307\317\320\321\322\"\"\313\314\aB\237\315#\323\300\b\b\243\206k\315$\211@Y\203\223\x05\311=\203\204\316\x06\300\x06\n$\202\305\312@\x04GZ\\ABB\202\305\x05\311=\203\272\316\x06\300\x06\n$\x02@Z\321\324\325\326\327\330\x06!\331\"\332\333%\"\266\202\202\305\312\x05GZ\\ABB\266\203\207" [#[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] pcomplete--env-regexp nil string-match 0 1 getenv match-string 2 t boundaries mapconcat identity "" complete-with-action apply + mapcar length completion-boundaries make-byte-code 257 "\211\300\301O\207" vconcat vector [nil] 4 "\n\n(fn S)"] 17 "\n\n(fn STRING PRED ACTION)"])
  pcomplete-stub("/" #[771 "\302G\303	\x06\"\203)\x04\304\305\224OB\262\306\307\310\a\"!B\262\x04\305\225\302O\262\202\0\203:\x02\311=\204O\x02\242\312=\204O\313\314\x06B\237\315#\316\300\x06\b$\262\202\307\317\320\321\322\"\"\313\314\aB\237\315#\323\300\b\b\243\206k\315$\211@Y\203\223\x05\311=\203\204\316\x06\300\x06\n$\202\305\312@\x04GZ\\ABB\202\305\x05\311=\203\272\316\x06\300\x06\n$\x02@Z\321\324\325\326\327\330\x06!\331\"\332\333%\"\266\202\202\305\312\x05GZ\\ABB\266\203\207" [#[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] pcomplete--env-regexp nil string-match 0 1 getenv match-string 2 t boundaries mapconcat identity "" complete-with-action apply + mapcar length completion-boundaries make-byte-code 257 "\211\300\301O\207" vconcat vector [nil] 4 "\n\n(fn S)"] 17 "\n\n(fn STRING PRED ACTION)"])
  pcomplete-do-complete("/" #[771 "\302G\303	\x06\"\203)\x04\304\305\224OB\262\306\307\310\a\"!B\262\x04\305\225\302O\262\202\0\203:\x02\311=\204O\x02\242\312=\204O\313\314\x06B\237\315#\316\300\x06\b$\262\202\307\317\320\321\322\"\"\313\314\aB\237\315#\323\300\b\b\243\206k\315$\211@Y\203\223\x05\311=\203\204\316\x06\300\x06\n$\202\305\312@\x04GZ\\ABB\202\305\x05\311=\203\272\316\x06\300\x06\n$\x02@Z\321\324\325\326\327\330\x06!\331\"\332\333%\"\266\202\202\305\312\x05GZ\\ABB\266\203\207" [#[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] pcomplete--env-regexp nil string-match 0 1 getenv match-string 2 t boundaries mapconcat identity "" complete-with-action apply + mapcar length completion-boundaries make-byte-code 257 "\211\300\301O\207" vconcat vector [nil] 4 "\n\n(fn S)"] 17 "\n\n(fn STRING PRED ACTION)"])
  pcomplete(1)
  funcall-interactively(pcomplete 1)
  call-interactively(pcomplete record nil)
  command-execute(pcomplete record)
  execute-extended-command(nil "pcomplete" "pcomplete")
  funcall-interactively(execute-extended-command nil "pcomplete" "pcomplete")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> What do you expect (file-name-directory "/davs:") to return else?

IIUC "/davs:hello.c" is parsed as a reference to a file on the host
"davs" and we nowadays signal an error in that case telling the user to
use an explicit "/ssh:davs:hello.c" instead.

At least that's how I interpret the

   tramp-file-name-handler: Host name must not match method "davs"

(replace "davs" with "adb" and friends depending on your particular
case) we get in the recipe.

So, assuming this interpretation is correct, I think "/davs:" should not
be considered a valid "directory" name.

So I see two options:

- (file-name-directory "/davs:") returns "/"
- (file-name-directory "/davs:") returns "/davs:" and Tramp accepts
  (file-name-all-completions "" "/davs:") without signaling an error.

> And yes, in this case no trailing slash is added. Hmm, maybe it shall
> return "/davs:./". Would this be better?

I don't know what are the tradeoffs in this regard, so I have no
preference in this regard.  But what (file-name-directory "/davs:")
returns should be a directory name that file-name-all-completions can
use and to the extent that "/davs:" is the prefix of valid file names,
file-name-all-completions should not signal an error (it's acceptable if
it doesn't actually lists all those valid file names, OTOH).


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>   tramp-send-command([#("scp" 0 3 (tramp-default t)) nil "adb" "" nil] "cd ~ && pwd")
>   tramp-sh-handle-expand-file-name("/adb:" "/")
>   apply(tramp-sh-handle-expand-file-name ("/adb:" "/"))
>   tramp-sh-file-name-handler(expand-file-name "/adb:" "/")
>   apply(tramp-sh-file-name-handler expand-file-name ("/adb:" "/"))
>   tramp-file-name-handler(expand-file-name "/adb:" "/")
>   expand-file-name("adb:" "/")
>   apply(expand-file-name ("adb:" "/"))
>   tramp-completion-run-real-handler(expand-file-name ("adb:" "/"))
>   tramp-completion-file-name-handler(expand-file-name "adb:" "/")
>   file-directory-p("adb:")

And I think here we have another related problem: (file-directory-p
"/adb:") should not try to connect to the "adb" host, because this name
is ambiguous.

A big part of the problem here is our support for the
"/<host>:" syntax.  I wonder if we could demote this syntax.  One option
is of course to drop it altogether (I'm biased: I never use it).
But maybe another is to only support it in substitute-in-file-name,
where it'd get expanded to its more verbose equivalent.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

Hi Stefan,

> A big part of the problem here is our support for the
> "/<host>:" syntax.  I wonder if we could demote this syntax.  One option
> is of course to drop it altogether (I'm biased: I never use it).
> But maybe another is to only support it in substitute-in-file-name,
> where it'd get expanded to its more verbose equivalent.

I fully agree with you, I haven't been ever in favor of the
"/mehod:host:" syntax, and its abbreviation "/host:". Unfortunately,
this was decided before I've entered Tramp maintenance (2002 or so).

However, before we start with such radical changes, why do you oppose to
use non-essential in pcomplete? According to my tests, this solves the problem.

I know that you've planned to use non-essential for several use
cases. But according to the status quo in the sources, until now it is
used only for indication of file name completion being in progress.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> However, before we start with such radical changes, why do you oppose
> to use non-essential in pcomplete? According to my tests, this solves
> the problem.

non-essential means that what we're doing is a skippable/secondary (aka
non-essential) side-effect of the main operation.  When the user hits
TAB, performing completion is not a skippable side-effect of some other
main operation.  It *is* the main operation.

When the user types "cd /ssh:myhost:src/ema TAB" she *wants* Tramp to
prompt her for a password to then give her the valid completions.

It'd be correct for company-mode to bind non-essential around the code
which computes the completion data in response to the user's normal
editing of the buffer (in order to decide whether the
auto-popup a completion menu), because the completion is not the main
operation.

Similarly it's correct for icomplete to bind non-essential while
computing the completion list, since it's done "in the background" in
response to other editing commands and shouldn't get in the way of those
editing commands.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> However, before we start with such radical changes, why do you oppose
>> to use non-essential in pcomplete? According to my tests, this solves
>> the problem.
>
> non-essential means that what we're doing is a skippable/secondary (aka
> non-essential) side-effect of the main operation.  When the user hits
> TAB, performing completion is not a skippable side-effect of some other
> main operation.  It *is* the main operation.
>
> When the user types "cd /ssh:myhost:src/ema TAB" she *wants* Tramp to
> prompt her for a password to then give her the valid completions.

non-essential is essential for Tramp only until the completion of method
and host is finished. As soon as the filename contains two colons, or a
slash, it completes. No problem with your example, therefore. You'll get 
"/ssh:myhost:src/emacs/" after hitting TAB, even if non-essential is
bound to t.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> non-essential is essential for Tramp only until the completion of method
> and host is finished. As soon as the filename contains two colons, or a
> slash, it completes. No problem with your example, therefore. You'll get 
> "/ssh:myhost:src/emacs/" after hitting TAB, even if non-essential is
> bound to t.

If the connection was not yet established, I think it'd be risky for
Tramp to provide the completion when non-essential is non-nil, because
establishing a connection risks being too intrusive for a non-essential
operation (it's probably OK to reuse an existing connection, OTOH).


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/18/2016 07:53 PM, Stefan Monnier wrote:

> It'd be correct for company-mode to bind non-essential around the code
> which computes the completion data in response to the user's normal
> editing of the buffer (in order to decide whether the
> auto-popup a completion menu), because the completion is not the main
> operation.

Makes sense. Applied in the upstream.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> non-essential is essential for Tramp only until the completion of method
>> and host is finished. As soon as the filename contains two colons, or a
>> slash, it completes. No problem with your example, therefore. You'll get 
>> "/ssh:myhost:src/emacs/" after hitting TAB, even if non-essential is
>> bound to t.
>
> If the connection was not yet established, I think it'd be risky for
> Tramp to provide the completion when non-essential is non-nil, because
> establishing a connection risks being too intrusive for a non-essential
> operation (it's probably OK to reuse an existing connection, OTOH).

When we have "/ssh:myhost:src/em" to complete, and the user indicates
she wants completion (pressing TAB or so), she expresses her wish to
open a connection to myhost. No reason to deny this.

non-essential comes in play only when hostname completion hasn't been
finished.

This is how Tramp behaves for years. I haven't heard complaints about
(except the cases non-essential isn't let-bound, like the one we're
discussing here).

Again, I still don't understand why it hurts to bind non-essential in
pcomplete.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> non-essential comes in play only when hostname completion hasn't been
> finished.

Could you expand on this?

Do you mean when completing things like "/ssh:toto" ?
In that case I agree that Tramp shouldn't open a connection, but I think
this should be regardless of non-essential.

But do you also mean that completing "/ssh:host:tmp/ema" will open a new
connection regardless of non-essential?  If so, as mentioned I think
that would be a mistake.

> This is how Tramp behaves for years. I haven't heard complaints about
> (except the cases non-essential isn't let-bound, like the one we're
> discussing here).

Sounds like we agree, then: this should not be affected by non-essential.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> non-essential comes in play only when hostname completion hasn't been
>> finished.
>
> Could you expand on this?
>
> Do you mean when completing things like "/ssh:toto" ?
> In that case I agree that Tramp shouldn't open a connection, but I think
> this should be regardless of non-essential.

I'm not speaking about opening a connection. This is controlled
independent of non-essential settings.

I'm speaking about this error, that Tramp reports "/adb:" not being a
valid hostname. This does not happen if non-essential is bound to
non-nil, indicating to Tramp that the given filename is not complete yet.

> But do you also mean that completing "/ssh:host:tmp/ema" will open a new
> connection regardless of non-essential?  If so, as mentioned I think
> that would be a mistake.

I don't believe it is a mistake. The presence of a slash indicates to
Tramp, that the (Emacs) user has provided everything what's needed to
say about method, (remote) user, and hostname to Tramp. So it is safe to
open the connection. Otherwise, you would never be able to complete
local pathnames for remote hosts, if the connection hasn't been
established yet. Inconvenient.

It is my daily scenario to type

C-x C-f / / / s s h : h o s <TAB> / u s <TAB>

The leading slashes are needed because I work usually on other remote
machines in parallel. It's my typing style to start with three
consecutive slashes :-)

The first <TAB> completes to "/ssh:host:" The second <TAB> completes to
"/ssh:host:/usr". And so on.

I also perform very often "ediff" over machine boundaries. Usually, the
first filename is on the machine I'm working on, and the second filename
on another machine on a similar location. Once I have started ediff, I
want to complete the (slightliy) different pathname of the second file
while being asked from ediff for that filename. If I would need to open
a connection to the second host in advance, this workflow would be
impossible.

>> This is how Tramp behaves for years. I haven't heard complaints about
>> (except the cases non-essential isn't let-bound, like the one we're
>> discussing here).
>
> Sounds like we agree, then: this should not be affected by non-essential.

Yes, here we agree. Opening a new connection is not related to non-essential.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

Hi Michael,

It seems we have a very fundamental misunderstanding about what
"essential" means in this context.

Can you explain *your* understanding of `non-essential', then?
What does this var mean, for you?
And how do you relate that to its documented meaning in its docstring?


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Hi Michael,

Hi Stefan,

> It seems we have a very fundamental misunderstanding about what
> "essential" means in this context.

Yes. It is not the first time we discuss this, see for example
<http://thread.gmane.org/gmane.emacs.diffs/121948> or
<http://thread.gmane.org/gmane.emacs.diffs/114769>.

> Can you explain *your* understanding of `non-essential', then?
> What does this var mean, for you?

For Tramp, it is an indication that file name completion is in
progress. 

> And how do you relate that to its documented meaning in its docstring?

"... it can be used to prevent Tramp from prompting the
user for a password when we are simply scanning a set of files in the
background or displaying possible completions before the user even asked
for it."

If the user has typed "/ssh:host:tmp/ema", and she requests for
completion by typing TAB, the existence of the slash in the local file
name part is an indication that "the user even asked for it" (completion
on "host").

Without the slash in the local file name, Tramp does not perform
completion on "host" when there is no connection yet. In a previous
message I was wrong about this, saying that completion happens already
when there are two colons.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>> Can you explain *your* understanding of `non-essential', then?
>> What does this var mean, for you?
> For Tramp, it is an indication that file name completion is in
> progress.

Huh?  Why would it need such a variable for that, instead of getting
this info from the fact that you're replying to
`file-name-all-completions' (or `file-name-completion') rather than to
some other request?

And why would it be called `non-essential' instead of `in-completion'?

>> And how do you relate that to its documented meaning in its docstring?
> "... it can be used to prevent Tramp from prompting the
> user for a password

That's an example.

> when we are simply scanning a set of files in the background or
> displaying possible completions before the user even asked for it."

That's the particular justification for not prompting the user.

> If the user has typed "/ssh:host:tmp/ema", and she requests for
> completion by typing TAB, the existence of the slash in the local file
> name part is an indication that "the user even asked for it" (completion
> on "host").

The fact that she hit TAB is enough indication.  But in icomplete and
company, we request completion data even though the user hasn't hit
anything like TAB.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Drew Adams]

> "... it can be used to prevent Tramp from prompting the
> user for a password when we are simply scanning a set of files in the
> background or displaying possible completions before the user even asked
> for it."

Those are two examples.  More generally, it can be used to prevent
Tramp from prompting _for any reason_ and _in any context_ where a
file name is read (any context where Tramp is invoked - e.g. by a
file handler).

> If the user has typed "/ssh:host:tmp/ema", and she requests for
> completion by typing TAB, the existence of the slash in the local file
> name part is an indication that "the user even asked for it" (completion
> on "host").
> 
> Without the slash in the local file name, Tramp does not perform
> completion on "host" when there is no connection yet. In a previous
> message I was wrong about this, saying that completion happens already
> when there are two colons.

Those are explanations of why someone using vanilla Emacs might
want to inhibit Tramp from prompting.  They might be sufficient
reasons for Tramp not to prompt, in that vanilla context.

But they are not _necessary_ conditions to inhibit prompting.

Phrases such as "by typing TAB" and "slash in the local file name
part" are inappropriate considerations here.  They do not belong
in a Tramp consideration of what it means for `non-essential' to
be non-nil.

Tramp should simply honor `non-essential' without question.  It
is not Tramp's business what the reasons might be for inhibiting
its prompting.  It should not make any assumptions about the kind
of completion being used or the mechanism of completion - or even
whether any completion is currently being done.  It should not
care why it is being inhibited from prompting - no second-guessing.

It should not even look at the input string if `non-essential'
is non-nil.  If it is non-nil then Tramp should not consider
perhaps prompting if the input string is this or that or contains
this or that.  If non-nil then hands-off, please - no questions
asked.

It is typically the file-name reading code that binds/sets
`non-essential', and it is up to whatever code that does that
to decide whether Tramp should be able to prompt for connection
info (password etc.).

Tramp needs to keep its hands off if `non-essential' is non-nil,
and not second-guess _why_ it is non-nil and what that might
really mean to the code that made it non-nil.

Of course, it's possible I'm missing something big here, in
which case I apologize.  But my impression from what I've seen
so far is that you are now digging into the weeds in territory
where Tramp does not really belong.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>> Can you explain *your* understanding of `non-essential', then?
>>> What does this var mean, for you?
>> For Tramp, it is an indication that file name completion is in
>> progress.
>
> Huh?  Why would it need such a variable for that, instead of getting
> this info from the fact that you're replying to
> `file-name-all-completions' (or `file-name-completion') rather than to
> some other request?

Because these aren't the only file name operations being called. You
have seen it in the backtraces I've shown earlier.

> And why would it be called `non-essential' instead of `in-completion'?

I wanted to introduce `completion-only'. You have said, that it would be
too specific, and I shall use `non-essential' instead.

<http://thread.gmane.org/gmane.emacs.bugs/35968/focus=36234>

As I understand the codebase of Emacs 25, no other use of
`non-essential' has appeared since then. Six years later.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Drew Adams <drew.adams@oracle.com> writes:

> Of course, it's possible I'm missing something big here, in
> which case I apologize.  But my impression from what I've seen
> so far is that you are now digging into the weeds in territory
> where Tramp does not really belong.

Maybe you are right, and I'll happily discuss this. But think about the
difference of "a connection has already been established to a host", and
"no connection has been established to a host". Tramp shall support file
name completion at least in the first case. I don't know whether
completion packages want to go into the business of deciding this
question. Therefore, there must be a trade-off between completion
packages and Tramp in order to decide whether to complete.

Anyway, the purpose of *this* bug report is that non-essential hasn't
been bound, and nobody could explain to me why it is wrong to bind it in
pcomplete. I would like to fix *this* problem in the bug report, and
discuss proper usage of non-essential somewhere else, in emacs-devel or
another bug report. Otherwise, we loose focus.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/20/2016 05:54 PM, Michael Albinus wrote:

> Anyway, the purpose of *this* bug report is that non-essential hasn't
> been bound, and nobody could explain to me why it is wrong to bind it in
> pcomplete.

Probably because, if I have typed a valid remote file name for which 
there's no connection, and I explicitly invoke pcomplete, I *want* you 
to prompt me for login and password.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>>>> Can you explain *your* understanding of `non-essential', then?
>>>> What does this var mean, for you?
>>> For Tramp, it is an indication that file name completion is in
>>> progress.
>> Huh?  Why would it need such a variable for that, instead of getting
>> this info from the fact that you're replying to
>> `file-name-all-completions' (or `file-name-completion') rather than to
>> some other request?
> Because these aren't the only file name operations being called. You
> have seen it in the backtraces I've shown earlier.

But in that backtrace, it's OK for Tramp to open a new connection, since
the user hit TAB.

>> And why would it be called `non-essential' instead of `in-completion'?
> I wanted to introduce `completion-only'.

The crucial distinction to be made is not between "performing
completion" and "not performing completion", but between "any normal
operation, including completion in response to TAB" and "side-operations
like on-the-fly completion à la icomplete or company or background data
collection (like semantic might perform)".

> As I understand the codebase of Emacs 25, no other use of
> `non-essential' has appeared since then. Six years later.

It should be bound in desktop.el when reloading a desktop, but instead
desktop.el opted to use ad-hoc file-remote-p tests, IIUC.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Drew Adams]

> > Of course, it's possible I'm missing something big here, in
> > which case I apologize.  But my impression from what I've seen
> > so far is that you are now digging into the weeds in territory
> > where Tramp does not really belong.
> 
> Maybe you are right, and I'll happily discuss this. But think about the
> difference of "a connection has already been established to a host", and
> "no connection has been established to a host". Tramp shall support file
> name completion at least in the first case.

Yes, but I don't see how that contradicts what I said.  Tramp should
(I think) always interpret non-nil `non-essential' as an inhibition
to prompt.  If there is already a connection then presumably
(hopefully) Tramp would not prompt anyway, regardless of the value
of `non-essential'.

> I don't know whether completion packages want to go into the business
> of deciding this question.

They are not _required_ to decide it.  But if they set/bind
`non-essential' to non-nil then I think Tramp should consider
that they _have_ decided it - they've decided that Tramp should
not prompt.

> Therefore, there must be a trade-off between completion
> packages and Tramp in order to decide whether to complete.

I think the trade-off is here: When `non-essential' is nil
then Tramp gets to decide (e.g., depending on whether there
is already a connection - or the phase of the moon, for that
matter).  When `non-essential' is non-nil, Tramp does not
get to decide - the decision has been made that Tramp must
not prompt.

Again, though, I'm no expert on any of this.  Maybe there
are legitimate reasons why Tramp should sometimes prompt
even when `non-essential' is non-nil.  But if there are
then they are beyond my (current) understanding.

> Anyway, the purpose of *this* bug report is that non-essential hasn't
> been bound, and nobody could explain to me why it is wrong to bind it in
> pcomplete. 

I see.  Sorry for intruding here then.  I have nothing to
say about whether pcomplete should bind `non-essential'
to non-nil.  That's apparently a question about pcomplete
behavior and not about Tramp behavior in the face of non-nil
`non-essential'.

> I would like to fix *this* problem in the bug report, and
> discuss proper usage of non-essential somewhere else,
> in emacs-devel or another bug report. Otherwise, we loose focus.

Agreed.  Sorry if my messages here were not helpful.

I would like to know, though, whether you agree generally
that non-nil `non-essential' should inhibit prompting by
Tramp.

If you don't then maybe we can discuss it off line or in
emacs-devel.  I'd like to understand that better, as (I
think) I need to know how to control such prompting in
my code.  I thought that it was sufficient to bind
`non-essential' to non-nil to prevent prompting.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Drew Adams]

Sorry, but please forget all that I said.  In general, whatever the
current behavior of Tramp is for `non-essential' I've found to be
reasonable.  I wouldn't want to suggest any change, being mostly
ignorant of this area.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/20/2016 05:54 PM, Michael Albinus wrote:
>
>> Anyway, the purpose of *this* bug report is that non-essential hasn't
>> been bound, and nobody could explain to me why it is wrong to bind it in
>> pcomplete.
>
> Probably because, if I have typed a valid remote file name for which
> there's no connection, and I explicitly invoke pcomplete, I *want* you
> to prompt me for login and password.

I'll do, once you have typed the first slash. This doesn't depend on
settings of non-essential.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>>>> Can you explain *your* understanding of `non-essential', then?
>>>>> What does this var mean, for you?
>>>> For Tramp, it is an indication that file name completion is in
>>>> progress.
>>> Huh?  Why would it need such a variable for that, instead of getting
>>> this info from the fact that you're replying to
>>> `file-name-all-completions' (or `file-name-completion') rather than to
>>> some other request?
>> Because these aren't the only file name operations being called. You
>> have seen it in the backtraces I've shown earlier.
>
> But in that backtrace, it's OK for Tramp to open a new connection, since
> the user hit TAB.

Tramp does not know that the user hit TAB. It checks for `non-essential'.

>>> And why would it be called `non-essential' instead of `in-completion'?
>> I wanted to introduce `completion-only'.
>
> The crucial distinction to be made is not between "performing
> completion" and "not performing completion", but between "any normal
> operation, including completion in response to TAB" and "side-operations
> like on-the-fly completion à la icomplete or company or background data
> collection (like semantic might perform)".

I don't understand. Tramp doe not know where it has been called
from. It operates stateless. `non-essential' provides some context,
that's all.

>> As I understand the codebase of Emacs 25, no other use of
>> `non-essential' has appeared since then. Six years later.
>
> It should be bound in desktop.el when reloading a desktop, but instead
> desktop.el opted to use ad-hoc file-remote-p tests, IIUC.

I do not care desktop.el just now, it is the case we were discussing 6
years ago. As of today, there is no other use case for `non-essential'
in the codebase but the Tramp case.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Drew Adams <drew.adams@oracle.com> writes:

> Sorry, but please forget all that I said.

Done :-)

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/20/2016 10:31 PM, Michael Albinus wrote:

> I'll do, once you have typed the first slash. This doesn't depend on
> settings of non-essential.

That sounds wrong.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/20/2016 10:31 PM, Michael Albinus wrote:
>
>> I'll do, once you have typed the first slash. This doesn't depend on
>> settings of non-essential.
>
> That sounds wrong.

No. `non-essential' doesn't dictate what Tramp shall do, for example
whether it could open a new connection. It tells Tramp something about
the context ("Be careful, we are in file name completion!"). Tramp
decides then, whether it could perform an operation, or not.

Seeing a slash in the local file name part, as when you type
"/ssh:host:tmp/em TAB", gives Tramp the confidence that it is safe to
open a connection to "host".

I was too sloppy with my statement above. The first slash I've meant is
NOT the slash in "/ssh".

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/20/2016 10:53 PM, Michael Albinus wrote:

> No. `non-essential' doesn't dictate what Tramp shall do, for example
> whether it could open a new connection. It tells Tramp something about
> the context ("Be careful, we are in file name completion!"). Tramp
> decides then, whether it could perform an operation, or not.

According to the docstring, it only tells Tramp not to ask for the 
credentials (when non-essential is non-nil). Whether Tramp is allowed to 
open a connection it already knows credentials for, is not specified 
(though I probably wouldn't).

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>> But in that backtrace, it's OK for Tramp to open a new connection, since
>> the user hit TAB.
> Tramp does not know that the user hit TAB. It checks for `non-essential'.

Exactly: pcomplete tells Tramp that it's OK to prompt for a password by
*not* setting non-essential.  That's how Tramp can know.

>>>> And why would it be called `non-essential' instead of `in-completion'?
>>> I wanted to introduce `completion-only'.
>> The crucial distinction to be made is not between "performing
>> completion" and "not performing completion", but between "any normal
>> operation, including completion in response to TAB" and "side-operations
>> like on-the-fly completion à la icomplete or company or background data
>> collection (like semantic might perform)".
> I don't understand.  Tramp does not know where it has been called
> from. It operates stateless.  `non-essential' provides some context,
> that's all.

That's right.  What I'm pointing out is that the context that Tramp
needs is not "are we performing some kind of completion", but "are we
allowed to prompt the user for a password" (admittedly, `non-essential'
is not limited to "passwords" but more generally means that we should
stay discrete.  E.g. it also means we shouldn't block Emacs for too
long).

> I do not care desktop.el just now, it is the case we were discussing 6
> years ago. As of today, there is no other use case for `non-essential'
> in the codebase but the Tramp case.

Admittedly, the fact that the two sides (let-binder and var-reader)
don't agree on what that variable means, reduces its
usefulness significantly.

In my view, Tramp should never prompt the user for a password (nor
signal an error, tho emitting some warning message might be OK in some
cases) when non-essential is non-nil.

BTW, to clarify:
- Someone reported a bug about company's interaction with Tramp
  (presumably via pcomplete).  I suspect this should be fixed by having
  company bind non-essential and IIUC Dmitry did just that (don't know
  if it does/did fix the problem).
- As part of that company bug, this bug#23006 was filed, which has
  nothing to do with non-essential since it's about a user interaction
  which is "essential".  I've pointed out a few issues that have to do
  with the interaction between file-name-all-completions and
  file-name-directory which might lead to fixing this bug, but AFAIK
  this part of the discussion has not been followed yet.
Could we get back to the interaction between file-name-all-completions
and file-name-directory?


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> According to the docstring, it only tells Tramp not to ask for the
> credentials (when non-essential is non-nil). Whether Tramp is allowed to
> open a connection it already knows credentials for, is not specified (though
> I probably wouldn't).

Indeed, the issue is "don't interfere with the main operation", so
prompting is a clear no-no.  Opening a new connection might be
acceptable (as long as it doesn't block Emacs for too long, for
example).


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/21/2016 12:17 AM, Stefan Monnier wrote:

> BTW, to clarify:
> - Someone reported a bug about company's interaction with Tramp
>   (presumably via pcomplete).

Via pcomplete-completions-at-point.

>  I suspect this should be fixed by having
>   company bind non-essential and IIUC Dmitry did just that (don't know
>   if it does/did fix the problem).

Not really fixed, no. That bug wasn't limited to idle completion, which 
is the only case when I'm binding `non-essential' to t, like you suggested.

And even that bug wasn't about being prompted for credentials. The 
difference in scenarios between that one and this one is very small: 
https://github.com/company-mode/company-mode/pull/487

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/20/2016 10:53 PM, Michael Albinus wrote:
>
>> No. `non-essential' doesn't dictate what Tramp shall do, for example
>> whether it could open a new connection. It tells Tramp something about
>> the context ("Be careful, we are in file name completion!"). Tramp
>> decides then, whether it could perform an operation, or not.
>
> According to the docstring, it only tells Tramp not to ask for the
> credentials (when non-essential is non-nil). Whether Tramp is allowed
> to open a connection it already knows credentials for, is not
> specified (though I probably wouldn't).

Tramp doesn't know in advance whether it has sufficient credentials for
a given connection. Therefore, it is only about whether Tramp shall open
a new connection. Asking for the credentials is the user visible part of
opening a new connection.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/21/2016 05:46 PM, Michael Albinus wrote:

> Tramp doesn't know in advance whether it has sufficient credentials for
> a given connection. Therefore, it is only about whether Tramp shall open
> a new connection. Asking for the credentials is the user visible part of
> opening a new connection.

Then it shouldn't open a new connection, I guess. Or somehow abort as 
soon as it finds out that it doesn't have the sufficient credentials.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> According to the docstring, it only tells Tramp not to ask for the
>> credentials (when non-essential is non-nil). Whether Tramp is allowed to
>> open a connection it already knows credentials for, is not specified (though
>> I probably wouldn't).
>
> Indeed, the issue is "don't interfere with the main operation", so
> prompting is a clear no-no.  Opening a new connection might be
> acceptable (as long as it doesn't block Emacs for too long, for
> example).

Again, Tramp has no chance to determine this in advance. Whether new
credentials are required interactively, or not, is decided only after
Tramp has started to open a new connection.

Therefore, Tramp maps this decision to "shall I open a new connection,
or not". It does not make sense to start with the ping-pong of opening a
new connection, and to stop half the way due to missing credentials.


>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>> But in that backtrace, it's OK for Tramp to open a new connection, since
>>> the user hit TAB.
>> Tramp does not know that the user hit TAB. It checks for `non-essential'.
>
> Exactly: pcomplete tells Tramp that it's OK to prompt for a password by
> *not* setting non-essential.  That's how Tramp can know.

But that's not the case in *this* bug. Opening a new connection is not
the only decision Tramp takes depending on the setting of
non-essential. The other decision is, whether a remote file name like
"/adb:" is intended by the user as complete file name, or whether the
completion machinery is still working, and Tramp could expext also a
hostname after the method "adb". That's the problem in this bug; Tramp
has not been informed that we are still in completion mode, and it takes
"/adb:" serious (allowed to perform essential operations).

>>>>> And why would it be called `non-essential' instead of `in-completion'?
>>>> I wanted to introduce `completion-only'.
>>> The crucial distinction to be made is not between "performing
>>> completion" and "not performing completion", but between "any normal
>>> operation, including completion in response to TAB" and "side-operations
>>> like on-the-fly completion à la icomplete or company or background data
>>> collection (like semantic might perform)".
>> I don't understand.  Tramp does not know where it has been called
>> from. It operates stateless.  `non-essential' provides some context,
>> that's all.
>
> That's right.  What I'm pointing out is that the context that Tramp
> needs is not "are we performing some kind of completion", but "are we
> allowed to prompt the user for a password" (admittedly, `non-essential'
> is not limited to "passwords" but more generally means that we should
> stay discrete.  E.g. it also means we shouldn't block Emacs for too
> long).

As said the other email, this decision doesn't make sense to Tramp. It
can only decide "are we allowed to open a new connection".

>> I do not care desktop.el just now, it is the case we were discussing 6
>> years ago. As of today, there is no other use case for `non-essential'
>> in the codebase but the Tramp case.
>
> Admittedly, the fact that the two sides (let-binder and var-reader)
> don't agree on what that variable means, reduces its
> usefulness significantly.
>
> In my view, Tramp should never prompt the user for a password (nor
> signal an error, tho emitting some warning message might be OK in some
> cases) when non-essential is non-nil.

Again, pls speak about opening a new connection. And also again, this
bug is *not* about Tramp prompting a password when it is not
ecxpected. This bug is about Tramp taking "/adb:" as the final remote file
name a user wants to be treated, because it wasn't informed that we are
still in file name completion.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/21/2016 05:46 PM, Michael Albinus wrote:
>
>> Tramp doesn't know in advance whether it has sufficient credentials for
>> a given connection. Therefore, it is only about whether Tramp shall open
>> a new connection. Asking for the credentials is the user visible part of
>> opening a new connection.
>
> Then it shouldn't open a new connection, I guess. Or somehow abort as
> soon as it finds out that it doesn't have the sufficient credentials.

The latter does not make sense. Starting with "ssh -l user host" and
then aborting due to a missing password is nonsense. More serious, it
could be regarded as attack by the server we try to connect. After
several such attempts (every time you are in file name completion and
must abort due to the missing credentials), the server shall blacklist us.

The first case makes more sense. Once non-essential is non-nil, Tramp
does not open a new connection until it is obvious that the user wants
this. The indication is a slash in the local file name part.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/21/2016 06:03 PM, Michael Albinus wrote:

> The latter does not make sense. Starting with "ssh -l user host" and
> then aborting due to a missing password is nonsense. More serious, it
> could be regarded as attack by the server we try to connect. After
> several such attempts (every time you are in file name completion and
> must abort due to the missing credentials), the server shall blacklist us.

OK.

> The first case makes more sense. Once non-essential is non-nil, Tramp
> does not open a new connection until it is obvious that the user wants
> this. The indication is a slash in the local file name part.

That seems undesirable, and against non-essential's purpose.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> The first case makes more sense. Once non-essential is non-nil, Tramp
>> does not open a new connection until it is obvious that the user wants
>> this. The indication is a slash in the local file name part.
>
> That seems undesirable, and against non-essential's purpose.

Whatever `non-essential's purpose is. *I* have asked for an indication
that file name completion is ongoing. Stefan wanted to use it in a more
general way. I haven't seen the other use cases yet.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/21/2016 06:25 PM, Michael Albinus wrote:

> *I* have asked for an indication
> that file name completion is ongoing.

Do that in Tramp's file-name-all-completions handler?

> Stefan wanted to use it in a more
> general way. I haven't seen the other use cases yet.

Why do you need other cases? This *is* the more general way, and it 
makes more sense to me, as an author of a completion UI.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/21/2016 06:25 PM, Michael Albinus wrote:
>
>> *I* have asked for an indication
>> that file name completion is ongoing.
>
> Do that in Tramp's file-name-all-completions handler?

It's not only file-name-all-completions. Other file name operations are
involved also.

>> Stefan wanted to use it in a more
>> general way. I haven't seen the other use cases yet.
>
> Why do you need other cases? This *is* the more general way, and it
> makes more sense to me, as an author of a completion UI.

I don't need the other cases, and I've never asked for. The hypethetical
existence of such a more general meaning is the argument, that
non-essential does not simply indicate that file name completion is on
the way.

And I don't understand why Tramp shall refuse to provide file name
completion once the user has asked for this explicitely. Typing
"/ssh:host:tmp/em TAB" is a very convincing argument to me, that the
user wants a new connection to "host", and she wants also to get
completion for "tmp/ema" on that host. I cannot see why Tramp shall be
forbidden to do this.

But again, *this* bug report is not about what Tramp does when
non-essential is bound to non-nil. *This* bug report is about, that
non-essential is nil, although file name completion is
undergoing. Remember, we are in eshell, the user has typed "cd /", and
afterwards she has called pcomplete.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/21/2016 06:55 PM, Michael Albinus wrote:

> And I don't understand why Tramp shall refuse to provide file name
> completion once the user has asked for this explicitely. Typing
> "/ssh:host:tmp/em TAB" is a very convincing argument to me, that the
> user wants a new connection to "host", and she wants also to get
> completion for "tmp/ema" on that host. I cannot see why Tramp shall be
> forbidden to do this.

You don't know that the user has pressed TAB. There are contexts where 
the user hasn't pressed TAB, but we still call 
file-name-all-completions, e.g. in an idle timer callback, to show what 
*would* happen *if* they press tab.

> But again, *this* bug report is not about what Tramp does when
> non-essential is bound to non-nil. *This* bug report is about, that
> non-essential is nil, although file name completion is
> undergoing. Remember, we are in eshell, the user has typed "cd /", and
> afterwards she has called pcomplete.

Sure. I'm the author of this report. Still, it would be nice to have 
`non-essential' have the intended effect.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/21/2016 06:55 PM, Michael Albinus wrote:
>
>> And I don't understand why Tramp shall refuse to provide file name
>> completion once the user has asked for this explicitely. Typing
>> "/ssh:host:tmp/em TAB" is a very convincing argument to me, that the
>> user wants a new connection to "host", and she wants also to get
>> completion for "tmp/ema" on that host. I cannot see why Tramp shall be
>> forbidden to do this.
>
> You don't know that the user has pressed TAB. There are contexts where
> the user hasn't pressed TAB, but we still call
> file-name-all-completions, e.g. in an idle timer callback, to show
> what *would* happen *if* they press tab.

Yes, that would be a legitimate scenario to ask Tramp not to open a new
connection. It's the first time somebody explained this, as far as I'm
aware of, thanks!

So maybe we need two different flags for Tramp: One to suppress opening a
new connection under any circumstances, and one flag to tell Tramp that
we are in the file name completion business, and it shall open a new
connection only it is safe (like seeing a slash in the local file name
part). Remember my scenario about ediff'ing remote files; I *want* file
name completion for the second file, even if there's no established
connection yet.

>> But again, *this* bug report is not about what Tramp does when
>> non-essential is bound to non-nil. *This* bug report is about, that
>> non-essential is nil, although file name completion is
>> undergoing. Remember, we are in eshell, the user has typed "cd /", and
>> afterwards she has called pcomplete.
>
> Sure. I'm the author of this report. Still, it would be nice to have
> `non-essential' have the intended effect.

Still, it would be nice to have let-bound non-essential in order to get
an effect.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> And I don't understand why Tramp shall refuse to provide file name
> completion once the user has asked for this explicitely. Typing
> "/ssh:host:tmp/em TAB" is a very convincing argument to me, that the
> user wants a new connection to "host",

We all agree.  And in that case, non-essential will be nil, so Tramp
can open a connection.  It can even do that without the second slash.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>> Indeed, the issue is "don't interfere with the main operation", so
>> prompting is a clear no-no.  Opening a new connection might be
>> acceptable (as long as it doesn't block Emacs for too long, for
>> example).
> Again, Tramp has no chance to determine this in advance. Whether new
> credentials are required interactively, or not, is decided only after
> Tramp has started to open a new connection.

That's OK.  I'm only talking about the intended meaning of
non-essential, independently of any particular use such as in Tramp.
If there's a risk of problems and non-essential is non-nil, then don't
open a new connection.  If there's always such a risk, then never open
a new connection (when non-essential is non-nil).


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/21/2016 08:36 PM, Michael Albinus wrote:

> Yes, that would be a legitimate scenario to ask Tramp not to open a new
> connection. It's the first time somebody explained this, as far as I'm
> aware of, thanks!

My pleasure.

> So maybe we need two different flags for Tramp: One to suppress opening a
> new connection under any circumstances, and one flag to tell Tramp that
> we are in the file name completion business, and it shall open a new
> connection only it is safe (like seeing a slash in the local file name
> part). Remember my scenario about ediff'ing remote files; I *want* file
> name completion for the second file, even if there's no established
> connection yet.

Do you have any reason to expect that `non-essential' would be set to 
non-nil in that context?

 From your description, I see the reason for the connection to be 
established. It does not describe the slash-related caveats, and what 
would make you avoid establishing a connection, e.g. in the case of ediff.

>> Sure. I'm the author of this report. Still, it would be nice to have
>> `non-essential' have the intended effect.
>
> Still, it would be nice to have let-bound non-essential in order to get
> an effect.

It's not that simple. First of all, I came to this bug from a report by 
a company-mode user. And apparently we're getting the same error, 
through pcomplete-completion-at-point, from two different user 
interactions: one that should set `non-essential' (like Stefan 
suggested), that would be the "idle completion", and one that shouldn't 
set `non-essential', that would be manually initiated completion.

So setting `non-essential' in pcomplete-completion-at-point, or one of 
its callees, looks like a wrong solution to me.

But getting back to this bug report:

I've just typed `M-x pcomplete', which should be a good indicator that I 
probably want to be prompted for credentials, if required. 
`non-essential' should be nil in this scenario. And yet, I shouldn't see 
that error.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> And I don't understand why Tramp shall refuse to provide file name
>> completion once the user has asked for this explicitely. Typing
>> "/ssh:host:tmp/em TAB" is a very convincing argument to me, that the
>> user wants a new connection to "host",
>
> We all agree.  And in that case, non-essential will be nil, so Tramp
> can open a connection.  It can even do that without the second slash.

As Dmitry has shown, there might be cases Tramp shall not open a new
connection even in this case (called from an idle timer). So we need a
distinction between "it is always blocked to open a new connection" and
"We recommend Tramp to open a new connection only when it is safe for
Tramp". One single value of non-essential doesn't fit this bill.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>>> Indeed, the issue is "don't interfere with the main operation", so
>>> prompting is a clear no-no.  Opening a new connection might be
>>> acceptable (as long as it doesn't block Emacs for too long, for
>>> example).
>> Again, Tramp has no chance to determine this in advance. Whether new
>> credentials are required interactively, or not, is decided only after
>> Tramp has started to open a new connection.
>
> That's OK.  I'm only talking about the intended meaning of
> non-essential, independently of any particular use such as in Tramp.
> If there's a risk of problems and non-essential is non-nil, then don't
> open a new connection.  If there's always such a risk, then never open
> a new connection (when non-essential is non-nil).

Unless we are running into the scenario given by Dmitry ...

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> So maybe we need two different flags for Tramp: One to suppress opening a
>> new connection under any circumstances, and one flag to tell Tramp that
>> we are in the file name completion business, and it shall open a new
>> connection only it is safe (like seeing a slash in the local file name
>> part). Remember my scenario about ediff'ing remote files; I *want* file
>> name completion for the second file, even if there's no established
>> connection yet.
>
> Do you have any reason to expect that `non-essential' would be set to
> non-nil in that context?

Yes. The reason is that it has been always my understanding of
non-essential. See the discussion between Stefan and me six years ago,
when non-essential was born.

Tramp needs an indication that file name completion is in
progress. Otherwise, it would regard remote file names like "/adb:" safe
to be handled, and it would report an error then. Again, we are speaking
about arbitray file name operations but file-name-all-completions and
file-name-completion. Every completion package is free to call any magic
file name operation while file name completion is in progress, it isn't
forbidden anywhere.

> From your description, I see the reason for the connection to be
> established. It does not describe the slash-related caveats, and what
> would make you avoid establishing a connection, e.g. in the case of
> ediff.

I start to believe that we need two different indications. Let's assume
non-essential provides two different non-nil values, `strict' and
`best-effort'. (I don't insist in these names, I even don't insist it
must be indicated by non-essential. Any other variable would do).

In the `strict' case, Tramp must not open a new connection under any
circumstances. That's your idle timer case. There might be also other
cases, but it shall be used with care, because it restricts Tramp's
convenience.

In the `best-effort' case, Tramp is said that file name completion is in
progress, and it shouldn't open a new connection even if the file name
to be completed looks like a valid remote file name, like "/adb:". Tramp
opens a new connection only when it is safe, for example when it has
detected a slash in the local file name part of the file name to be
completed. That's the case when the user has hit TAB, for example. File
name completion packages, which bind non-essential to true these days,
shall change this to a binding of `best-effort'.

>> Still, it would be nice to have let-bound non-essential in order to get
>> an effect.
>
> It's not that simple. First of all, I came to this bug from a report
> by a company-mode user. And apparently we're getting the same error,
> through pcomplete-completion-at-point, from two different user
> interactions: one that should set `non-essential' (like Stefan
> suggested), that would be the "idle completion", and one that
> shouldn't set `non-essential', that would be manually initiated
> completion.
>
> So setting `non-essential' in pcomplete-completion-at-point, or one of
> its callees, looks like a wrong solution to me.
>
> But getting back to this bug report:
>
> I've just typed `M-x pcomplete', which should be a good indicator that
> I probably want to be prompted for credentials, if
> required. `non-essential' should be nil in this scenario. And yet, I
> shouldn't see that error.

`strict' is superior to `best-effort'. Coming back to this bug report, I
would expect that non-essential is bound to `best-effort' somewhere in
pcomplete or the involved completion package(s). When one of these
packages determine, that they run in an idle timer (or something
comparable dangerous), they should overwrite this binding of non-essential
with `strict'.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>>> And I don't understand why Tramp shall refuse to provide file name
>>> completion once the user has asked for this explicitely. Typing
>>> "/ssh:host:tmp/em TAB" is a very convincing argument to me, that the
>>> user wants a new connection to "host",
>> We all agree.  And in that case, non-essential will be nil, so Tramp
>> can open a connection.  It can even do that without the second slash.
> As Dmitry has shown, there might be cases Tramp shall not open a new
> connection even in this case (called from an idle timer).

In the idle timer case, non-essential will be non-nil, so Tramp should
never open a connection.

> So we need a distinction between "it is always blocked to open a new
> connection" and "We recommend Tramp to open a new connection only when
> it is safe for Tramp". One single value of non-essential doesn't fit
> this bill.

I must have missed something.  Can you repeat which are the three
different cases you have in mind?
[ In this message, I only see two cases, and I'm not sure what the
  third would be.  ]


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> Unless we are running into the scenario given by Dmitry ...

Which scenario is that?


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> Tramp needs an indication that file name completion is in progress.

That's where we disagree.

> Otherwise, it would regard remote file names like "/adb:" safe
> to be handled, and it would report an error then.

This bug report is not about "cd /adb: TAB signals an error" but about
"cd / TAB signals an error".  The "/adb:" part comes from Tramp because
of a problem in Tramp's implementation of file-name-directory, AFAICT.
It has nothing to do with file name completion.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Unless we are running into the scenario given by Dmitry ...
>
> Which scenario is that?

Idle timer.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>> And I don't understand why Tramp shall refuse to provide file name
>>>> completion once the user has asked for this explicitely. Typing
>>>> "/ssh:host:tmp/em TAB" is a very convincing argument to me, that the
>>>> user wants a new connection to "host",
>>> We all agree.  And in that case, non-essential will be nil, so Tramp
>>> can open a connection.  It can even do that without the second slash.
>> As Dmitry has shown, there might be cases Tramp shall not open a new
>> connection even in this case (called from an idle timer).
>
> In the idle timer case, non-essential will be non-nil, so Tramp should
> never open a connection.
>
>> So we need a distinction between "it is always blocked to open a new
>> connection" and "We recommend Tramp to open a new connection only when
>> it is safe for Tramp". One single value of non-essential doesn't fit
>> this bill.
>
> I must have missed something.  Can you repeat which are the three
> different cases you have in mind?
> [ In this message, I only see two cases, and I'm not sure what the
>   third would be.  ]

I'm speaking only about two different cases. File name completion
started by user activity, like hitting TAB. And file name completion
started in the background, for example via idle timers.

If I have said "three" somewhere else, it was a mistake.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Tramp needs an indication that file name completion is in progress.
>
> That's where we disagree.
>
>> Otherwise, it would regard remote file names like "/adb:" safe
>> to be handled, and it would report an error then.
>
> This bug report is not about "cd /adb: TAB signals an error" but about
> "cd / TAB signals an error".  The "/adb:" part comes from Tramp because
> of a problem in Tramp's implementation of file-name-directory, AFAICT.
> It has nothing to do with file name completion.

cd / TAB calls file-name-all-completions. This returns several
candidates, like "/adb:". That's OK.

Afterwards, (file-directory-p "adb:") is called from completion-file-name-table:

  ...
  expand-file-name("adb:" "/")
  apply(expand-file-name ("adb:" "/"))
  (let* ((inhibit-file-name-handlers (cons (quote tramp-completion-file-name-handler) (cons (quote cygwin-mount-name-hook-function) (cons (quote cygwin-mount-map-drive-hook-function) (and (eq inhibit-file-name-operation operation) inhibit-file-name-handlers))))) (inhibit-file-name-operation operation)) (apply operation args))
  tramp-completion-run-real-handler(expand-file-name ("adb:" "/"))
  (if (and fn tramp-mode (or (eq tramp-syntax (quote sep)) (featurep (quote tramp)) (and (boundp (quote partial-completion-mode)) (symbol-value (quote partial-completion-mode))) (featurep (quote ido)) (featurep (quote icicles)))) (let ((save-match-data-internal (match-data))) (unwind-protect (progn (apply (cdr fn) args)) (set-match-data save-match-data-internal (quote evaporate)))) (tramp-completion-run-real-handler operation args))
  (let ((fn (assoc operation tramp-completion-file-name-handler-alist))) (if (and fn tramp-mode (or (eq tramp-syntax (quote sep)) (featurep (quote tramp)) (and (boundp (quote partial-completion-mode)) (symbol-value (quote partial-completion-mode))) (featurep (quote ido)) (featurep (quote icicles)))) (let ((save-match-data-internal (match-data))) (unwind-protect (progn (apply (cdr fn) args)) (set-match-data save-match-data-internal (quote evaporate)))) (tramp-completion-run-real-handler operation args)))
  tramp-completion-file-name-handler(expand-file-name "adb:" "/")
  file-directory-p("adb:")
  #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"]("adb:")
  completion-file-name-table("/" #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] t)
  comint-completion-file-name-table("/" #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] t)
  complete-with-action(t comint-completion-file-name-table "/" #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"])
  completion-table-with-predicate(comint-completion-file-name-table #[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] strict "/" nil t)
  #[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"]("/" nil t)
  complete-with-action(t #[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] "/" nil)
  #[771 "\302G\303	\x06\"\203)\x04\304\305\224OB\262\306\307\310\a\"!B\262\x04\305\225\302O\262\202\0\203:\x02\311=\204O\x02\242\312=\204O\313\314\x06B\237\315#\316\300\x06\b$\262\202\307\317\320\321\322\"\"\313\314\aB\237\315#\323\300\b\b\243\206k\315$\211@Y\203\223\x05\311=\203\204\316\x06\300\x06\n$\202\305\312@\x04GZ\\ABB\202\305\x05\311=\203\272\316\x06\300\x06\n$\x02@Z\321\324\325\326\327\330\x06!\331\"\332\333%\"\266\202\202\305\312\x05GZ\\ABB\266\203\207" [#[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] pcomplete--env-regexp nil string-match 0 1 getenv match-string 2 t boundaries mapconcat identity "" complete-with-action apply + mapcar length completion-boundaries make-byte-code 257 "\211\300\301O\207" vconcat vector [nil] 4 "\n\n(fn S)"] 17 "\n\n(fn STRING PRED ACTION)"]("/" nil t)
  all-completions("/" #[771 "\302G\303	\x06\"\203)\x04\304\305\224OB\262\306\307\310\a\"!B\262\x04\305\225\302O\262\202\0\203:\x02\311=\204O\x02\242\312=\204O\313\314\x06B\237\315#\316\300\x06\b$\262\202\307\317\320\321\322\"\"\313\314\aB\237\315#\323\300\b\b\243\206k\315$\211@Y\203\223\x05\311=\203\204\316\x06\300\x06\n$\202\305\312@\x04GZ\\ABB\202\305\x05\311=\203\272\316\x06\300\x06\n$\x02@Z\321\324\325\326\327\330\x06!\331\"\332\333%\"\266\202\202\305\312\x05GZ\\ABB\266\203\207" [#[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] pcomplete--env-regexp nil string-match 0 1 getenv match-string 2 t boundaries mapconcat identity "" complete-with-action apply + mapcar length completion-boundaries make-byte-code 257 "\211\300\301O\207" vconcat vector [nil] 4 "\n\n(fn S)"] 17 "\n\n(fn STRING PRED ACTION)"])
  pcomplete-stub("/" #[771 "\302G\303	\x06\"\203)\x04\304\305\224OB\262\306\307\310\a\"!B\262\x04\305\225\302O\262\202\0\203:\x02\311=\204O\x02\242\312=\204O\313\314\x06B\237\315#\316\300\x06\b$\262\202\307\317\320\321\322\"\"\313\314\aB\237\315#\323\300\b\b\243\206k\315$\211@Y\203\223\x05\311=\203\204\316\x06\300\x06\n$\202\305\312@\x04GZ\\ABB\202\305\x05\311=\203\272\316\x06\300\x06\n$\x02@Z\321\324\325\326\327\330\x06!\331\"\332\333%\"\266\202\202\305\312\x05GZ\\ABB\266\203\207" [#[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] pcomplete--env-regexp nil string-match 0 1 getenv match-string 2 t boundaries mapconcat identity "" complete-with-action apply + mapcar length completion-boundaries make-byte-code 257 "\211\300\301O\207" vconcat vector [nil] 4 "\n\n(fn S)"] 17 "\n\n(fn STRING PRED ACTION)"])
  pcomplete-do-complete("/" #[771 "\302G\303	\x06\"\203)\x04\304\305\224OB\262\306\307\310\a\"!B\262\x04\305\225\302O\262\202\0\203:\x02\311=\204O\x02\242\312=\204O\313\314\x06B\237\315#\316\300\x06\b$\262\202\307\317\320\321\322\"\"\313\314\aB\237\315#\323\300\b\b\243\206k\315$\211@Y\203\223\x05\311=\203\204\316\x06\300\x06\n$\202\305\312@\x04GZ\\ABB\202\305\x05\311=\203\272\316\x06\300\x06\n$\x02@Z\321\324\325\326\327\330\x06!\331\"\332\333%\"\266\202\202\305\312\x05GZ\\ABB\266\203\207" [#[771 "\211\305=\203\0	\203\0\305\306\307B\310\x05#ABB\207\311\n\x1c\312\313\300\314\x06\x06\x06&*\207" [#[257 "\302\203\n\302!\205\0\301\203\0\301!\205\0\300?\206\0\300!\207" [file-directory-p #[257 "\211GSH\302=\203\0\301\205\0\303\301\"\202\0\300\205\0\303\300\"?\207" ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"] nil] 3 "\n\n(fn F)"] pcomplete-compare-entry-function pcomplete-ignore-case completion-ignore-case completion-ignored-extensions metadata cycle-sort-function #[257 "\301\b\"\207" [pcomplete-compare-entry-function sort] 4 "\n\n(fn COMPS)"] completion-file-name-table nil completion-table-with-predicate comint-completion-file-name-table strict] 10 "\n\n(fn S P A)"] pcomplete--env-regexp nil string-match 0 1 getenv match-string 2 t boundaries mapconcat identity "" complete-with-action apply + mapcar length completion-boundaries make-byte-code 257 "\211\300\301O\207" vconcat vector [nil] 4 "\n\n(fn S)"] 17 "\n\n(fn STRING PRED ACTION)"])
  pcomplete(1)
  funcall-interactively(pcomplete 1)
  call-interactively(pcomplete record nil)
  command-execute(pcomplete record)
  execute-extended-command(nil "pcomplete" "pcomplete")
  funcall-interactively(execute-extended-command nil "pcomplete" "pcomplete")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

Tramp has no information, that file-name-completion is still in
progress, and does its job. It tries to connect to the remote host
"/adb:" in order to check.

If Tramp would have been said, that file name completion is still in
progress, it would behave correctly.

And it isn't only about file-directory-p, that's just an example. *Any*
file name operation could be called while file name completion is in
progress. That's why Tramp needs this information.

And yes, this is our basic disagreement. I'm not able to implement
proper Tramp operation without this information. Why do you refuse to
tell this to Tramp?

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> I'm speaking only about two different cases. File name completion
> started by user activity, like hitting TAB. And file name completion
> started in the background, for example via idle timers.

Also known as: the case where non-essential is nil and the case where
it's non-nil.

> If I have said "three" somewhere else, it was a mistake.

You said:

    One single value of non-essential doesn't fit this bill.


-- Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>>> Unless we are running into the scenario given by Dmitry ...
>> Which scenario is that?
> Idle timer.

That's the scenario where non-essential is non-nil.
IOW, the same scenario as icomplete.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I'm speaking only about two different cases. File name completion
>> started by user activity, like hitting TAB. And file name completion
>> started in the background, for example via idle timers.
>
> Also known as: the case where non-essential is nil and the case where
> it's non-nil.

Stefan, just a simple question: do you agree that Tramp shall be
informed that file name completion is in progress? I do not insist
that's via non-essential, any way this information bit is given will be
appreciated.

And in case you refuse this: why?

> -- Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> cd / TAB calls file-name-all-completions. This returns several
> candidates, like "/adb:". That's OK.

Indeed.

> Afterwards, (file-directory-p "adb:") is called from completion-file-name-table:

Aha, now we're going somewhere:

>   file-directory-p("adb:")
>   #[257 "\302\203\n\302!\205\301\203\301!\205\300?\206\300!\207"
> [file-directory-p #[257
> "\211GSH\302=\203\301\205\303\301\"\202\300\205\303\300\"?\207"
> ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"]
> nil] 3 "\n\n(fn F)"]("adb:")
>   completion-file-name-table("/" #[257
> "\302\203\n\302!\205\301\203\301!\205\300?\206\300!\207"
> [file-directory-p #[257
> "\211GSH\302=\203\301\205\303\301\"\202\300\205\303\300\"?\207"
> ["~\\'" "\\`\\(\\.\\.?\\|CVS\\)/\\'" 47 string-match] 4 "\n\n(fn FILE)"]
> nil] 3 "\n\n(fn F)"] t)

So, it seems the issue is that the completion predicate calls
file-directory-p and that this happens in a part of the file name that's
just at the boundary between "a local file name" and "a remote file
name", so while file-name-all-completions did not need receive a remote
file name it returned (apparently) remote file names.

One avenue that may be needed then is to bind non-essential while
checking the completion predicate (somewhere in pcomplete.el, or
comint.el, or minibuffer.el.).

But in this specific instance, (file-directory-p "/adb:") should simply
return nil, IMO.

> Tramp has no information, that file-name-completion is still in
> progress, and does its job. It tries to connect to the remote host
> "/adb:" in order to check.

For me (file-directory-p "/adb:") doesn't try to connect to "adb" but
instead signals "Host name must not match method name 'adb'".  I think
it's correct not to try to connect to "adb", but I think it should just
return nil because "/adb:" is not a directory.

> And yes, this is our basic disagreement.  I'm not able to implement
> proper Tramp operation without this information.  Why do you refuse to
> tell this to Tramp?

Because I don't see any fundamental difference between "completion" and
"non completion".  I do see that in some cases, the completion code may
try to "generate" new file names, and maybe in these particular
sub-cases it would make sense to bind non-essential, but not wholesale
around the whole completion operation.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> Stefan, just a simple question: do you agree that Tramp shall be
> informed that file name completion is in progress?

No.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Tramp has no information, that file-name-completion is still in
>> progress, and does its job. It tries to connect to the remote host
>> "/adb:" in order to check.
>
> For me (file-directory-p "/adb:") doesn't try to connect to "adb" but
> instead signals "Host name must not match method name 'adb'".  I think
> it's correct not to try to connect to "adb", but I think it should just
> return nil because "/adb:" is not a directory.

As said, it is not about file-directory-p. Any file name operation could
be called while file name completion is in progress.

>         Stefan

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Stefan, just a simple question: do you agree that Tramp shall be
>> informed that file name completion is in progress?
>
> No.

I give up. EOD.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>>> Tramp has no information, that file-name-completion is still in
>>> progress, and does its job. It tries to connect to the remote host
>>> "/adb:" in order to check.
>> For me (file-directory-p "/adb:") doesn't try to connect to "adb" but
>> instead signals "Host name must not match method name 'adb'".  I think
>> it's correct not to try to connect to "adb", but I think it should just
>> return nil because "/adb:" is not a directory.
> As said, it is not about file-directory-p. Any file name operation could
> be called while file name completion is in progress.

I think it's worth going into the details to see why/where an actual
real problem comes up.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

Hey Michael,

On 03/22/2016 03:35 PM, Michael Albinus wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> I'm speaking only about two different cases. File name completion
>>> started by user activity, like hitting TAB. And file name completion
>>> started in the background, for example via idle timers.
>>
>> Also known as: the case where non-essential is nil and the case where
>> it's non-nil.

This looks reasonable to me.

> Stefan, just a simple question: do you agree that Tramp shall be
> informed that file name completion is in progress?

How would that happen? I don't have to use file-name-all-completions to 
implement file completion operation. Until recently, company-files used 
directory-files at least for some inputs, to implement file completion.

Would every third-party author who decided to implement their own file 
completion mechanism have to find out about some new, Tramp-related 
variable, and bind it to t in their code?

That doesn't sound great.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/22/2016 11:47 AM, Michael Albinus wrote:

>>> part). Remember my scenario about ediff'ing remote files; I *want* file
>>> name completion for the second file, even if there's no established
>>> connection yet.
>>
>> Do you have any reason to expect that `non-essential' would be set to
>> non-nil in that context?
>
> Yes. The reason is that it has been always my understanding of
> non-essential. See the discussion between Stefan and me six years ago,
> when non-essential was born.

I've only managed to find the 2012's discussion in the archive.

> In the `strict' case, Tramp must not open a new connection under any
> circumstances. That's your idle timer case. There might be also other
> cases, but it shall be used with care, because it restricts Tramp's
> convenience.

So far, company-files has been doing this with

   (or (not (file-remote-p file))
       (file-remote-p file nil t))

where file is the current input.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

> How would that happen? I don't have to use file-name-all-completions to
> implement file completion operation. Until recently, company-files used
> directory-files at least for some inputs, to implement file completion.

That's exactly Michael's point: that Tramp can't know that the current
operation is part of some kind of completion, just by looking at which
Tramp functions are called.

The question is: does Tramp need to know whether or not this
`directory-files', or `file-directory-p' (or
`file-name-all-completions', which may also be called in non-completion
contexts) is called as part of a completion operation?


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

On 03/24/2016 03:15 PM, Stefan Monnier wrote:

> That's exactly Michael's point: that Tramp can't know that the current
> operation is part of some kind of completion, just by looking at which
> Tramp functions are called.

That seems to be everybody's point.

> The question is: does Tramp need to know whether or not this
> `directory-files', or `file-directory-p' (or
> `file-name-all-completions', which may also be called in non-completion
> contexts) is called as part of a completion operation?

A general purpose function cannot know.

file-name-all-completions cannot know either, because it can be called 
outside of completion context.

A completion UI cannot know, in general, because it can be used to 
complete just about anything, and it doesn't always know what it's 
completing.

Can completion-file-name-table know?

Maybe a niche completion backend like company-files is the only kind of 
code that *can* know because it's not used for anything else, for now. 
But it's third-party code anyway.

Maybe "we're in a completion operation" is not something that's reliably 
knowable, in a loosely coupled system like Emacs.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Stefan Monnier]

>> The question is: does Tramp need to know whether or not this
>> `directory-files', or `file-directory-p' (or
>> `file-name-all-completions', which may also be called in non-completion
>> contexts) is called as part of a completion operation?
>
> A general purpose function cannot know.
> file-name-all-completions cannot know either, because it can be called
> outside of completion context.
>
> A completion UI cannot know, in general, because it can be used to complete
> just about anything, and it doesn't always know what it's completing.

AFAIK, by definition, a completion UI knows that it's performing completion.
So, I don't see much difficulty in telling Tramp that we're
performing completion: just let-bind some new `in-completion' variable
in the few completion UIs we have.

The problem is that I don't see any reason why Tramp would need to know.


        Stefan

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

> As mentioned in a recent discussion:
>
> 1. M-x eshell
> 2. type 'cd /'
> 3. M-: (require 'tramp) ;; New step!
> 4. M-x pcomplete
>
> I'm getting the error 'tramp-file-name-handler: Host name must not match
> method "adb"'.

I've committed a patch to master which ought to solve this (and
hopefully other completion related) bugs for Tramp. Could you, pls
check?

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

> As mentioned in a recent discussion:
>
> 1. M-x eshell
> 2. type 'cd /'
> 3. M-: (require 'tramp) ;; New step!
> 4. M-x pcomplete
>
> I'm getting the error 'tramp-file-name-handler: Host name must not match
> method "adb"'.

According to the recent changes of Tramp in the master branch, this
error does not happen anymore. Could you, pls, confirm?

I would expect pcomplete also to show possible Tramp methods like
"sudo:". It doesn't. But this is rather related to pcomplete, so it is
not relevant for *this* bug.

Best regards, Michael.

bug#23006: 25.0.92; Loading Tramp breaks pcomplete in eshell-mode

[Dmitry Gutov]

Hi Michael,

On 15.03.2017 14:42, Michael Albinus wrote:

>> As mentioned in a recent discussion:
>>
>> 1. M-x eshell
>> 2. type 'cd /'
>> 3. M-: (require 'tramp) ;; New step!
>> 4. M-x pcomplete
>>
>> I'm getting the error 'tramp-file-name-handler: Host name must not match
>> method "adb"'.
> 
> According to the recent changes of Tramp in the master branch, this
> error does not happen anymore. Could you, pls, confirm?

Looks fixed, thanks! Closing.