Re: TRAMP VC optimization fails: non-TRAMP filenames handled incorrectly in async operations.

[Daniel Pittman <slippycheeze@google.com>]

[-- Attachment #1: Type: text/plain, Size: 3361 bytes --]

On Wed, Mar 27, 2019 at 3:55 PM Michael Albinus <michael.albinus@gmx.de>
wrote:

> Daniel Pittman <slippycheeze@google.com> writes:
>
> > I don't have a patch, but I'll see if I can figure out how to improve
> > this.  I fear the situation is impossible, however, and that this
> > attempt to improve the performance of `vc-registered` is doomed to
> > failure unless an async protocol is defined to replace the current,
> > synchronous, version.
>
> Thanks for the very thourogh report, much appreciated! I will check in
> parallel to you what could be done.
>

No problem.


> There exists a branch in the git repo, "feature/tramp-thread-safe". This
> could solve the issue, if "async", as you've said, means Emacs
> threads. Tramp uses mutexes, which should protect its
> asynchrosity. However, the work on this branch is stalled due to serious
> problems I'm not able to solve myself.
>
> If you mean something else with "async", then we need even another
> approach.


Sorry, this response turned out long.  I'm afraid I haven't the time to
condense further, but a summary: threads are not necessary, I believe, and
so we should disable this optimization for now, then bring it back if it is
provably safe.

The long form with analysis and so forth:

I believe that threads are not necessary: Emacs can dispatch, for example,
any timer, network connection, or external process, callback when it enters
the main wait loop.  Since tramp is going there while it awaits feedback
from the remote system (as far as I understand both modern tramp, and the
event loop) then it can enter essentially arbitrary code.

So, if it blocked any other file operations that'd be a deadlock if the
code – for example, the buffer and network connection for `(url-retrieve "
http://example.com" (lambda (&rest uh-oh) ...))` could call into anything,
which in the case of, eg, the package.el system fetching an update to the
package list, anticipates being able to write the file locally.

I'm confident it was possible to have arbitrary callbacks like that trigger
while tramp was awaiting remote data a decade or so ago, when I was last
working on it.  I presume, but can't be certain, that this remains true
today, so most of my concrete analysis comes with "I believe, but could be
wrong" attached.  It doesn't seem to have changed, though.

My best guess is that we should disable that optimization for now, and if
desirable, reapproach it.

One possibly functional strategy, but that I have not considered all
possible angles of, might be to fetch the path and then if `(not
(tramp-file-name-p ...))` dispatch to the original file name handlers.  I
think that would absolutely work as long as none of those callbacks
interacted with a tramp path at all, and it .... might, but probably
wouldn't, if they did.

Historically, I attached a tramp operation to copy the generated
server(-start) key to a remote tramp path, since I used a TCP listener, ssh
forwarding, and that shared secret to allow remote emacsclient to work.
That could have triggered at any point after tramp reconnected, as it
advised a fairly low level tramp function.  I'm not sure how common (or
cared about) that sort of nastly hack is, but I'm not confident that more
legitimate ways to do the same could be in use in the wild.

[-- Attachment #2: Type: text/html, Size: 4249 bytes --]