From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Daniel Pittman Newsgroups: gmane.emacs.devel Subject: TRAMP VC optimization fails: non-TRAMP filenames handled incorrectly in async operations. Date: Wed, 27 Mar 2019 15:38:23 +0000 Message-ID: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000008e5d370585153ced" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="202335"; mail-complaints-to="usenet@blaine.gmane.org" To: emacs-devel Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Mar 27 16:40:18 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h9Af1-000qT0-9n for ged-emacs-devel@m.gmane.org; Wed, 27 Mar 2019 16:40:15 +0100 Original-Received: from localhost ([127.0.0.1]:49628 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h9Af0-0004fA-9V for ged-emacs-devel@m.gmane.org; Wed, 27 Mar 2019 11:40:14 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:52381) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h9Ads-0004KG-85 for emacs-devel@gnu.org; Wed, 27 Mar 2019 11:39:05 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h9Adq-00010A-Sl for emacs-devel@gnu.org; Wed, 27 Mar 2019 11:39:04 -0400 Original-Received: from mail-it1-x131.google.com ([2607:f8b0:4864:20::131]:37093) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1h9Adq-0000zV-GP for emacs-devel@gnu.org; Wed, 27 Mar 2019 11:39:02 -0400 Original-Received: by mail-it1-x131.google.com with SMTP id u65so848806itc.2 for ; Wed, 27 Mar 2019 08:39:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=6B5I5L8ayOV2YY+VPLI5YT0cYyoha4h2RM5J7b3gS4Y=; b=ASSBDs8542FXR55gHK5GT05mcpwUXD85Kxph1OiKH18q0PE9qW/LlAKqGu5fVAWPEf qIOZ8JztQD//3QuyIC+yJSFQM1ySL8fPbqtFGgumudpepJPk+OiAzc9olahst0VSS203 N8bMKVG9Gtbt5o7j6o7hNgVIdq18I3fgRBPs8t4WWzwCV9gNYFt6PtKNF1NNKjPfykA8 EUs1OsOH5YeRUGWDfqzr4+v8e0jz24nI7QOiTa62Zkf6jewg/KtZ+7phsZAdHsex4Rvb LMau7N/FXMhz2CGf6/N/IfZol4XzJf3fPuJ5n1VrR5jr+UPei+IrZQ7LrlgsZhLSIzmA aumQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=6B5I5L8ayOV2YY+VPLI5YT0cYyoha4h2RM5J7b3gS4Y=; b=GPurJb3xfk7MwZXkSimMGZgiPB5EQC1OPkxhZPtU22Ort7vL0LO0E9iQaFEVx8D/mw sjk0SnyOEAaZMiz0aa4CBoFZOSX4d6CwfCvGexIOqHbXZvCV0NLlfwEWkeo6tthBKe7W dwjkxghVBRkZahix3afQoS7Y8yUFHJmE+NiaYCIZ8wOpkWZOWw4vP8jORJTLtdCCSvl8 H3Gh0miLV/unRzzh8Q8XK2M9uFj3B8yrkc5SMq2xn46a+z5/29bRjatyEtZAmQCYHT0g ccQ6HDKQpY4iDIl77NMLx5TRcYBd3uuHVl28qGWRLVF90Bddju9cKJ5pehhwTQrziCpI gDLA== X-Gm-Message-State: APjAAAU05QuvCWR3iU4BcHcWiVA0SwElkdexWB3YLpZxy4y+zmuRyMWc uyKntczygOoqlPjCRN1RAyTXJkzkkNCmj3TeVDUYiNpL86JcEw== X-Google-Smtp-Source: APXvYqwL35kFNzGpk2Tdu1Bk5CZ6z6S8usZL2TKUrE3X0d+pY5C1YdFbe1OI10b1H5rx2fuMRUAL5QTmQygp4NRAVVM= X-Received: by 2002:a24:16d4:: with SMTP id a203mr3516622ita.52.1553701140304; Wed, 27 Mar 2019 08:39:00 -0700 (PDT) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::131 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:234774 Archived-At: --0000000000008e5d370585153ced Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable G'day. In HEAD, tramp-sh has some optimizations in place for the `vc-registered` operation, intended to minimize the number of round trips. Sadly, these don't seem to be safe in a single threaded Emacs world, and even less safe in a threaded one. The function `tramp-sh-handle-vc-registered` attempts to optimize the process by emplacing a separate file-name-handler, running the operation(s), then performing a single remote call to obtain all the data. This is an optimization, no doubt, and would be great ... except that it assumes a synchronous operation where no other code can run while the new file handler is in effect. This is, sadly, not the case. The observable symptom is that we hit this error throw in `tramp-dissect-file-name`: (tramp-user-error nil "Not a Tramp file name: \"%s\"" name) The most common place I observed this was during startup, when it complained that the `nsm-settings-file`, which is the default value in my case, was not a TRAMP filename. It also happens for expanding `~`, and a handful of other cases, when I was reverting tramp buffers, etc. On debugging I discovered that the problems were springing from calls something akin to this: (let ((default-directory "/ssh:slippycheeze@example.com:/tmp")) (expand-file-name "~" default-directory)) That uses, via `tramp-vc-file-name-handler`, `tramp-file-name-for-operation`, which in the case of `expand-file-name` returns "~". I believe that is correct, but if not, that may be the root cause of the problem. In any case, that is then passed to `tramp-dissect-file-name` by `tramp-vc-file-name-handler`, and (definitely correctly) triggers the error that a non-tramp filename is being parsed. This triggered from an async lisp callback firing, while the current buffer happened to be on tramp. The previously mentioned `nsm-settings-file` is referenced when an async update of ELPA package lists =E2=80=93 triggered i= n my init.el file =E2=80=93 is running, and desktop.el is busy reloading my buff= ers, including the ones via tramp. So, basically, the optimization would work great if only we never called any lisp code asynchronously to it =E2=80=93 but we do, and that is probabl= y made more visible when I'm using an ssh-backed method to talk to a machine ~ 250ms away from my Emacs instance than when it is ~ 25ms away. A bigger race, where TRAMP is waiting for data from an external process, and so other external processes can also fire. I don't have a patch, but I'll see if I can figure out how to improve this. I fear the situation is impossible, however, and that this attempt to improve the performance of `vc-registered` is doomed to failure unless an async protocol is defined to replace the current, synchronous, version. I also cannot say if the problem extends beyond `expand-file-name`, but it probably hits at minimum anything else that canonicalizes paths. --0000000000008e5d370585153ced Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
G'day.=C2= =A0 In HEAD, tramp-sh has some optimizations in place for the `vc-registere= d` operation, intended to minimize the number of round trips.=C2=A0 Sadly, = these don't seem to be safe in a single threaded Emacs world, and even = less safe in a threaded one.

The function `tramp-sh-hand= le-vc-registered` attempts to optimize the process by emplacing a separate = file-name-handler, running the operation(s), then performing a single remot= e call to obtain all the data.

This is an optimiza= tion, no doubt, and would be great ... except that it assumes a synchronous= operation where no other code can run while the new file handler is in eff= ect.=C2=A0 This is, sadly, not the case.

The obser= vable symptom is that we hit this error throw in `tramp-dissect-file-name`:=
(tramp-user-error nil "Not a Tramp file name: \"%s\&qu= ot;" name)

The most common place I observ= ed this was during startup, when it complained that the `nsm-settings-file`= , which is the default value in my case, was not a TRAMP filename.=C2=A0 It= also happens for expanding `~`, and a handful of other cases, when I was r= everting tramp buffers, etc.

On debugging I discov= ered that the problems were springing from calls something akin to this:
(let ((default-directory "/ssh:slippycheeze@example.com:/tmp&q= uot;))
=C2=A0 (expand-file-name "~" default-directory))=

That uses, via `tramp-vc-file-name-handler`, `tra= mp-file-name-for-operation`, which in the case of `expand-file-name` return= s "~".=C2=A0 I believe that is correct, but if not, that may be t= he root cause of the problem.

In any case, that is= then passed to `tramp-dissect-file-name` by `tramp-vc-file-name-handler`, = and (definitely correctly) triggers the error that a non-tramp filename is = being parsed.

This triggered from an async lisp ca= llback firing, while the current buffer happened to be on tramp.=C2=A0 The = previously mentioned `nsm-settings-file` is referenced when an async update= of ELPA package lists =E2=80=93 triggered in my init.el file =E2=80=93 is = running, and desktop.el is busy reloading my buffers, including the ones vi= a tramp.

So, basically, the optimization would wor= k great if only we never called any lisp code asynchronously to it =E2=80= =93 but we do, and that is probably made more visible when I'm using an= ssh-backed method to talk to a machine ~ 250ms away from my Emacs instance= than when it is ~ 25ms away.=C2=A0 A bigger race, where TRAMP is waiting f= or data from an external process, and so other external processes can also = fire.

I don't have a patch, but I'll see i= f I can figure out how to improve this.=C2=A0 I fear the situation is impos= sible, however, and that this attempt to improve the performance of `vc-reg= istered` is doomed to failure unless an async protocol is defined to replac= e the current, synchronous, version.

I also cannot= say if the problem extends beyond `expand-file-name`, but it probably hits= at minimum anything else that canonicalizes paths.
=
--0000000000008e5d370585153ced--