From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Michael Albinus Newsgroups: gmane.emacs.devel Subject: Re: Introducing thread-safe Tramp Date: Tue, 24 Jul 2018 14:52:03 +0200 Message-ID: <87wotkn6do.fsf@gmx.de> References: <8736wa9c5s.fsf@gmx.de> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1532436618 14089 195.159.176.226 (24 Jul 2018 12:50:18 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 24 Jul 2018 12:50:18 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: Dmitry Gutov , emacs-devel@gnu.org To: Ken Raeburn Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Jul 24 14:50:13 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fhwlZ-0003ZL-4i for ged-emacs-devel@m.gmane.org; Tue, 24 Jul 2018 14:50:13 +0200 Original-Received: from localhost ([::1]:40370 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fhwnf-0002e8-M5 for ged-emacs-devel@m.gmane.org; Tue, 24 Jul 2018 08:52:23 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48603) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fhwnW-0002dq-Fi for emacs-devel@gnu.org; Tue, 24 Jul 2018 08:52:15 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fhwnQ-00069M-Vc for emacs-devel@gnu.org; Tue, 24 Jul 2018 08:52:14 -0400 Original-Received: from mout.gmx.net ([212.227.15.18]:33903) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fhwnQ-000689-LE for emacs-devel@gnu.org; Tue, 24 Jul 2018 08:52:08 -0400 Original-Received: from detlef.gmx.de ([178.20.95.13]) by mail.gmx.com (mrgmx003 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MYx2x-1fTyfW1hml-00VjFK; Tue, 24 Jul 2018 14:52:05 +0200 In-Reply-To: (Ken Raeburn's message of "Tue, 24 Jul 2018 04:25:12 -0400") X-Provags-ID: V03:K1:2lCSLKzmh3d4cnN4CyRnJSuiB3f0Geu9PxK+XtgwMjDRCkN1rdT nW5U9zT1WW5gRDmp5bz9wm8Hko46cWip67lpNzgyC7SPD7PafFYmpVIWlrVVNC4cnR+y25F jtopUKsHSxP+1fE1bnn6ShT+2kmMoi1OFBsN3k4O65VO0J/JDa9qXWPgDBZZZABcqTnawIS sUyptfllu70GX7uYlUxNQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:hV6zE4VR47s=:XEC//oz87vdFJ7B2ZCUx9f h4QOjW4lHG9UOEE4/vgiwNBvUL3BJNWAfOOBz86dQAkG66aZTMPgErU4aD4ixTfR0WNjkjKhA Y1KmsSyvI2Ld6CQdiTG8z+YN0LMbVBoC9bg2uMfmQ6xq8vGXZL42s+xuZDdd37LiSaHaflw4i U8eG3fhgyqQpyP/Zs3HPZGGNdEkdgkgyA8iBga2UR09wVta14lqFwYb0x/GFvHeADmaO8EK5x BDRLQURB2HdeWQOBnFrNuO7Cvr4+gw9mn31OpYK+gYd3n2NIr5PfszRfiSCmWPq/ktKZYYHRf VcNmmLWqcZiroGqTtBj4NI0V/jixPRrhcaCLBcY18NRkYebgIwOg0hVhzaH6zpdVIxF9fDVDV W6OKeJWOfsrrIqUaQb5T/RUJEOTSR+mMMKJAJfhqwT/cFdaGQCgj51h4ohWGv0uoNEY2ml5vf 7YEEZ8UOq9Trys8uT1zmk4PxRZ9afW8foHR0UkMBnT2T8sbeV+DmMQUfbkibv5cxCXaXrieMD GIhxtYc5RMSf3UEjEIQNJe1iZVJXp33MUpxAY7tN4++un5kdu/6CXuMkaQwJWJnmF3Y1e5Cil SlG0Y6bQoG7xdVGd7XkanVhtJlX5GQL9VWu+IpfLxsJR6GhgyxPMn55Q1u5dNdBk3WMvpM/Oc aeiYcC6FTZwkoch6tusxat4ia4nt5rD0LKmGMj4nfW0ZKL6I3WhQUB4NnXYKQ25MVnq/RdaIr uJv4U1eIqefqGgfdiD3Q3M+k2WJrJ36GbNZHiscbQKjcwLoSYvwGx1MiUvs/D8s6w+EDcouZ X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 212.227.15.18 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:227761 Archived-At: Ken Raeburn writes: Hi Ken, > The changes you describe to the find-file interfaces relate to another > idea I=E2=80=99ve been thinking of for a while, though I wanted to wait u= ntil > I had time to look into it a bit more myself. Namely, make some of the > C-level local file system interactions release the global lock, and > use additional threads for invoking some of those operations. Yes. But it might be better to use an own mutex for that. In Tramp, I have introduced one mutex per connection. Whenever any Tramp file name operation is called, it goes via `tramp-file-name-handler'. Here I determine the responsible mutex (according to the connection characteristics), and I lock this mutex. Once the operation has finished, at the end of `tramp-file-name-handler', the mutex is unlocked. (To be honest, I simply use `with-mutex'.) By this, only one file operation per connection can run at a given time. This is still better than w/o threads, because if you access concurrently "/ssh:..." and "/sudo:...", the operations could run in parallel. And, due to the threads created by find-file, Tramp operations do not block global editing functions. Of course, this is a coarse approach. On my todo list is a change, to apply conection-oriented mutexes not on the file operations level, but on process related level (sending commands remotely, and retrieve the output). By this, more concurrency shall be possible, with (hopefully) better performance. Similar to what you have in mind. > So, for example, while loading a large file from a slow NFS server, we > can also be processing subprocess output, running garbage collection, > etc. If the NFS server hangs, ^G should interrupt the main UI thread > that=E2=80=99s sitting waiting for a notification from the file I/O thread > (which is hung until the server comes back or the access times out; > however, other file I/O threads could be used to access other files or > file systems), break it out of the wait, and let the user go do > something else. In auto-save-mode or auto-revert-mode, read/write/stat > calls shouldn=E2=80=99t cause delays in the UI just because a file server= is > slow. Same would be true for Tramp then. > There are plenty of details I haven=E2=80=99t worked out, like what to do > about these background threads acting on a buffer the user is actively > modifying at the same time. But one thing that had occurred to me was > that as far as the user interaction is concerned, a multithreaded > Tramp would be just the way to try out some of these sorts of changes. Yep. Currently I'm sitting on the problem of handling the minibuffer properly, when several find-file operations are in progress in parallel. Something like confirmation of risky local variables in a file, and alike. Another problem will be cached values. Tramp caches a lot of file information, in order to reduce roundtrips. Those cached values might become invalid due to operations in another thread. So there's still a lot to do. I hope to get much feedback from you and other people. > Ken Best regards, Michael.