all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Remote asynchronous processes
@ 2020-04-13 10:19 Michael Albinus
  2020-04-13 20:32 ` Philippe Vaucher
                   ` (2 more replies)
  0 siblings, 3 replies; 27+ messages in thread
From: Michael Albinus @ 2020-04-13 10:19 UTC (permalink / raw)
  To: Philipp Stephani; +Cc: emacs-devel

Hi,

On the Tramp ML, there is a discussion about performance of remote
asynchronous processes. start-file-process / make-process take too much
time to finish.

One of the reasons is, that Tramp opens first a shell on the remote
host, performs sanity checks, and runs the command after that. Well, I
cannot change this in general; the sanity checks have been added due to
feedback from users.

One idea to change the situation is, to remove all sanity checks from
make-process. That is, if a user has a default directory
"/ssh:user@host:/path/to/dir", and if he calls

--8<---------------cut here---------------start------------->8---
(make-process
 :name "test"
 :buffer (current-buffer)
 :command '("cmd")
 :file-handler t))
--8<---------------cut here---------------end--------------->8---

this is translated directly into

--8<---------------cut here---------------start------------->8---
ssh -l user -o ControlMaster=auto -o ControlPath='tramp.%C' \
  -o ControlPersist=no host "cd /path/to/dir; cmd"
--8<---------------cut here---------------end--------------->8---

This would improve performance significantly. The drawback is, that
Tramp does not perform convenience checks, like password handling.

start-file-process would not be changed, and it behaves like before.

Comments?

Best regards, Michael.



^ permalink raw reply	[flat|nested] 27+ messages in thread
* Re: Remote asynchronous processes
@ 2020-07-29 16:58 Felipe Lema
  2020-07-31 10:22 ` Michael Albinus
  0 siblings, 1 reply; 27+ messages in thread
From: Felipe Lema @ 2020-07-29 16:58 UTC (permalink / raw)
  To: michael.albinus, emacs-devel

Hello, there (I hope I'm replying correctly)

Answering as I'm interested in this matter, although I'm not a formal
Emacs maintainer myself.

I've tried this patch for async processes and, given that I deal with
far away servers that have a significant delay in answering queries, I
find this most useful. It cuts starting processes from about 5s to less
than 1s.

While this looks promising, I was also wondering about an alternative
solution.

I understand that users have requested dealing with very specific
setups in remote servers and that currently tramp-sh deals with them:
It sends some commands to the server and adjusts the current "command
to be sent".

Instead of sending-and-waiting-for commands, can't these results be
stored in memory? That way tramp-sh can construct the "command to be
sent" /without/ doing queries. If necessary, the results stored in
memory can be updated frequently in the background. Maybe they can even
be persisted for future use (using the `persist` package).

If this is not feasible, then I can manage with this "non-checks-just-
run-this-command-as-is" approach.

Gards

Michael Albinus <address@hidden> wrote:

>Hi,
>
>> On the Tramp ML, there is a discussion about performance of remote
>> asynchronous processes. start-file-process / make-process take too
much
>> time to finish.
>>
>> One of the reasons is, that Tramp opens first a shell on the remote
>> host, performs sanity checks, and runs the command after that. Well,
I
>> cannot change this in general; the sanity checks have been added due
to
>> feedback from users.
>>
>> One idea to change the situation is, to remove all sanity checks
from
>> make-process. That is, if a user has a default directory
>> "/ssh:user@host:/path/to/dir", and if he calls
>>
>> (make-process
>>  :name "test"
>>  :buffer (current-buffer)
>>  :command '("cmd")
>>  :file-handler t))
>>
>> this is translated directly into
>>
>> ssh -l user -o ControlMaster=auto -o ControlPath='tramp.%C' \
>>   -o ControlPersist=no host "cd /path/to/dir; cmd"
>>
>> This would improve performance significantly. The drawback is, that
>> Tramp does not perform convenience checks, like password handling.
>
>I have played with this idea, and the output is the appended file
>tramp-make-process.el. It changes the make-process implementation of
>Tramp for all methods defined in tramp-sh.el (like "ssh") and
>tramp-adb.el.
>
>This is a proof-of-concept, and shouldn't be used for production. Read
>the commentary in the file for limitations.
>
>However, the speed optimization is remarkable. I've tested it with the
>following code snippet:
>
>--8<---------------cut here---------------start------------->8---
>(let ((tramp-verbose 0)
>      (default-directory "/ssh::/"))
>  ;; Fill the caches.
>  (start-file-process "" nil "true")
>  ;; Run benchmark.
>  (benchmark-run 10
>    (start-file-process "" nil "true")))
>--8<---------------cut here---------------end--------------->8---
>
>In the default case, the result is
>
>--8<---------------cut here---------------start------------->8---
>(3.623666842 80 1.183512312)
>--8<---------------cut here---------------end--------------->8---
>
>If tramp-make-process.el is loaded, the result is
>
>--8<---------------cut here---------------start------------->8---
>(0.022762177 0 0.0)
>--8<---------------cut here---------------end--------------->8---
>
>Similar results, if I use "/adb::/" as default directory:
>
>--8<---------------cut here---------------start------------->8---
>(4.599374061 2 0.03429497299999973)
>--8<---------------cut here---------------end--------------->8---
>
>vs
>
>--8<---------------cut here---------------start------------->8---
>(0.013003183 0 0.0)
>--8<---------------cut here---------------end--------------->8---
>
>Comments?
>
>Best regards, Michael.
>




^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2020-08-21 22:28 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-04-13 10:19 Remote asynchronous processes Michael Albinus
2020-04-13 20:32 ` Philippe Vaucher
2020-04-14  9:03   ` Michael Albinus
2020-04-14 12:34     ` Philippe Vaucher
2020-04-14 14:28       ` Michael Albinus
2020-08-04 17:00       ` Philipp Stephani
2020-04-14 14:48     ` Stefan Monnier
2020-04-14 15:30       ` Michael Albinus
2020-08-04 17:01       ` Philipp Stephani
2020-08-04 16:56     ` Philipp Stephani
2020-08-04 17:35       ` Michael Albinus
2020-08-10 14:42         ` Philipp Stephani
2020-05-06 11:59 ` Michael Albinus
2020-08-04 16:48 ` Philipp Stephani
2020-08-04 17:31   ` Michael Albinus
2020-08-10 14:56     ` Philipp Stephani
  -- strict thread matches above, loose matches on Subject: below --
2020-07-29 16:58 Felipe Lema
2020-07-31 10:22 ` Michael Albinus
2020-08-04 12:27   ` Michael Albinus
2020-08-04 16:06     ` Felipe Lema
2020-08-06 19:08     ` Sean Whitton
2020-08-09  7:22       ` Michael Albinus
2020-08-09 14:47         ` Philipp Stephani
2020-08-09 17:06           ` Michael Albinus
2020-08-12 10:46         ` Michael Albinus
2020-08-21 22:28           ` Sean Whitton
2020-08-07 16:28   ` Philipp Stephani

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.