From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Paul Pogonyshev Newsgroups: gmane.emacs.bugs Subject: bug#56342: TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks Date: Sun, 3 Jul 2022 21:52:55 +0200 Message-ID: References: <8735fjh5ge.fsf@gmx.de> <87v8sefl2f.fsf@gmx.de> <874jzyc9u9.fsf@gmx.de> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000d9855f05e2ebf70a" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1715"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 56342@debbugs.gnu.org To: Michael Albinus Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Jul 03 21:54:20 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o85ff-0000Ej-Bm for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 03 Jul 2022 21:54:19 +0200 Original-Received: from localhost ([::1]:45080 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o85fe-0007yH-8q for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 03 Jul 2022 15:54:18 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:43062) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o85fP-0007y9-8R for bug-gnu-emacs@gnu.org; Sun, 03 Jul 2022 15:54:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:51624) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o85fN-0002jC-P4 for bug-gnu-emacs@gnu.org; Sun, 03 Jul 2022 15:54:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1o85fN-0003Rx-Lu for bug-gnu-emacs@gnu.org; Sun, 03 Jul 2022 15:54:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Paul Pogonyshev Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 03 Jul 2022 19:54:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56342 X-GNU-PR-Package: emacs Original-Received: via spool by 56342-submit@debbugs.gnu.org id=B56342.165687799513208 (code B ref 56342); Sun, 03 Jul 2022 19:54:01 +0000 Original-Received: (at 56342) by debbugs.gnu.org; 3 Jul 2022 19:53:15 +0000 Original-Received: from localhost ([127.0.0.1]:45521 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o85ec-0003Qy-BE for submit@debbugs.gnu.org; Sun, 03 Jul 2022 15:53:14 -0400 Original-Received: from mail-yb1-f175.google.com ([209.85.219.175]:40624) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o85ea-0003Ql-S9 for 56342@debbugs.gnu.org; Sun, 03 Jul 2022 15:53:13 -0400 Original-Received: by mail-yb1-f175.google.com with SMTP id o2so8160367yba.7 for <56342@debbugs.gnu.org>; Sun, 03 Jul 2022 12:53:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Y6nTzgBZF6uyYLCi3FOGFGUC5Pr8bjYuApmIWbAbww8=; b=OIoVpAfy4ZXc+2nmPaj7dKMAvXsYJzgeGuUmyAC6uCuE1QO4F+hm4VOsGNEyT0f3Pt WntbOExra7rBz+qP0fzNKeIQZIxT9ceIv3Ig18zOzBa2NRlm/TvFZsi8rcisJ2C5KF69 YJG+82MBKdwCkR1oxAZAKz4tm2hY8n7fndRzZb6cyuM/+i0ZUvQTdCJX6rTphPgoSOL0 etzQ+hwQUIiODi5DD9W5B79EaN7zMAuuShcbuKesowC3TCZIuYAW2eXqp24jT7rNCaM3 KEMkSgB4LAuxhVhd9YFr8afuhEKWugMiBacGW6AzgiRIRiJNmOo4R7g0uuvkbA+434o9 CKvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Y6nTzgBZF6uyYLCi3FOGFGUC5Pr8bjYuApmIWbAbww8=; b=lgNsrIL2slheyDyWmt9HxfWcyCax/ZC0eqrcvDYjnLKzciypfDLSXkwqqdHMKn+RNz HvChPcyVIvRNH8KI2zrHhbKd1Ux+G0lIs6qrNYp3S1Z79YIcCih4LvwdVDaygVgbBRQ5 Li51sMDiGDzJuDYXEM+q60wcgU3jr7gaTqgWIhtWFZ+gmHWO9RaZw/bNrK2d768gdaHI 75FKSwKwFsPDB0J4SHw8Ksx3+Lj/bIPc3rsz3ou3nGNlmUHDK/Ocwu7BMDJjTW2Z9cW5 9RF4xmlZgsyUrZc/opbiz6bp4yrCwGTmTlA+grP2iCRa7OMNXzqxgYDBlneOUa7f+C15 ZOAQ== X-Gm-Message-State: AJIora+ZCdaEK4HZVM+1fg5+emjuSbM/VjU5+/mHOxqKTEHAOE2adYEA /AzHOwXgBghOg8k+wz5f40YL8BT7JEjg2NluIw== X-Google-Smtp-Source: AGRyM1sZlDcIci6yR9h2PtXpySS3lMgI+4YA2JrY0h4X8C667yrvwSp/Hsz0DZdhIHojNdAfuSWe8a1lbtopvVx5QfU= X-Received: by 2002:a25:6ed5:0:b0:669:8b84:bb57 with SMTP id j204-20020a256ed5000000b006698b84bb57mr26798090ybc.227.1656877986971; Sun, 03 Jul 2022 12:53:06 -0700 (PDT) In-Reply-To: <874jzyc9u9.fsf@gmx.de> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:236022 Archived-At: --000000000000d9855f05e2ebf70a Content-Type: text/plain; charset="UTF-8" > I will see whether I can do something along these lines. Thank you. I'm just spewing out ideas here, you know the code better and what will and will not work. > Perhaps changing the order: First try > to insert the file contents, and if this errs out, a call with > file-exists-p in order to raise the error. I'll play with this. Yes, this would be a better way from my point of view. I don't care how slowly (unless it is like 50 times slower) it fails, that is an unlikely case. The most likely one, when the file exists, should be the benchmark. > Doing it asynchronously would require a second connection to the remote > host. Performance would rather degrade. Maybe not really asynchronously, just let it return early, not waiting for the result? I'm not sure how TRAMP receives responses, but can it just keep executing commands sequentially, as now, but give control back to the caller in case of commands where the result doesn't really matter (cleanup, e.g. deleting a temporary file). Of course, this means that if an "important" command is issued right away, it has to wait for the response to the previous inessential command. But when such an inessential command is the last in a batch, this waiting would be effectively merged with Emacs' idling in the normal UI command loop, making things more responsive for the user. > One thing you could do w/o code change is to change the value of > remote-file-name-inhibit-cache. Thank you for the hint, will try. Paul On Sun, 3 Jul 2022 at 20:47, Michael Albinus wrote: > Paul Pogonyshev writes: > > Hi Paul, > > > Maybe you could then single out commands that are supposed to return > > result nearly-instantly anyway, e.g. `stat' or `test' (though the > > latter I have suggested to replace with `stat' before). If they fail > > to produce the result in 10 seconds, the connection can be assumed > > dead, just as with `echo are you awake'. Commands that involve > > transmitting (potentially) large amount of data this or that way can > > work as now. So, you would usually avoid one extra command (which with > > a high ping means sth. like 0.2 s, already pretty noticeable in UI) > > and achieve the same result. > > > > Sanity checks are good. but not if they visibly slow down normal > > operation. > > I will see whether I can do something along these lines. > > > It might be easier to convince the rest of Emacs developers to use an > > extra variable to possibly skip `file-exists-p' on only remote files > > than to remove the call altogether. Though in either case it is > > required that `file-missing' error is handled properly which is, > > incidentally, easier to test if the call to `file-exists-p' is > > dropped. A target here would be `insert-file-contents' and whatever it > > calls, and this is a part of Emacs, not some external library. > > Tramp has an own implementation of insert-file-contents, called > tramp-handle-insert-file-contents. And indeed, file-exists-p is called > in order to raise the file-missing error if needed. I have no idea how > we could generate this otherwise. Perhaps changing the order: First try > to insert the file contents, and if this errs out, a call with > file-exists-p in order to raise the error. I'll play with this. > > OTOH, with longer cache expiration time (see below), file-exists-p > doesn't cost a roundtrip. > > > Yet another idea: removing temporary should be done asynchronously (I > > haven't checked, maybe it's already the case, but likely not). The > > caller doesn't really care about call result and even if it has > > succeeded. Again, not sure if this is easy to achieve interface-wise, > > maybe it's from a higher level. But I suspect it's something like > > `with-temp-file' and can be optimized: one-time optimization are worth > > it even if that costs readability. > > Doing it asynchronously would require a second connection to the remote > host. Performance would rather degrade. > > One thing you could do w/o code change is to change the value of > remote-file-name-inhibit-cache. Its default value is 10, meaning caches > expire already after 10 seconds. With your slow connection, a higher > value should help already. > > > Paul > > Best regards, Michael. > --000000000000d9855f05e2ebf70a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
> I will see whether I can do something along these lin= es.

Thank you. I'm just spewing out ideas here, = you know the code better and what will and will not work.

> Perhaps changing the order: First try
> to insert the = file contents, and if this errs out, a call with
> file-exists-p in o= rder to raise the error. I'll play with this.

Yes, t= his would be a better way from my point of view. I don't care how slowl= y (unless it is like 50 times slower) it fails, that is an unlikely case. T= he most likely one, when the file exists, should be the benchmark.

> Doing it asynchronously would require a second connec= tion to the remote
> host. Performance would rather degrade.

Maybe not really asynchronously, just let it return early,= not waiting for the result? I'm not sure how TRAMP receives responses,= but can it just keep executing commands sequentially, as now, but give con= trol back to the caller in case of commands where the result doesn't re= ally matter (cleanup, e.g. deleting a temporary file). Of course, this mean= s that if an "important" command is issued right away, it has to = wait for the response to the previous inessential command. But when such an= inessential command is the last in a batch, this waiting would be effectiv= ely merged with Emacs' idling in the normal UI command loop, making thi= ngs more responsive for the user.

> One thing you could do w/o co= de change is to change the value of
> remote-file-name-inhibit-cache.=

Thank you for the hint, will try.

<= /div>
Paul


On Sun, 3 Jul 2022 at 20:47, Michael= Albinus <michael.albinus@gmx.= de> wrote:
pogonyshev@gmail.com> writes:

Hi Paul,

> Maybe you could then single out commands that are supposed to return > result nearly-instantly anyway, e.g. `stat' or `test' (though = the
> latter I have suggested to replace with `stat' before). If they fa= il
> to produce the result in 10 seconds, the connection can be assumed
> dead, just as with `echo are you awake'. Commands that involve
> transmitting (potentially) large amount of data this or that way can > work as now. So, you would usually avoid one extra command (which with=
> a high ping means sth. like 0.2 s, already pretty noticeable in UI) > and achieve the same result.
>
> Sanity checks are good. but not if they visibly slow down normal
> operation.

I will see whether I can do something along these lines.

> It might be easier to convince the rest of Emacs developers to use an<= br> > extra variable to possibly skip `file-exists-p' on only remote fil= es
> than to remove the call altogether. Though in either case it is
> required that `file-missing' error is handled properly which is, > incidentally, easier to test if the call to `file-exists-p' is
> dropped. A target here would be `insert-file-contents' and whateve= r it
> calls, and this is a part of Emacs, not some external library.

Tramp has an own implementation of insert-file-contents, called
tramp-handle-insert-file-contents. And indeed, file-exists-p is called
in order to raise the file-missing error if needed. I have no idea how
we could generate this otherwise. Perhaps changing the order: First try
to insert the file contents, and if this errs out, a call with
file-exists-p in order to raise the error. I'll play with this.

OTOH, with longer cache expiration time (see below), file-exists-p
doesn't cost a roundtrip.

> Yet another idea: removing temporary should be done asynchronously (I<= br> > haven't checked, maybe it's already the case, but likely not).= The
> caller doesn't really care about call result and even if it has > succeeded. Again, not sure if this is easy to achieve interface-wise,<= br> > maybe it's from a higher level. But I suspect it's something l= ike
> `with-temp-file' and can be optimized: one-time optimization are w= orth
> it even if that costs readability.

Doing it asynchronously would require a second connection to the remote
host. Performance would rather degrade.

One thing you could do w/o code change is to change the value of
remote-file-name-inhibit-cache. Its default value is 10, meaning caches
expire already after 10 seconds. With your slow connection, a higher
value should help already.

> Paul

Best regards, Michael.
--000000000000d9855f05e2ebf70a--