From: Michael Albinus <michael.albinus@gmx.de>
To: "João Távora" <joaotavora@gmail.com>
Cc: Thomas Koch <thomas@koch.ro>, 61350@debbugs.gnu.org
Subject: bug#61350: Eglot over Tramp freezes with large project
Date: Thu, 02 Mar 2023 12:01:32 +0100 [thread overview]
Message-ID: <87y1ofct83.fsf@gmx.de> (raw)
In-Reply-To: <87ilfkh89k.fsf@gmail.com> ("João Távora"'s message of "Wed, 01 Mar 2023 14:10:47 +0000")
João Távora <joaotavora@gmail.com> writes:
Hi João,
> But JUST-THIS-ONE _is_ relevant when there is more than one process.
> Here, there is. There's one process, the jsonrpc.el process, henceforth
> 'jprocess', and the Tramp process, henceforth 'tprocess'. jprocess
> receives only JSONRPC data from the LSP server. It "thinks" it is
> talking directly to a JSONRPC server, but in Tramp scenarios it is being
> fed data from tprocess, which is the process connected to the remote
> host. In tprocess, other things, such as shell interactions are going
> on.
>
> Michael can probably confirm, correct or deny this.
More or less correct. But I still can't say which process gets output
when, because I cannot debug accept-process-output (it's a C
function). And running Emacs under gdb changes timings, which is
important I believe.
> When one (accept-process-output tprocess nil nil 'JUST-THIS-ONE=t) one
> must be absolutely sure that tprocess is going to send _something_
> "soon". If it doesn't, we'll hang indefinitely (until the process dies
> or the user quits)
Yes. But Tramp calls accept-process-output only, if it has send a
command to the remote shell, and it expects something to be returned. At
least the shell prompt.
During my tests I have also changed this to
--8<---------------cut here---------------start------------->8---
(while (accept-process-output tprocess 0 nil 'JUST-THIS-ONE))
--8<---------------cut here---------------end--------------->8---
but it didn't help either.
> That's what has been confirmed through a backtrace. It's a particular
> accept-process-output call in tramp-wait-for-regexp that hangs,
> understandibly so.
Yes.
> Now, 'tramp-check-for-regexp' uses a somewhat non-standard technique of
> searching for messages: it searches them from the back, from the end of
> the tprocess's buffer. I don't know what motivated this, but I find it
> odd. I find one of its callees, tramp-search-regexp, particularly
> suspicious:
>
> (defun tramp-search-regexp (regexp)
> "Search for REGEXP backwards, starting at point-max.""
> (goto-char (point-max))
> ;; We restrict ourselves to the last 256 characters. There were
> ;; reports of a shell command "git ls-files -zco --exclude-standard"
> ;; with 85k files involved, which has blocked Tramp forever.
> (re-search-backward regexp (max (point-min) (- (point) 256)) 'noerror))
>
> See the comment there? Only 256 characters back are inspected.
Yes. But the regexp it searches for is the final shell prompt. Something
like "///4b3b7d4fa561141e84c94a1cf25e8575#$", which is shorter than 256
bytes for sure.
> So, finally, here's my conjecture:
>
> 1. Tramp goes into 'tramp-wait-for-regexp'. tprocess's buffer already
> the message that 'found' is supposed to return, but it also has a lot
> more stuff, say a lot of JSONRPC data from the LSP server that also
> came into that tprocess buffer and is awaiting to be delivered to
> jprocess.
>
> 2. This data is for piping into jprocess, where the JSONRPC message will
> be decoded, but it will probably never arrive at its destination.
>
> 3. 'found' will be nil in tramp-wait-for-regexp, because of the
> tramp-search-regexp limitation.
>
> 4. tramp-wait-for-regexp will issue the "risky" accept-process-output
> call.
>
> 5. there is no more data that accept-process-output wants to put in the
> buffer, because the LSP server is fine for the moment.
>
> 6. Emacs hang
>
> Just a conjecture.
Yes, this is more or less the scenario. But I still don't understand why
not all data are delivered through the socket ssh is using. Could it be
there is a limitation, how much data could be buffered by ssh?
>> I have a vague feeling, that Tramp could be improved with a work queue
>> such that requests to tramp from notification or timer threads get
>> blocked while another tramp command is still waiting for a
>> reply.
>
> There are no (usable) threads in Emacs.
There are. I made Tramp using threads, and it worked fine, when no
interactive dialogue inside a thread happened.
> Timers are events, and so are runs of each processe's process filter.
> Those two are what creates asynchronicity and the emulation of
> simultaneity in Emacs. When jprocess's filter sees a whole JSONRPC
> message, it calls the message handler.
Timers and process filters are the cause of the "Forbidden reentrant
call in Tramp" errors. Wwe must do anything, solving this.
> João
Best regards, Michael.
next prev parent reply other threads:[~2023-03-02 11:01 UTC|newest]
Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-07 16:33 bug#61350: Eglot over Tramp freezes with large project Thomas Koch
2023-02-17 9:54 ` Michael Albinus
2023-02-17 10:33 ` Thomas Koch
2023-02-18 11:10 ` Thomas Koch
2023-02-18 12:07 ` Michael Albinus
2023-02-23 11:55 ` Thomas Koch
2023-02-25 14:36 ` Michael Albinus
2023-02-23 12:17 ` João Távora
2023-02-23 14:18 ` João Távora
2023-02-23 14:47 ` Thomas Koch
2023-02-23 15:22 ` João Távora
2023-02-24 17:19 ` Michael Albinus
2023-02-24 17:45 ` João Távora
2023-02-25 14:27 ` Michael Albinus
2023-02-25 23:09 ` João Távora
2023-02-26 10:24 ` Thomas Koch
2023-02-26 15:58 ` Michael Albinus
2023-02-26 17:23 ` Michael Albinus
2023-02-26 21:13 ` João Távora
2023-02-26 21:45 ` João Távora
2023-02-27 7:53 ` Michael Albinus
2023-02-27 9:42 ` João Távora
2023-02-27 20:11 ` Michael Albinus
2023-02-27 7:47 ` Michael Albinus
2023-02-27 9:35 ` João Távora
2023-02-27 20:10 ` Michael Albinus
2023-02-28 0:10 ` João Távora
2023-02-28 10:38 ` Michael Albinus
2023-02-28 11:33 ` João Távora
2023-02-28 12:59 ` Michael Albinus
2023-02-28 14:41 ` João Távora
2023-02-28 14:18 ` Michael Albinus
2023-02-28 14:51 ` João Távora
2023-02-28 15:01 ` Michael Albinus
2023-02-28 17:55 ` Thomas Koch
2023-03-01 14:10 ` João Távora
2023-03-01 16:19 ` João Távora
2023-03-02 11:01 ` Michael Albinus [this message]
2023-03-02 11:22 ` João Távora
2023-03-02 11:50 ` Michael Albinus
2023-03-05 11:21 ` Michael Albinus
2023-03-05 11:45 ` Thomas Koch
2023-03-05 12:23 ` Michael Albinus
2023-03-07 12:49 ` Michael Albinus
2023-03-07 13:04 ` Thomas Koch
2023-03-07 13:33 ` João Távora
2023-03-07 13:52 ` Michael Albinus
2023-03-07 14:03 ` João Távora
2023-03-07 14:31 ` Michael Albinus
2023-03-11 9:00 ` Michael Albinus
2023-03-11 10:14 ` Thomas Koch
2023-03-11 11:47 ` João Távora
2023-03-11 12:27 ` Michael Albinus
2023-03-11 11:42 ` João Távora
2023-03-11 12:44 ` Michael Albinus
2023-03-11 14:01 ` João Távora
2023-03-11 14:25 ` Michael Albinus
2023-03-12 0:48 ` João Távora
2023-03-12 10:22 ` Michael Albinus
2023-03-14 11:01 ` Michael Albinus
2023-03-14 15:00 ` Michael Albinus
2023-03-14 15:19 ` João Távora
2023-03-14 15:42 ` Michael Albinus
2023-03-15 17:47 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2023-03-15 18:05 ` João Távora
2023-03-15 18:30 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2023-03-15 19:44 ` João Távora
2023-03-15 20:14 ` João Távora
2023-03-15 21:34 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2023-03-15 21:55 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2023-03-16 13:28 ` João Távora
2023-03-18 12:34 ` Michael Albinus
2023-03-15 21:43 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2023-03-15 21:49 ` João Távora
2023-03-16 6:24 ` Jim Porter
2023-03-16 13:25 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2023-03-16 13:28 ` João Távora
2023-03-16 15:58 ` João Távora
2023-03-16 20:36 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2023-03-16 22:04 ` João Távora
2023-03-07 13:47 ` Michael Albinus
2023-03-06 12:42 ` Michael Albinus
2023-03-06 13:45 ` João Távora
2023-03-06 13:42 ` João Távora
2023-03-02 10:40 ` Michael Albinus
2023-02-28 19:37 ` João Távora
2023-03-01 8:44 ` Michael Albinus
2023-03-01 11:15 ` João Távora
2023-03-01 10:46 ` Gregory Heytings
2023-03-01 11:08 ` João Távora
2023-03-01 11:23 ` Gregory Heytings
2023-03-01 11:37 ` João Távora
2023-03-01 14:51 ` Michael Albinus
2023-03-01 15:02 ` Gregory Heytings
2023-04-24 1:44 ` Aaron Madlon-Kay
2023-05-05 11:32 ` Michael Albinus
2023-05-05 13:14 ` Ruijie Yu via Bug reports for GNU Emacs, the Swiss army knife of text editors
2023-05-05 14:53 ` Michael Albinus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87y1ofct83.fsf@gmx.de \
--to=michael.albinus@gmx.de \
--cc=61350@debbugs.gnu.org \
--cc=joaotavora@gmail.com \
--cc=thomas@koch.ro \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).