From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Michael Albinus Newsgroups: gmane.emacs.bugs Subject: bug#61350: Eglot over Tramp freezes with large project Date: Thu, 02 Mar 2023 11:40:33 +0100 Message-ID: <87356ne8ri.fsf@gmx.de> References: <87y1ootw2t.fsf@gmail.com> <69968923.705640.1677163650760@office.mailbox.org> <87a613f0b7.fsf@gmx.de> <87r0udvmzr.fsf@gmx.de> <878rglxrzm.fsf@gmail.com> <87cz5wmjbx.fsf@gmx.de> <87h6v8f7u9.fsf@gmail.com> <87o7pflfcd.fsf@gmx.de> <87wn43e9ht.fsf@gmail.com> <874jr6oont.fsf@gmx.de> <87sfeqd4zi.fsf@gmail.com> <877cw1swjm.fsf@gmx.de> <87k0016dgo.fsf@gmx.de> <1458446553.50372.1677606917251@office.mailbox.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5945"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= , 61350@debbugs.gnu.org To: Thomas Koch Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Mar 02 11:41:15 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pXgN9-0001OR-MW for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 02 Mar 2023 11:41:15 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pXgMy-0005PY-5W; Thu, 02 Mar 2023 05:41:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pXgMw-0005PP-Hs for bug-gnu-emacs@gnu.org; Thu, 02 Mar 2023 05:41:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pXgMw-0002yS-9e for bug-gnu-emacs@gnu.org; Thu, 02 Mar 2023 05:41:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pXgMv-000163-O4 for bug-gnu-emacs@gnu.org; Thu, 02 Mar 2023 05:41:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Michael Albinus Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 02 Mar 2023 10:41:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61350 X-GNU-PR-Package: emacs Original-Received: via spool by 61350-submit@debbugs.gnu.org id=B61350.16777536464185 (code B ref 61350); Thu, 02 Mar 2023 10:41:01 +0000 Original-Received: (at 61350) by debbugs.gnu.org; 2 Mar 2023 10:40:46 +0000 Original-Received: from localhost ([127.0.0.1]:55898 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pXgMg-00015Q-7A for submit@debbugs.gnu.org; Thu, 02 Mar 2023 05:40:46 -0500 Original-Received: from mout.gmx.net ([212.227.15.19]:45815) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pXgMd-00015B-Hw for 61350@debbugs.gnu.org; Thu, 02 Mar 2023 05:40:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1677753637; i=michael.albinus@gmx.de; bh=Qf8z0U/CHC5mtiR/Y365DA3xx+BdfzUlt0jdpI5e0YA=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=OkaCBXyl6iat9KSdkEMs6jblBvrMc/eABA796UCgZvSXl0ezeyUWpWgPja9EO2/8m 5loplu5iPe8yLibwzPiktbygL1903gs0eixrDz3FnSQV3Em4z8No05HjP8CSP2OxYD l2nJTGqYErI487jyqTxCoIqzOnG7Tf+9frcBWGMfN0TOJrJDhMWB++ZSYyqtTLbYy3 afhGmA1AG+VbASGrC215HxXG3Xg5tiCFoj3ijcFXE7ypTKv0+47x6fSVJw18Z2HVkS AoMH92XMyTFBD1A2sTTgdvxC2Vwy9ca8xzN591NxXQd01Sm+jzCDpYa/NNb5uapo6l mrIO6r39ecLyQ== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Original-Received: from gandalf.gmx.de ([185.89.39.22]) by mail.gmx.net (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1M8ygO-1pdiF83vHi-0065eh; Thu, 02 Mar 2023 11:40:37 +0100 In-Reply-To: <1458446553.50372.1677606917251@office.mailbox.org> (Thomas Koch's message of "Tue, 28 Feb 2023 19:55:17 +0200 (EET)") X-Provags-ID: V03:K1:RnRk4JvvW3UtPghQYmGHb73tN+ZfYXtAFlGLPJaVJB/u2goxIe9 wb3UFHGa57jIN7JLCYcpU7ny6l2IvSKWwRwacxi6SVqbcrYE2/Pn2ALzIycTgojiteVLZa/ 8X1YJxqA/iY0ThOavD2ub6+rbycqqIt6jGVvh/l0GfuCV+vCecpSeNtYem7lnVlOIJQLZvz gK5ta+EPvr6uId0wQDz1A== UI-OutboundReport: notjunk:1;M01:P0:mbqvDP5uL6c=;bNhf5GSuNmqBRc9PX7fHkwAGARq 9M5o3k6ZsXOjfUlEYYBmNBdPZV+uY3H3Qdsfh6vfPbhKWfwCKaPPJEjpaGdgtJYq7LlTlrBvl nqJC40VxbPT8ze2jah0TMINotQ8liK0Id43YeeYhKWHYDy0mfs1ix5P1ozeHDHnJDvvlrDT+N eSpCLGukBzHR7EcVcH7Xu10+k9M8+d7LXO0DWUdyGToIB3IaW026rhmP0kZEqz2qoHhXWKz3B aHzJCTfAo2wcv8pAoxf2Tuys2ouXuj0TuTI7ckDPAWDzASLJYSPc9k3kSZL3VdnwsSNX35k/R R2zFxsD3zsmMoLrAXE/5QrUrX5dCW+/LXIJtv00eFsokMX/dmCgYE24XHLVqvdo6JqBgZYoFW 20GbUbz9HHAJ5TsICCCGoSKpyt5g4CbHPmbZ9pZwBFIn7BoUVPR2h93EFzDAi8A9x1nMaPOLs C2y8j8/A9nxLQSfRuVHqXT+73rDmfJTcmoosNNv5LKHmIJja5VMxzmfYs8rhz8ayUBnOc/fGx fJ0zx3hRpcWFJHZANhFWET1RbR0cSD4wuHQxQXCNOaVkR/nJ18IBUMi1ZZjndiCzrZMuNlBO5 Zg+gW86zSQn8uK3saKkf9dfByOMzqbnh6kGQk1csbTnZGzV06isOc6dOMjltlAN3B75wz1Xc5 Yed//siDXpoDaZVz7+JBt7Spcj5Iqu2b59uh7q1RfbAr0l5DxoonFeZRv1pi5JX3wqxTrnNDO eXq6rd0c6+LdzHwyJhA0gBgIFtgtsG4OVScuyODynQ6GCz3vHP47KIv/rVNu+s1ffaieOACw X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:257127 Archived-At: Thomas Koch writes: Hi Thomas, > My current working theory is: > > 1. There is some buffer in SSH (or TCP) that gets filled by the language server sending data to eglot. > > 2. Tramp sends a command over SSH. > > 3. Tramp sets the JUST-THIS-ONE option of accept-process-output to t since https://debbugs.gnu.org/12145. > > 4. The response to 2. can not arrive due to the buffer being filled in 1. Tramp blocks the emacs main thread. It is a little bit more specific. Tramp speaks with the remote shell in order to execute commands like '(file-truename "...")'. This is transformed by Tramp into the shell command --8<---------------cut here---------------start------------->8--- \readlink --canonicalize-missing /home/albinus/src/yacy_search_server/examples/SimpleSearchClient/pom.xml 2>/dev/null; echo tramp_exit_status $? --8<---------------cut here---------------end--------------->8--- Tramp expects the result of the readlink call, and the return value (in order to know whether this result can be used). And it expects a special shell prompt, which is the indication, that the remote command has been finished. So it calls tramp-accept-process-output, unitl the connection buffer contains something like --8<---------------cut here---------------start------------->8--- /home/albinus/src/yacy_search_server/examples/SimpleSearchClient/pom.xml tramp_exit_status 0 ///4b3b7d4fa561141e84c94a1cf25e8575#$ --8<---------------cut here---------------end--------------->8--- This special shell prompt is initialized at process startup, by setting PS1. This works fine, until there is the concurrency with the Eglot process. According to my debugging, Tramp sees only part of the output. The final shell prompt, ///4b3b7d4fa561141e84c94a1cf25e8575#$, doesn't arrive, and so Tramp is blocked. > I tested my theory by deleting (and thus disabling) the JUST-THIS-ONE > argument in all calls to accept-process-output in tramp.el and > tramp-sh.el. Eglot did not freeze anymore in two tests with two > different systems (but the same jdtls binary on the same Debian > version). Yes. But I still don't understand what's the difference in scenario. > In one test however I saw this in the message buffer: > > [jsonrpc] (warning) Invalid JSON: (control character 0xd 1 > 381 381) > {"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///home/thk/git/yacy_search_server/source/net/yacy/htroot/ConfigSearchBox.java","diagnostics":[{"range":{"start":{"line":35,"character":86},"end":{"line":35,"character":94}},"severity":3,"code":"1102","source":"Java","message":"At > least one of the problems in category \u0027unuseContent-Length: 798 > > {"jsonrpc":"2.0","method":"textDocument/publ > > I already sent the above to Michael in an out-of-band thread (thanks > for your patience!). I suspect this is due to the "Forbidden reentrant call", but it isn't obvious. > I have a vague feeling, that Tramp could be improved with a work queue > such that requests to tramp from notification or timer threads get > blocked while another tramp command is still waiting for a > reply. Exactly. This is what I want to achieve by using threads for protecting the communication with the remote processes. Or at least with the main connection process. > Additionally I understand that Joao has an idea to use markers > controlled by Tramp. - I'm sorry that I can not (yet) contribute with > putting both ideas into code. > > There are two statements in this bug thread that I don't yet understand and that might be worth more elaboration. I volunteer as a debugging rubber duck[1]. However if you both understand each other then never mind: > > [1] https://en.wikipedia.org/wiki/Rubber_duck_debugging > > Joao: "markers in the process buffer is what is used commonly" > > I understand that tramp sends only one command at a time per > connection. Otherwise the reentrant error is thrown. The command > result gets written to a connection-specific buffer. The output is > parsed from point-min and the buffer content deleted after parsing and > before sending the next command. So what should there be improved? > Jsonrpc needs its own markers because N messages can arrive at any > given time and the buffer can not be deleted after each message. Meanwhile I don't believe any longer that markers are the problem. > Michael: "If another process has consumed the output, even if it is pushed into the correct buffer, Tramp doesn't know." > > What exact scenario should that be? Emacs writes the output (if > there's no filter) in the correct buffer for the process. Tramp might > call accept-process-output unnecessarily because the output is already > in the buffer due to a accept-process-output call of some other > code. But in this case Tramp will find the output there eventually, > maybe after having waited until a timeout. This could be improved by > checking for already present output before calling > accept-process-output? I stand corrected. The problem isn't that another process has consumed the output, but rather that not everything arrives. For whatever reason. Best regards, Michael.