From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= Newsgroups: gmane.emacs.bugs Subject: bug#61350: Eglot over Tramp freezes with large project Date: Thu, 02 Mar 2023 11:22:49 +0000 Message-ID: <87356n8kja.fsf@gmail.com> References: <87y1ootw2t.fsf@gmail.com> <69968923.705640.1677163650760@office.mailbox.org> <87a613f0b7.fsf@gmx.de> <87r0udvmzr.fsf@gmx.de> <878rglxrzm.fsf@gmail.com> <87cz5wmjbx.fsf@gmx.de> <87h6v8f7u9.fsf@gmail.com> <87o7pflfcd.fsf@gmx.de> <87wn43e9ht.fsf@gmail.com> <874jr6oont.fsf@gmx.de> <87sfeqd4zi.fsf@gmail.com> <877cw1swjm.fsf@gmx.de> <87k0016dgo.fsf@gmx.de> <1458446553.50372.1677606917251@office.mailbox.org> <87ilfkh89k.fsf@gmail.com> <87y1ofct83.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="15550"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Thomas Koch , 61350@debbugs.gnu.org To: Michael Albinus Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Mar 02 12:22:29 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pXh12-0003tm-SQ for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 02 Mar 2023 12:22:28 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pXh0f-0007ow-5y; Thu, 02 Mar 2023 06:22:05 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pXh0c-0007oY-R6 for bug-gnu-emacs@gnu.org; Thu, 02 Mar 2023 06:22:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pXh0c-00058p-Jc for bug-gnu-emacs@gnu.org; Thu, 02 Mar 2023 06:22:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pXh0c-0002B0-2t for bug-gnu-emacs@gnu.org; Thu, 02 Mar 2023 06:22:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 02 Mar 2023 11:22:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61350 X-GNU-PR-Package: emacs Original-Received: via spool by 61350-submit@debbugs.gnu.org id=B61350.16777560648300 (code B ref 61350); Thu, 02 Mar 2023 11:22:02 +0000 Original-Received: (at 61350) by debbugs.gnu.org; 2 Mar 2023 11:21:04 +0000 Original-Received: from localhost ([127.0.0.1]:55963 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pXgzf-00029n-Oo for submit@debbugs.gnu.org; Thu, 02 Mar 2023 06:21:04 -0500 Original-Received: from mail-wm1-f52.google.com ([209.85.128.52]:36388) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pXgzc-00028x-Bz for 61350@debbugs.gnu.org; Thu, 02 Mar 2023 06:21:01 -0500 Original-Received: by mail-wm1-f52.google.com with SMTP id j19-20020a05600c191300b003eb3e1eb0caso1468911wmq.1 for <61350@debbugs.gnu.org>; Thu, 02 Mar 2023 03:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1677756054; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rTPK0RLDWFGBiGRPQfZR79huO7yb3tIiukbwohllQL8=; b=TsxMoW5Ox+l9tQbShVEqei+sICvMCFc5+TaqJYbnAKOaTm0cRQonI0TFsgbIiP6QUH GuQQ9ilUk4RAv7glWF+zsXUIaY9dXXZEI3FvVXN4dxpqO+tq/5tb99NJsWNgf1eeL5AK wLzyz4qNix7oNO4hULdLTU0L0BHIYPCeSrFsl45LoqY549dmyg0XLAzKmIsvHc+1Ogwy /q3Y/V5bCe4Pmd6G0UbBvt4qB78k29I9RdzfvjDDgiE3pQO/KnwnXrD7hwRgCJjXbyRo ffKA+JisnZWhK6QublMWYty79UMP2v3rCZaQ7m2Jru086Cyqqp7sEUFTvmQ0iIEmAP3A ycsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677756054; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=rTPK0RLDWFGBiGRPQfZR79huO7yb3tIiukbwohllQL8=; b=pTpz2GtOECgSILU1VmF2F17PI22SMNB/9qk3nNk8BGbUEFzum7ztlDQpUTIayWwd8q Xf0t4Llqm4USn/pbWbNpbUDuOdhe4oSVGKtdFvfn2ylvv0Ljc1F5H3/2Y5UUcKunY7dA TFQnyRD8CztGO6rKsl4KX/5eZCJOtXLqvohInCTZ73LEmtAT19s6kUP3588zfmSHulOP N4OGRg6zz8QP3J24tktnnLYo9roh8NbEIpR8HB4fRRVXkPjsM3jMnXvTzYjbL4vVQ22x UUl79IB8C2QaORijTB1SWMN4m7o2feAVHDzY5EN0AUkWndOlCNgpQ6B7KaWhG95F8zo9 /C9g== X-Gm-Message-State: AO0yUKUtmO1OAGsbACCSw8bSllyAStPJwOWJTLCikd+UeHo4T7OsWj7s F2SPYq1KngfIrw7qDnvUB5EDnOOL1GE= X-Google-Smtp-Source: AK7set/sBQ00FgcAsNQd6QZKPnr6rwMjcHdENKMdNCUOFm18eREhRgd/cErt8b+TXzSEzEkPPxJd/Q== X-Received: by 2002:a05:600c:3319:b0:3eb:2b88:9adc with SMTP id q25-20020a05600c331900b003eb2b889adcmr7393964wmp.25.1677756054135; Thu, 02 Mar 2023 03:20:54 -0800 (PST) Original-Received: from krug (87-196-72-142.net.novis.pt. [87.196.72.142]) by smtp.gmail.com with ESMTPSA id s19-20020a05600c45d300b003df7b40f99fsm2986957wmo.11.2023.03.02.03.20.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:20:53 -0800 (PST) In-Reply-To: <87y1ofct83.fsf@gmx.de> (Michael Albinus's message of "Thu, 02 Mar 2023 12:01:32 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:257136 Archived-At: Michael Albinus writes: >> Michael can probably confirm, correct or deny this. > More or less correct. Actually, I don't think it is correct, at least not the way I meant it: JSONRPC data never gets into the tprocess's buffer. Experiments also seem to disprove it. > But I still can't say which process gets output > when, because I cannot debug accept-process-output (it's a C > function). And running Emacs under gdb changes timings, which is > important I believe. Yes, we must probably write some gdb scripts. Eli is expert at that, but I've done some it myself. >> When one (accept-process-output tprocess nil nil 'JUST-THIS-ONE=3Dt) one >> must be absolutely sure that tprocess is going to send _something_ >> "soon". If it doesn't, we'll hang indefinitely (until the process dies >> or the user quits) > > Yes. But Tramp calls accept-process-output only, if it has send a > command to the remote shell, and it expects something to be returned. At > least the shell prompt. Yes. Tramp is doing the right thing. It really expects a response to come. And more often than not, it does. But sometimes it doesn't, and that's when we hang. >> See the comment there? Only 256 characters back are inspected. > > Yes. But the regexp it searches for is the final shell prompt. Something > like "///4b3b7d4fa561141e84c94a1cf25e8575#$", which is shorter than 256 > bytes for sure. OK, but why can't one search for it from where you last left parsing, (i.e. point) up to process-mark? Anyway, I increased that value significantly and it didn't make a difference, so this is probably a red herring. >> So, finally, here's my conjecture: >> >> 1. Tramp goes into 'tramp-wait-for-regexp'. tprocess's buffer already >> the message that 'found' is supposed to return, but it also has a lot >> more stuff, say a lot of JSONRPC data from the LSP server that also >> came into that tprocess buffer and is awaiting to be delivered to >> jprocess. >> >> 2. This data is for piping into jprocess, where the JSONRPC message will >> be decoded, but it will probably never arrive at its destination. >> >> 3. 'found' will be nil in tramp-wait-for-regexp, because of the >> tramp-search-regexp limitation. >> >> 4. tramp-wait-for-regexp will issue the "risky" accept-process-output >> call. >> >> 5. there is no more data that accept-process-output wants to put in the >> buffer, because the LSP server is fine for the moment. >> >> 6. Emacs hang >> >> Just a conjecture. > > Yes, this is more or less the scenario. But I still don't understand why > not all data are delivered through the socket ssh is using. Could it be > there is a limitation, how much data could be buffered by ssh? After testing with a enhanced tramp-backtrace that prints out the contents of every process buffer, I don't think my conjecture is correct. diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el index df4b7dfca2c..f24a3b51074 100644 --- a/lisp/net/tramp.el +++ b/lisp/net/tramp.el @@ -2212,12 +2212,21 @@ tramp-backtrace If VEC-OR-PROC is nil, the buffer *debug tramp* is used. FORCE forces the backtrace even if `tramp-verbose' is less than 10. This function is meant for debugging purposes." - (let ((tramp-verbose (if force 10 tramp-verbose))) + (let ((tramp-verbose (if force 10 tramp-verbose)) + (bt (lambda () + (backtrace) + (dolist (p (process-list)) + (let* ((buf (process-buffer p)) + (name (and buf (buffer-name buf)))) + (when buf + (princ (format "\n--8<---- begin contents of `%s' ----= -->8---\n" name)) + (princ (with-current-buffer buf (buffer-string))) + (princ (format "\n--8<---- end contents of `%s' ------= >8---\n" name)))))))) (when (>=3D tramp-verbose 10) (if vec-or-proc (tramp-message - vec-or-proc 10 "\n%s" (with-output-to-string (backtrace))) - (with-output-to-temp-buffer "*debug tramp*" (backtrace)))))) + vec-or-proc 10 "\n%s" (with-output-to-string (funcall bt))) + (with-output-to-temp-buffer "*debug tramp*" (funcall bt)))))) When you press C-g after the hang occurs, the backtrace is correct but the tprocess buffer is simply empty, according to the new logs. IOW the response Tramp was waiting for never arrived. >> There are no (usable) threads in Emacs. > There are. I made Tramp using threads, and it worked fine, when no > interactive dialogue inside a thread happened. Right. There are so-called green threads, which could fit input-waiting situations like this one, not for achieving true simultaneity. In my opinion, they don't present any advantage over the evented model, which we understand much better (wait... except we don't :-) ) I don't think we should do that until we understand what is happening. >> Timers are events, and so are runs of each processe's process filter. >> Those two are what creates asynchronicity and the emulation of >> simultaneity in Emacs. When jprocess's filter sees a whole JSONRPC >> message, it calls the message handler. > > Timers and process filters are the cause of the "Forbidden reentrant > call in Tramp" errors. I've had problems with reentrant calls to process filters before, and I solved them with a timer. Not sure what that error is. > Wwe must do anything, solving this. I don't understand what you mean here. Jo=C3=A3o