From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= Newsgroups: gmane.emacs.bugs Subject: bug#61350: Eglot over Tramp freezes with large project Date: Wed, 15 Mar 2023 20:14:06 +0000 Message-ID: <874jql6acx.fsf@gmail.com> References: <87y1ootw2t.fsf@gmail.com> <31964054.380679.1678194256931@office.mailbox.org> <87r0u0u0rl.fsf@gmx.de> <87mt4otyy1.fsf@gmx.de> <87pm9fk6ht.fsf@gmx.de> <87mt4jzf8q.fsf@gmail.com> <87fsabh2z2.fsf@gmx.de> <87edpvxu7w.fsf@gmail.com> <87bkkzgyb8.fsf@gmx.de> <87lek2x09t.fsf@gmail.com> <875yb646d1.fsf@gmx.de> <87fsa7mw9x.fsf@gmx.de> <87fsa7l6o6.fsf@gmx.de> <87a60fl4p7.fsf@gmx.de> <878rfx6bqe.fsf@gmail.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="20088"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Thomas Koch , Michael Albinus , 61350@debbugs.gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Mar 15 21:13:27 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pcXV1-00051s-Nb for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 15 Mar 2023 21:13:27 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pcXUi-00022d-7L; Wed, 15 Mar 2023 16:13:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pcXUc-00020p-VY for bug-gnu-emacs@gnu.org; Wed, 15 Mar 2023 16:13:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pcXUc-0002dr-JY for bug-gnu-emacs@gnu.org; Wed, 15 Mar 2023 16:13:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pcXUc-0000UD-B7 for bug-gnu-emacs@gnu.org; Wed, 15 Mar 2023 16:13:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 15 Mar 2023 20:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61350 X-GNU-PR-Package: emacs Original-Received: via spool by 61350-submit@debbugs.gnu.org id=B61350.16789111361808 (code B ref 61350); Wed, 15 Mar 2023 20:13:02 +0000 Original-Received: (at 61350) by debbugs.gnu.org; 15 Mar 2023 20:12:16 +0000 Original-Received: from localhost ([127.0.0.1]:40600 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pcXTr-0000T5-Dw for submit@debbugs.gnu.org; Wed, 15 Mar 2023 16:12:16 -0400 Original-Received: from mail-wr1-f44.google.com ([209.85.221.44]:42811) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pcXTp-0000Sr-0X for 61350@debbugs.gnu.org; Wed, 15 Mar 2023 16:12:13 -0400 Original-Received: by mail-wr1-f44.google.com with SMTP id j2so18381676wrh.9 for <61350@debbugs.gnu.org>; Wed, 15 Mar 2023 13:12:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678911127; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=9msQmoLJ1K6zmJZjhzF/+WzuInVxTkpnqY6hGSaqels=; b=EngwL9uhZkt5S2LWSrbmuz6Nhlgxf2Otg1v1Rjp/dsGWpUFRg8z0robd3JfVt/QtLo 3WNPAXJYnHiqFhiNjjF3VTxyIFPny5Cdpd8KUNIKu/Rg6SdvGeX7/5vHSteAilXqr2N/ tNy4GV/Pfp96/nQzUOKR+99unOto0R6BIRU85ZwEgvLOY8HTEwgoMnSq49LnTxF80NgH skc9rSM6T3p5+krpTnUuWQiYNqpcKhao/uaVOOQgUEWUE4dMZBMxcOKT9FrPLzAUhxia CKKc40ZBL1e+bxyX8TEIkkwx1tqNgHC8J6Bv6GjPyIkSrq6BA59KkarYHSYV3SBLZifl jAXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678911127; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=9msQmoLJ1K6zmJZjhzF/+WzuInVxTkpnqY6hGSaqels=; b=7J7drUimJHGSyojgoYfowrOZSY+JLi3ei6G+hVW+JmjPBVbQrCndGtjcIxADtMvlMi r8AHQ5fgRZUJw7IWVrkEH83DlYtv+rY+66lEPhFk2k6BVbB7iMI7pOwB2xUAbJgNh5ci ADZYdmBO1M/HiSeFsqne+paB/yEgeB4iQ/zjzwMnxm+K8EHVlchXozibnwDh3ktb2Kdg SWbWVsR/+6CbZQz37TFWzpflQmnnl1NahjKRafqKq2WP3tMWw5gQbzlq/ItC2V6u6Wdh WnlFRZ4EHtRbaCJxEwai7kux49svHS1nOQd5KhcZJqRGDDtMLvrZJLoRTI904xC6osds ACbA== X-Gm-Message-State: AO0yUKUNFQE7USI8NlfoHP9o2EbncVB8fj/xaiBtUtLHXGs1uOh/MJwU k3NnNGBUguIuTnwgkpBsgd/5YXoOIYQ= X-Google-Smtp-Source: AK7set/X5FN8Cq4+p37x3HJIt0Cn+q13HvZuXLVB0/PcB9SYk3Yb47FnDywmF3lS9wkwFkIj/aFvZA== X-Received: by 2002:a5d:608d:0:b0:2c7:d7c:7d0 with SMTP id w13-20020a5d608d000000b002c70d7c07d0mr2740072wrt.22.1678911126645; Wed, 15 Mar 2023 13:12:06 -0700 (PDT) Original-Received: from krug (87-196-72-142.net.novis.pt. [87.196.72.142]) by smtp.gmail.com with ESMTPSA id s20-20020a7bc394000000b003ed2433aa4asm2732494wmj.41.2023.03.15.13.12.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 13:12:05 -0700 (PDT) In-Reply-To: <878rfx6bqe.fsf@gmail.com> ("=?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?="'s message of "Wed, 15 Mar 2023 19:44:25 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:257982 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Jo=C3=A3o T=C3=A1vora writes: > Stefan Monnier writes: > Does that "some" include _this_ problem? Let's take this one for > example. > > make -C test filenotify-tests > SELECTOR=3Dfile-notify-test04-autorevert Sorry, here I meant to type 'file-notify-test04-autorevert-remote' > We know this was introduced by the recent changes > 54ef338ba3670415cf47fabc33a92d4904707c7e. FWIW, I've run the following experiment: 1. Reverted 54ef338ba3670415cf47fabc33a92d4904707c7e 2. Applied a simpler, more conservative, patch, discussed with Michael before. The patch keeps the protocol for tramp-a-p-o untouched. It simply adds a "while" to the 'a-p-o' call issued for "related" processes. 3. Ran the filenotify test suite. All but one tests pass. 'file-notify-test04-autorevert-remote' now passes. The failure I get, 'file-notify-test06-dir-validity-remote', is _also_ in Emacs 29 4. Removed the Tramp-specific Eglot workaround in lisp/progmodes/eglot.el which disabled Controlmaster. 5. Ran the Eglot-over-Tramp experiment with the JDTLS LSP server multiple times with 0% failures. 6. The two Tramp-related failures in the eglot-tests.el test suite are _also_ fixed. Ran this multiple times, 0% failures. It seems evident that my simpler patch puts us in a much better place than we are now, so I propose we push them. Patches are attached. Of course, we can continue to explore other more elegant solutions. Jo=C3=A3o --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-Revert-Improve-Tramp-processes-to-accept-output-over.patch >From f8ea932928fe37573d28c197c1c9d2da9a08089d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jo=C3=A3o=20T=C3=A1vora?= Date: Wed, 15 Mar 2023 19:18:00 +0000 Subject: [PATCH 1/3] Revert "Improve Tramp processes to accept output over the same socket" This reverts commit 54ef338ba3670415cf47fabc33a92d4904707c7e. --- lisp/net/tramp-adb.el | 2 +- lisp/net/tramp-gvfs.el | 2 +- lisp/net/tramp-sh.el | 14 +------------- lisp/net/tramp-smb.el | 6 +++--- lisp/net/tramp-sudoedit.el | 2 +- lisp/net/tramp.el | 33 +++++++++------------------------ 6 files changed, 16 insertions(+), 43 deletions(-) diff --git a/lisp/net/tramp-adb.el b/lisp/net/tramp-adb.el index d338201ab72..64f45e7958d 100644 --- a/lisp/net/tramp-adb.el +++ b/lisp/net/tramp-adb.el @@ -990,7 +990,7 @@ tramp-adb-handle-make-process (progn (goto-char (point-min)) (not (search-forward "\n" nil t))) - (tramp-accept-process-output p)) + (tramp-accept-process-output p 0)) (delete-region (point-min) (point))) ;; Provide error buffer. This shows only ;; initial error messages; messages diff --git a/lisp/net/tramp-gvfs.el b/lisp/net/tramp-gvfs.el index c1ad37de1d2..266724c587f 100644 --- a/lisp/net/tramp-gvfs.el +++ b/lisp/net/tramp-gvfs.el @@ -1469,7 +1469,7 @@ tramp-gvfs-handle-file-notify-add-watch (set-process-sentinel p #'tramp-file-notify-process-sentinel) ;; There might be an error if the monitor is not supported. ;; Give the filter a chance to read the output. - (while (tramp-accept-process-output p)) + (while (tramp-accept-process-output p 0)) (unless (process-live-p p) (tramp-error p 'file-notify-error "Monitoring not supported for `%s'" file-name)) diff --git a/lisp/net/tramp-sh.el b/lisp/net/tramp-sh.el index 882b79b3ee7..48ebfff6cfe 100644 --- a/lisp/net/tramp-sh.el +++ b/lisp/net/tramp-sh.el @@ -2424,10 +2424,6 @@ tramp-do-copy-or-rename-file-out-of-band copy-program copy-args))) (tramp-message v 6 "%s" (string-join (process-command p) " ")) (process-put p 'vector v) - ;; This is neded for ssh or PuTTY based processes, and - ;; only if the respective options are set. Perhaps, - ;; the setting could be more fine-grained. - (process-put p 'shared-socket t) (process-put p 'adjust-window-size-function #'ignore) (set-process-query-on-exit-flag p nil) @@ -3757,10 +3753,6 @@ tramp-sh-handle-file-notify-add-watch (string-join sequence " ")) (tramp-message v 6 "Run `%s', %S" (string-join sequence " ") p) (process-put p 'vector v) - ;; This is neded for ssh or PuTTY based processes, and only if - ;; the respective options are set. Perhaps, the setting could - ;; be more fine-grained. - (process-put p 'shared-socket t) ;; Needed for process filter. (process-put p 'events events) (process-put p 'watch-name localname) @@ -3769,7 +3761,7 @@ tramp-sh-handle-file-notify-add-watch (set-process-sentinel p #'tramp-file-notify-process-sentinel) ;; There might be an error if the monitor is not supported. ;; Give the filter a chance to read the output. - (while (tramp-accept-process-output p)) + (while (tramp-accept-process-output p 0)) (unless (process-live-p p) (tramp-error p 'file-notify-error "Monitoring not supported for `%s'" file-name)) @@ -5121,10 +5113,6 @@ tramp-maybe-open-connection ;; Set sentinel and query flag. Initialize variables. (set-process-sentinel p #'tramp-process-sentinel) (process-put p 'vector vec) - ;; This is neded for ssh or PuTTY based processes, and - ;; only if the respective options are set. Perhaps, - ;; the setting could be more fine-grained. - (process-put p 'shared-socket t) (process-put p 'adjust-window-size-function #'ignore) (set-process-query-on-exit-flag p nil) (setq tramp-current-connection (cons vec (current-time))) diff --git a/lisp/net/tramp-smb.el b/lisp/net/tramp-smb.el index bb4ab9e3057..1aa4520eeb6 100644 --- a/lisp/net/tramp-smb.el +++ b/lisp/net/tramp-smb.el @@ -757,7 +757,7 @@ tramp-smb-action-get-acl "Read ACL data from connection buffer." (unless (process-live-p proc) ;; Accept pending output. - (while (tramp-accept-process-output proc)) + (while (tramp-accept-process-output proc 0)) (with-current-buffer (tramp-get-connection-buffer vec) ;; There might be a hidden password prompt. (widen) @@ -1363,7 +1363,7 @@ tramp-smb-action-set-acl "Set ACL data." (unless (process-live-p proc) ;; Accept pending output. - (while (tramp-accept-process-output proc)) + (while (tramp-accept-process-output proc 0)) (tramp-message vec 10 "\n%s" (tramp-get-buffer-string (tramp-get-connection-buffer vec))) (throw 'tramp-action 'ok))) @@ -2023,7 +2023,7 @@ tramp-smb-wait-for-output ;; Read pending output. (while (not (re-search-forward tramp-smb-prompt nil t)) - (while (tramp-accept-process-output p)) + (while (tramp-accept-process-output p 0)) (goto-char (point-min))) (tramp-message vec 6 "\n%s" (buffer-string)) diff --git a/lisp/net/tramp-sudoedit.el b/lisp/net/tramp-sudoedit.el index 3cacde2468c..abb9afc570b 100644 --- a/lisp/net/tramp-sudoedit.el +++ b/lisp/net/tramp-sudoedit.el @@ -692,7 +692,7 @@ tramp-sudoedit-action-sudo "Check, whether a sudo process has finished. Remove unneeded output." ;; There might be pending output for the exit status. (unless (process-live-p proc) - (while (tramp-accept-process-output proc)) + (while (tramp-accept-process-output proc 0)) ;; Delete narrowed region, it would be in the way reading a Lisp form. (goto-char (point-min)) (widen) diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el index b6e985db6b1..47173b95bea 100644 --- a/lisp/net/tramp.el +++ b/lisp/net/tramp.el @@ -5087,11 +5087,6 @@ tramp-handle-make-process ;; t. See Bug#51177. (when filter (set-process-filter p filter)) - (process-put p 'vector v) - ;; This is neded for ssh or PuTTY based processes, and - ;; only if the respective options are set. Perhaps, the - ;; setting could be more fine-grained. - (process-put p 'shared-socket t) (process-put p 'remote-command orig-command) (tramp-set-connection-property p "remote-command" orig-command) @@ -5494,7 +5489,7 @@ tramp-handle-file-notify-rm-watch ;; There might be pending output. Avoid problems with reentrant ;; call of Tramp. (ignore-errors - (while (tramp-accept-process-output proc))) + (while (tramp-accept-process-output proc 0))) (tramp-message proc 6 "Kill %S" proc) (delete-process proc)) @@ -5646,13 +5641,13 @@ tramp-action-process-alive "Check, whether a process has finished." (unless (process-live-p proc) ;; There might be pending output. - (while (tramp-accept-process-output proc)) + (while (tramp-accept-process-output proc 0)) (throw 'tramp-action 'process-died))) (defun tramp-action-out-of-band (proc vec) "Check, whether an out-of-band copy has finished." ;; There might be pending output for the exit status. - (while (tramp-accept-process-output proc)) + (while (tramp-accept-process-output proc 0)) (cond ((and (not (process-live-p proc)) (zerop (process-exit-status proc))) (tramp-message vec 3 "Process has finished.") @@ -5683,7 +5678,7 @@ tramp-process-one-action (while (not found) ;; Reread output once all actions have been performed. ;; Obviously, the output was not complete. - (while (tramp-accept-process-output proc)) + (while (tramp-accept-process-output proc 0)) (setq todo actions) (while todo (setq item (pop todo) @@ -5800,21 +5795,11 @@ with-tramp-locked-connection ,@body) (tramp-flush-connection-property ,proc "locked")))) -(defun tramp-accept-process-output (proc &optional _timeout) +(defun tramp-accept-process-output (proc &optional timeout) "Like `accept-process-output' for Tramp processes. This is needed in order to hide `last-coding-system-used', which is set for process communication also. If the user quits via `C-g', it is propagated up to `tramp-file-name-handler'." - (declare (advertised-calling-convention (proc) "29.2")) - ;; There could be other processes which use the same socket for - ;; communication. This could block the output for the current - ;; process. Read such output first. (Bug#61350) - (when-let (((process-get proc 'shared-socket)) - (v (process-get proc 'vector))) - (dolist (p (delq proc (process-list))) - (when (tramp-file-name-equal-p v (process-get p 'vector)) - (accept-process-output p 0 nil t)))) - (with-current-buffer (process-buffer proc) (let ((inhibit-read-only t) last-coding-system-used @@ -5824,10 +5809,10 @@ tramp-accept-process-output ;; JUST-THIS-ONE is set due to Bug#12145. `with-local-quit' ;; returns t in order to report success. (if (with-local-quit - (setq result (accept-process-output proc 0 nil t)) t) + (setq result (accept-process-output proc timeout nil t)) t) (tramp-message - proc 10 "%s %s %s\n%s" - proc (process-status proc) result (buffer-string)) + proc 10 "%s %s %s %s\n%s" + proc timeout (process-status proc) result (buffer-string)) ;; Propagate quit. (keyboard-quit))) result))) @@ -6840,7 +6825,7 @@ tramp-interrupt-process (tramp-get-remote-null-device (process-get proc 'vector)))) ;; Wait, until the process has disappeared. If it doesn't, ;; fall back to the default implementation. - (while (tramp-accept-process-output proc)) + (while (tramp-accept-process-output proc 0)) (not (process-live-p proc)))))) (add-hook 'interrupt-process-functions #'tramp-interrupt-process) -- 2.39.2 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0003-Simpler-fix-for-bug-61350.patch >From 16e93416780e416d27ed514e3d8a876405a4f4bc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jo=C3=A3o=20T=C3=A1vora?= Date: Wed, 15 Mar 2023 20:02:43 +0000 Subject: [PATCH 3/3] Simpler fix for bug#61350 * lisp/net/tramp.el (tramp-accept-process-output): Accept process from related processes. --- lisp/net/tramp.el | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el index 47173b95bea..885b29f9471 100644 --- a/lisp/net/tramp.el +++ b/lisp/net/tramp.el @@ -5800,6 +5800,11 @@ tramp-accept-process-output This is needed in order to hide `last-coding-system-used', which is set for process communication also. If the user quits via `C-g', it is propagated up to `tramp-file-name-handler'." + (when-let (((process-get proc 'shared-socket)) + (v (process-get proc 'vector))) + (dolist (p (delq proc (process-list))) + (when (tramp-file-name-equal-p v (process-get p 'vector)) + (while (accept-process-output p 0 nil t))))) (with-current-buffer (process-buffer proc) (let ((inhibit-read-only t) last-coding-system-used -- 2.39.2 --=-=-=--