From: "João Távora" <joaotavora@gmail.com>
To: 60088@debbugs.gnu.org, eliz@gnu.org
Subject: bug#60088: 29.0.60; Eglot loses messages from gopls
Date: Fri, 16 Dec 2022 08:56:56 +0000 [thread overview]
Message-ID: <CALDnm51Xmj+00uGR=gnOKGBkW03e_J8xZiwWA7Vb-QBvw6k6Hg@mail.gmail.com> (raw)
In-Reply-To: <87pmckg9sw.fsf@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 5334 bytes --]
I've now pushed the patch/bugfix to emacs-29 and bumped
version of jsonrpc.el so we should get a new GNU ELPA package
soon.
GNU ELPA Eglot users should wait for the package to be bumped
too.
João
On Thu, Dec 15, 2022 at 11:55 AM João Távora <joaotavora@gmail.com> wrote:
> [Eli, I have CC'ed you directly because this bug concerns a technique to
> avoid recursive process filters, which seems like an area you are expert
> in.]
>
> Hi,
>
> In some situations, Eglot is losing some messages from the Go language
> server, 'gopls'. The problem has been reproduced and identified.
>
> For context, this started in:
>
> * https://github.com/joaotavora/eglot/issues/587#issuecomment-1221072833
> * https://github.com/golang/go/issues/54559
>
> (In the former link, there are reports of what seem to be other
> problems. The reports are missing clear reproduction recipes, so they
> may not be the same problem I'm about to describe.)
>
> Reproduction:
>
> Ensure go-mode is installed from https://melpa.org/#/go-mode
> $ go install golang.org/x/tools/gopls@latest
> $ git clone git clone https://github.com/google/nftables
> $ emacs -Q -f package-initialize nftables/nftables_test.go
> M-x eglot
>
> The last step will connect to the server, exchange some messages. It
> will not block Emacs, but the Eglot session will be completely
> non-functional.
>
> Analysis:
>
> Many essential JSONRPC messages have not been exchanged, they have been
> lost. The bytes of the lost messages from the gopls side can be seen in
> the process buffer (M-x list-processes, then click the link to the
> hidden process buffer).
>
> The problem happens in jsonrpc.el's jsonrpc--process-filter. The code
> does not expect this function to be called recursively, but that can
> happen if the client code invoked by its jsonrpc--connection-receive
> callee eventually sends a follow-up message to the server. This is a
> common scenario, especially during LSP initialiation. It has been
> working fine for a number of years and a large number of servers.
>
> However:
>
> * if that follow-up message is large enough (this is why the largish
> nftables_test.go file is needed) AND
>
> * the server has already sent us some output with just the right (wrong)
> timing.
>
> then the process-send-string call that is eventually reached will in
> fact cause more output to arrive from the process and a recursive
> invocation of the process filter.
>
> Resolution:
>
> I've looked at a number of ways to solve this problem, from avoiding the
> follow up message in Eglot's side to different techniques on the jsonrpc
> side. None of them are as satisfactory as the patch after my sig, which
> simply re-schedules the would-be recursive call to re-run from within a
> timer stack as soon as possible.
>
> The only problem I can envision with this approach would be if multiple
> recursive calls are launched from the same jsonrpc--connection-receive
> stack frame and would somehow not be executed in the correct order.
> That doesn't seem to be a problem after some experiments like
>
> (progn
> (run-at-time 0 nil #'insert "a")
> (run-at-time 0 nil #'insert "b")
> (run-at-time 0 nil #'insert "c"))
>
> which inserts the predicatbly sane "abc". If this in-order execution is
> somehow not guaranteed, there is the alternate more complex technique of
> inserting the output into the process buffer immediately, and schedule a
> single processing call thereafter.
>
> I will install the following patch soon unless someone objects to this
> approach.
>
> João
>
> diff --git a/lisp/jsonrpc.el b/lisp/jsonrpc.el
> index 90833e1c1d..76bbf28138 100644
> --- a/lisp/jsonrpc.el
> +++ b/lisp/jsonrpc.el
> @@ -548,11 +548,27 @@ jsonrpc--process-sentinel
> (delete-process proc)
> (funcall (jsonrpc--on-shutdown connection) connection)))))
>
> -(defun jsonrpc--process-filter (proc string)
> +(defvar jsonrpc--in-process-filter nil
> + "Non-nil if inside `jsonrpc--process-filter'.")
> +
> +(cl-defun jsonrpc--process-filter (proc string)
> "Called when new data STRING has arrived for PROC."
> + (when jsonrpc--in-process-filter
> + ;; Problematic recursive process filters may happen if
> + ;; `jsonrpc--connection-receive', called by us, eventually calls
> + ;; client code which calls `process-send-string' (which see) to,
> + ;; say send a follow-up message. If that happens to writes enough
> + ;; bytes for additional output to be received, we have a problem.
> + ;;
> + ;; In that case, remove recursiveness by re-scheduling ourselves to
> + ;; run from within a timer as soon as possible.
> +
> + (run-at-time 0 nil #'jsonrpc--process-filter proc string)
> + (cl-return-from jsonrpc--process-filter))
> (when (buffer-live-p (process-buffer proc))
> (with-current-buffer (process-buffer proc)
> (let* ((inhibit-read-only t)
> + (jsonrpc--in-process-filter t)
> (connection (process-get proc 'jsonrpc-connection))
> (expected-bytes (jsonrpc--expected-bytes connection)))
> ;; Insert the text, advancing the process marker.
>
>
>
>
>
>
>
>
>
>
>
>
>
--
João Távora
[-- Attachment #2: Type: text/html, Size: 6730 bytes --]
next prev parent reply other threads:[~2022-12-16 8:56 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-15 11:56 bug#60088: 29.0.60; Eglot loses messages from gopls João Távora
2022-12-16 8:56 ` João Távora [this message]
2023-09-05 23:51 ` Stefan Kangas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CALDnm51Xmj+00uGR=gnOKGBkW03e_J8xZiwWA7Vb-QBvw6k6Hg@mail.gmail.com' \
--to=joaotavora@gmail.com \
--cc=60088@debbugs.gnu.org \
--cc=eliz@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).