bug#61350: Eglot over Tramp freezes with large project

[Michael Albinus <michael.albinus@gmx.de>]

Thomas Koch <thomas@koch.ro> writes:

Hi Thomas,

> Michael and I met yesterday for a debugging session.
>
> Eventually we tried without SSH ControlMaster (via tramp-use-ssh-controlmaster-options). Eglot did not freeze anymore.

Yep. It looks, like neither Eglot nor Tramp do it wrong.

> I actually found the hint to ControlMaster already early on but probably made a mistake when trying it:
>
> - Either I set tramp-ssh-controlmaster-options to nil, which is the default and does not have any effect.

Do not use this variable, it is Tramp internal. Instead, set the user
option tramp-use-ssh-controlmaster-options to nil *prior* establishing a
connection, as explained in the Tramp manual.

> - Or SSH still used ControlMaster due to my ssh config:
>
>   Host *
>      ControlMaster auto

This shouldn't be set for hosts you're communicating via Eglot.

> I still see one
>
> error in process filter: tramp-error: peculiar error: "Forbidden reentrant call of Tramp"

This is *another* error, it doesn't block Emacs as described in this
report. And since we have other bug reports about (bug#49954,
bug#60534), I prefer to not discuss it in *this* bug report.

> I don't consider this a solution but just a workaround. After all I
> had tramp freeze my emacs also before I used eglot.

I don't believe it is a workaround only. Using ssh's ControlMaster seems
to be problematic for large data to be transferred, for whatever
reason. It doesn't look like it is an error in Emacs. I might be wrong,
but as I said already, after reading the code again and again (and also
after extensive debugging) I have no clue what could be wrong in Eglot
or Tramp.

Therefore, I recommend that Eglot and Tramp manuals warn about using ssh
ControlMaster option when large data are on the wire.

> After reading jsonrpc and tramp source code yesterday I also believe
> that Joãos proposal might help to make Tramp more stable.

I'm not against changes in Eglot or jsonrpc. But I believe this won't
help with the "Forbidden reentrant call of Tramp" problem.

> I was thinking whether there could be an abstract (and reliable)
> process interaction library to be used by tramp, jsonrpc and
> others. In my init.el I found at least magit (magit-process.el) and
> maybe org-export (ox.el) which also call external processes and need
> to reliably parse process output.

magit-process.el offers some wrappers for process-file,
start-file-process, and even for tramp-sh-handle-process-file and
tramp-sh-handle-start-file-process. So the real work is still done in
Tramp.

In ox.el, start-process is used with a sentinel in
org-export-async-start. However, I don't see where accept-process-output
is called here, it might be distributed over the Org source tree (which
is much too large for me to read).

> jsonrpc is special since it needs to match process output by message
> id to previous process input.
>
> Otherwise it seems to me that there is a lot of shared logic?

What do you mean by this?

Best regards, Michael.