Eglot tests on EMBA

Eglot tests on EMBA

[Michael Albinus]

Hi,

FTR, I have changed test/infra/Dockerfile.emba to install also clangd on
the server. This shall enable some of the Eglot tests, which are skipped
at all ATM. Pushed to the emacs-29 branch.

Best regards, Michael.

Re: Eglot tests on EMBA

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi,

> FTR, I have changed test/infra/Dockerfile.emba to install also clangd on
> the server. This shall enable some of the Eglot tests, which are skipped
> at all ATM. Pushed to the emacs-29 branch.

This has been extended for the master branch. On EMBA, there are now two
jobs build-image-eglot and test-eglot, which run in every scheduled
pipeline for the master branch. See for example
<https://emba.gnu.org/emacs/emacs/-/jobs/65322> and
<https://emba.gnu.org/emacs/emacs/-/jobs/65324>. The result,
eglot-tests.log from the latter job, can be retrieved as artifact via
<https://emba.gnu.org/emacs/emacs/-/jobs/65324/artifacts/download>.

While 36 (of 50) tests pass, we have also one failing and 13 skipped jobs:

--8<---------------cut here---------------start------------->8---
Ran 50 tests, 36 results as expected, 1 unexpected, 13 skipped (2023-03-29 14:07:00+0000, 75.388590 sec)

1 unexpected results:
   FAILED  eglot-test-multiline-eldoc  "eglot--tests-force-full-eldoc didn't deliver"

13 skipped results:
  SKIPPED  eglot-test-eclipse-connect  ((skip-unless (executable-find "jdtls")) :form (executable-find "jdtls") :value nil)
  SKIPPED  eglot-test-javascript-basic  ((skip-unless (and (executable-find "typescript-language-server") (executable-find "tsserver"))) :form (and (executable-find "typescript-language-server") (executable-find "tsserver")) :value nil)
  SKIPPED  eglot-test-json-basic  ((skip-unless (executable-find "vscode-json-languageserver")) :form (executable-find "vscode-json-languageserver") :value nil)
  SKIPPED  eglot-test-path-to-uri-windows  ((skip-unless (eq system-type 'windows-nt)) :form (eq gnu/linux windows-nt) :value nil)
  SKIPPED  eglot-test-project-wide-diagnostics-rust-analyzer  ((skip-unless (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer") :value nil)
  SKIPPED  eglot-test-project-wide-diagnostics-typescript  ((skip-unless (and (executable-find "typescript-language-server") (executable-find "tsserver"))) :form (and (executable-find "typescript-language-server") (executable-find "tsserver")) :value nil)
  SKIPPED  eglot-test-python-autopep-formatting  ((skip-unless (and (executable-find "pylsp") (executable-find "autopep8"))) :form (and (executable-find "pylsp") (executable-find "autopep8")) :value nil)
  SKIPPED  eglot-test-python-yapf-formatting  ((skip-unless (and (executable-find "pylsp") (not (executable-find "autopep8")) (or (executable-find "yapf") (executable-find "yapf3")))) :form (and (executable-find "pylsp") (not (executable-find "autopep8")) (or (executable-find "yapf") (executable-find "yapf3"))) :value nil)
  SKIPPED  eglot-test-rust-analyzer-hover-after-edit  ((skip-unless (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer") :value nil)
  SKIPPED  eglot-test-rust-analyzer-watches-files  ((skip-unless (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer") :value nil)
  SKIPPED  eglot-test-rust-on-type-formatting  ((skip-unless (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer") :value nil)
  SKIPPED  eglot-test-snippet-completions  ((skip-unless (and (executable-find "pylsp") (functionp 'yas-minor-mode))) :form (and (executable-find "pylsp") (functionp 'yas-minor-mode)) :value nil)
  SKIPPED  eglot-test-snippet-completions-with-company  ((skip-unless (and (executable-find "pylsp") (functionp 'yas-minor-mode) (functionp 'company-complete))) :form (and (executable-find "pylsp") (functionp 'yas-minor-mode) (functionp 'company-complete)) :value nil)
--8<---------------cut here---------------end--------------->8---

Could somebody check why the test eglot-test-multiline-eldoc fails? And
are there recipes how to install the missing dependencies in Debian
bullseye, in order to activate the missing tests? I've digged for
recipes of clangd and pylisp (see test/infra/Dockerfile.emba target
emacs-eglot); doing it for the other dependencies would be too much for
me.

Best regards, Michael.

Re: Eglot tests on EMBA

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 4474 bytes --]

I can check the eldoc/eglot tests later, looks like I introduced it in a
recent commit and the test might need updating.

João

On Wed, Mar 29, 2023, 15:34 Michael Albinus <michael.albinus@gmx.de> wrote:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
> Hi,
>
> > FTR, I have changed test/infra/Dockerfile.emba to install also clangd on
> > the server. This shall enable some of the Eglot tests, which are skipped
> > at all ATM. Pushed to the emacs-29 branch.
>
> This has been extended for the master branch. On EMBA, there are now two
> jobs build-image-eglot and test-eglot, which run in every scheduled
> pipeline for the master branch. See for example
> <https://emba.gnu.org/emacs/emacs/-/jobs/65322> and
> <https://emba.gnu.org/emacs/emacs/-/jobs/65324>. The result,
> eglot-tests.log from the latter job, can be retrieved as artifact via
> <https://emba.gnu.org/emacs/emacs/-/jobs/65324/artifacts/download>.
>
> While 36 (of 50) tests pass, we have also one failing and 13 skipped jobs:
>
> --8<---------------cut here---------------start------------->8---
> Ran 50 tests, 36 results as expected, 1 unexpected, 13 skipped (2023-03-29
> 14:07:00+0000, 75.388590 sec)
>
> 1 unexpected results:
>    FAILED  eglot-test-multiline-eldoc  "eglot--tests-force-full-eldoc
> didn't deliver"
>
> 13 skipped results:
>   SKIPPED  eglot-test-eclipse-connect  ((skip-unless (executable-find
> "jdtls")) :form (executable-find "jdtls") :value nil)
>   SKIPPED  eglot-test-javascript-basic  ((skip-unless (and
> (executable-find "typescript-language-server") (executable-find
> "tsserver"))) :form (and (executable-find "typescript-language-server")
> (executable-find "tsserver")) :value nil)
>   SKIPPED  eglot-test-json-basic  ((skip-unless (executable-find
> "vscode-json-languageserver")) :form (executable-find
> "vscode-json-languageserver") :value nil)
>   SKIPPED  eglot-test-path-to-uri-windows  ((skip-unless (eq system-type
> 'windows-nt)) :form (eq gnu/linux windows-nt) :value nil)
>   SKIPPED  eglot-test-project-wide-diagnostics-rust-analyzer
> ((skip-unless (executable-find "rust-analyzer")) :form (executable-find
> "rust-analyzer") :value nil)
>   SKIPPED  eglot-test-project-wide-diagnostics-typescript  ((skip-unless
> (and (executable-find "typescript-language-server") (executable-find
> "tsserver"))) :form (and (executable-find "typescript-language-server")
> (executable-find "tsserver")) :value nil)
>   SKIPPED  eglot-test-python-autopep-formatting  ((skip-unless (and
> (executable-find "pylsp") (executable-find "autopep8"))) :form (and
> (executable-find "pylsp") (executable-find "autopep8")) :value nil)
>   SKIPPED  eglot-test-python-yapf-formatting  ((skip-unless (and
> (executable-find "pylsp") (not (executable-find "autopep8")) (or
> (executable-find "yapf") (executable-find "yapf3")))) :form (and
> (executable-find "pylsp") (not (executable-find "autopep8")) (or
> (executable-find "yapf") (executable-find "yapf3"))) :value nil)
>   SKIPPED  eglot-test-rust-analyzer-hover-after-edit  ((skip-unless
> (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer")
> :value nil)
>   SKIPPED  eglot-test-rust-analyzer-watches-files  ((skip-unless
> (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer")
> :value nil)
>   SKIPPED  eglot-test-rust-on-type-formatting  ((skip-unless
> (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer")
> :value nil)
>   SKIPPED  eglot-test-snippet-completions  ((skip-unless (and
> (executable-find "pylsp") (functionp 'yas-minor-mode))) :form (and
> (executable-find "pylsp") (functionp 'yas-minor-mode)) :value nil)
>   SKIPPED  eglot-test-snippet-completions-with-company  ((skip-unless (and
> (executable-find "pylsp") (functionp 'yas-minor-mode) (functionp
> 'company-complete))) :form (and (executable-find "pylsp") (functionp
> 'yas-minor-mode) (functionp 'company-complete)) :value nil)
> --8<---------------cut here---------------end--------------->8---
>
> Could somebody check why the test eglot-test-multiline-eldoc fails? And
> are there recipes how to install the missing dependencies in Debian
> bullseye, in order to activate the missing tests? I've digged for
> recipes of clangd and pylisp (see test/infra/Dockerfile.emba target
> emacs-eglot); doing it for the other dependencies would be too much for
> me.
>
> Best regards, Michael.
>

[-- Attachment #2: Type: text/html, Size: 5766 bytes --]

Re: Eglot tests on EMBA

[João Távora]

Michael, eglot-test-multiline-eldoc doesn't fail locally for me (or
in the GitHub CI system (https://github.com/joaotavora/eglot/actions)
though that is not running master.

After reading the logs,  I think it comes down to how you installed
pylsp on EMBA.   I think you should do it with

   pip install "python-lsp-server[all]"

which install its "providers", which are some type of contribs.

We could have code in eglot-tests.el to check if these "providers"
are installed into pylsp, but I think it's better to just install
pylsp like I provided, and start thinking of some other less
problematic server for these tests.

Maybe move the test to clangd.  I think it would be equally
valid.  But I've not made up my mind if Eglot tests should test
a wide or narrow field of language servers.  Opinions welcome.

João

On Wed, Mar 29, 2023 at 3:46 PM João Távora <joaotavora@gmail.com> wrote:
>
> I can check the eldoc/eglot tests later, looks like I introduced it in a recent commit and the test might need updating.
>
> João
>
> On Wed, Mar 29, 2023, 15:34 Michael Albinus <michael.albinus@gmx.de> wrote:
>>
>> Michael Albinus <michael.albinus@gmx.de> writes:
>>
>> Hi,
>>
>> > FTR, I have changed test/infra/Dockerfile.emba to install also clangd on
>> > the server. This shall enable some of the Eglot tests, which are skipped
>> > at all ATM. Pushed to the emacs-29 branch.
>>
>> This has been extended for the master branch. On EMBA, there are now two
>> jobs build-image-eglot and test-eglot, which run in every scheduled
>> pipeline for the master branch. See for example
>> <https://emba.gnu.org/emacs/emacs/-/jobs/65322> and
>> <https://emba.gnu.org/emacs/emacs/-/jobs/65324>. The result,
>> eglot-tests.log from the latter job, can be retrieved as artifact via
>> <https://emba.gnu.org/emacs/emacs/-/jobs/65324/artifacts/download>.
>>
>> While 36 (of 50) tests pass, we have also one failing and 13 skipped jobs:
>>
>> --8<---------------cut here---------------start------------->8---
>> Ran 50 tests, 36 results as expected, 1 unexpected, 13 skipped (2023-03-29 14:07:00+0000, 75.388590 sec)
>>
>> 1 unexpected results:
>>    FAILED  eglot-test-multiline-eldoc  "eglot--tests-force-full-eldoc didn't deliver"
>>
>> 13 skipped results:
>>   SKIPPED  eglot-test-eclipse-connect  ((skip-unless (executable-find "jdtls")) :form (executable-find "jdtls") :value nil)
>>   SKIPPED  eglot-test-javascript-basic  ((skip-unless (and (executable-find "typescript-language-server") (executable-find "tsserver"))) :form (and (executable-find "typescript-language-server") (executable-find "tsserver")) :value nil)
>>   SKIPPED  eglot-test-json-basic  ((skip-unless (executable-find "vscode-json-languageserver")) :form (executable-find "vscode-json-languageserver") :value nil)
>>   SKIPPED  eglot-test-path-to-uri-windows  ((skip-unless (eq system-type 'windows-nt)) :form (eq gnu/linux windows-nt) :value nil)
>>   SKIPPED  eglot-test-project-wide-diagnostics-rust-analyzer  ((skip-unless (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer") :value nil)
>>   SKIPPED  eglot-test-project-wide-diagnostics-typescript  ((skip-unless (and (executable-find "typescript-language-server") (executable-find "tsserver"))) :form (and (executable-find "typescript-language-server") (executable-find "tsserver")) :value nil)
>>   SKIPPED  eglot-test-python-autopep-formatting  ((skip-unless (and (executable-find "pylsp") (executable-find "autopep8"))) :form (and (executable-find "pylsp") (executable-find "autopep8")) :value nil)
>>   SKIPPED  eglot-test-python-yapf-formatting  ((skip-unless (and (executable-find "pylsp") (not (executable-find "autopep8")) (or (executable-find "yapf") (executable-find "yapf3")))) :form (and (executable-find "pylsp") (not (executable-find "autopep8")) (or (executable-find "yapf") (executable-find "yapf3"))) :value nil)
>>   SKIPPED  eglot-test-rust-analyzer-hover-after-edit  ((skip-unless (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer") :value nil)
>>   SKIPPED  eglot-test-rust-analyzer-watches-files  ((skip-unless (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer") :value nil)
>>   SKIPPED  eglot-test-rust-on-type-formatting  ((skip-unless (executable-find "rust-analyzer")) :form (executable-find "rust-analyzer") :value nil)
>>   SKIPPED  eglot-test-snippet-completions  ((skip-unless (and (executable-find "pylsp") (functionp 'yas-minor-mode))) :form (and (executable-find "pylsp") (functionp 'yas-minor-mode)) :value nil)
>>   SKIPPED  eglot-test-snippet-completions-with-company  ((skip-unless (and (executable-find "pylsp") (functionp 'yas-minor-mode) (functionp 'company-complete))) :form (and (executable-find "pylsp") (functionp 'yas-minor-mode) (functionp 'company-complete)) :value nil)
>> --8<---------------cut here---------------end--------------->8---
>>
>> Could somebody check why the test eglot-test-multiline-eldoc fails? And
>> are there recipes how to install the missing dependencies in Debian
>> bullseye, in order to activate the missing tests? I've digged for
>> recipes of clangd and pylisp (see test/infra/Dockerfile.emba target
>> emacs-eglot); doing it for the other dependencies would be too much for
>> me.
>>
>> Best regards, Michael.



-- 
João Távora

Re: Eglot tests on EMBA

[Michael Albinus]

João Távora <joaotavora@gmail.com> writes:

Hi João,

> After reading the logs,  I think it comes down to how you installed
> pylsp on EMBA.   I think you should do it with
>
>    pip install "python-lsp-server[all]"
>
> which install its "providers", which are some type of contribs.

I'm not happy with this. EMBA installs a conservative list of software;
they shall be available as Debian bullseye package.

Since Debian bullseye does not offer the package python3-pylsp (which
will be available with a later Debian release), I install on EMBA
python3-pyls. This shall be sufficient, because (according to
eglot-server-programs) "pyls" is supported.

eglot-tests.el does not dertect this, it checks only for "pylsp". I
believe this shall be changed, the check shall be for all alternatives
configured in eglot-server-programs. And not only for python, but also
for other languages.

Because of this deficiency, I run "ln -s /usr/bin/pyls /usr/bin/pylsp"
on EMBA. But I dislike it.

> We could have code in eglot-tests.el to check if these "providers"
> are installed into pylsp, but I think it's better to just install
> pylsp like I provided, and start thinking of some other less
> problematic server for these tests.

Hmm, again, I don't believe we shall adapt everything just that the
tests run. It must be the other direction: eglot-tests must check the
environment, and do what's possible. You cannot always assume that
people are using bleeding edge software. It is a valid assumption, that
people use stable distributions, like Debian bullseye.

I have no idea what is needed internally, but Debian bullseye offers
also the packages

--8<---------------cut here---------------start------------->8---
Package python3-pyls-black
    bullseye (stable) (python): Black plugin for the Python Language Server
    0.4.6-3: all

Package python3-pyls-jsonrpc
    bullseye (stable) (python): Python server implementation of the JSON RPC 2.0 protocol
    0.4.0-2: all

Package python3-pyls-spyder
    bullseye (stable) (python): Spyder plugin for the Python Language Server
    0.3.0-3: all
--8<---------------cut here---------------end--------------->8---

Is there something we shall install?

> Maybe move the test to clangd.  I think it would be equally
> valid.  But I've not made up my mind if Eglot tests should test
> a wide or narrow field of language servers.  Opinions welcome.

Your decision.

> João

Best regards, Michael.

Re: Eglot tests on EMBA

[João Távora]

On Fri, Mar 31, 2023 at 11:31 AM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> Hi João,
>
> > After reading the logs,  I think it comes down to how you installed
> > pylsp on EMBA.   I think you should do it with
> >
> >    pip install "python-lsp-server[all]"
> >
> > which install its "providers", which are some type of contribs.
>
> I'm not happy with this. EMBA installs a conservative list of software;
> they shall be available as Debian bullseye package.

Then you should complain to Debian and uninstall pylsp on EMBA is the
meantime.

> Since Debian bullseye does not offer the package python3-pylsp (which
> will be available with a later Debian release), I install on EMBA
> python3-pyls. This shall be sufficient, because (according to
> eglot-server-programs) "pyls" is supported.
>
> eglot-tests.el does not dertect this, it checks only for "pylsp". I
> believe this shall be changed,

No, it shan't :-) I'm not even sure how to do so portably.

Just undo your installation of pylsp on EMBA and I'll look
into replacing that test with some other language server.

Or run `pip install "python-lsp-server[all]"` in the dockerfile.

Let's not overcomplicate eglot-tests.el with these things.

> the check shall be for all alternatives
> configured in eglot-server-programs. And not only for python, but also
> for other languages.

No, that's an immense amount of work and that's not what these
tests are really for.  People add stuff to eglot-server-programs
liberally: it's a huge database now. I'm not crazy about that but
is has always been the most  fair practice to acommodate server
preferences, and people seem like to see their favourite server at
least represented  there.

But some of these servers don't work, are deprecated, have
inconsistent executable names on different platforms: it's
a mess.  Testing that is way beyond the scope of eglot-tests.el.

You may lobby for a eglot-server-program-tests.el instead,
and I won't oppose it, but I personally don't have time for that,
nor do I see a lot of value at the moment.

eglot-tests.el is testing the server-agnostic Elisp logic in
eglot.el -- Eglot _is_ server agnostic.  We just happen to need
someone speaking LSP to be able to exercise features.  We could just
as well have a (compliant) Elisp LSP server simulator and not worry
about  external language servers in eglot-tests.el at all.  But that's quite
a bit of work.  So some common installations of language servers are needed.

`clangd` seems to be a fairly reliable one so I've been looking
at transferring most of the tests that I can to clangd.  But it
doesn't exercise _all_ the features that Eglot supports, like
file watching.

I'll look into adapting this test for clangd instead of pylsp.

In the meantime, just uninstall python-lsp-server on EMBA and
let this test be skipped for now.  Or install it like I suggested.

Re: Eglot tests on EMBA

[Michael Albinus]

João Távora <joaotavora@gmail.com> writes:

Hi João,

>> I'm not happy with this. EMBA installs a conservative list of software;
>> they shall be available as Debian bullseye package.
>
> Then you should complain to Debian and uninstall pylsp on EMBA is the
> meantime.

Nothing to complain to Debian. They offer pylsp in the testing version;
we have simply decided not to use it. We use the stable version.

>> Since Debian bullseye does not offer the package python3-pylsp (which
>> will be available with a later Debian release), I install on EMBA
>> python3-pyls. This shall be sufficient, because (according to
>> eglot-server-programs) "pyls" is supported.
>>
>> eglot-tests.el does not dertect this, it checks only for "pylsp". I
>> believe this shall be changed,
>
> No, it shan't :-) I'm not even sure how to do so portably.

The most simple way would be

--8<---------------cut here---------------start------------->8---
(skip-unless (or (executable-find "pylsp") (executable-find "pyls")))
--8<---------------cut here---------------end--------------->8---

But this is a sledge hammer approach I guess.

> Just undo your installation of pylsp on EMBA and I'll look
> into replacing that test with some other language server.
>
> Or run `pip install "python-lsp-server[all]"` in the dockerfile.
>
> Let's not overcomplicate eglot-tests.el with these things.

eglot-tests.el run for your environment. But there are people with other
environments (like Debian stable), which could errors running
eglot-tests.el.

>> the check shall be for all alternatives
>> configured in eglot-server-programs. And not only for python, but also
>> for other languages.
>
> No, that's an immense amount of work and that's not what these
> tests are really for.  People add stuff to eglot-server-programs
> liberally: it's a huge database now. I'm not crazy about that but
> is has always been the most  fair practice to acommodate server
> preferences, and people seem like to see their favourite server at
> least represented  there.

Is it that hard to extract all relevant server executables for a given
major-mode, and iterate over them with executable-find?

> But some of these servers don't work, are deprecated, have
> inconsistent executable names on different platforms: it's
> a mess.  Testing that is way beyond the scope of eglot-tests.el.

In such a case, the test shall fail. And that would be OK, by this you
get feedback what doesn't work (given people write bug reports).

> You may lobby for a eglot-server-program-tests.el instead,
> and I won't oppose it, but I personally don't have time for that,
> nor do I see a lot of value at the moment.
>
> eglot-tests.el is testing the server-agnostic Elisp logic in
> eglot.el -- Eglot _is_ server agnostic.  We just happen to need
> someone speaking LSP to be able to exercise features.  We could just
> as well have a (compliant) Elisp LSP server simulator and not worry
> about  external language servers in eglot-tests.el at all.  But that's quite
> a bit of work.  So some common installations of language servers are needed.

Sure, possible. I was just speaking about the given state of eglot-tests.el.

> `clangd` seems to be a fairly reliable one so I've been looking
> at transferring most of the tests that I can to clangd.  But it
> doesn't exercise _all_ the features that Eglot supports, like
> file watching.
>
> I'll look into adapting this test for clangd instead of pylsp.

OK.

> In the meantime, just uninstall python-lsp-server on EMBA and
> let this test be skipped for now.  Or install it like I suggested.

Will do.

Best regards, Michael.

Re: Eglot tests on EMBA

[João Távora]

On Fri, Mar 31, 2023 at 1:14 PM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> Hi João,
>
> >> I'm not happy with this. EMBA installs a conservative list of software;
> >> they shall be available as Debian bullseye package.
> >
> > Then you should complain to Debian and uninstall pylsp on EMBA is the
> > meantime.
>
> Nothing to complain to Debian. They offer pylsp in the testing version;
> we have simply decided not to use it. We use the stable version.
>
> >> Since Debian bullseye does not offer the package python3-pylsp (which
> >> will be available with a later Debian release), I install on EMBA
> >> python3-pyls. This shall be sufficient, because (according to
> >> eglot-server-programs) "pyls" is supported.
> >>
> >> eglot-tests.el does not dertect this, it checks only for "pylsp". I
> >> believe this shall be changed,
> >
> > No, it shan't :-) I'm not even sure how to do so portably.
>
> The most simple way would be
>
> --8<---------------cut here---------------start------------->8---
> (skip-unless (or (executable-find "pylsp") (executable-find "pyls")))
> --8<---------------cut here---------------end--------------->8---
>
> But this is a sledge hammer approach I guess.

Hmm, I'm confused.  Why are we talking about 'pyls'?  The
test is for `pylsp`.  It used to be for `pyls`, but now it's
not.  'pyls' would probably have the same "provider" issue.

My point is that I don't know how to ask the `pylsp`
executable easily what "providers" it supports.  pylsp --version
doesn't hint at it. Maybe there is an easy way, but I don't know it.

Maybe you meant these kinds of changes to eglot-tests.el?


diff --git a/test/lisp/progmodes/eglot-tests.el
b/test/lisp/progmodes/eglot-tests.el
index b11ce942b7d..7215dc24b3e 100644
--- a/test/lisp/progmodes/eglot-tests.el
+++ b/test/lisp/progmodes/eglot-tests.el
@@ -570,7 +570,8 @@ eglot-test-non-unique-completions
       '(("project" . (("something.py" . "foo=1\nfoobar=2\nfoo"))))
     (with-current-buffer
         (eglot--find-file-noselect "project/something.py")
-      (should (eglot--tests-connect))
+      (let ((eglot-server-programs '((python-mode . ("pylsp")))))
+        (should (eglot--tests-connect)))
       (goto-char (point-max))
       (completion-at-point))
     ;; FIXME: `current-message' doesn't work here :-(


This makes sense to me.  It's an oversight that eglot-server-programs
isn't restricted there like it is in other tests.

To avoid hurting readability too much, this override could
be an option to the eglot--with-fixture macro.

> eglot-tests.el run for your environment. But there are people with other
> environments (like Debian stable), which could errors running
> eglot-tests.el.

Yes, I understand, but I also doubt the size of the set intesecting
people who have simplistic pylsp installations and people running
the Emacs test suite.  So I'm not very worried.  And most people are
using pyright these days anyway.

> >> the check shall be for all alternatives
> >> configured in eglot-server-programs. And not only for python, but also
> >> for other languages.
> >
> > No, that's an immense amount of work and that's not what these
> > tests are really for.  People add stuff to eglot-server-programs
> > liberally: it's a huge database now. I'm not crazy about that but
> > is has always been the most  fair practice to acommodate server
> > preferences, and people seem like to see their favourite server at
> > least represented  there.
>
> Is it that hard to extract all relevant server executables for a given
> major-mode, and iterate over them with executable-find?

That's not hard, but what do you want to do with this information?
What test do you want to design?  I'm not following.  Maybe I'm
missing something.

> > But some of these servers don't work, are deprecated, have
> > inconsistent executable names on different platforms: it's
> > a mess.  Testing that is way beyond the scope of eglot-tests.el.
>
> In such a case, the test shall fail. And that would be OK, by this you
> get feedback what doesn't work (given people write bug reports).

I don't follow.  What test fails?  What would be considered "failure"?
Not having all possible programs mentioned in `eglot-server-programs`
installed?  Can't be that, that's just tyranny :-)  There's a lot
of stuff in there.

> > You may lobby for a eglot-server-program-tests.el instead,
> > and I won't oppose it, but I personally don't have time for that,
> > nor do I see a lot of value at the moment.
> >
> > eglot-tests.el is testing the server-agnostic Elisp logic in
> > eglot.el -- Eglot _is_ server agnostic.  We just happen to need
> > someone speaking LSP to be able to exercise features.  We could just
> > as well have a (compliant) Elisp LSP server simulator and not worry
> > about  external language servers in eglot-tests.el at all.  But that's quite
> > a bit of work.  So some common installations of language servers are needed.
>
> Sure, possible. I was just speaking about the given state of eglot-tests.el.

It's not super, because of these outside dependencies, but it's not a
very bad state, I would say.  It helped me catch a bug the other day :-)

João

Re: Eglot tests on EMBA

[João Távora]

Just a follow-up, because I just noticed that I was mixing up pylsp and pyls
as well

pylsp is pip install python-lsp-server
pyls is pip install python-language-server

The second one is deprecated according to people familiar with
the matter.  Eglot now "prefers" the first, and so do the tests.
Maybe this helps.

João

On Fri, Mar 31, 2023 at 2:35 PM João Távora <joaotavora@gmail.com> wrote:
>
> On Fri, Mar 31, 2023 at 1:14 PM Michael Albinus <michael.albinus@gmx.de> wrote:
> >
> > João Távora <joaotavora@gmail.com> writes:
> >
> > Hi João,
> >
> > >> I'm not happy with this. EMBA installs a conservative list of software;
> > >> they shall be available as Debian bullseye package.
> > >
> > > Then you should complain to Debian and uninstall pylsp on EMBA is the
> > > meantime.
> >
> > Nothing to complain to Debian. They offer pylsp in the testing version;
> > we have simply decided not to use it. We use the stable version.
> >
> > >> Since Debian bullseye does not offer the package python3-pylsp (which
> > >> will be available with a later Debian release), I install on EMBA
> > >> python3-pyls. This shall be sufficient, because (according to
> > >> eglot-server-programs) "pyls" is supported.
> > >>
> > >> eglot-tests.el does not dertect this, it checks only for "pylsp". I
> > >> believe this shall be changed,
> > >
> > > No, it shan't :-) I'm not even sure how to do so portably.
> >
> > The most simple way would be
> >
> > --8<---------------cut here---------------start------------->8---
> > (skip-unless (or (executable-find "pylsp") (executable-find "pyls")))
> > --8<---------------cut here---------------end--------------->8---
> >
> > But this is a sledge hammer approach I guess.
>
> Hmm, I'm confused.  Why are we talking about 'pyls'?  The
> test is for `pylsp`.  It used to be for `pyls`, but now it's
> not.  'pyls' would probably have the same "provider" issue.
>
> My point is that I don't know how to ask the `pylsp`
> executable easily what "providers" it supports.  pylsp --version
> doesn't hint at it. Maybe there is an easy way, but I don't know it.
>
> Maybe you meant these kinds of changes to eglot-tests.el?
>
>
> diff --git a/test/lisp/progmodes/eglot-tests.el
> b/test/lisp/progmodes/eglot-tests.el
> index b11ce942b7d..7215dc24b3e 100644
> --- a/test/lisp/progmodes/eglot-tests.el
> +++ b/test/lisp/progmodes/eglot-tests.el
> @@ -570,7 +570,8 @@ eglot-test-non-unique-completions
>        '(("project" . (("something.py" . "foo=1\nfoobar=2\nfoo"))))
>      (with-current-buffer
>          (eglot--find-file-noselect "project/something.py")
> -      (should (eglot--tests-connect))
> +      (let ((eglot-server-programs '((python-mode . ("pylsp")))))
> +        (should (eglot--tests-connect)))
>        (goto-char (point-max))
>        (completion-at-point))
>      ;; FIXME: `current-message' doesn't work here :-(
>
>
> This makes sense to me.  It's an oversight that eglot-server-programs
> isn't restricted there like it is in other tests.
>
> To avoid hurting readability too much, this override could
> be an option to the eglot--with-fixture macro.
>
> > eglot-tests.el run for your environment. But there are people with other
> > environments (like Debian stable), which could errors running
> > eglot-tests.el.
>
> Yes, I understand, but I also doubt the size of the set intesecting
> people who have simplistic pylsp installations and people running
> the Emacs test suite.  So I'm not very worried.  And most people are
> using pyright these days anyway.
>
> > >> the check shall be for all alternatives
> > >> configured in eglot-server-programs. And not only for python, but also
> > >> for other languages.
> > >
> > > No, that's an immense amount of work and that's not what these
> > > tests are really for.  People add stuff to eglot-server-programs
> > > liberally: it's a huge database now. I'm not crazy about that but
> > > is has always been the most  fair practice to acommodate server
> > > preferences, and people seem like to see their favourite server at
> > > least represented  there.
> >
> > Is it that hard to extract all relevant server executables for a given
> > major-mode, and iterate over them with executable-find?
>
> That's not hard, but what do you want to do with this information?
> What test do you want to design?  I'm not following.  Maybe I'm
> missing something.
>
> > > But some of these servers don't work, are deprecated, have
> > > inconsistent executable names on different platforms: it's
> > > a mess.  Testing that is way beyond the scope of eglot-tests.el.
> >
> > In such a case, the test shall fail. And that would be OK, by this you
> > get feedback what doesn't work (given people write bug reports).
>
> I don't follow.  What test fails?  What would be considered "failure"?
> Not having all possible programs mentioned in `eglot-server-programs`
> installed?  Can't be that, that's just tyranny :-)  There's a lot
> of stuff in there.
>
> > > You may lobby for a eglot-server-program-tests.el instead,
> > > and I won't oppose it, but I personally don't have time for that,
> > > nor do I see a lot of value at the moment.
> > >
> > > eglot-tests.el is testing the server-agnostic Elisp logic in
> > > eglot.el -- Eglot _is_ server agnostic.  We just happen to need
> > > someone speaking LSP to be able to exercise features.  We could just
> > > as well have a (compliant) Elisp LSP server simulator and not worry
> > > about  external language servers in eglot-tests.el at all.  But that's quite
> > > a bit of work.  So some common installations of language servers are needed.
> >
> > Sure, possible. I was just speaking about the given state of eglot-tests.el.
>
> It's not super, because of these outside dependencies, but it's not a
> very bad state, I would say.  It helped me catch a bug the other day :-)
>
> João



-- 
João Távora

Re: Eglot tests on EMBA

[Michael Albinus]

João Távora <joaotavora@gmail.com> writes:

Hi João,

> The second one is deprecated according to people familiar with
> the matter.  Eglot now "prefers" the first, and so do the tests.
> Maybe this helps.

Don't worry. I have removed pyls(p) from EMBA as suggested by
you. Cause I don't know what's all about this.

If you want to enable this later, once we switch to Debian bookworm, let
me know.

> João

Best regards, Michael.