Making `eglot-server-programs' a custom variable?

Making `eglot-server-programs' a custom variable?

[Arash Esbati]

Hi all,

I tried eglot only once and I had to add a lsp-server to
`eglot-server-programs'.  From this experience, is there a plan to make
this a custom variable?  If not, I suggest to change the example in
eglot manual from

  (add-to-list 'eglot-server-programs
               '(foo-mode . ("fools" "--stdio")))

to

  (with-eval-after-load 'eglot
    (add-to-list 'eglot-server-programs
                 '(foo-mode . ("fools" "--stdio"))))

for those who will copy&paste this into their init files and then wonder
why it throws an error.

Best, Arash

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Arash Esbati <arash@gnu.org> writes:

> Hi all,
>
> I tried eglot only once and I had to add a lsp-server to
> `eglot-server-programs'.  From this experience, is there a plan to make
> this a custom variable?  If not, I suggest to change the example in
> eglot manual from
>
>   (add-to-list 'eglot-server-programs
>                '(foo-mode . ("fools" "--stdio")))
>
> to
>
>   (with-eval-after-load 'eglot
>     (add-to-list 'eglot-server-programs
>                  '(foo-mode . ("fools" "--stdio"))))
>
> for those who will copy&paste this into their init files and then wonder
> why it throws an error.

How would making `eglot-server-programs' help in that respect?  If the
`defvar' were just to be replaced by a `defcustom', the result would
still just be a variable, that couldn't be `add-to-list'ed before it is
loaded.

From what I see `add-to-list' has no special handling of user options.

> Best, Arash

I've CC'ed João.

Re: Making `eglot-server-programs' a custom variable?

[Arash Esbati]

Philip Kaludercic <philipk@posteo.net> writes:

> How would making `eglot-server-programs' help in that respect?

If it's meant to be extended by users, then one could use the custom
interface for it, not add-to-list then.

> If the `defvar' were just to be replaced by a `defcustom', the result
> would still just be a variable, that couldn't be `add-to-list'ed
> before it is loaded.

This will depend on the implementation.  Say the current content of
`eglot-server-programs' is in `eglot-server-programs-builtin' and
`eglot-server-programs' is a custom variable, and you have a function
like this in eglot.el:

(defun eglot-server-programs ()
  (append eglot-server-programs
          eglot-server-programs-builtin))

then a user can just setq the custom eglot-server-programs without being
worry about the rest.

Best, Arash

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Date: Wed, 09 Nov 2022 21:25:53 +0100
> 
> I tried eglot only once and I had to add a lsp-server to
> `eglot-server-programs'.  From this experience, is there a plan to make
> this a custom variable?  If not, I suggest to change the example in
> eglot manual from
> 
>   (add-to-list 'eglot-server-programs
>                '(foo-mode . ("fools" "--stdio")))
> 
> to
> 
>   (with-eval-after-load 'eglot
>     (add-to-list 'eglot-server-programs
>                  '(foo-mode . ("fools" "--stdio"))))
> 
> for those who will copy&paste this into their init files and then wonder
> why it throws an error.

The intent is for this variable to include enough servers to make it
unnecessary to add to the list.  With that in mind, I wouldn't
complicate the manual by making sure this is copy/paste-able.

Is the LSP server you needed to add still missing from the database
currently on master?  If so, please suggest the addition(s).

Re: Making `eglot-server-programs' a custom variable?

[Tim Cross]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arash Esbati <arash@gnu.org>
>> Date: Wed, 09 Nov 2022 21:25:53 +0100
>> 
>> I tried eglot only once and I had to add a lsp-server to
>> `eglot-server-programs'.  From this experience, is there a plan to make
>> this a custom variable?  If not, I suggest to change the example in
>> eglot manual from
>> 
>>   (add-to-list 'eglot-server-programs
>>                '(foo-mode . ("fools" "--stdio")))
>> 
>> to
>> 
>>   (with-eval-after-load 'eglot
>>     (add-to-list 'eglot-server-programs
>>                  '(foo-mode . ("fools" "--stdio"))))
>> 
>> for those who will copy&paste this into their init files and then wonder
>> why it throws an error.
>
> The intent is for this variable to include enough servers to make it
> unnecessary to add to the list.  With that in mind, I wouldn't
> complicate the manual by making sure this is copy/paste-able.
>
> Is the LSP server you needed to add still missing from the database
> currently on master?  If so, please suggest the addition(s).

I'm not sure that intent will hold. It has the underlying assumption
there is a definitive language server for each language. There are
multiple servers for many languages and individual
preferences/requirements can differ.

The first thing I had to do in order to use eglot was modify this
variable to add my preferred Javascript server. From an end user
perspective, this was harder to do than making the same change with
lsp-mode. For reference, here is what I ended up doing to get it working

  (defclass eglot-deno (eglot-lsp-server) ()
     :documentation "A custom class for deno lsp.")
   (cl-defmethod eglot-initialization-options ((server eglot-deno))
     "Passes through required deno initialization options"
     (list :enable t
           :lint t))
  (add-to-list 'eglot-server-programs '((js-mode typescript-mode) . (eglot-deno "deno" "lsp")))

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Thu, 10 Nov 2022 18:56:01 +1100
> 
> > Is the LSP server you needed to add still missing from the database
> > currently on master?  If so, please suggest the addition(s).
> 
> I'm not sure that intent will hold. It has the underlying assumption
> there is a definitive language server for each language.

No, it doesn't.  Please see the current value of the variable and its
documentation in the manual.

> There are multiple servers for many languages and individual
> preferences/requirements can differ.

Yes, and the variable already supports that.

> The first thing I had to do in order to use eglot was modify this
> variable to add my preferred Javascript server.

Which one?  Can it be added to the variable if it's still missing?

Re: Making `eglot-server-programs' a custom variable?

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> The intent is for this variable to include enough servers to make it
> unnecessary to add to the list.

Ok, that's a different story.  Would it then make sense to add a comment
to the manual about each server and if they work on the platforms Emacs
runs?  Currently, "digestif" doesn't run on Windows, "texlab" (see
below) does.  I know, this is work and possibly out of Eglot's scope,
but it could give users some direction.

> Is the LSP server you needed to add still missing from the database
> currently on master?  If so, please suggest the addition(s).

The addition would look like this:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/progmodes/eglot.el b/lisp/progmodes/eglot.el
index 2eaa396386..f69d392b46 100644
--- a/lisp/progmodes/eglot.el
+++ b/lisp/progmodes/eglot.el
@@ -217,7 +217,7 @@ eglot-server-programs
                                 (scala-mode . ("metals-emacs"))
                                 (racket-mode . ("racket" "-l" "racket-langserver"))
                                 ((tex-mode context-mode texinfo-mode bibtex-mode)
-                                 . ("digestif"))
+                                 . ,(eglot-alternatives '("digestif" "texlab")))
                                 (erlang-mode . ("erlang_ls" "--transport" "stdio"))
                                 (yaml-mode . ("yaml-language-server" "--stdio"))
                                 (nix-mode . ,(eglot-alternatives '("nil" "rnix-lsp")))
--8<---------------cut here---------------end--------------->8---

Best, Arash

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Thu, 10 Nov 2022 10:18:43 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The intent is for this variable to include enough servers to make it
> > unnecessary to add to the list.
> 
> Ok, that's a different story.  Would it then make sense to add a comment
> to the manual about each server and if they work on the platforms Emacs
> runs?

IMO, that'd be too much stuff that is not really related to Emacs.

> Currently, "digestif" doesn't run on Windows, "texlab" (see
> below) does.  I know, this is work and possibly out of Eglot's scope,
> but it could give users some direction.

Users need to read the documentation of the servers they want to
install anyway.  We cannot provide that in Emacs, in the same way we
don't provide this information about other programs Emacs uses.

> > Is the LSP server you needed to add still missing from the database
> > currently on master?  If so, please suggest the addition(s).
> 
> The addition would look like this:

Thanks, I'll let João comment on this.

Re: Making `eglot-server-programs' a custom variable?

[Tim Cross]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tim Cross <theophilusx@gmail.com>
>> Date: Thu, 10 Nov 2022 18:56:01 +1100
>> 
>> > Is the LSP server you needed to add still missing from the database
>> > currently on master?  If so, please suggest the addition(s).
>> 
>> I'm not sure that intent will hold. It has the underlying assumption
>> there is a definitive language server for each language.
>
> No, it doesn't.  Please see the current value of the variable and its
> documentation in the manual.
>

I guess you will have to explain as I've looked at the documentation
again and still don't see how you can have two different language
servers for the same language. While you can have multiple entries for
the same language modes, only the first one in the list take effect.

>> There are multiple servers for many languages and individual
>> preferences/requirements can differ.
>
> Yes, and the variable already supports that.
>

How? Please show as it isn't clear to me from the documentation and
there isn't a single entry which shows support for multiple language
servers for the same language (where only one is active at a time of course). .

>> The first thing I had to do in order to use eglot was modify this
>> variable to add my preferred Javascript server.
>
> Which one?  Can it be added to the variable if it's still missing?

I included it at the end of my post and you deleted it. You can certainly add it,
but it will only take effect if it comes before the existing entry for
javascript and of course, if you do that, it will be irritating for those who want the
original value.

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1587 bytes --]

Arash, I think your suggestion of recommending `with-eval-after-load`
is pertinent and should be added to the manual.

Eli, even though we provide a healthy dose of built-in server invocations
in that variable, we can't and shouldn't aim at being exhaustive.

Eglot's manual is meant to be a go-to guide for knowing how to resolve
simple problems such as this one, which is very common.  The problems
solved by eglot-workspace-configuration and other variables are also
very common. The manual should be reasonably self sufficient, not too
terse or relying on the user's knowledge.

The "Quick Start" section that Eli added is a great example of that
pragmatism, but other parts need to follow suit.

So we really do need a copy-paste recipe for things like this.  I'll be
committing Arash's trivial and effective suggestion today or tomorrow
unless someone seriously objects.

That said, I have no objection to converting eglot-server-programs into
a defcustom, if someone will commit to translating and maintaining all
the complex combination of options into "widget" form.  I don't have the
time or inclination for this task, but maybe someone has.

Tim, I'm on board with making sure the documentation can
demonstrate to users how to perform complex customizations
of this variable, like the one you suggest.   But here, I think
I tend to prefer documenting such complex things in one place only,
and I see no problem in teaching the user to C-h f to the variable's
docstring, which is the canonical source of truth to that interface.

Thanks,
João

[-- Attachment #2: Type: text/html, Size: 2069 bytes --]

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1451 bytes --]

On Thu, Nov 10, 2022 at 9:35 AM Eli Zaretskii <eliz@gnu.org> wrote:

> Users need to read the documentation of the servers they want to
> install anyway.  We cannot provide that in Emacs, in the same way we
> don't provide this information about other programs Emacs uses.

Yes, I agree.  We won't mention details like server's operating
systems in the manual, at least not in that section.  The only mentions to
servers so far are in the example of eglot-workspace-configurations, because
they its a fairly representative example, and no better one has been
submitted.
And we also mention, in the troubleshooting section, that clangd and pylsp
are
good servers to try to reproduce a bug report, because they're fairly easy
to install.

> > The addition would look like this:
> [...]
> Thanks, I'll let João comment on this.

The addition is fine.

But I disagree that we should endeavor to be exhaustive in that list
and think that tweaking the variable is a rare event.  Servers come and
go for many languages, it's a very volatile landscape.  And
eglot-server-programs is just a starter database of simple server
invocations that are more or less known to work out of the box.  It is
designed to be tweaked, passing command line arguments to servers,
putting in absolute paths for experiments, etc.

Maybe the variable could be autoloaded, but that probably has
some pitfalls I'm not seeing at the moment.

João

[-- Attachment #2: Type: text/html, Size: 1685 bytes --]

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Thu, 10 Nov 2022 20:34:49 +1100
> 
> >> I'm not sure that intent will hold. It has the underlying assumption
> >> there is a definitive language server for each language.
> >
> > No, it doesn't.  Please see the current value of the variable and its
> > documentation in the manual.
> 
> I guess you will have to explain as I've looked at the documentation
> again and still don't see how you can have two different language
> servers for the same language. While you can have multiple entries for
> the same language modes, only the first one in the list take effect.

That's not my understanding.

> >> There are multiple servers for many languages and individual
> >> preferences/requirements can differ.
> >
> > Yes, and the variable already supports that.
> >
> 
> How?

Via eglot-alternatives.  There are entries in the value which already
use that.  And the change you suggested used that as well.  So I
wonder what kind of misunderstanding are we having here.

> Please show as it isn't clear to me from the documentation and
> there isn't a single entry which shows support for multiple language
> servers for the same language (where only one is active at a time of course). .

You mean, there's a difference between several servers for the same
mode and several servers for the same language?  What is the
difference?  Or is that also part of the misunderstanding?

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 10 Nov 2022 10:15:41 +0000
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
> Arash, I think your suggestion of recommending `with-eval-after-load`
> is pertinent and should be added to the manual.

How about adding a command to add servers to the list instead?  See
below.

> Eli, even though we provide a healthy dose of built-in server invocations
> in that variable, we can't and shouldn't aim at being exhaustive.
> 
> Eglot's manual is meant to be a go-to guide for knowing how to resolve
> simple problems such as this one, which is very common.  The problems
> solved by eglot-workspace-configuration and other variables are also 
> very common. The manual should be reasonably self sufficient, not too
> terse or relying on the user's knowledge.

IMNSHO, it is not the mission of the Eglot manual to teach people
with-eval-after-load and in general how to update lists that aren't
autoloaded or preloaded.  This is a slippery slope.

> That said, I have no objection to converting eglot-server-programs into
> a defcustom, if someone will commit to translating and maintaining all
> the complex combination of options into "widget" form.  I don't have the 
> time or inclination for this task, but maybe someone has.

It is not a good idea to have a defcustom whose value should be such a
complex data structure.  If we think users will have to augment the
value, it is better to add a command to do that, and in that command
hide all the complexity and subtleties of the value.

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 2466 bytes --]

On Thu, Nov 10, 2022 at 11:23 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: João Távora <joaotavora@gmail.com>
> > Date: Thu, 10 Nov 2022 10:15:41 +0000
> > Cc: emacs-devel <emacs-devel@gnu.org>
> >
> > Arash, I think your suggestion of recommending `with-eval-after-load`
> > is pertinent and should be added to the manual.
>
> How about adding a command to add servers to the list instead?  See
> below.
>
> > Eli, even though we provide a healthy dose of built-in server invocations
> > in that variable, we can't and shouldn't aim at being exhaustive.
> >
> > Eglot's manual is meant to be a go-to guide for knowing how to resolve
> > simple problems such as this one, which is very common.  The problems
> > solved by eglot-workspace-configuration and other variables are also
> > very common. The manual should be reasonably self sufficient, not too
> > terse or relying on the user's knowledge.
>
> IMNSHO, it is not the mission of the Eglot manual to teach people
> with-eval-after-load and in general how to update lists that aren't
> autoloaded or preloaded.  This is a slippery slope.
>

I understand your opinion, and I share it mostly.  But this is not that
slippery.
We also have short mentions in passing of directory-local variables because
they are essential to setting up eglot-workspace-configuration.  Short
mentions
of Emacs's facilities and links to other documentation deepening those
concepts is the best strategy. I've used it in the README.md and MANUAL.md
that preceded this manual and I can attest that it worked well.

If making the variable autoloaded can make use skip the
with-eval-after-load,
and has no other pitfalls, then I'm OK with that too.  Whatever works and
makes the manual self-sufficient in this very simple and common task.

> That said, I have no objection to converting eglot-server-programs into
> > a defcustom, if someone will commit to translating and maintaining all
> > the complex combination of options into "widget" form.  I don't have the
> > time or inclination for this task, but maybe someone has.
>
> It is not a good idea to have a defcustom whose value should be such a
> complex data structure.


I agree, but a command is a not a solution IMO: this belongs in the
user's configuration file.

A good docstring (can it  be improved?) containing good descriptions
and some ready-made recipes is the current and best approach to these
things.

[-- Attachment #2: Type: text/html, Size: 3387 bytes --]

Re: Making `eglot-server-programs' a custom variable?

[Stefan Monnier]

> Arash, I think your suggestion of recommending `with-eval-after-load`
> is pertinent and should be added to the manual.

I don't have an objection to that but I consider uses of
`with-eval-after-load` as hints that maybe we should do things differently.
IOW we should try and change the way this customization is done such that it
does not need `with-eval-after-load` any more.

The suggestion to split the var in two (one plain var and a custom var
that defaults to nil) is such a possible solution.

> Eli, even though we provide a healthy dose of built-in server invocations
> in that variable, we can't and shouldn't aim at being exhaustive.

Seeing the wild number of LSP servers available for some languages, I'd
agree, sadly.

Maybe to reduce the problem we should allow multiple entries per
major mode and use the first that works, without needing to go through
`eglot-alternatives`?

> That said, I have no objection to converting eglot-server-programs into
> a defcustom, if someone will commit to translating and maintaining all
> the complex combination of options into "widget" form.  I don't have the
> time or inclination for this task, but maybe someone has.

That variable's type is arguably too complex for a `defcustom`.
Maybe the corresponding info should be split into several variables to
make it more accessible to users, but I don't have any good idea how to
do that.
[ I'll note in passing that it's common to use strings for TCP port
  numbers, especially once they are standardized enough to appear in
  /etc/services, so maybe the syntax for that TCP connection should
  replace (HOST PORT ...) with something like (:tcp HOST PORT ...).  ]


        Stefan

Re: Making `eglot-server-programs' a custom variable?

[Tim Cross]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tim Cross <theophilusx@gmail.com>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 10 Nov 2022 20:34:49 +1100
>> 
>> >> I'm not sure that intent will hold. It has the underlying assumption
>> >> there is a definitive language server for each language.
>> >
>> > No, it doesn't.  Please see the current value of the variable and its
>> > documentation in the manual.
>> 
>> I guess you will have to explain as I've looked at the documentation
>> again and still don't see how you can have two different language
>> servers for the same language. While you can have multiple entries for
>> the same language modes, only the first one in the list take effect.
>
> That's not my understanding.
>
>> >> There are multiple servers for many languages and individual
>> >> preferences/requirements can differ.
>> >
>> > Yes, and the variable already supports that.
>> >
>> 
>> How?
>
> Via eglot-alternatives.  There are entries in the value which already
> use that.  And the change you suggested used that as well.  So I
> wonder what kind of misunderstanding are we having here.
>

That function is not mentioned or referenced in the documentation for
eglot-server-programs. I did see that function when I was configuring
eglot, but it wasn't at all clear to me how it fitted into the
picture. I cold not find any reference to eglot-alternatives in the
eglot manual and it isn't listed in the index, so after you mentioning
it, while I *think* I understand what it does, I still don't see how it
is relevant in this situation (unless you use it in conjunction with
other elisp to implement a new command for selecting which server to
use?).

Not sure how you feel it was part of the change I suggested given I've
not suggested any change - all I did was copy and paste the
configuration I had to add in order to get eglot to work with my
preferred language server for javascript. I really added it to show that
adding a new/different server isn't always straight-forward and that it
requires editing that eglot-server-programs variable. 

>> Please show as it isn't clear to me from the documentation and
>> there isn't a single entry which shows support for multiple language
>> servers for the same language (where only one is active at a time of course). .
>
> You mean, there's a difference between several servers for the same
> mode and several servers for the same language?  What is the
> difference?  Or is that also part of the misunderstanding?

I don't think 'mode' adds much here. It is really just an emacs specific
notion which langauge servers know nothing about and used to identify
the source language. For example, we have
js-mode and typescript-mode, both modes for editing/developing
javascript. However, there is more than one language server mode for
javascript. The 'default' is typescript-language-server, my preference
is deno. Both are language servers for javascript and typescript. 

Different language servers for the same language can offer different
features or different strategies in supporting a feature. One server
might have different snippets, different lint rules, different
formatting rules or use a different strategy for implementing things
like code completion or simply have a different performance profile.

As it stands now, you have an alist with keys that are emacs mode names
and values which are the language server to run (along with command line
arguments etc) when you use eglot in one of the modes used as the key.

I cannot see how you can have two separate entries for (js-mode
typescript-mode) or if you do, how the user indicates which entry to
use.

In my case, my configuration of deno works because it appears in the
list before the default entry which uses
typescript-language-server. Your suggestion was that if there is a
server not in the list, it should just be added to the list. I still
don't see how this would work in the general case. If there are two
entries in the list with the same key, how does the user identify which
entry they want to be used?

It was this which made be suggest that your suggestion that this
variable would typically not need to be modified by end users may not
hold. I think it will be something which users will often need to do -
especially as this space is still evolving and the status of servers for
many languages are still somewhat in flux.

Note that I may not agree 100% with the OPs position that the manual
should be updated. When adding code to your init.el file, cut and paste
examples frequently get people into more trouble than help. I think that
if your going to add calls to add-to-list in your init.el file, you
relaly should understand what that means and why you cannot add to a
list which doesn't yet exist etc. However, I disagree with the
suggestion this variable is some internal variable which won't need to
be modified by users.

I also wanted to highlight that in this specific case of adding deno as
a server, it was much easier and more straight-forward to do so with
lsp-mode than it was in eglot. This is not a suggestion that lsp-mode is
better - for many reasons, I very much prefer the architecture and
design goals of eglot over lsp-mode. It was mentioned as just a data
point from someone who has used both. Note also that I did initially try
adding deno useing the simple/naive approach as outlined in the manual,
but could not get that to work. In the end, I got it working after some
help from IRC. 

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 10 Nov 2022 12:07:30 +0000
> Cc: arash@gnu.org, theophilusx@gmail.com, emacs-devel@gnu.org
> 
>  > That said, I have no objection to converting eglot-server-programs into
>  > a defcustom, if someone will commit to translating and maintaining all
>  > the complex combination of options into "widget" form.  I don't have the 
>  > time or inclination for this task, but maybe someone has.
> 
>  It is not a good idea to have a defcustom whose value should be such a
>  complex data structure. 
> 
> I agree, but a command is a not a solution IMO: this belongs in the 
> user's configuration file. 

??? The user init file can call the command, cannot it?

And adding a command is not a replacement for having a variable:
people who know enough Lisp can do what they want with the variable.
The command is proposed as a replacement for a defcustom, because
interactively customizing such a complex variable with a very long
value is problematic at best.

> A good docstring (can it  be improved?) containing good descriptions 
> and some ready-made recipes is the current and best approach to these 
> things.

Sorry, but I disagree.  No doc string can reasonably teach users
advanced Lisp.  That's a non-starter.

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Arash, I think your suggestion of recommending `with-eval-after-load`
>> is pertinent and should be added to the manual.
> I don't have an objection to that but I consider uses of
> `with-eval-after-load` as hints that maybe we should do things
> differently.

Why?  This is in a user's init file.  I can more or less see that
argument for inter-library dependencies.  But here, w-e-a-load is
exactly what's needed.  I use it all the time in my config.

The reason people probably didn't need this earlier is because they were
using Eglot from a package and were simply adding

  (require 'eglot)
  (add-to-list ...)

Or they were doing that use-package magic, which probably does the same
something similar to the require or the w-e-a-load strategy.

> The suggestion to split the var in two (one plain var and a custom var
> that defaults to nil) is such a possible solution.

That would just break user's configurations and complicate things for no
reason.  add-to-list is the way to go, and with-eval-after-load (or
use-package if you must) is just there for that.  Composing existing
pieces that do one job well is better than inventing and maintaining new
pieces.

>> Eli, even though we provide a healthy dose of built-in server invocations
>> in that variable, we can't and shouldn't aim at being exhaustive.
> Seeing the wild number of LSP servers available for some languages, I'd
> agree, sadly.

I don't think this is sad :-)

> Maybe to reduce the problem we should allow multiple entries per
> major mode and use the first that works, without needing to go through
> `eglot-alternatives`?

Again, why?  Why add more semantics to an already complicated variable
when the functional eglot-alternatives plugin works fine?  Furthermore,
I'd like to this particular configuration for a future
multiple-simultaneous-server-in-one-mode idea.

> [ I'll note in passing that it's common to use strings for TCP port
>   numbers, especially once they are standardized enough to appear in
>   /etc/services, so maybe the syntax for that TCP connection should
>   replace (HOST PORT ...) with something like (:tcp HOST PORT ...).  ]

If you can make that change without breaking backward compatibility, I
have no objections.

Re: Making `eglot-server-programs' a custom variable?

[Dmitry Gutov]

On 10.11.2022 17:19, Eli Zaretskii wrote:
>> From: João Távora <joaotavora@gmail.com>
>> Date: Thu, 10 Nov 2022 12:07:30 +0000
>> Cc: arash@gnu.org, theophilusx@gmail.com, emacs-devel@gnu.org
>>
>>   > That said, I have no objection to converting eglot-server-programs into
>>   > a defcustom, if someone will commit to translating and maintaining all
>>   > the complex combination of options into "widget" form.  I don't have the
>>   > time or inclination for this task, but maybe someone has.
>>
>>   It is not a good idea to have a defcustom whose value should be such a
>>   complex data structure.
>>
>> I agree, but a command is a not a solution IMO: this belongs in the
>> user's configuration file.
> 
> ??? The user init file can call the command, cannot it?
> 
> And adding a command is not a replacement for having a variable:
> people who know enough Lisp can do what they want with the variable.
> The command is proposed as a replacement for a defcustom, because
> interactively customizing such a complex variable with a very long
> value is problematic at best.

If you just have a command (to be used interactively, right?), it will 
only affect the value in the current session.

defcustom provides a facility to persist the changes between sessions.

I suppose we could have both a defcustom, and a command or two to modify 
the value.

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Date: Thu, 10 Nov 2022 12:07:30 +0000
>> Cc: arash@gnu.org, theophilusx@gmail.com, emacs-devel@gnu.org
>> 
>>  > That said, I have no objection to converting eglot-server-programs into
>>  > a defcustom, if someone will commit to translating and maintaining all
>>  > the complex combination of options into "widget" form.  I don't have the 
>>  > time or inclination for this task, but maybe someone has.
>> 
>>  It is not a good idea to have a defcustom whose value should be such a
>>  complex data structure. 
>> 
>> I agree, but a command is a not a solution IMO: this belongs in the 
>> user's configuration file. 
>
> ??? The user init file can call the command, cannot it?

Yeah, but then why make it a command, i.e. an interactive function?  If
whatever you're proposing is going to be called interactively, it won't
persist through sessions.  So if you're proposing a simple function, I
think there's no need to for a function that simply calls add-to-list.

> And adding a command is not a replacement for having a variable:
> people who know enough Lisp can do what they want with the variable.
> The command is proposed as a replacement for a defcustom, because
> interactively customizing such a complex variable with a very long
> value is problematic at best.

Again I cannot see the point of a interactive command.  If you mean a
simple non-interactive function to be called from the user's init file,
I can't see much advantage of using that over using add-to-list, other
than auto-loading.  And auto-loading isn't necessarily an advantage: I
break out many Emacs sessions where I don't M-x eglot at all.
with-eval-after-load is the tool for the job here.

>> A good docstring (can it  be improved?) containing good descriptions 
>> and some ready-made recipes is the current and best approach to these 
>> things.
>
> Sorry, but I disagree.  No doc string can reasonably teach users
> advanced Lisp.  That's a non-starter.

Funny, that's how I learned all my Elisp until I learned of the Elisp
manual a few years ago.  And this is not "advanced" stuff at all.  This
is all that's needed:

diff --git a/lisp/progmodes/eglot.el b/lisp/progmodes/eglot.el
index 204121045a0..a6df8371425 100644
--- a/lisp/progmodes/eglot.el
+++ b/lisp/progmodes/eglot.el
@@ -298,7 +298,10 @@ eglot-server-programs
   the call is interactive, the function can ask the user for
   hints on finding the required programs, etc.  Otherwise, it
   should not ask the user for any input, and return nil or signal
-  an error if it can't produce a valid CONTACT.")
+  an error if it can't produce a valid CONTACT.  This option can
+  be combined with the helper function
+  `eglot-alternatives' (which see) to define more than one
+  alternative server for a given MAJOR-MODE.")
 
 (defface eglot-highlight-symbol-face
   '((t (:inherit bold)))

Throwing in an example 3-line snippet in the manual for
eglot-alternatives is also fine:

(with-eval-after-load 'eglot
  (add-to-list 'eglot-server-programs
               `(foo-mode .
                  ,(eglot-alternatives '("fools" ("bazls" "--foo"))))))

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> Date: Thu, 10 Nov 2022 17:35:55 +0200
> Cc: arash@gnu.org, theophilusx@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> If you just have a command (to be used interactively, right?), it will 
> only affect the value in the current session.

The command could do whatever we want it to do, including writing to
the user's init file, if the user wants that.  And if the command is
invoked from the init file, it will affect the current session, in
every session.

> defcustom provides a facility to persist the changes between sessions.

So do commands.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: arash@gnu.org,  theophilusx@gmail.com,  emacs-devel@gnu.org
> Date: Thu, 10 Nov 2022 15:38:32 +0000
> 
> >> I agree, but a command is a not a solution IMO: this belongs in the 
> >> user's configuration file. 
> >
> > ??? The user init file can call the command, cannot it?
> 
> Yeah, but then why make it a command, i.e. an interactive function?

For the same reason a defcustom doesn't force the user to make the
changes permanent.

> If
> whatever you're proposing is going to be called interactively, it won't
> persist through sessions.

It could, but I don't see why it must.  There's nothing magic in
saving a customization.  And the same command could be put in the
user's init file, in which case no need for it to write anything.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 10 Nov 2022 10:25:11 +0000
> Cc: Arash Esbati <arash@gnu.org>, emacs-devel@gnu.org
> 
> > > The addition would look like this:
> > [...]
> > Thanks, I'll let João comment on this.
> 
> The addition is fine.

I installed that, thanks.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: arash@gnu.org,  theophilusx@gmail.com,  emacs-devel@gnu.org
> Date: Thu, 10 Nov 2022 15:38:32 +0000
> 
> >> A good docstring (can it  be improved?) containing good descriptions 
> >> and some ready-made recipes is the current and best approach to these 
> >> things.
> >
> > Sorry, but I disagree.  No doc string can reasonably teach users
> > advanced Lisp.  That's a non-starter.
> 
> Funny, that's how I learned all my Elisp until I learned of the Elisp
> manual a few years ago.  And this is not "advanced" stuff at all.  This
> is all that's needed:

We are miscommunicating: I thought you were suggesting to explain the
structure of eglot-server-programs's data structure and the methods of
augmenting it in the doc string.  (You could have guessed that I
didn't mean such a simple addition to a doc string, the moment you
wrote the "funny" and the "advanced" parts of your reply.  You know me
well enough for that.)

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Thu, 10 Nov 2022 10:25:11 +0000
> Cc: Arash Esbati <arash@gnu.org>, emacs-devel@gnu.org
> 
> But I disagree that we should endeavor to be exhaustive in that list
> and think that tweaking the variable is a rare event.

This is a project-level goal of the Emacs project, and as such is up
to the maintainers to set and pursue.  Even if the maintainers are
wrong, in your opinion.

Re: Making `eglot-server-programs' a custom variable?

[Stefan Monnier]

João Távora [2022-11-10 15:21:46] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> Arash, I think your suggestion of recommending `with-eval-after-load`
>>> is pertinent and should be added to the manual.
>> I don't have an objection to that but I consider uses of
>> `with-eval-after-load` as hints that maybe we should do things
>> differently.
> Why?  This is in a user's init file.  I can more or less see that
> argument for inter-library dependencies.  But here, w-e-a-load is
> exactly what's needed.  I use it all the time in my config.

I have 4 uses of it in my ~75kB init file.

It's a great tool, but still one where I consider that imposing it on
the user is a sign of a deficiency (a bit like advice, tho much less
severely so).

That doesn't mean we have to find a way to avoid its use, just that it
would be nice to find such a way.

>>> Eli, even though we provide a healthy dose of built-in server invocations
>>> in that variable, we can't and shouldn't aim at being exhaustive.
>> Seeing the wild number of LSP servers available for some languages, I'd
>> agree, sadly.
> I don't think this is sad :-)

In my experience it's usually a reflection of the fact the LSP protocol
leaves a bit too much control to the server, which should focus on
providing "impartial info" about the language, leaving more of the
choices of how to use that info somewhere in the hands of the end user.

"Choose an LSP server" seems to me like a very poor way to customize the
behavior of the server.

IOW the design of the LSP protocol is not "Emacs-y" enough :-)

>> Maybe to reduce the problem we should allow multiple entries per
>> major mode and use the first that works, without needing to go through
>> `eglot-alternatives`?
>
> Again, why?  Why add more semantics to an already complicated variable
> when the functional eglot-alternatives plugin works fine?  Furthermore,
> I'd like to this particular configuration for a future
> multiple-simultaneous-server-in-one-mode idea.

I'm just suggesting directions.  I don't claim they're the right answer.
My impression is that this variable is currently both "too complex" and
"not flexible enough".  Don't get me wrong, it's good enough for now.

But no, I don't have a good replacement to suggest.  My gut just tells
me that there is a better arrangement.

>> [ I'll note in passing that it's common to use strings for TCP port
>>   numbers, especially once they are standardized enough to appear in
>>   /etc/services, so maybe the syntax for that TCP connection should
>>   replace (HOST PORT ...) with something like (:tcp HOST PORT ...).  ]
> If you can make that change without breaking backward compatibility, I
> have no objections.

I think currently an entry cannot start with a keyword, so te syntax
I propose wouldn't break compatibility.  But it would be different in
style from what's there currently, making it more complex for the user.
So I think it would only make sense if/when we make other changes at the
same time.


        Stefan

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Arash Esbati <arash@gnu.org> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> How would making `eglot-server-programs' help in that respect?
>
> If it's meant to be extended by users, then one could use the custom
> interface for it, not add-to-list then.
>
>> If the `defvar' were just to be replaced by a `defcustom', the result
>> would still just be a variable, that couldn't be `add-to-list'ed
>> before it is loaded.
>
> This will depend on the implementation.  Say the current content of
> `eglot-server-programs' is in `eglot-server-programs-builtin' and
> `eglot-server-programs' is a custom variable, and you have a function
> like this in eglot.el:
>
> (defun eglot-server-programs ()
>   (append eglot-server-programs
>           eglot-server-programs-builtin))
>
> then a user can just setq the custom eglot-server-programs without being
> worry about the rest.

Honestly, I think that there should be a command like `add-to-option'
that can handle these things automatically and defer loading if
possible.  There are plenty of cases where you would either do something
like this, or have a ...-user-list that is appended together with some
build-in list, and even though the actual problem is shared among all
user options.

Re: Making `eglot-server-programs' a custom variable?

[Stefan Monnier]

Philip Kaludercic [2022-11-10 17:47:36] wrote:
> Honestly, I think that there should be a command like `add-to-option'
> that can handle these things automatically and defer loading if
> possible.

Might be a good choice.  We could even change `add-to-list` to do that.

I suspect it would not be able to use `with-eval-after-load` because it
wouldn't know which file holds/defines that var.  But we could make it
work for Custom vars, OTOH.


        Stefan

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Philip Kaludercic [2022-11-10 17:47:36] wrote:
>> Honestly, I think that there should be a command like `add-to-option'
>> that can handle these things automatically and defer loading if
>> possible.
>
> Might be a good choice.  We could even change `add-to-list` to do that.
>
> I suspect it would not be able to use `with-eval-after-load` because it
> wouldn't know which file holds/defines that var.  But we could make it
> work for Custom vars, OTOH.

I have the hunch that modifying `add-to-list' along these lines could
break a lot of subtle programs, especially when they use `add-to-list'
in place of `push'.

Re: Making `eglot-server-programs' a custom variable?

[Dmitry Gutov]

On 10.11.2022 18:50, Eli Zaretskii wrote:
>> Date: Thu, 10 Nov 2022 17:35:55 +0200
>> Cc:arash@gnu.org,theophilusx@gmail.com,emacs-devel@gnu.org
>> From: Dmitry Gutov<dgutov@yandex.ru>
>>
>> If you just have a command (to be used interactively, right?), it will
>> only affect the value in the current session.
> The command could do whatever we want it to do, including writing to
> the user's init file, if the user wants that.  And if the command is
> invoked from the init file, it will affect the current session, in
> every session.

The user can normally choose where the customizations are stored (inside 
the init file, or in some file alongside it).

If you're suggesting the proposed command reimplements that logic (and 
honors the value of custom-file? ignores it?), that seems like a poor 
choice in my POV.

No opinion on the rest of this discussion.

Re: Making `eglot-server-programs' a custom variable?

[Stefan Monnier]

>> I suspect it would not be able to use `with-eval-after-load` because it
>> wouldn't know which file holds/defines that var.  But we could make it
>> work for Custom vars, OTOH.
>
> I have the hunch that modifying `add-to-list' along these lines could
> break a lot of subtle programs,

I'd imagine the change would only affect the case where the var is
unbound, i.e. where we currently signal an error.  For that reason, it
seems safe enough.

> especially when they use `add-to-list' in place of `push'.

It should be safe enough even for such bad-style cases.


        Stefan

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> Date: Thu, 10 Nov 2022 20:22:00 +0200
> Cc: joaotavora@gmail.com, arash@gnu.org, theophilusx@gmail.com,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 10.11.2022 18:50, Eli Zaretskii wrote:
> >> Date: Thu, 10 Nov 2022 17:35:55 +0200
> >> Cc:arash@gnu.org,theophilusx@gmail.com,emacs-devel@gnu.org
> >> From: Dmitry Gutov<dgutov@yandex.ru>
> >>
> >> If you just have a command (to be used interactively, right?), it will
> >> only affect the value in the current session.
> > The command could do whatever we want it to do, including writing to
> > the user's init file, if the user wants that.  And if the command is
> > invoked from the init file, it will affect the current session, in
> > every session.
> 
> The user can normally choose where the customizations are stored (inside 
> the init file, or in some file alongside it).

When they use defcustoms? only globally, for all the defcustoms, not
individually for each defcustom.

> If you're suggesting the proposed command reimplements that logic (and 
> honors the value of custom-file? ignores it?), that seems like a poor 
> choice in my POV.

There's no need to reimplement anything, because we can use the same
APIs that Custom uses.  For example Options->Save options from the
menu bar already does that.

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> I suspect it would not be able to use `with-eval-after-load` because it
>>> wouldn't know which file holds/defines that var.  But we could make it
>>> work for Custom vars, OTOH.
>>
>> I have the hunch that modifying `add-to-list' along these lines could
>> break a lot of subtle programs,
>
> I'd imagine the change would only affect the case where the var is
> unbound, i.e. where we currently signal an error.  For that reason, it
> seems safe enough.

If that is all the function does, I think it should be safe.  Other
questions are if an error should be raised when there is a type-mismatch
or if the custom setter/getter ought to be used.

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

> (You could have guessed that I didn't mean such a simple addition to a
> doc string, the moment you wrote the "funny" and the "advanced" parts
> of your reply.  You know me well enough for that.)

Sorry, I just found the idea that I was proposing to teach advanced
Elisp in docstrings both an exaggeration and oddly familiar.  Because it
really speaks to how I learned Elisp with Emacs's excellent docstrings.

This is why I wrote "funny": as in "curious", not as in "ridiculous".

But, pragmatically, I mean only this:

* A reference to eglot-alternatives in eglot-server-programs, just like
  the one I proposed.

* A good docstring for eglot-alternatives.  I think the current one is
  fine (but suggestion are welcome).

* An additional simple eglot-alternatives example in the Eglot manual.

I've done pushed these ideas to master.

João

Re: Making `eglot-server-programs' a custom variable?

[Augusto Stoffel]

On Thu, 10 Nov 2022 at 10:18, Arash Esbati wrote:

> Currently, "digestif" doesn't run on Windows

FWIW, this should be easy to fix by a volunteer with access to a Windows
machine.  I might also be able to fix things if a detailed bug report is
provided.  (In fact, I wouldn't be surprised if it just works by now.)

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Date: Thu, 10 Nov 2022 10:25:11 +0000
>> Cc: Arash Esbati <arash@gnu.org>, emacs-devel@gnu.org
>> 
>> But I disagree that we should endeavor to be exhaustive in that list
>> and think that tweaking the variable is a rare event.
>
> This is a project-level goal of the Emacs project, and as such is up
> to the maintainers to set and pursue.  Even if the maintainers are
> wrong, in your opinion.

eglot-server-programs grew formidably over the last 4 years, and I have
no objections to keep growing it.  Being exhaustive is going to be
reasonably hard, because so many servers will continue to appear.  Even
faster than programming languages.  And some servers are just abandoned,
which is another annoyance.  And we should not assume the user won't
want to tweak to add flags or servers we don't know about.  That would
contradicts my lengthy experience with the uses of this variable.

Anyway, I prefer to focus my limited time on other aspects of Eglot, but
you're certainly not "wrong" if you make adding new servers a priority.

João

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Why?  This is in a user's init file.  I can more or less see that
>> argument for inter-library dependencies.  But here, w-e-a-load is
>> exactly what's needed.  I use it all the time in my config.
>
> I have 4 uses of it in my ~75kB init file.

Heh, I have it in almost every one of my ~75-750 small elisp files, each
one more or less dedicated to configuring a package that I may or may
not use for 7500 years.

> That doesn't mean we have to find a way to avoid its use, just that it
> would be nice to find such a way.

I think with-eval-after-load sits nicely at the intersection of lazy
autoloads and user Elisp snippets.  Is there another tool that does
this?  use-package, maybe?  But won't that use the same thing
underneath?  Anyway I tend to prefer things that do one thing well and
don't make me learn new DSLs.  with-eval-after-load hasn't let me down.

> "Choose an LSP server" seems to me like a very poor way to customize the
> behavior of the server.
>
> IOW the design of the LSP protocol is not "Emacs-y" enough :-)

Ah.  Yes.  You're probably right about that, how could you not be ;-) ?
I also think it's more about these servers using different backend
strategies themselves.  Last I looked at the C++ server landscape, all
of them provided more or less the same, but consuming different amount
resources.  There's also lots of forking going on.  I think this is
good.

>>> Maybe to reduce the problem we should allow multiple entries per
>>> major mode and use the first that works, without needing to go through
>>> `eglot-alternatives`?
>>
>> Again, why?  Why add more semantics to an already complicated variable
>> when the functional eglot-alternatives plugin works fine?  Furthermore,
>> I'd like to this particular configuration for a future
>> multiple-simultaneous-server-in-one-mode idea.
>
> I'm just suggesting directions.  I don't claim they're the right answer.
> My impression is that this variable is currently both "too complex" and
> "not flexible enough".  Don't get me wrong, it's good enough for now.
>
> But no, I don't have a good replacement to suggest.  My gut just tells
> me that there is a better arrangement.

It's not a very pretty variable, I admit, but it's good enough for now
and could be a lot worse :-)

I once thought it would be a good idea to have a buffer-local variable
eglot-server-program (singular) that major-modes could set as a stronger
suggestion of what to use.  So a future smc++-mode could be based
entirely on tree-sitter for syntax-highlighting and some good C++ server
for everything else, including some LSP-backed smc++-frob-kittens
command that isn't in eglot.el.  The "good C++ server" would be set in
eglot-server-program, called from smc++-mode.  Maybe that "good server"
would even be bundled with the smc++-mode package so that M-x smc++-mode
would probably start Eglot by default.  Can you ask your gut if this is
an acceptable arrangement?

João

Re: Making `eglot-server-programs' a custom variable?

[Yuri Khan]

On Fri, 11 Nov 2022 at 04:45, João Távora <joaotavora@gmail.com> wrote:

> eglot-server-programs grew formidably over the last 4 years, and I have
> no objections to keep growing it.  Being exhaustive is going to be
> reasonably hard, because so many servers will continue to appear.  Even
> faster than programming languages.  And some servers are just abandoned,
> which is another annoyance.  And we should not assume the user won't
> want to tweak to add flags or servers we don't know about.  That would
> contradicts my lengthy experience with the uses of this variable.

I might be using Eglot unconventionally, but this is how I do it:

There are multiple projects, with their specific compiler versions and
library dependencies. If I installed all of that directly on my
machine, I’d have version hell. Therefore, for each project, I build a
docker image with all the external dependencies packed in. I mount my
source tree into a container and invoke the build in it. Emacs, on the
other hand, runs on the host machine. Thus, my compile-command is
defined in .dir-locals.el and starts with “docker run --rm”.

Similarly, I put the language server(s) into the same container image.
This way, it has access to external library sources. So, my
eglot-server-programs is overridden in each project to run a script
that starts e.g. clangd in a container spun up from the
project-specific development tools image.

I do not bother writing out a copy of the full default value of
eglot-server-programs with my dockerized server substituted or
prepended. The expectation is that each project only uses one or no
more than a handful of language servers.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Thu, 10 Nov 2022 21:45:59 +0000
> 
> > This is a project-level goal of the Emacs project, and as such is up
> > to the maintainers to set and pursue.  Even if the maintainers are
> > wrong, in your opinion.
> 
> eglot-server-programs grew formidably over the last 4 years, and I have
> no objections to keep growing it.  Being exhaustive is going to be
> reasonably hard, because so many servers will continue to appear.  Even
> faster than programming languages.  And some servers are just abandoned,
> which is another annoyance.  And we should not assume the user won't
> want to tweak to add flags or servers we don't know about.  That would
> contradicts my lengthy experience with the uses of this variable.
> 
> Anyway, I prefer to focus my limited time on other aspects of Eglot, but
> you're certainly not "wrong" if you make adding new servers a priority.

As long as you don't start objecting to adding more servers, there's
no contradiction between our goals.

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> On Fri, 11 Nov 2022 at 04:45, João Távora <joaotavora@gmail.com> wrote:
>
>> eglot-server-programs grew formidably over the last 4 years, and I have
>> no objections to keep growing it.  Being exhaustive is going to be
>> reasonably hard, because so many servers will continue to appear.  Even
>> faster than programming languages.  And some servers are just abandoned,
>> which is another annoyance.  And we should not assume the user won't
>> want to tweak to add flags or servers we don't know about.  That would
>> contradicts my lengthy experience with the uses of this variable.
>
> I might be using Eglot unconventionally, but this is how I do it:

Dir-local eglot-server-programs sounds fine, as long as the project
files you intend for Eglot to manage are contained entirely inside one
directory, which they usually are.

João

Re: Making `eglot-server-programs' a custom variable?

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 10 Nov 2022 21:45:59 +0000
>> 
>> > This is a project-level goal of the Emacs project, and as such is up
>> > to the maintainers to set and pursue.  Even if the maintainers are
>> > wrong, in your opinion.
>> 
>> eglot-server-programs grew formidably over the last 4 years, and I have
>> no objections to keep growing it.  Being exhaustive is going to be
>> reasonably hard, because so many servers will continue to appear.  Even
>> faster than programming languages.  And some servers are just abandoned,
>> which is another annoyance.  And we should not assume the user won't
>> want to tweak to add flags or servers we don't know about.  That would
>> contradicts my lengthy experience with the uses of this variable.
>> 
>> Anyway, I prefer to focus my limited time on other aspects of Eglot, but
>> you're certainly not "wrong" if you make adding new servers a priority.
>
> As long as you don't start objecting to adding more servers, there's
> no contradiction between our goals.

I don't think I ever objected to it, and I don't plan on starting.
Personally, I use a tiny fraction of that list, and it's been gleefully
and carefully curated by many of Eglot's smaller contributors, to which
I am very thankful.

Re: Making `eglot-server-programs' a custom variable?

[Arash Esbati]

Augusto Stoffel <arstoffel@gmail.com> writes:

> On Thu, 10 Nov 2022 at 10:18, Arash Esbati wrote:
>
>> Currently, "digestif" doesn't run on Windows
>
> FWIW, this should be easy to fix by a volunteer with access to a Windows
> machine.  I might also be able to fix things if a detailed bug report is
> provided.  (In fact, I wouldn't be surprised if it just works by now.)

It still doesn't work.  I ran the installation script with this change
in order to use lua (v5.4.4) and not texlua:

  # Name of the Lua interpreter to use
  LUA=lua

This is what I get in a ming64 shell:

--8<---------------cut here---------------start------------->8---
Digestif not found in /z/myhome/.digestif, fetching it now
Cloning into '/z/myhome/.digestif'...
remote: Enumerating objects: 70, done.
remote: Counting objects: 100% (70/70), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 70 (delta 2), reused 20 (delta 0), pack-reused 0
Receiving objects: 100% (70/70), 644.32 KiB | 3.95 MiB/s, done.
Resolving deltas: 100% (2/2), done.
Done! If you are running this interactively, press Control-C to quit.
The system cannot find the specified path.
The system cannot find the specified path.
Content-Length: 175

[It hangs here and now I hit C-c and the rest follows:]

{"error":{"message":"z:/myhome/.digestif/digestif\\util.lua:569: bad argument #2 to 'match' (string expected, got nil)","code":-32700},"jsonrpc":"2.0","id":null,"result":false}C:\pathto\mingw64\bin\lua.exe: z:/myhome/.digestif/digestif\langserver.lua:388: interrupted!
stack traceback:
        [C]: in function 'io.write'
        z:/myhome/.digestif/digestif\langserver.lua:388: in upvalue 'write_msg'
        z:/myhome/.digestif/digestif\langserver.lua:405: in upvalue 'rpc_send'
        z:/myhome/.digestif/digestif\langserver.lua:418: in upvalue 'rpc_receive'
        z:/myhome/.digestif/digestif\langserver.lua:430: in upvalue 'process_request'
        z:/myhome/.digestif/digestif\langserver.lua:538: in function 'digestif.langserver.main'
        z:/myhome/.digestif/bin/digestif:2: in main chunk
        [C]: in ?
--8<---------------cut here---------------end--------------->8---

Please feel free to contact me directly if I can provide more input
since this is off-topic on emacs-devel.

Best, Arash

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Fri, 11 Nov 2022 09:12:17 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: João Távora <joaotavora@gmail.com>
> >> Cc: emacs-devel@gnu.org
> >> Date: Thu, 10 Nov 2022 21:45:59 +0000
> >> 
> >> > This is a project-level goal of the Emacs project, and as such is up
> >> > to the maintainers to set and pursue.  Even if the maintainers are
> >> > wrong, in your opinion.
> >> 
> >> eglot-server-programs grew formidably over the last 4 years, and I have
> >> no objections to keep growing it.  Being exhaustive is going to be
> >> reasonably hard, because so many servers will continue to appear.  Even
> >> faster than programming languages.  And some servers are just abandoned,
> >> which is another annoyance.  And we should not assume the user won't
> >> want to tweak to add flags or servers we don't know about.  That would
> >> contradicts my lengthy experience with the uses of this variable.
> >> 
> >> Anyway, I prefer to focus my limited time on other aspects of Eglot, but
> >> you're certainly not "wrong" if you make adding new servers a priority.
> >
> > As long as you don't start objecting to adding more servers, there's
> > no contradiction between our goals.
> 
> I don't think I ever objected to it

And I don't think I ever said you did.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 11 Nov 2022 11:05:19 +0100
> 
> This is what I get in a ming64 shell:
> 
> --8<---------------cut here---------------start------------->8---
> Digestif not found in /z/myhome/.digestif, fetching it now
> Cloning into '/z/myhome/.digestif'...

This sounds like some MSYS2 executable is involved, which might get
you in trouble due to different file-name syntax and other
incompatibilities between MSYS2 and native Windows programs.

> remote: Enumerating objects: 70, done.
> remote: Counting objects: 100% (70/70), done.
> remote: Compressing objects: 100% (68/68), done.
> remote: Total 70 (delta 2), reused 20 (delta 0), pack-reused 0
> Receiving objects: 100% (70/70), 644.32 KiB | 3.95 MiB/s, done.
> Resolving deltas: 100% (2/2), done.
> Done! If you are running this interactively, press Control-C to quit.
> The system cannot find the specified path.
> The system cannot find the specified path.

Which program this tries to invoke and with what command-line
arguments?

Re: Making `eglot-server-programs' a custom variable?

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Arash Esbati <arash@gnu.org>
>> Cc: emacs-devel@gnu.org
>> Date: Fri, 11 Nov 2022 11:05:19 +0100
>> 
>> This is what I get in a ming64 shell:
>> 
>> --8<---------------cut here---------------start------------->8---
>> Digestif not found in /z/myhome/.digestif, fetching it now
>> Cloning into '/z/myhome/.digestif'...
>
> This sounds like some MSYS2 executable is involved, which might get
> you in trouble due to different file-name syntax and other
> incompatibilities between MSYS2 and native Windows programs.

This part comes from git which is indeed a MSYS2 executable and
actually succeeds (see below) and the repo is cloned to my HD.

>> remote: Enumerating objects: 70, done.
>> remote: Counting objects: 100% (70/70), done.
>> remote: Compressing objects: 100% (68/68), done.
>> remote: Total 70 (delta 2), reused 20 (delta 0), pack-reused 0
>> Receiving objects: 100% (70/70), 644.32 KiB | 3.95 MiB/s, done.
>> Resolving deltas: 100% (2/2), done.
>> Done! If you are running this interactively, press Control-C to quit.
>> The system cannot find the specified path.
>> The system cannot find the specified path.
>
> Which program this tries to invoke and with what command-line
> arguments?

This is lua.exe which is installed as mingw-w64-x86_64-lua:

-> which lua
/mingw64/bin/lua

The installation script is here:
https://raw.githubusercontent.com/astoff/digestif/master/scripts/digestif

The problematic part can be condensed to this I think:

DIGESTIF_HOME="$HOME/.digestif"
LUA=lua

export LUA_PATH="$DIGESTIF_HOME/?.lua"
export DIGESTIF_PRERELEASE="$(git -C "$DIGESTIF_HOME" rev-parse --short HEAD)"

"$LUA" "$DIGESTIF_HOME/bin/digestif" "$@"

Best, Arash

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: arstoffel@gmail.com,  emacs-devel@gnu.org
> Date: Fri, 11 Nov 2022 13:22:53 +0100
> 
> >> Cloning into '/z/myhome/.digestif'...
> >
> > This sounds like some MSYS2 executable is involved, which might get
> > you in trouble due to different file-name syntax and other
> > incompatibilities between MSYS2 and native Windows programs.
> 
> This part comes from git which is indeed a MSYS2 executable and
> actually succeeds (see below) and the repo is cloned to my HD.

Why are using an MSYS2 build of Git, when a native Windows build
exists and is fully functional?

> >> The system cannot find the specified path.
> >> The system cannot find the specified path.
> >
> > Which program this tries to invoke and with what command-line
> > arguments?
> 
> This is lua.exe which is installed as mingw-w64-x86_64-lua:
> 
> -> which lua
> /mingw64/bin/lua
> 
> The installation script is here:
> https://raw.githubusercontent.com/astoff/digestif/master/scripts/digestif
> 
> The problematic part can be condensed to this I think:
> 
> DIGESTIF_HOME="$HOME/.digestif"
> LUA=lua
> 
> export LUA_PATH="$DIGESTIF_HOME/?.lua"
> export DIGESTIF_PRERELEASE="$(git -C "$DIGESTIF_HOME" rev-parse --short HEAD)"
> 
> "$LUA" "$DIGESTIF_HOME/bin/digestif" "$@"

So which of the files/directories involved is the one that "the system
cannot find"?  That message comes from cmd.exe, AFAIK.

Maybe it's LUA_PATH, because Windows doesn't allow question marks in
file names.

Re: Making `eglot-server-programs' a custom variable?

[Augusto Stoffel]

On Fri, 11 Nov 2022 at 14:33, Eli Zaretskii wrote:

>> export LUA_PATH="$DIGESTIF_HOME/?.lua"
>> export DIGESTIF_PRERELEASE="$(git -C "$DIGESTIF_HOME" rev-parse --short HEAD)"
>> 
>> "$LUA" "$DIGESTIF_HOME/bin/digestif" "$@"
>
> So which of the files/directories involved is the one that "the system
> cannot find"?  That message comes from cmd.exe, AFAIK.
>
> Maybe it's LUA_PATH, because Windows doesn't allow question marks in
> file names.

The LUA_PATH is (probably) fine; the question mark is interpreted by Lua
and means that package x.y.z is to be found at
"$DIGESTIF_HOME/x/y/z.lua".

Re: Making `eglot-server-programs' a custom variable?

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> Why are using an MSYS2 build of Git, when a native Windows build
> exists and is fully functional?

I just took the one which was available through pacman, and I saw only

  msys/git

There is no

  mingw64/mingw-w64-x86_64-git

AFAICT.  Or are you thinking about 'Git for Windows' releases?  And
until now, I had no problems with MSYS2 git.

> So which of the files/directories involved is the one that "the system
> cannot find"?

I can't tell where that comes from.

> That message comes from cmd.exe, AFAIK.

Yes, I get the same result when doing the excercise within cmd.exe.

Best, Arash

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: arstoffel@gmail.com,  emacs-devel@gnu.org
> Date: Fri, 11 Nov 2022 14:48:30 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Why are using an MSYS2 build of Git, when a native Windows build
> > exists and is fully functional?
> 
> I just took the one which was available through pacman, and I saw only
> 
>   msys/git
> 
> There is no
> 
>   mingw64/mingw-w64-x86_64-git
> 
> AFAICT.  Or are you thinking about 'Git for Windows' releases?

Yes.

> And until now, I had no problems with MSYS2 git.

That's just sheer luck, IME.

> > So which of the files/directories involved is the one that "the system
> > cannot find"?
> 
> I can't tell where that comes from.

Understanding that will probably allow fixing the problem, so I'd try
very hard if I were you or Augusto.

Re: Making `eglot-server-programs' a custom variable?

[Brian Cully via Emacs development discussions.]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> I might be using Eglot unconventionally, but this is how I do it:
>
> There are multiple projects, with their specific compiler versions and
> library dependencies. If I installed all of that directly on my
> machine, I’d have version hell. Therefore, for each project, I build a
> docker image with all the external dependencies packed in. I mount my
> source tree into a container and invoke the build in it. Emacs, on the
> other hand, runs on the host machine. Thus, my compile-command is
> defined in .dir-locals.el and starts with “docker run --rm”.

This is also how I use eglot. I only do my coding inside of explicitly
versioned and reproducible containers.

FWIW, eglot natively supports Tramp, meaning it will spawn the LSP on
the remote, as long as you are also accessing those files
remotely. There's no need to change the value of ‘eglot-server-programs’
in this case.

-bjc

Re: Making `eglot-server-programs' a custom variable?

[Jim Porter]

On 11/10/2022 9:47 AM, Philip Kaludercic wrote:
> Honestly, I think that there should be a command like `add-to-option'
> that can handle these things automatically and defer loading if
> possible.  There are plenty of cases where you would either do something
> like this, or have a ...-user-list that is appended together with some
> build-in list, and even though the actual problem is shared among all
> user options.

Assuming I understand what you mean, I agree that this would be very 
useful. I think it would be super-helpful for there to be a way in the 
Customize interface to indicate that you want to add A, B, C to a 
defcustom list, and/or remove X, Y, Z. For another example where this 
would be helpful, see bug#54977.

(For people who build their Emacs configs by writing Elisp, this 
probably isn't too big a deal, since they probably know enough to do the 
right thing, but it couldn't hurt to make this easier in Elisp too. 
Although use-package already makes this about as easy as can be, since 
you can do all this in a ':config' block, and it Just Works.)

Re: Making `eglot-server-programs' a custom variable?

[chad]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On Fri, Nov 11, 2022 at 10:48 PM Jim Porter <jporterbugs@gmail.com> wrote:

> Although use-package already makes this about as easy as can be, since
> you can do all this in a ':config' block, and it Just Works.
>

This sort of behavior is one of the reasons I asked earlier about the
possibility of getting use-package included for 29; there's a lot of
cargo-cult configuration going on out there in places like reddit and
Planet Emacs, and much of it is enabled by use-package, so it would be good
(imho) to make that prerequisite step easier.

~Chad

[-- Attachment #2: Type: text/html, Size: 920 bytes --]

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

chad <yandros@gmail.com> writes:

> On Fri, Nov 11, 2022 at 10:48 PM Jim Porter <jporterbugs@gmail.com> wrote:
>
>> Although use-package already makes this about as easy as can be, since
>> you can do all this in a ':config' block, and it Just Works.
>
> This sort of behavior is one of the reasons I asked earlier about the
> possibility of getting use-package included for 29; there's a lot of
> cargo-cult configuration going on out there in places like reddit and
> Planet Emacs, and much of it is enabled by use-package, so it would be good
> (imho) to make that prerequisite step easier.

See <87iljk1voy.fsf@posteo.net>.

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Jim Porter <jporterbugs@gmail.com> writes:

> On 11/10/2022 9:47 AM, Philip Kaludercic wrote:
>> Honestly, I think that there should be a command like `add-to-option'
>> that can handle these things automatically and defer loading if
>> possible.  There are plenty of cases where you would either do something
>> like this, or have a ...-user-list that is appended together with some
>> build-in list, and even though the actual problem is shared among all
>> user options.
>
> Assuming I understand what you mean, I agree that this would be very
> useful. I think it would be super-helpful for there to be a way in the
> Customize interface to indicate that you want to add A, B, C to a
> defcustom list, and/or remove X, Y, Z. For another example where this
> would be helpful, see bug#54977.
>
> (For people who build their Emacs configs by writing Elisp, this
> probably isn't too big a deal, since they probably know enough to do
> the right thing, but it couldn't hurt to make this easier in Elisp
> too. Although use-package already makes this about as easy as can be,
> since you can do all this in a ':config' block, and it Just Works.)

Just for the sake of completeness, I'd like to mention what setup.el
also provides.

The `:option' macro (which is comparable to `:custom' in use-package)
can take a complex option name.  The default is something like

    (:option c-default-style '((java-mode . "java")
                               (awk-mode  . "awk")
                               (other     . "k&r")))
                               
but you could also use

    (:option (prepend c-default-style) '(other . "k&r"))

to load the variable and then prepend a value to a list.  In retrospect,
I don't like the syntax that much, and the fact that `:option' calls
`custom-load-symbol' unconditionally can really slow down a
configuration.  This last thing is something that an `add-to-option` --
be it a standalone function or an extension of `add-to-list' must avoid IMO.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> Date: Fri, 11 Nov 2022 19:47:52 -0800
> Cc: emacs-devel <emacs-devel@gnu.org>, João Távora
>  <joaotavora@gmail.com>
> From: Jim Porter <jporterbugs@gmail.com>
> 
> I think it would be super-helpful for there to be a way in the 
> Customize interface to indicate that you want to add A, B, C to a 
> defcustom list, and/or remove X, Y, Z.

We already have that, AFAIU what you mean: see, for example

  M-x customize-variable RET image-load-path RET

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Fri, 11 Nov 2022 19:47:52 -0800
>> Cc: emacs-devel <emacs-devel@gnu.org>, João Távora
>>  <joaotavora@gmail.com>
>> From: Jim Porter <jporterbugs@gmail.com>
>> 
>> I think it would be super-helpful for there to be a way in the 
>> Customize interface to indicate that you want to add A, B, C to a 
>> defcustom list, and/or remove X, Y, Z.
>
> We already have that, AFAIU what you mean: see, for example
>
>   M-x customize-variable RET image-load-path RET

My issue here is that while you can modify the list, when saved you will
store the entire modified list, and no the modifications you made on the
base variable.

E.g. considering `eglot-server-programs', just because I want to an
additional server entry, I don't want to fix the list of servers in a
`custom-set-variables' block and ignore any future updates that have
nothing to do with the additional entry.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Jim Porter <jporterbugs@gmail.com>,  arash@gnu.org,
>   emacs-devel@gnu.org,  joaotavora@gmail.com
> Date: Sat, 12 Nov 2022 08:03:51 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Date: Fri, 11 Nov 2022 19:47:52 -0800
> >> Cc: emacs-devel <emacs-devel@gnu.org>, João Távora
> >>  <joaotavora@gmail.com>
> >> From: Jim Porter <jporterbugs@gmail.com>
> >> 
> >> I think it would be super-helpful for there to be a way in the 
> >> Customize interface to indicate that you want to add A, B, C to a 
> >> defcustom list, and/or remove X, Y, Z.
> >
> > We already have that, AFAIU what you mean: see, for example
> >
> >   M-x customize-variable RET image-load-path RET
> 
> My issue here is that while you can modify the list, when saved you will
> store the entire modified list, and no the modifications you made on the
> base variable.

The way to do this is to have a basic value as a defvar, and store the
corrections to that in a defcustom.

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Jim Porter <jporterbugs@gmail.com>,  arash@gnu.org,
>>   emacs-devel@gnu.org,  joaotavora@gmail.com
>> Date: Sat, 12 Nov 2022 08:03:51 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Date: Fri, 11 Nov 2022 19:47:52 -0800
>> >> Cc: emacs-devel <emacs-devel@gnu.org>, João Távora
>> >>  <joaotavora@gmail.com>
>> >> From: Jim Porter <jporterbugs@gmail.com>
>> >> 
>> >> I think it would be super-helpful for there to be a way in the 
>> >> Customize interface to indicate that you want to add A, B, C to a 
>> >> defcustom list, and/or remove X, Y, Z.
>> >
>> > We already have that, AFAIU what you mean: see, for example
>> >
>> >   M-x customize-variable RET image-load-path RET
>> 
>> My issue here is that while you can modify the list, when saved you will
>> store the entire modified list, and no the modifications you made on the
>> base variable.
>
> The way to do this is to have a basic value as a defvar, and store the
> corrections to that in a defcustom.

But in that case you need to load the library to get the default value,
in which case you might as well store the default value in the
defcustom.  Also, I don't think this helps people who use the Customize
interface?

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>   joaotavora@gmail.com
> Date: Sat, 12 Nov 2022 08:45:38 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> My issue here is that while you can modify the list, when saved you will
> >> store the entire modified list, and no the modifications you made on the
> >> base variable.
> >
> > The way to do this is to have a basic value as a defvar, and store the
> > corrections to that in a defcustom.
> 
> But in that case you need to load the library to get the default value,

I don't see why.  And even if it's so, why is it a problem to load the
library?

> in which case you might as well store the default value in the
> defcustom.

But you already explained why storing in a defcustom doesn't solve the
problem?  So why are you suggesting to do that anyway?

> Also, I don't think this helps people who use the Customize
> interface?

Why doesn't it help?

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com
>> Date: Sat, 12 Nov 2022 08:45:38 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> My issue here is that while you can modify the list, when saved you will
>> >> store the entire modified list, and no the modifications you made on the
>> >> base variable.
>> >
>> > The way to do this is to have a basic value as a defvar, and store the
>> > corrections to that in a defcustom.
>> 
>> But in that case you need to load the library to get the default value,
>
> I don't see why.  And even if it's so, why is it a problem to load the
> library?

(I'm not just talking about Eglot right now)  If the default value is
defined in an non-autoloaded variable, you have to load the library to
access the value -- otherwise it simply wasn't loaded.

The "issue" here is just that loading everything you want to modify
during initialisation can get slow.

>> in which case you might as well store the default value in the
>> defcustom.
>
> But you already explained why storing in a defcustom doesn't solve the
> problem?  So why are you suggesting to do that anyway?

^^ I am not sure I explained that?  If you have to load the library to
get the default, writing

    (setopt foo (cons 'bar foo))

or

    (setopt foo (cons 'bar foo-default))

doesn't make much of a difference to me.

>> Also, I don't think this helps people who use the Customize
>> interface?
>
> Why doesn't it help?

Maybe I have missed something, if a user option has a `repeat' or
`alist' type, you can't just say "append this and that value to the end
of some other value".  All you get to modify is the entire list, and all
you get to store is the entire list.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>   joaotavora@gmail.com
> Date: Sat, 12 Nov 2022 09:40:23 +0000
> 
> >> > The way to do this is to have a basic value as a defvar, and store the
> >> > corrections to that in a defcustom.
> >> 
> >> But in that case you need to load the library to get the default value,
> >
> > I don't see why.  And even if it's so, why is it a problem to load the
> > library?
> 
> (I'm not just talking about Eglot right now)  If the default value is
> defined in an non-autoloaded variable, you have to load the library to
> access the value -- otherwise it simply wasn't loaded.
> 
> The "issue" here is just that loading everything you want to modify
> during initialisation can get slow.

Whether or not it is necessary to load the library depends on how the
:set function of the defcustom is implemented.  I can see several ways
of implementing it that won't require loading the library right away,
and I'm sure you can see those ways as well.

> >> in which case you might as well store the default value in the
> >> defcustom.
> >
> > But you already explained why storing in a defcustom doesn't solve the
> > problem?  So why are you suggesting to do that anyway?
> 
> ^^ I am not sure I explained that?

You said:

> >   M-x customize-variable RET image-load-path RET
> 
> My issue here is that while you can modify the list, when saved you will
> store the entire modified list, and no the modifications you made on the
> base variable.

To me, this says that storing the value in a defcustom hits that
"issue" to which you were alluding, and for which I proposed a
solution of having the defcustom be an add-on to the baseline value.

> If you have to load the library to
> get the default, writing
> 
>     (setopt foo (cons 'bar foo))
> 
> or
> 
>     (setopt foo (cons 'bar foo-default))
> 
> doesn't make much of a difference to me.

Sorry, I don't see the relevance of setopt to what I was trying to
suggest.

> >> Also, I don't think this helps people who use the Customize
> >> interface?
> >
> > Why doesn't it help?
> 
> Maybe I have missed something, if a user option has a `repeat' or
> `alist' type, you can't just say "append this and that value to the end
> of some other value".  All you get to modify is the entire list, and all
> you get to store is the entire list.

That's a job for the :set function of the defcustom.

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com
>> Date: Sat, 12 Nov 2022 09:40:23 +0000
>> 
>> >> > The way to do this is to have a basic value as a defvar, and store the
>> >> > corrections to that in a defcustom.
>> >> 
>> >> But in that case you need to load the library to get the default value,
>> >
>> > I don't see why.  And even if it's so, why is it a problem to load the
>> > library?
>> 
>> (I'm not just talking about Eglot right now)  If the default value is
>> defined in an non-autoloaded variable, you have to load the library to
>> access the value -- otherwise it simply wasn't loaded.
>> 
>> The "issue" here is just that loading everything you want to modify
>> during initialisation can get slow.
>
> Whether or not it is necessary to load the library depends on how the
> :set function of the defcustom is implemented.  I can see several ways
> of implementing it that won't require loading the library right away,
> and I'm sure you can see those ways as well.

Actually no, I am not sure I do.

>> >> in which case you might as well store the default value in the
>> >> defcustom.
>> >
>> > But you already explained why storing in a defcustom doesn't solve the
>> > problem?  So why are you suggesting to do that anyway?
>> 
>> ^^ I am not sure I explained that?
>
> You said:
>
>> >   M-x customize-variable RET image-load-path RET
>> 
>> My issue here is that while you can modify the list, when saved you will
>> store the entire modified list, and no the modifications you made on the
>> base variable.
>
> To me, this says that storing the value in a defcustom hits that
> "issue" to which you were alluding, and for which I proposed a
> solution of having the defcustom be an add-on to the baseline value.

I see.  The issue is that if I just set the user option directly, say
even before loading the library I overwrite the default value.  But if I
load the library, I will have access to the default value before
modifying it and setting the user option.

>> If you have to load the library to
>> get the default, writing
>> 
>>     (setopt foo (cons 'bar foo))
>> 
>> or
>> 
>>     (setopt foo (cons 'bar foo-default))
>> 
>> doesn't make much of a difference to me.
>
> Sorry, I don't see the relevance of setopt to what I was trying to
> suggest.

There are two fundamental ways to work with user options, right?  Using
the Customise interface and programmatically, e.g. using `setopt'.  The
discussion began with finding a programmatic way of modifying the
default value of a user option that contains some repetitive type
(repeat, alist, plist), which we were calling `add-to-

>> >> Also, I don't think this helps people who use the Customize
>> >> interface?
>> >
>> > Why doesn't it help?
>> 
>> Maybe I have missed something, if a user option has a `repeat' or
>> `alist' type, you can't just say "append this and that value to the end
>> of some other value".  All you get to modify is the entire list, and all
>> you get to store is the entire list.
>
> That's a job for the :set function of the defcustom.

I am not sure I know what you are thinking of, but wouldn't this mean
all user options that have already been marked as having a `repeat' or
`alist' type, that these would now require an additional :set function?

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>   joaotavora@gmail.com
> Date: Sat, 12 Nov 2022 13:46:40 +0000
> 
> >> (I'm not just talking about Eglot right now)  If the default value is
> >> defined in an non-autoloaded variable, you have to load the library to
> >> access the value -- otherwise it simply wasn't loaded.
> >> 
> >> The "issue" here is just that loading everything you want to modify
> >> during initialisation can get slow.
> >
> > Whether or not it is necessary to load the library depends on how the
> > :set function of the defcustom is implemented.  I can see several ways
> > of implementing it that won't require loading the library right away,
> > and I'm sure you can see those ways as well.
> 
> Actually no, I am not sure I do.

Any way that stored the changes of the variable's value in a data
structure whose execution is deferred to when the library is first
loaded.  This includes ` backquoted forms, eval-after-load, mode
hooks, etc.

> > To me, this says that storing the value in a defcustom hits that
> > "issue" to which you were alluding, and for which I proposed a
> > solution of having the defcustom be an add-on to the baseline value.
> 
> I see.  The issue is that if I just set the user option directly, say
> even before loading the library I overwrite the default value.

Once again, I'm talking about the user option being used to _augment_
the default value of a variable.  Such a user option should by default
have a nil value, so setting the value of the option doesn't overwrite
the baseline value of the variable which the option will augment.

I feel there's a misunderstanding here, because I don't see why these
obvious aspects need to be explained.  So let me provide an example as
a possible clarification.

Under my proposal, the variable eglot-server-programs remains a
defvar, and contains the baseline list of the servers.  To customize
the list, users change the value of a separate user option, say,
eglot-user-server-programs.  This user option's value is nil by
default, and it allows users to specify both additions of servers to
the baseline value of eglot-server-programs and removal of servers
from that value.  The :set function of eglot-user-server-programs then
takes care of doing whatever is needed to make sure that the value of
eglot-server-programs is modified according to
eglot-user-server-programs's value when Eglot is started.

> >> Maybe I have missed something, if a user option has a `repeat' or
> >> `alist' type, you can't just say "append this and that value to the end
> >> of some other value".  All you get to modify is the entire list, and all
> >> you get to store is the entire list.
> >
> > That's a job for the :set function of the defcustom.
> 
> I am not sure I know what you are thinking of, but wouldn't this mean
> all user options that have already been marked as having a `repeat' or
> `alist' type, that these would now require an additional :set function?

No, of course not.  I didn't mean any changes to the infrastructure
that we use for Customize and user options in general.  See above, I
hope I now explained what I had in mind.

Re: Making `eglot-server-programs' a custom variable?

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> This is a project-level goal of the Emacs project, and as such is up
> to the maintainers to set and pursue.

I'm not sure if this point is already addressed: With the goal above to
have a large number of server names built-in, maybe it should also be
mentioned if the license of the servers is relevant for their addition.
The one I added is released under GPL v3.0.

Best, Arash

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: João Távora <joaotavora@gmail.com>,
>   emacs-devel@gnu.org
> Date: Sat, 12 Nov 2022 15:44:12 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > This is a project-level goal of the Emacs project, and as such is up
> > to the maintainers to set and pursue.
> 
> I'm not sure if this point is already addressed: With the goal above to
> have a large number of server names built-in, maybe it should also be
> mentioned if the license of the servers is relevant for their addition.
> The one I added is released under GPL v3.0.

I hope we are only adding servers which are Free Software.  If that is
so, why does it matter under which free license they are distributed?
People who care can easily look that up, but most users will not care,
I think.

Re: Making `eglot-server-programs' a custom variable?

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> I hope we are only adding servers which are Free Software.  If that is
> so, why does it matter under which free license they are distributed?

Personally, I agree with the above.  I think this should be spelled out
once and people know what they can suggest for future additions.

Best, Arash

Re: Making `eglot-server-programs' a custom variable?

[Michael Albinus]

Brian Cully via "Emacs development discussions." <emacs-devel@gnu.org>
writes:

Hi,

> FWIW, eglot natively supports Tramp, meaning it will spawn the LSP on
> the remote, as long as you are also accessing those files
> remotely. There's no need to change the value of ‘eglot-server-programs’
> in this case.

If the program is different on your local host and on the remote hosts,
it might be useful to have different values for eglot-server-programs,
depending on where you are.

> -bjc

Best regards, Michael.

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com
>> Date: Sat, 12 Nov 2022 13:46:40 +0000
>> 
>> >> (I'm not just talking about Eglot right now)  If the default value is
>> >> defined in an non-autoloaded variable, you have to load the library to
>> >> access the value -- otherwise it simply wasn't loaded.
>> >> 
>> >> The "issue" here is just that loading everything you want to modify
>> >> during initialisation can get slow.
>> >
>> > Whether or not it is necessary to load the library depends on how the
>> > :set function of the defcustom is implemented.  I can see several ways
>> > of implementing it that won't require loading the library right away,
>> > and I'm sure you can see those ways as well.
>> 
>> Actually no, I am not sure I do.
>
> Any way that stored the changes of the variable's value in a data
> structure whose execution is deferred to when the library is first
> loaded.  This includes ` backquoted forms, eval-after-load, mode
> hooks, etc.
>
>> > To me, this says that storing the value in a defcustom hits that
>> > "issue" to which you were alluding, and for which I proposed a
>> > solution of having the defcustom be an add-on to the baseline value.
>> 
>> I see.  The issue is that if I just set the user option directly, say
>> even before loading the library I overwrite the default value.
>
> Once again, I'm talking about the user option being used to _augment_
> the default value of a variable.  Such a user option should by default
> have a nil value, so setting the value of the option doesn't overwrite
> the baseline value of the variable which the option will augment.

Ok.

> I feel there's a misunderstanding here, because I don't see why these
> obvious aspects need to be explained.  So let me provide an example as
> a possible clarification.

That might very well be possible.

> Under my proposal, the variable eglot-server-programs remains a
> defvar, and contains the baseline list of the servers.  To customize
> the list, users change the value of a separate user option, say,
> eglot-user-server-programs.  This user option's value is nil by
> default, and it allows users to specify both additions of servers to
> the baseline value of eglot-server-programs and removal of servers
> from that value.  

How would this look like?

>                   The :set function of eglot-user-server-programs then
> takes care of doing whatever is needed to make sure that the value of
> eglot-server-programs is modified according to
> eglot-user-server-programs's value when Eglot is started.

This I understand, but I don't see how this is preferable to a general
solution that doesn't require explicit support for any user option.

>> >> Maybe I have missed something, if a user option has a `repeat' or
>> >> `alist' type, you can't just say "append this and that value to the end
>> >> of some other value".  All you get to modify is the entire list, and all
>> >> you get to store is the entire list.
>> >
>> > That's a job for the :set function of the defcustom.
>> 
>> I am not sure I know what you are thinking of, but wouldn't this mean
>> all user options that have already been marked as having a `repeat' or
>> `alist' type, that these would now require an additional :set function?
>
> No, of course not.  I didn't mean any changes to the infrastructure
> that we use for Customize and user options in general.  See above, I
> hope I now explained what I had in mind.

I suppose it does.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>   joaotavora@gmail.com
> Date: Sun, 13 Nov 2022 00:20:35 +0000
> 
> > Under my proposal, the variable eglot-server-programs remains a
> > defvar, and contains the baseline list of the servers.  To customize
> > the list, users change the value of a separate user option, say,
> > eglot-user-server-programs.  This user option's value is nil by
> > default, and it allows users to specify both additions of servers to
> > the baseline value of eglot-server-programs and removal of servers
> > from that value.  
> 
> How would this look like?

I don't understand what you are asking here.  How what will look like?

> >                   The :set function of eglot-user-server-programs then
> > takes care of doing whatever is needed to make sure that the value of
> > eglot-server-programs is modified according to
> > eglot-user-server-programs's value when Eglot is started.
> 
> This I understand, but I don't see how this is preferable to a general
> solution that doesn't require explicit support for any user option.

What is the general solution you have in mind?  And what problems does
that general solution solve?

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com
>> Date: Sun, 13 Nov 2022 00:20:35 +0000
>> 
>> > Under my proposal, the variable eglot-server-programs remains a
>> > defvar, and contains the baseline list of the servers.  To customize
>> > the list, users change the value of a separate user option, say,
>> > eglot-user-server-programs.  This user option's value is nil by
>> > default, and it allows users to specify both additions of servers to
>> > the baseline value of eglot-server-programs and removal of servers
>> > from that value.  
>> 
>> How would this look like?
>
> I don't understand what you are asking here.  How what will look like?

Err, the question doesn't make sense in retrospect, I should have
removed it before sending out my message.

>> >                   The :set function of eglot-user-server-programs then
>> > takes care of doing whatever is needed to make sure that the value of
>> > eglot-server-programs is modified according to
>> > eglot-user-server-programs's value when Eglot is started.
>> 
>> This I understand, but I don't see how this is preferable to a general
>> solution that doesn't require explicit support for any user option.
>
> What is the general solution you have in mind?  And what problems does
> that general solution solve?

That all user options containing lists can be modified, without each
user option containing lists explicitly accommodating the fact using
some other variable.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>   joaotavora@gmail.com
> Date: Sun, 13 Nov 2022 07:11:33 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> >                   The :set function of eglot-user-server-programs then
> >> > takes care of doing whatever is needed to make sure that the value of
> >> > eglot-server-programs is modified according to
> >> > eglot-user-server-programs's value when Eglot is started.
> >> 
> >> This I understand, but I don't see how this is preferable to a general
> >> solution that doesn't require explicit support for any user option.
> >
> > What is the general solution you have in mind?  And what problems does
> > that general solution solve?
> 
> That all user options containing lists can be modified, without each
> user option containing lists explicitly accommodating the fact using
> some other variable.

The problem is only with user options that have complex structures.
Letting users edit such data structures directly is wrought with
unnecessary risks, and requires users to understand very well the
semantics of the data structure and the effect of every change in it.
Alternatively, it requires adding infrastructure to Custom to make
these aspects safer and more easily understandable (something I'm not
even sure is feasible).

My proposal is free from all those disadvantages, because it leaves
the complexity to the :set function, which can do whatever it takes
without bothering users.

In general, I consider user options whose values are complex data
structure to be a Bad Thing, due to the difficulties they cause in
customizing them.  If we limit such options to just the absolute
minimum, the "each user option containing lists" problem you envision
should not exist.  We should strive to present to users a simpler
customization interface, in the form of simple variables whose effects
are easily documented and understood, and make their :set functions
change the complex data structures as needed under the hood.

The proliferation of user options with complex values we have
currently in Emacs is a bad tendency that is at least in part due to
the fact that the developers have no problems dealing with such data
structures.  IMNSHO, we don't think enough about our users when we
introduce such options.

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com
>> Date: Sun, 13 Nov 2022 07:11:33 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> >                   The :set function of eglot-user-server-programs then
>> >> > takes care of doing whatever is needed to make sure that the value of
>> >> > eglot-server-programs is modified according to
>> >> > eglot-user-server-programs's value when Eglot is started.
>> >> 
>> >> This I understand, but I don't see how this is preferable to a general
>> >> solution that doesn't require explicit support for any user option.
>> >
>> > What is the general solution you have in mind?  And what problems does
>> > that general solution solve?
>> 
>> That all user options containing lists can be modified, without each
>> user option containing lists explicitly accommodating the fact using
>> some other variable.
>
> The problem is only with user options that have complex structures.
> Letting users edit such data structures directly is wrought with
> unnecessary risks, and requires users to understand very well the
> semantics of the data structure and the effect of every change in it.

Where do you draw the line to what is "complex" and what isn't?
`eglot-server-programs' is complex in the trivial sense that it is made
up of multiple parts, but each entry is relatively independent, and
pretending an element to the beginning of the list shouldn't involve any
unexpected surprises.

> Alternatively, it requires adding infrastructure to Custom to make
> these aspects safer and more easily understandable (something I'm not
> even sure is feasible).

Like `setopt' does with primitive type checking?

  ;; Check that the type is correct.
  (when-let ((type (get variable 'custom-type)))
    (unless (widget-apply (widget-convert type) :match value)
      (user-error "Value `%S' does not match type %s" value type)))

> My proposal is free from all those disadvantages, because it leaves
> the complexity to the :set function, which can do whatever it takes
> without bothering users.
>
> In general, I consider user options whose values are complex data
> structure to be a Bad Thing, due to the difficulties they cause in
> customizing them.  If we limit such options to just the absolute
> minimum, the "each user option containing lists" problem you envision
> should not exist.  We should strive to present to users a simpler
> customization interface, in the form of simple variables whose effects
> are easily documented and understood, and make their :set functions
> change the complex data structures as needed under the hood.

FWIW I agree that user options shouldn't be too complicated, but knowing
how to simplify a user option is an art in itself.

> The proliferation of user options with complex values we have
> currently in Emacs is a bad tendency that is at least in part due to
> the fact that the developers have no problems dealing with such data
> structures.  IMNSHO, we don't think enough about our users when we
> introduce such options.

Again, I am uncertain what you mean by complex.  Is `auto-mode-alist'
complex?  Or `display-buffer-alist'?

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>   joaotavora@gmail.com
> Date: Tue, 15 Nov 2022 17:50:25 +0000
> 
> > The problem is only with user options that have complex structures.
> > Letting users edit such data structures directly is wrought with
> > unnecessary risks, and requires users to understand very well the
> > semantics of the data structure and the effect of every change in it.
> 
> Where do you draw the line to what is "complex" and what isn't?

Basically, anything that is not an atom is "complex" for this purpose,
in my book.

> `eglot-server-programs' is complex in the trivial sense that it is made
> up of multiple parts, but each entry is relatively independent, and
> pretending an element to the beginning of the list shouldn't involve any
> unexpected surprises.

Let's look at this from the user's POV, okay?

  (defvar eglot-server-programs `((rust-mode . ,(eglot-alternatives '("rust-analyzer" "rls")))
				  (cmake-mode . ("cmake-language-server"))
				  (vimrc-mode . ("vim-language-server" "--stdio"))
				  (python-mode
				   . ,(eglot-alternatives
				       '("pylsp" "pyls" ("pyright-langserver" "--stdio") "jedi-language-server")))
				  ((js-json-mode json-mode) . ,(eglot-alternatives '(("vscode-json-language-server" "--stdio") ("json-languageserver" "--stdio"))))

Here we have:

 . a multi-level list
 . elements that are alists
 . a "backquote construct" with evaluated parts in 

How much Lisp do we require a user to know?  Imagine a user who just
wants to add one more server, either for an existing mode or for a new
mode not in the list.  Do we really expect him or her to understand
all that?

> > Alternatively, it requires adding infrastructure to Custom to make
> > these aspects safer and more easily understandable (something I'm not
> > even sure is feasible).
> 
> Like `setopt' does with primitive type checking?

Yes, but much more complex.  Essentially, display the above list in a
form that is easy to understand, and allow updating it in that form.

> FWIW I agree that user options shouldn't be too complicated, but knowing
> how to simplify a user option is an art in itself.

Yes, but IMO we should bite that bullet every time.

> > The proliferation of user options with complex values we have
> > currently in Emacs is a bad tendency that is at least in part due to
> > the fact that the developers have no problems dealing with such data
> > structures.  IMNSHO, we don't think enough about our users when we
> > introduce such options.
> 
> Again, I am uncertain what you mean by complex.  Is `auto-mode-alist'
> complex?

To some extent, yes.

> Or `display-buffer-alist'?

Definitely!

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com
>> Date: Tue, 15 Nov 2022 17:50:25 +0000
>> 
>> > The problem is only with user options that have complex structures.
>> > Letting users edit such data structures directly is wrought with
>> > unnecessary risks, and requires users to understand very well the
>> > semantics of the data structure and the effect of every change in it.
>> 
>> Where do you draw the line to what is "complex" and what isn't?
>
> Basically, anything that is not an atom is "complex" for this purpose,
> in my book.

Ok.

>> `eglot-server-programs' is complex in the trivial sense that it is made
>> up of multiple parts, but each entry is relatively independent, and
>> pretending an element to the beginning of the list shouldn't involve any
>> unexpected surprises.
>
> Let's look at this from the user's POV, okay?
>
>   (defvar eglot-server-programs `((rust-mode . ,(eglot-alternatives '("rust-analyzer" "rls")))
> 				  (cmake-mode . ("cmake-language-server"))
> 				  (vimrc-mode . ("vim-language-server" "--stdio"))
> 				  (python-mode
> 				   . ,(eglot-alternatives
> 				       '("pylsp" "pyls" ("pyright-langserver" "--stdio") "jedi-language-server")))
> 				  ((js-json-mode json-mode) . ,(eglot-alternatives '(("vscode-json-language-server" "--stdio") ("json-languageserver" "--stdio"))))
>
> Here we have:
>
>  . a multi-level list
>  . elements that are alists
>  . a "backquote construct" with evaluated parts in 
>
> How much Lisp do we require a user to know?  Imagine a user who just
> wants to add one more server, either for an existing mode or for a new
> mode not in the list.  Do we really expect him or her to understand
> all that?

For a simple modification, it appears that 

  (add-to-list 'eglot-server-programs '(foo-mode "foo-lsp" "--stdio"))

is enough.

>> > Alternatively, it requires adding infrastructure to Custom to make
>> > these aspects safer and more easily understandable (something I'm not
>> > even sure is feasible).
>> 
>> Like `setopt' does with primitive type checking?
>
> Yes, but much more complex.  Essentially, display the above list in a
> form that is easy to understand, and allow updating it in that form.

I agree that that would be a good thing to have, but that appears to be
something that would require reworking the widget framework, right?

>> FWIW I agree that user options shouldn't be too complicated, but knowing
>> how to simplify a user option is an art in itself.
>
> Yes, but IMO we should bite that bullet every time.

Do you mean "we" as in the Emacs core developers?  Then yes, but there
are plenty of packages on ELPA that don't scrutinise themselves to the
same degree.  The reality of this complexity shouldn't just be ignored.

>> > The proliferation of user options with complex values we have
>> > currently in Emacs is a bad tendency that is at least in part due to
>> > the fact that the developers have no problems dealing with such data
>> > structures.  IMNSHO, we don't think enough about our users when we
>> > introduce such options.
>> 
>> Again, I am uncertain what you mean by complex.  Is `auto-mode-alist'
>> complex?
>
> To some extent, yes.
>
>> Or `display-buffer-alist'?
>
> Definitely!

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>   joaotavora@gmail.com
> Date: Wed, 16 Nov 2022 13:05:46 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >   (defvar eglot-server-programs `((rust-mode . ,(eglot-alternatives '("rust-analyzer" "rls")))
> > 				  (cmake-mode . ("cmake-language-server"))
> > 				  (vimrc-mode . ("vim-language-server" "--stdio"))
> > 				  (python-mode
> > 				   . ,(eglot-alternatives
> > 				       '("pylsp" "pyls" ("pyright-langserver" "--stdio") "jedi-language-server")))
> > 				  ((js-json-mode json-mode) . ,(eglot-alternatives '(("vscode-json-language-server" "--stdio") ("json-languageserver" "--stdio"))))
> >
> > Here we have:
> >
> >  . a multi-level list
> >  . elements that are alists
> >  . a "backquote construct" with evaluated parts in 
> >
> > How much Lisp do we require a user to know?  Imagine a user who just
> > wants to add one more server, either for an existing mode or for a new
> > mode not in the list.  Do we really expect him or her to understand
> > all that?
> 
> For a simple modification, it appears that 
> 
>   (add-to-list 'eglot-server-programs '(foo-mode "foo-lsp" "--stdio"))
> 
> is enough.

And we expect a random user to know this how?

> >> > Alternatively, it requires adding infrastructure to Custom to make
> >> > these aspects safer and more easily understandable (something I'm not
> >> > even sure is feasible).
> >> 
> >> Like `setopt' does with primitive type checking?
> >
> > Yes, but much more complex.  Essentially, display the above list in a
> > form that is easy to understand, and allow updating it in that form.
> 
> I agree that that would be a good thing to have, but that appears to be
> something that would require reworking the widget framework, right?

Probably.  Which is why I think my original proposal, not to ask users
to customize such variables directly, is much easier to implement.

> >> FWIW I agree that user options shouldn't be too complicated, but knowing
> >> how to simplify a user option is an art in itself.
> >
> > Yes, but IMO we should bite that bullet every time.
> 
> Do you mean "we" as in the Emacs core developers?

Yes.

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com
>> Date: Wed, 16 Nov 2022 13:05:46 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >   (defvar eglot-server-programs `((rust-mode . ,(eglot-alternatives '("rust-analyzer" "rls")))
>> > 				  (cmake-mode . ("cmake-language-server"))
>> > 				  (vimrc-mode . ("vim-language-server" "--stdio"))
>> > 				  (python-mode
>> > 				   . ,(eglot-alternatives
>> > 				       '("pylsp" "pyls" ("pyright-langserver" "--stdio") "jedi-language-server")))
>> > 				  ((js-json-mode json-mode) . ,(eglot-alternatives '(("vscode-json-language-server" "--stdio") ("json-languageserver" "--stdio"))))
>> >
>> > Here we have:
>> >
>> >  . a multi-level list
>> >  . elements that are alists
>> >  . a "backquote construct" with evaluated parts in 
>> >
>> > How much Lisp do we require a user to know?  Imagine a user who just
>> > wants to add one more server, either for an existing mode or for a new
>> > mode not in the list.  Do we really expect him or her to understand
>> > all that?
>> 
>> For a simple modification, it appears that 
>> 
>>   (add-to-list 'eglot-server-programs '(foo-mode "foo-lsp" "--stdio"))
>> 
>> is enough.
>
> And we expect a random user to know this how?

I believe it to be no more or less reasonable to know than how to
manipulate `auto-mode-alist', and that involves Elisp regular
expressions.

If something like this is mentioned in the manual, I think that should
suffice for a "learning-by-template" approach, which is my impression of
how most people use Emacs.

>> >> > Alternatively, it requires adding infrastructure to Custom to make
>> >> > these aspects safer and more easily understandable (something I'm not
>> >> > even sure is feasible).
>> >> 
>> >> Like `setopt' does with primitive type checking?
>> >
>> > Yes, but much more complex.  Essentially, display the above list in a
>> > form that is easy to understand, and allow updating it in that form.
>> 
>> I agree that that would be a good thing to have, but that appears to be
>> something that would require reworking the widget framework, right?
>
> Probably.  Which is why I think my original proposal, not to ask users
> to customize such variables directly, is much easier to implement.

I don't think that either or differs too much in difficulty, this is
more a question of approach.

Re: Making `eglot-server-programs' a custom variable?

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>  joaotavora@gmail.com
> Date: Wed, 16 Nov 2022 14:12:39 +0000
> 
> >> >  . a multi-level list
> >> >  . elements that are alists
> >> >  . a "backquote construct" with evaluated parts in 
> >> >
> >> > How much Lisp do we require a user to know?  Imagine a user who just
> >> > wants to add one more server, either for an existing mode or for a new
> >> > mode not in the list.  Do we really expect him or her to understand
> >> > all that?
> >> 
> >> For a simple modification, it appears that 
> >> 
> >>   (add-to-list 'eglot-server-programs '(foo-mode "foo-lsp" "--stdio"))
> >> 
> >> is enough.
> >
> > And we expect a random user to know this how?
> 
> I believe it to be no more or less reasonable to know than how to
> manipulate `auto-mode-alist', and that involves Elisp regular
> expressions.

I think this variable is way harder to grasp that auto-mode-alist.

> > Probably.  Which is why I think my original proposal, not to ask users
> > to customize such variables directly, is much easier to implement.
> 
> I don't think that either or differs too much in difficulty, this is
> more a question of approach.

I think adding infrastructure to Custom widgets is much harder than
writing a :set function which adds an element to the likes of
eglot-server-programs.  But extensions of Custom in this direction
will be most welcome, of course.

Re: Making `eglot-server-programs' a custom variable?

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: jporterbugs@gmail.com,  arash@gnu.org,  emacs-devel@gnu.org,
>>  joaotavora@gmail.com
>> Date: Wed, 16 Nov 2022 14:12:39 +0000
>> 
>> >> >  . a multi-level list
>> >> >  . elements that are alists
>> >> >  . a "backquote construct" with evaluated parts in 
>> >> >
>> >> > How much Lisp do we require a user to know?  Imagine a user who just
>> >> > wants to add one more server, either for an existing mode or for a new
>> >> > mode not in the list.  Do we really expect him or her to understand
>> >> > all that?
>> >> 
>> >> For a simple modification, it appears that 
>> >> 
>> >>   (add-to-list 'eglot-server-programs '(foo-mode "foo-lsp" "--stdio"))
>> >> 
>> >> is enough.
>> >
>> > And we expect a random user to know this how?
>> 
>> I believe it to be no more or less reasonable to know than how to
>> manipulate `auto-mode-alist', and that involves Elisp regular
>> expressions.
>
> I think this variable is way harder to grasp that auto-mode-alist.

This might just be familiarity speaking, (regexp . major-mode) vs
(major-mode . command-list) -- in the simple case, the case that most
users are interested in -- doesn't seem that big of a difference.

>> > Probably.  Which is why I think my original proposal, not to ask users
>> > to customize such variables directly, is much easier to implement.
>> 
>> I don't think that either or differs too much in difficulty, this is
>> more a question of approach.
>
> I think adding infrastructure to Custom widgets is much harder than
> writing a :set function which adds an element to the likes of
> eglot-server-programs.  But extensions of Custom in this direction
> will be most welcome, of course.

To clarify, my proposal wasn't to add additional infrastructure to
custom widgets, but to either extend add-to-list or add a similar
command that would defer itself until the package is loaded, to avoid
overriding a variable that has a default value.