Emacs as an Org LSP server

Emacs as an Org LSP server

[TEC]

Hi Everyone,

From the Org standardisation effort the idea of using Emacs as the 
basis
of an LSP server for Org has been mentioned a few times.

I thought this deserved it's own thread so here it is :)

I'm quite keen to investigate the viability of this idea.
Some key questions that I think need addressing are:
 1. How can we 'package' Emacs into an LSP client?
 2. Assuming we use some language as the basis for the host how do 
 we
    want to pick it? LSP library? Lisp? Are there any outstanding
    contenders.
 3. How much effort is involved? Is it worth it to try to make Org 
 more
    approachable* (without Emacs)?

Lastly, but perhaps even more crucially --- who would be 
interested in
working on this? I certainly am, but this feels like something 
that
would be more viable with a small working group.

Who's interested?

Timothy.


* I can't help but think that this hypothetical LSP server may 
  serve as
  a 'gateway drug' to Org in Emacs 😉

Re: Emacs as an Org LSP server

[TEC]

A little progress update.

https://github.com/tecosaur/org-lsp now exists.

I have no idea what I'm doing, so if anyone has feedback on the current
idea, that would be much appreciated.

TEC <tecosaur@gmail.com> writes:

> Hi Everyone,
>
> From the Org standardisation effort the idea of using Emacs as the basis
> of an LSP server for Org has been mentioned a few times.
>
> I thought this deserved it's own thread so here it is :)
>
> I'm quite keen to investigate the viability of this idea.
> Some key questions that I think need addressing are:
> 1. How can we 'package' Emacs into an LSP client?
> 2. Assuming we use some language as the basis for the host how do  we
>    want to pick it? LSP library? Lisp? Are there any outstanding
>    contenders.
> 3. How much effort is involved? Is it worth it to try to make Org  more
>    approachable* (without Emacs)?
>
> Lastly, but perhaps even more crucially --- who would be interested in
> working on this? I certainly am, but this feels like something that
> would be more viable with a small working group.
>
> Who's interested?
>
> Timothy.
>
>
> * I can't help but think that this hypothetical LSP server may   serve as
>  a 'gateway drug' to Org in Emacs 😉

Re: Emacs as an Org LSP server

[Bill Burdick]

[-- Attachment #1: Type: text/plain, Size: 1588 bytes --]

Excellent idea!

I frequently use Eclipse and, although I do always have an Emacs open, the
idea of seamlessly using org-mode inside Eclipse is very attractive...


-- Bill


On Sun, Dec 13, 2020 at 12:44 PM TEC <tecosaur@gmail.com> wrote:

>
> A little progress update.
>
> https://github.com/tecosaur/org-lsp now exists.
>
> I have no idea what I'm doing, so if anyone has feedback on the current
> idea, that would be much appreciated.
>
> TEC <tecosaur@gmail.com> writes:
>
> > Hi Everyone,
> >
> > From the Org standardisation effort the idea of using Emacs as the basis
> > of an LSP server for Org has been mentioned a few times.
> >
> > I thought this deserved it's own thread so here it is :)
> >
> > I'm quite keen to investigate the viability of this idea.
> > Some key questions that I think need addressing are:
> > 1. How can we 'package' Emacs into an LSP client?
> > 2. Assuming we use some language as the basis for the host how do  we
> >    want to pick it? LSP library? Lisp? Are there any outstanding
> >    contenders.
> > 3. How much effort is involved? Is it worth it to try to make Org  more
> >    approachable* (without Emacs)?
> >
> > Lastly, but perhaps even more crucially --- who would be interested in
> > working on this? I certainly am, but this feels like something that
> > would be more viable with a small working group.
> >
> > Who's interested?
> >
> > Timothy.
> >
> >
> > * I can't help but think that this hypothetical LSP server may   serve as
> >  a 'gateway drug' to Org in Emacs 😉
>
>
>

[-- Attachment #2: Type: text/html, Size: 2386 bytes --]

Re: Emacs as an Org LSP server

[Jean Louis]

* TEC <tecosaur@gmail.com> [2020-12-13 13:44]:
> 
> A little progress update.
> 
> https://github.com/tecosaur/org-lsp now exists.

As Org-mode does not have collaboration neither was initially designed
for other editor, such idea is welcome.

From a perspective that some server has to know what user is writing
it is advisable to use one own's servers. But if idea gets popular
some company will commercialize it and centralize user's data and
privacy is gone.

Re: Emacs as an Org LSP server

[TEC]

Jean Louis <bugs@gnu.support> writes:

> * TEC <tecosaur@gmail.com> [2020-12-13 13:44]:
>> 
>> A little progress update.
>> 
>> https://github.com/tecosaur/org-lsp now exists.
>
> As Org-mode does not have collaboration neither was initially designed
> for other editor, such idea is welcome.
>
> From a perspective that some server has to know what user is writing
> it is advisable to use one own's servers. But if idea gets popular
> some company will commercialize it and centralize user's data and
> privacy is gone.

FYI the nature of LSP (as I understand it) is that the "server" is a
locally running service that responds to signals from a "client" (code
editor / IDE).

Hope that clears things up,

Timothy

Re: Emacs as an Org LSP server

[Jean Louis]

* TEC <tecosaur@gmail.com> [2020-12-13 20:35]:
> > From a perspective that some server has to know what user is writing
> > it is advisable to use one own's servers. But if idea gets popular
> > some company will commercialize it and centralize user's data and
> > privacy is gone.
> 
> FYI the nature of LSP (as I understand it) is that the "server" is a
> locally running service that responds to signals from a "client" (code
> editor / IDE).

That is how it starts until corporation like Github or somebody else
takes it over. Just look at Github pattern. Git was decentralized
system that they centralized for 50 million developers and included
eye candies that one cannot self-host as one wants.

Jean

Re: Emacs as an Org LSP server

[Gerry Agbobada]

[-- Attachment #1: Type: text/plain, Size: 1805 bytes --]

On Sun, Dec 13, 2020, at 21:23, Jean Louis wrote:
> * TEC <tecosaur@gmail.com> [2020-12-13 20:35]:
> > > From a perspective that some server has to know what user is writing
> > > it is advisable to use one own's servers. But if idea gets popular
> > > some company will commercialize it and centralize user's data and
> > > privacy is gone.
> > 
> > FYI the nature of LSP (as I understand it) is that the "server" is a
> > locally running service that responds to signals from a "client" (code
> > editor / IDE).
> 
> That is how it starts until corporation like Github or somebody else
> takes it over. Just look at Github pattern. Git was decentralized
> system that they centralized for 50 million developers and included
> eye candies that one cannot self-host as one wants.
> 

Hello,

The "server" in Language Server Protocol is a program that answers to LSP requests that's all. It could just be a program written in a FOSS licence (like Palantir pyls 
https://github.com/palantir/python-language-server ) that needs to read the files on your computer in order to answer requests. Data (i.e your org files on your filesystem) does not need to be centralized for it to work.

Git was eventually ""centralized"" by github because version control systems and software forges are based on sharing the data between multiple users, so someone can (and will) offer the tradeoff to make the sharing easier at the cost of privacy/freedom etc.

LSP servers are just file indexers that implement a common protocol to make writing integrations easier. They are called servers because they are long running process listening to messages, but really everything could (and most of the time do) run offline, with file watches over your "project" and sockets for I/O with clients that run locally


Gerry Agbobada

[-- Attachment #2: Type: text/html, Size: 2461 bytes --]

Re: Emacs as an Org LSP server

[Tim Cross]

Gerry Agbobada <emacs-orgmode@gagbo.net> writes:

> On Sun, Dec 13, 2020, at 21:23, Jean Louis wrote:
>> * TEC <tecosaur@gmail.com> [2020-12-13 20:35]:
>> > > From a perspective that some server has to know what user is writing
>> > > it is advisable to use one own's servers. But if idea gets popular
>> > > some company will commercialize it and centralize user's data and
>> > > privacy is gone.
>> >
>> > FYI the nature of LSP (as I understand it) is that the "server" is a
>> > locally running service that responds to signals from a "client" (code
>> > editor / IDE).
>>
>> That is how it starts until corporation like Github or somebody else
>> takes it over. Just look at Github pattern. Git was decentralized
>> system that they centralized for 50 million developers and included
>> eye candies that one cannot self-host as one wants.
>>
>
> Hello,
>
> The "server" in Language Server Protocol is a program that answers to LSP requests that's all. It could just be a program written in a FOSS licence (like Palantir pyls
> https://github.com/palantir/python-language-server ) that needs to read the files on your computer in order to answer requests. Data (i.e your org files on your filesystem) does not need to be centralized for it to work.
>
> Git was eventually ""centralized"" by github because version control systems and software forges are based on sharing the data between multiple users, so someone can (and will) offer the tradeoff to make the sharing easier at the cost of privacy/freedom etc.
>
> LSP servers are just file indexers that implement a common protocol to make writing integrations easier. They are called servers because they are long running process listening to messages, but really everything could (and most of the time do) run offline, with file watches over your "project" and sockets for I/O with clients that run locally
>
>

Good clarification and content. It is important to separate
implementations from protocol. LSP is just a protocol to allow an
interface between an editor and a service which can provide additional
functionality in an editor independent manner.

--
Tim Cross

Re: Emacs as an Org LSP server

[George Mauer]

[-- Attachment #1: Type: text/plain, Size: 1286 bytes --]

I think maybe you might be thrown off by the word "server"? Lsp is just a
standardization of how an editor can do language-specific things. The fact
that standardization exists makes the whole thing pluggable by various
services. These typically run in a separate process - which is a good idea
anyways - on the same machine and the plugin just starts that prices and
communicated to it.

Typescript, c#, I think python, and JavaScript (and maybe Java?) plugins
already do this

On Sun, Dec 13, 2020, 14:34 Jean Louis <bugs@gnu.support> wrote:

> * TEC <tecosaur@gmail.com> [2020-12-13 20:35]:
> > > From a perspective that some server has to know what user is writing
> > > it is advisable to use one own's servers. But if idea gets popular
> > > some company will commercialize it and centralize user's data and
> > > privacy is gone.
> >
> > FYI the nature of LSP (as I understand it) is that the "server" is a
> > locally running service that responds to signals from a "client" (code
> > editor / IDE).
>
> That is how it starts until corporation like Github or somebody else
> takes it over. Just look at Github pattern. Git was decentralized
> system that they centralized for 50 million developers and included
> eye candies that one cannot self-host as one wants.
>
> Jean
>
>

[-- Attachment #2: Type: text/html, Size: 1758 bytes --]

Re: Emacs as an Org LSP server

[Neil Jerram]

[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]

Could you describe a use case?  Apologies if I missed this in earlier
threads.


On Sun, 13 Dec 2020 at 10:44, TEC <tecosaur@gmail.com> wrote:

>
> A little progress update.
>
> https://github.com/tecosaur/org-lsp now exists.
>
> I have no idea what I'm doing, so if anyone has feedback on the current
> idea, that would be much appreciated.
>
> TEC <tecosaur@gmail.com> writes:
>
> > Hi Everyone,
> >
> > From the Org standardisation effort the idea of using Emacs as the basis
> > of an LSP server for Org has been mentioned a few times.
> >
> > I thought this deserved it's own thread so here it is :)
> >
> > I'm quite keen to investigate the viability of this idea.
> > Some key questions that I think need addressing are:
> > 1. How can we 'package' Emacs into an LSP client?
> > 2. Assuming we use some language as the basis for the host how do  we
> >    want to pick it? LSP library? Lisp? Are there any outstanding
> >    contenders.
> > 3. How much effort is involved? Is it worth it to try to make Org  more
> >    approachable* (without Emacs)?
> >
> > Lastly, but perhaps even more crucially --- who would be interested in
> > working on this? I certainly am, but this feels like something that
> > would be more viable with a small working group.
> >
> > Who's interested?
> >
> > Timothy.
> >
> >
> > * I can't help but think that this hypothetical LSP server may   serve as
> >  a 'gateway drug' to Org in Emacs 😉
>
>
>

[-- Attachment #2: Type: text/html, Size: 2128 bytes --]

Re: Emacs as an Org LSP server

[TEC]

[-- Attachment #1.1.1: Type: text/html, Size: 6333 bytes --]

[-- Attachment #1.1.2: model.png --]
[-- Type: image/png, Size: 39468 bytes --]

Re: Emacs as an Org LSP server

[Neil Jerram]

[-- Attachment #1.1: Type: text/plain, Size: 2595 bytes --]

Thanks Timothy.  I did read the README, but I'm afraid I still can't quite
picture a specific use.


On Mon, 14 Dec 2020 at 15:28, TEC <tecosaur@gmail.com> wrote:

> Hi Neil,
>
> I’m going to quote you the readme from the linked github repo:
>
> Allow the unwashed masses to use Org, without using Emacs, using Emacs.
>
> Here’s the image from the readme [image: model.png]
>
> And here’s the first line from the first result of a google search for
> &ldquoLSP”:
>
> The Language Server Protocol (LSP) defines the protocol used between an
> editor or IDE and a language server that provides language features like
> auto complete, go to definition, find all references etc.
>
> That should give you an idea of the intent here.
>
> All the best,
> *Timothy*
>
> * From*: Neil Jerram <%22Neil+Jerram%22+%3Cneiljerram@gmail.com%3E>
> * Subject*: Re: Emacs as an Org LSP server
> * To*: TEC <%22TEC%22+%3Ctecosaur@gmail.com%3E>
> * Cc*: "org-mode-email" <emacs-orgmode@gnu.org>
> * Date*: Mon, 14 Dec 2020 19:41:05 +0800
> Could you describe a use case?  Apologies if I missed this in earlier
> threads.
>
>
> On Sun, 13 Dec 2020 at 10:44, TEC <tecosaur@gmail.com> wrote:
>
>>
>> A little progress update.
>>
>> https://github.com/tecosaur/org-lsp now exists.
>>
>> I have no idea what I'm doing, so if anyone has feedback on the current
>> idea, that would be much appreciated.
>>
>> TEC <tecosaur@gmail.com> writes:
>>
>> > Hi Everyone,
>> >
>> > From the Org standardisation effort the idea of using Emacs as the basis
>> > of an LSP server for Org has been mentioned a few times.
>> >
>> > I thought this deserved it's own thread so here it is :)
>> >
>> > I'm quite keen to investigate the viability of this idea.
>> > Some key questions that I think need addressing are:
>> > 1. How can we 'package' Emacs into an LSP client?
>> > 2. Assuming we use some language as the basis for the host how do  we
>> >    want to pick it? LSP library? Lisp? Are there any outstanding
>> >    contenders.
>> > 3. How much effort is involved? Is it worth it to try to make Org  more
>> >    approachable* (without Emacs)?
>> >
>> > Lastly, but perhaps even more crucially --- who would be interested in
>> > working on this? I certainly am, but this feels like something that
>> > would be more viable with a small working group.
>> >
>> > Who's interested?
>> >
>> > Timothy.
>> >
>> >
>> > * I can't help but think that this hypothetical LSP server may   serve
>> as
>> >  a 'gateway drug' to Org in Emacs 😉
>>
>>
>>

[-- Attachment #1.2: Type: text/html, Size: 6496 bytes --]

[-- Attachment #2: model.png --]
[-- Type: image/png, Size: 39468 bytes --]

Re: Emacs as an Org LSP server

[TEC]

[-- Attachment #1.1: Type: text/html, Size: 9839 bytes --]

Re: Emacs as an Org LSP server

[Jean Louis]

It may all look nice and shiny. But what you people don't understand
is that it is Microsoft and deep meaning of Microsoft one can know if
one researches the history as only so one can see the present and look
into future. Microsoft never changed its strategies. Language server
protocol is just another branch of possible strategies to take away
people's computing. It is matter of advertising and making it popular,
when all the fish are in the net that is where final result comes, and
that is to take away people's freedom and computing to centralized
places.

If Microsoft is really so friendly, then instead of server based
language service they could provide generic definitions how editor
could act, and editor could load those generic definitions locally
without server/client paradigm.

Now Emacs, as prime tool of the GNU project, as free software, is then
supposed to communicate with something external to receive information
on how to do its functions? One big LOL on that!

What will be next? Maybe computers without hard disks that simply load
all they need from Microsoft.

Let us give away our computing to Microsoft.

What people do not understand is that large and evil corporation such
as Microsoft never does any move without strategic planning and
without objective. Try to recognize patterns.

Then better right away stop developing language packages for Emacs and
give away computing to corporations.

Jean

Re: Emacs as an Org LSP server

[TEC]

Jean Louis <bugs@gnu.support> writes:

> [LSP is a evil plot from microsoft]

Hi Jean,

I can see that you're overly concerned about Microsoft being able to
somehow exert control over this. It may assuage your concerns to see an
example "technology stack" that Org-LSP could fit into.

1. Org / Emacs, all GPL-3
2. Rust LSP server + Rust cargo extensions, none of which are written by M$ (all GPL-compatable)
3. Kakoune LSP = Rust, using the "unlicence" licence
4. Kakoune (an experimental text editor, with /no/ relation to M$)

Microsoft has provided a /standard/ that a huge number of editors/IDEs
have adopted with /independent implementations/. At this point there is
/nothing/ M$ could do to interfere with how the above works.

You seem to be focusing on the term "server" in the name. This seems to
be a red herring in this case. In LSP the server is analogous to "emacs
--daemon" and the client to "emacsclient".

I appreciate your concerns Jean, and am aware of Microsoft's history,
however I do not believe there is any factual basis for your conclusions
in this instance.

There is no need to loose sleep over an LSP Server for Org existing :)
On the contrary, I think it has the potential to ultimately enrich the
Org community (see previous discussions).

--
Timothy.

Re: Emacs as an Org LSP server

[Neil Jerram]

[-- Attachment #1: Type: text/plain, Size: 4363 bytes --]

I'm afraid things still aren't clear for me.  Is there a reason it's so
hard to give a concrete example?

If I try to analogise from how LSP works for golang, I believe the LSP
server does things like
- complete symbol beginning with "Xyz"
- tell me where so-and-so function is defined (e.g. so that the client
editor can jump to it).
I'm not sure if operations like that make sense for Org.

Another possibility might be interacting, from a 3rd party editor, with a
body of Org content that has been primarily written and managed in Emacs.
If so, what would those interactions be?  Marking a task as done?
Something more complex than that?

Or is it like: 3rd party editor opens an Org file and the user types some
<random key sequence>.  Editor asks the LSP server (Emacs) "what does
<random key sequence> mean?", and the server replies "it means the Org
entry should now look like this: ..."


On Mon, 14 Dec 2020 at 15:58, TEC <tecosaur@gmail.com> wrote:

> Hi Neil,
>
> Good to hear that you did take a look at the readme 🙂.
>
> You can think of the LSP as a specification for cross-editor/IDE
> extensions. The intent of this is to make some of Org’s functionality
> accessible to the ~95% of people who don’t use Emacs, by hooking into Emacs
> itself.
>
> Does that clear things up for you? You can also see
> https://langserver.org/.
>
> All the best,
> *Timothy*
>
> * From*: Neil Jerram <%22Neil+Jerram%22+%3Cneiljerram@gmail.com%3E>
> * Subject*: Re: Emacs as an Org LSP server
> * To*: TEC <%22TEC%22+%3Ctecosaur@gmail.com%3E>
> * Cc*: "org-mode-email" <emacs-orgmode@gnu.org>
> * Date*: Mon, 14 Dec 2020 23:46:12 +0800
> Thanks Timothy.  I did read the README, but I'm afraid I still can't quite
> picture a specific use.
>
>
> On Mon, 14 Dec 2020 at 15:28, TEC <tecosaur@gmail.com> wrote:
>
>> Hi Neil,
>>
>> I’m going to quote you the readme from the linked github repo:
>>
>> Allow the unwashed masses to use Org, without using Emacs, using Emacs.
>>
>> Here’s the image from the readme [image: model.png]
>>
>> And here’s the first line from the first result of a google search for
>> &ldquoLSP”:
>>
>> The Language Server Protocol (LSP) defines the protocol used between an
>> editor or IDE and a language server that provides language features like
>> auto complete, go to definition, find all references etc.
>>
>> That should give you an idea of the intent here.
>>
>> All the best,
>> *Timothy*
>>
>> * From*: Neil Jerram <%22Neil+Jerram%22+%3Cneiljerram@gmail.com%3E>
>> * Subject*: Re: Emacs as an Org LSP server
>> * To*: TEC <%22TEC%22+%3Ctecosaur@gmail.com%3E>
>> * Cc*: "org-mode-email" <emacs-orgmode@gnu.org>
>> * Date*: Mon, 14 Dec 2020 19:41:05 +0800
>> Could you describe a use case?  Apologies if I missed this in earlier
>> threads.
>>
>>
>> On Sun, 13 Dec 2020 at 10:44, TEC <tecosaur@gmail.com> wrote:
>>
>>>
>>> A little progress update.
>>>
>>> https://github.com/tecosaur/org-lsp now exists.
>>>
>>> I have no idea what I'm doing, so if anyone has feedback on the current
>>> idea, that would be much appreciated.
>>>
>>> TEC <tecosaur@gmail.com> writes:
>>>
>>> > Hi Everyone,
>>> >
>>> > From the Org standardisation effort the idea of using Emacs as the
>>> basis
>>> > of an LSP server for Org has been mentioned a few times.
>>> >
>>> > I thought this deserved it's own thread so here it is :)
>>> >
>>> > I'm quite keen to investigate the viability of this idea.
>>> > Some key questions that I think need addressing are:
>>> > 1. How can we 'package' Emacs into an LSP client?
>>> > 2. Assuming we use some language as the basis for the host how do  we
>>> >    want to pick it? LSP library? Lisp? Are there any outstanding
>>> >    contenders.
>>> > 3. How much effort is involved? Is it worth it to try to make Org  more
>>> >    approachable* (without Emacs)?
>>> >
>>> > Lastly, but perhaps even more crucially --- who would be interested in
>>> > working on this? I certainly am, but this feels like something that
>>> > would be more viable with a small working group.
>>> >
>>> > Who's interested?
>>> >
>>> > Timothy.
>>> >
>>> >
>>> > * I can't help but think that this hypothetical LSP server may   serve
>>> as
>>> >  a 'gateway drug' to Org in Emacs 😉
>>>
>>>
>>>

[-- Attachment #2: Type: text/html, Size: 10933 bytes --]

Re: Emacs as an Org LSP server

[TEC]

[-- Attachment #1.1: Type: text/html, Size: 14573 bytes --]

Re: Emacs as an Org LSP server

[Gerry Agbobada]

[-- Attachment #1.1: Type: text/plain, Size: 2758 bytes --]


> It may all look nice and shiny. But what you people don't understand
> is that it is Microsoft and deep meaning of Microsoft one can know if
> one researches the history as only so one can see the present and look
> into future. Microsoft never changed its strategies. Language server
> protocol is just another branch of possible strategies to take away
> people's computing. It is matter of advertising and making it popular,
> when all the fish are in the net that is where final result comes, and
> that is to take away people's freedom and computing to centralized
> places.
> 
> 

Hello,

You made is a very clear point about _not_ understanding that "server" does not mean "machine controlled by Microsoft". As long as you refuse to understand that, it's very hard to discuss seriously on the matter. I attached as a picture how I plan to summarize LSP to new Doom Emacs users that don't know much about Emacs either. Hopefully this will be clear enough and show that :
- The "server" program does not need to run in the cloud/on 3rd party hardware (it would be really inefficient actually, since everything is IO bound when it comes to LSP)
- You can use any server program you want, even a GPLv3 and later software in the complete stack if you want.

Furthermore, I find that spending so much time and energy to prevent people from spending their time on what they think is right, is pretty harmful.

With that out of the way :

> If Microsoft is really so friendly, then instead of server based
> language service they could provide generic definitions how editor
> could act, and editor could load those generic definitions locally
> without server/client paradigm.
This is literally what MS did.
The protocol is entirely public : https://microsoft.github.io/language-server-protocol/ Of course the governance is terrible and MS might choose to change the protocol in a non-forward compatible once "all the fish are in the net" and they want to force everyone to use VSCode because it becomes the only client that's fully compliant.

Even if that were to happen, nothing prevents people to say "I don't want to follow MS protocol anymore, the program I distribute as LSP server follows the last protocol version I agree with, i.e. 3.15" and linking to that protocol. the LSP clients that want to use that lsp server to enhance editing capabilities (or allow smart manipulations of org-files from editors that are not Emacs*) for org-mode files will have to follow the 3.15 protocol if they want the features.

* which seems to be the goal of this endeavour.

I don't feel like answering much more about this subject, I'm more interested in the technical stack to make that thing work than the usual GNU politics to be honest.

Gerry Agbobada



[-- Attachment #1.2: Type: text/html, Size: 3491 bytes --]

[-- Attachment #2: lsp-doom.png --]
[-- Type: image/png, Size: 253038 bytes --]

Re: Emacs as an Org LSP server

[Russell Adams]

On Mon, Dec 14, 2020 at 05:22:55PM +0000, Neil Jerram wrote:
> If I try to analogise from how LSP works for golang, I believe the LSP
> server does things like
> - complete symbol beginning with "Xyz"
> - tell me where so-and-so function is defined (e.g. so that the client
> editor can jump to it).
> I'm not sure if operations like that make sense for Org.

LSP is also REST based, so your editor how has to talk to a web
*server* over a network. This could be central, and not just on your
machine. How would you know in an update that didn't happen?

> Another possibility might be interacting, from a 3rd party editor, with a
> body of Org content that has been primarily written and managed in Emacs.
> If so, what would those interactions be?  Marking a task as done?
> Something more complex than that?

I'm not interested in spending any time improving an LSP for Org which
would give non-free editors additional functionality with Org files.

That Microsoft is involved in the LSP specification seals the deal for
me.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Emacs as an Org LSP server

[TEC]

Russell Adams <RLAdams@AdamsInfoServ.Com> writes:

> LSP is also REST based, so your editor how has to talk to a web
> *server* over a network. This could be central, and not just on your
> machine. How would you know in an update that didn't happen?

This just ... isn't right.
It's not even REST based, it's using JSON-RPC, and most servers use
stdout + stdin. I'm afraid this simply isn't accurate.


I'm going to ignore the genetic fallacy re: Microsoft.

> I'm not interested in spending any time improving an LSP for Org which
> would give non-free editors additional functionality with Org files.

Because I feel that the rest of the points have been addressed, I'll
just cover this. Looking at https://langserver.org/, the list of current
editors that have LSP clients is:

- Acme
- PROPRIETRY! C++ Builder
- PROPRIETRY! Delphi
- Eclipse
- Eclipse Che
- Emacs (x2)
- GNATStudio
- PROPRIETRY! IntelliJ
- Kakoune
- Kate
- Moonshine IDE
- Oni
- VSCode
- NeoVim (x5)
- PROPRIETRY! Sublime Text 3
- Atom
- CodeMirror
- Theia
- Spyder IDE
- Qt Creator
- Ycmd
- Brackets
- JupyterLab

Note that the majority of the above, (and if considering usage: vast
majority), are *free*.

If your issue is that there is the potential for some non-free
applications to also benefit from this ... the logical conclusion is
that we should stop using the GPL licence, because it allows *anyone*
(including non-free applications) to benefit --- thus inherently making
the work itself /less/ free 😑.

--
Timothy.

Re: Emacs as an Org LSP server

[Neil Jerram]

[-- Attachment #1: Type: text/plain, Size: 5497 bytes --]

Yes, thanks, I'm seeing the picture now.  I guess that some of those things
would require extensions to the LSP standard/protocol, as well as just
implementation, wouldn't they?

On Mon, 14 Dec 2020 at 17:31, TEC <tecosaur@gmail.com> wrote:

> Hi Neil,
>
> Ah, I see what you’re getting at now. I’ll try to give you an idea of what
> I think could apply.
>
>    - Provide nice text manipulation actions, e.g. structural editing
>    - Completion, with company
>    - Org Export
>    - Run Babel blocks
>    - Org syntax highlighting (potentially)
>    - Folding (maybe)
>    - All the nice stuff like table alignment, checkbox state propagation…
>
> Does that help?
>
> All the best,
> *Timothy*
>
> * From*: Neil Jerram <%22Neil+Jerram%22+%3Cneiljerram@gmail.com%3E>
> * Subject*: Re: Emacs as an Org LSP server
> * To*: TEC <%22TEC%22+%3Ctecosaur@gmail.com%3E>
> * Cc*: "org-mode-email" <emacs-orgmode@gnu.org>
> * Date*: Tue, 15 Dec 2020 01:22:55 +0800
> I'm afraid things still aren't clear for me.  Is there a reason it's so
> hard to give a concrete example?
>
> If I try to analogise from how LSP works for golang, I believe the LSP
> server does things like
> - complete symbol beginning with "Xyz"
> - tell me where so-and-so function is defined (e.g. so that the client
> editor can jump to it).
> I'm not sure if operations like that make sense for Org.
>
> Another possibility might be interacting, from a 3rd party editor, with a
> body of Org content that has been primarily written and managed in Emacs.
> If so, what would those interactions be?  Marking a task as done?
> Something more complex than that?
>
> Or is it like: 3rd party editor opens an Org file and the user types some
> <random key sequence>.  Editor asks the LSP server (Emacs) "what does
> <random key sequence> mean?", and the server replies "it means the Org
> entry should now look like this: ..."
>
>
> On Mon, 14 Dec 2020 at 15:58, TEC <tecosaur@gmail.com> wrote:
>
>> Hi Neil,
>>
>> Good to hear that you did take a look at the readme 🙂.
>>
>> You can think of the LSP as a specification for cross-editor/IDE
>> extensions. The intent of this is to make some of Org’s functionality
>> accessible to the ~95% of people who don’t use Emacs, by hooking into Emacs
>> itself.
>>
>> Does that clear things up for you? You can also see
>> https://langserver.org/.
>>
>> All the best,
>> *Timothy*
>>
>> * From*: Neil Jerram <%22Neil+Jerram%22+%3Cneiljerram@gmail.com%3E>
>> * Subject*: Re: Emacs as an Org LSP server
>> * To*: TEC <%22TEC%22+%3Ctecosaur@gmail.com%3E>
>> * Cc*: "org-mode-email" <emacs-orgmode@gnu.org>
>> * Date*: Mon, 14 Dec 2020 23:46:12 +0800
>> Thanks Timothy.  I did read the README, but I'm afraid I still can't
>> quite picture a specific use.
>>
>>
>> On Mon, 14 Dec 2020 at 15:28, TEC <tecosaur@gmail.com> wrote:
>>
>>> Hi Neil,
>>>
>>> I’m going to quote you the readme from the linked github repo:
>>>
>>> Allow the unwashed masses to use Org, without using Emacs, using Emacs.
>>>
>>> Here’s the image from the readme [image: model.png]
>>>
>>> And here’s the first line from the first result of a google search for
>>> &ldquoLSP”:
>>>
>>> The Language Server Protocol (LSP) defines the protocol used between an
>>> editor or IDE and a language server that provides language features
>>> like auto complete, go to definition, find all references etc.
>>>
>>> That should give you an idea of the intent here.
>>>
>>> All the best,
>>> *Timothy*
>>>
>>> * From*: Neil Jerram <%22Neil+Jerram%22+%3Cneiljerram@gmail.com%3E>
>>> * Subject*: Re: Emacs as an Org LSP server
>>> * To*: TEC <%22TEC%22+%3Ctecosaur@gmail.com%3E>
>>> * Cc*: "org-mode-email" <emacs-orgmode@gnu.org>
>>> * Date*: Mon, 14 Dec 2020 19:41:05 +0800
>>> Could you describe a use case?  Apologies if I missed this in earlier
>>> threads.
>>>
>>>
>>> On Sun, 13 Dec 2020 at 10:44, TEC <tecosaur@gmail.com> wrote:
>>>
>>>>
>>>> A little progress update.
>>>>
>>>> https://github.com/tecosaur/org-lsp now exists.
>>>>
>>>> I have no idea what I'm doing, so if anyone has feedback on the current
>>>> idea, that would be much appreciated.
>>>>
>>>> TEC <tecosaur@gmail.com> writes:
>>>>
>>>> > Hi Everyone,
>>>> >
>>>> > From the Org standardisation effort the idea of using Emacs as the
>>>> basis
>>>> > of an LSP server for Org has been mentioned a few times.
>>>> >
>>>> > I thought this deserved it's own thread so here it is :)
>>>> >
>>>> > I'm quite keen to investigate the viability of this idea.
>>>> > Some key questions that I think need addressing are:
>>>> > 1. How can we 'package' Emacs into an LSP client?
>>>> > 2. Assuming we use some language as the basis for the host how do  we
>>>> >    want to pick it? LSP library? Lisp? Are there any outstanding
>>>> >    contenders.
>>>> > 3. How much effort is involved? Is it worth it to try to make Org
>>>> more
>>>> >    approachable* (without Emacs)?
>>>> >
>>>> > Lastly, but perhaps even more crucially --- who would be interested in
>>>> > working on this? I certainly am, but this feels like something that
>>>> > would be more viable with a small working group.
>>>> >
>>>> > Who's interested?
>>>> >
>>>> > Timothy.
>>>> >
>>>> >
>>>> > * I can't help but think that this hypothetical LSP server may
>>>>  serve as
>>>> >  a 'gateway drug' to Org in Emacs 😉
>>>>
>>>>
>>>>

[-- Attachment #2: Type: text/html, Size: 14845 bytes --]

Re: Emacs as an Org LSP server

[TEC]

[-- Attachment #1.1: Type: text/html, Size: 17605 bytes --]

Re: Emacs as an Org LSP server

[Russell Adams]

On Tue, Dec 15, 2020 at 01:08:47AM +0800, TEC wrote:
>
> Jean Louis <bugs@gnu.support> writes:
>
> > [LSP is a evil plot from microsoft]
>
> I can see that you're overly concerned about Microsoft being able to
> somehow exert control over this. It may assuage your concerns to see an
> example "technology stack" that Org-LSP could fit into.

REST API calls to a remote server as a core part of editing text in
your editor isn't concerning? How remote? How would you know? If they
use HTTPS could you even see what is sent?

> Microsoft has provided a /standard/ that a huge number of editors/IDEs
> have adopted with /independent implementations/. At this point there is
> /nothing/ M$ could do to interfere with how the above works.

Microsoft doesn't make standards that it can't corrupt or take
advantage of. See LDAP/AD, HTML extensions, programming language
extensions that makes their solutions incompatible with standards.

> You seem to be focusing on the term "server" in the name. This seems to
> be a red herring in this case. In LSP the server is analogous to "emacs
> --daemon" and the client to "emacsclient".

REST = web server. Using to make JSON requests over what you are
editing and your editor requiring the ability to send/receive to a
potential remote web server is a valid concern.

Emacs daemon is a local socket interface (by default) for
communication between processes on the same box.

> I appreciate your concerns Jean, and am aware of Microsoft's history,
> however I do not believe there is any factual basis for your conclusions
> in this instance.

Tainted, definitions quoted from https://www.thefreedictionary.com/tainted

 - To affect or associate with something undesirable or reprehensible:
    a reputation that was tainted by allegations of illegal activity.

 - An undesirable or corrupting influence or association: wanted to
    avoid the taint of an accounting scandal.

This is the point. Given Microsoft's shameful history, any project
they are supporting is *tainted* by their corrupting influence and
association. That LSP is pushed by MS makes it undesirable due to
their reputation. That Github is now owned by MS makes it tainted by
their reputation.

Companies, just like individuals should be judged by their actions.
Microsoft's well earned poor reputation is sufficient reason to
exclude them from any open source effort.

I must conclude that MS is supporting LSP because they believe it will
increase market share for their proprietary editors. This is due to
their reputation and historic behavior. Thus I have no desire to
support LSP and thus not support MS indirectly.

You might be tired of this kind of debate, but imagine how those of us
who have been in IT for 20 or 30 years are tired of being told that
the abuse we have repeatedly endured from MS is somehow no longer
relevant. That somehow we're wrong to point out we have suffered abuse
from a technology monopoly, and that we are weary and intolerant of
those enabling it (ie: govts, CIOs, end users with fancy toys).

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Emacs as an Org LSP server

[TEC]

Russell Adams <RLAdams@AdamsInfoServ.Com> writes:

> REST API calls to a remote server as a core part of editing text in
> your editor isn't concerning? How remote? How would you know? If they
> use HTTPS could you even see what is sent?

I'm not concerned about REST API calls to a remote server, because:
1. There are no REST API calls
2. There is no remote server

> Microsoft doesn't make standards that it can't corrupt or take
> advantage of. See LDAP/AD, HTML extensions, programming language
> extensions that makes their solutions incompatible with standards.

Sure, but I can choose not to support a certain standard, as can other
LSP-Client/Server FLOSS devs, and you can install a particular version
of either.


> REST = web server. Using to make JSON requests over what you are
> editing and your editor requiring the ability to send/receive to a
> potential remote web server is a valid concern.

No REST, just JSON-RPC, which is just a data format. I don't think JSON
is evil. Oh, and once again, no web servers.

> Emacs daemon is a local socket interface (by default) for
> communication between processes on the same box.

Yep, like LSP. Hence the analogy.

> [ MS Taint ]

I'm a stats student, so if you'll excuse the slightly odd perspective, I
see the chance of MS being dodgy as a bayesian process. Previous
knowledge creates an informed prior. It does not allow you to make
conclusions without examining each instance on a case-by-case basis,
only predictions. To do otherwise is to commit the genetic fallacy.

--
Timothy

Re: Emacs as an Org LSP server

[Jean Louis]

* Gerry Agbobada <emacs-orgmode@gagbo.net> [2020-12-14 20:32]:
> 
> > It may all look nice and shiny. But what you people don't understand
> > is that it is Microsoft and deep meaning of Microsoft one can know if
> > one researches the history as only so one can see the present and look
> > into future. Microsoft never changed its strategies. Language server
> > protocol is just another branch of possible strategies to take away
> > people's computing. It is matter of advertising and making it popular,
> > when all the fish are in the net that is where final result comes, and
> > that is to take away people's freedom and computing to centralized
> > places.
> > 
> > 
> 
> Hello,
> 
> You made is a very clear point about _not_ understanding that
> "server" does not mean "machine controlled by Microsoft". As long as
> you refuse to understand that, it's very hard to discuss seriously
> on the matter. I attached as a picture how I plan to summarize LSP
> to new Doom Emacs users that don't know much about Emacs
> either. Hopefully this will be clear enough and show that :

Well that is your way of understanding what I said, while I never said
that it cannot run locally neither I said that it will be run by
Microsoft, though it probably will if it get popularized.

Microsoft have filed patent for LSP languag server protocol:
https://uspto.report/patent/app/20190149346

Why should Emacs develop and be built on protocol that Microsoft wish
to protect as their own, to control and use to subjugate population of
developers?

Emacs is free software and shall remain far dubious activities of
Microsoft.

You can tell me how everybody can host one's own website, but reality
is that people don't.

Everybody can host one's own code, software, git, packages, but
reality is that people don't and majority look for gratis solutions
online. Corporations take over the control and have to earn money
somehow, so they trap users into their strategies.

Just as git can be decentralized and installed on many various online
servers, it is rather not and it is centralized on Github mostly that
further traps developers by their proprietary eye candies. In general
Github/Microsoft take away specific control and freedom 0 from users
to run the software as they wish:
https://sanctum.geek.nz/why-not-github.html

When such tool that takes away computing becomes popular by marketing
and advertising it will be definitely offered by largest corporations,
which in turn will trap developers in future to their online tools and
online server side computing, it opens plethora of future privacy
issues, centralization and control from entities like Microsoft and
others.

It really does not matter that it is all nicely packed into marketing
pitch about "open source" and "helpful to developers" and that it is
now not centralized and that people find it good. Marketing from those
corporations created that environment of acceptance that computing may
be taken away.

Their marketing pushes that people adopt LSP as from God granted. And
people currently do not see that flying to the source of light will
not let all flies survive.

Microsoft NEVER does any move in the public without having clearly
defined strategy on how to gain control over people's computing.

Jean

See the future.

Re: Emacs as an Org LSP server

[TEC]

Jean Louis <bugs@gnu.support> writes:

> Microsoft have filed patent for LSP languag server protocol:
> https://uspto.report/patent/app/20190149346

This isn't a patent for LSP (it's an open standard), this is a patent
for their Remote Development package:
https://code.visualstudio.com/docs/remote/remote-overview.

Rather different.

> Why should Emacs develop and be built on protocol that Microsoft wish
> to protect as their own, to control and use to subjugate population of
> developers?

This simply isn't what's happening here. I'm just starting work on my
own little project to give non-emacs people a taste of Org's
capabilities. I didn't think the way I spend my time was such a matter
of public concern to the Emacs community.

I guess this criticism may apply to the lsp-mode / eglot packages, but
neither of those have any relation to me.

--
Timothy.

LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[Jean Louis]

* TEC <tecosaur@gmail.com> [2020-12-14 20:24]:
> 
> Jean Louis <bugs@gnu.support> writes:
> 
> > [LSP is a evil plot from microsoft]
> 
> Hi Jean,
> 
> I can see that you're overly concerned about Microsoft being able to
> somehow exert control over this. It may assuage your concerns to see an
> example "technology stack" that Org-LSP could fit into.

Not interested in patented processes. Before any Emacs or GNU software
such as Org within Emacs or Emacs or other software start interacting
by using patented protocols one shall consult attorneys of GNU. Once
attorney confirm that it is alright then go ahead.

See:
https://www.gnu.org/philosophy/software-patents.html

and
https://www.gnu.org/philosophy/fighting-software-patents.html

> Microsoft has provided a /standard/ that a huge number of editors/IDEs
> have adopted with /independent implementations/. At this point there is
> /nothing/ M$ could do to interfere with how the above works.

In Emacs world we have Emacs standard. There is no need to rely on
Micro$oft's patented LSP language server protocols. There is so much
more than you think that M$ can interfer with how the above works.

> You seem to be focusing on the term "server" in the name. This seems to
> be a red herring in this case. In LSP the server is analogous to "emacs
> --daemon" and the client to "emacsclient".

Yes? Don't insist on something that is not, I fully understand what it
is. I am talking of bigger picture and giving you references that may
or may not expand your awareness.

> I appreciate your concerns Jean, and am aware of Microsoft's history,
> however I do not believe there is any factual basis for your conclusions
> in this instance.

See the patent
https://uspto.report/patent/app/20190149346

> There is no need to loose sleep over an LSP Server for Org existing :)
> On the contrary, I think it has the potential to ultimately enrich the
> Org community (see previous discussions).

Enrich it with unencumbered patent-free solutions.

Adopting patented technologies in GNU projects shall be verified by
GNU attorneys.

Jean

Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[TEC]

Hi Jean,

Please read my previous emails before re-iterating the same points.

LSP is not patented, it's just referenced in a patent about MS's fancy
remote development extension.

Jean Louis <bugs@gnu.support> writes:

> Enrich it with unencumbered patent-free solutions.

That's what I'm doing :)

--
Timothy.

Re: Emacs as an Org LSP server

[Jean Louis]

* TEC <tecosaur@gmail.com> [2020-12-14 21:35]:
> 
> Jean Louis <bugs@gnu.support> writes:
> 
> > Microsoft have filed patent for LSP languag server protocol:
> > https://uspto.report/patent/app/20190149346
> 
> This isn't a patent for LSP (it's an open standard), this is a patent
> for their Remote Development package:
> https://code.visualstudio.com/docs/remote/remote-overview.

Are you sure? I wish it would be mistake. But then again did you
research USPTO that there is no patent for LSP?

I see there:

0064] Here, it will be appreciated that a base tool (e.g., Tool A,
Tool B, or Tool C) may be a service or other type of function/tool
that is generally common across many or all of the different types of
client applications. For example, in the context of code editing, the
base tools 420 may include a code completion service, a code debugging
service (e.g., a source code error checking tool), a code highlighting
service, a code navigation operation/service, a code colorization
service (e.g., syntax highlighting in which different colors are
applied to the syntax depending on what category a syntax term belongs
to), a code refactoring service (e.g., restructuring code without
altering its behavior), a code hinting service (e.g., code
completion), a source code search tool, a source code control tool,
and/or a lightbulb service (e.g., an icon service that provides an
expanded display of options).

> I guess this criticism may apply to the lsp-mode / eglot packages, but
> neither of those have any relation to me.

Now, apart from those LSP/Microsoft issues, what is your solution? You
wish to provide LSP server based Org editing to various other editors?

Jean

Re: Emacs as an Org LSP server

[Bastien]

Hi Jean,

you quoted the GNU Kind Communication Guidelines already in this
list: https://www.gnu.org/philosophy/kind-communication.en.html

May I draw your attention to this specific sentence:

  Rather than trying to have the last word, look for the times when
  there is no need to reply, perhaps because you already made the
  relevant point clear enough.

From the last 1000 messages, 174 messages come from you and the vast
majority of them are not about fixing bugs or directly providing an
answer, they are about sharing your opinion on something.  Sometimes
it is on-topic, sometimes it is not, it is hard to tell, because your
message tend to be very long.

I strongly suggest you try to think twice about this sentence from
the GKCG before writing.

Thanks,

-- 
 Bastien

Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[Jean Louis]

* TEC <tecosaur@gmail.com> [2020-12-14 21:48]:
> 
> Hi Jean,
> 
> Please read my previous emails before re-iterating the same points.
> 
> LSP is not patented, it's just referenced in a patent about MS's fancy
> remote development extension.

Do you have evidence it is not patented?

A patent need not be implemented fully. What LSP is is described in
that patent I have referenced. I wish it is not patented. Can you
provide reference that disputes the reference I gave you?

Remember, I wish it would not be so.

Jean

Re: Emacs as an Org LSP server

[Russell Adams]

On Tue, Dec 15, 2020 at 02:12:43AM +0800, TEC wrote:
> > [ MS Taint ]
>
> I'm a stats student, so if you'll excuse the slightly odd perspective, I
> see the chance of MS being dodgy as a bayesian process. Previous
> knowledge creates an informed prior. It does not allow you to make
> conclusions without examining each instance on a case-by-case basis,
> only predictions. To do otherwise is to commit the genetic fallacy.

I don't credit MS as the source of the idea, only a supporter. So
let's omit MS from the discussion and distill this down.

Emacs is a unique and amazing editor. Emacs has special features that
enables truly remarkable data management and text editing in
Org-mode. Other editors cannot or have not been able to replicate
these features, or Emacs Org-mode would not be so uniquely
desirable. Thus if users want to use Org-mode, they should use
Emacs. It is freely available and like all worthwhile tools Emacs
takes some time to learn.

I understand LSP is about editor agnostic support for common
programming languages and editing operations, reducing code
duplication, and improving editing experience in all editors using
LSP.

As someone using and supporting Emacs, why should I care about LSP?
Perhaps if Emacs lacks a decent editing mode for a language then Emacs
could use LSP to provide missing features.

On the other hand if Emacs can provide an LSP to provide our unique
features to other editors which are not Emacs, does that hurt Emacs?

Emacs and Org are volunteer written and maintained. Volunteer time is
scarce and valuable because it is not paid and the pool of qualified
individuals to provide the specialty labor is small. I don't count
myself among that talent pool, however I advocate for the careful
utilization of their attention and try to contribute from the
sideline.

Given volunteer time for Emacs and Org is valuable, why should that
time be spent on technology that could ultimately decrease Emacs
market share? It seems self defeating to contribute to an effort which
could reduce future interest in Emacs, leading to less volunteer time.

If users and programmers for other editors want to try and replicate
the success and features of Org in their editor, they are welcome to
do so. However why should I want to actively contribute to that
effort?

I see it as a choice between choosing to spend our limited time on
maintaining and improving Emacs and Org, or spend time helping other
editors catch up. This is where MS enters, because they will benefit
and I find that strongly unpalatable.

I do understand I'm being protectionist, yet is that wrong? I support
the idea of other open source editors, but we do compete for users. I
expect other editors to be responsible for implementing their own
features.

If we had more volunteers and a surplus of their valuable time, and
Org didn't struggle for time and attention for maintenance and
improvement, perhaps I would be more supportive of collaborative
efforts between editors. Perhaps I could even ignore that evil
monopolists might indirectly profit.

So in summary, why should anyone contribute to exporting our unique
features to other editors instead of investing that time making Emacs
better?

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Emacs as an Org LSP server

[Russell Adams]

On Tue, Dec 15, 2020 at 02:26:30AM +0800, TEC wrote:
> This simply isn't what's happening here. I'm just starting work on my
> own little project to give non-emacs people a taste of Org's
> capabilities. I didn't think the way I spend my time was such a matter
> of public concern to the Emacs community.

I don't think you're being criticized for how you choose to spend your
time. It's certainly an interesting coding project to try and do. I've
been very impressed with your ability. For instance discussing the
Latex templates you made and your updates to the website. Please don't
think this is a personal beef with you, or dismissive of your talents.

Your original post to the list on this topic was to solicit
collaboration from other Org users regarding making an LSP server for
Org, and that's where the disagreement comes from.

Clearly there are strong opinions over the direction, degrees of
freedom, and potential misuse of LSP as a technology. The issue
regarding MS's involvement in the LSP standard isn't your problem, nor
under your control. I'd suggest you just accidentally stepped into
"it". You clearly have valid reasons to like LSP, and I admire many of
the goals as well. I (and others) appear to disagree with the LSP
project implementation.

From my other email, I see this thread as discussing whether the Emacs
community supports spending time on competing technologies, or whether
that time is better focused on improving Emacs and Org.

I think the criticism has been to the effect that there are a variety
of reasons not to support spending community time on competing
technologies like LSP without significant consideration. That
additional consideration is likely larger than us both, and could up
being a project level governance concern and even a legal concern.

In summary I'm positive over you doing a proof of concept code to
satisfy yourself, but I share the concerns over it being a larger
project.

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: Emacs as an Org LSP server

[Tim Cross]

Jean Louis <bugs@gnu.support> writes:

> It may all look nice and shiny. But what you people don't understand
> is that it is Microsoft and deep meaning of Microsoft one can know if
> one researches the history as only so one can see the present and look
> into future. Microsoft never changed its strategies. Language server
> protocol is just another branch of possible strategies to take away
> people's computing. It is matter of advertising and making it popular,
> when all the fish are in the net that is where final result comes, and
> that is to take away people's freedom and computing to centralized
> places.
>

This is just ill informed nonsense. The LSP is nothing more than a
specification. The fact it was initially defined/proposed by Microsoft
is completely irrelevant.

> If Microsoft is really so friendly, then instead of server based
> language service they could provide generic definitions how editor
> could act, and editor could load those generic definitions locally
> without server/client paradigm.
>

It is NOT server based in the sense you mean. In fact, it is actually
precisely what you argue it should be. LSP is simply a "generic definitions how editor
could act, and editor could load those generic definitions locally."

There is absolutely nothing intrinsically wrong with a client server
model. This is exactly the same model you use for your all singing and
dancing extension to org based on a postgres database backend (another
client server model). The 'model' doesn't even need to use TCP
networking, you could do it using just Unix sockets.

Have a look at the existing LSP implementations for Emacs. None of them
require an external server. None of them have anything tied to Microsoft
or github at all (apart from downloading the code). None of them even
require an external network connection.

LSP is nothing more than a specification for a program and a client
which defines the interface between the two and what the supported API
should be. Yes, this has a benefit for Microsoft because it means that
it does not have to implement specific support for every possible
language in their editors like VSCode. However, this is also a benefit
for other editors, like Emacs, because it too can take advantage of this
facility because Microsoft has made the protocol public (they had no
choice other than to make it public because they want others to
implement the servers, not Microsoft).

If your going to speak with authority on some subject and claim we don't
understand it, perhaps you should first make sure you have done your own
research and have a basic understanding of what it is rather than making
inaccurate claims based on a misunderstanding of the use of the word
Server and some baseless conspiracy theory.

Re: Emacs as an Org LSP server

[Jean Louis]

* Russell Adams <RLAdams@AdamsInfoServ.Com> [2020-12-14 22:20]:
  :PROPERTIES:
  :CREATED:  [2020-12-14 Mon 23:18]
  :ID:       a24a5299-11e6-4ecf-a6c5-4622f0d6c28b
  :END:
> On Tue, Dec 15, 2020 at 02:12:43AM +0800, TEC wrote:
> > > [ MS Taint ]
> >
> > I'm a stats student, so if you'll excuse the slightly odd perspective, I
> > see the chance of MS being dodgy as a bayesian process. Previous
> > knowledge creates an informed prior. It does not allow you to make
> > conclusions without examining each instance on a case-by-case basis,
> > only predictions. To do otherwise is to commit the genetic fallacy.
> 
> I don't credit MS as the source of the idea, only a supporter. So
> let's omit MS from the discussion and distill this down.
> 
> Emacs is a unique and amazing editor. Emacs has special features that
> enables truly remarkable data management and text editing in
> Org-mode. Other editors cannot or have not been able to replicate
> these features, or Emacs Org-mode would not be so uniquely
> desirable. Thus if users want to use Org-mode, they should use
> Emacs. It is freely available and like all worthwhile tools Emacs
> takes some time to learn.

There are now other editors using Org slowly slowly, not full. There
exists Org mode for Vim editor. Various Org based tools like Orgzly
for mobile devices have been developed.

https://github.com/jceb/vim-orgmode

Features like outline, TODO/DONE, properties, tags, and various dates
can be implemented by editor macros in other editors.  The very basic
functionality is open for any editor. Finally all those basic tags,
properties, dates can be as well written by hand. Macros are just
handy there.

There is Perl parser for Org:
https://metacpan.org/pod/Org::Parser

If there are parsing engines than most basic features can be
implemented in other editors. 

> If users and programmers for other editors want to try and replicate
> the success and features of Org in their editor, they are welcome to
> do so. However why should I want to actively contribute to that
> effort?

Maybe for compatibility and better collaboration. Observe the basic
structure as such can be definitely written by any editor. Macros
bound to keys can quickly switch TODO/DONE items, insert SCHEDULED,
DEADLINE by using external calendaring tools such as zenity or
question and answer principles, few variables if supported by editor
may hold various properties and tags to be chosen from.

** TODO Headline                                                  :topublish:
   SCHEDULED: <2020-12-14 Mon>
   :PROPERTIES:
   :DESCRIPTION: My first
   :CREATED:  [2020-12-14 Mon 23:18]
   :ID:       d93f73cf-c420-4d4b-b5c8-db53725e26e4
   :END:

Then searching for various properties, tags, TODO/DONE items becomes
easy in any editor. Command line greping or other types of search also
helps to find specific headlines. It need not be necessarily all Emacs
based.

It helps in collaboration. People using various editors can provide
Org type structure and submit their reports or contributions.

> So in summary, why should anyone contribute to exporting our unique
> features to other editors instead of investing that time making Emacs
> better?

When editing files on remote servers not always I have Emacs available
neither I can always install it (at least not as quick). But few handy
macros that one may fetch from WWW server can temporarily serve to
construct basic Org headlines.

Using Emacs on mobile devices is tedious. I do use it but normally
over SSH. Sometimes directly. It is not user friendly on mobile
devices. If there would be Android/LineageOS/Replicant OS editor that
supports macros, I could at least enter some notes with little
structured text for later. Just that I did not find editor with macros.

I use Emacs on mobile devices in console mode. Somebody made Emacs for
Android as GUI, but it is crushing.

In general, it should be useful from Org website to provide macros for
other editors that support macros, as that way more users may come to Emacs as well.

Jean

Re: Emacs as an Org LSP server

[Tim Cross]

I am no fan of Microsoft. I have run Linux as my primary desktop since
1994. I have been working as a developer since 1988 and have first hand
experience regarding many of the poor business practices of Microsoft.
However, I think the LSP is actually a positive imitative and a
potential benefit to all developers and all editors and development
tools, both closed and open source. I have outlined some points I think
are relevant below.

Russell Adams <RLAdams@AdamsInfoServ.Com> writes:

> On Tue, Dec 15, 2020 at 01:08:47AM +0800, TEC wrote:
>>
>> Jean Louis <bugs@gnu.support> writes:
>>
>> > [LSP is a evil plot from microsoft]
>>
>> I can see that you're overly concerned about Microsoft being able to
>> somehow exert control over this. It may assuage your concerns to see an
>> example "technology stack" that Org-LSP could fit into.
>
> REST API calls to a remote server as a core part of editing text in
> your editor isn't concerning? How remote? How would you know? If they
> use HTTPS could you even see what is sent?
>

LSP is not restricted to REST. It uses JSON as the message format, but
can use any form of remote procedure call. It could be REST, it could be
basic Unix socket interface, it could be some other TCP interface. Any
interface both sides understand will work.

>> Microsoft has provided a /standard/ that a huge number of editors/IDEs
>> have adopted with /independent implementations/. At this point there is
>> /nothing/ M$ could do to interfere with how the above works.
>
> Microsoft doesn't make standards that it can't corrupt or take
> advantage of. See LDAP/AD, HTML extensions, programming language
> extensions that makes their solutions incompatible with standards.
>

Yes, Microsoft does have a poor reputation when it comes to standards.
However, if you consider why they have proposed the LSP and what their
business model is, it becomes fairly obvious there is no benefit for
them in changing this specification without good technical reasons.

Microsoft has proposed the LSP because it has the potential to make
their editors more popular and able to support more languages. They want
others to implement the LSP server so that they can support the language
in their editor or development tools with minimal development effort,
increasing market share and reducing maintenance costs. Nothing unusual
with that - basic business principal. They won't want to modify or
change the protocol unnecessarily as that will break their own
integration with these servers. Their business model relies on
maintaining the standard they have proposed.

The key point here is that other technologies, including free software
tools like Emacs, can also benefit from this technology. I'm sure
Microsoft would prefer to prevent this, but they can't if they want
others to develop the language server modules.

One of the biggest challenges for editors like Emacs is how to provide
support for new languages which include all the features people expect
in a modern editor. Often, it is extremely difficult to provide these
features without incorporating some form of language parser or compiler.
this is difficult to do with just Elisp. To try and work around these
limitations, Emacs has used things like ECLIM to interface with the
Eclipse editor and leverage off the internal facilities it provides.
What the LSP does is provide a generic interface definition which works
in a similar fashion, but is not dependent on an external server like
Eclipse.

Consider the potential of a future where in addition to defining a new
language, tools to compile/parse and execute the language the projects
which develop/maintain that language also provide an LSP server for that
language. this would mean that in order to use that language in your
editor, all you need to do is configure your LSP client to communicate
with that server.

I currently use 4 different LSP servers in my Emacs setup. None of them
require a web server. They are all just scripts/executables which sit in
my bin directory. When I open a file in a language which has a LSP
server, Emacs starts the server and communicates with it to handle
completion, linting, format hints etc. There is no external network
connection required, no remote services, no reporting to MS. Even if
Microsoft changes the specification, it has no impact on my current use.

>> You seem to be focusing on the term "server" in the name. This seems to
>> be a red herring in this case. In LSP the server is analogous to "emacs
>> --daemon" and the client to "emacsclient".
>
> REST = web server. Using to make JSON requests over what you are
> editing and your editor requiring the ability to send/receive to a
> potential remote web server is a valid concern.
>

Except that isn't how it works. There is no requirement for a web
server and there is no requirement for it to be based on REST. The
message format is JSON, but how these messages are passed between the
editor and the server is not restricted to a HTTP protocol.


> Emacs daemon is a local socket interface (by default) for
> communication between processes on the same box.
>

As are some of the LSP interfaces.

>> I appreciate your concerns Jean, and am aware of Microsoft's history,
>> however I do not believe there is any factual basis for your conclusions
>> in this instance.
>
> Tainted, definitions quoted from https://www.thefreedictionary.com/tainted
>
>  - To affect or associate with something undesirable or reprehensible:
>     a reputation that was tainted by allegations of illegal activity.
>
>  - An undesirable or corrupting influence or association: wanted to
>     avoid the taint of an accounting scandal.
>
> This is the point. Given Microsoft's shameful history, any project
> they are supporting is *tainted* by their corrupting influence and
> association. That LSP is pushed by MS makes it undesirable due to
> their reputation. That Github is now owned by MS makes it tainted by
> their reputation.
>

And yet Microsoft has submitted and had patches accepted into the Linux
kernel. Does this mean the Linux kernel is now tainted?

> Companies, just like individuals should be judged by their actions.
> Microsoft's well earned poor reputation is sufficient reason to
> exclude them from any open source effort.
>
> I must conclude that MS is supporting LSP because they believe it will
> increase market share for their proprietary editors. This is due to
> their reputation and historic behavior. Thus I have no desire to
> support LSP and thus not support MS indirectly.
>

Just because something is a benefit for Microsofts market share is no
reason it can not also be a benefit to others. I have no illusions that
Microsoft has proposed the LSP because they can see a benefit for their
editors and tools. However, there is such a thing as mutual benefit. The
LSP is a good idea regardless of its origin. Similar ideas have been
proposed before, but have failed to gain traction because they lacked
sufficient support. In fact, LSP could be a benefit to open source
because it now means editors like Emacs could incorporate feature rich
support for closed languages, such as Microsoft's .net and visual basic
and make it more feasible to use Emacs in environments where this was
not possible or as productive as using a native Microsoft editor.

As to the question of indirect support for Microsoft, I have some bad
news. It is more than likely, if you are using Emacs, you are already
providing indirect support. I recently had a look at the packages in the
GNU ELPA repository. I found 98 of these packages, all of them with
copyright assigned to the FSF, are maintained and developed on github.
If you are using GNU ELPA packages, it is highly likely you are using a
package which relies on github for maintenance of the code base,
supporting github which is owned by Microsoft.

We are right to be sceptical about anything proposed by Microsoft.
However, we also need to look more deeply than just a "everything
microsoft does is bad". Even bad people can have good ideas. We need to
evaluate the idea on its own merit and use it to meet our own
requirements. In this case, an LSP server for org mode has the potential
to bring org mode to others who don't use Emacs. I have reservations
about how successful this can be because much of what I think makes org
mode such a powerful tool is too highly tied to Emacs, but I think it is
a great experiment which might prove to be beneficial to both org mode
and free software more generally.

--
Tim Cross

Re: Emacs as an Org LSP server

[Tim Cross]

Russell Adams <RLAdams@AdamsInfoServ.Com> writes:

> So in summary, why should anyone contribute to exporting our unique
> features to other editors instead of investing that time making Emacs
> better?
>

You cannot know that such an effort won't also benefit Emacs org mode
users. The greater the user base, the greater the pool of ideas for
enhancing and extending the system. Emacs users and developers don't
have a monopoly on good ideas.

At the end of the day this is about volunteer effort and if someone
wants to volunteer to do something, in general, we should support that
effort (with some caveats of course). If anyone doesn't want to support
it, that is also fine. However, once we start to try and control what or
how people volunteer their time, we venture out into dangerous waters.
At the end of the day, success or failure of an imitative will depend on
whether people use it or not. Most original initiatives never actually
see the light of day and some which do can often have benefits which
were not recognised initially. I prefer to encourage ideas and see what
fruit they will bare rather than discourage them before they begin.

--
Tim Cross

Re: Emacs as an Org LSP server

[Tom Gillespie]

See also. https://lists.gnu.org/archive/html/emacs-devel/2017-04/msg00798.html
and https://www.reddit.com/r/emacs/comments/696pv1/rms_supports_language_server_protocol_integration/
for some discussion. Best,
Tom

On Mon, Dec 14, 2020 at 4:31 PM Tim Cross <theophilusx@gmail.com> wrote:
>
>
>
> I am no fan of Microsoft. I have run Linux as my primary desktop since
> 1994. I have been working as a developer since 1988 and have first hand
> experience regarding many of the poor business practices of Microsoft.
> However, I think the LSP is actually a positive imitative and a
> potential benefit to all developers and all editors and development
> tools, both closed and open source. I have outlined some points I think
> are relevant below.
>
> Russell Adams <RLAdams@AdamsInfoServ.Com> writes:
>
> > On Tue, Dec 15, 2020 at 01:08:47AM +0800, TEC wrote:
> >>
> >> Jean Louis <bugs@gnu.support> writes:
> >>
> >> > [LSP is a evil plot from microsoft]
> >>
> >> I can see that you're overly concerned about Microsoft being able to
> >> somehow exert control over this. It may assuage your concerns to see an
> >> example "technology stack" that Org-LSP could fit into.
> >
> > REST API calls to a remote server as a core part of editing text in
> > your editor isn't concerning? How remote? How would you know? If they
> > use HTTPS could you even see what is sent?
> >
>
> LSP is not restricted to REST. It uses JSON as the message format, but
> can use any form of remote procedure call. It could be REST, it could be
> basic Unix socket interface, it could be some other TCP interface. Any
> interface both sides understand will work.
>
> >> Microsoft has provided a /standard/ that a huge number of editors/IDEs
> >> have adopted with /independent implementations/. At this point there is
> >> /nothing/ M$ could do to interfere with how the above works.
> >
> > Microsoft doesn't make standards that it can't corrupt or take
> > advantage of. See LDAP/AD, HTML extensions, programming language
> > extensions that makes their solutions incompatible with standards.
> >
>
> Yes, Microsoft does have a poor reputation when it comes to standards.
> However, if you consider why they have proposed the LSP and what their
> business model is, it becomes fairly obvious there is no benefit for
> them in changing this specification without good technical reasons.
>
> Microsoft has proposed the LSP because it has the potential to make
> their editors more popular and able to support more languages. They want
> others to implement the LSP server so that they can support the language
> in their editor or development tools with minimal development effort,
> increasing market share and reducing maintenance costs. Nothing unusual
> with that - basic business principal. They won't want to modify or
> change the protocol unnecessarily as that will break their own
> integration with these servers. Their business model relies on
> maintaining the standard they have proposed.
>
> The key point here is that other technologies, including free software
> tools like Emacs, can also benefit from this technology. I'm sure
> Microsoft would prefer to prevent this, but they can't if they want
> others to develop the language server modules.
>
> One of the biggest challenges for editors like Emacs is how to provide
> support for new languages which include all the features people expect
> in a modern editor. Often, it is extremely difficult to provide these
> features without incorporating some form of language parser or compiler.
> this is difficult to do with just Elisp. To try and work around these
> limitations, Emacs has used things like ECLIM to interface with the
> Eclipse editor and leverage off the internal facilities it provides.
> What the LSP does is provide a generic interface definition which works
> in a similar fashion, but is not dependent on an external server like
> Eclipse.
>
> Consider the potential of a future where in addition to defining a new
> language, tools to compile/parse and execute the language the projects
> which develop/maintain that language also provide an LSP server for that
> language. this would mean that in order to use that language in your
> editor, all you need to do is configure your LSP client to communicate
> with that server.
>
> I currently use 4 different LSP servers in my Emacs setup. None of them
> require a web server. They are all just scripts/executables which sit in
> my bin directory. When I open a file in a language which has a LSP
> server, Emacs starts the server and communicates with it to handle
> completion, linting, format hints etc. There is no external network
> connection required, no remote services, no reporting to MS. Even if
> Microsoft changes the specification, it has no impact on my current use.
>
> >> You seem to be focusing on the term "server" in the name. This seems to
> >> be a red herring in this case. In LSP the server is analogous to "emacs
> >> --daemon" and the client to "emacsclient".
> >
> > REST = web server. Using to make JSON requests over what you are
> > editing and your editor requiring the ability to send/receive to a
> > potential remote web server is a valid concern.
> >
>
> Except that isn't how it works. There is no requirement for a web
> server and there is no requirement for it to be based on REST. The
> message format is JSON, but how these messages are passed between the
> editor and the server is not restricted to a HTTP protocol.
>
>
> > Emacs daemon is a local socket interface (by default) for
> > communication between processes on the same box.
> >
>
> As are some of the LSP interfaces.
>
> >> I appreciate your concerns Jean, and am aware of Microsoft's history,
> >> however I do not believe there is any factual basis for your conclusions
> >> in this instance.
> >
> > Tainted, definitions quoted from https://www.thefreedictionary.com/tainted
> >
> >  - To affect or associate with something undesirable or reprehensible:
> >     a reputation that was tainted by allegations of illegal activity.
> >
> >  - An undesirable or corrupting influence or association: wanted to
> >     avoid the taint of an accounting scandal.
> >
> > This is the point. Given Microsoft's shameful history, any project
> > they are supporting is *tainted* by their corrupting influence and
> > association. That LSP is pushed by MS makes it undesirable due to
> > their reputation. That Github is now owned by MS makes it tainted by
> > their reputation.
> >
>
> And yet Microsoft has submitted and had patches accepted into the Linux
> kernel. Does this mean the Linux kernel is now tainted?
>
> > Companies, just like individuals should be judged by their actions.
> > Microsoft's well earned poor reputation is sufficient reason to
> > exclude them from any open source effort.
> >
> > I must conclude that MS is supporting LSP because they believe it will
> > increase market share for their proprietary editors. This is due to
> > their reputation and historic behavior. Thus I have no desire to
> > support LSP and thus not support MS indirectly.
> >
>
> Just because something is a benefit for Microsofts market share is no
> reason it can not also be a benefit to others. I have no illusions that
> Microsoft has proposed the LSP because they can see a benefit for their
> editors and tools. However, there is such a thing as mutual benefit. The
> LSP is a good idea regardless of its origin. Similar ideas have been
> proposed before, but have failed to gain traction because they lacked
> sufficient support. In fact, LSP could be a benefit to open source
> because it now means editors like Emacs could incorporate feature rich
> support for closed languages, such as Microsoft's .net and visual basic
> and make it more feasible to use Emacs in environments where this was
> not possible or as productive as using a native Microsoft editor.
>
> As to the question of indirect support for Microsoft, I have some bad
> news. It is more than likely, if you are using Emacs, you are already
> providing indirect support. I recently had a look at the packages in the
> GNU ELPA repository. I found 98 of these packages, all of them with
> copyright assigned to the FSF, are maintained and developed on github.
> If you are using GNU ELPA packages, it is highly likely you are using a
> package which relies on github for maintenance of the code base,
> supporting github which is owned by Microsoft.
>
> We are right to be sceptical about anything proposed by Microsoft.
> However, we also need to look more deeply than just a "everything
> microsoft does is bad". Even bad people can have good ideas. We need to
> evaluate the idea on its own merit and use it to meet our own
> requirements. In this case, an LSP server for org mode has the potential
> to bring org mode to others who don't use Emacs. I have reservations
> about how successful this can be because much of what I think makes org
> mode such a powerful tool is too highly tied to Emacs, but I think it is
> a great experiment which might prove to be beneficial to both org mode
> and free software more generally.
>
> --
> Tim Cross
>

Re: Emacs as an Org LSP server

[Jean Louis]

There is definitely nothing wrong in providing Org language server
that runs for different editors who could support the LSP protocol, it
will boost collaboration.

That is pretty much separate subject of the centralization and
strategies we spoke about.

* Tim Cross <theophilusx@gmail.com> [2020-12-14 23:19]:
> This is just ill informed nonsense. The LSP is nothing more than a
> specification. The fact it was initially defined/proposed by Microsoft
> is completely irrelevant.

I truly wish it would be that simple. 

There are many tools and inventions by Microsoft. Some of them appear
to be free, but all of them are there to contribute to profit
making. I am not against profit making. But we have to look into the
tool as having the purpose to contribute to THEIR profit
making. History of Microsoft is clear. Sorry, I do not share the
narrowed viewpoint that they will invest so much money "to help other
free software developers". That it is defined by Microsoft in
collaboration with others is very relevant there.

First question to clarify is if it is really patented or not. While
you as user you can download some Rust server software or Java
software and run server that will work with various editors, somebody
else may not be able to do so if there is patent on it. That imposes
freedom obstacles in future.

Does this patent description correspond to the subject:
https://uspto.report/patent/app/20190149346

> It is NOT server based in the sense you mean. In fact, it is
> actually precisely what you argue it should be. LSP is simply a
> "generic definitions how editor could act, and editor could load
> those generic definitions locally."

I am well aware that you as user may download the piece of software
and run it as server on your computer and that you wish to distinguish
how user may not need a remote server. We clarified this
already.

Corporation will not have use of your personal use, they will promote
their servers and push people to get hooked and trapped into it. It
will become questionable if other entities become able to do the same
if such process is patented.

That it is server based should be undisputable. The whole protocol
speaks of sparing client's CPU time, so CPU time will be spared when
process does not run on the same CPU. You can run it now for
yourself. Sure. But the strategy is visible from their very open
descriptions. Large company is not interested in those single
users. Single users had "git" under their control but nobody had
enough money and power to centralize 50 million developers.

Innocent example is: https://melpa.org/#/lsp-pascal package that
requires: https://github.com/arjanadriaanse/pascal-language-server

But it is made and designed as a server for third parties to take
advantage of it one time in future.

https://code.visualstudio.com/api/language-extensions/language-server-extension-guide

If one would like to improve all editors to use centralized
specifications than that could be done also by providing server-less
specification that every editor could load and thus function in the
same way. Then editor developers could make their underlying language
module that would understand the extension
specificiation. Then users would just need to import or load the
general specification something like XML file or similar type of a
document that says how specific programming language would be linted,
completed, highlighted and so on. And all free software editors would
likely comply and adopt that, would that option be popularized.

That option was not popularized and server based model have been
chosen as only so one can take away computing control from people and
gain larger market share.

Microsoft engineers are not stupid to provide a useful tool and in
addition to put money to promote such tool for other editors as there
would be no gain for them in that strategy.

Maybe not everytime user need to use third software to provide
specification for some language, but most of time. I do understand
that language server provides same service to various editors provided
they use LSP protocol. I do understand it can minimize code writing
which is definitely sound and reasonable. It is just our narrow view
on it. Read the patent and wrong me if that patent does not
apply. Read their plans of server based designed and third party
registry and wrong me if it is not so. 

Instead of some larger Emacs Lisp package for specific language mode,
we will load somewhat or considerably shorter Emacs Lisp PLUS the
external software to provide us server to Emacs supporting that
specific software.

Even the server has to be developed, but apparently minimizes efforts
in larger group of editor developers.

- servers can be downloaded currently by users and used on single
  computer. Yet intention of the protocol is not to be used on single
  computer but to take away the CPU time/effort from clients onto the
  server side. The logic for long term strategy is "third party"
  providing a service. They may or may not implement it. It sounds
  like CPU is not enough for Emacs to handle multiple languages, but
  alright, let us follow their logic.

From:
https://www.infoworld.com/article/3088698/microsoft-backed-langauge-server-protocol-strives-for-language-tools-interoperability.html

"Either a third-party registry or a bundling of the language server
inside of an IDE is necessary to enable Language Server Protocol. An
open governance model for the protocol still must be developed,
according to Jewell."

Now we are using bundling method. But the "third party registry" is
an option for future. That clearly speaks of long term strategy.

You speak of bundling method. I speak of the overall long term
strategy. Once we all start using it, then there will be new
innovation: there will be no need to download all those various
language servers, there will be united centralized place which on can
use.

In general, I find the strategy smart, but not humanitarian as it
appears to be. It is competition for control of the market share, run
for profit, and possible spying and tracking of users (once third
party registry come into place) and a way to take away computing from
people.

When there is enough general acceptance, people will be centralized,
trapped and tracked.

It is one way for Emacs to die. The more control there is the more
funnel effect will be there for only few software pieces to remain on
the market, probably those belonging to Microsoft. Developers are
moved into direction how corporations want it. Not how they want it.

Who controls the specification controls all editors and their related
users.

Jean

Re: Emacs as an Org LSP server

[Dominik Schrempf]

Hello!

I am infrequent active participant on this list but follow some discussions.
This one I found particularly interesting. I do see both of your points Tim
Cross, and Jean Louis, thank you for your detailed explanations including the
references.

As a user of Emacs and Org mode (and not so much as a developer), I am mostly
interested in an editor that works well with the features and languages I use.
For example, I am writing a lot of Haskell, and the Haskell Language Server
provides excellent development support. The Haskell Language Server is not
developed exclusively by Emacs users. To the contrary, it's probably developed
mostly by non-Emacs users. Would I use Emacs to write Haskell code if I could
not use the Haskell Language Server? I don't know although I love Emacs. (I am
sure I would, but I would be a little disappointed).

More generally, on https://langserver.org/ I found a very good argument for why
we need a specification such as the LSP.

I quote:

The problem: "The Matrix"
       Go	Java	TypeScript	...
Emacs  X  X     X
Vim    X  X     X
VSCode X  X     X
...

The solution: Lang[uage] servers and clients
Go         X
Java       X
TypeScript X
...

Emacs  X
Vim    X
VSCode X

I think this is an excellent idea. However, I am not familiar with the legal
aspects mentioned by Jean. So far I had good experiences with language servers.
On the other side, Org mode is Emacs specific, so this argument does not really
apply. Do we want Org mode to stay Emacs specific? I don't know.

Dominik

Jean Louis <bugs@gnu.support> writes:

> There is definitely nothing wrong in providing Org language server
> that runs for different editors who could support the LSP protocol, it
> will boost collaboration.
>
> That is pretty much separate subject of the centralization and
> strategies we spoke about.
>
> * Tim Cross <theophilusx@gmail.com> [2020-12-14 23:19]:
>> This is just ill informed nonsense. The LSP is nothing more than a
>> specification. The fact it was initially defined/proposed by Microsoft
>> is completely irrelevant.
>
> I truly wish it would be that simple.
>
> There are many tools and inventions by Microsoft. Some of them appear
> to be free, but all of them are there to contribute to profit
> making. I am not against profit making. But we have to look into the
> tool as having the purpose to contribute to THEIR profit
> making. History of Microsoft is clear. Sorry, I do not share the
> narrowed viewpoint that they will invest so much money "to help other
> free software developers". That it is defined by Microsoft in
> collaboration with others is very relevant there.
>
> First question to clarify is if it is really patented or not. While
> you as user you can download some Rust server software or Java
> software and run server that will work with various editors, somebody
> else may not be able to do so if there is patent on it. That imposes
> freedom obstacles in future.
>
> Does this patent description correspond to the subject:
> https://uspto.report/patent/app/20190149346
>
>> It is NOT server based in the sense you mean. In fact, it is
>> actually precisely what you argue it should be. LSP is simply a
>> "generic definitions how editor could act, and editor could load
>> those generic definitions locally."
>
> I am well aware that you as user may download the piece of software
> and run it as server on your computer and that you wish to distinguish
> how user may not need a remote server. We clarified this
> already.
>
> Corporation will not have use of your personal use, they will promote
> their servers and push people to get hooked and trapped into it. It
> will become questionable if other entities become able to do the same
> if such process is patented.
>
> That it is server based should be undisputable. The whole protocol
> speaks of sparing client's CPU time, so CPU time will be spared when
> process does not run on the same CPU. You can run it now for
> yourself. Sure. But the strategy is visible from their very open
> descriptions. Large company is not interested in those single
> users. Single users had "git" under their control but nobody had
> enough money and power to centralize 50 million developers.
>
> Innocent example is: https://melpa.org/#/lsp-pascal package that
> requires: https://github.com/arjanadriaanse/pascal-language-server
>
> But it is made and designed as a server for third parties to take
> advantage of it one time in future.
>
> https://code.visualstudio.com/api/language-extensions/language-server-extension-guide
>
> If one would like to improve all editors to use centralized
> specifications than that could be done also by providing server-less
> specification that every editor could load and thus function in the
> same way. Then editor developers could make their underlying language
> module that would understand the extension
> specificiation. Then users would just need to import or load the
> general specification something like XML file or similar type of a
> document that says how specific programming language would be linted,
> completed, highlighted and so on. And all free software editors would
> likely comply and adopt that, would that option be popularized.
>
> That option was not popularized and server based model have been
> chosen as only so one can take away computing control from people and
> gain larger market share.
>
> Microsoft engineers are not stupid to provide a useful tool and in
> addition to put money to promote such tool for other editors as there
> would be no gain for them in that strategy.
>
> Maybe not everytime user need to use third software to provide
> specification for some language, but most of time. I do understand
> that language server provides same service to various editors provided
> they use LSP protocol. I do understand it can minimize code writing
> which is definitely sound and reasonable. It is just our narrow view
> on it. Read the patent and wrong me if that patent does not
> apply. Read their plans of server based designed and third party
> registry and wrong me if it is not so.
>
> Instead of some larger Emacs Lisp package for specific language mode,
> we will load somewhat or considerably shorter Emacs Lisp PLUS the
> external software to provide us server to Emacs supporting that
> specific software.
>
> Even the server has to be developed, but apparently minimizes efforts
> in larger group of editor developers.
>
> - servers can be downloaded currently by users and used on single
>   computer. Yet intention of the protocol is not to be used on single
>   computer but to take away the CPU time/effort from clients onto the
>   server side. The logic for long term strategy is "third party"
>   providing a service. They may or may not implement it. It sounds
>   like CPU is not enough for Emacs to handle multiple languages, but
>   alright, let us follow their logic.
>
> From:
> https://www.infoworld.com/article/3088698/microsoft-backed-langauge-server-protocol-strives-for-language-tools-interoperability.html
>
> "Either a third-party registry or a bundling of the language server
> inside of an IDE is necessary to enable Language Server Protocol. An
> open governance model for the protocol still must be developed,
> according to Jewell."
>
> Now we are using bundling method. But the "third party registry" is
> an option for future. That clearly speaks of long term strategy.
>
> You speak of bundling method. I speak of the overall long term
> strategy. Once we all start using it, then there will be new
> innovation: there will be no need to download all those various
> language servers, there will be united centralized place which on can
> use.
>
> In general, I find the strategy smart, but not humanitarian as it
> appears to be. It is competition for control of the market share, run
> for profit, and possible spying and tracking of users (once third
> party registry come into place) and a way to take away computing from
> people.
>
> When there is enough general acceptance, people will be centralized,
> trapped and tracked.
>
> It is one way for Emacs to die. The more control there is the more
> funnel effect will be there for only few software pieces to remain on
> the market, probably those belonging to Microsoft. Developers are
> moved into direction how corporations want it. Not how they want it.
>
> Who controls the specification controls all editors and their related
> users.
>
> Jean

Re: Emacs as an Org LSP server

[Jean Louis]

* Dominik Schrempf <dominik.schrempf@gmail.com> [2020-12-15 01:36]:
> I think this is an excellent idea. However, I am not familiar with the legal
> aspects mentioned by Jean.

I hope there will be no legal problems and that my statements will be
proven as wrong.

> So far I had good experiences with language servers.  On the other
> side, Org mode is Emacs specific, so this argument does not really
> apply. Do we want Org mode to stay Emacs specific? I don't know.

Org mode has many connections to Open Hyperdocument Template by Doug
Engelbart. The Augment system that has been demonstrated back in maybe
1968 fantastic features that Org mode does not support today in
2020. One of those features is collaboration.

To make Org mode non-software centric is good motion. As if we claim
it is plain text, then plain text may be implemented by other
editors. That boosts or opens first steps to more collaboration. I do
not see LSP server that as good collaboration method, but it can open
various editors to that.

What would be more collaborative is to extend the Org-LSP to read
information from Emacs instance and provide Org structure to its
clients. As Org is not just a programming language that one may do
with any editor, it is dependent on Emacs. There are some other
implementations of Org mode, but it is mostly dependent on Emacs.

Org-LSP implementation that could, at least optionally, read
structured data in a running Emacs instance, or read such data in an
Org file where server resides, then it could eventually provide to its
clients access to more features than commonly expected from LSP, such
as lists of tags, properties, timestamps, including capture templates,
and agenda features. Then if server would be really remote, users
could at least collaborate in the sense that they could work on
similar or same set of structured Org data. They could assign tasks to
same people or have same types of TODO keywords, common LaTeX and PDF
export decorations or other specific settings.

Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Do you have evidence it is not patented?

That sort of question is not useful to ask.
No one _ever_ has evidence that any given thing
is not patented.

Unless we see a specific practical problem,
the thing to do is just ignore the danger of patents.
The danger does exist, but worrying about it in advance
is futile and damaging.  It is like worrying that
a meteorite might fall and hit you.  There is a small
chance of that, but worrying about it is useless.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[Jean Louis]

* Richard Stallman <rms@gnu.org> [2020-12-15 08:48]:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>   > Do you have evidence it is not patented?
> 
> That sort of question is not useful to ask.
> No one _ever_ has evidence that any given thing
> is not patented.

I was expecting a reference where Microsoft explains it is free in one
way or the other, whereby I could not find it myself.

Jean

Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[Christopher Dimech]

Daniel Ravicher found 283 software patents that, if upheld as valid by the courts, could potentially be used to support patent claims upon the Linux Kernel.  I wonder how many more for Free Software in general!

---------------------
Christopher Dimech
General Administrator - Naiad Informatics - GNU Project (Geocomputation)
- Geophysical Simulation
- Geological Subsurface Mapping
- Disaster Preparedness and Mitigation
- Natural Resource Exploration and Production
- Free Software Advocacy


> Sent: Tuesday, December 15, 2020 at 6:50 AM
> From: "Jean Louis" <bugs@gnu.support>
> To: "Richard Stallman" <rms@gnu.org>
> Cc: neiljerram@gmail.com, emacs-orgmode@gnu.org, tecosaur@gmail.com
> Subject: Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server
>
> * Richard Stallman <rms@gnu.org> [2020-12-15 08:48]:
> > [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> > [[[ whether defending the US Constitution against all enemies,     ]]]
> > [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> >
> >   > Do you have evidence it is not patented?
> >
> > That sort of question is not useful to ask.
> > No one _ever_ has evidence that any given thing
> > is not patented.
>
> I was expecting a reference where Microsoft explains it is free in one
> way or the other, whereby I could not find it myself.
>
> Jean
>
>

Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[Jean Louis]

I can understand that GNU and Org shall ignore patents and continue without putting attention.


Jean

Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[Christopher Dimech]

> Sent: Tuesday, December 15, 2020 at 7:25 AM
> From: "Jean Louis" <bugs@gnu.support>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: neiljerram@gmail.com, emacs-orgmode@gnu.org, "Richard Stallman" <rms@gnu.org>, tecosaur@gmail.com
> Subject: Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server
>
> I can understand that GNU and Org shall ignore patents and continue without putting attention.

Lawyers worth their salt will also tell you to ignore them.

> Jean
>
>

Re: Emacs as an Org LSP server

[Bill Burdick]

[-- Attachment #1: Type: text/plain, Size: 373 bytes --]

On Mon, Dec 14, 2020 at 7:35 PM Gerry Agbobada <emacs-orgmode@gagbo.net>
wrote:

> Furthermore, I find that spending so much time and energy to prevent
> people from spending their time on what they think is right, is pretty
> harmful.
>

This is really key.

Timothy, please keep up the good work and pay no attention to people who
are trying to discourage you!


-- Bill

[-- Attachment #2: Type: text/html, Size: 743 bytes --]

Re: LSP is Microsoft's patented protocol - Re: Emacs as an Org LSP server

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Daniel Ravicher found 283 software patents that, if upheld as valid by the courts, could potentially be used to support patent claims upon the Linux Kernel.  I wonder how many more for Free Software in general!

I used to estimate around 100,000 patents for a 2000-era GNU/Linux distro,
based solely on the size of code base.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Emacs as an Org LSP server

[Bastien]

TEC <tecosaur@gmail.com> writes:

> A little progress update.
>
> https://github.com/tecosaur/org-lsp now exists.

I encourage everyone to work with Timothy on how to make real progress
on org-lsp.  If needed so, please use https://github.com/tecosaur/org-lsp
for reporting issues and suggestions.

Timothy, I'm removing this thread from the "Help requests" section on
updates.orgmode.org, because perhaps the code is not mature enough to
receive concrete help -- but feel free to reopen another call when it
is a good time.

Thanks,

-- 
 Bastien