Progress on merging Eglot?

Progress on merging Eglot?

[Payas Relekar]

Hi,

I remember reading a discussion on possibly merging Eglot (language
server client) to Emacs core. I also distinctly remember author wished
to hack on it in the core after merging.

Since it is already on ELPA, are there any more blockers to merge it?

We have a tree-sitter branch, perhaps we can also have one for Eglot?

cc Joao Tavora (Eglot author/maintainer)

Thanks,
Payas Relekar

--

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 2750 bytes --]

On Thu, Sep 8, 2022 at 1:34 PM Payas Relekar <relekarpayas@gmail.com> wrote:

> I remember reading a discussion on possibly merging Eglot (language
> server client) to Emacs core. I also distinctly remember author wished
> to hack on it in the core after merging.
>
> Since it is already on ELPA, are there any more blockers to merge it?


Hi,

The only blocker -- as far as I am aware -- is the amount of time I have
to dedicate to this.

A sequence of concrete actions needs to be laid down to reach these
objectives:

1. Eglot in core, as a :core ELPA package, so that anyone with
   emacs.git commit rights can push to it to release new versions to
   ELPA. People running trunk emacs.git can get the Eglot bleeding
   edge of course.

2. Preserve the full Git history of the eglot.el file after it has been
   merged.

3. Adapt the tests in eglot-tests.el into Emacs's test suite (shouldn't
   be terribly difficult, they are both ert-based). The most interesting
  and useful tests need LSP servers, so one should investigate if these
  can be installed in Hydra.

4. Check carefully for copyright assignments for Eglot's many
   contributors.  I've been careful with this since day 1, but this
   needs to be double checked.  Some contributions are
   "Copyright-paperwork-exempt".

5. Arrange for the eglot.el file in Eglot's current upstream --
   https://github.com/joaotavora/eglot -- to mirror
   src/progmodes/eglot.el, ideally (but not necessarily) automatically

   This is so that:

   5.1 Users of bare Git clones can continue mostly unimpeded.

   5.2 GitHub users can still suggest changes as "Pull Requests"
       via GitHub's interface

   5.3 The current GitHub actions CI -- where installation of third-party
       language servers is relatively easy -- can run at least the current
       eglot-tests.el.

6. Adapt the README.md into a manual section in TexInfo format.

Items 2 and 5 are the ones I'm not 100% sure on how to make
happen. But I'm pretty sure they are possible given sufficient Git
skills.

If anyone can help with any of these items, it would be most welcome
and speed up the process.

"Help" can mean simply fleshing out the item into concrete actions,
anticipating problems, etc..  It can also mean actually doing those
actions.  I don't think there are many dependencies between the items,
so multiple people can work concurrently..

I'm also interested in comments about this plan, its objectives,
alternatives, etc... Those are welcome, though I'd rather not waste
too much time on alternatives such as "forget about Git history",
"just bury the GitHub repo", "the manual can come later",
"ditch the CI", etc...

Thank you,
João

[-- Attachment #2: Type: text/html, Size: 3483 bytes --]

Re: Progress on merging Eglot?

[Philip Kaludercic]

João Távora <joaotavora@gmail.com> writes:

> On Thu, Sep 8, 2022 at 1:34 PM Payas Relekar <relekarpayas@gmail.com> wrote:
>
>> I remember reading a discussion on possibly merging Eglot (language
>> server client) to Emacs core. I also distinctly remember author wished
>> to hack on it in the core after merging.
>>
>> Since it is already on ELPA, are there any more blockers to merge it?
>
>
> Hi,
>
> The only blocker -- as far as I am aware -- is the amount of time I have
> to dedicate to this.
>
> A sequence of concrete actions needs to be laid down to reach these
> objectives:

[...]

> 2. Preserve the full Git history of the eglot.el file after it has been
>    merged.

[...]

> 5. Arrange for the eglot.el file in Eglot's current upstream --
>    https://github.com/joaotavora/eglot -- to mirror
>    src/progmodes/eglot.el, ideally (but not necessarily) automatically
>
>    This is so that:
>
>    5.1 Users of bare Git clones can continue mostly unimpeded.
>
>    5.2 GitHub users can still suggest changes as "Pull Requests"
>        via GitHub's interface
>
>    5.3 The current GitHub actions CI -- where installation of third-party
>        language servers is relatively easy -- can run at least the current
>        eglot-tests.el.

[...]

> Items 2 and 5 are the ones I'm not 100% sure on how to make
> happen. But I'm pretty sure they are possible given sufficient Git
> skills.

It might be possible to add Eglot's history as a branch, then move
elgot.el to the right position and merge the branches.  Another approach
could be to "rebase" Eglot's repository with artificial commits that
match those in the actual repository.  In either case, the main problem
is that the (908) commits in the repository don't appear to be in the
conventional changelog format, and I don't think there is an easy way to
automatically fix that.

Re: Progress on merging Eglot?

[João Távora]

Philip Kaludercic <philipk@posteo.net> writes:

> In either case, the main problem is that the (908) commits in the
> repository don't appear to be in the conventional changelog format,
> and I don't think there is an easy way to automatically fix that.

Can you point to examples?  And why do you say it's a "main problem"? 

Anyway I use and enforce the same format in both my Eglot and emacs.git
commits, and I've never had big complaints about the format of the
latter (that I remember).

If you're concerned about the ChangeLog style

  * eglot.el (foo): bar.

Not making sense once eglot.el is in lisp/progmodes/eglot.el, then I
don't think it's a problem.

Also, rewriting all commits once is fine with me, as long as the long
string of commits is preserved.

João

Re: Progress on merging Eglot?

[Philip Kaludercic]

João Távora <joaotavora@gmail.com> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> In either case, the main problem is that the (908) commits in the
>> repository don't appear to be in the conventional changelog format,
>> and I don't think there is an easy way to automatically fix that.
>
> Can you point to examples?  And why do you say it's a "main problem"? 

I should have been more concrete, a lot of messages appear to reference
issues in the elgot issue tracker, that wouldn't make much sense in
emacs.git.

> Anyway I use and enforce the same format in both my Eglot and emacs.git
> commits, and I've never had big complaints about the format of the
> latter (that I remember).
>
> If you're concerned about the ChangeLog style
>
>   * eglot.el (foo): bar.
>
> Not making sense once eglot.el is in lisp/progmodes/eglot.el, then I
> don't think it's a problem.
>
> Also, rewriting all commits once is fine with me, as long as the long
> string of commits is preserved.
>
> João

Re: Progress on merging Eglot?

[Payas Relekar]

Philip Kaludercic <philipk@posteo.net> writes:

> João Távora <joaotavora@gmail.com> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> In either case, the main problem is that the (908) commits in the
>>> repository don't appear to be in the conventional changelog format,
>>> and I don't think there is an easy way to automatically fix that.
>>
>> Can you point to examples?  And why do you say it's a "main problem"?
>
> I should have been more concrete, a lot of messages appear to reference
> issues in the elgot issue tracker, that wouldn't make much sense in
> emacs.git.

If we choose to recreate commit history one time, can we simply replace
the #num with Github issue link? Should be relatively simple regex find
and replace.

Thanks,
Payas

--

Re: Progress on merging Eglot?

[João Távora]

Philip Kaludercic <philipk@posteo.net> writes:

> João Távora <joaotavora@gmail.com> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> In either case, the main problem is that the (908) commits in the
>>> repository don't appear to be in the conventional changelog format,
>>> and I don't think there is an easy way to automatically fix that.
>>
>> Can you point to examples?  And why do you say it's a "main problem"? 
>
> I should have been more concrete, a lot of messages appear to reference
> issues in the elgot issue tracker, that wouldn't make much sense in
> emacs.git.

Ah, yes.  Well, in that case, I wouldn't mind rewriting the the whole
history -- once -- and adding a search and replace.  So item 2 can
become:

   ...
    
   2. Preserve the full Git history of the eglot.el file after it has been
      merged.
    
      Before merging, it's OK to rewrite commit messages with mechanical
      replacements as: 
    
         #[0-9]+ -> https://github.com/joaotavora/eglot/issues/\1
    
   ...

In my view, this should catch _most_ of the issue references, but I'm
personally OK with missing a few.  Many messages start with 

  'Fix #123: Bar kittens'
  'Close #123: Blither Foo

They can be adapted to be just "Bar kittens" and "Blither foo" with the
aforementioned link in some other line of the commit message.  Scripts
for doing this are appreciated.

João

Re: Progress on merging Eglot?

[Philip Kaludercic]

Payas Relekar <relekarpayas@gmail.com> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> João Távora <joaotavora@gmail.com> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> In either case, the main problem is that the (908) commits in the
>>>> repository don't appear to be in the conventional changelog format,
>>>> and I don't think there is an easy way to automatically fix that.
>>>
>>> Can you point to examples?  And why do you say it's a "main problem"?
>>
>> I should have been more concrete, a lot of messages appear to reference
>> issues in the elgot issue tracker, that wouldn't make much sense in
>> emacs.git.
>
> If we choose to recreate commit history one time, can we simply replace
> the #num with Github issue link? Should be relatively simple regex find
> and replace.

Since many of the #nums are in the first commit line, this could cause
too long commit messages.

> Thanks,
> Payas
>
> --

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 531 bytes --]

On Thu, Sep 8, 2022 at 3:44 PM Philip Kaludercic <philipk@posteo.net> wrote:

>
> > If we choose to recreate commit history one time, can we simply replace
> > the #num with Github issue link? Should be relatively simple regex find
> > and replace.
>
> Since many of the #nums are in the first commit line, this could cause
> too long commit messages.
>

See my other reply.  Fixing these requires a slightly more complicated edit,
but nothing transcendental.  Basically just put the link on some other line.

João

[-- Attachment #2: Type: text/html, Size: 923 bytes --]

Re: Progress on merging Eglot?

[Payas Relekar]

João Távora <joaotavora@gmail.com> writes:

> On Thu, Sep 8, 2022 at 3:44 PM Philip Kaludercic <philipk@posteo.net> wrote:
>
>>
>> > If we choose to recreate commit history one time, can we simply replace
>> > the #num with Github issue link? Should be relatively simple regex find
>> > and replace.
>>
>> Since many of the #nums are in the first commit line, this could cause
>> too long commit messages.
>>
>
> See my other reply.  Fixing these requires a slightly more complicated edit,
> but nothing transcendental.  Basically just put the link on some other line.

Regardless, commit history is only for record and back-reference, and
this is one time action only.

Considering Emacs timeline, if development and consequently bug-reports
continue with core Emacs, over time the #num will become less common and
less needed.

Thanks,
Payas

--

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1875 bytes --]

[Sorry, I took part of this conversation off-list by mistake.  Bringing it
back.]

On Thu, Sep 8, 2022 at 3:53 PM Payas Relekar <relekarpayas@gmail.com> wrote:

> João Távora <joaotavora@gmail.com> writes:
> > On Thu, Sep 8, 2022 at 3:25 PM Payas Relekar <relekarpayas@gmail.com>
> wrote:
> >
> > If we choose to recreate commit history one time, can we simply replace
> >> the #num with Github issue link? Should be relatively simple regex find
> >> and replace.
> >>
> >
> > I think so too. If you'd like to work on these scripts, test them, and
> put
> > the result somewhere we
> > can inspect them, it'd be very appreciated.
>
> Sure. I'm not much versed on Git side of things, but I'll give it a try
> over weekend. At the very least, creating commit messages shouldn't be
> much trouble.
>

Cool, I'll put you down for item 2 then!  I recommend the tool
https://github.com/newren/git-filter-repo.  Here's an example of how it's
used.

https://stackoverflow.com/questions/60863773/git-bulk-change-of-commit-dates

This tool seems to be preferred over `git filter-branch` nowadays and is
easy to
combine with whatever scripting language you're comfortable with.

I've used one of the two in the past (can't remember which).  There are
others, so choose your poison. I also vaguely  remember that running the
script on windows was about a million times slower than on a GNU/linux
box.
Also not sure if that holds anymore.

So the inputs should be Eglot's main repo (master branch) and some branch
of the emacs.git repo.  The output should be something like a new
lisp/progmodes/eglot.el added and then some 900+ commits on top of it.
Place this on a git branch somewhere so we can look at it.

We'll probably need to make adjustments, so make sure to post the script
somewhere with instructions on how to use it.

João

[-- Attachment #2: Type: text/html, Size: 2966 bytes --]

Re: Progress on merging Eglot?

[Payas Relekar]

João Távora <joaotavora@gmail.com> writes:

> [Sorry, I took part of this conversation off-list by mistake.  Bringing it
> back.]
>
> Cool, I'll put you down for item 2 then!  I recommend the tool
> https://github.com/newren/git-filter-repo.  Here's an example of how it's
> used.
>
> https://stackoverflow.com/questions/60863773/git-bulk-change-of-commit-dates
>
> This tool seems to be preferred over `git filter-branch` nowadays and is
> easy to
> combine with whatever scripting language you're comfortable with.
>
> I've used one of the two in the past (can't remember which).  There are
> others, so choose your poison. I also vaguely  remember that running the
> script on windows was about a million times slower than on a GNU/linux
> box.
> Also not sure if that holds anymore.
>
> So the inputs should be Eglot's main repo (master branch) and some branch
> of the emacs.git repo.  The output should be something like a new
> lisp/progmodes/eglot.el added and then some 900+ commits on top of it.
> Place this on a git branch somewhere so we can look at it.
>
> We'll probably need to make adjustments, so make sure to post the script
> somewhere with instructions on how to use it.

Thanks a ton for pointers! I think I got most of it down, if any more
questions, will reach back out.

One thing I'm thinking is, is there any possibility of PRs being
referred in commit messages? If so, the */issues/num won't work, and
*/pulls/num will have to be used. That means probably involving curl.

This looks fun :)

--
Payas
--

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

On Thu, Sep 8, 2022 at 6:03 PM Payas Relekar <relekarpayas@gmail.com> wrote:


> One thing I'm thinking is, is there any possibility of PRs being
> referred in commit messages? If so, the */issues/num won't work, and
> */pulls/num will have to be used. That means probably involving curl.
>

But I think GitHub automatically redirects "issues" -> "pulls"

So if you want to be super clean and heroic, yes do the curl scraping.
Could be useful that day GitHub is no more and we need the wayback
machine to read the discussion.

But don't let that block you either.


> This looks fun :)


Thanks a lot for your help!

João

[-- Attachment #2: Type: text/html, Size: 1263 bytes --]

Re: Progress on merging Eglot?

[Payas Relekar]

João Távora <joaotavora@gmail.com> writes:

> On Thu, Sep 8, 2022 at 6:03 PM Payas Relekar <relekarpayas@gmail.com> wrote:
>
>
>> One thing I'm thinking is, is there any possibility of PRs being
>> referred in commit messages? If so, the */issues/num won't work, and
>> */pulls/num will have to be used. That means probably involving curl.
>>
>
> But I think GitHub automatically redirects "issues" -> "pulls"

That's convenient :)

> So if you want to be super clean and heroic, yes do the curl scraping.
> Could be useful that day GitHub is no more and we need the wayback
> machine to read the discussion.

I think I'll stay clear of heroics for v0, perhaps stretch goal later..

> But don't let that block you either.
>
>
>> This looks fun :)
>
>
> Thanks a lot for your help!

Okay, some dumb questions to boot. I see eglot repo has below relevant files:

1. .dir-locals.el
2. Makefile
3. NEWS.md
4. eglot-tests.el
5. eglot.el

From your description above, I see we only care about #4 and #5, is that
correct? Any of the files omitted here (gifs and .gitignore etc) relevant?

I took a quick skim and there are some commits with `* Makefile', can
these be safely ignored? If we are skipping NEWS.md and .dir-locals.el
files, can we skip commits that are limited to them as well?


Thanks,
Payas
--

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 484 bytes --]

On Thu, Sep 8, 2022 at 6:12 PM Payas Relekar <relekarpayas@gmail.com> wrote:

> Thanks a lot for your help!
>
> Okay, some dumb questions to boot. I see eglot repo has below relevant
> files:
>
> 1. .dir-locals.el
> 2. Makefile
> 3. NEWS.md
> 4. eglot-tests.el
> 5. eglot.el


I'd say focus just on keeping the history of eglot.el.  The history of
eglot-tests.el would
be nice too, but nowhere as important as eglot.el.

Don't worry about the other files.

João

[-- Attachment #2: Type: text/html, Size: 905 bytes --]

Re: Progress on merging Eglot?

[Payas Relekar]

João Távora <joaotavora@gmail.com> writes:

> On Thu, Sep 8, 2022 at 6:12 PM Payas Relekar <relekarpayas@gmail.com> wrote:
>
>> Thanks a lot for your help!
>>
>> Okay, some dumb questions to boot. I see eglot repo has below relevant
>> files:
>>
>> 1. .dir-locals.el
>> 2. Makefile
>> 3. NEWS.md
>> 4. eglot-tests.el
>> 5. eglot.el
>
>
> I'd say focus just on keeping the history of eglot.el.  The history of
> eglot-tests.el would
> be nice too, but nowhere as important as eglot.el.
>
> Don't worry about the other files.
>
> João

Very well. Thank you!

Will get back once there is some progress.

--
Payas

--

Re: Progress on merging Eglot?

[Payas Relekar]

Hello,

A bit later than I expected, but $LIFE and all..

I made a naive attempt to get eglot.el into emacs with history (took too
long to call it QnD, but its definitely dirty).

Here's the steps I followed:

***
# move to a directory where both repos can be cloned
cd ~/git

# clone repos
git clone https://github.com/joaotavora/eglot.git

git clone git clone -b master git://git.sv.gnu.org/emacs.git

# filter eglot with only eglot.el and find+replace # in commit messages
git filter-repo --path eglot.el --message-callback '
      return re.sub(b" #", b" https://github.com/joaotavora/eglot/issues/", message)
      return re.sub(b",#", b",https://github.com/joaotavora/eglot/issues/", message)
      return re.sub(b"(#", b"(https://github.com/joaotavora/eglot/issues/", message)
      return re.sub(b"Fix#", b"Fix: https://github.com/joaotavora/eglot/issues/", message)
      return re.sub(b"github#", b"Github: https://github.com/joaotavora/eglot/issues/", message)
      '

# back to emacs repo
cd ../emacs

# add filtered eglot as upstream
git remote add eglot ../eglot/

git fetch eglot master

# prepare emacs repo for merge
git merge remotes/eglot/master --allow-unrelated-histories --no-commit

# reset eglot to initial commit
cd ../eglot && git reset --hard 4970e7e1b605510c665c52bc1ddd83bcd1e255d4

# merge by pushing eglot.el into lisp/progmodes/
cd ../emacs && git read-tree --prefix=lisp/progmodes/ -u eglot/master

# make a commit
git commit -m "; Merge from joaotavora/eglot"

# push eglot to latest
cd ../eglot && git reset --hard 2f9cf0dcfb2d37f45ec8b9ea65ac9063da033b70

# pull all eglot history to emacs
cd ../emacs && git pull -s subtree eglot master
--allow-unrelated-histories
***

This is the repo I pushed for you to see:
https://github.com/bhankas/emacs/commits/master

There are couple of glaring issues with this:

1. commit message length is not validated, but can be part of 'git filter-repo' script
2. history does not show for eglot.el or lisp/progmodes, but only as part
  of entire emacs repo

I'm kinda unsure where to go from here, for #1 I can definitely use some
help, my bash skills are primitive at best.

As for #2, I've tried quite a few solutions after scouring Google  and
this seems to be the least bad. Another approach was to export patches
from our filtered eglot repo and apply them to particular file in emacs
repo, but that way leads to many many merge conflicts, that I am unsure
how to resolve automatically. I'll be happy to provide the patches to
anybody if it can help.

Further help appreciated.

Thanks,
Payas

Payas Relekar <relekarpayas@gmail.com> writes:

> João Távora <joaotavora@gmail.com> writes:
> I'd say focus just on keeping the history of eglot.el.  The history of
>> eglot-tests.el would
>> be nice too, but nowhere as important as eglot.el.
>>
>> Don't worry about the other files.
>>
>> João
>
> Very well. Thank you!
>
> Will get back once there is some progress.

--

Re: Progress on merging Eglot?

[Payas Relekar]

Payas Relekar <relekarpayas@gmail.com> writes:

A convenient gist for the steps, for those who'd like it that way:
https://gist.github.com/bhankas/422c34db2e987d4dc3f243e26ba156a3

Thanks,
Payas

> Hello,
>
> A bit later than I expected, but $LIFE and all..
>
> I made a naive attempt to get eglot.el into emacs with history (took too
> long to call it QnD, but its definitely dirty).
>
> Here's the steps I followed:
>
> ***
> # move to a directory where both repos can be cloned
> cd ~/git
>
> # clone repos
> git clone https://github.com/joaotavora/eglot.git
>
> git clone git clone -b master git://git.sv.gnu.org/emacs.git
>
> # filter eglot with only eglot.el and find+replace # in commit messages
> git filter-repo --path eglot.el --message-callback '
>       return re.sub(b" #", b" https://github.com/joaotavora/eglot/issues/", message)
>       return re.sub(b",#", b",https://github.com/joaotavora/eglot/issues/", message)
>       return re.sub(b"(#", b"(https://github.com/joaotavora/eglot/issues/", message)
>       return re.sub(b"Fix#", b"Fix: https://github.com/joaotavora/eglot/issues/", message)
>       return re.sub(b"github#", b"Github: https://github.com/joaotavora/eglot/issues/", message)
>       '
>
> # back to emacs repo
> cd ../emacs
>
> # add filtered eglot as upstream
> git remote add eglot ../eglot/
>
> git fetch eglot master
>
> # prepare emacs repo for merge
> git merge remotes/eglot/master --allow-unrelated-histories --no-commit
>
> # reset eglot to initial commit
> cd ../eglot && git reset --hard 4970e7e1b605510c665c52bc1ddd83bcd1e255d4
>
> # merge by pushing eglot.el into lisp/progmodes/
> cd ../emacs && git read-tree --prefix=lisp/progmodes/ -u eglot/master
>
> # make a commit
> git commit -m "; Merge from joaotavora/eglot"
>
> # push eglot to latest
> cd ../eglot && git reset --hard 2f9cf0dcfb2d37f45ec8b9ea65ac9063da033b70
>
> # pull all eglot history to emacs
> cd ../emacs && git pull -s subtree eglot master
> --allow-unrelated-histories
> ***
>
> This is the repo I pushed for you to see:
> https://github.com/bhankas/emacs/commits/master
>
> There are couple of glaring issues with this:
>
> 1. commit message length is not validated, but can be part of 'git filter-repo' script
> 2. history does not show for eglot.el or lisp/progmodes, but only as part
>   of entire emacs repo
>
> I'm kinda unsure where to go from here, for #1 I can definitely use some
> help, my bash skills are primitive at best.
>
> As for #2, I've tried quite a few solutions after scouring Google  and
> this seems to be the least bad. Another approach was to export patches
> from our filtered eglot repo and apply them to particular file in emacs
> repo, but that way leads to many many merge conflicts, that I am unsure
> how to resolve automatically. I'll be happy to provide the patches to
> anybody if it can help.
>
> Further help appreciated.
>
> Thanks,
> Payas
>
> Payas Relekar <relekarpayas@gmail.com> writes:
>
>> João Távora <joaotavora@gmail.com> writes:
>> I'd say focus just on keeping the history of eglot.el.  The history of
>>> eglot-tests.el would
>>> be nice too, but nowhere as important as eglot.el.
>>>
>>> Don't worry about the other files.
>>>
>>> João
>>
>> Very well. Thank you!
>>
>> Will get back once there is some progress.
>
> --
>
>

--

Re: Progress on merging Eglot?

[Philip Kaludercic]

Payas Relekar <relekarpayas@gmail.com> writes:

> There are couple of glaring issues with this:
>
> 1. commit message length is not validated, but can be part of 'git filter-repo' script

A script to do the job might not be trivial, but I think you write
something "good enough", then manually fix anything that the rough
attempt missed. 

> 2. history does not show for eglot.el or lisp/progmodes, but only as part
>   of entire emacs repo

I know that João said he didn't want to discuss the topic, but I'll say
it anyway.  The necessity to preserve the history seems superfluous to
me.  emacs.git has projects like Org or the Modus Themes that are
developed out of tree and are regularly synchronised back without any
major issues.

It seems to me that this is an unnecessary hindrance towards merging
Eglot, that might be nice to have but is really stretching what you can
do with Git.

Re: Progress on merging Eglot?

[Dmitry Gutov]

On 18.09.2022 23:30, Philip Kaludercic wrote:
>> 2. history does not show for eglot.el or lisp/progmodes, but only as part
>>    of entire emacs repo
> I know that João said he didn't want to discuss the topic, but I'll say
> it anyway.  The necessity to preserve the history seems superfluous to
> me.  emacs.git has projects like Org or the Modus Themes that are
> developed out of tree and are regularly synchronised back without any
> major issues.

Org is still being developed out of tree, though. So any history digs 
are probably going to occur in its own repo.

IIUC, Joao want to move Eglot entirely into Emacs and be done with 
github at some soon-ish point.

Re: Progress on merging Eglot?

[Philip Kaludercic]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 18.09.2022 23:30, Philip Kaludercic wrote:
>>> 2. history does not show for eglot.el or lisp/progmodes, but only as part
>>>    of entire emacs repo
>> I know that João said he didn't want to discuss the topic, but I'll say
>> it anyway.  The necessity to preserve the history seems superfluous to
>> me.  emacs.git has projects like Org or the Modus Themes that are
>> developed out of tree and are regularly synchronised back without any
>> major issues.
>
> Org is still being developed out of tree, though. So any history digs
> are probably going to occur in its own repo.

> IIUC, Joao want to move Eglot entirely into Emacs and be done with
> github at some soon-ish point.

I am not sure about that, in his message he wrote the following:

    5. Arrange for the eglot.el file in Eglot's current upstream --
       https://github.com/joaotavora/eglot -- to mirror
       src/progmodes/eglot.el, ideally (but not necessarily) automatically

       This is so that:

       5.1 Users of bare Git clones can continue mostly unimpeded.

       5.2 GitHub users can still suggest changes as "Pull Requests"
           via GitHub's interface

       5.3 The current GitHub actions CI -- where installation of third-party
           language servers is relatively easy -- can run at least the current
           eglot-tests.el.

Maybe there would be a point in "copying" the history into emacs.git,
but this is a continuous process that would have to be synchronised on
both sides (emacs.git and GitHub) all the time.  That sounds like more
effort than the history is worth.

Re: Progress on merging Eglot?

[Dmitry Gutov]

On 19.09.2022 00:00, Philip Kaludercic wrote:
> Dmitry Gutov<dgutov@yandex.ru>  writes:
> 
>> On 18.09.2022 23:30, Philip Kaludercic wrote:
>>>> 2. history does not show for eglot.el or lisp/progmodes, but only as part
>>>>     of entire emacs repo
>>> I know that João said he didn't want to discuss the topic, but I'll say
>>> it anyway.  The necessity to preserve the history seems superfluous to
>>> me.  emacs.git has projects like Org or the Modus Themes that are
>>> developed out of tree and are regularly synchronised back without any
>>> major issues.
>> Org is still being developed out of tree, though. So any history digs
>> are probably going to occur in its own repo.
>> IIUC, Joao want to move Eglot entirely into Emacs and be done with
>> github at some soon-ish point.
> I am not sure about that, in his message he wrote the following:
> 
>      5. Arrange for the eglot.el file in Eglot's current upstream --
>         https://github.com/joaotavora/eglot  -- to mirror
>         src/progmodes/eglot.el, ideally (but not necessarily) automatically
> 
>         This is so that:
> 
>         5.1 Users of bare Git clones can continue mostly unimpeded.
> 
>         5.2 GitHub users can still suggest changes as "Pull Requests"
>             via GitHub's interface
> 
>         5.3 The current GitHub actions CI -- where installation of third-party
>             language servers is relatively easy -- can run at least the current
>             eglot-tests.el.
> 
> Maybe there would be a point in "copying" the history into emacs.git,
> but this is a continuous process that would have to be synchronised on
> both sides (emacs.git and GitHub) all the time.  That sounds like more
> effort than the history is worth.

Maybe I misunderstood, then.

But if the full move is never going to happen, I think the elpa-bundling 
approach seems like a better option all-around. It just need a little 
more work and (perhaps significantly) more attention from the maintainers.

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1237 bytes --]

Dmitry is right. The goal is to be done with GitHub soonish. Soonish
meaning phased.

Philip, this is item number 5.  Not only is it secondary to item 2, which
Payas is handling, but
it has to be handled after 2 is done.

So, once Eglot's mainline is the same as Emacs's, I will find some way for
the
current Github repo to somehow mirror only that part of Emacs.  Of course
by
force-pushing to the current repo.  This will be a "best effort thing".  If
it's a dump copy
of eglot.el from emacs.git -> git@github.com:joaotavora/eglot every 24h
then I'm perfectly
happy.  If something more sophisticated can be done, so much the better.
Regardless,
keeping history in that repo is not important because it won't be the
upstream by then.

Maybe there would be a point in "copying" the history into emacs.git,
> but this is a continuous process that would have to be synchronised on
> both sides (emacs.git and GitHub) all the time.  That sounds like more
> effort than the history is worth.
>

That's not the plan.  But if that _were_ the plan, then I can think of no
better
arguments than keeping history.  Good (Git) history is a benediction that
should
be preserved at (almost) all costs.

João Távora

[-- Attachment #2: Type: text/html, Size: 1745 bytes --]

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 212 bytes --]

On Sun, Sep 18, 2022 at 10:11 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 19.09.2022 00:00, Philip Kaludercic wrote:
> > Dmitry Gutov<dgutov@yandex.ru>  writes:
>
> Maybe I misunderstood, then.


You didn't.

[-- Attachment #2: Type: text/html, Size: 591 bytes --]

Re: Progress on merging Eglot?

[Philip Kaludercic]

João Távora <joaotavora@gmail.com> writes:

>> but this is a continuous process that would have to be synchronised on
>> both sides (emacs.git and GitHub) all the time.  That sounds like more
>> effort than the history is worth.
>>
>
> That's not the plan.  But if that _were_ the plan, then I can think of no
> better
> arguments than keeping history.  

In that case my apologies, I had misunderstood.

>                                  Good (Git) history is a benediction that
> should
> be preserved at (almost) all costs.
>
> João Távora

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 453 bytes --]

On Sun, Sep 18, 2022 at 10:20 PM Philip Kaludercic <philipk@posteo.net>
wrote:

> João Távora <joaotavora@gmail.com> writes:
>
> >> but this is a continuous process that would have to be synchronised on
> >> both sides (emacs.git and GitHub) all the time.  That sounds like more
> >> effort than the history is worth.
> > That's not the plan.
> In that case my apologies, I had misunderstood.
>

No need to apologize, of course!

João

[-- Attachment #2: Type: text/html, Size: 877 bytes --]

Re: Progress on merging Eglot?

[Stefan Monnier]

> # filter eglot with only eglot.el and find+replace # in commit messages
> git filter-repo --path eglot.el --message-callback '
>       return re.sub(b" #", b" https://github.com/joaotavora/eglot/issues/", message)
>       return re.sub(b",#", b",https://github.com/joaotavora/eglot/issues/", message)
>       return re.sub(b"(#", b"(https://github.com/joaotavora/eglot/issues/", message)
>       return re.sub(b"Fix#", b"Fix: https://github.com/joaotavora/eglot/issues/", message)
>       return re.sub(b"github#", b"Github: https://github.com/joaotavora/eglot/issues/", message)

I suspect you can get a slightly better result if you tweak this filter
script so as to rename `eglot.el` to `lisp/progmodes/eglot.el` in each commit.

Then you can also skip the `git read-tree`.


        Stefan

Re: Progress on merging Eglot?

[Payas Relekar]

Thank you! Indeed moving to subdirectory makes history apply to
correct file and shows up as expected. Was also quite straightforward
with git-filter-repo, so here's the updates:

Updates gist:
https://gist.github.com/bhankas/422c34db2e987d4dc3f243e26ba156a3

Updated repo:
https://github.com/bhankas/emacs

Updated history:
https://github.com/bhankas/emacs/commits/master/lisp/progmodes/eglot.el

Now the only remaining issue is commit length. Can someone please
advice on how we can go about it? My brute force thinking is to simply
add newline where length exceeds, delimited at word boundary, but
obviously that's not good enough nor covers sufficient edge cases.
Perhaps a Python/Perl wizard has a one/two liner to take that pain
away?

Thanks,
Payas

On Mon, 19 Sep, 2022, 08:08 Stefan Monnier, <monnier@iro.umontreal.ca> wrote:
>
> > # filter eglot with only eglot.el and find+replace # in commit messages
> > git filter-repo --path eglot.el --message-callback '
> >       return re.sub(b" #", b" https://github.com/joaotavora/eglot/issues/", message)
> >       return re.sub(b",#", b",https://github.com/joaotavora/eglot/issues/", message)
> >       return re.sub(b"(#", b"(https://github.com/joaotavora/eglot/issues/", message)
> >       return re.sub(b"Fix#", b"Fix: https://github.com/joaotavora/eglot/issues/", message)
> >       return re.sub(b"github#", b"Github: https://github.com/joaotavora/eglot/issues/", message)
>
> I suspect you can get a slightly better result if you tweak this filter
> script so as to rename `eglot.el` to `lisp/progmodes/eglot.el` in each commit.
>
> Then you can also skip the `git read-tree`.
>
>
>         Stefan
>

Re: Progress on merging Eglot?

[Payas Relekar]

Another way I can think of is to keep #nums where they are right now,
but add their respective links in the message body below.

e.g.
Fixes #234,#456

this fixes two issues

Becomes =>

Fixes #234,#456

this fixes two issues
#234: https://github.com/joaotavora/eglot/234
#456: https://github.com/joaotavora/eglot/456

Inputs welcome, since this is not 100% convenient, but much simpler to
implement, with minimal to none manual readjustment needed.

Thanks,
Payas

On 9/19/22, Payas Relekar <relekarpayas@gmail.com> wrote:
> Thank you! Indeed moving to subdirectory makes history apply to
> correct file and shows up as expected. Was also quite straightforward
> with git-filter-repo, so here's the updates:
>
> Updates gist:
> https://gist.github.com/bhankas/422c34db2e987d4dc3f243e26ba156a3
>
> Updated repo:
> https://github.com/bhankas/emacs
>
> Updated history:
> https://github.com/bhankas/emacs/commits/master/lisp/progmodes/eglot.el
>
> Now the only remaining issue is commit length. Can someone please
> advice on how we can go about it? My brute force thinking is to simply
> add newline where length exceeds, delimited at word boundary, but
> obviously that's not good enough nor covers sufficient edge cases.
> Perhaps a Python/Perl wizard has a one/two liner to take that pain
> away?
>
> Thanks,
> Payas
>
> On Mon, 19 Sep, 2022, 08:08 Stefan Monnier, <monnier@iro.umontreal.ca>
> wrote:
>>
>> > # filter eglot with only eglot.el and find+replace # in commit messages
>> > git filter-repo --path eglot.el --message-callback '
>> >       return re.sub(b" #", b"
>> > https://github.com/joaotavora/eglot/issues/", message)
>> >       return re.sub(b",#",
>> > b",https://github.com/joaotavora/eglot/issues/", message)
>> >       return re.sub(b"(#",
>> > b"(https://github.com/joaotavora/eglot/issues/", message)
>> >       return re.sub(b"Fix#", b"Fix:
>> > https://github.com/joaotavora/eglot/issues/", message)
>> >       return re.sub(b"github#", b"Github:
>> > https://github.com/joaotavora/eglot/issues/", message)
>>
>> I suspect you can get a slightly better result if you tweak this filter
>> script so as to rename `eglot.el` to `lisp/progmodes/eglot.el` in each
>> commit.
>>
>> Then you can also skip the `git read-tree`.
>>
>>
>>         Stefan
>>
>

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 2779 bytes --]

Hi Payas,

I haven't had the to look at this in depth, but this looks mighty fine,
including this last idea of putting two "fixes" found in the subject line
in two lines of the body.

Many thanks,
João

On Mon, Sep 19, 2022, 05:42 Payas Relekar <relekarpayas@gmail.com> wrote:

> Another way I can think of is to keep #nums where they are right now,
> but add their respective links in the message body below.
>
> e.g.
> Fixes #234,#456
>
> this fixes two issues
>
> Becomes =>
>
> Fixes #234,#456
>
> this fixes two issues
> #234: https://github.com/joaotavora/eglot/234
> #456: https://github.com/joaotavora/eglot/456
>
> Inputs welcome, since this is not 100% convenient, but much simpler to
> implement, with minimal to none manual readjustment needed.
>
> Thanks,
> Payas
>
> On 9/19/22, Payas Relekar <relekarpayas@gmail.com> wrote:
> > Thank you! Indeed moving to subdirectory makes history apply to
> > correct file and shows up as expected. Was also quite straightforward
> > with git-filter-repo, so here's the updates:
> >
> > Updates gist:
> > https://gist.github.com/bhankas/422c34db2e987d4dc3f243e26ba156a3
> >
> > Updated repo:
> > https://github.com/bhankas/emacs
> >
> > Updated history:
> > https://github.com/bhankas/emacs/commits/master/lisp/progmodes/eglot.el
> >
> > Now the only remaining issue is commit length. Can someone please
> > advice on how we can go about it? My brute force thinking is to simply
> > add newline where length exceeds, delimited at word boundary, but
> > obviously that's not good enough nor covers sufficient edge cases.
> > Perhaps a Python/Perl wizard has a one/two liner to take that pain
> > away?
> >
> > Thanks,
> > Payas
> >
> > On Mon, 19 Sep, 2022, 08:08 Stefan Monnier, <monnier@iro.umontreal.ca>
> > wrote:
> >>
> >> > # filter eglot with only eglot.el and find+replace # in commit
> messages
> >> > git filter-repo --path eglot.el --message-callback '
> >> >       return re.sub(b" #", b"
> >> > https://github.com/joaotavora/eglot/issues/", message)
> >> >       return re.sub(b",#",
> >> > b",https://github.com/joaotavora/eglot/issues/", message)
> >> >       return re.sub(b"(#",
> >> > b"(https://github.com/joaotavora/eglot/issues/", message)
> >> >       return re.sub(b"Fix#", b"Fix:
> >> > https://github.com/joaotavora/eglot/issues/", message)
> >> >       return re.sub(b"github#", b"Github:
> >> > https://github.com/joaotavora/eglot/issues/", message)
> >>
> >> I suspect you can get a slightly better result if you tweak this filter
> >> script so as to rename `eglot.el` to `lisp/progmodes/eglot.el` in each
> >> commit.
> >>
> >> Then you can also skip the `git read-tree`.
> >>
> >>
> >>         Stefan
> >>
> >
>

[-- Attachment #2: Type: text/html, Size: 4914 bytes --]

Re: Progress on merging Eglot?

[Payas Relekar]

João Távora <joaotavora@gmail.com> writes:

> I haven't had the to look at this in depth, but this looks mighty fine,
> including this last idea of putting two "fixes" found in the subject line
> in two lines of the body.
>
> On Mon, Sep 19, 2022, 05:42 Payas Relekar <relekarpayas@gmail.com> wrote:
>
>> Another way I can think of is to keep #nums where they are right now,
>> but add their respective links in the message body below.
>>
>> e.g.
>> Fixes #234,#456
>>
>> this fixes two issues
>>
>> Becomes =>
>>
>> Fixes #234,#456
>>
>> this fixes two issues
>> #234: https://github.com/joaotavora/eglot/234
>> #456: https://github.com/joaotavora/eglot/456
>>

Alright! I think we have everything we need. I took the append URLs to
bottom of commit message approach. Here's what we have:

- History visible for file lisp/progmodes/eglot.el
- Commit message length undisturbed
- every #num is appended with appropriate URL at the bottom of commit
  message

These are the latest and (hopefully) final URLs:

Code:
https://gist.github.com/bhankas/422c34db2e987d4dc3f243e26ba156a3

(Please ensure eglot2emacs.py is in same directory as eglot/ and emacs/
repos)

Emacs repo with eglot.el merged:
https://github.com/bhankas/emacs

eglot.el file history within merged emacs repo:
https://github.com/bhankas/emacs/commits/master/lisp/progmodes/eglot.el

If all is good, that concludes #2 on João's list.

Let me know for any corrections and/or improvements.

Thanks,
Payas

--

Re: Progress on merging Eglot?

[Payas Relekar]

[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]

João Távora <joaotavora@gmail.com> writes:

> I haven't had the to look at this in depth, but this looks mighty fine,
> including this last idea of putting two "fixes" found in the subject line
> in two lines of the body.
>
> On Mon, Sep 19, 2022, 05:42 Payas Relekar <relekarpayas@gmail.com> wrote:
>
>> Another way I can think of is to keep #nums where they are right now,
>> but add their respective links in the message body below.
>>
>> e.g.
>> Fixes #234,#456
>>
>> this fixes two issues
>>
>> Becomes =>
>>
>> Fixes #234,#456
>>
>> this fixes two issues
>> #234: https://github.com/joaotavora/eglot/234
>> #456: https://github.com/joaotavora/eglot/456
>>

Alright! I think we have everything we need. I took the append URLs to
bottom of commit message approach. Here's what we have:

- History visible for file lisp/progmodes/eglot.el
- Commit message length undisturbed
- every #num is appended with appropriate URL at the bottom of commit
  message

These are the latest and (hopefully) final URLs:

Code:
https://gist.github.com/bhankas/422c34db2e987d4dc3f243e26ba156a3

(Please ensure eglot2emacs.py is in same directory as eglot/ and emacs/
repos)

Emacs repo with eglot.el merged:
https://github.com/bhankas/emacs

eglot.el file history within merged emacs repo:
https://github.com/bhankas/emacs/commits/master/lisp/progmodes/eglot.el

If all is good, that concludes #2 on João's list.

Let me know for any corrections and/or improvements.

Thanks,
Payas

--

[-- Attachment #2: Type: text/html, Size: 2622 bytes --]

Re: Progress on merging Eglot?

[Payas Relekar]

Payas Relekar <relekarpayas@gmail.com> writes:

> João Távora <joaotavora@gmail.com> writes:
>
>> I haven't had the to look at this in depth, but this looks mighty fine,
>> including this last idea of putting two "fixes" found in the subject line
>> in two lines of the body.
>>
> Alright! I think we have everything we need. I took the append URLs to
> bottom of commit message approach. Here's what we have:
>
> - History visible for file lisp/progmodes/eglot.el
> - Commit message length undisturbed
> - every #num is appended with appropriate URL at the bottom of commit
>   message
>
> These are the latest and (hopefully) final URLs:
>
> Code:
> https://gist.github.com/bhankas/422c34db2e987d4dc3f243e26ba156a3
>
> (Please ensure eglot2emacs.py is in same directory as eglot/ and emacs/
> repos)
>
> Emacs repo with eglot.el merged:
> https://github.com/bhankas/emacs
>
> eglot.el file history within merged emacs repo:
> https://github.com/bhankas/emacs/commits/master/lisp/progmodes/eglot.el

Jumped a bit :)
Made another, this time final final correction (properly strip excess
before # in URL) and ready for review.

git is updated with revision 4, all other links are same.

> If all is good, that concludes #2 on João's list.
>
> Let me know for any corrections and/or improvements.
>
> Thanks,
> Payas

Thanks,
Payas

--

Re: Progress on merging Eglot?

[Stefan Monnier]

Payas Relekar [2022-09-19 10:12:34] wrote:
> Another way I can think of is to keep #nums where they are right now,
> but add their respective links in the message body below.

I was about to say that the overlong lines are perfectly fine (the main
focus is to preserve information, rather than make it pretty), but this
is even better.


        Stefan

Re: Progress on merging Eglot?

[Payas Relekar]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Payas Relekar [2022-09-19 10:12:34] wrote:
>> Another way I can think of is to keep #nums where they are right now,
>> but add their respective links in the message body below.
>
> I was about to say that the overlong lines are perfectly fine (the main
> focus is to preserve information, rather than make it pretty), but this
> is even better.
>

Thanks. I think the script is generic enough that it can be run second
time with minor changes to have the tests file too. That can maybe help
with #3 on Joao's list (adjusting tests).

#1 seems trivial enough (:core ELPA package) and #5 (emacs core ->
 github sync for eglot.el) is low/non priority, so I am looking at #4.

#4 being checking/verifying copyright assignments. Joao mentioned some
 are copyright exempt, and I wouldn't know where to start.

Any pointers on that? Probably only one of Emacs maintainers can verify
assignments, but anything you need on the way? List of author names
and/or emails?

Thanks,
Payas
--

Re: Progress on merging Eglot?

[João Távora]

Payas Relekar <relekarpayas@gmail.com> writes:

> Alright! I think we have everything we need. I took the append URLs to
> bottom of commit message approach. Here's what we have:
>
> - History visible for file lisp/progmodes/eglot.el
> - Commit message length undisturbed
> - every #num is appended with appropriate URL at the bottom of commit
>   message
>
> These are the latest and (hopefully) final URLs:
>
> Code:
> https://gist.github.com/bhankas/422c34db2e987d4dc3f243e26ba156a3
>
> (Please ensure eglot2emacs.py is in same directory as eglot/ and emacs/
> repos)
>
> Emacs repo with eglot.el merged:
> https://github.com/bhankas/emacs
>
> eglot.el file history within merged emacs repo:
> https://github.com/bhankas/emacs/commits/master/lisp/progmodes/eglot.el
>
> If all is good, that concludes #2 on João's list.
>
> Let me know for any corrections and/or improvements.

Thanks very much Payas, your work provided a most excellent start.

I took your GitHub gist, forked it, and reworked it.  That version now
lives at

   https://gist.github.com/joaotavora/2ed97f2ec85958986983d5cb78202770

and I ask that any work proceeds from there.

Most of my work happened in the eglot2emacs.py script, which I have
renamed eglot2emacs-message-callback.py.  After many tweaks, I ran it
through git filter-repo and think this version is now my preferred one.
For instance it transforms:

   Close #974, #776: Update docstring of eglot-events-buffer-size
    
   * eglot.el (eglot-events-buffer-size): Mention that you need to<CARRIAGE-RETURN>
   restart the connection for 'eglot-events-buffer-size' to take effect.

into

   Update docstring of eglot-events-buffer-size

   * eglot.el (eglot-events-buffer-size): Mention that you need to
   restart the connection for 'eglot-events-buffer-size' to take effect.
    
   GitHub-reference: Close https://github.com/joaotavora/eglot/issues/974
   GitHub-reference: Close https://github.com/joaotavora/eglot/issues/776

It also does other small tricks (and maybe some more can be added)

The only problem is that I can't get run the full eglot2emacs.sh to run.
The process is blocked, I think, by an Emacs commit hook that refuses to
accept some trailing changes in some unrelated files.  Can you see
what's up with that?  Did I mess up eglot2emacs.sh?

João

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1143 bytes --]

On Mon, Sep 19, 2022 at 2:57 PM Payas Relekar <relekarpayas@gmail.com>
wrote:

>
> #1 seems trivial enough (:core ELPA package) and #5 (emacs core ->
>  github sync for eglot.el) is low/non priority, so I am looking at #4.
>

#5 is a priority, it's just not the _top_ priority right now.  But it's
pretty important.
Let's just make sure we understand what it means.  It does _not_
necessarily
mean a perfect history sync emacs.git -> by-then-old GitHub repo.

#1 means changing elpa.git.  And should be reasonably easy to do once #2
is done


> #4 being checking/verifying copyright assignments. Joao mentioned some
>  are copyright exempt, and I wouldn't know where to start.
>
Any pointers on that? Probably only one of Emacs maintainers can verify
> assignments, but anything you need on the way? List of author names
> and/or emails?
>

You're right.  Only people with access to the copyright file can check that.

A unique list of author names and emails is very welcome.  You can filter
out the authors whose only commit contains a line

Copyright-paperwork-exempt: yes

In the commit message.

João

[-- Attachment #2: Type: text/html, Size: 1988 bytes --]

Re: Progress on merging Eglot?

[Payas Relekar]

João Távora <joaotavora@gmail.com> writes:

> The only problem is that I can't get run the full eglot2emacs.sh to run.
> The process is blocked, I think, by an Emacs commit hook that refuses to
> accept some trailing changes in some unrelated files.  Can you see
> what's up with that?  Did I mess up eglot2emacs.sh?
>

I'll probably won't be able to test again today, but from what I see
your .sh is pretty much identical to mine.

.py is well beyond my kindergarten python, but if I had to make a guess
its probably generating something that Emacs doesn't like. My script did
not add any newlines to commit messages, and yours does, but there's
enough stuff I don't know about git/python/bash to make any conclusive
statement.

Thanks,
Payas

--

Re: Progress on merging Eglot?

[Payas Relekar]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]


João Távora <joaotavora@gmail.com> writes:

> You're right.  Only people with access to the copyright file can check that.
>
> A unique list of author names and emails is very welcome.  You can filter
> out the authors whose only commit contains a line
>
> Copyright-paperwork-exempt: yes
>
> In the commit message.

Please find attached the list of authors without copyright exemption
string in commit message.

I tried to remove duplicates, but there are some by name and some by
email. The list is small enough that they can be identified with a
glance.

This is the filter used:

eglot_authors.py:
######
log = b""
if (not re.search(b"Copyright-paperwork-exempt: yes", commit.message)):
    log += b"\n" + commit.author_name + b"\t" + commit.author_email
    print(commit.author_name + b"\t" + commit.author_email)
    with open("temp", "a") as text_file:
        text_file.write(log.decode("utf-8"))
######

command:
*****
cd eglot && git filter-repo --commit-callback ../eglot_authors.py && cat \
temp | sort -h | uniq | sed 's/\t/\t\t/g'
*****

QnD, but good enough IMO.

Let me know for any corrections.

Thanks,
Payas

--

[-- Attachment #2: eglot_authors.txt --]
[-- Type: text/plain, Size: 3268 bytes --]

Alan Zimmerman          alan.zimm@gmail.com
Alex Branham            jabranham@users.noreply.github.com
Andrii Kolomoiets       andreyk.mad@gmail.com
Augusto Stoffel         arstoffel@gmail.com
Augusto Stoffel         astoff@users.noreply.github.com
Basil L. Contovounesios contovob@tcd.ie
Billy.Zheng             vil963@gmail.com
Brian Cully             bjc@kublai.com
Brian Leung             29217594+leungbk@users.noreply.github.com
Brian Leung             bkleung89@gmail.com
Brian Leung             leungbk@mailfence.com
Brian Leung             leungbk@posteo.net
brotzeit                brotzeitmacher@gmail.com
Christian Garbs         mitch@cgarbs.de
Damien Merenne          damien@bloom-life.com
Dan Davison             dandavison7@gmail.com
David Florness          edwargix@gmail.com
Derek Passen            dpassen1@gmail.com
Dmitry Gutov            dgutov@yandex.ru
Evgeni Kolev            evgenysw@gmail.com
Fangrui Song            i@maskray.me
Felicián Németh         felician.nemeth@gmail.com
Fredrik Bergroth        fbergroth@gmail.com
Garret Buell            gmbuell@gmail.com
Illia Danko             illia@aigent.com
Ingo Lohmar             ingo.lohmar@github.com
Ingo Lohmar             ingo.lohmar@posteo.net
James N         		james@jojojames.com
jgart           		47760695+jgarte@users.noreply.github.com
jicksaw         		jicksaw@pm.me
Jim Porter          	826865+jimporter@users.noreply.github.com
Jim Porter          	itsjimporter@gmail.com
João Távora         	joaotavora@gmail.com
Josh Elsasser       	jelsasser@appneta.com
Jürgen Hötzel       	juergen@archlinux.org
lorniu/sz           	lorniu@gmail.com
Manuel Uberti       	manuel-uberti@users.noreply.github.com
Marcus Swanson      	marcus.swanson91@gmail.com
Mario Rodas         	marsam@users.noreply.github.com
Martin Carlson      	spearalot@gmail.com
Michael Livshin     	repo@cmm.kakpryg.net
Michał K            	k.michal@zoho.com
Michał Krzywkowski  	k.michal@zoho.com
mkcms           		k.michal@zoho.com
Mohsin Kaleem       	mohkale@kisara.moe
muffinmad           	andreyk.mad@gmail.com
NA              		na@aisrntairetnraoitn.com
Omar Polo           	op@omarpolo.com
Paul M. Rodriguez   	paulmrodriguez@gmail.com
Philip K            	philip@warpmail.net
Philipp Edelmann    	edelmann@fs.tum.de
Philipp Stephani    	p.stephani2@gmail.com
rbrtb               	104695105+rbrtb@users.noreply.github.com
Rudolf Schlatte     	rudi@constantly.at
Sergey Kostyaev     	s-kostyaev@users.noreply.github.com
Stefan Kangas       	stefankangas@gmail.com
Stefan Kangas       	stefan@marxist.se
Stefan Monnier      	monnier@iro.umontreal.ca
Stephen Leake       	stephen.leake84@gmail.com
Stephen Leake       	stephen_leake@stephe-leake.org
Steve Purcell       	steve@sanityinc.com
Theodor Thornhill   	theodorthornhill@icloud.com
Theodor Thornhill   	theo@thornhill.no
Theodor Thornhill   	theothornhill@pm.me
theothornhill       	theodorthornhill@icloud.com
Tobias Rittweiler   	trittweiler@gmail.com
Tomasz Hołubowicz   	45176912+alternateved@users.noreply.github.com
Tom Tromey          	tom@tromey.com
Trevor Murphy       	trevormurphy@google.com
Xiang Ji            	hi@xiangji.me
Yuan Fu         		casouri@gmail.com

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]

On Mon, Sep 19, 2022 at 4:52 PM Payas Relekar <relekarpayas@gmail.com>
wrote:

>
> > The only problem is that I can't get run the full eglot2emacs.sh to run.
> > The process is blocked, I think, by an Emacs commit hook that refuses to
> > accept some trailing changes in some unrelated files.  Can you see
> > what's up with that?  Did I mess up eglot2emacs.sh?
> >
>
> .py is well beyond my kindergarten python, but if I had to make a guess
> its probably generating something that Emacs doesn't like.
>

No, that wasn't it.  It probably worked for you because your script clones
a fresh
emacs.git repo without installing the pre-commit hooks.  When they are in
place,
the merge is prevented.  I haven't investigated why: there is some logic in
~/.git/hooks/pre-commit that supposedly nullifies the hook for merges.

Anyway, when the hook is laid to rest, the result is something like what I
just uploaded to the scratch/eglot2emacs branch:

This branch will probably be deleted and re-created frequently as we tweak
the script.
I hope that isn't a problem.  Stefan?

João

[-- Attachment #2: Type: text/html, Size: 1624 bytes --]

Re: Progress on merging Eglot?

[Payas Relekar]

João Távora <joaotavora@gmail.com> writes:

> On Mon, Sep 19, 2022 at 4:52 PM Payas Relekar <relekarpayas@gmail.com>
> wrote:
>
>> .py is well beyond my kindergarten python, but if I had to make a guess
>> its probably generating something that Emacs doesn't like.
>>
>
> No, that wasn't it.  It probably worked for you because your script clones
> a fresh
> emacs.git repo without installing the pre-commit hooks.  When they are in
> place,
> the merge is prevented.  I haven't investigated why: there is some logic in
> ~/.git/hooks/pre-commit that supposedly nullifies the hook for merges.

Possible, every time I made changes to script, I had emacs repo hard
reset to upstream master. Maybe try that first?

> Anyway, when the hook is laid to rest, the result is something like what I
> just uploaded to the scratch/eglot2emacs branch:
>
> This branch will probably be deleted and re-created frequently as we tweak
> the script.
> I hope that isn't a problem.  Stefan?

Excellente! And away we go....

Payas

--

Re: Progress on merging Eglot?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]

On Mon, Sep 19, 2022 at 5:54 PM Payas Relekar <relekarpayas@gmail.com>
wrote:

> João Távora <joaotavora@gmail.com> writes:
>
> > No, that wasn't it.  It probably worked for you because your script
> clones
> > a fresh
> > emacs.git repo without installing the pre-commit hooks.  When they are in
> > place,
> > the merge is prevented.  I haven't investigated why: there is some logic
> in
> > ~/.git/hooks/pre-commit that supposedly nullifies the hook for merges.
>
> Possible, every time I made changes to script, I had emacs repo hard
> reset to upstream master. Maybe try that first?
>

That doesn't affect the hooks already commited to ~/.git/hooks
The reason we're getting bitten by this is because I'm using a "git
worktree"
working tree which shared the ~/.git/ directory with the principal working
tree.

I'll add a comment to the gist to warn about this pitfall.  Meanwhile,
using a fresh
non-git-worktree clone also fixes it.

> Anyway, when the hook is laid to rest, the result is something like what I
> > just uploaded to the scratch/eglot2emacs branch:
> >
> > This branch will probably be deleted and re-created frequently as we
> tweak
> > the script.
> > I hope that isn't a problem.  Stefan?
>
> Excellente! And away we go....
>

Yes, thanks for kicking things off!

João

[-- Attachment #2: Type: text/html, Size: 2060 bytes --]

Progress on merging Eglot: update

[João Távora]

Hi,

We've worked a little bit last week on getting Eglot merged into the
core and this is the latest status:

1. TODO Eglot in emacs.git, published to ELPA as a :core package

   I think this is very easy consists of tweaking elpa.git with this
   patch once item 2 is done:

   diff --git a/elpa-packages b/elpa-packages
   index d87a9f9e4e..10379a9c32 100644
   --- a/elpa-packages
   +++ b/elpa-packages
   @@ -139,7 +139,7 @@
   - ("eglot"               :url "https://github.com/joaotavora/eglot.git")
   + ("eglot"               :core "lisp/progmodes/eglot.el")

2. DONE Preserve full Git history in lisp/progmodes/eglot.el file after merge

   A final version of the scripts lives in:

   https://gist.github.com/joaotavora/2ed97f2ec85958986983d5cb78202770

   I think the commit messages look fine.  I tweaked the emails to match
   what we have in the copyright file and made the author's names
   uniform.

   The scratch/eglot2emacs branch contains the latest result of running
   the ./eglot2emacs.sh script.

3. TODO Adapt tests in eglot-tests.el into Emacs's test suite

   Volunteers welcome for this, but reasonably low priority.

4. DONE Check carefully for copyright assignments for Eglot's many
   contributors

   I've done this.  The scripts mentioned in item 2 describe how I
   proceeded.

5. HELP rewrite https://github.com/joaotavora/eglot
   to list/progmodes/eglot.el

   I need to come up with a git incantation to run periodically so that
   the patches that happen in list/progmodes/eglot.el upstream can
   somehow be applied to the GitHub downstream-used-to-be-upstream.

   Then I need to know if there's some kind of FSF machine where a cron
   job can run these scripts periodically.  Maybe where the Elpa.git
   stuff runs?

6. DOING Adapt the README.md into a manual section in TexInfo format.

   Eli Zaretskii will handle this part once I hand him a slightly better
   structured README.md


João

Re: Progress on merging Eglot: update

[Payas Relekar]

Thanks a lot for all the work! Looking forward to Eglot in Emacs 29.1.

João Távora <joaotavora@gmail.com> writes:

> 5. HELP rewrite https://github.com/joaotavora/eglot
>    to list/progmodes/eglot.el
>
>    I need to come up with a git incantation to run periodically so that
>    the patches that happen in list/progmodes/eglot.el upstream can
>    somehow be applied to the GitHub downstream-used-to-be-upstream.

Can we use git-filter repo in similar manner as we did now, but instead
of moving to child dir (lisp/progmodes/), we move to parent dir
(../../)?

Then we can simply force push to github. Admittedly I haven't tested
this yet, but sounds doable. That way commit history will be preserved
both ways.

>    Then I need to know if there's some kind of FSF machine where a cron
>    job can run these scripts periodically.  Maybe where the Elpa.git
>    stuff runs?

Thanks,
Payas

--

Re: Progress on merging Eglot: update

[Payas Relekar]

Payas Relekar <relekarpayas@gmail.com> writes:

> João Távora <joaotavora@gmail.com> writes:
>
>> 5. HELP rewrite https://github.com/joaotavora/eglot
>>    to list/progmodes/eglot.el
>>
>>    I need to come up with a git incantation to run periodically so that
>>    the patches that happen in list/progmodes/eglot.el upstream can
>>    somehow be applied to the GitHub downstream-used-to-be-upstream.
>
> Can we use git-filter repo in similar manner as we did now, but instead
> of moving to child dir (lisp/progmodes/), we move to parent dir
> (../../)?
>
> Then we can simply force push to github. Admittedly I haven't tested
> this yet, but sounds doable. That way commit history will be preserved
> both ways.

Okay, tested and apparently working. It's a bit convoluted, but seems to
be working..

Here's the steps (assuming eglot/ and emacs/ repos are cloned in same
dir):

***
cd emacs/

# reduce to lisp/progmodes
git filter-repo --subdirectory-filter lisp/progmodes/

# reduce to eglot.el
git filter-repo --path eglot.el

cd ../eglot/

# remove eglot.el (to be pulled from emacs)
git filter-repo --invert-paths --path eglot.el

# add emacs.git as upstream
git remote add emacs ../emacs/

# fetch latest emacs.git with eglot.el
git fetch emacs master

# merge both repos
git merge remotes/emacs/master --allow-unrelated-histories --no-commit

# make a commit
git commit -m "; Merge from emacs.git"
***

Here's a repo with test commit added from emacs.git on top:
https://github.com/bhankas/eglot

I did not check extensively, but on (very) quick skim did not notice any
obvious duplication in the log.

Let me know if its helpful or needs more work.

Thanks,
Payas

--

Re: Progress on merging Eglot: update

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >    I need to come up with a git incantation to run periodically so that
  > >    the patches that happen in list/progmodes/eglot.el upstream can
  > >    somehow be applied to the GitHub downstream-used-to-be-upstream.

  > Can we use git-filter repo in similar manner as we did now, but instead
  > of moving to child dir (lisp/progmodes/), we move to parent dir
  > (../../)?

  > Then we can simply force push to github. Admittedly I haven't tested
  > this yet, but sounds doable. That way commit history will be preserved
  > both ways.

Using GitHub this way is no disaster in practice, but it is embarrassing.
Could we move that repo to a better place, such as sourcehut?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Progress on merging Eglot: update

[Payas Relekar]

Richard Stallman <rms@gnu.org> writes:

> Using GitHub this way is no disaster in practice, but it is embarrassing.
> Could we move that repo to a better place, such as sourcehut?

I believe Joao's reasons were to keep the users who target the github
repo from having their config broken, as well as provide another avenue
to submit issues/patches.

Modus-themes is hosted on sourcehut but also provides mirrors on Github
and Gitlab for similar reasons. Prot (modus-themes author/maintainer)
uses sourcehut as his primary hosting provider, but Joao plans to use
emacs.git for that purpose. As such, yet another mirror using mailing
list for issues/patches seems redundant. It will be his call in the end.

--

Re: Progress on merging Eglot: update

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: joaotavora@gmail.com, emacs-devel@gnu.org, eliz@gnu.org
> Date: Sat, 24 Sep 2022 22:46:18 -0400
> 
> Using GitHub this way is no disaster in practice, but it is embarrassing.
> Could we move that repo to a better place, such as sourcehut?

I don't think it makes sense to invest such an effort now, since we
are going to import Eglot into Emacs very soon, as soon as the
remaining technical issues are resolved.  Moving it to another hosting
service now will just delay the merge with Emacs, which is undesirable
since we want Eglot in Emacs before we start the Emacs 29 release
cycle.

Re: Progress on merging Eglot: update

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I don't think it makes sense to invest such an effort now, since we
  > are going to import Eglot into Emacs very soon, as soon as the
  > remaining technical issues are resolved.  Moving it to another hosting
  > service now will just delay the merge with Emacs, which is undesirable
  > since we want Eglot in Emacs before we start the Emacs 29 release
  > cycle.

Are you saying that this connection with GitHub is temporary?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Progress on merging Eglot: update

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 977 bytes --]

On Mon, Sep 26, 2022 at 1:10 PM Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > I don't think it makes sense to invest such an effort now, since we
>   > are going to import Eglot into Emacs very soon, as soon as the
>   > remaining technical issues are resolved.  Moving it to another hosting
>   > service now will just delay the merge with Emacs, which is undesirable
>   > since we want Eglot in Emacs before we start the Emacs 29 release
>   > cycle.
>
> Are you saying that this connection with GitHub is temporary?
>

That's correct.  My plan is to sever it completely, but in the short- to
medium-term it is still useful.  At any rate, GitHub will cease to be the
"upstream" of Eglot source once the merge to core is done.

João

[-- Attachment #2: Type: text/html, Size: 1401 bytes --]

Re: Progress on merging Eglot: update

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Are you saying that this connection with GitHub is temporary?
  > >

  > That's correct.  My plan is to sever it completely, but in the short- to
  > medium-term it is still useful.  At any rate, GitHub will cease to be the
  > "upstream" of Eglot source once the merge to core is done.

Thanks.  I agree it's not something to worry about if it is temporary.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)