Guile 3 and wip-elisp/Emacs

Guile 3 and wip-elisp/Emacs

[Gregg Sangster]

Hello Guile Hackers,

I've rebased the wip-elisp branch on top of commit 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe: v3.0.4-64-g33232cb5c4).  It's published here:

https://git.sr.ht/~g20r/guile

There are two additional failed tests which appear to be new tests added since the last work on the wip-elisp branch.  Also, ./configure requires the --disable-jit option as the build fails otherwise.  I have not yet investigated any of these issues.

I don't have much experience with Guile itself or the Guile codebase.  The next commit has a conflict I was not able to resolve: that's why it's not rebased all the way up to the tip of main.  My next tasks are to get the current Guile Emacs running on this version of Guile followed by fixing the JIT build failure, rebasing this to the tip of main and rebasing Guile Emacs on a more recent Emacs (I believe it is currently v24.something).

My question is if this is worthwhile work?  I don't have much time to spend on it but am happy to keep hacking away if there's still upstream interest.

--
Gregg

Guile 3 and wip-elisp/Emacs

[Gregg Sangster]

Hello Guile Hackers,

I've rebased the wip-elisp branch on top of commit 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe: v3.0.4-64-g33232cb5c4).  It's published here:

https://git.sr.ht/~g20r/guile

There are two additional failed tests which appear to be new tests added since the last work on the wip-elisp branch.  Also, ./configure requires the --disable-jit option as the build fails otherwise.  I have not yet investigated any of these issues.

I don't have much experience with Guile itself or the Guile codebase.  The next commit has a conflict I was not able to resolve: that's why it's not rebased all the way up to the tip of main.  My next tasks are to get the current Guile Emacs running on this version of Guile followed by fixing the JIT build failure, rebasing this to the tip of main and rebasing Guile Emacs on a more recent Emacs (I believe it is currently v24.something).

My question is if this is worthwhile work?  I don't have much time to spend on it but am happy to keep hacking away if there's still upstream interest.

--
Gregg

Guile 3 and wip-elisp/Emacs

[Gregg Sangster]

Hello Guile Hackers,

I've rebased the wip-elisp branch on top of commit 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe: v3.0.4-64-g33232cb5c4).  It's published here:

https://git.sr.ht/~g20r/guile

There are two additional failed tests which appear to be new tests added since the last work on the wip-elisp branch.  Also, ./configure requires the --disable-jit option as the build fails otherwise.  I have not yet investigated any of these issues.

I don't have much experience with Guile itself or the Guile codebase.  The next commit has a conflict I was not able to resolve: that's why it's not rebased all the way up to the tip of main.  My next tasks are to get the current Guile Emacs running on this version of Guile followed by fixing the JIT build failure, rebasing this to the tip of main and rebasing Guile Emacs on a more recent Emacs (I believe it is currently v24.something).

My question is if this is worthwhile work?  I don't have much time to spend on it but am happy to keep hacking away if there's still upstream interest.

--
Gregg

Re: Guile 3 and wip-elisp/Emacs

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]

Hello Gregg,
"Gregg Sangster" <gregg@thesangsters.ca> writes:

> I've rebased the wip-elisp branch on top of commit 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe: v3.0.4-64-g33232cb5c4).  It's published here:
>
> https://git.sr.ht/~g20r/guile

I’m not a Guile core developer, but I think that this is awesome!

> My question is if this is worthwhile work?  I don't have much time to spend on it but am happy to keep hacking away if there's still upstream interest.

I think that there is upstream interest in Guile to improve the elisp
implementation, but I’m not sure about Emacs. They added improvements to
elisp that also provide better performance and they keep improving their
elisp implementation.

That said: Tools like fibers would be great to have in Emacs.

The workload to finish this is considerable, though: IIRC You’ll need to
solve some deeper problems that prevent Guile Emacs from using
byte-compiled files (that’s why it currently has a very high startup
time).

That said, there is a guile-emacs package in guix, and Guile 3.0.7 is
much faster than Guile 2.

(please someone correct me if I’m wrong!)

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Guile 3 and wip-elisp/Emacs

[Christine Lemmer-Webber]

"Dr. Arne Babenhauserheide" <arne_bab@web.de> writes:

> [[PGP Signed Part:Undecided]]
> Hello Gregg,
> "Gregg Sangster" <gregg@thesangsters.ca> writes:
>
>> I've rebased the wip-elisp branch on top of commit
>> 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe:
>> v3.0.4-64-g33232cb5c4).  It's published here:
>>
>> https://git.sr.ht/~g20r/guile
>
> I’m not a Guile core developer, but I think that this is awesome!
>
>> My question is if this is worthwhile work?  I don't have much time to
>> spend on it but am happy to keep hacking away if there's still
>> upstream interest.
>
> I think that there is upstream interest in Guile to improve the elisp
> implementation, but I’m not sure about Emacs. They added improvements to
> elisp that also provide better performance and they keep improving their
> elisp implementation.

I would personally think it would be good if we could get the wip-elisp
branch merged.  It would be good not to bitrot.  It seems more likely to
survive if it gets included in emacs proper.

> That said: Tools like fibers would be great to have in Emacs.
>
> The workload to finish this is considerable, though: IIRC You’ll need to
> solve some deeper problems that prevent Guile Emacs from using
> byte-compiled files (that’s why it currently has a very high startup
> time).
>
> That said, there is a guile-emacs package in guix, and Guile 3.0.7 is
> much faster than Guile 2.

Yes, to complete *guile-emacs*... that will be a lot of work.  But it
will be less work if guile's elisp branch is no longer this separate
thing that isn't kept up to date with the rest of guile.  More likely
that someone could finish and pick it up.

> (please someone correct me if I’m wrong!)
>
> Best wishes,
> Arne

Re: Guile 3 and wip-elisp/Emacs

[Ricardo Wurmus]

Hi Gregg,

> I've rebased the wip-elisp branch on top of commit 
> 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe: 
> v3.0.4-64-g33232cb5c4).  It's published here:
>
> https://git.sr.ht/~g20r/guile

FWIW I had also rebased wip-elisp in 2020:

    https://git.elephly.net/?p=software/guile.git;a=shortlog;h=refs/heads/wip-elisp

-- 
Ricardo

Re: Guile 3 and wip-elisp/Emacs

[Gerry Agbobada]

Hello,

Just my 2 cents as a drive-by watcher: I agree that having wip-elisp branch in guile tree would be nice to have, this way it's easier to find out where progress is being made for this endeavour, with potentially a checked out TODO or TASKS file to see if I can ever help (if only with testing, I know nothing of compilation so I don't think I'll be able to contribute to an .elc parser/evaluator/loader)

Regards,

-- 
Gerry

Re: Guile 3 and wip-elisp/Emacs

[Gregg Sangster]

Hi Ricardo,

Thanks,  your changes fixed a couple of the test errors I saw.

On Monday, September 13, 2021 06:56 EDT, Ricardo Wurmus <rekado@elephly.net> wrote:

>
> Hi Gregg,
>
> > I've rebased the wip-elisp branch on top of commit
> > 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe:
> > v3.0.4-64-g33232cb5c4).  It's published here:
> >
> > https://git.sr.ht/~g20r/guile
>
> FWIW I had also rebased wip-elisp in 2020:
>
>     https://git.elephly.net/?p=software/guile.git;a=shortlog;h=refs/heads/wip-elisp
>
> --
> Ricardo
>

Re: Guile 3 and wip-elisp/Emacs

[Gregg Sangster]

Hello,

I have wip-elisp rebased all the way up to main as of a few days ago (e60469c8b6936575c079faaffa40a340e1d49f3c) plus two changes from Ricardo.  It's available here:

https://git.sr.ht/~g20r/guile

There is one test failure in "make check" on test-out-of-memory.  I haven't investigated it yet but I get the same failure on main.  If that's not an expected failure, it might be a problem with my environment.

There is also an emacs repo here:

https://git.sr.ht/~g20r/emacs

which uses the 3.0-based wip-elisp.  It builds and runs but segfaults easily.  I've started rebasing it on a more current emacs.  Those changes aren't published yet.

I haven't been able to get the guile-emacs package in guix working.  The build gets stuck at collecting/processing OKURI-NASI entries after a few hours.  For comparison, the 3.0-based guile-emacs builds in about 20 minutes on my machine.  If anyone has that running, it would be nice to have a comparison of the build/run speed.

On Sunday, September 12, 2021 22:13 EDT, Christine Lemmer-Webber <cwebber@dustycloud.org> wrote:

> "Dr. Arne Babenhauserheide" <arne_bab@web.de> writes:
>
> > [[PGP Signed Part:Undecided]]
> > Hello Gregg,
> > "Gregg Sangster" <gregg@thesangsters.ca> writes:
> >
> >> I've rebased the wip-elisp branch on top of commit
> >> 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe:
> >> v3.0.4-64-g33232cb5c4).  It's published here:
> >>
> >> https://git.sr.ht/~g20r/guile
> >
> > I’m not a Guile core developer, but I think that this is awesome!
> >
> >> My question is if this is worthwhile work?  I don't have much time to
> >> spend on it but am happy to keep hacking away if there's still
> >> upstream interest.
> >
> > I think that there is upstream interest in Guile to improve the elisp
> > implementation, but I’m not sure about Emacs. They added improvements to
> > elisp that also provide better performance and they keep improving their
> > elisp implementation.
>
> I would personally think it would be good if we could get the wip-elisp
> branch merged.  It would be good not to bitrot.  It seems more likely to
> survive if it gets included in emacs proper.
>
> > That said: Tools like fibers would be great to have in Emacs.
> >
> > The workload to finish this is considerable, though: IIRC You’ll need to
> > solve some deeper problems that prevent Guile Emacs from using
> > byte-compiled files (that’s why it currently has a very high startup
> > time).
> >
> > That said, there is a guile-emacs package in guix, and Guile 3.0.7 is
> > much faster than Guile 2.
>
> Yes, to complete *guile-emacs*... that will be a lot of work.  But it
> will be less work if guile's elisp branch is no longer this separate
> thing that isn't kept up to date with the rest of guile.  More likely
> that someone could finish and pick it up.
>
> > (please someone correct me if I’m wrong!)
> >
> > Best wishes,
> > Arne
>

Re: Guile 3 and wip-elisp/Emacs

[Ricardo Wurmus]

Hi Gregg,

"Gregg Sangster" <gregg@thesangsters.ca> writes:

> I have wip-elisp rebased all the way up to main as of a few days 
> ago (e60469c8b6936575c079faaffa40a340e1d49f3c) plus two changes 
> from Ricardo.  It's available here:
>
> https://git.sr.ht/~g20r/guile

Excellent!

> There is one test failure in "make check" on test-out-of-memory. 
> I
> haven't investigated it yet but I get the same failure on main. 
> If
> that's not an expected failure, it might be a problem with my
> environment.
>
> There is also an emacs repo here:
>
> https://git.sr.ht/~g20r/emacs
>
> which uses the 3.0-based wip-elisp.  It builds and runs but 
> segfaults easily.  I've started rebasing it on a more current 
> emacs.  Those changes aren't published yet.

I also had he same experience.  Guile Emacs would segfault very 
quickly.  That’s why I started a rebase on top of the nearest 
Emacs release, which is 25.2.  I had to abandon the rebase, 
because after about 45 commits it became too difficult for me to 
understand what exactly I had to do to resolve all these 
conflicts.  There are about 112 commits left to apply.

If you think you could benefit from the work I already did I could 
upload the full git repo as it is, mid-rebase and all.

> I haven't been able to get the guile-emacs package in guix 
> working.
> The build gets stuck at collecting/processing OKURI-NASI entries 
> after
> a few hours.  For comparison, the 3.0-based guile-emacs builds 
> in
> about 20 minutes on my machine.  If anyone has that running, it 
> would
> be nice to have a comparison of the build/run speed.

I only got the 2.x-based Guile Emacs to build.  Haven’t tried with 
3.x because I had not dared rebase beyond 2.2.

-- 
Ricardo

Re: Guile 3 and wip-elisp/Emacs

[Christine Lemmer-Webber]

This is awesome.

So, what's stopping us from merging this into guile main now before it
bitrots again? :)

"Gregg Sangster" <gregg@thesangsters.ca> writes:

> Hi Ricardo,
>
> Thanks,  your changes fixed a couple of the test errors I saw.
>  
> On Monday, September 13, 2021 06:56 EDT, Ricardo Wurmus <rekado@elephly.net> wrote: 
>  
>> 
>> Hi Gregg,
>> 
>> > I've rebased the wip-elisp branch on top of commit 
>> > 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe: 
>> > v3.0.4-64-g33232cb5c4).  It's published here:
>> >
>> > https://git.sr.ht/~g20r/guile
>> 
>> FWIW I had also rebased wip-elisp in 2020:
>> 
>>     https://git.elephly.net/?p=software/guile.git;a=shortlog;h=refs/heads/wip-elisp
>> 
>> -- 
>> Ricardo
>>

Re: Guile 3 and wip-elisp/Emacs

[Christine Lemmer-Webber]

I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
again, making some naming adjustments for myself and a couple of
adjustments having talked to Robin.

If nobody objects, I'd like to merge this into main.  Maintainers, if
you have any objections, speak now or forever hold these commits!

Christine Lemmer-Webber <cwebber@dustycloud.org> writes:

> This is awesome.
>
> So, what's stopping us from merging this into guile main now before it
> bitrots again? :)
>
> "Gregg Sangster" <gregg@thesangsters.ca> writes:
>
>> Hi Ricardo,
>>
>> Thanks,  your changes fixed a couple of the test errors I saw.
>>  
>> On Monday, September 13, 2021 06:56 EDT, Ricardo Wurmus <rekado@elephly.net> wrote: 
>>  
>>> 
>>> Hi Gregg,
>>> 
>>> > I've rebased the wip-elisp branch on top of commit 
>>> > 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe: 
>>> > v3.0.4-64-g33232cb5c4).  It's published here:
>>> >
>>> > https://git.sr.ht/~g20r/guile
>>> 
>>> FWIW I had also rebased wip-elisp in 2020:
>>> 
>>>     https://git.elephly.net/?p=software/guile.git;a=shortlog;h=refs/heads/wip-elisp
>>> 
>>> -- 
>>> Ricardo
>>>

Re: Guile 3 and wip-elisp/Emacs

[Ludovic Courtès]

Hello!

Christine Lemmer-Webber <cwebber@dustycloud.org> skribis:

> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
> again, making some naming adjustments for myself and a couple of
> adjustments having talked to Robin.
>
> If nobody objects, I'd like to merge this into main.  Maintainers, if
> you have any objections, speak now or forever hold these commits!

I haven’t looked at the branch, but I think it’s great to see it live
and it’s great if it can be merged!

Some things to pay attention to before merging to ‘main’, since it
corresponds to the current 3.0 stable branch:

  • Make sure no backward incompatibilities are introduced in
    preexisting modules;

  • Make sure the ABI of libguile-3.0.so and that of public modules
    is unchanged, or is changed compatibly;

  • Make sure there are reasonable tests and doc so it can be maintained
    (and used!) going forward;

  • Robin has a copyright assignment on file, so we should be fine
    (whether we’ll keep doing copyright assignment for Guile is still
    unclear, but we can discuss that separately).

I think we should also wait for a green light from Andy.

Thanks Gregg, Christine, Robin, and everyone involved!

Ludo’.

Re: Guile 3 and wip-elisp/Emacs

[Christine Lemmer-Webber]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Christine Lemmer-Webber <cwebber@dustycloud.org> skribis:
>
>> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
>> again, making some naming adjustments for myself and a couple of
>> adjustments having talked to Robin.
>>
>> If nobody objects, I'd like to merge this into main.  Maintainers, if
>> you have any objections, speak now or forever hold these commits!
>
> I haven’t looked at the branch, but I think it’s great to see it live
> and it’s great if it can be merged!

I just compiled the rebased version and will be playing with it little
bits over the next few days to make sure it's reasonably good.

> Some things to pay attention to before merging to ‘main’, since it
> corresponds to the current 3.0 stable branch:
>
>   • Make sure no backward incompatibilities are introduced in
>     preexisting modules;
>
>   • Make sure the ABI of libguile-3.0.so and that of public modules
>     is unchanged, or is changed compatibly;

There are, I think, two commits that could use review, but I am NOT the
right person to do this.

  4e96211eb666751b8666beb918bf3108aa1c725b intern arbitrary constants
  433fc448ddb018767906f8808203c9668c68cd83 multiple obarrays

>   • Make sure there are reasonable tests and doc so it can be maintained
>     (and used!) going forward;

Three tests across elisp-compiler.test,elisp-reader.test, elisp.test.
They seem to cover enough.

>   • Robin has a copyright assignment on file, so we should be fine
>     (whether we’ll keep doing copyright assignment for Guile is still
>     unclear, but we can discuss that separately).

Cool.

> I think we should also wait for a green light from Andy.

Yes, I think so too.

Note that I will be without consistent internet over the next few days
so will be replying intermittently until the weekend, but I want to see
this happen.  Andy, I think you're the best person to review the
relevant commits above (and maybe the "guile-private-ref" and "allow
arbitrary constants in cps" commits look relevant too).

> Thanks Gregg, Christine, Robin, and everyone involved!

Yes, also Ricardo for doing the previous rebase!

This is awesome stuff!  Let's get it in!

> Ludo’.

Re: Guile 3 and wip-elisp/Emacs

[Christine Lemmer-Webber]

Christine Lemmer-Webber <cwebber@dustycloud.org> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Christine Lemmer-Webber <cwebber@dustycloud.org> skribis:
>>
>>> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
>>> again, making some naming adjustments for myself and a couple of
>>> adjustments having talked to Robin.
>>>
>>> If nobody objects, I'd like to merge this into main.  Maintainers, if
>>> you have any objections, speak now or forever hold these commits!
>>
>> I haven’t looked at the branch, but I think it’s great to see it live
>> and it’s great if it can be merged!
>
> I just compiled the rebased version and will be playing with it little
> bits over the next few days to make sure it's reasonably good.
>
>> Some things to pay attention to before merging to ‘main’, since it
>> corresponds to the current 3.0 stable branch:
>>
>>   • Make sure no backward incompatibilities are introduced in
>>     preexisting modules;
>>
>>   • Make sure the ABI of libguile-3.0.so and that of public modules
>>     is unchanged, or is changed compatibly;
>
> There are, I think, two commits that could use review, but I am NOT the
> right person to do this.
>
>   4e96211eb666751b8666beb918bf3108aa1c725b intern arbitrary constants
>   433fc448ddb018767906f8808203c9668c68cd83 multiple obarrays
>
>>   • Make sure there are reasonable tests and doc so it can be maintained
>>     (and used!) going forward;
>
> Three tests across elisp-compiler.test,elisp-reader.test, elisp.test.
> They seem to cover enough.

Also I just verified, only one commit is failing, which is the out of
memory test, but that's failing on the main branch too.  So looks good
on that front.

Re: Guile 3 and wip-elisp/Emacs

[Gregg Sangster]

Hi Christine,

Thanks for looking at this!  I just noticed that my email address as the committer on 1ba3d7854cac0524f80d3c6113da770505c9eda9 is incorrect.  Before merging, can you please change it to gregg@thesangsters.ca?

On Tuesday, October 19, 2021 17:59 EDT, Christine Lemmer-Webber <cwebber@dustycloud.org> wrote:

> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
> again, making some naming adjustments for myself and a couple of
> adjustments having talked to Robin.
>
> If nobody objects, I'd like to merge this into main.  Maintainers, if
> you have any objections, speak now or forever hold these commits!
>
> Christine Lemmer-Webber <cwebber@dustycloud.org> writes:
>
> > This is awesome.
> >
> > So, what's stopping us from merging this into guile main now before it
> > bitrots again? :)
> >
> > "Gregg Sangster" <gregg@thesangsters.ca> writes:
> >
> >> Hi Ricardo,
> >>
> >> Thanks,  your changes fixed a couple of the test errors I saw.
> >>
> >> On Monday, September 13, 2021 06:56 EDT, Ricardo Wurmus <rekado@elephly.net> wrote:
> >>
> >>>
> >>> Hi Gregg,
> >>>
> >>> > I've rebased the wip-elisp branch on top of commit
> >>> > 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe:
> >>> > v3.0.4-64-g33232cb5c4).  It's published here:
> >>> >
> >>> > https://git.sr.ht/~g20r/guile
> >>>
> >>> FWIW I had also rebased wip-elisp in 2020:
> >>>
> >>>     https://git.elephly.net/?p=software/guile.git;a=shortlog;h=refs/heads/wip-elisp
> >>>
> >>> --
> >>> Ricardo
> >>>
>

Re: Guile 3 and wip-elisp/Emacs

[Gregg Sangster]

Hi Ricardo,

On Thursday, October 14, 2021 13:34 EDT, Ricardo Wurmus <rekado@elephly.net> wrote:
> I also had he same experience.  Guile Emacs would segfault very
> quickly.  That’s why I started a rebase on top of the nearest
> Emacs release, which is 25.2.  I had to abandon the rebase,
> because after about 45 commits it became too difficult for me to
> understand what exactly I had to do to resolve all these
> conflicts.  There are about 112 commits left to apply.
>
> If you think you could benefit from the work I already did I could
> upload the full git repo as it is, mid-rebase and all.

If you'd like to upload it, I think it would be a great reference.

I've taken the strategy of rebasing forwards along the master branch a few commits at a time.  So far I'm about a thousand commits ahead.  It's time consuming but the conflicts to resolve are smaller.  There are just more of them. :)

--
Gregg

Re: Guile 3 and wip-elisp/Emacs

[Robin Templeton]

"Gregg Sangster" <gregg@thesangsters.ca> writes:

> Hello Guile Hackers,
>
> I've rebased the wip-elisp branch on top of commit
> 449f50dd84a081aea16ef678e32bf37abe429ff6 (git describe:
> v3.0.4-64-g33232cb5c4).  It's published here:
>
> https://git.sr.ht/~g20r/guile

This is wonderful, thank you! (IIUC, Christine and Ricardo wrote
ChangeLog entries and did some rebasing, but I don't think it's been
fully rebased for some time, so this is a big step forward!)

N.B. The commit messages need minor rewording, so this will require
trivial changes before merging even if's otherwise perfect.

> There are two additional failed tests which appear to be new tests
> added since the last work on the wip-elisp branch.  Also, ./configure
> requires the --disable-jit option as the build fails otherwise.  I
> have not yet investigated any of these issues.

Christine noted some "questionable" compiler changes, where my changes
might conflict with Andy's, so this isn't surprising but it's useful to
have potential confirmation. The elisp changes were fairly simple, so it
shouldn't be too difficult to fix, if that is indeed the problem.

> I don't have much experience with Guile itself or the Guile codebase.
> The next commit has a conflict I was not able to resolve: that's why
> it's not rebased all the way up to the tip of main.  My next tasks are
> to get the current Guile Emacs running on this version of Guile
> followed by fixing the JIT build failure, rebasing this to the tip of
> main and rebasing Guile Emacs on a more recent Emacs (I believe it is
> currently v24.something).

Rebasing onto a somewhat newer Emacs should be pretty straightforword,
and certainly worthwhile on its on. However, rebasing onto emacs master
is a major task that'll probably take several person-months of fairly
tedious work...

> My question is if this is worthwhile work?  I don't have much time to
> spend on it but am happy to keep hacking away if there's still
> upstream interest.

From my perspective, it certainly is! There is plenty of interest from
the Guile side AFAICT; the Emacs maintainers have been (understandably)
skeptical about the project in general, but if we can make it correct
*and* fast I think they'll be more interested, especially if Guile 3's
JIT outperforms emacs native-comp in practice.

Thanks,
Robin

-- 
<robin> "Digital Imperialism: The Highest Stage of Computing Capitalism"
<dsk>   -- the Lenin of Emacs

Re: Guile 3 and wip-elisp/Emacs

[Robin Templeton]

"Dr. Arne Babenhauserheide" <arne_bab@web.de> writes:

> The workload to finish this is considerable, though: IIRC You’ll need to
> solve some deeper problems that prevent Guile Emacs from using
> byte-compiled files (that’s why it currently has a very high startup
> time).

To clarify, Guile-Emacs intentionally ditches the existing Emacs
bytecode compiler entirely (though it'd probably be possible to emulate
the bytecode interpreter to allow loading old '.elc' files). I haven't
followed Guile development closely enough to know whether these are
practical solutions, but my main ideas for improving startup speed were
to either add some form of AOT compilation to Guile, or to rely on
cached JIT-compiled code. IIRC Emacs now uses a portable dumper (rather
than the old system which dumped a running Emacs instance to an
executable), which seems analogous to the latter approach.

> That said, there is a guile-emacs package in guix, and Guile 3.0.7 is
> much faster than Guile 2.
>
> (please someone correct me if I’m wrong!)
>
> Best wishes,
> Arne

Regards,
Robin

-- 
<Robert> But speaking of programming, I was dreaming that I had some
coding problem and suddenly robin walked by, so I asked what the common
lisp way of solving it would be.

Re: Guile 3 and wip-elisp/Emacs

[Christopher Allan Webber]

Christine Lemmer-Webber <cwebber@dustycloud.org> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Christine Lemmer-Webber <cwebber@dustycloud.org> skribis:
>>
>>> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
>>> again, making some naming adjustments for myself and a couple of
>>> adjustments having talked to Robin.
>>>
>>> If nobody objects, I'd like to merge this into main.  Maintainers, if
>>> you have any objections, speak now or forever hold these commits!
>>
>> I haven’t looked at the branch, but I think it’s great to see it live
>> and it’s great if it can be merged!
>
> I just compiled the rebased version and will be playing with it little
> bits over the next few days to make sure it's reasonably good.

Well, *something* isn't ready to go:

scheme@(guile-user)> (+ 1 2)
$3 = 3
scheme@(guile-user)> ,L elisp
Happy hacking with Emacs Lisp!  To switch back, type `,L scheme'.
elisp@(guile-user)> (+ 1 2)
$4 = 3
elisp@(guile-user)> (cons 'foo '())
$25 = (foo)
elisp@(guile-user) [10]> 

Hoo, 10 levels deep!

>> Some things to pay attention to before merging to ‘main’, since it
>> corresponds to the current 3.0 stable branch:
>>
>>   • Make sure no backward incompatibilities are introduced in
>>     preexisting modules;
>>
>>   • Make sure the ABI of libguile-3.0.so and that of public modules
>>     is unchanged, or is changed compatibly;
>
> There are, I think, two commits that could use review, but I am NOT the
> right person to do this.
>
>   4e96211eb666751b8666beb918bf3108aa1c725b intern arbitrary constants
>   433fc448ddb018767906f8808203c9668c68cd83 multiple obarrays

Re: Guile 3 and wip-elisp/Emacs

[Ricardo Wurmus]

Hi Gregg,

> On Thursday, October 14, 2021 13:34 EDT, Ricardo Wurmus 
> <rekado@elephly.net> wrote: 
>> I also had he same experience.  Guile Emacs would segfault very 
>> quickly.  That’s why I started a rebase on top of the nearest 
>> Emacs release, which is 25.2.  I had to abandon the rebase, 
>> because after about 45 commits it became too difficult for me 
>> to 
>> understand what exactly I had to do to resolve all these 
>> conflicts.  There are about 112 commits left to apply.
>> 
>> If you think you could benefit from the work I already did I 
>> could 
>> upload the full git repo as it is, mid-rebase and all.
>
> If you'd like to upload it, I think it would be a great 
> reference.

You can download it here:

    https://bimsbstatic.mdc-berlin.de/akalin/guile-emacs-repo.tar.xz

-- 
Ricardo

Re: Guile 3 and wip-elisp/Emacs

[Robin Templeton]

"Gregg Sangster" <gregg@thesangsters.ca> writes:

> Hello,
>
> I have wip-elisp rebased all the way up to main as of a few days ago
> (e60469c8b6936575c079faaffa40a340e1d49f3c) plus two changes from
> Ricardo.  It's available here:
>
> https://git.sr.ht/~g20r/guile
>
> There is one test failure in "make check" on test-out-of-memory.  I
> haven't investigated it yet but I get the same failure on main.  If
> that's not an expected failure, it might be a problem with my
> environment.
>
> There is also an emacs repo here:
>
> https://git.sr.ht/~g20r/emacs
>
> which uses the 3.0-based wip-elisp.  It builds and runs but segfaults
> easily.  I've started rebasing it on a more current emacs.  Those
> changes aren't published yet.

It's not a good sign that it segfaults easily; Someone(TM) should figure
out the source of the crashes even if rebasing happens to fix them,
which I wouldn't necessarily count on. I'll try to do some debugging in
the next few days.

> I haven't been able to get the guile-emacs package in guix working.  The
> build gets stuck at collecting/processing OKURI-NASI entries after a few
> hours.  For comparison, the 3.0-based guile-emacs builds in about 20
> minutes on my machine.  If anyone has that running, it would be nice to
> have a comparison of the build/run speed.

I can confirm that, it seems to get stuck at:

Reading file "/tmp/guix-build-[...]/[...]/leim/SKK-DIC/SKK-JISYO.L" ...
[...]
Collecting OKURI-NASI entries ...
[...]
collected 70% ...

even after running for quite a while:

guix build -c32 guile-emacs  5.29s user 0.47s system 0% cpu 28:02:53.21 total

I'm not sure what's going on here, as I've definitely installed the
guile-emacs package before (though not recently) and it doesn't use the
standard emacs and guile packages, for obvious reasons. I can probably
use guix time-machine to figure out what's happening. That build phase
was always excruciatingly slow, but not *that* slow.

-- 
„Die Philosophen haben die Welt nur verschieden *interpretiert*, es
kömmt drauf an, sie zu *verändern*.“ -- Karl Marx

Re: Guile 3 and wip-elisp/Emacs

[Robin Templeton]

Christine Lemmer-Webber <cwebber@dustycloud.org> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Christine Lemmer-Webber <cwebber@dustycloud.org> skribis:
>>
>>> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
>>> again, making some naming adjustments for myself and a couple of
>>> adjustments having talked to Robin.
>>>
>>> If nobody objects, I'd like to merge this into main.  Maintainers, if
>>> you have any objections, speak now or forever hold these commits!
>>
>> I haven’t looked at the branch, but I think it’s great to see it live
>> and it’s great if it can be merged!
>
> I just compiled the rebased version and will be playing with it little
> bits over the next few days to make sure it's reasonably good.
>
>> Some things to pay attention to before merging to ‘main’, since it
>> corresponds to the current 3.0 stable branch:
>>
>>   • Make sure no backward incompatibilities are introduced in
>>     preexisting modules;
>>
>>   • Make sure the ABI of libguile-3.0.so and that of public modules
>>     is unchanged, or is changed compatibly;
>
> There are, I think, two commits that could use review, but I am NOT the
> right person to do this.
>
>   4e96211eb666751b8666beb918bf3108aa1c725b intern arbitrary constants
>   433fc448ddb018767906f8808203c9668c68cd83 multiple obarrays

I'll take a look at these...

> [...] (and maybe the "guile-private-ref" and "allow
> arbitrary constants in cps" commits look relevant too).

and these, but concur that Andy is the best person to review them, and I
agree that Andy should approve the merge overall in any case.

Still reviewing the wip-elisp-rebased branch in my spare time; so far I
haven't found any noteworthy problems, not that I was expecting to :)

A minor point: IMHO the "(Best-ability ChangeLog annotation ...)" lines
aren't the ideal way to credit you, in terms of commit message
formatting. I'd prefer using git 'trailers' for in-commit credit, so
that it's obvious to both humans and git that it's commit metadata. I'm
not sure there's a conventional git trailer for this...something like
'ChangeLog-by:' would be probably be clear enough. But if you prefer the
current formatting and it works with Guile's ChangeLog generator, I
don't mind leaving it as-is (except perhaps "s/Best-ability //" after
I've reviewed everything :)) WDYT?

Thanks,
Robin

-- 
"You have to act as if it were possible to radically transform the
world. And you have to do it all the time." -- Angela Davis

Re: Guile 3 and wip-elisp/Emacs

[Christine Lemmer-Webber]

Robin Templeton <robin@terpri.org> writes:

> Christine Lemmer-Webber <cwebber@dustycloud.org> writes:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Hello!
>>>
>>> Christine Lemmer-Webber <cwebber@dustycloud.org> skribis:
>>>
>>>> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
>>>> again, making some naming adjustments for myself and a couple of
>>>> adjustments having talked to Robin.
>>>>
>>>> If nobody objects, I'd like to merge this into main.  Maintainers, if
>>>> you have any objections, speak now or forever hold these commits!
>>>
>>> I haven’t looked at the branch, but I think it’s great to see it live
>>> and it’s great if it can be merged!
>>
>> I just compiled the rebased version and will be playing with it little
>> bits over the next few days to make sure it's reasonably good.
>>
>>> Some things to pay attention to before merging to ‘main’, since it
>>> corresponds to the current 3.0 stable branch:
>>>
>>>   • Make sure no backward incompatibilities are introduced in
>>>     preexisting modules;
>>>
>>>   • Make sure the ABI of libguile-3.0.so and that of public modules
>>>     is unchanged, or is changed compatibly;
>>
>> There are, I think, two commits that could use review, but I am NOT the
>> right person to do this.
>>
>>   4e96211eb666751b8666beb918bf3108aa1c725b intern arbitrary constants
>>   433fc448ddb018767906f8808203c9668c68cd83 multiple obarrays
>
> I'll take a look at these...
>
>> [...] (and maybe the "guile-private-ref" and "allow
>> arbitrary constants in cps" commits look relevant too).
>
> and these, but concur that Andy is the best person to review them, and I
> agree that Andy should approve the merge overall in any case.
>
> Still reviewing the wip-elisp-rebased branch in my spare time; so far I
> haven't found any noteworthy problems, not that I was expecting to :)
>
> A minor point: IMHO the "(Best-ability ChangeLog annotation ...)" lines
> aren't the ideal way to credit you, in terms of commit message
> formatting. I'd prefer using git 'trailers' for in-commit credit, so
> that it's obvious to both humans and git that it's commit metadata. I'm
> not sure there's a conventional git trailer for this...something like
> 'ChangeLog-by:' would be probably be clear enough. But if you prefer the
> current formatting and it works with Guile's ChangeLog generator, I
> don't mind leaving it as-is (except perhaps "s/Best-ability //" after
> I've reviewed everything :)) WDYT?

I'm fine changing it.  The original reason for the "best ability" notes
was that I was more or less making up text to make it compatible with
the changelog style based off of reading your code.  Honestly the
attribution part isn't the most important to me but the "ChangeLog-by is
an interesting idea".  Most importantly, I want to get it in! :)

> Thanks,
> Robin

Re: Guile 3 and wip-elisp/Emacs

[Christine Lemmer-Webber]

Christine Lemmer-Webber <cwebber@dustycloud.org> writes:

> Well, *something* isn't ready to go:
>
> scheme@(guile-user)> (+ 1 2)
> $3 = 3
> scheme@(guile-user)> ,L elisp
> Happy hacking with Emacs Lisp!  To switch back, type `,L scheme'.
> elisp@(guile-user)> (+ 1 2)
> $4 = 3
> elisp@(guile-user)> (cons 'foo '())
> $25 = (foo)
> elisp@(guile-user) [10]> 
>
> Hoo, 10 levels deep!

I missed earlier in the thread that Gregg said that we should use
--disable-jit.  Once I did that it was fine.  So ignore this bit!

 - Christine

PS: An aside: wrong name in previous email; I never use gnus anymore,
apparently loading it accidentally the other day loaded a stale
~/.gnus.el and lead to me deadnaming myself with an old
  (setq user-full-name ...)
!!!  Watch out for stale elisp!

Re: Guile 3 and wip-elisp/Emacs

[Christine Lemmer-Webber]

Christine Lemmer-Webber <cwebber@dustycloud.org> writes:

> Christine Lemmer-Webber <cwebber@dustycloud.org> writes:
>
>> Well, *something* isn't ready to go:
>>
>> scheme@(guile-user)> (+ 1 2)
>> $3 = 3
>> scheme@(guile-user)> ,L elisp
>> Happy hacking with Emacs Lisp!  To switch back, type `,L scheme'.
>> elisp@(guile-user)> (+ 1 2)
>> $4 = 3
>> elisp@(guile-user)> (cons 'foo '())
>> $25 = (foo)
>> elisp@(guile-user) [10]> 
>>
>> Hoo, 10 levels deep!
>
> I missed earlier in the thread that Gregg said that we should use
> --disable-jit.  Once I did that it was fine.  So ignore this bit!
>
>  - Christine

It wasn't this either!  It turns out it's that ,L elisp and geiser don't
seem to get along (yet).  The same thing happens with me in Guile's
stable release.

Also notably, --disable-jit doesn't seem to have been the culprit for me
then; I got it to compile fine without it, I think?

Re: Guile 3 and wip-elisp/Emacs

[Gregg Sangster]

Hi Christine,

On Friday, October 22, 2021 12:17 EDT, Christine Lemmer-Webber <cwebber@dustycloud.org> wrote:

> Christine Lemmer-Webber <cwebber@dustycloud.org> writes:
> > I missed earlier in the thread that Gregg said that we should use
> > --disable-jit.  Once I did that it was fine.  So ignore this bit!
> >
> >  - Christine
>
> It wasn't this either!  It turns out it's that ,L elisp and geiser don't
> seem to get along (yet).  The same thing happens with me in Guile's
> stable release.
>
> Also notably, --disable-jit doesn't seem to have been the culprit for me
> then; I got it to compile fine without it, I think?

--disable-jit isn't needed.  My initial rebasing was done in the guile-for-guile-emacs guix environment and around the guile 2 to 3 transition it became necessary.  After switching to the guile-next environment, the jit version compiles fine.

--
Gregg