Re: Compiling Elisp to a native code with a GCC plugin

[Wojciech Meyer <wojciech.meyer@googlemail.com>]

Tom Tromey <tromey@redhat.com> writes:

>>>>>> "Wojciech" == Wojciech Meyer <wojciech.meyer@googlemail.com> writes:
>
> Tom> So, lots of grunge work, just to get the point where you could start
> Tom> actually working on the GC.  I would look at automated rewriting to
> Tom> make this work -- that worked out great on the concurrent branch.
>
> Wojciech> Maybe that work should be actually done even without thinking
> Wojciech> currently about GC. AFAIR MT Emacs rewriting was in Elisp,
> Wojciech> ideally maybe using GCC would be better at some point.
>
> You would have to hack GCC a little bit, because most of the code
> locations you want to change arise from macro expansion, and GCC does
> not keep all that information.  (Though there's a WIP patch for this.)

Yes, I am aware about impreciseness of this. Currently I may not think
about this for some other unrelated reasons as well...

>
> Maybe it could be done more simply using a simple parser in elisp that
> recognizes just the needed forms.  Or maybe something based on clang.
>
> This would get you most of the way there, though there are still some
> bad things you have to fix up by hand.
>
>
> For the concurrency stuff, we did two kinds of automated rewriting.

If you could point me out with the tools you used for this job, I would
be grateful, any points to git?

>
> One was just pure elisp that searched the .c for DEFVAR_LISP and then
> made various changes.
>
> The other one modified the source (in a compile-breaking way), then ran
> the compiler, then visited each error to perform a rewrite.  This
> approach might also work for the GC problem, I am not certain.

That is clever and cheap to do, and it is worth to try (and it looks a
bit like humans do re-factoring). However clang error messages are
currently are more precise than GCC (yes, we can match-replace regex,
and it will work fine in most cases).

>
> These scripts are both in src/ on the concurrency branch.
>
> One problem with any compiler-based approach is that it only works on
> the sources it sees.  That is, the not-taken #if branches won't get
> rewritten.  This argues for trying some kind of custom parser.

Yes, this is a major problem, different configurations, different
systems, and you are never sure if it will not break something, on some
machine (assuming that the changes are required to generate correct code).

>
> Another problem we ran into is that this approach doesn't work if the
> problem code itself appears in a macro.  There were a few spots that we
> had to fix by hand -- no big deal, the automation is still worthwhile
> even if it only does 85% of the work.

Definitely it is worthwhile, however I need to know what to rewrite at
first...

>
>
> My advice is to try to do this bulk rewriting work on head, so that it
> doesn't rot.  I think that's been a problem for the concurrency work
> :-(

Any chances to get it back to life? It would be nice to push it back...
If you would like to get it back, I would volunteer with any help.

If the maintainers are happy to accept patches with incremental
improvements then it is OK, however I still think in terms `I want - but
it will be hard and even nobody started it before'. If there will be any
progress on this I will just let everybody know. (anyway starting such
project is even OK for just *learning* about internals).

Shall we setup a public repo somewhere then?

I can commit there my generational gc as a sub-module, straight away.
(however it is not the best starting point, as was said).

>
> Tom

Thanks,
Wojciech