GNU is looking for Google Summer of Code Projects

GNU is looking for Google Summer of Code Projects

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

In another list I see that GNU has been accepted for Summer of Code and is
looking for projects.

My own favorite ones regarding GNU Emacs have to do with beefing up the
Emacs Lisp runtime and bytecode system. In particular giving proper
callback information from bytecode (bytecode offset, mapping information
from bytecode to line numbers). The bytecode decompiler I started, while it
works on simple examples, I think I could get going in a much more solid
and reliable way.

And of course on the elisp package side, realgud has always been hurting
for help, multii-display windows in the debugger.

But enough about me. What is most in need of help in GNU Emacs that a
summer student might reasonably make progress on?

Discuss the ideas here (please cc me since I don't regularly follow) and
contact me offline and I'll forward the GNU contacts. (Or you probably can
look them up for yourself if so inclined. I am not a coordinator, I am just
a backup mentor).

[-- Attachment #2: Type: text/html, Size: 1104 bytes --]

Re: GNU is looking for Google Summer of Code Projects

[Stefan Monnier]

> My own favorite ones regarding GNU Emacs have to do with beefing up the
> Emacs Lisp runtime and bytecode system. In particular giving proper
> callback information from bytecode (bytecode offset, mapping information
> from bytecode to line numbers). The bytecode decompiler I started, while it
> works on simple examples, I think I could get going in a much more solid
> and reliable way.

It should be easy (much smaller than a summer project) to change the
C code so that a bytecode offset can be extracted from the backtrace.

The harder and more interesting part is how to propagate source
information (line numbers and/or lexical variable names and location) to
byte-code.  There are many parts to this, so it's definitely possible to
get some summer project(s) out of it.  E.g. one such project is to change
the reader so it outputs "fat cons cells" (i.e. cons-cells with line-num
info), then arrange for that info to survive `macroexpand-all` and
`cconv.el`.  That could already be used to give more precise line
numbers in bytecompiler warnings.

Another is to devise a way to annotate bytecode objects with a map from
byte-offsets to information about the lexical vars in-scope at that point
and their location (i.e. position in the stack or in the closure).
And then teach Emacs's debugger to use that info.

> But enough about me. What is most in need of help in GNU Emacs that a
> summer student might reasonably make progress on?

I'm sure there are lots of desires.  One I'd suggest is to introduce an
"object description" that can be used both by the GC and pdump code (and
maybe also by `equal` and `print--preprocess`?), so that when changing
the representation of objects or introducing new types we don't have to
make corresponding changes in so many different places.  XEmacs had such
a thing, so there's previous experience on which we can build.
It could also be a step towards replacing our GC with one that's
incremental such the one in XEmacs (or even better: concurrent, unlike
that of XEmacs).


        Stefan

Re: GNU is looking for Google Summer of Code Projects

[Andrea Corallo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> It should be easy (much smaller than a summer project) to change the
> C code so that a bytecode offset can be extracted from the backtrace.
>
> The harder and more interesting part is how to propagate source
> information (line numbers and/or lexical variable names and location) to
> byte-code.  There are many parts to this, so it's definitely possible to
> get some summer project(s) out of it.  E.g. one such project is to change
> the reader so it outputs "fat cons cells" (i.e. cons-cells with line-num
> info), then arrange for that info to survive `macroexpand-all` and
> `cconv.el`.  That could already be used to give more precise line
> numbers in bytecompiler warnings.
>
> Another is to devise a way to annotate bytecode objects with a map from
> byte-offsets to information about the lexical vars in-scope at that point
> and their location (i.e. position in the stack or in the closure).
> And then teach Emacs's debugger to use that info.
>
>> But enough about me. What is most in need of help in GNU Emacs that a
>> summer student might reasonably make progress on?
>
> I'm sure there are lots of desires.  One I'd suggest is to introduce an
> "object description" that can be used both by the GC and pdump code (and
> maybe also by `equal` and `print--preprocess`?), so that when changing
> the representation of objects or introducing new types we don't have to
> make corresponding changes in so many different places.  XEmacs had such
> a thing, so there's previous experience on which we can build.
> It could also be a step towards replacing our GC with one that's
> incremental such the one in XEmacs (or even better: concurrent, unlike
> that of XEmacs).
>
>
>         Stefan

It's probably definitely early to discuss but can't resist.

Do we really need some dedicated low level object?  This should be all
overhead that disappears with compilation anyway.

Also wanted to ask, am I wrong or something has been attempted in this
field?

I'm quite curious on this because the day we get source locations
crossing byte-code we could use the native compiler also as a diagnostic
tool.

  Andrea

-- 
akrl@sdf.org

Re: GNU is looking for Google Summer of Code Projects

[Andrea Corallo]

Andrea Corallo <akrl@sdf.org> writes:


> Also wanted to ask, am I wrong or something has been attempted on this
                                                     ^^^
                                                   already

-- 
akrl@sdf.org

Re: GNU is looking for Google Summer of Code Projects

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 2836 bytes --]

On Thu, Mar 19, 2020 at 1:56 PM Andrea Corallo <akrl@sdf.org> wrote:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> > It should be easy (much smaller than a summer project) to change the
> > C code so that a bytecode offset can be extracted from the backtrace.
> >
> > The harder and more interesting part is how to propagate source
> > information (line numbers and/or lexical variable names and location) to
> > byte-code.  There are many parts to this, so it's definitely possible to
> > get some summer project(s) out of it.  E.g. one such project is to change
> > the reader so it outputs "fat cons cells" (i.e. cons-cells with line-num
> > info), then arrange for that info to survive `macroexpand-all` and
> > `cconv.el`.  That could already be used to give more precise line
> > numbers in bytecompiler warnings.
> >
> > Another is to devise a way to annotate bytecode objects with a map from
> > byte-offsets to information about the lexical vars in-scope at that point
> > and their location (i.e. position in the stack or in the closure).
> > And then teach Emacs's debugger to use that info.
> >
> >> But enough about me. What is most in need of help in GNU Emacs that a
> >> summer student might reasonably make progress on?
> >
> > I'm sure there are lots of desires.  One I'd suggest is to introduce an
> > "object description" that can be used both by the GC and pdump code (and
> > maybe also by `equal` and `print--preprocess`?), so that when changing
> > the representation of objects or introducing new types we don't have to
> > make corresponding changes in so many different places.  XEmacs had such
> > a thing, so there's previous experience on which we can build.
> > It could also be a step towards replacing our GC with one that's
> > incremental such the one in XEmacs (or even better: concurrent, unlike
> > that of XEmacs).
> >
> >
> >         Stefan
>
> It's probably definitely early to discuss but can't resist.
>
> Do we really need some dedicated low level object?  This should be all
> overhead that disappears with compilation anyway.
>
> Also wanted to ask, am I wrong or something has been attempted in this
> field?
>

In the bit that I have come across looking over byteocde work and history
e,g. see http://rocky.github.io/elisp-bytecode.pdf it has become extremely
clear that there are precious few who understand how the bytecode and
runtime system work. And the people who wrote this initially, e.g. rms, and
later jwz, no longer do so.

No slight to Stefan, Jim Blandy, Paul Eggert or Tom Tromey, but if nothing
else we need a new generation of people to pick up the torch and carry on.


>
> I'm quite curious on this because the day we get source locations
> crossing byte-code we could use the native compiler also as a diagnostic
> tool.
>
>   Andrea
>
> --
> akrl@sdf.org
>

[-- Attachment #2: Type: text/html, Size: 3840 bytes --]

Re: GNU is looking for Google Summer of Code Projects

[Stefan Monnier]

> Do we really need some dedicated low level object?

I don't know what you mean, sorry.

> This should be all overhead that disappears with compilation anyway.

I get the impression that you were referring to the part where I talked
about the "object description" for the runtime system.  Compilation is
of no help here.  It's already all happening in C code.

Maybe rewriting in a language with a bit more introspection might make
an "object description" more-or-less readily available (maybe the
Remacs work might qualify), but we'd still need to connect that with
a GC and with pdump etc...


        Stefan

Re: GNU is looking for Google Summer of Code Projects

[Andrea Corallo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Do we really need some dedicated low level object?
>
> I don't know what you mean, sorry.
>
>> This should be all overhead that disappears with compilation anyway.
>
> I get the impression that you were referring to the part where I talked
> about the "object description" for the runtime system.  Compilation is
> of no help here.  It's already all happening in C code.
>
> Maybe rewriting in a language with a bit more introspection might make
> an "object description" more-or-less readily available (maybe the
> Remacs work might qualify), but we'd still need to connect that with
> a GC and with pdump etc...

Ops I now understand, we are talking about 4 different problems:

1 source location going through the compilation pipeline
2 debug information into bytecode to debug
3 autogenerate GC and pdumper code from obj description
4 GC

Clear to me thanks.

  Andrea

--
akrl@sdf.org

Re: GNU is looking for Google Summer of Code Projects

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 1285 bytes --]

On Thu, Mar 19, 2020 at 5:45 PM Andrea Corallo <akrl@sdf.org> wrote:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> >> Do we really need some dedicated low level object?
> >
> > I don't know what you mean, sorry.
> >
> >> This should be all overhead that disappears with compilation anyway.
> >
> > I get the impression that you were referring to the part where I talked
> > about the "object description" for the runtime system.  Compilation is
> > of no help here.  It's already all happening in C code.
> >
> > Maybe rewriting in a language with a bit more introspection might make
> > an "object description" more-or-less readily available (maybe the
> > Remacs work might qualify), but we'd still need to connect that with
> > a GC and with pdump etc...
>
> Ops I now understand, we are talking about 4 different problems:
>
> 1 source location going through the compilation pipeline
> 2 debug information into bytecode to debug
>

The above two I think a summer student could do.
Clarification of item 2. There is *reporting* location information
especially in traceback information on an error, which I suppose could be
considered "to debug".

3 autogenerate GC and pdumper code from obj description
> 4 GC
>
> Clear to me thanks.
>
>   Andrea
>
> --
> akrl@sdf.org
>

[-- Attachment #2: Type: text/html, Size: 2220 bytes --]

Re: GNU is looking for Google Summer of Code Projects

[Zhu Zihao]

> It could also be a step towards replacing our GC with one that's
> incremental such the one in XEmacs (or even better: concurrent, unlike
> that of XEmacs).

Ravenbrook's MPS may suit this case (pure C implementation, efficient GC
algorithm), but I' m not sure whether its license is compatible.

Ravenbrook's MPS: https://github.com/Ravenbrook/mps

Re: GNU is looking for Google Summer of Code Projects

[Stefan Monnier]

>> It could also be a step towards replacing our GC with one that's
>> incremental such the one in XEmacs (or even better: concurrent, unlike
>> that of XEmacs).
> Ravenbrook's MPS may suit this case (pure C implementation, efficient GC
> algorithm),

Indeed, it seems like it might be worth investigating.
It'd be great to take advantage of someone else's efforts.

> but I' m not sure whether its license is compatible.

Not sure either, but at least nothing struck me as an obvious
problem ;-)


        Stefan