guile-emacs for gsoc

guile-emacs for gsoc

[Brian Templeton]

Hello Guile hackers,

I'm preparing Summer of Code applications and am considering working on
Emacs-Guile integration. Here's my understanding of the current
situation, from skimming list archives, source code, etc.:

* Guile now has a partial implementation of Emacs Lisp, and it mostly
  works although a few features are missing

* Ken Raeburn has done proof-of-concept work to make Emacs use Guile's
  object representation and GC for Emacs Lisp, which is now outdated but
  shows that it's feasible (and perhaps it could be forward-ported?)

* Emacs Lisp interaction is pervasive throughout much of the Emacs C
  code, but I don't know how well-abstracted it is

* Once Guile's Emacs Lisp support is sufficiently good, Guile can be
  swapped in for the Emacs Lisp evaluator

I'm not yet familiar with the relevant bits of Guile and Emacs internals
(though I've worked on other Lisp implementations), and I only started
following Guile development recently. So my question is, where could I
best focus my efforts for two months of full-time work (plus a month of
relatively free time before GSoC officially begins)? One obvious project
would be to finish up the Emacs Lisp implementation; I just started
looking at it last night, so I don't know how complete it really is.
Another might be to go ahead with replacing the Emacs Lisp evaluator
even if Guile's Emacs Lisp implementation is not quite complete, enough
to get it booting (and then improving the Emacs Lisp language
implementation would have immediate benefits). And then there are side
projects like writing a bytecode interpreter.

If you have feedback for my application, you'll have to respond before
19:00 UTC on Friday. (My apologies for the late message! I was working
on a rather different proposal, and only found out recently that there
is significant active interest in Guile-Emacs.) Of course I'm still
interested in discussing it after then, and any suggestions are helpful.

thanks,
bpt
-- 
Inteligenta persono lernas la lingvon Esperanton rapide kaj facile.
Esperanto estas moderna, kultura lingvo por la mondo. Simpla, fleksebla,
belsona, Esperanto estas la praktika solvo de la problemo de universala
interkompreno. Lernu la interlingvon Esperanton! http://lernu.net/

Re: guile-emacs for gsoc

[Daniel Kraft]

Hi Brian,

Brian Templeton wrote:
> I'm preparing Summer of Code applications and am considering working on
> Emacs-Guile integration. Here's my understanding of the current
> situation, from skimming list archives, source code, etc.:

this sounds great!  I did the current elisp implementation in last 
year's GSoC, so I'd love to see this covered further!  Unfortunatly I 
had nearly no time at all to work on this since last autumn because of 
my studies, even though I would have liked to.

I do not really know about Ken's guile-emacs or emacs itself, but I hope 
I can give you some hints on the "pure" elisp implementation:

First of all, see README in module/language/elisp.  And of course, 
module/language/elisp itself, there's where (nearly) all of it is 
concentrated.

One important open thing I'm aware is "proper" handling of Elisp's nil 
vs. Scheme's #f and '() -- this has been source of discussion for quite 
some while now, and you should find plenty of threads in the archives. 
And another thing is that Guile's "backend" got recently support for 
things like optional arguments, while last summer I implemented this in 
the elisp "front end"; so maybe a good first start (I don't know if an 
easy start, though) could be to switch elisp's implementation to the 
available Guile infrastructure instead.

Of the missing elisp bits, some are mentioned in the README file.  Other 
things, especially what I had in mind as doing "next", would be for 
instance buffer- and frame-local bindings.

I think there may even be some threads in the mailing list archives 
where I wrote some ideas about that; I'm not sure, though.  I think my 
idea was to mark the bindings in a special way (similarly to what is 
currently done for void values) and back them by some kind of hash-table 
that is indexed with the current buffer/frame/whatever on retrieval. 
But if you want to work on that, you will no doubt find a good way to do 
so :)

If you want to work (also) on the real emacs integration, I can not tell 
you anything about that.  But if that's your main focus of interest, 
maybe I can even find some time again during the summer to work on the 
elisp bits and help you that way -- I guess that the emacs work may be 
far more than a single GSoC project...  (I will not apply for Guile as 
GSoC project though, this would be just "for fun" -- more so than GSoC 
already is.)

Happy hacking!

Daniel

guile-emacs for gsoc

[Brian Templeton]

Hello Guile hackers,

I'm preparing Summer of Code applications and am considering working on
Emacs-Guile integration. Here's my understanding of the current
situation, from skimming list archives, source code, etc.:

* Guile now has a partial implementation of Emacs Lisp, and it mostly
  works although a few features are missing

* Ken Raeburn has done proof-of-concept work to make Emacs use Guile's
  object representation and GC for Emacs Lisp, which is now outdated but
  shows that it's feasible (and perhaps it could be forward-ported?)

* Emacs Lisp interaction is pervasive throughout much of the Emacs C
  code, but I don't know how well-abstracted it is

* Once Guile's Emacs Lisp support is sufficiently good, Guile can be
  swapped in for the Emacs Lisp evaluator

I'm not yet familiar with the relevant bits of Guile and Emacs internals
(though I've worked on other Lisp implementations), and I only started
following Guile development recently. So my question is, where could I
best focus my efforts for two months of full-time work (plus a month of
relatively free time before GSoC officially begins)? One obvious project
would be to finish up the Emacs Lisp implementation; I just started
looking at it last night, so I don't know how complete it really is.
Another might be to go ahead with replacing the Emacs Lisp evaluator
even if Guile's Emacs Lisp implementation is not quite complete, enough
to get it booting (and then improving the Emacs Lisp language
implementation would have immediate benefits). And then there are side
projects like writing a bytecode interpreter.

If you have feedback for my application, you'll have to respond before
19:00 UTC on Friday. (My apologies for the late message! I was working
on a rather different proposal, and only found out recently that there
is significant active interest in Guile-Emacs.) Of course I'm still
interested in discussing it after then, and any suggestions are helpful.

thanks,
bpt
-- 
Inteligenta persono lernas la lingvon Esperanton rapide kaj facile.
Esperanto estas moderna, kultura lingvo por la mondo. Simpla, fleksebla,
belsona, Esperanto estas la praktika solvo de la problemo de universala
interkompreno. Lernu la interlingvon Esperanton! http://lernu.net/

Re: guile-emacs for gsoc

[Andy Wingo]

Hi!

Briefly, as time grows short: I'm excited that you're interested in
hacking Guile into Emacs. I can offer some mentorship, though my
bandwidth is a bit strained at this point. I would prefer if someone
else stepped up to mentor, though I will do it if no one else can.

Also, the project is a bit nebulous at this point -- Ken has come at it
from the C-and-emacs side, and Daniel, Neil and I have come at it from
the Elisp-in-Guile side, and it's really unclear what's in the middle.
That's part of the hack, is finding that out.

The job seems to me more of painstakingly correct refactorings than of
strokes of creative brilliance. You do have to be a bit daring -- the
whole project is a little crazy -- but precise in the execution.

Still interested? I would be very happy to accept a proposal from you.
But please, do us all a favor, and really immerse yourself in the state
of things as they are now, as much as is possible tomorrow -- raeburn's
guilemacs tree and Guile's elisp implementation.

> One obvious project would be to finish up the Emacs Lisp
> implementation; I just started looking at it last night, so I don't
> know how complete it really is.

I don't really know that either. Daniel probably knows best; but I know
that there is some work to do on Guile's elisp still: the compiler needs
to take advantage of advances in Guile's compiler (prompts/aborts,
"native" fluid binding, optional/keyword args) -- unfortunately
documenting some of this is still next on my to-document list.
(literally next)

> Another might be to go ahead with replacing the Emacs Lisp evaluator
> even if Guile's Emacs Lisp implementation is not quite complete,
> enough to get it booting (and then improving the Emacs Lisp language
> implementation would have immediate benefits).

This would be good; but tough, as I mentioned.

> And then there are side
> projects like writing a bytecode interpreter.

I don't think we should be interpreting existing compiled elisp
bytecode. we should compile elisp source to guile's vm.

What do you think?

Andy
-- 
http://wingolog.org/

Re: guile-emacs for gsoc

[Ludovic Courtès]

Hi,

Andy Wingo <wingo@pobox.com> writes:

> Also, the project is a bit nebulous at this point -- Ken has come at it
> from the C-and-emacs side, and Daniel, Neil and I have come at it from
> the Elisp-in-Guile side, and it's really unclear what's in the middle.
> That's part of the hack, is finding that out.

FWIW I think finding out how to put it all together should be central to
the project.  I think it would help the Guile-in-Emacs project to start
working on this as much as possible before further improving the elisp
compiler front-end.

Thanks,
Ludo’.

Re: guile-emacs for gsoc

[Ken Raeburn]

Hi, Brian.  It's great to hear that you're interested in this project!

On Apr 7, 2010, at 15:55, Brian Templeton wrote:
> * Guile now has a partial implementation of Emacs Lisp, and it mostly
>  works although a few features are missing

Some of them are pretty significant.  I think the buffer- and frame-local variables are pretty crucial -- though we probably don't want the basic Guile elisp support to have "current buffer" and "current frame" as concepts.  Something a bit more abstract, where code in Emacs could nail it down to looking for a specially designated "buffer" object or whatever, would be cleaner.

There are also certain classes of variables, only created in C code, which can only hold integer values, or only hold boolean values.  Like buffer-local variables, these are implemented in Emacs by storing special object types known to the evaluator in the "value" slot; those objects include pointers to C variables of appropriate types, which are also used directly by Emacs C code.

Scheme code that messes around with these special Lisp variables should probably see the same restrictions on new values, and have the same impact on the behavior of the C code.

There's also the chance that by the time we get through this work, Emacs will have added optional lexical scoping, and some measure of thread support.  Both of these projects are in the works now (in fact, I think the lexical scoping work may be finished and just waiting to merge for Emacs 24), and if we want to potentially be able to swap in Guile's elisp support someday, it'll need to behave the same, except perhaps in corner cases that don't matter -- but the Emacs developers would be deciding which cases matter.

You don't need all of Emacs to be able to reproduce the sort of environment where you can exercise these language features.  For example, a "smob" type (Guile's application-defined object type) called "buffer" could have a C structure associated with it that holds a few Scheme variables, and a C function called from Scheme could print out or return the values stored in the structure associated with a buffer object denoted by a global variable "the-current-buffer", which could be changed by calling "set-buffer", etc.  This buffer type doesn't have to actually hold any character data for you to be able to work on the low-level Lisp stuff.

> * Ken Raeburn has done proof-of-concept work to make Emacs use Guile's
>  object representation and GC for Emacs Lisp, which is now outdated but
>  shows that it's feasible (and perhaps it could be forward-ported?)

It's in the works.  Emacs switched version control systems recently, and I started a new job around the same time, so it's been taking me a while.  But I've got a few (not directly Guile related, but helpful for this project) patches about ready to submit upstream, and am working on rebasing the Guile changes onto a more recent version from the new upstream repository.  Probably I should target guile 1.8 with that, since that's what the old version was working with; once that's working, then update to 2.0.

Once it's brought forward to a current Emacs code base, there are lots of smaller sub-projects to tackle -- convert Lisp strings use Guile strings (and figure out what to do with text properties); convert Lisp vectors; convert hashes (and especially get weak hashes right!); etc.

Would my rebased version be ready for someone else to hack on soon?  *sigh*  I don't know... I hope so.

> * Emacs Lisp interaction is pervasive throughout much of the Emacs C
>  code, but I don't know how well-abstracted it is

Not very, in some places.  That's been a big part of my project.  Among my just-about-ready changes are macro-izing access to some fields of Lisp symbol objects, instead of using hard-coded C struct field names.

Another possible side project that's cropped up is making Emacs work with the Boehm GC library that Guile 2.0 will use.  There's an old development branch in Emacs to do that, but it never got folded in to the trunk.  I'm not sure if that's going to be needed to get Emacs to use Guile 2.0.

> * Once Guile's Emacs Lisp support is sufficiently good, Guile can be
>  swapped in for the Emacs Lisp evaluator
> 
> I'm not yet familiar with the relevant bits of Guile and Emacs internals
> (though I've worked on other Lisp implementations), and I only started
> following Guile development recently. So my question is, where could I
> best focus my efforts for two months of full-time work (plus a month of
> relatively free time before GSoC officially begins)? One obvious project
> would be to finish up the Emacs Lisp implementation; I just started
> looking at it last night, so I don't know how complete it really is.

> Another might be to go ahead with replacing the Emacs Lisp evaluator
> even if Guile's Emacs Lisp implementation is not quite complete, enough
> to get it booting (and then improving the Emacs Lisp language
> implementation would have immediate benefits).

Interesting idea... but I think we're not quite ready to meet in the middle yet.

My Guile-Emacs work has "smob" types for Lisp symbols and strings, because (1) they're somewhat different from Scheme ones, and (2) I haven't finished abstracting the C bits well enough.  So the Guile elisp code would work on Guile strings and symbols, and the other Emacs code needs smobs for now....

There's probably a lot of code comparing for equality with "nil" that would need tweaking to handle Guile's approach to the whole nil/()/#f issue.

As Ludo' indicated, just *how* we should meet in the middle is still an open question...

> And then there are side
> projects like writing a bytecode interpreter.

Well... Andy's right about compiling elisp source being the biggest win.  But from personal experience, I don't think it's good to assume that the .el file is always handy.  Although it's binary in form, overall I think it'd be a much simpler "language" to compile than Scheme or Lisp.  It doesn't have to be blindingly fast, but I think maybe we should consider supporting it...

Ken