Guile Summer of Code project

Guile Summer of Code project

[Luis Araujo]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

Hello everyone , how are you?

Considering that the GNU project is participating on the google summer 
of code of this year, I am wondering if we could probably get a nice 
Guile/Scheme project onto it.

Here I send a brief and general proposal about my project, a Guile C 
Parser. A parser written on Scheme hooked up into Guile for parsing C files.

As I said, it is a very brief proposal, so, many implementations details 
would still need to be figured out or might change on the road, but I 
think it is detailed enough as to show what I am after with it.

Please let me know if you are interested on further information,

Thanks and Regards,

[-- Attachment #2: GCP_guile.txt --]
[-- Type: text/plain, Size: 5557 bytes --]

-- Title: Guile C Parser (GCP)

-- What is GCP?

GCP as its name stands, it is a parser for ANSI C code written on Scheme 
to be included into Guile. 

The main idea is to represent in a structured and modular way all the components
of a C source file into Guile. 

This tool would allow to easily parser C headers files and generate 
automatically Guile interfaces for their libraries among other things.

-- Design and Development

My initial idea is to give an object representation of a C source code file
from Guile, and implement from there on, functions and data types , probably
using GOOPS for many of this representation for easy handling of the C code
from Guile itself.

Briefly and general explained the project will consist initially of 
the following Scheme modules:

    c-parser.scm:
	This module contains the main Scheme procedures for parsing C code.
	It should be an easy interface from the user point of view, though
	very powerful in the other hand, because it will implement the core
	parsing operations.

	For example, we can have a main entry function for the whole parser
	called:

		(define (parser-c <string>)
			...)

	This function would take a single string value containing a proper C code
	program or file and will be the responsible of building the complete AST 
	for the code returning either a:
		a -  GCP data structure. Something like an abstract syntax tree
		wrapped around data structures to ease handle (we could 
		probably use GOOPS here).
		
		b - An error message or structure spepcifying failure of
		parsing.

	We could also include in this module handy things like for 
	example, different parsing modes for a C standard or language extension.

     c-lexer.scm:
	This module could be included in certain extent into c-parser.scm ,
	nevertheless , considering the nature of the project and given the
	representation I want to give to each components , I will probably have 
	to implement a lexer offering some kind of object representation
	for tokens, so it probably make it worthy to have it in a different
	file.

	It will contain also a main entry function like:

	   	(define (lexer-c <string>)
			...)
	
	Taking a string of proper C code, and returning lexems probably 
	wrapped in somekind of data types. It could also return a lexical error.

	This function and module could be also used standalone , but practically
	will be invoked from c-parser.scm through parse-c.
	
     c-syntax.scm:

	This module will contain the whole ANSI C code abstract syntax described
	and represented in terms of Scheme structures. For example,
	we can have here defined objects for the different C statements 
	(if, while ..) , the different structures (struct , enum ...).

	It will also be the module containing a representation of all the
	available standard ANSI types.

	In general, this module will offer the mapping between the C 
	syntax and its Scheme equivalent, among the necessary data structures
	(abstract data types/objects) to be handled by the rest of the modules.
 
     c-pretty.scm:

	This module will contain the fuctions to 'pretty print' output
	of the parser and lexer. It might also contain functions for proper
	visulation of the data structures of the syntax representation on
	Scheme.

     c-misc.scm:

	It will contain extra functions and objects. For example, we can
	add here routines to extend the parser, like parser-c that instead
	of taking a string value would take a whole file or C program directory,
	it can also containg Guile-specific extensions.

-- Timeline:

   Considering the previous brief and general description of this project,
   I will take a schedule similar to the following:

     	   - I would start designing the c-syntax.scm module, this could
	   take big part of the time , probably around  3 weeks, and it should
	   be in my opinion the first module, since it will contain the main
	   design desicions affecting the rest of the modules operations.
	   All the data structures handled by the rest of the program will
	   be described and implemented here.

	   - In 2 weeks could be greatly implemented the whole lexer.

	   - The parser will be the more complex and time consuming task.
	   I expect that its development takes at least 1 month and a half.

	   - The modules c-pretty and c-misc , depending of the needs could
	   take longer than expected, but something basic and usable 
	   accomplishing their main functions could be done in around 3 weeks.

After this time, we should be able to have a very usable parser and framework,
that we can hook into Guile and start using very straight-forward.

-- Technical details:

   I would fully implement this code using latest 1.8 Guile version
   (by the time of this writing 1.8.4).

-- Community benefits.

   Scheme is a small programming language , with a very loyal community
   who has been able to deploy Scheme through many fields.

   Guile is a particular Scheme project intended to make available such a
   language as an embeddable library into other applications, mainly 
   coded on C.
   
   Therefore, being able to parse and access C code from Scheme/Guile itself,
   and handling such a code like any other Scheme object would clearly 
   benefit the community opening many new possibilities.

   For example, creating Guile bindings through a foreign function generator
   using the parser on headers for popular C libraries and applications
   would be plain easy.

   This project can also greatly help with the internals, analyzing and
   checking proper C code using Guile.

Re: Guile Summer of Code project

[Neil Jerram]

Luis Araujo <araujo@gnu.org> writes:

> Hello everyone , how are you?

Hi Luis!

Thanks for posting this proposal.  I haven't read every detail yet,
but have one initial query, and would be interested in your thoughts.

It seems like it would be a big job to write a C parser in Guile, and
it also seems that the really interesting results would flow from what
one could do with the parser output - but that the existence of the
parser on its own would not be that exciting.

Therefore I'm wondering if there isn't already a C parser -
implemented in some other language - that you could use to work first
on the interesting possible applications of having a C parser.

This is all assuming, of course, that there is a C parser available
somewhere that would generate the right kind of output, and I haven't
actually researched that.  But it seems to me that the advantages of
doing things this way round would be that

- it would bring concrete advantages to Guile more quickly

- the process of working on applications would probably make clear
  whether a Guile-implemented C parser would provide extra benefits,
  and would probably indicate what kind of parser interface and/or
  output would be most useful and convenient.

What do you think?

    Neil

Re: Guile Summer of Code project

[Kjetil S. Matheussen]

Neil Jerram:
>
> Luis Araujo <araujo@gnu.org> writes:
>
>> Hello everyone , how are you?
>
> Hi Luis!
>
> Thanks for posting this proposal.  I haven't read every detail yet,
> but have one initial query, and would be interested in your thoughts.
>
> It seems like it would be a big job to write a C parser in Guile, and
> it also seems that the really interesting results would flow from what
> one could do with the parser output - but that the existence of the
> parser on its own would not be that exciting.
>
> Therefore I'm wondering if there isn't already a C parser -
> implemented in some other language - that you could use to work first
> on the interesting possible applications of having a C parser.
>
> This is all assuming, of course, that there is a C parser available
> somewhere that would generate the right kind of output, and I haven't
> actually researched that.  But it seems to me that the advantages of
> doing things this way round would be that
>
> - it would bring concrete advantages to Guile more quickly
>
> - the process of working on applications would probably make clear
>  whether a Guile-implemented C parser would provide extra benefits,
>  and would probably indicate what kind of parser interface and/or
>  output would be most useful and convenient.
>
> What do you think?
>

For me it sounds like you would achieve exactly the same just
by combining the output of either swig[1] or gwrap[2] and
feed it into eval-c[3-4]. Perhaps 5-6 hours of work, probably
less...

[1] http://www.swig.org
[2] http://www.nongnu.org/g-wrap/
[3] http://snd.cvs.sourceforge.net/*checkout*/snd/cvs-snd/eval-c.scm
[4] http://www.notam02.no/~kjetism/sndrt_lac2008.pdf

Re: Guile Summer of Code project

[Kjetil S. Matheussen]

On Tue, 25 Mar 2008, Kjetil S. Matheussen wrote:

>
> Neil Jerram:
>>
>>  Luis Araujo <araujo@gnu.org> writes:
>> 
>> >  Hello everyone , how are you?
>>
>>  Hi Luis!
>>
>>  Thanks for posting this proposal.  I haven't read every detail yet,
>>  but have one initial query, and would be interested in your thoughts.
>>
>>  It seems like it would be a big job to write a C parser in Guile, and
>>  it also seems that the really interesting results would flow from what
>>  one could do with the parser output - but that the existence of the
>>  parser on its own would not be that exciting.
>>
>>  Therefore I'm wondering if there isn't already a C parser -
>>  implemented in some other language - that you could use to work first
>>  on the interesting possible applications of having a C parser.
>>
>>  This is all assuming, of course, that there is a C parser available
>>  somewhere that would generate the right kind of output, and I haven't
>>  actually researched that.  But it seems to me that the advantages of
>>  doing things this way round would be that
>>
>>  - it would bring concrete advantages to Guile more quickly
>>
>>  - the process of working on applications would probably make clear
>>   whether a Guile-implemented C parser would provide extra benefits,
>>   and would probably indicate what kind of parser interface and/or
>>   output would be most useful and convenient.
>>
>>  What do you think?
>> 
>
> For me it sounds like you would achieve exactly the same just
> by combining the output of either swig[1] or gwrap[2] and
> feed it into eval-c[3-4]. Perhaps 5-6 hours of work, probably
> less...
>
> [1] http://www.swig.org
> [2] http://www.nongnu.org/g-wrap/
> [3] http://snd.cvs.sourceforge.net/*checkout*/snd/cvs-snd/eval-c.scm
> [4] http://www.notam02.no/~kjetism/sndrt_lac2008.pdf
>
>

Also see this mail from december:

http://www.mail-archive.com/guile-devel@gnu.org/msg02034.html

Re: Guile Summer of Code project

[Neil Jerram]

"Kjetil S. Matheussen" <k.s.matheussen@notam02.no> writes:

> On Tue, 25 Mar 2008, Kjetil S. Matheussen wrote:
>
>>
>> Neil Jerram:
>>>
>>>  Luis Araujo <araujo@gnu.org> writes:
>>>
>>> >  Hello everyone , how are you?
>>>
>>>  Hi Luis!
>>>
>>>  Thanks for posting this proposal.  I haven't read every detail yet,
>>>  but have one initial query, and would be interested in your thoughts.
>>>
>>>  It seems like it would be a big job to write a C parser in Guile, and
>>>  it also seems that the really interesting results would flow from what
>>>  one could do with the parser output - but that the existence of the
>>>  parser on its own would not be that exciting.
>>>
>>>  Therefore I'm wondering if there isn't already a C parser -
>>>  implemented in some other language - that you could use to work first
>>>  on the interesting possible applications of having a C parser.
>>>
>>>  This is all assuming, of course, that there is a C parser available
>>>  somewhere that would generate the right kind of output, and I haven't
>>>  actually researched that.  But it seems to me that the advantages of
>>>  doing things this way round would be that
>>>
>>>  - it would bring concrete advantages to Guile more quickly
>>>
>>>  - the process of working on applications would probably make clear
>>>   whether a Guile-implemented C parser would provide extra benefits,
>>>   and would probably indicate what kind of parser interface and/or
>>>   output would be most useful and convenient.
>>>
>>>  What do you think?
>>>
>>
>> For me it sounds like you would achieve exactly the same just
>> by combining the output of either swig[1] or gwrap[2] and
>> feed it into eval-c[3-4]. Perhaps 5-6 hours of work, probably
>> less...
>>
>> [1] http://www.swig.org
>> [2] http://www.nongnu.org/g-wrap/
>> [3] http://snd.cvs.sourceforge.net/*checkout*/snd/cvs-snd/eval-c.scm
>> [4] http://www.notam02.no/~kjetism/sndrt_lac2008.pdf
>>
>>
>
> Also see this mail from december:
>
> http://www.mail-archive.com/guile-devel@gnu.org/msg02034.html

In general terms, this - i.e. duplicating what can already be done -
is my concern too.  From a very quick googling session today, I also
came across sparse (http://www.kernel.org/pub/software/devel/sparse/),
and I wonder whether you/we could get more mileage from using (and
contributing to) that, instead of from creating a new implementation.

Luis, have you considered using and building upon these existing
technologies?  If you have, can you say more to justify your
particular proposed approach?  I think that will help to attract
support from GNU project members who will vote on the available
applications.

Regards,
        Neil

Re: Guile Summer of Code project

[Ludovic Courtès]

Hi,

Overall, I concur with what others have already said.  Just adding my
two cents...

Luis Araujo <araujo@gnu.org> writes:

> -- What is GCP?
>
> GCP as its name stands, it is a parser for ANSI C code written on Scheme 
> to be included into Guile. 
>
> The main idea is to represent in a structured and modular way all the components
> of a C source file into Guile. 
>
> This tool would allow to easily parser C headers files and generate 
> automatically Guile interfaces for their libraries among other things.

Obviously, a parser alone is not going to help much towards this goal:
interfacing Scheme code with C libraries is a job on its own (see
G-Wrap, Swig, etc.), and parsing headers to help automate things is
probably an optional part of such tools (G-Wrap does not support it at
all whereas Swig relies on it---the downside is that Swig-generated
bindings usually have to be hand-tweaked to provide a nicer Scheme API).

>     c-parser.scm:

Surely, you would want to use `lalr-scm' [0], which is available in
Guile-Lib [1] for that purpose.

>      c-lexer.scm:

Likewise, SILex [2] would allow you to generate the lexer, in a way
similar to Flex.

> -- Community benefits.
>
>    Scheme is a small programming language , with a very loyal community
>    who has been able to deploy Scheme through many fields.
>
>    Guile is a particular Scheme project intended to make available such a
>    language as an embeddable library into other applications, mainly 
>    coded on C.
>    
>    Therefore, being able to parse and access C code from Scheme/Guile itself,
>    and handling such a code like any other Scheme object would clearly 
>    benefit the community opening many new possibilities.
>
>    For example, creating Guile bindings through a foreign function generator
>    using the parser on headers for popular C libraries and applications
>    would be plain easy.

Again, I'm not convinced that such a tool would suffice to make FFI
generation "plain easy".

Besides, others have already pointed out similar projects/tools such as
`eval-c'.  While apparently somewhat different, the (old) `ctax' project
[3] might also be of interest.

Thanks,
Ludovic.

[0] http://www.iro.umontreal.ca/~boucherd/Lalr/documentation/lalr.html
[1] http://home.gna.org/guile-lib/doc/ref/text.parse-lalr/
[2] http://www.iro.umontreal.ca/~dube/
[3] http://cvs.savannah.gnu.org/viewvc/guile/guile/guile-rgx-ctax/

Re: Guile Summer of Code project

[Kjetil S. Matheussen]

Ludovic Court?s:
>>    Therefore, being able to parse and access C code from Scheme/Guile itself,
>>    and handling such a code like any other Scheme object would clearly
>>    benefit the community opening many new possibilities.
>>
>>    For example, creating Guile bindings through a foreign function generator
>>    using the parser on headers for popular C libraries and applications
>>    would be plain easy.
>
> Again, I'm not convinced that such a tool would suffice to make FFI
> generation "plain easy".
>

One project who actually makes this conversion "plain easy", is jsext:
   http://www.thescripts.com/forum/thread528081.html
jsext is a library for javascript which automatically creates wrappers
on the fly out of c header files. I know it works, because I once got
a personal demonstration from the author of it. :-) Might be worth looking
at as well.

Re: Guile Summer of Code project

[Kjetil S. Matheussen]

On Thu, 27 Mar 2008, Kjetil S. Matheussen wrote:

>
> Ludovic Court?s:
>> >     Therefore, being able to parse and access C code from Scheme/Guile 
>> >     itself,
>> >     and handling such a code like any other Scheme object would clearly
>> >     benefit the community opening many new possibilities.
>> > 
>> >     For example, creating Guile bindings through a foreign function 
>> >     generator
>> >     using the parser on headers for popular C libraries and applications
>> >     would be plain easy.
>>
>>  Again, I'm not convinced that such a tool would suffice to make FFI
>>  generation "plain easy".
>> 
>
> One project who actually makes this conversion "plain easy", is jsext:
>  http://www.thescripts.com/forum/thread528081.html
> jsext is a library for javascript which automatically creates wrappers
> on the fly out of c header files. I know it works, because I once got
> a personal demonstration from the author of it. :-) Might be worth looking
> at as well.
>
>

I should have provided the main link for the project as well:

   http://www.jsext.net/main.jsx

I think jsext has developed a lot since the posting of the
link I gave. However, the info in the link above demonstrates very well 
the intention of jsext, which seems to be exactly the same as this gnu 
summer of code project proposal. Perhaps, if jsext is written in C
(I'm not sure, maybe it is), its just porting jsext to guile, and thats
it...