Lua

Lua

[No Itisnt]

Hello, I'm going to be implementing Lua 5.1 for Guile as a Google
Summer of Code under the GNU Project this year. My project is defined
as "the Lua 5.1 programming language and standard library excluding
the string, coroutine, and debug modules" I left a little bit of
wiggle room in the standard library in case I get burnt out.

- I'm going by the 5.1 manual, billed as the "official definition of
the Lua language": <http://www.lua.org/manual/5.1/>.
- Per GSOC rules, I can't accept contributions. Feedback is always welcome.
- I'm going to be working under the "lua" branch of the official
repository, which can be viewed through the web at
<http://git.savannah.gnu.org/cgit/guile.git/log/?h=lua>.
- I'll keep my personal notes in the 'NOTES-LUA' file in the toplevel
directory, and keep it updated with known issues etc.
- One major thing omitted from this project is a wrapper for the C
API. I'd still like to see it happen, maybe later this year.
- GSOC starts on May 24th and ends on August 16th.

Feel free to use this thread for discussion or feedback. And good luck
to my fellow Guile Summer of Code hackers.

Re: Lua

[Ludovic Courtès]

Hello,

Good to hear, this all sounds nice to me.  Happy summer hacking!

Ludo’.

Re: Lua

[No Itisnt]

Couple of questions:

- Right now I've stuffed everything into one file. When it doesn't
recompile automatically (as in I run it without making changes, after
a run that autocompiled it) it can't resolve MAKE-APPLICATION from
(language tree-il) so a bunch of my tests cause errors. Any known bugs
that would cause that?

- Will Guile correctly optimize explicit returns as tail calls? I
couldn't tell from a quick look but my gut feeling is that it only
optimizes implicit returns, as opposed to those using (primitive-ref
return)?

Re: Lua

[Andy Wingo]

Heya,

On Thu 03 Jun 2010 10:32, No Itisnt <theseaisinhere@gmail.com> writes:

> - Right now I've stuffed everything into one file. When it doesn't
> recompile automatically (as in I run it without making changes, after
> a run that autocompiled it) it can't resolve MAKE-APPLICATION from
> (language tree-il) so a bunch of my tests cause errors. Any known bugs
> that would cause that?

Not a known bug, no. Can you make a test case?

http://www.slate.com/id/2081042

> - Will Guile correctly optimize explicit returns as tail calls? I
> couldn't tell from a quick look but my gut feeling is that it only
> optimizes implicit returns, as opposed to those using (primitive-ref
> return)?

No, it will not. The primitive-ref return thing is a hack; the proper
way to do this (I think?) is to use prompts and aborts, and also enhance
Guile's compiler to do source-to-source translation of Tree-IL in which
an intraprocedural abort to a prompt with a fresh tag that cannot escape
the prompt expression does some kind of CPS on the expression to leave
the return expression in tail context.

Otherwise we could add `return' to Tree-IL, but that is nasty I think;
or another hack (nasty, but perhaps expedient). Better to CPS, in the
long run anyway.

Andy
-- 
http://wingolog.org/

Re: Lua

[No Itisnt]

[-- Attachment #1: Type: text/plain, Size: 1029 bytes --]

> Not a known bug, no. Can you make a test case?

Yes, attached. It appears to happen when (rnrs control) is used. This
is with the latest master commit.

> No, it will not. The primitive-ref return thing is a hack; the proper
> way to do this (I think?) is to use prompts and aborts, and also enhance
> Guile's compiler to do source-to-source translation of Tree-IL in which
> an intraprocedural abort to a prompt with a fresh tag that cannot escape
> the prompt expression does some kind of CPS on the expression to leave
> the return expression in tail context.
>
> Otherwise we could add `return' to Tree-IL, but that is nasty I think;
> or another hack (nasty, but perhaps expedient). Better to CPS, in the
> long run anyway.

It's not urgent, but Lua does mandate tail call optimization.

On a related note, what would be the best way to benchmark a compiler
optimization? I see "benchmark-guile"; is there a way to easily run it
with an optimization enabled and disabled?  And maybe get some fancy
charts and graphs, too?

[-- Attachment #2: test.scm --]
[-- Type: application/octet-stream, Size: 151 bytes --]

(define-module (module)
  #:use-module ((rnrs control) #:version (6))
  #:use-module (language tree-il))

(make-const #f 2)

(display 'done)
(newline)

Re: Lua

[Andy Wingo]

On Thu 03 Jun 2010 22:36, No Itisnt <theseaisinhere@gmail.com> writes:

>> Not a known bug, no. Can you make a test case?
>
> Yes, attached. It appears to happen when (rnrs control) is used. This
> is with the latest master commit.

Thanks!

> It's not urgent, but Lua does mandate tail call optimization.

I think the basic thing would be the equivalent of:

  (lambda args ... (return (foo bar ...)) ...)

  =>
  (lambda args
    (let ((unique-tag (make-prompt-tag)))
      (% unique-tag 
         (...
          (abort-to-prompt unique-tag (lambda () (foo bar ...)))
          ...)
         (lambda (unused-cont thunk) (thunk)))))
      
And from there the compiler would be resposible for removing the prompt,
if possible.

But, I could be totally misled here and there's some much more simple
solution.

> On a related note, what would be the best way to benchmark a compiler
> optimization? I see "benchmark-guile"; is there a way to easily run it
> with an optimization enabled and disabled?  And maybe get some fancy
> charts and graphs, too?

Just running benchmark-guile before and after, I'm afraid... Once
they're in master, there's http://ossau.homelinux.net/~neil/.

Cheers,

Andy
-- 
http://wingolog.org/

Re: Lua

[Andy Wingo]

On Thu 03 Jun 2010 22:36, No Itisnt <theseaisinhere@gmail.com> writes:

> (define-module (test)
>   #:use-module ((rnrs control) #:version (6))
>   #:use-module (language tree-il))
>
> (make-const #f 2)
>
> (display 'done)
> (newline)

I put this as test.scm in my guile's root, and then ran:

  wingo@unquote:~/src/guile$ meta/guile
  GNU Guile 1.9.11.7-3d8b6
  Copyright (C) 1995-2010 Free Software Foundation, Inc.

  Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
  This program is free software, and you are welcome to redistribute it
  under certain conditions; type `,show c' for details.

  Enter `,help' for help.
  scheme@(guile-user)> (use-modules (test))
  ;;; note: autocompilation is enabled, set GUILE_AUTO_COMPILE=0
  ;;;       or pass the --no-autocompile argument to disable.
  ;;; compiling /home/wingo/src/guile/test.scm
  ;;; compiling /opt/guile/share/guile/2.0/rnrs//6/control.scm
  ;;; compiling /opt/guile/share/guile/2.0/rnrs//6/base.scm
  ;;; compiled /home/wingo/src/guile/cache/guile/ccache/2.0-0.Q-LE-4/opt/guile/share/guile/2.0/rnrs/6/base.scm.go
  ;;; compiled /home/wingo/src/guile/cache/guile/ccache/2.0-0.Q-LE-4/opt/guile/share/guile/2.0/rnrs/6/control.scm.go
  ;;; compiled /home/wingo/src/guile/cache/guile/ccache/2.0-0.Q-LE-4/home/wingo/src/guile/test.scm.go
  done
  scheme@(guile-user)> 
  wingo@unquote:~/src/guile$ meta/guile
  GNU Guile 1.9.11.7-3d8b6
  Copyright (C) 1995-2010 Free Software Foundation, Inc.

  Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
  This program is free software, and you are welcome to redistribute it
  under certain conditions; type `,show c' for details.

  Enter `,help' for help.
  scheme@(guile-user)> (use-modules (test))
  done
  scheme@(guile-user)> 

So, I don't see the issue here. Can you try again?

I don't know why it chose to autocompile the rnrs modules, though;
perhaps something version-related? Yuk...

Andy
-- 
http://wingolog.org/

Re: Lua

[No Itisnt]

Hey, now would be a great time for a critique of my Lua project if you
are willing, so I can incorporate any criticisms or ideas you may have
before the GSOC midterm date. I'm closing in on finishing the base
language and I hope to complete it this week if time permits.

- I am using #nil for Lua's nil right now. Lua considers nil false for
the purpose of conditionals and it's also the value of unset variables
or table slots. Maybe it should be replaced with a gensym? I didn't
follow the Emacs nil thing very closely so I'm not sure whether that
approach might be problematic.

- Lua represents numbers as the C 'double' type by default -- what's
the best way to approximate that in Guile?

- How should I implement multiple return values? Lua allows multiple
variables to be assigned in a single statement, e.g. "a,b = c()", but
it's also permissive in that an assignment need not be balanced --
there can be extra variables on the left or expressions on the right.
So I think I need to be able to inspect the return value of function
calls somewhat at runtime. I was hoping for something elegant but
perhaps I will just return vectors and inspect the return values of
functions at runtime.

Thanks.

Re: Lua

[Andy Wingo]

Hello!

On Sun 13 Jun 2010 23:03, No Itisnt <theseaisinhere@gmail.com> writes:

> Hey, now would be a great time for a critique of my Lua project if you
> are willing, so I can incorporate any criticisms or ideas you may have
> before the GSOC midterm date. I'm closing in on finishing the base
> language and I hope to complete it this week if time permits.

Great news. I haven't had time to really dive in, but hope to do so
towards the end of the week.

> - I am using #nil for Lua's nil right now. Lua considers nil false for
> the purpose of conditionals and it's also the value of unset variables
> or table slots. Maybe it should be replaced with a gensym? I didn't
> follow the Emacs nil thing very closely so I'm not sure whether that
> approach might be problematic.

It could be OK. #f would also be fine. Does this value have any other
kinds of semantics? You will want to use an immediate like #f or nil or
something, going through a symbol will be too slow.

> - Lua represents numbers as the C 'double' type by default -- what's
> the best way to approximate that in Guile?

Hmm. Here I think we should use Guile's numbers by default. That way you
get the speed of inums in most normal cases. You should probably coerce
division into inexact numbers (i.e. 3 / 4 translating to the equivalent
of (exact->inexact (/ 3 4))).

Looking farther, having immediate (unboxed) floating point numbers might
be a win. See the discussion in
http://www.nue.riec.tohoku.ac.jp/ppl2008/proceedings/1-06.pdf; I don't
think a special GC nursery is the right thing for us though. I'd rather
try to tag immediate floating point numbers as in
http://t-t-travails.blogspot.com/2007/07/tagged-unboxed-floating-point-numbers.html.

> - How should I implement multiple return values? Lua allows multiple
> variables to be assigned in a single statement, e.g. "a,b = c()", but
> it's also permissive in that an assignment need not be balanced --
> there can be extra variables on the left or expressions on the right.
> So I think I need to be able to inspect the return value of function
> calls somewhat at runtime. I was hoping for something elegant but
> perhaps I will just return vectors and inspect the return values of
> functions at runtime.

I don't know really. This was the sticking point for me when I looked at
implementing Lua. It would be nice to avoid heap allocation...

Here is one option:

  (letrec ((cont (case-lambda
                   ((a b c) ; the expected number of values
                    do-the-thing)
                   (args    ; some other number of values
                    (cont (first-arg arg) (second-arg arg) (third-arg arg))))))
    (call-with-values (lambda () right-hand-side ...)
      cont))

This should be optimizable in the future (in particular, rendering the
case-lambda body inline, as normal lambdas are).

Happy hacking,

Andy
-- 
http://wingolog.org/

Re: Lua

[No Itisnt]

>> Hey, now would be a great time for a critique of my Lua project if you
>> are willing, so I can incorporate any criticisms or ideas you may have
>> before the GSOC midterm date. I'm closing in on finishing the base
>> language and I hope to complete it this week if time permits.
>
> Great news. I haven't had time to really dive in, but hope to do so
> towards the end of the week.

Great! Please excuse the mess. The parser is awful, awful, awful and
will probably be the first thing to go. Maybe it will get replaced
with a PEG later on :) But it works for now.

>> - I am using #nil for Lua's nil right now. Lua considers nil false for
>> the purpose of conditionals and it's also the value of unset variables
>> or table slots. Maybe it should be replaced with a gensym? I didn't
>> follow the Emacs nil thing very closely so I'm not sure whether that
>> approach might be problematic.
>
> It could be OK. #f would also be fine. Does this value have any other
> kinds of semantics? You will want to use an immediate like #f or nil or
> something, going through a symbol will be too slow.

It has to be distinct from false. I'll just stick with #nil and
hopefully nothing will go wrong

>> - Lua represents numbers as the C 'double' type by default -- what's
>> the best way to approximate that in Guile?
>
> Hmm. Here I think we should use Guile's numbers by default. That way you
> get the speed of inums in most normal cases. You should probably coerce
> division into inexact numbers (i.e. 3 / 4 translating to the equivalent
> of (exact->inexact (/ 3 4))).

Sounds good

Thanks