full moon, vm status update

[Andy Wingo <wingo@pobox.com>]

'Sup,

I was going to divide this update into the good, the bad, and the ugly,
but the bad is the ugly, and the good contains within it the bad as
well.

The good:

  * I've hacked the disassembler to weave together the bits of
    information that it has into a more coherent text.

    So for example where in the past you would have a separate section
    after the program text listing the instruction <-> source
    associations, now the instruction disassembly is annotated with the
    source information directly.

    This required me to become format's daddy.
    http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=module/system/vm/disasm.scm;h=5ec1c004b42092a621792c1bc93e1c1b566a11fc;hb=vm#l171

    I've appended a bit of disassembly to this mail.

  * Fixed another bug in quasiquote compilation. I don't expect any
    more, but then again, I didn't expect any more.

  * The VM stack is now marked precisely.

  * There is a now a debugging mode, currently turned on, in which we
    try ensure that the top of the stack is non-NULL and that all
    elements past the top are set to NULL. There are a number of checks
    in various places that this is the case. The idea is to avoid lost
    references when GC runs, and the heap structure's idea of the VM
    registers is out of sync with what the VM regs actually are; or
    there is some sloppy programming somewhere. When turned off, this
    code incurs no overhead.

    This mode helped to catch a number of stack GC bugs.

  * All of Guile's test suites pass now, with (ice-9 ...) compiled,
    including boot-9.

The bad:
  (It's like a C switch statement, it all falls through to the ugly.)

The ugly:

  * Actually the bit about all of the test suites passing was a lie in
    another respect: the elisp test fails, with a C stack overflow,
    indicating too much recursion into the interpreter.

    I suspect this is because some part of the elisp code depends on
    tail calls within interpreted code, but you don't get tail call
    semantics when calling between VM code and interpreted code, just as
    you do not with calling between e.g. interpreted and C code.

    I was hoping that the stack calibration patches would land sooner or
    later in master so that I could merge and pick up this code, because
    the truth is that it's all too easy to hit the limit, for whatever
    reason.

    I have not yet tested the calibration patches.

  * I had to disable compilation of popen.scm in order to get the tests
    to pass, as it was causing really strange, nondeterministic things
    to happen. I don't know why.

    My current speculation is that when you compile --with-threads, as I
    do, that the socketpair between the signal receiving thread and the
    main thread is not closed after the fork, therefore signals in the
    child might reach the parent or vice versa, causing random code to
    run which itself might cause VM problems.

    Julian: if you have read this far and have the cycles, can you say
    something about the state of the signal socketpair after a fork?

  * Still there are a couple other modules not being compiles.

The future:

  * Compile more modules -- srfi, third-level modules -- (ice-9 foo
    bar)

  * Fix GOOPS to be VM-compatible

  * Compile *and* interpret the test suites

  * Update docs (maybe merging with guile.texi)

  and, once things are correct...

  * Profile, profile, profile

My goal is: correct execution of all existing code that:
  * does not do runtime side-effects in macros
  * does not call (the-environment)
  * does not unquote in values into macros

Well that's a long enough update! Questions, comments, and help are most
welcome.

Happy hacking,

Andy


ps. Here's the disassembly I promised:

scheme@(guile-user)> ,x with-output-to-file
Disassembly of #<program with-output-to-file (file thunk)>:

nargs = 2  nrest = 0  nlocs = 2  nexts = 0

Bytecode:

   0    (late-variable-ref 0)           ;; `open-output-file'
   2    (local-ref 0)                   ;; `file' (arg)
   4    (call 1)                                              at r4rs.scm:145:16
   6    (local-set 2)                   ;; `nport'
   8    (late-variable-ref 1)           ;; `with-output-to-port'
  10    (local-ref 2)                   ;; `nport'
  12    (local-ref 1)                   ;; `thunk' (arg)
  14    (call 2)                                              at r4rs.scm:146:14
  16    (local-set 3)                   ;; `ans'
  18    (late-variable-ref 2)           ;; `close-port'
  20    (local-ref 2)                   ;; `nport'
  22    (call 1)                                              at r4rs.scm:147:4
  24    (drop)                          
  25    (local-ref 3)                   ;; `ans'
  27    (return)                        

Properties:

  #f    (documentation . "THUNK must be a procedure of no arguments, and FILE must be a
string naming a file.  The effect is unspecified if the file already exists. 
The file is opened for output, an output port connected to it is made
the default value returned by `current-output-port', 
and the THUNK is called with no arguments.
When the THUNK returns, the port is closed and the previous
default is restored.  Returns the value yielded by THUNK.  If an
escape procedure is used to escape from the continuation of these
procedures, their behavior is implementation dependent.")

(If you read this and worry about runtime costs, don't -- the
annotations and properties are not loaded until they are asked for)
-- 
http://wingolog.org/