frames, stacks, traps, continuations, backtraces, oh my oh my.

frames, stacks, traps, continuations, backtraces, oh my oh my.

[Andy Wingo]

Hey folks,

Over the weekend I started to work on tidying up some loose ends in the
vm branch. One of the big loose ends is the debugging story: getting
good backtraces between interpreted and compiled code, etc. My mind is
ajumble with code right now, but I figured I'd dump some observations,
to see if they jog any neurons in anyone's head.

 * make-stack with an outer cut that is inside vm-compiled code will not
   find the outer cut, thus it returns no frames.

 * It is insane to have VM compilation affect the general debugging
   interface -- the primitives that the we provide already (make-stack
   and start-stack, operating either on the current or a captured
   continuation) are a bit goofy, but they have well-defined and
   sufficient semantics.

 * Calling scm_backtrace() should show backtraces regardless of whether
   the code is running interpreted or compiled.

 * It is truly irritating to have to deal with stacks.[ch] and
   debug.[ch].

 * It is a bit distressing to have to shoehorn my beautiful vm-frame
   stuff into the old stack and debugging infrastructure. Perhaps things
   can be made nicer on the C side.

 * "Info frames" are what are contained in a #<stack> object; they are
   copies of information from a real continuation. Backtraces rely on
   this information only, not the live information.

 * "Debug frames" are two things: (1) a chain of data structures on the
   C stack, describing "frames" in a continuation; (2) Scheme objects
   that point to those C structures. The Scheme objects only make their
   way into scheme as the result of a "trap", a low-level hook run by
   the evaluator. The latter are sometimes called debug objects.

 * Debug frames aren't there if you aren't using the debugging
   evaluator.

 * Debug objects (on the Scheme side) need to be cheap to create,
   otherwise running code with traps enabled would be quite slow. For
   that reason we can't operate on full continuations, given how
   continuations are implemented.

 * Perhaps apropos the former, did I mention that debug.h sucks?

 * As yet unpushed, the VM has an abstraction similar to debug objects,
   but more pleasant to work with. Since debug objects aren't used
   except as arguments to make-stack, maybe I can avoid them entirely.

Actually, the last point is convincing me. Also, the tea is convincing
me to go to sleep. Perhaps finally the code can come together later this
week.

Thanks for listening, it's been helpful to me ;)

Andy
-- 
http://wingolog.org/

Re: frames, stacks, traps, continuations, backtraces, oh my oh my.

[Neil Jerram]

2008/12/22 Andy Wingo <wingo@pobox.com>:
> Hey folks,

Hi Andy!

I see that the following answers are now a bit late, but for
completeness anyway....

> Over the weekend I started to work on tidying up some loose ends in the
> vm branch. One of the big loose ends is the debugging story: getting
> good backtraces between interpreted and compiled code, etc. My mind is
> ajumble with code right now, but I figured I'd dump some observations,
> to see if they jog any neurons in anyone's head.
>
>  * make-stack with an outer cut that is inside vm-compiled code will not
>   find the outer cut, thus it returns no frames.

Not sure what you're saying here.  Was this just a bug statement at
the time you wrote this email?  Is it still the case (following your
"vm status update" email)?

>  * It is insane to have VM compilation affect the general debugging
>   interface -- the primitives that the we provide already (make-stack
>   and start-stack, operating either on the current or a captured
>   continuation) are a bit goofy, but they have well-defined and
>   sufficient semantics.
>
>  * Calling scm_backtrace() should show backtraces regardless of whether
>   the code is running interpreted or compiled.

Agree x2

>  * It is truly irritating to have to deal with stacks.[ch] and
>   debug.[ch].

I roughly remember that too.  What in particular though?

>  * It is a bit distressing to have to shoehorn my beautiful vm-frame
>   stuff into the old stack and debugging infrastructure. Perhaps things
>   can be made nicer on the C side.

Don't understand here, as I'd say the "old" infrastructure is OK.
Have things now worked out to your satisfaction?

>  * "Info frames" are what are contained in a #<stack> object; they are
>   copies of information from a real continuation. Backtraces rely on
>   this information only, not the live information.

Yes.  And I would assume that it's fairly clean to transform (or
interleave) VM frames into these.

>  * "Debug frames" are two things: (1) a chain of data structures on the
>   C stack, describing "frames" in a continuation; (2) Scheme objects
>   that point to those C structures. The Scheme objects only make their
>   way into scheme as the result of a "trap", a low-level hook run by
>   the evaluator. The latter are sometimes called debug objects.

Yes.  You may also see the phrase "cheap traps", which means using a
relatively lightweight debug object, and passing that out to Scheme,
instead of making a continuation and passing that out.  I removed the
non-cheap continuation option a few years back, on the grounds that
the trap code can use call/cc itself if it wants a full continuation.

(Also note that debug frames rely on a real on-the-stack alloca(), and
won't work with a heap-simulated one.)

>  * Debug frames aren't there if you aren't using the debugging
>   evaluator.

Yes, this is one of the differences between DEVAL and CEVAL.

>  * Debug objects (on the Scheme side) need to be cheap to create,
>   otherwise running code with traps enabled would be quite slow. For
>   that reason we can't operate on full continuations, given how
>   continuations are implemented.

Agreed.  See above.

>  * Perhaps apropos the former, did I mention that debug.h sucks?
>
>  * As yet unpushed, the VM has an abstraction similar to debug objects,
>   but more pleasant to work with. Since debug objects aren't used
>   except as arguments to make-stack, maybe I can avoid them entirely.

Does any part of the API expose what is inside a debug object?  As far
as I'm aware, they are really only a closure parameter from the trap
call to make-stack.  So if you have something "more pleasant", maybe
it will be fine just to redefine "debug object" to be that thing?

> Actually, the last point is convincing me. Also, the tea is convincing
> me to go to sleep. Perhaps finally the code can come together later this
> week.
>
> Thanks for listening, it's been helpful to me ;)

Better late than never, I hope!

     Neil