tracepoints/breakpoints and native compilation

tracepoints/breakpoints and native compilation

[Andy Wingo]

Hi :)

A design question for everyone.  I am wondering how to support
breakpoints, tracepoints, and the like in a Guile with native-code
compilation.  If you are not familiar with what Guile does currently,
see:

  https://www.gnu.org/software/guile/manual/html_node/Traps.html

Basically Guile supports calling out to user-defined procedures when:

  (1) pushing a new continuation (stack frame)
  (2) tail-calling a procedure
  (3) popping a continuation (e.g. on return)
  (4) non-local return (after abort, or after calling a call/cc continuation)
  (5) advancing the instruction pointer

This last one is obviously expensive.  To mitigate this cost, Guile
builds the VM twice: once with hooks and once without.  When run with
--debug, you get the VM with hooks; otherwise, no hooks for you.

Note that without hooks, you still get interrupts, so you can do
statistical profiling, or interrupt a loop.  But unlike hooks,
interrupts are placed in the code.  Practically speaking, without hooks,
what you lose is the ability to set a breakpoint.  You also lose the
ability to trace a procedure, but because tracing can recursively invoke
the right VM, that's not a problem in practice.

Generally you only care about hook performance when they are disabled,
so what you need to do is minimize overhead in that case.  In a bytecode
interpreter (as Guile has now), the hooks do add some overhead, but they
are very predictable branches, so it's not a big deal.  Debug mode is on
by default in the Guile REPL and I think it adds some 10 or 20%
overhead.

However in native-code compilation, hooks are more of a problem.  They
increase the executable size, as each hook invocation has to be present
in the text somewhere.  They pollute the branch predictor, as there are
many more branches.  They increase instruction cache size, even in the
best case where the slow branches are all out of line.  Boo.

So... I have a proposal.  Right now in the short term I am going to make
a JIT for Guile 3.0 (master branch) bytecode.  I am still working on
massaging the bytecode into a state where it is very easily JITtable,
but that will be soon I hope (weeks) and then, outside circumstances
permitting, we can have a JIT within the next 3 months.  For JIT
compilation, the "hook" problem resolves itself very easily: we can JIT
in two modes, one that adds hooks and one that doesn't.  We can alter
the hook API to allow the VM to know when to re-JIT code; adding a
breakpoint doesn't have to re-JIT everything.  Or of course we can keep
the current API and just re-JIT everything.

For the bytecode interpreter, we can keep the two VMs.  Also easy.

However if we ship native code in the .go files -- what do we do?  Three
options, I see -- one, ship "regular" code (no hooks) -- fast, but no
breakpoints from Guile.  Two, ship "debug" code (with hooks).  Or three,
ship the bytecode also, and a JIT compiler, so that we can re-JIT if
needed.

The first possibility would mean that some Guile-compiled code is not
really debuggable in a nice way.  It's not ideal.  The second is OK, but
it would be slow.  However the third option seems to offer a good choice
for general-purpose installs.  If we we ship the bytecode in .go files
by default and the built Guile supports JIT compilation, then maybe we
can get all the advantages -- peak performance with native code without
embedded hook calls, but still the ability to insert hooks if needed.

So I am looking at going with the third option.  It also opens the door
to potential experiments with trace compilation and optimization.  I am
currently not looking at adaptive optimization otherwise, but anyway
that's a bit far off; we need native compilation first to get good
startup time.

Thoughts welcome!

Andy

Re: tracepoints/breakpoints and native compilation

[Mark H Weaver]

Hi Andy,

Andy Wingo <wingo@pobox.com> writes:

> A design question for everyone.  I am wondering how to support
> breakpoints, tracepoints, and the like in a Guile with native-code
> compilation.  If you are not familiar with what Guile does currently,
> see:
>
>   https://www.gnu.org/software/guile/manual/html_node/Traps.html
>
> Basically Guile supports calling out to user-defined procedures when:
>
>   (1) pushing a new continuation (stack frame)
>   (2) tail-calling a procedure
>   (3) popping a continuation (e.g. on return)
>   (4) non-local return (after abort, or after calling a call/cc continuation)
>   (5) advancing the instruction pointer

I think we should consider deprecating these legacy debug hooks, and
instead to support debugging features similar to GDB, using the same
implementation methods that GDB uses.

Alternatively, I wonder if it would be feasible to enhance GDB itself to
support debugging native Guile code.

> However if we ship native code in the .go files -- what do we do?  Three
> options, I see -- one, ship "regular" code (no hooks) -- fast, but no
> breakpoints from Guile.  Two, ship "debug" code (with hooks).  Or three,
> ship the bytecode also, and a JIT compiler, so that we can re-JIT if
> needed.

If we keep the hooks, then I agree that option three is best.

     Thanks!
       Mark

Re: tracepoints/breakpoints and native compilation

[Andy Wingo]

On Tue 15 May 2018 05:08, Mark H Weaver <mhw@netris.org> writes:

> I think we should consider deprecating these legacy debug hooks, and
> instead to support debugging features similar to GDB, using the same
> implementation methods that GDB uses.
>
> Alternatively, I wonder if it would be feasible to enhance GDB itself to
> support debugging native Guile code.

Tx for feedback.  FWIW GDB will patch the executable code in-place.  It
does so while the program being debugged is stopped; it uses ptrace for
this AFAIU.  I guess in Guile we'd set asyncs on all active threads to
synchronize.  Actually modifying the executable code is pretty gnarly
and target-specific though.

FWIW JS and Java VMs will often "deoptimize" functions that have
breakpoints or tracepoints.  Some can avoid deoptimization in some
cases, but the mechanism is the same: recompile functions.

The hooks that Guile has aren't so legacy, and aren't immutable API --
they showed up in 2.0, changed a bit in 2.2, and if we keep them they
will change a bit more in 3.0.  Their users are already deep in the
engine and can adapt.

Regarding GDB, it would certainly be feasible to enhance it to
understand Guile's native code.  Probably it's all possible to do from
Scheme.  But for tracepoints and and for when you don't have an
appropriate GDB you probably do want to be able to also have in-process
debugging capabilities.

Cheers,

Andy