coverage/profiling

coverage/profiling

[Han-Wen Nienhuys]

Hi,

I'd like to run a coverage check on the lilypond source. 
How do I do this in GUILE?

  
-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

frames / stacks / source? was Re: coverage/profiling

[Han-Wen Nienhuys]

Han-Wen Nienhuys escreveu:
> I'd like to run a coverage check on the lilypond source. 
> How do I do this in GUILE?

Since noone responded, I decided to take a look myself.
My overall idea was to do the following:

 - at the top of deval(), find out current source file and line
 
 - invoke some kind of callback that increments a counter for the
source location

 - produce pretty pretty coverage graphs with the contents of the hash

The last two steps are trivial, but I'm getting lost with the first
step.

I tried to follow what happens when an error backtrace is generated.
My try is below. Unfortunately, I can't find much documentation on how
frames/stacks/etc. interact. Can anyone gently nudge me in the right
direction?


diff -p -u -u -r1.408 eval.c
--- eval.c      9 Oct 2006 23:10:31 -0000       1.408
+++ eval.c      8 Jan 2007 23:39:39 -0000
@@ -3230,6 +3230,13 @@ eval_letrec_inits (SCM env, SCM init_for
  * If, however, x represents some form that requires to evaluate a sequence of
  * expressions like (begin exp1 exp2 ...), then recursive calls to CEVAL are
  * performed for all but the last expression of that sequence.  */
+#include "stacks.h"
+#include <stdio.h>
+
+void
+read_frame (scm_t_debug_frame const *dframe, scm_t_ptrdiff offset,
+           scm_t_info_frame *iframe);
+int scm_do_profiling;
 
 static SCM
 CEVAL (SCM x, SCM env)
@@ -3263,6 +3270,34 @@ CEVAL (SCM x, SCM env)
     }
 #endif
 
+
+#ifdef DEVAL
+  if (scm_do_profiling)
+    {
+      struct scm_t_info_frame info_frame;
+      SCM source;
+      SCM file;
+      SCM line;
+    
+      read_frame (&debug, 0, &info_frame);
+
+      source = info_frame.source;
+      file = SCM_MEMOIZEDP (source) ? scm_source_property (source, scm_sym_filename) : SCM_BOOL_F;
+      line = (SCM_MEMOIZEDP (source)) ? scm_source_property (source, scm_sym_line) : SCM_BOOL_F;
+
+      if (scm_is_true (line)
+         && scm_is_true (file))
+       {
+         printf ("%s %d - size %ld\n", scm_i_string_chars (file), scm_to_int (line), scm_debug_eframe
_size);
+       }
+      else
+       {
+         scm_display (source, scm_current_output_port ());
+         scm_puts ("\n", scm_current_output_port ());
+       }
+    }
+#endif


-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Ludovic Courtès]

Hi,

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:

> Han-Wen Nienhuys escreveu:
>> I'd like to run a coverage check on the lilypond source. 
>> How do I do this in GUILE?
>
> Since noone responded, I decided to take a look myself.

[...]

> +int scm_do_profiling;

Are you interested in just profiling or coverage?  C-level or
Scheme-level?  (I assume the latter.)

For Scheme-level profiling, you may want to use the `statprof' module
currently available in guile-lib.  It's very easy to set up and provides
results similar to `gprof' (it's a statistical profiler, too).

For Scheme-level coverage tests (à la `gcov'), I don't know of any tool
that would help.

Hope this helps!
Ludovic.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Han-Wen Nienhuys]

Ludovic Courtès escreveu:
> [...]
> 
>> +int scm_do_profiling;
> 
> Are you interested in just profiling or coverage?  C-level or
> Scheme-level?  (I assume the latter.)

they are related, but mainly coverage a la gcov for now.


-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Andy Wingo]

Hi,

On Tue, 2007-01-09 at 10:10 +0100, Ludovic Courtès wrote:
> For Scheme-level profiling, you may want to use the `statprof' module
> currently available in guile-lib.  It's very easy to set up and provides
> results similar to `gprof' (it's a statistical profiler, too).
> 
> For Scheme-level coverage tests (à la `gcov'), I don't know of any tool
> that would help.

Statprof[0] can also do instrumenting profiling, so that you can see
when you enter and leave frames. Won't help getting you the different
branches of an if, though.

[0] http://home.gna.org/guile-lib/doc/ref/statprof/

Cheers,

Andy.
-- 
http://wingolog.org/



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Kevin Ryde]

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:
>
>  - at the top of deval(), find out current source file and line

I suppose even cuter would be tracking each form being evaluated, not
just by line.  Does the debugger interface have some single step that
could help?  (I've never gotten into that.)

I was thinking about some profiling the other day, to see if there's
some bottlenecks in my program.  Not the same as test coverage, but
close.  (I had a distinct lack of success with both a gprof and a
"instrument-functions" build of the C code, to check the C level.)


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Neil Jerram]

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:

> Since noone responded, I decided to take a look myself.
> My overall idea was to do the following:
>
>  - at the top of deval(), find out current source file and line

OK, but with the code where you have it at the moment, you'll miss
tail-recursive calls.

>  - invoke some kind of callback that increments a counter for the
> source location
>
>  - produce pretty pretty coverage graphs with the contents of the hash
>
> The last two steps are trivial, but I'm getting lost with the first
> step.
>
> I tried to follow what happens when an error backtrace is generated.
> My try is below. Unfortunately, I can't find much documentation on how
> frames/stacks/etc. interact. Can anyone gently nudge me in the right
> direction?

Do you think you need to do this in C?  (You might do, for reasonable
performance - I genuinely don't know yet.)  The evaluator already has
hooks (see "Evaluator trap options" in the manual) that allow you to
call out to arbitrary Scheme code at the entry and exit of every
frame.

You could use these directly, or you could use/modify/tweak one of the
following three modules that provides a higher level abstraction on
top of these.

1. The (ice-9 debug) trace code.  This traces procedure calls, though,
   not source coverage.

2. statprof, as others have already mentioned.  This is statistical,
   though.

3. My guile-debugging stuff (which is now in Guile CVS, if that's what
   you're using).

guile-debugging should be able to accumulate coverage of all source
expressions - say in a particular file - but it might incur a big
performance cost in so doing.  I'll try this out and let you know what
I find.

> +#ifdef DEVAL
> +  if (scm_do_profiling)
> +    {
> +      struct scm_t_info_frame info_frame;
> +      SCM source;
> +      SCM file;
> +      SCM line;
> +    
> +      read_frame (&debug, 0, &info_frame);
> +
> +      source = info_frame.source;
> +      file = SCM_MEMOIZEDP (source) ? scm_source_property (source, scm_sym_filename) : SCM_BOOL_F;
> +      line = (SCM_MEMOIZEDP (source)) ? scm_source_property (source, scm_sym_line) : SCM_BOOL_F;

Why SCM_MEMOIZEDP?  I believe non-memoized expressions have source
properties too.

> +
> +      if (scm_is_true (line)
> +         && scm_is_true (file))
> +       {
> +         printf ("%s %d - size %ld\n", scm_i_string_chars (file), scm_to_int (line), scm_debug_eframe
> _size);

That looks OK, although I'm not sure what the point of printing
scm_debug_eframe_size is.

> +       }
> +      else
> +       {
> +         scm_display (source, scm_current_output_port ());
> +         scm_puts ("\n", scm_current_output_port ());
> +       }
> +    }
> +#endif

So what do you find happens with this code?

Regards,
     Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Han-Wen Nienhuys]

Neil Jerram escreveu:
>>  - invoke some kind of callback that increments a counter for the
>> source location
>>
>>  - produce pretty pretty coverage graphs with the contents of the hash
>>
>> The last two steps are trivial, but I'm getting lost with the first
>> step.
>>
>> I tried to follow what happens when an error backtrace is generated.
>> My try is below. Unfortunately, I can't find much documentation on how
>> frames/stacks/etc. interact. Can anyone gently nudge me in the right
>> direction?
> 
> Do you think you need to do this in C?  (You might do, for reasonable
> performance - I genuinely don't know yet.)  The evaluator already has

I don't know, it was the first idea that came to mind. It doesn't have to be
super-fast, 10x slower is acceptable, 100x not.

> hooks (see "Evaluator trap options" in the manual) that allow you to
> call out to arbitrary Scheme code at the entry and exit of every
> frame.
> 

I'll have a look.

>> +      source = info_frame.source;
>> +      file = SCM_MEMOIZEDP (source) ? scm_source_property (source, scm_sym_filename) : SCM_BOOL_F;
>> +      line = (SCM_MEMOIZEDP (source)) ? scm_source_property (source, scm_sym_line) : SCM_BOOL_F;
> 
> Why SCM_MEMOIZEDP?  I believe non-memoized expressions have source
> properties too.

I actually don't have a clue what I'm doing ; this was a bit of cut &
paste from backtrace.c. 

> So what do you find happens with this code?

I don't get line numbers, just the memoized expressions from
the 2nd (display .. ) call.

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Han-Wen Nienhuys]

Neil Jerram escreveu:
> Do you think you need to do this in C?  (You might do, for reasonable
> performance - I genuinely don't know yet.)  The evaluator already has
> hooks (see "Evaluator trap options" in the manual) that allow you to
> call out to arbitrary Scheme code at the entry and exit of every
> frame.

[hanwen@haring info]$ guile
guile> (trap-enable 'enter-frame-handler)
throw from within critical section.
Abortado
[hanwen@haring info]$ 

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Neil Jerram]

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:

> Neil Jerram escreveu:
>> Do you think you need to do this in C?  (You might do, for reasonable
>> performance - I genuinely don't know yet.)  The evaluator already has
>> hooks (see "Evaluator trap options" in the manual) that allow you to
>> call out to arbitrary Scheme code at the entry and exit of every
>> frame.
>
> [hanwen@haring info]$ guile
> guile> (trap-enable 'enter-frame-handler)
> throw from within critical section.
> Abortado
> [hanwen@haring info]$ 

Takes effect quickly, doesn't it? :-)

What you need is something more like this:

(trap-set! enter-frame-handler
  (lambda (key continuation . args)
    ...))
(trap-enable 'enter-frame)

And it's important to do them in this order.

Regards,
     Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Han-Wen Nienhuys]

Neil Jerram escreveu:
>>> Do you think you need to do this in C?  (You might do, for reasonable
>>> performance - I genuinely don't know yet.)  The evaluator already has
>>> hooks (see "Evaluator trap options" in the manual) that allow you to
>>> call out to arbitrary Scheme code at the entry and exit of every
>>> frame.
>> [hanwen@haring info]$ guile
>> guile> (trap-enable 'enter-frame-handler)
>> throw from within critical section.
>> Abortado
>> [hanwen@haring info]$ 
> 
> Takes effect quickly, doesn't it? :-)
> 
> What you need is something more like this:
> 
> (trap-set! enter-frame-handler
>   (lambda (key continuation . args)
>     ...))
> (trap-enable 'enter-frame)
> 
> And it's important to do them in this order.

Hi,

perhaps the documentation could be extended to include a small example,
such as

"The following example demonstrates the use of traps. The program
displays each source line as it is executed, by installing a handler
that examines the origin of each continuation.  The handlers are
installed and enabled with @code{trap-set!} and @code{trap-enable},
but they only become active if traps are switched on with
@code{(trap-enable 'traps)}.



(define (test-add x y)
  (+
   x
   (/
    x
    y)))



(define (record-coverage key continuation . args)
  (let*
      ((stack (make-stack continuation))
       (frame (stack-ref stack 0))
       (source (frame-source frame))
       (line (and source (source-property source 'line))))
    (if (and line (< line 10))
	(format #t "line ~a ~a ~a \n"  (1+ line) key args))))

(trap-set! apply-frame-handler record-coverage)
(trap-set! enter-frame-handler record-coverage)
(trap-set! exit-frame-handler record-coverage)

(trap-enable 'apply-frame 'exit-frame 'enter-frame)

(trap-enable 'traps)

(define z (test-add 1 2))

(trap-disable 'traps)

when this is put in a file and executed with --debug, the following
is shown

line 2 enter-frame (#t (+ x (/ x y))) 
line 4 enter-frame (#f (/ x y)) 
line 4 apply-frame (#f) 
line 4 exit-frame (1/2) 
line 2 apply-frame (#t) 
line 2 exit-frame (3/2) 

"


(I'm not sure what the #f / #t mean, though)      



-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Ludovic Courtès]

Hi,

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:

> [hanwen@haring info]$ guile
> guile> (trap-enable 'enter-frame-handler)
> throw from within critical section.
> Abortado

Same here with HEAD and 1.8.0.  Here's what happens:

  #0  0x0fc13f6c in raise () from /lib/tls/libc.so.6
  #1  0x0fc15a0c in abort () from /lib/tls/libc.so.6
  #2  0x0ffa4a1c in scm_ithrow (key=0x10015410, args=0x300a2558, noreturn=<value optimized out>) at throw.c:699
  #3  0x0ff3718c in scm_error_scm (key=0x10015410, subr=0x300781f0, message=0x300781d0, args=0x300a2578, data=0x4) at error.c:92
  #4  0x0ff37220 in scm_error (key=0x10015410, subr=<value optimized out>, message=0xffc5594 "Unknown option name: ~S", args=0x300a2578, 
      rest=0x4) at error.c:58
  #5  0x0ff37268 in scm_misc_error (subr=0xfa <Address 0xfa out of bounds>, message=0x3001bb60 "", args=0x0) at error.c:268
  #6  0x0ff721a8 in scm_options (args=0x300a25e8, options=0xffe1b40, n=7, s=0xffc117c "evaluator-traps-interface") at options.c:202
  #7  0x0ff3a1e4 in scm_evaluator_traps (setting=0x300a25c8) at eval.c:3134
  #8  0x0ff3ec50 in deval (x=<value optimized out>, env=0x300a2668) at eval.c:4219

(Note the "Unknown option name"...)

And `(trap-enable 'enter-frame)' loops forever and yields a stack
overflow:

  #0  scm_gc_mark_dependencies (p=0x3019b0b0) at gc-mark.c:199
  #1  0x0ff4b6cc in scm_gc_mark (ptr=0x3019b0b0) at gc-mark.c:169
  #2  0x0ff4b34c in scm_gc_mark_dependencies (p=<value optimized out>) at gc-mark.c:218
  #3  0x0ff4b6cc in scm_gc_mark (ptr=0x3019b0b0) at gc-mark.c:169
  #4  0x0ff4b34c in scm_gc_mark_dependencies (p=<value optimized out>) at gc-mark.c:218
  [ repeated ad infinitum... ]

Deserves further investigation...  ;-)

Thanks,
Ludovic.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Han-Wen Nienhuys]

Ludovic Courtès escreveu:
> Hi,
> 
> Han-Wen Nienhuys <hanwen@xs4all.nl> writes:
> 
>> [hanwen@haring info]$ guile
>> guile> (trap-enable 'enter-frame-handler)
>> throw from within critical section.
>> Abortado
> 
> Same here with HEAD and 1.8.0.  Here's what happens:
> 
>   #0  0x0fc13f6c in raise () from /lib/tls/libc.so.6
>   #1  0x0fc15a0c in abort () from /lib/tls/libc.so.6
>   #2  0x0ffa4a1c in scm_ithrow (key=0x10015410, args=0x300a2558, noreturn=<value optimized out>) at throw.c:699
>   #3  0x0ff3718c in scm_error_scm (key=0x10015410, subr=0x300781f0, message=0x300781d0, args=0x300a2578, data=0x4) at error.c:92
>   #4  0x0ff37220 in scm_error (key=0x10015410, subr=<value optimized out>, message=0xffc5594 "Unknown option name: ~S", args=0x300a2578, 
>       rest=0x4) at error.c:58
>   #5  0x0ff37268 in scm_misc_error (subr=0xfa <Address 0xfa out of bounds>, message=0x3001bb60 "", args=0x0) at error.c:268
>   #6  0x0ff721a8 in scm_options (args=0x300a25e8, options=0xffe1b40, n=7, s=0xffc117c "evaluator-traps-interface") at options.c:202
>   #7  0x0ff3a1e4 in scm_evaluator_traps (setting=0x300a25c8) at eval.c:3134
>   #8  0x0ff3ec50 in deval (x=<value optimized out>, env=0x300a2668) at eval.c:4219
> 
> (Note the "Unknown option name"...)

I have added 

 /* the jump buffer data structure */
@@ -695,7 +696,24 @@ scm_ithrow (SCM key, SCM args, int noret
 
   if (scm_i_critical_section_level)
     {
+      SCM s = args;
+      int i = 0;
+
+      /*
+       We have much better routines for displaying Scheme, but we're
+       already inside a pernicious error, and it's unlikely that they
+       are available to us. We try to print something useful anyway,
+       so users don't need a debugger to find out what went wrong.
+       */
       fprintf (stderr, "throw from within critical section.\n");
+      if (scm_is_symbol (key))
+       fprintf (stderr, "error key: %s\n", scm_i_symbol_chars (key));
+
+      
+      for (; scm_is_pair (s); s = scm_cdr (s), i++)
+       if (scm_is_string (scm_car (s)))
+         fprintf (stderr, "argument %d: %s\n", i, scm_i_string_chars (scm_car (s)));
+      
       abort ();
     }
 

I wonder, is there any generic routine for safely printing immediates 
and strings/symbols?  This might also be a good idea to apply to 1.8.
With lilypond, we occasionally have this problem as well, and it's
a PITA to have to use a debugger to find out about a user-level fault.


 
-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Kevin Ryde]

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:
>
> This might also be a good idea to apply to 1.8.

Yes, it's a bit obscure otherwise.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Han-Wen Nienhuys]

Neil Jerram escreveu:
> Han-Wen Nienhuys <hanwen@xs4all.nl> writes:
> 
>> Since noone responded, I decided to take a look myself.
>> My overall idea was to do the following:
>>
>>  - at the top of deval(), find out current source file and line
> 
> OK, but with the code where you have it at the moment, you'll miss
> tail-recursive calls.
> 
>>  - invoke some kind of callback that increments a counter for the
>> source location
>>
>>  - produce pretty pretty coverage graphs with the contents of the hash
>>
>> The last two steps are trivial, but I'm getting lost with the first
>> step.
>>
>> I tried to follow what happens when an error backtrace is generated.
>> My try is below. Unfortunately, I can't find much documentation on how
>> frames/stacks/etc. interact. Can anyone gently nudge me in the right
>> direction?
> 
> Do you think you need to do this in C?  (You might do, for reasonable

Unfortunately, this is way too slow.

**
[lilydev@haring lilypond]$ time lilypond input/example-1
GNU LilyPond 2.11.10

Hangup

real    0m2.534s
user    0m2.456s
sys     0m0.063s


[lilydev@haring lilypond]$ time lilypond -dcoverage input/example-1
GNU LilyPond 2.11.10

Hangup

real    1m22.184s
user    1m19.808s
sys     0m0.235s
**

Perhaps the better option is to somehow instrument the code such that
memoization of an expression records the coverage. Then we won't get 
execution counts, but it should be almost as fast as normal running.

   

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Ludovic Courtès]

Hi,

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:

> Unfortunately, this is way too slow.
>
> **
> [lilydev@haring lilypond]$ time lilypond input/example-1
> GNU LilyPond 2.11.10
>
> Hangup
>
> real    0m2.534s
> user    0m2.456s
> sys     0m0.063s
>
>
> [lilydev@haring lilypond]$ time lilypond -dcoverage input/example-1
> GNU LilyPond 2.11.10
>
> Hangup
>
> real    1m22.184s
> user    1m19.808s
> sys     0m0.235s
> **

Is this with just the `enter-frame' trap enabled (with corresponding
handler), or are there any additional traps?  What do(es) the handler(s)
do?

Besides, it _really_ doesn't work here (see backtrace in my previous
post):

  $ guile-1.8.0 
  guile> (trap-set! enter-frame-handler (lambda args (format #f "args: ~a~%" args)))
  (exit-frame-handler #f apply-frame-handler #f enter-frame-handler #<procedure #f args> traps)
  guile> (trap-enable 'enter-frame)
  Segmentation fault (core dumped)

Did I miss something, or did I mess up with my Guile?

> Perhaps the better option is to somehow instrument the code such that
> memoization of an expression records the coverage. Then we won't get 
> execution counts, but it should be almost as fast as normal running.

If such instrumentation were to be added to `eval', we'd have to make
sure that it gets commented out when we don't want it (just like
debugging code is pruned from `ceval ()').

Thanks,
Ludovic.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Han-Wen Nienhuys]

Ludovic Courtès escreveu:
> Hi,
> 
> Han-Wen Nienhuys <hanwen@xs4all.nl> writes:
> 
>> Unfortunately, this is way too slow.
>>
>> **
>> [lilydev@haring lilypond]$ time lilypond input/example-1
>> GNU LilyPond 2.11.10
>>
>> Hangup
>>
>> real    0m2.534s
>> user    0m2.456s
>> sys     0m0.063s
>>
>>
>> [lilydev@haring lilypond]$ time lilypond -dcoverage input/example-1
>> GNU LilyPond 2.11.10
>>
>> Hangup
>>
>> real    1m22.184s
>> user    1m19.808s
>> sys     0m0.235s
>> **
> 
> Is this with just the `enter-frame' trap enabled (with corresponding
> handler), or are there any additional traps?  What do(es) the handler(s)
> do?

this is with enter frame, which does 

 (define (record-coverage key continuation . args)
  (let*
      ((frame (last-stack-frame continuation))
       (source (frame-source frame))
       (file (and source (source-property source 'filename)))
       (cov-vector (and file (hash-ref covered-files file)))
       (line (and cov-vector (source-property source 'line)))
       )
    
    (if line
        (vector-set! cov-vector
                     line
                     (1+ (vector-ref cov-vector line))
                     ))))

> 
> Besides, it _really_ doesn't work here (see backtrace in my previous
> post):

the program I posted was tested with GUILE 1.8, perhaps you can start from there.
  


-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Ludovic Courtès]

Hi,

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:

>> Besides, it _really_ doesn't work here (see backtrace in my previous
>> post):
>
> the program I posted was tested with GUILE 1.8, perhaps you can start from there.

That's what I did (see my previous message).  :-(

Thanks,
Ludovic.

PS: BTW, the messages you post to this list are delivered twice.  I
    think the reason is that, when you reply to a message, in addition
    to the `Newgroups:' field (which contains Gmane's newsgroup for this
    list), your news/mail client keeps `guile-devel@gnu.org' in the
    `To:' or `Cc:' list.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Neil Jerram]

Neil Jerram <neil@ossau.uklinux.net> writes:

> guile-debugging should be able to accumulate coverage of all source
> expressions - say in a particular file - but it might incur a big
> performance cost in so doing.  I'll try this out and let you know what
> I find.

For the record, here is how one can do code coverage using
guile-debugging.  I appreciate this may in practice be too slow for
complex programs, but it seems to work reasonably for small amounts of
code.

First we implement a basic code coverage trap:

  (use-modules (ice-9 debugging traps))

  (define covered-file-names '())

  (define coverage-hash (make-hash-table 31))

  (define (cover-entry trap-context)
    (let ((source-pos (frame->source-position (tc:frame trap-context))))
      (if (and source-pos
               (member (car source-pos) covered-file-names))
          (hash-set! coverage-hash
                     source-pos
                     (+ (or (hash-ref coverage-hash source-pos) 0) 1)))))

  (define coverage-trap (make <entry-trap> #:behaviour cover-entry))

Then here's an example of how to use this trap:

  (use-modules (ice-9 debugging example-fns))

  (set! covered-file-names
    '("/usr/share/guile/ice-9/debugging/example-fns.scm "))

  (install-trap coverage-trap)

  (fact2 5)

  (uninstall-trap coverage-trap) ;; The trap does hit performance a
  ;; bit while it is installed, so this uninstall is just to get back to
  ;; normal performance.

And here are the results that I get from this, which look correct to me:

  (hash-fold (lambda (key value acc)
               (format #t "~S: ~S\n" key value))
             #f
             coverage-hash)
  |=
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 11 21): 5
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 3 0): 1
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 8 0): 1
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 9 2): 6
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 13 0): 1
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 9 6): 6
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 14 2): 1
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 11 6): 5
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 11 13): 5
  ("/usr/share/guile/ice-9/debugging/example-fns.scm" 0 0): 3

For profiling, this could be extended to record the time when
evaluation of each source expression begins, and to subtract this from
the time when the frame for each source expression is exited.  The
guile-debugging "at-exit" procedure allows you to do something when a
frame is exited, so `cover-entry' would be extended as follows.

  (use-modules (ice-9 debugging steps))  ; for at-exit

  (define (cover-entry trap-context)
    (let ((source-pos (frame->source-position (tc:frame trap-context)))
          (entry-time #f))
      (if (and source-pos
               (member (car source-pos) covered-file-names))
          (begin
            (hash-set! coverage-hash
                       source-pos
                       (+ (or (hash-ref coverage-hash source-pos) 0) 1))
            (at-exit (tc:depth trap-context)
              (lambda (ignored)
                (hash-set! accumulated-time-hash
                           source-pos
                           (+ (or (hash-ref accumulated-time-hash source-pos) 0)
                              (- (get-time-now) entry-time)))))
            (set! entry-time (get-time-now))))))

I've glossed over details of how to get the current time, and time
arithmetic, so this probably won't work as is, but the intent should
be clear.

It will also be slow, and will incorrectly increase the time of
non-"leaf" code by the time taken by the coverage/profiling code
itself.  guile-statprof does a better job of trying to mitigate the
latter factor, so is in fact a better current bet for profiling Guile
code.

Regards,
     Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Kevin Ryde]

Neil Jerram <neil@ossau.uklinux.net> writes:
>
>   (define coverage-trap (make <entry-trap> #:behaviour cover-entry))

You don't have to use goops to get a trap do you?

> guile-statprof does a better job of trying to mitigate the
> latter factor, so is in fact a better current bet for profiling Guile
> code.

I expect there's a place for both extry/exit and prof style.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: frames / stacks / source? was Re: coverage/profiling

[Neil Jerram]

Kevin Ryde <user42@zip.com.au> writes:

> Neil Jerram <neil@ossau.uklinux.net> writes:
>>
>>   (define coverage-trap (make <entry-trap> #:behaviour cover-entry))
>
> You don't have to use goops to get a trap do you?

In terms of the low-level call outs from the evaluator, no.  In terms
of the infrastructure that (ice-9 debugging traps) builds on top of
that, yes.

Regards,
     Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel