Code coverage is broken

Code coverage is broken

[Nala Ginrut]

Hi folks!
I'm trying to add code coverage for certain program.
However, the function callings are always unrecorded.
--------------------------------------------------------------------------------------
(use-modules (system vm coverage)
                         (system vm vm))
   (call-with-values
        (lambda ()
           (with-code-coverage
             (lambda () (display "hello\n"))))
        (lambda (data result)
          (let ((port (open-output-file "1.info")))
            (coverage-data->lcov data port)
            (close port))))
---------------------------------------------------------------------------------------

(To someone who uses old 2.0, with-code-coverage interface has been
changed to accept only one thunk)

When I use lcov to generate coverage graph, it always complains that
--------------------------------------------
 lines......: 0.0% (0 of 36447 lines)
  functions..: no data found
  branches...: no data found
--------------------------------------------
I found the FN tag in info file is always missing.

Then I found in 581a4eb82b1534970060e3cbd79b9a96d351edf9

-------------------------------------------------
+      #;
         (for-each dump-function procs)
-------------------------------------------------
It seems that FN printing was skipped intended.

Is it a bug?
Anyone who has tried code coverage?

Thanks!

Re: Code coverage is broken

[Amirouche Boubekki]

[-- Attachment #1: Type: text/plain, Size: 3367 bytes --]

code coverage work with guile-2.2

Here is the test run:

amirouche@ubujan19:~/src/scheme/guile/coverage$ guile --version
guile (GNU Guile) 2.2.4
Copyright (C) 2018 Free Software Foundation, Inc.

License LGPLv3+: GNU LGPL 3 or later <http://gnu.org/licenses/lgpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
amirouche@ubujan19:~/src/scheme/guile/coverage$ ls
Makefile  run-tests-with-coverage.scm  test.scm
amirouche@ubujan19:~/src/scheme/guile/coverage$ cat Makefile
all:
    @guile --debug run-tests-with-coverage.scm test.scm
    @genhtml test.scm.info | tail -n 3
amirouche@ubujan19:~/src/scheme/guile/coverage$ cat
run-tests-with-coverage.scm
(use-modules (system vm coverage)
             (system vm vm))


(define (run-test-with-coverage test)
  (call-with-values (lambda ()
                      (with-code-coverage
                       (lambda ()
                         (load test))))
    (lambda (data result)
      (let ((port (open-output-file (string-append test ".info"))))
        (coverage-data->lcov data port)
        (close port)))))

(run-test-with-coverage (cadr (program-arguments)))
amirouche@ubujan19:~/src/scheme/guile/coverage$ cat test.scm
(display "test echo\n")
amirouche@ubujan19:~/src/scheme/guile/coverage$ make
test echo
Overall coverage rate:
  lines......: 1.5% (135 of 9180 lines)
  functions..: no data found
amirouche@ubujan19:~/src/scheme/guile/coverage$

Get the code with: git clone https://github.com/a-guile-mind/coverage.git

The thing that is not implemented is getting the coverage for the "current
directory".
The percentage that genhtml gives is the overall coverage of all the code
that is loaded (I guess).

Le ven. 8 févr. 2019 à 21:41, Nala Ginrut <nalaginrut@gmail.com> a écrit :

> Hi folks!
> I'm trying to add code coverage for certain program.
> However, the function callings are always unrecorded.
>
> --------------------------------------------------------------------------------------
> (use-modules (system vm coverage)
>                          (system vm vm))
>    (call-with-values
>         (lambda ()
>            (with-code-coverage
>              (lambda () (display "hello\n"))))
>         (lambda (data result)
>           (let ((port (open-output-file "1.info")))
>             (coverage-data->lcov data port)
>             (close port))))
>
> ---------------------------------------------------------------------------------------
>
> (To someone who uses old 2.0, with-code-coverage interface has been
> changed to accept only one thunk)
>
> When I use lcov to generate coverage graph, it always complains that
> --------------------------------------------
>  lines......: 0.0% (0 of 36447 lines)
>   functions..: no data found
>   branches...: no data found
> --------------------------------------------
> I found the FN tag in info file is always missing.
>

> Then I found in 581a4eb82b1534970060e3cbd79b9a96d351edf9
>
> -------------------------------------------------
> +      #;
>          (for-each dump-function procs)
> -------------------------------------------------
> It seems that FN printing was skipped intended.
>
> Is it a bug?
> Anyone who has tried code coverage?
>
> Thanks!
>
>

[-- Attachment #2: Type: text/html, Size: 4542 bytes --]

Re: Code coverage is broken

[Amirouche Boubekki]

[-- Attachment #1: Type: text/plain, Size: 61 bytes --]

Sorry! My test is not good. Ignore what I posted previously.

[-- Attachment #2: Type: text/html, Size: 97 bytes --]

Re: Code coverage is broken

[Nala Ginrut]

Never mind, it's better than mine, at least there're lines records,
although functions calling are still missing.
Thanks ;-)

On Sat, Feb 9, 2019 at 8:27 AM Amirouche Boubekki
<amirouche.boubekki@gmail.com> wrote:
>
> Sorry! My test is not good. Ignore what I posted previously.

Re: Code coverage is broken

[Nala Ginrut]

I wonder if it's a bug that the FN was skipped.

On Sat, Feb 9, 2019 at 10:51 AM Nala Ginrut <nalaginrut@gmail.com> wrote:
>
> Never mind, it's better than mine, at least there're lines records,
> although functions calling are still missing.
> Thanks ;-)
>
> On Sat, Feb 9, 2019 at 8:27 AM Amirouche Boubekki
> <amirouche.boubekki@gmail.com> wrote:
> >
> > Sorry! My test is not good. Ignore what I posted previously.

Re: Code coverage is broken

[Amirouche Boubekki]

[-- Attachment #1: Type: text/plain, Size: 803 bytes --]

Le sam. 9 févr. 2019 à 14:48, Nala Ginrut <nalaginrut@gmail.com> a écrit :

> I wonder if it's a bug that the FN was skipped.
>

Yes it is expected, see
http://git.savannah.gnu.org/cgit/guile.git/tree/module/system/vm/coverage.scm#n309

Here is a patch that take into account function coverage, it is rather slow.

demo: https://screenshots.firefox.com/FmuctotjXTFRSiFW/null

HTH


> On Sat, Feb 9, 2019 at 10:51 AM Nala Ginrut <nalaginrut@gmail.com> wrote:
> >
> > Never mind, it's better than mine, at least there're lines records,
> > although functions calling are still missing.
> > Thanks ;-)
> >
> > On Sat, Feb 9, 2019 at 8:27 AM Amirouche Boubekki
> > <amirouche.boubekki@gmail.com> wrote:
> > >
> > > Sorry! My test is not good. Ignore what I posted previously.
>

[-- Attachment #2: Type: text/html, Size: 1785 bytes --]

Re: Code coverage is broken

[Amirouche Boubekki]

[-- Attachment #1.1: Type: text/plain, Size: 980 bytes --]

with the attachement this time.

Le sam. 9 févr. 2019 à 18:10, Amirouche Boubekki <
amirouche.boubekki@gmail.com> a écrit :

>
>
> Le sam. 9 févr. 2019 à 14:48, Nala Ginrut <nalaginrut@gmail.com> a écrit :
>
>> I wonder if it's a bug that the FN was skipped.
>>
>
> Yes it is expected, see
> http://git.savannah.gnu.org/cgit/guile.git/tree/module/system/vm/coverage.scm#n309
>
> Here is a patch that take into account function coverage, it is rather
> slow.
>
> demo: https://screenshots.firefox.com/FmuctotjXTFRSiFW/null
>
> HTH
>
>
>> On Sat, Feb 9, 2019 at 10:51 AM Nala Ginrut <nalaginrut@gmail.com> wrote:
>> >
>> > Never mind, it's better than mine, at least there're lines records,
>> > although functions calling are still missing.
>> > Thanks ;-)
>> >
>> > On Sat, Feb 9, 2019 at 8:27 AM Amirouche Boubekki
>> > <amirouche.boubekki@gmail.com> wrote:
>> > >
>> > > Sorry! My test is not good. Ignore what I posted previously.
>>
>

[-- Attachment #1.2: Type: text/html, Size: 2223 bytes --]

[-- Attachment #2: 0001-add-function-coverage.patch --]
[-- Type: text/x-patch, Size: 2947 bytes --]

From d32da0570d3f6aa1857331c821ca071b0eca96f2 Mon Sep 17 00:00:00 2001
From: Amirouche <amirouche.boubekki@gmail.com>
Date: Sat, 9 Feb 2019 18:08:45 +0100
Subject: [PATCH] add function coverage


diff --git a/module/system/vm/coverage.scm b/module/system/vm/coverage.scm
index 0d51e261a..f151912f7 100644
--- a/module/system/vm/coverage.scm
+++ b/module/system/vm/coverage.scm
@@ -21,6 +21,7 @@
   #:use-module (system vm frame)
   #:use-module (system vm program)
   #:use-module (system vm debug)
+  #:use-module (system xref)
   #:use-module (ice-9 format)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9)
@@ -309,11 +310,11 @@ gathered, even if their code was not executed."
   ;; FIXME: Re-enable this code, but using for-each-elf-symbol on each source
   ;; chunk.  Use that to build a map of file -> proc-addr + line + name.  Then
   ;; use something like procedure-execution-count to get the execution count.
-  #;
+
   (define (dump-function proc)
     ;; Dump source location and basic coverage data for PROC.
     (and (or (program? proc))
-         (let ((sources (program-sources* data proc)))
+         (let ((sources (program-sources proc)))
            (and (pair? sources)
                 (let* ((line (source:line-for-user (car sources)))
                        (name (or (procedure-name proc)
@@ -330,8 +331,9 @@ gathered, even if their code was not executed."
                 (if (string? path)
                     (begin
                       (format port "SF:~A~%" path)
-                      #;
-                      (for-each dump-function procs)
+                      (let ((procs (file-procedures file)))
+                        (when procs
+                          (for-each dump-function procs)))
                       (for-each (lambda (line+count)
                                   (let ((line  (car line+count))
                                         (count (cdr line+count)))
diff --git a/module/system/xref.scm b/module/system/xref.scm
index e335f9481..d0ac49615 100644
--- a/module/system/xref.scm
+++ b/module/system/xref.scm
@@ -26,7 +26,8 @@
             procedure-callees
             procedure-callers
             source-closures
-            source-procedures))
+            source-procedures
+            file-procedures))
 
 ;;;
 ;;; The cross-reference database: who calls whom.
@@ -371,3 +372,11 @@ pair of the form (module-name . variable-name), "
                  (false-if-exception (open-input-file file))))
          (file (if port (port-filename port) file)))
     (lookup-source-procedures file line *sources-db*)))
+
+(define (lookup-procedures file-table)
+  (apply append (hash-fold (lambda (key value acc) (cons value acc)) '() file-table)))
+
+(define (file-procedures file)
+  "Retrieve all procedures defined in FILE. Can return multiple times the same procedure"
+  (ensure-sources-db #f)
+  (and=> (hash-ref *sources-db* file) lookup-procedures))
-- 
2.19.1