Difference when calling guile script from C vs interpreter

Difference when calling guile script from C vs interpreter

[Jean Rene Dawin]

Hi,

when following guile script: 
______ gp.guile _________

(use-modules (statprof))
(use-modules (system vm program))

;Scalar product
(define (sp a b)
   (+ 
      (* (car a)(car b)) 
      (* (cdr a)(cdr b))
   )
)
(define size 444100)
(define (run)
   (let ((accu 0.001) 
         (r (make-f32vector size 0.0))
        )
         (let lp ((j 0) )
            (when (< j size)
               (set! accu (+ accu 0.001))
               (f32vector-set! r j (sp (cons accu (* 2 accu)) (cons (* 3 accu) (* 5 accu))))
               (lp (+ j 1) )
            )
         )
   	 (display (f32vector-ref r (- size 1)))(newline)
   )
)
(statprof run)
(display (program? run))(newline)

______________________________

is run from the interpreter, the output shows the following:

scheme@(guile-user)> (load "gp.guile")
;;; note: source file /home/gp.guile
;;;       newer than compiled /home/.cache/guile/ccache/2.2-LE-8-3.A/home/gp.guile.go
;;; compiling /home/gp.guile
;;; compiled /home/.cache/guile/ccache/2.2-LE-8-3.A/home/gp.guile.go
2563934.0
%     cumulative   self             
time   seconds     seconds  procedure
 50.00      0.33      0.16  /home/gp.guile:12:0:run
 28.57      0.09      0.09  %after-gc-thunk
 21.43      0.14      0.07  /home/gp.guile:5:0:sp
  0.00      0.09      0.00  anon #x7fe262d41380
---
Sample count: 14
Total time: 0.327908556 seconds (0.121140347 seconds in GC)
#t



When the same script is loaded from following C program:
_____ gp.c _______
#include <libguile.h>

int main(int argc, char **argv)
{
   scm_init_guile();
   scm_c_primitive_load("/home/gp.guile");
   return 0;
}
___________________

the result looks like this:

$ gcc -o gp gp.c $(guile-config compile) $(guile-config link)
$ ./gp 
2563934.0
%     cumulative   self             
time   seconds     seconds  procedure
 15.60      0.30      0.30  ice-9/eval.scm:226:7
 12.84      0.59      0.25  ice-9/eval.scm:159:9
 12.84      0.25      0.25  ice-9/eval.scm:182:7
  6.42      3.49      0.12  ice-9/eval.scm:618:6
  6.42      0.55      0.12  ice-9/eval.scm:159:9
  5.50      0.25      0.11  ice-9/eval.scm:625:6
  5.50      0.11      0.11  ice-9/eval.scm:123:11
  3.67      0.07      0.07  ice-9/eval.scm:282:4
  3.67      0.07      0.07  ice-9/eval.scm:124:11
  3.67      0.07      0.07  srfi/srfi-4.scm:85:20:f32vector-set!
  2.75      0.53      0.05  ice-9/eval.scm:159:9
  2.75      0.20      0.05  ice-9/eval.scm:191:12
  2.75      0.12      0.05  ice-9/eval.scm:263:9
  2.75      0.07      0.05  ice-9/eval.scm:155:9
  1.83      1.10      0.04  ice-9/eval.scm:163:9
  1.83      0.46      0.04  ice-9/eval.scm:159:9
  1.83      0.04      0.04  ice-9/eval.scm:336:13
  1.83      0.04      0.04  ice-9/eval.scm:222:7
  1.83      0.04      0.04  ice-9/eval.scm:590:16
  1.83      0.04      0.04  ice-9/eval.scm:333:13
  0.92      0.02      0.02  ice-9/eval.scm:273:7
  0.92      0.02      0.02  ice-9/eval.scm:224:11
  0.00      0.05      0.00  ice-9/eval.scm:155:9
---
Sample count: 109
Total time: 1.939099187 seconds (0.570765622 seconds in GC)
#t

Is this difference expected?
It is 6 times slower when called from C versus from the interpreter.
And the statprof output of the C version shows calls in ice-9/eval.scm
but from the interpreter it doesn't. As I'm new to guile, any
hints would be helpful.

This is GNU Guile 2.2.6 on x86_64 GNU/Linux.

Regards,
Jean Rene Dawin

Re: Difference when calling guile script from C vs interpreter

[Taylan Kammer]

On 20.10.2020 14:58, Jean Rene Dawin wrote:
> Hi,
> 
> when following guile script:
> ______ gp.guile _________
> 
> (snip)
> ______________________________
> 
> is run from the interpreter, the output shows the following:
> 
> (snip)
> Total time: 0.327908556 seconds (0.121140347 seconds in GC)
> #t
> 
> 
> 
> When the same script is loaded from following C program:
> _____ gp.c _______
> #include <libguile.h>
> 
> int main(int argc, char **argv)
> {
>     scm_init_guile();
>     scm_c_primitive_load("/home/gp.guile");
>     return 0;
> }
> ___________________
> 
> the result looks like this:
> 
> (snip)
> Total time: 1.939099187 seconds (0.570765622 seconds in GC)
> #t
> 
> Is this difference expected?

When you use the "guile" executable to run a file, it automatically 
compiles it first, then runs the compiled version.  (For this reason I 
wouldn't use the term "interpreter" in this case.)

When using primitive-load, it doesn't auto-compile it (and I think it 
doesn't even look for an already compiled version), instead it directly 
calls the interpreter on the source code.  (This time it really is the 
interpreter.)

It's generally preferable to *extend* Guile with C code, rather than to 
*embed* it in C code.  This means you turn your C code into libraries, 
which are turned into Guile modules, then you write a Guile program that 
uses those modules.  I.e. you call to C code from Guile, not to Guile 
code from C.

If you want to really prefer to embed and not extend, then perhaps you 
can make your Guile files available in the %load-path and use 
scm_primitive_load_path (see documentation).

I don't know if/how the Guile compiler can be called from C.  Maybe 
someone more experienced can help further.


- Taylan

Re: Difference when calling guile script from C vs interpreter

[Roel Janssen]

On Tue, 2020-10-20 at 16:27 +0200, Taylan Kammer wrote:
> On 20.10.2020 14:58, Jean Rene Dawin wrote:
> > Hi,
> > 
> > when following guile script:
> > ______ gp.guile _________
> > 
> > (snip)
> > ______________________________
> > 
> > is run from the interpreter, the output shows the following:
> > 
> > (snip)
> > Total time: 0.327908556 seconds (0.121140347 seconds in GC)
> > #t
> > 
> > 
> > 
> > When the same script is loaded from following C program:
> > _____ gp.c _______
> > #include <libguile.h>
> > 
> > int main(int argc, char **argv)
> > {
> >     scm_init_guile();
> >     scm_c_primitive_load("/home/gp.guile");
> >     return 0;
> > }
> > ___________________
> > 
> > the result looks like this:
> > 
> > (snip)
> > Total time: 1.939099187 seconds (0.570765622 seconds in GC)
> > #t
> > 
> > Is this difference expected?
> 
> When you use the "guile" executable to run a file, it automatically 
> compiles it first, then runs the compiled version.  (For this reason
> I 
> wouldn't use the term "interpreter" in this case.)
> 
> When using primitive-load, it doesn't auto-compile it (and I think it
> doesn't even look for an already compiled version), instead it
> directly 
> calls the interpreter on the source code.  (This time it really is
> the 
> interpreter.)
> 
> It's generally preferable to *extend* Guile with C code, rather than
> to 
> *embed* it in C code.  This means you turn your C code into
> libraries, 
> which are turned into Guile modules, then you write a Guile program
> that 
> uses those modules.  I.e. you call to C code from Guile, not to Guile
> code from C.
> 
> If you want to really prefer to embed and not extend, then perhaps
> you 
> can make your Guile files available in the %load-path and use 
> scm_primitive_load_path (see documentation).
> 
> I don't know if/how the Guile compiler can be called from C.  Maybe 
> someone more experienced can help further.
> 
> 
> - Taylan
> 
> 

Once you've compiled the Guile module, it should be enough to set the
GUILE_LOAD_COMPILED_PATH environment variable before booting Guile in
C.  Here's what I use:

https://github.com/UMCUGenetics/sparqling-genomics/blob/master/web/sg-web.c.in#L314-L356

Kind regards,
Roel Janssen

Re: Difference when calling guile script from C vs interpreter

[Jean Rene Dawin]

Hi,

thanks for your responses.

> It's generally preferable to *extend* Guile with C code, rather than to *embed* it in C code.
> This means you turn your C code into libraries, which are turned into Guile modules, then you write a Guile program
> that uses those modules.  I.e. you call to C code from Guile, not to Guile code from C.

Well, in my case I want my C program to load/compile guile scripts during run time.
The guile scripts are intended to be altered and then reloaded/compiled from within the running C program.

> If you want to really prefer to embed and not extend, then perhaps you can make your Guile files available
> in the %load-path and use scm_primitive_load_path (see documentation). 

> Once you've compiled the Guile module, it should be enough to set the
> GUILE_LOAD_COMPILED_PATH environment variable before booting Guile in C

Right. When I use

   scm_primitive_load_path(scm_from_locale_string("gp.guile"));

and set the environment variables

GUILE_LOAD_PATH=""
GUILE_LOAD_COMPILED_PATH=""

the code is indeed compiled:

$ ./gp 
;;; note: source file gp.guile
;;;       newer than compiled /home/.cache/guile/ccache/2.2-LE-8-3.A/home/gp.guile.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling gp.guile
;;; compiled /home/.cache/guile/ccache/2.2-LE-8-3.A/home/gp.guile.go


I'm not sure why the empty environment variables work, because the corresponding variables in guile look like this:

%load-path: ( /usr/share/guile/2.2 /usr/share/guile/site/2.2 /usr/share/guile/site /usr/share/guile)
%load-compiled-path:( /usr/lib/guile/2.2/ccache /usr/lib/guile/2.2/site-ccache)

so my home and .cache directory are not included.

What also works is to give the full path to scm_primitive_load_path and unset the environment variables:

   scm_primitive_load_path(scm_from_locale_string("/home/gp.guile"));

$ unset GUILE_LOAD_COMPILED_PATH
$ unset GUILE_LOAD_PATH
$ ./gp 
;;; note: source file /home/gp.guile
;;;       newer than compiled /home/.cache/guile/ccache/2.2-LE-8-3.A/home/gp.guile.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/gp.guile
;;; compiled /home/.cache/guile/ccache/2.2-LE-8-3.A/home/gp.guile.go

For now I'm fine with these solutions. Thanks again.

Regards,
Jean Rene