Compilation and load-extension

Compilation and load-extension

[Tristan Colgate]

Hi All,

  I'm using guile and swig for some library binding and get
compilation errors/warning
that seem to be due to the compiler not knowing that a load-extension
has supplied
a module (this is with library bindings generated by swig).

  Is there a way to get rid of the warnings (other then turning off
compilation?). Everything
works well (after some minor changes to swig), it just looks a bit untidy.

  Cheers,

-- 
Tristan Colgate-McFarlane
----
  "You can get all your daily vitamins from 52 pints of guiness, and a
glass of milk"

Re: Compilation and load-extension

[Ludovic Courtès]

Hi,

Tristan Colgate <tcolgate@gmail.com> writes:

>   I'm using guile and swig for some library binding and get
> compilation errors/warning
> that seem to be due to the compiler not knowing that a load-extension
> has supplied
> a module (this is with library bindings generated by swig).

Can you copy/paste the errors/warnings?

Thanks,
Ludo’.

Re: Compilation and load-extension

[Tristan Colgate]

Ah, sorry....

;;; note: autocompilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-autocompile argument to disable.
;;; compiling /usr/local/share/guile/site/snmp/net-snmp.scm
;;; WARNING: compilation of
/usr/local/share/guile/site/snmp/net-snmp.scm failed:
;;; key misc-error, throw args (#f "~A ~S" ("no code for module" (snmp
net-snmp-primitive)) #f)
STRING: Unknown (edit /etc/snmp/snmpd.conf)
STRING: pinot

the snmp net-snmp-primitive module is provided by a load-extension in
the swig wrapper module.
it's loaded by the module that swig has generated (snmp net-snmp)

2009/12/18 Ludovic Courtès <ludo@gnu.org>:
> Hi,
>
> Tristan Colgate <tcolgate@gmail.com> writes:
>
>>   I'm using guile and swig for some library binding and get
>> compilation errors/warning
>> that seem to be due to the compiler not knowing that a load-extension
>> has supplied
>> a module (this is with library bindings generated by swig).
>
> Can you copy/paste the errors/warnings?
>
> Thanks,
> Ludo’.
>
>
>
>



-- 
Tristan Colgate-McFarlane
----
  "You can get all your daily vitamins from 52 pints of guiness, and a
glass of milk"

Re: Compilation and load-extension

[Ludovic Courtès]

Hi,

Tristan Colgate <tcolgate@gmail.com> writes:

> Ah, sorry....
>
> ;;; note: autocompilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;       or pass the --no-autocompile argument to disable.
> ;;; compiling /usr/local/share/guile/site/snmp/net-snmp.scm
> ;;; WARNING: compilation of
> /usr/local/share/guile/site/snmp/net-snmp.scm failed:
> ;;; key misc-error, throw args (#f "~A ~S" ("no code for module" (snmp
> net-snmp-primitive)) #f)
> STRING: Unknown (edit /etc/snmp/snmpd.conf)
> STRING: pinot
>
> the snmp net-snmp-primitive module is provided by a load-extension in
> the swig wrapper module.
> it's loaded by the module that swig has generated (snmp net-snmp)

So does the .so that’s loaded via ‘load-extension’ calls
‘scm_c_define_module ()’, right?  And ‘net-snmp.scm’ does:

  (define-module (snmp net-snmp)
    #:export (...))

  (load-extension ...)

  (use-modules (snmp net-snmp-primitive))

Right?

Could you make sure with strace(1) or some such that the right .so gets
loaded?

Thanks,
Ludo’.

Re: Compilation and load-extension

[Tristan Colgate]

yes, swig generates snmp/net-snmp.scm with the following header...

(define-module (snmp net-snmp))

(load-extension "libguile_snmp_net-snmp.so" "scm_init_snmp_net_snmp_module")

(use-modules (oop goops) (Swig common))
(use-modules ((snmp net-snmp-primitive) :renamer (symbol-prefix-proc
'primitive:)))

the net-snmp_wrap.c contains...

SCM
scm_init_snmp_net_snmp_module (void)
{
scm_c_define_module("snmp net-snmp-primitive",
  SWIG_init_helper, NULL);
return SCM_UNSPECIFIED;
}

I can confirm that the correct versions of the various files are being
loaded (have confirmed via strace
previously whilst debugging other issues).

  As mentioned before, this is purely cosmetic in that the code does
actually get run and work, it is the
compilation that fails (thhough things carry on anyway)

2009/12/18 Ludovic Courtès <ludo@gnu.org>:
> Hi,
>
> Tristan Colgate <tcolgate@gmail.com> writes:
>
>> Ah, sorry....
>>
>> ;;; note: autocompilation is enabled, set GUILE_AUTO_COMPILE=0
>> ;;;       or pass the --no-autocompile argument to disable.
>> ;;; compiling /usr/local/share/guile/site/snmp/net-snmp.scm
>> ;;; WARNING: compilation of
>> /usr/local/share/guile/site/snmp/net-snmp.scm failed:
>> ;;; key misc-error, throw args (#f "~A ~S" ("no code for module" (snmp
>> net-snmp-primitive)) #f)
>> STRING: Unknown (edit /etc/snmp/snmpd.conf)
>> STRING: pinot
>>
>> the snmp net-snmp-primitive module is provided by a load-extension in
>> the swig wrapper module.
>> it's loaded by the module that swig has generated (snmp net-snmp)
>
> So does the .so that’s loaded via ‘load-extension’ calls
> ‘scm_c_define_module ()’, right?  And ‘net-snmp.scm’ does:
>
>  (define-module (snmp net-snmp)
>    #:export (...))
>
>  (load-extension ...)
>
>  (use-modules (snmp net-snmp-primitive))
>
> Right?
>
> Could you make sure with strace(1) or some such that the right .so gets
> loaded?
>
> Thanks,
> Ludo’.
>
>
>
>



-- 
Tristan Colgate-McFarlane
----
  "You can get all your daily vitamins from 52 pints of guiness, and a
glass of milk"

Re: Compilation and load-extension

[Andy Wingo]

Hi Tristan,

>> Tristan Colgate <tcolgate@gmail.com> writes:
>>
>>> Ah, sorry....
>>>
>>> ;;; note: autocompilation is enabled, set GUILE_AUTO_COMPILE=0
>>> ;;;       or pass the --no-autocompile argument to disable.
>>> ;;; compiling /usr/local/share/guile/site/snmp/net-snmp.scm
>>> ;;; WARNING: compilation of
>>> /usr/local/share/guile/site/snmp/net-snmp.scm failed:
>>> ;;; key misc-error, throw args (#f "~A ~S" ("no code for module" (snmp
>>> net-snmp-primitive)) #f)

I can't see how this is happening, though I am sure it is indeed
happening for you :) Can you put a tarball up somewhere so that we can
have a look at it?

Andy
-- 
http://wingolog.org/

Re: Compilation and load-extension

[Tristan Colgate]

2009/12/28 Andy Wingo <wingo@pobox.com>:
> I can't see how this is happening, though I am sure it is indeed
> happening for you :) Can you put a tarball up somewhere so that we can
> have a look at it?

I've created a stripped down module that uses that same style as the
swig ones and attached it to the following
bug report

http://savannah.gnu.org/bugs/index.php?28442

Could it be the return value of the init function? is that supposed to
return anything particular?
-- 
Tristan Colgate-McFarlane
----
  "You can get all your daily vitamins from 52 pints of guiness, and a
glass of milk"

Re: Compilation and load-extension

[Andy Wingo]

Hi Tristan,

On Wed 30 Dec 2009 16:27, Tristan Colgate <tcolgate@gmail.com> writes:

> 2009/12/28 Andy Wingo <wingo@pobox.com>:
>> I can't see how this is happening, though I am sure it is indeed
>> happening for you :) Can you put a tarball up somewhere so that we can
>> have a look at it?
>
> I've created a stripped down module that uses that same style as the
> swig ones and attached it to the following
> bug report
>
> http://savannah.gnu.org/bugs/index.php?28442

Thanks for the test case. Here's dir/mymodule.scm:

    (define-module (dir mymodule))
    (load-extension "test-module.so" "scm_init_my_module")
    (use-modules ((dir mymodule-prim) :renamer (symbol-prefix-proc 'primitive:)))

What happens when Guile tries to compile this file is that the forms are
expanded one by one, then compiled all together. Since expanders are
written in Scheme they can do any computation -- and indeed we have
`define-module' and `use-modules' hooked up to define and import modules
at expansion-time as well as when you load the compiled file
(load-time).

But the problem is that the load-extension form is just a function call,
not a macro, so it's not going to cause any code to run at expand-time.
Specifically it will not cause the extension to be loaded at expand
time, so later when you try to use (dir mymodule-prim) Guile rightly
complains about there being no such module.

Guile then switches to the interpreter, after compilation failed -- and
since the interpreter evaluates each form after expanding it, indeed the
extension does get loaded in time for the use-modules to see it.

The solution is to enclose the (load-extension ...) in an eval-when:

    (eval-when (eval load compile)
      (load-extension "test-module.so" "scm_init_my_module"))

This ensures that the primitives module gets defined at compile-time.

See also the NEWS section entitled, "Functions needed by macros at
expand-time need to be present at expand-time.".

If this code is generated by SWIG, as you had mentioned before, please
let the SWIG maintainers know about this issue, and point them to this
message.

Regards,

Andy
-- 
http://wingolog.org/

Re: Compilation and load-extension

[Tristan Colgate]

2010/1/3 Andy Wingo <wingo@pobox.com>:
> Hi Tristan,
...
>    (eval-when (eval load compile)
>      (load-extension "test-module.so" "scm_init_my_module"))
>
  Thanks for the explanation and the fix, it works nicely and I now
have previous code running cleanly, a guile-lib based unit test
framework on its way.

> If this code is generated by SWIG, as you had mentioned before, please
> let the SWIG maintainers know about this issue, and point them to this
> message.

  On further inspection this is actually my code in my swig binding,
so trivial to update on my end. swig does still need some changes but
I haven't got a good enough understanding yet to go to the swig guys
(memory allocation is the main issue, and they use some deprecated
functions.

  Cheers, and thanks for all the great work on guile.

-- 
Tristan Colgate-McFarlane
----
  "You can get all your daily vitamins from 52 pints of guiness, and a
glass of milk"