A [serious] problem with module integration

A [serious] problem with module integration

[Panicz Maciej Godek]

Hi,
I've been making some tiny changes in the way my application
uses modules. Previously I was including the scheme code right
into the C binary. This wasn't particularly elegant sollution, as it
didn't allow the code to be compiled, so I decided that I use guile
modules wherever I can.
And now I have a problem: the modules that I wrote make use
of the symbols defined by my application (using scm_c_define...),
but they are unavailable outside my application, i.e. for external
modules.

The result is that the modules don't work, because certain symbols
are undefined that are needed to build the necessary definitions.

If I'd taken a different approach, e.g. if I'd decided to use shared
libraries instead of having the main application call scm_with_guile,
the solution would be easy -- I could just define the wrappers around
the procedures exported by the library.

But now I'm in the big trouble.

Is there any way for the symbols introduced using scm_c_define...
to become available for external modules (that are meant to be
used within the application that I create)? I've tried creating
an empty module that includes the symbols exported by my app,
and nothing else, but it doesn't seem to help.

Perhaps there's some other idea that you could suggest?

Re: A [serious] problem with module integration

[Mike Gran]

Hello Panicz-

> From: Panicz Maciej Godek <godek.maciek@gmail.com>
> And now I have a problem: the modules that I wrote make use
> of the symbols defined by my application (using scm_c_define...),
> but they are unavailable outside my application, i.e. for external
> modules.

[snip]

I read this, and I am having a bit of trouble visualizing
what you've done, but, perhaps your problem can be fixed
by simply replacing scm_c_define with scm_c_export.
 
So try that first.

In any case, I pretty sure I did what you are trying to
when I added guile handling to a private build of the Zile
editor.  Re http://github.com/spk121/zile .  This is a bit
more elaborate because I wanted my exported functions
to be in their own named module

If I recall correctly, (cause I haven't touched this code
in a while), in my C code that defines the Guile exports,
I did this.

- created a new module called 'zile' to hold my functions using
  scm_c_resolve_module("zile")
- defined my functions into that module with 
  scm_c_export
- switched back to the (guile user) environment
  with scm_c_resolve_module("guile-user")
- loaded my newly defined module using
  scm_c_use_module("zile")

Hope this correct, and that it helps.

-Mike Gran

Re: A [serious] problem with module integration

[Mike Gran]

> From: Mike Gran <spk121@yahoo.com>
> To: Panicz Maciej Godek <godek.maciek@gmail.com>; "guile-user@gnu.org" <guile-user@gnu.org>
> Cc: 
> Sent: Friday, November 16, 2012 8:03 AM
> Subject: Re: A [serious] problem with module integration
> 
> Hello Panicz-
> 
>> From: Panicz Maciej Godek <godek.maciek@gmail.com>
>> And now I have a problem: the modules that I wrote make use
>> of the symbols defined by my application (using scm_c_define...),
>> but they are unavailable outside my application, i.e. for external
>> modules.
> 
> [snip]
> 
> I read this, and I am having a bit of trouble visualizing
> what you've done, but, perhaps your problem can be fixed
> by simply replacing scm_c_define with scm_c_export.

I'm replying to my own e-mail, because this is unclear.
You don't *replace* scm_c_define with scm_c_export.
You add a call to scm_c_export after your calls to scm_c_define.

Re: A [serious] problem with module integration

[Panicz Maciej Godek]

2012/11/16 Mike Gran <spk121@yahoo.com>:

> I read this, and I am having a bit of trouble visualizing
> what you've done, but, perhaps your problem can be fixed
> by simply replacing scm_c_define with scm_c_export.
>
> So try that first.

Oh yes, it helped. Thanks a lot!

Re: A [serious] problem with module integration

[Mark H Weaver]

Mike Gran <spk121@yahoo.com> writes:

>> From: Panicz Maciej Godek <godek.maciek@gmail.com>
>> And now I have a problem: the modules that I wrote make use
>> of the symbols defined by my application (using scm_c_define...),
>> but they are unavailable outside my application, i.e. for external
>> modules.
>
> If I recall correctly, (cause I haven't touched this code
> in a while), in my C code that defines the Guile exports,
> I did this.
>
> - created a new module called 'zile' to hold my functions using
>   scm_c_resolve_module("zile")
> - defined my functions into that module with 
>   scm_c_export
> - switched back to the (guile user) environment
>   with scm_c_resolve_module("guile-user")
> - loaded my newly defined module using
>   scm_c_use_module("zile")

What Mike wrote here is almost correct, but there were a few mistakes.

First, I should explain what went wrong with what Panicz did.
'scm_c_define' and 'scm_c_define_gsubr' add the new bindings to the
current module, but they are not exported by default.  Unless you have
set it yourself, the current module in Guile 2.0 will probably be
"guile-user", but it's best not to depend on that.  Better to arrange
for your C bindings to go into a specific module.

Mike tried to do that with 'scm_c_resolve_module', but that only
*returns* the named module, it does not change the current module.

The preferred way to do this is to use 'scm_c_define_module'.  Use it to
call an auxillary C function INIT, where all the 'scm_c_define',
'scm_c_define_gsubr', and 'scm_c_export' calls are done.  During
execution of INIT, the current module will be temporarily set to the new
module, but the current module will be automatically restored to its
previous setting before returning.

Then, I'd recommend calling 'use-modules' from within any Scheme module
that needs to access your C bindings.

A relevant example can be found near the end of the following page:

   http://www.gnu.org/software/guile/manual/html_node/C-Extensions.html

Although that example assumes that the C code is in a shared library, it
should work just as well if the C code is the main program.  Just leave
out the call to 'load-extension', but keep the call to 'use-modules'.

     Mark

Re: A [serious] problem with module integration

[Mike Gran]

> From: Mark H Weaver <mhw@netris.org>
> Mike Gran <spk121@yahoo.com> writes:
> 
> What Mike wrote here is almost correct, but there were a few mistakes.

Thanks for clarifying.

-Mike Gran

Re: A [serious] problem with module integration

[Ludovic Courtès]

Hi!

Panicz Maciej Godek <godek.maciek@gmail.com> skribis:

> If I'd taken a different approach, e.g. if I'd decided to use shared
> libraries instead of having the main application call scm_with_guile,
> the solution would be easy -- I could just define the wrappers around
> the procedures exported by the library.

Yes.  This is called /extending/ Guile, as opposed to /embedding/ it,
and it’s always more fruitful.  See, for instance, the discussion at
<http://twistedmatrix.com/users/glyph/rant/extendit.html>.

Thanks,
Ludo’.

Re: A [serious] problem with module integration

[rixed]

-[ Fri, Nov 16, 2012 at 02:54:50PM +0100, Panicz Maciej Godek ]----
> And now I have a problem: the modules that I wrote make use
> of the symbols defined by my application (using scm_c_define...),
> but they are unavailable outside my application, i.e. for external
> modules.
> 
> The result is that the modules don't work, because certain symbols
> are undefined that are needed to build the necessary definitions.

I have run into a similar problem recently: I'm defining from C the
module 'runtime' where I store all my scm_d_defined symbols and when
compiling a guile module (using "guile-tool compile") the compiler
looks for a file named runtime.scm and of course fails to find one
and stops.

To solve this I had to create a void runtime.scm file defining this
dummy module so that other modules can be compiled. You can see the
result here:

http://github.com/securactive/junkie/blob/master/guile/junkie/runtime.scm

There might be better solutions to this problem but this one is
particularly easy to implement.