Module unloading

Module unloading

[Zeeshan Ali]

Hello fellow Guilers,
    As most of you (who lurk on #guile) know that I've been writing an
xchat-guile plugin lately so people can write xchat plugins in
scheme/guile. The plugin is almost finished now and i am working on
the plugin loading and unloading functionality right now. I thought
that it would be a good idea that the xchat-guile plugins are the same
as guile modules. I know how to load the modules but i also need to
unload them and seems there is no 'straight' way of doing that in
guile, is there?
 
Regards,

Zeeshan Ali


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Neil Jerram]

Zeeshan Ali wrote:
> Hello fellow Guilers,
>     As most of you (who lurk on #guile) know that I've been writing an
> xchat-guile plugin lately so people can write xchat plugins in
> scheme/guile. The plugin is almost finished now and i am working on
> the plugin loading and unloading functionality right now. I thought
> that it would be a good idea that the xchat-guile plugins are the same
> as guile modules. I know how to load the modules but i also need to
> unload them and seems there is no 'straight' way of doing that in
> guile, is there?

There's no way of completely removing (=> undefining) the definitions 
that a module added.  I think it might be possible (in an undocumented 
and unsupported way) to "unuse" a module, though, i.e. to remove a 
plugin module from the module-uses list of your main module.  Is that 
any use to you?

	Neil


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Zeeshan Ali]

Hello,

> There's no way of completely removing (=> undefining) the definitions
> that a module added.

      Hmm... But the GC would take care of that, wouldn't it? If it
doesn'nt then the plugins must do that themselves in their
'xchat-plugin-deinit' procedure, which they must define and export.

>  I think it might be possible (in an undocumented
> and unsupported way) to "unuse" a module, though, i.e. to remove a
> plugin module from the module-uses list of your main module.  Is that
> any use to you?

    I think that is exactly what i need. Maybe there is no way defined
to do this as the GC also takes care of 'unused' modules as they are
also normal guile objects? Thanks for your help.

Regards,

Zeeshan Ali.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Neil Jerram]

Zeeshan Ali wrote:
> 
>>There's no way of completely removing (=> undefining) the definitions
>>that a module added.
> 
>       Hmm... But the GC would take care of that, wouldn't it? If it
> doesn'nt then the plugins must do that themselves in their
> 'xchat-plugin-deinit' procedure, which they must define and export.

It should do if you really can remove all the references to the module, 
but I'm not sure how easy that is; I think modules live (through 
membership of a global list?) even if no one is using them.

>> I think it might be possible (in an undocumented
>>and unsupported way) to "unuse" a module, though, i.e. to remove a
>>plugin module from the module-uses list of your main module.  Is that
>>any use to you?
> 
>     I think that is exactly what i need. Maybe there is no way defined
> to do this as the GC also takes care of 'unused' modules as they are
> also normal guile objects? Thanks for your help.

Well here's what I had in mind:

(use-modules (ossau example-fns))   ; this module exports `fact1'

(define (unuse-named-module name)
   (let ((intf (resolve-interface name)))
     (set-module-uses! (current-module)
		      (delq intf (module-uses (current-module))))))

fact1 => #<procedure fact1 (n)>

(unuse-named-module '(ossau example-fns))

fact1 => ERROR: Unbound variable: fact1

It seems to survive 5 minutes of testing by me, but there could well be 
problems lurking, so please test carefully in your context.

	Neil


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Zeeshan Ali]

Hello,

> It seems to survive 5 minutes of testing by me, but there could well be
> problems lurking, so please test carefully in your context.

   I'll try to test it here as well but i'll have to convert it to an
equivalent C code first.

BTW guys,

      I have been told that the fate of 'undefine' is in doubt. Is
that really true? Here is how i load modules and tell me if there is a
better way of doing the same:

1. load the module using scm_c_use_module ()
2. The module is responsible to define and export a few symbols (e.g
xchat-plugin-version)
3. I get the value of these symbols and
4 undefine these varriables

Regards,

Zeeshan Ali.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Neil Jerram]

Zeeshan Ali wrote:
> Hello,
> 
> 
>>It seems to survive 5 minutes of testing by me, but there could well be
>>problems lurking, so please test carefully in your context.
> 
> 
>    I'll try to test it here as well but i'll have to convert it to an
> equivalent C code first.

Out of interest, why is that?  (It's easy enough to call Scheme-defined 
code from C, isn't it?)

> 
> BTW guys,
> 
>       I have been told that the fate of 'undefine' is in doubt. Is
> that really true? Here is how i load modules and tell me if there is a
> better way of doing the same:
> 
> 1. load the module using scm_c_use_module ()
> 2. The module is responsible to define and export a few symbols (e.g
> xchat-plugin-version)
> 3. I get the value of these symbols and
> 4 undefine these varriables

That sounds more complex and magic than it needs to be, to me - where by 
"complex" I mean your infrastructure code using undefine, and by "magic" 
I mean that it requires plugin authors to follow a non-code-enforced 
convention about the names of the variables.  Why not use a registration 
API instead?  In other words, instead of this:

(define (do-plugin-thing-1 ...) ...)
(define (do-plugin-thing-2 ...) ...)
...

do this:

(let ()
   (define (do-plugin-thing-1 ...) ...)
   (define (do-plugin-thing-2 ...) ...)
   ...
   (register-plugin do-plugin-thing-1
                    do-plugin-thing-2
                    ...))

This way there is no magic, and the visibility of the plugin bindings is 
under the control of the plugin author.  Also, this way you could stay 
pure-R5RS if you like, as you don't really need any features of the 
module system.

Regards,
	Neil


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Zeeshan Ali]

Hello,

> >    I'll try to test it here as well but i'll have to convert it to an
> > equivalent C code first.
> 
> Out of interest, why is that?  (It's easy enough to call Scheme-defined
> code from C, isn't it?)

    Yes! I think i my mind is stuck in the world of C. :(

> That sounds more complex and magic than it needs to be, to me - where by
> "complex" I mean your infrastructure code using undefine, and by "magic"
> I mean that it requires plugin authors to follow a non-code-enforced
> convention about the names of the variables.

   I know my design sucks and that is why i am so desperate to hear
new ideas. Let's hear yours:

>  Why not use a registration
> API instead?  In other words, instead of this:
> 
> (define (do-plugin-thing-1 ...) ...)
> (define (do-plugin-thing-2 ...) ...)
> ...
> 
> do this:
> 
> (let ()
>    (define (do-plugin-thing-1 ...) ...)
>    (define (do-plugin-thing-2 ...) ...)
>    ...
>    (register-plugin do-plugin-thing-1
>                     do-plugin-thing-2
>                     ...))
> 
> This way there is no magic, and the visibility of the plugin bindings is
> under the control of the plugin author.  Also, this way you could stay
> pure-R5RS if you like, as you don't really need any features of the
> module system.

   Sounds pretty cool. I'll give it a try. Thanks a lot.

Regards,

Zeeshan Ali


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Rob Browning]

Neil Jerram <neil@ossau.uklinux.net> writes:

> This way there is no magic, and the visibility of the plugin bindings
> is under the control of the plugin author.  Also, this way you could
> stay pure-R5RS if you like, as you don't really need any features of
> the module system.

This made me wonder -- what happens if you dynamically load (dlopen) a
lib that depends on libguile, and then you later dlclose that lib?
What happens to libguile?  Presumably it will be unloaded too.  Does
Guile tolerate that?

If so, does Guile tolerate being re-loaded?  For example if you then
re-dlopen the lib that depends on libguile and start using it?

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Zeeshan Ali]

Hello again,

> >  Why not use a registration
> > API instead?  In other words, instead of this:
> >
> > (define (do-plugin-thing-1 ...) ...)
> > (define (do-plugin-thing-2 ...) ...)
> > ...
> >
> > do this:
> >
> > (let ()
> >    (define (do-plugin-thing-1 ...) ...)
> >    (define (do-plugin-thing-2 ...) ...)
> >    ...
> >    (register-plugin do-plugin-thing-1
> >                     do-plugin-thing-2
> >                     ...))

    This is the code of the plugin right? and register-plugin is a
function provided by me for the plugins to call? But i'll still need
an entry point inside the plugin: a procedure that i'll call from the
module as soon as i load it. So this approach doesn't seem to be of a
big advantage.

> > This way there is no magic, and the visibility of the plugin bindings is
> > under the control of the plugin author.

    I think I have figured-out a way out of this visibility problem:

(define amodule (resolve-module '(amodule)))
((variable-ref (module-local-variable amodule 'xchat-plugin-init)))

     This code seems to work and there is no need for the plugin to
export it's objects anymore.

>  Also, this way you could stay
> > pure-R5RS if you like, as you don't really need any features of the
> > module system.

   Wait a sec? Are you really thinking what i think you are thinking:
use the simple 'load' procedure combined with your suggested
registeration API instead of using the module system of guile?
    
Regards,

Zeeshan Ali


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Zeeshan Ali]

Hello Neil,
   I must remind you that i am anxiously waiting for more input on
this mater from you. Moreover, i tried to implement what you
recommended but i think i am in a problem: Since the loaded module
doesn't have access to binding of it's parent module (the module that
'uses' it), it is unable to access the 'xchat-register-plugin'
procedure that is defined in the parent module. Do you have a solution
for this in mind as well, or i took you all wrong.

Regards,

Zeeshan Ali.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Neil Jerram]

Zeeshan Ali wrote:
> Hello again,
> 
> 
>>> Why not use a registration
>>>API instead?  In other words, instead of this:
>>>
>>>(define (do-plugin-thing-1 ...) ...)
>>>(define (do-plugin-thing-2 ...) ...)
>>>...
>>>
>>>do this:
>>>
>>>(let ()
>>>   (define (do-plugin-thing-1 ...) ...)
>>>   (define (do-plugin-thing-2 ...) ...)
>>>   ...
>>>   (register-plugin do-plugin-thing-1
>>>                    do-plugin-thing-2
>>>                    ...))
> 
> 
>     This is the code of the plugin right? and register-plugin is a
> function provided by me for the plugins to call?

Yes, exactly.

> But i'll still need
> an entry point inside the plugin: a procedure that i'll call from the
> module as soon as i load it. So this approach doesn't seem to be of a
> big advantage.

I'm not sure I understand.  Do you need to call this procedure even
before the plugin code calls register-plugin (which would happen
automatically when the plugin code file is loaded)?  It's difficult to
see how there could be a requirement for this, given that it is
impossible to separate the loading and the register-plugin call.

If it is OK to call this procedure after register-plugin has been
called, then the procedure to call could be another argument to
register-plugin:

     (register-plugin init-proc
                      do-plugin-thing-1
                      do-plugin-thing-2
                      ...))

>     I think I have figured-out a way out of this visibility problem:
> 
> (define amodule (resolve-module '(amodule)))
> ((variable-ref (module-local-variable amodule 'xchat-plugin-init)))
> 
>      This code seems to work and there is no need for the plugin to
> export it's objects anymore.

Yes, this would be fine.

>> Also, this way you could stay
>>
>>>pure-R5RS if you like, as you don't really need any features of the
>>>module system.
> 
> 
>    Wait a sec? Are you really thinking what i think you are thinking:
> use the simple 'load' procedure combined with your suggested
> registeration API instead of using the module system of guile?

Yes.  It may indicate a deficiency in the module system, but I don't
think that modules are currently especially well aligned with the
requirements of acting as a plugin.  In my view modules are mostly
useful (and work very well) for organizing the code of a non-trivial
application (and with a thought of identifying and grouping together the
code that might be reusable by other applications).

To map this onto your program, I think it would be absolutely fine to
use just load to load plugins - in fact this would work well with a
standard File Selector GUI: "Browse for plugin ...", whereas for a
module-based invocation you'd need to invent a new dialog.  Please note 
that, even when a file is loaded using plain "load":

1. The plugin author can still choose to make their file a module, if 
that is of use to them, by beginning it with a (define-module ...) form.

2. The plugin author can still choose to use modules to organize their 
plugin code internally, by splitting out some of the code into modules 
and using (use-modules ...) from the main file to load them.

3. The plugin author can still use third-party modules, using 
(use-modules ...).

Regards,
	Neil


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Neil Jerram]

Zeeshan Ali wrote:
> Hello Neil,
>    I must remind you that i am anxiously waiting for more input on
> this mater from you. Moreover, i tried to implement what you
> recommended but i think i am in a problem: Since the loaded module
> doesn't have access to binding of it's parent module (the module that
> 'uses' it), it is unable to access the 'xchat-register-plugin'
> procedure that is defined in the parent module. Do you have a solution
> for this in mind as well, or i took you all wrong.

Yes.  This is covered in general terms by the comments in my earlier 
email today, but for the specific problem you describe, all you need to 
do is:

- export `xchat-register-plugin' from your main infrastructure module, 
which let's say is (xchat main)

- document that plugin authors need to write `(use-modules (xchat 
main))' at the start of their plugin code.

Hope this makes sense ...

	Neil


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Neil Jerram]

Rob Browning wrote:
> 
> This made me wonder -- what happens if you dynamically load (dlopen) a
> lib that depends on libguile, and then you later dlclose that lib?
> What happens to libguile?  Presumably it will be unloaded too.  Does
> Guile tolerate that?
> 
> If so, does Guile tolerate being re-loaded?  For example if you then
> re-dlopen the lib that depends on libguile and start using it?

What happens "inside" the library when it is dlopened?  I thought 
nothing, so I doubt that loading and unloading per se would be a problem.

If a single program called scm_init_guile() twice without the library 
being unloaded and reloaded first ... I doubt the code would handle 
that, but I'd say this is incorrect use of the API.

If a single program calls scm_init_guile(), later unloads libguile, the 
reloads libguile and calls scm_init_guile() ... I'd say this _ought_ to 
work, and it probably mostly does as long as reloading reinitializes 
libguile's global data.  Since there is no scm_term_guile(), though, I 
suspect we would also have leaked a lot of memory on the heap.

	Neil


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Rob Browning]

Neil Jerram <neil@ossau.uklinux.net> writes:

> What happens "inside" the library when it is dlopened?  I thought
> nothing, so I doubt that loading and unloading per se would be a
> problem.
>
> If a single program called scm_init_guile() twice without the library
> being unloaded and reloaded first ... I doubt the code would handle
> that, but I'd say this is incorrect use of the API.
>
> If a single program calls scm_init_guile(), later unloads libguile,
> the reloads libguile and calls scm_init_guile() ... I'd say this
> _ought_ to work, and it probably mostly does as long as reloading
> reinitializes libguile's global data.  Since there is no
> scm_term_guile(), though, I suspect we would also have leaked a lot of
> memory on the heap.

I was also concerned about any pending signals, callbacks registered
with other libraries, or anything else that has been set up by code
executed from guile.

I was concerned that there may be trouble unless there's some way of
enumerating all of these things and undoing them, and that seems like
it might be difficult in the general case.

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org; previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Neil Jerram]

Rob Browning wrote:
> 
> I was also concerned about any pending signals, callbacks registered
> with other libraries, or anything else that has been set up by code
> executed from guile.
> 
> I was concerned that there may be trouble unless there's some way of
> enumerating all of these things and undoing them, and that seems like
> it might be difficult in the general case.

Yes, good points.  This sounds pretty hard, so I guess it will be 
addressed only when someone has a real need.

	Neil


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Zeeshan Ali]

Hello Neil,
    Glad to hear from you again.

> 1. The plugin author can still choose to make their file a
> module, if that is of use to them, by beginning it with a
> (define-module ...) form.
> 
> 2. The plugin author can still choose to use modules to
> organize their plugin code internally, by splitting out some of
> the code into modules and using (use-modules ...) from the
> main file to load them.
> 
> 3. The plugin author can still use third-party modules, using
> (use-modules ...).

4. It will cause nonspace polution, which is what i won't want. :(

Regards,

Zeeshan Ali


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Module unloading

[Zeeshan Ali]

Hello,

> Yes.  This is covered in general terms by the comments in my earlier
> email today, but for the specific problem you describe, all you need to
> do is:
> 
> - export `xchat-register-plugin' from your main infrastructure module,
> which let's say is (xchat main)
> 
> - document that plugin authors need to write `(use-modules (xchat
> main))' at the start of their plugin code.

   Yes, Rob suggested this method on IRC this evening, but it seems to
further complicate the maters as i was trying to unref one module
reference and this way i'll get two to worry about.

    One thing that i found out is that module can't really be
unloaded. E.g if i load a module (either using use-modules or
resolve-module), unload/unuse it (using the code you showed me), edit
the module and then load it again, i don't get my changes; instead the
same old version of module get's loaded, which means the module didn't
get unloaded.

Regards,

Zeeshan Ali


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user