[PATCH] Doc: protecting procedure->pointer pointers from GC

[PATCH] Doc: protecting procedure->pointer pointers from GC

[Neil Jerram]

[-- Attachment #1: Type: text/plain, Size: 192 bytes --]

Following debugging of a strange issue where oFono appeared to be
sending a D-Bus signal from the wrong object, but I eventually realised
that the problem was in my own code...

        Neil


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Doc-protecting-procedure-pointer-pointers-from-GC.patch --]
[-- Type: text/x-diff, Size: 2392 bytes --]

From 02d80ad14c3bb8f89d7ba807dac658be6dd06df1 Mon Sep 17 00:00:00 2001
From: Neil Jerram <neil@ossau.homelinux.net>
Date: Mon, 30 Jan 2012 21:12:37 +0000
Subject: [PATCH] Doc: protecting procedure->pointer pointers from GC

* doc/ref/api-foreign.texi (Dynamic FFI): New text covering when pointers
  need to be retained.
---
 doc/ref/api-foreign.texi |   33 +++++++++++++++++++++++++++++++++
 1 files changed, 33 insertions(+), 0 deletions(-)

diff --git a/doc/ref/api-foreign.texi b/doc/ref/api-foreign.texi
index 6ece7f8..19d6b29 100644
--- a/doc/ref/api-foreign.texi
+++ b/doc/ref/api-foreign.texi
@@ -961,6 +961,39 @@ function can be made accessible to Scheme (@pxref{Array Sort Function,
 
 And voil@`a!
 
+If you use @code{procedure->pointer} to pass a Scheme procedure as a
+callback for a C library to invoke asynchronously later --- for example,
+as a handler for a D-Bus signal --- you must somehow keep a reference to
+the pointer that @code{procedure->pointer} returns, for as long as the C
+library might invoke the callback.  Otherwise the pointer can be
+prematurely garbage collected, leading to unsafe and arbitrary behaviour
+if the now-invalid callback is called.
+
+This example, which registers a Scheme procedure as a handler for a
+GObject signal, uses @code{make-object-property} to do that:
+
+@example
+(define gobject-signal-handlers (make-object-property))
+
+(define (gobject-connect object signal proc)
+  (let ((ptr (procedure->pointer void
+                                 (lambda (...C callback args...)
+                                   (proc ...Scheme args...))
+                                 (list ...C callback types...))))
+    ;; Protect the pointer from being prematurely collected.
+    (set! (gobject-signal-handlers object)
+          (acons signal
+                 ptr
+                 (or (gobject-signal-handlers object) '())))
+    ;; Register the handler.
+    (g_signal_connect_data object
+                           (string->pointer signal)
+                           ptr
+                           %null-pointer
+                           %null-pointer
+                           0)))
+@end example
+ 
 Note that @code{procedure->pointer} is not supported (and not defined)
 on a few exotic architectures.  Thus, user code may need to check
 @code{(defined? 'procedure->pointer)}.  Nevertheless, it is available on
-- 
1.7.8.3

Re: [PATCH] Doc: protecting procedure->pointer pointers from GC

[Andy Wingo]

On Mon 30 Jan 2012 22:32, Neil Jerram <neil@ossau.homelinux.net> writes:

> Following debugging of a strange issue where oFono appeared to be
> sending a D-Bus signal from the wrong object, but I eventually realised
> that the problem was in my own code...

The docs are good, but the example is irritating ;)  Is there a reason
you are unable to use guile-gnome for the object wrapping?  You can use
the FFI to call scm_from_gtype_instance or whatever it's called, after
loading (gnome gobject).

Andy
-- 
http://wingolog.org/

Re: [PATCH] Doc: protecting procedure->pointer pointers from GC

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

> On Mon 30 Jan 2012 22:32, Neil Jerram <neil@ossau.homelinux.net> writes:
>
>> Following debugging of a strange issue where oFono appeared to be
>> sending a D-Bus signal from the wrong object, but I eventually realised
>> that the problem was in my own code...
>
> The docs are good, but the example is irritating ;)  Is there a reason
> you are unable to use guile-gnome for the object wrapping?  You can use
> the FFI to call scm_from_gtype_instance or whatever it's called, after
> loading (gnome gobject).

Because Ludo and you have made it so easy to be productive with just
(system foreign) !

But your implication is right, it is a more interesting and complex
question than just that.

My current project is a phone application for the GTA04, which is an ARM
device with most of the promising software stacks being Debian-based.
Hence, aside from Guile itself - where I still have some fixing and
upstreaming to do - I'd ideally like not to take on any dependencies
other than things that are already available for ARM in Debian unstable.
I didn't actually go ahead and check beforehand whether a Guile
2-compatible guile-gnome was available, but I know that guile-2.0 itself
has only recently landed in Debian, so I guess I assumed a compatible
guile-gnome was very unlikely to be available yet.  (And, on checking, I
believe that's still correct.)

So then it would be a matter of building and packaging guile-gnome for
my device.  I remember that from Nokia 770 days...  It's quite doable,
but it takes time (and requires slib and g-wrap first, IIRC), and
compared with that it's way easier, and doesn't impede the hack, just to
grab the entry points that I need using pointer->procedure.

Another relevant factor is that I need relatively few guile-gnome entry
points because I'm only using gobject/glib to get gdbus, and the UI is
built with Edje, not Gtk/Gnome.  In fact it's a strange (but IMO fun)
hotchpotch:

- Edje/Evas for the UI (using 17 pointer->procedures)

- GDBus to access the oFono D-Bus API (using pointer->procedures: 3
  for basic GObject stuff, 11 for GVariant handling, and 2 for GDBus).

and I will probably add in more stuff for audio routing (PulseAudio?)
and accessing contact information (whatever the latest evolution of
Evolution Data Services is).

But even if I needed a lot more guile-gnome APIs, I'd (at least
initially) be tempted to grab a .defs file and try to hack something
based on that, rather than compiling C code...

I think the key benefit is that the (system foreign) approach is so
immediate.  I just add whatever API I need there, and the hack
continues...

My only reservation - and quite a big one - is that it's a bit like
going back in time to when compilers had no type checking for pointers.
In that sense it is rather fast and loose, compared to C programming.

But overall I love it, and I'd suggest that an alternative question (to
the one you asked above) would be why do you think we need bindings
anymore?

Well I hope that's of interest, and I'm sure you'll have interesting
counter-thoughts...  In case you or anyone wants to look at the code,
I've pushed it to http://git.savannah.gnu.org/cgit/ossaulib.git/.  The
main script is at scripts/phone.

        Neil

Re: [PATCH] Doc: protecting procedure->pointer pointers from GC

[Ludovic Courtès]

Hi Neil,

Neil Jerram <neil@ossau.homelinux.net> skribis:

> Another relevant factor is that I need relatively few guile-gnome entry
> points because I'm only using gobject/glib to get gdbus, and the UI is
> built with Edje, not Gtk/Gnome.  In fact it's a strange (but IMO fun)
> hotchpotch:
>
> - Edje/Evas for the UI (using 17 pointer->procedures)
>
> - GDBus to access the oFono D-Bus API (using pointer->procedures: 3
>   for basic GObject stuff, 11 for GVariant handling, and 2 for GDBus).
>
> and I will probably add in more stuff for audio routing (PulseAudio?)
> and accessing contact information (whatever the latest evolution of
> Evolution Data Services is).

Sounds like a lot of fun.  :-)

> I think the key benefit is that the (system foreign) approach is so
> immediate.  I just add whatever API I need there, and the hack
> continues...

Yes, that’s really neat.

> My only reservation - and quite a big one - is that it's a bit like
> going back in time to when compilers had no type checking for pointers.
> In that sense it is rather fast and loose, compared to C programming.

Indeed.  Plus, you sometimes have to be careful about memory management,
specifically making sure the lifetime of C objects matches that of their
companion Scheme objects.

> But overall I love it, and I'd suggest that an alternative question (to
> the one you asked above) would be why do you think we need bindings
> anymore?

Well, these are still bindings.  :-) It can be written more quickly, but
you still have to do a bit of polishing to make the API more Schemey,
write a test suite (esp. to mitigate the “loose” aspect of the FFI), etc.

> Well I hope that's of interest, and I'm sure you'll have interesting
> counter-thoughts...  In case you or anyone wants to look at the code,
> I've pushed it to http://git.savannah.gnu.org/cgit/ossaulib.git/.  The
> main script is at scripts/phone.

Looks great!  I had a glance at the E17 bindings, and I see that the API
uses raw pointer objects; using “wrapped pointer types” would provide
type checking, and a nicer UI if you add printers.  :-)

Cheers,
Ludo’.

Re: [PATCH] Doc: protecting procedure->pointer pointers from GC

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> Looks great!  I had a glance at the E17 bindings, and I see that the API
> uses raw pointer objects; using “wrapped pointer types” would provide
> type checking, and a nicer UI if you add printers.  :-)

That's an excellent point; yes, I'll do that.  Using wrapped pointer
types, it should then be the case that no naked pointer values ever
go in or out from the module that provides the binding - which feels
like a nice property. 

        Neil