* on coding a portable R6RS package supporting Guile and its FFI
@ 2013-01-17 10:53 Marco Maggi
2013-01-27 15:08 ` Ludovic Courtès
2013-02-05 17:38 ` Andy Wingo
0 siblings, 2 replies; 12+ messages in thread
From: Marco Maggi @ 2013-01-17 10:53 UTC (permalink / raw)
To: guile-user
Ciao,
in my latest package[1] I am using Guile 2.0.7 and its FFI
along with Mosh Scheme, Racket, Sagittarius Scheme, Vicare
Scheme, Ypsilon Scheme. I have the following observations:
* Guile does not come with the very simple SRFI 78:
lightweight testing. I had to include it[2][3].
* It appears that there is no facility to handle "output
arguments" from C functions; I mean the cases where a C
function accepts as argument a pointer to variable that
will be filled with some computed value. I am using a
WITH-LOCAL-STORAGE[4] macro which is maybe ugly, but
works.
Such arguments are common, and represent a nuisance to
handle. Racket has a sophisticated interface, complicated
to use when writing adapter code. Something simpler but
built in would be useful (this is the sort of thing a user
does not want to think about: it should be an already
solved problem).
* Whenever a callout to C accepts a pointer argument: a
bytevector argument is rejected. Is this not a useless
complication?
One can work around it by explicitly using
BYTEVECTOR->POINTER, so everything is ready in Guile. The
other Scheme implementations using a non-compacting
garbage collector already support this feature and I find
it truly convenient.
* There are no raw memory getters and setters[5]? The fact
that it is possible to create a wrapping bytevector is a
plus for sure, but I find it inconvenient to allocate a
bytevector when I do not need it (and raw getters and
setters are really small functions).
* The limit of 10 arguments for callouts to C is annoying.
It forced me to exclude some SOFA functions resulting in
amputated Guile support.
TIA
[1] <http://github.com/marcomaggi/r6rs-sofa/>
[2] <http://github.com/marcomaggi/r6rs-sofa/tree/master/lib/guile/srfi>
[3] <http://github.com/marcomaggi/vicare/blob/devel/doc/srfi-lightweight-testing.texi>
[4] <http://github.com/marcomaggi/r6rs-sofa/blob/master/lib/sofa/compat.guile.sls>
[5] <http://marcomaggi.github.com/docs/vicare.html/iklib-pointers.html>
--
Marco Maggi
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-01-17 10:53 on coding a portable R6RS package supporting Guile and its FFI Marco Maggi
@ 2013-01-27 15:08 ` Ludovic Courtès
2013-01-28 3:00 ` Daniel Hartwig
2013-02-05 17:38 ` Andy Wingo
1 sibling, 1 reply; 12+ messages in thread
From: Ludovic Courtès @ 2013-01-27 15:08 UTC (permalink / raw)
To: guile-user
Hi Marco,
Sorry for the delay.
Marco Maggi <marco.maggi-ipsu@poste.it> skribis:
> * Guile does not come with the very simple SRFI 78:
> lightweight testing. I had to include it[2][3].
Patches welcome. :-)
> * It appears that there is no facility to handle "output
> arguments" from C functions; I mean the cases where a C
> function accepts as argument a pointer to variable that
> will be filled with some computed value. I am using a
> WITH-LOCAL-STORAGE[4] macro which is maybe ugly, but
> works.
I typically roll my own allocation and dereference routines as well,
such as ‘make-int-pointer’ and ‘dereference-int’ at:
http://git.savannah.gnu.org/cgit/libchop.git/tree/guile2/chop/internal.scm#n255
Perhaps adding them to (system foreign) would help?
> Such arguments are common, and represent a nuisance to
> handle. Racket has a sophisticated interface, complicated
> to use when writing adapter code. Something simpler but
> built in would be useful (this is the sort of thing a user
> does not want to think about: it should be an already
> solved problem).
I realize it would be good to have facilities already available for
this.
However, I’m not familiar with Racket’s FFI, and it’s not clear to me
what a good generic API would be.
For instance, one could imagine a layer on top of ‘pointer->procedure’
that would allow to specify whether pointer arguments really are output
arguments. But then, you’d also have to allow programmers to tell how
output arguments are to be allocated (“pointerless” or not, ‘malloc’,
etc.), who owns them, what their life cycle is, etc.
All in all, it’s always seemed easier to me to do that manually, with
helpers specifically adapted to the C library I write bindings for.
WDYT?
> * Whenever a callout to C accepts a pointer argument: a
> bytevector argument is rejected. Is this not a useless
> complication?
>
> One can work around it by explicitly using
> BYTEVECTOR->POINTER, so everything is ready in Guile. The
> other Scheme implementations using a non-compacting
> garbage collector already support this feature and I find
> it truly convenient.
Well, the ‘pointer’ type is useful, because it’s inherently a more
low-level representation than bytevectors.
That said, the FFI call could implicitly convert bytevectors to
pointers. However, I generally prefer avoiding implicit type
conversions like these, for clarify.
Thoughts?
> * There are no raw memory getters and setters[5]?
There’s only ‘dereference-pointer’ now, but I agree we could add more of
these, as well as pointer arithmetic operators.
> * The limit of 10 arguments for callouts to C is annoying.
> It forced me to exclude some SOFA functions resulting in
> amputated Guile support.
Agreed.
Would you like to propose a patch in some of these areas?
Thanks for your feedback,
Ludo’.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-01-27 15:08 ` Ludovic Courtès
@ 2013-01-28 3:00 ` Daniel Hartwig
0 siblings, 0 replies; 12+ messages in thread
From: Daniel Hartwig @ 2013-01-28 3:00 UTC (permalink / raw)
To: guile-user
On 27 January 2013 23:08, Ludovic Courtès <ludo@gnu.org> wrote:
> Hi Marco,
>
> Sorry for the delay.
>
> Marco Maggi <marco.maggi-ipsu@poste.it> skribis:
>> * It appears that there is no facility to handle "output
>> arguments" from C functions; I mean the cases where a C
>> function accepts as argument a pointer to variable that
>> will be filled with some computed value. I am using a
>> WITH-LOCAL-STORAGE[4] macro which is maybe ugly, but
>> works.
>
> I typically roll my own allocation and dereference routines as well,
> such as ‘make-int-pointer’ and ‘dereference-int’ at:
Likewise.
>
> http://git.savannah.gnu.org/cgit/libchop.git/tree/guile2/chop/internal.scm#n255
>
> Perhaps adding them to (system foreign) would help?
Yes, in these generic cases.
>
>> Such arguments are common, and represent a nuisance to
>> handle. Racket has a sophisticated interface, complicated
>> to use when writing adapter code. Something simpler but
>> built in would be useful
Pointer arguments are common, yes, but they have a variety of usage
patterns across different APIs. This is evident in the complexity of
the Racket API for dealing with this.
In particular, the questions of who (caller or callee) allocates the
memory and who deallocates the memory complicate the handling. In
some APIs, the party in control of deallocation can change.
>> (this is the sort of thing a user
>> does not want to think about: it should be an already
>> solved problem).
Mapping any pointer interface requires some user thought to consider
its unique behaviour. It is risky to abstract some of that process
away from the programmer, even if only to accomodate a single, “most
common” usage.
IMO the best that could be done without complicating the FFI is to
bring in a small set of generic helpers such as Ludo's
make-int-pointer, etc..
> …
>
> All in all, it’s always seemed easier to me to do that manually, with
> helpers specifically adapted to the C library I write bindings for.
>
> WDYT?
Right. After considering the requirements for any given interface it
is only a small amount of work to create the appropriate helpers.
Being specific to the API under consideration, such helpers can
present an optimal interface.
>
>> * Whenever a callout to C accepts a pointer argument: a
>> bytevector argument is rejected. Is this not a useless
>> complication?
No, it is sensible type checking: bytevector is not pointer.
>>
>> One can work around it by explicitly using
>> BYTEVECTOR->POINTER, so everything is ready in Guile. The
>> other Scheme implementations using a non-compacting
>> garbage collector already support this feature and I find
>> it truly convenient.
>
> Well, the ‘pointer’ type is useful, because it’s inherently a more
> low-level representation than bytevectors.
>
> That said, the FFI call could implicitly convert bytevectors to
> pointers. However, I generally prefer avoiding implicit type
> conversions like these, for clarify.
>
> Thoughts?
This implicit conversion must be avoided. Instead, how about
supporting supporting SRFI-4 types (s16vector, etc.) or typed arrays
in the FFI. This would only cover some use cases, where the caller
controls the memory and the foreign argument is a /typed/ pointer.
This much could be implemented without seriously complicating the FFI.
For “void *” it is still required to explicitly pass a pointer object,
though it is not difficult to create helpers appropriate to the
/particular/ interface.
For callee allocated or deallocated memory it is still required to
interface using pointer objects.
SRFI-4 has the advantage that slots are always stored contiguously in
memory. Typed array slots can be stored non-contiguously, and any
such array would have to be rejected as “array-contents” does.
Regards
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-01-17 10:53 on coding a portable R6RS package supporting Guile and its FFI Marco Maggi
2013-01-27 15:08 ` Ludovic Courtès
@ 2013-02-05 17:38 ` Andy Wingo
2013-02-05 19:29 ` Noah Lavine
2013-02-05 22:31 ` Ludovic Courtès
1 sibling, 2 replies; 12+ messages in thread
From: Andy Wingo @ 2013-02-05 17:38 UTC (permalink / raw)
To: marco.maggi-ipsu; +Cc: guile-user
On Thu 17 Jan 2013 11:53, Marco Maggi <marco.maggi-ipsu@poste.it> writes:
> * Whenever a callout to C accepts a pointer argument: a
> bytevector argument is rejected. Is this not a useless
> complication?
>
> One can work around it by explicitly using
> BYTEVECTOR->POINTER, so everything is ready in Guile. The
> other Scheme implementations using a non-compacting
> garbage collector already support this feature and I find
> it truly convenient.
I agree that we would do well to allow bytevectors wherever an argument
type is a pointer. It avoids an allocation in a very common case
without changing the expressiveness of the interface or adding
significant additional overhead.
Ludovic sounded like he could go either way on this point. Daniel
brought up the argument that "a bytevector is not a pointer"; but for
many purposes it's not a terrible thing to think of it that way. IMHO.
Andy
--
http://wingolog.org/
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-02-05 17:38 ` Andy Wingo
@ 2013-02-05 19:29 ` Noah Lavine
2013-02-06 8:36 ` Andy Wingo
2013-02-05 22:31 ` Ludovic Courtès
1 sibling, 1 reply; 12+ messages in thread
From: Noah Lavine @ 2013-02-05 19:29 UTC (permalink / raw)
To: Andy Wingo; +Cc: Guile Mailing List
[-- Attachment #1: Type: text/plain, Size: 760 bytes --]
Hello,
That does sound very useful. The only thing that makes me worried is
On Tue, Feb 5, 2013 at 12:38 PM, Andy Wingo <wingo@pobox.com> wrote:
> On Thu 17 Jan 2013 11:53, Marco Maggi <marco.maggi-ipsu@poste.it> writes:
> > The other Scheme implementations using a non-compacting
> > garbage collector
>
Do we want to guarantee that our garbage collector will always be
non-compacting, or force major interface changes if it is? (I realize we've
already taken out smob mark procedures, but this is in some ways even
harder to add back.)
I'm not sure this is a big deal, but maybe we should consider what would
happen in the future if we did want to change. Using bytevector->pointer is
an acceptable cost to pay for flexibility, IMHO.
Noah
[-- Attachment #2: Type: text/html, Size: 1340 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-02-05 17:38 ` Andy Wingo
2013-02-05 19:29 ` Noah Lavine
@ 2013-02-05 22:31 ` Ludovic Courtès
2013-02-06 10:57 ` Andy Wingo
1 sibling, 1 reply; 12+ messages in thread
From: Ludovic Courtès @ 2013-02-05 22:31 UTC (permalink / raw)
To: guile-user
Andy Wingo <wingo@pobox.com> skribis:
> I agree that we would do well to allow bytevectors wherever an argument
> type is a pointer. It avoids an allocation in a very common case
> without changing the expressiveness of the interface or adding
> significant additional overhead.
I find it somewhat inelegant, though.
This would be less of a problem if we had “immediate pointers” [0].
Would the retagging in 2.1 allow this?
(At FOSDEM, Luca Saiu rightfully noted that we could use 4-bit tags
instead of 3-bit tags on 64-bit arches, which would give us the needed
room here.)
Ludo’.
[0] http://lists.gnu.org/archive/html/guile-devel/2011-01/msg00159.html
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-02-05 19:29 ` Noah Lavine
@ 2013-02-06 8:36 ` Andy Wingo
2013-02-06 13:25 ` Noah Lavine
0 siblings, 1 reply; 12+ messages in thread
From: Andy Wingo @ 2013-02-06 8:36 UTC (permalink / raw)
To: Noah Lavine; +Cc: Guile Mailing List
On Tue 05 Feb 2013 20:29, Noah Lavine <noah.b.lavine@gmail.com> writes:
> On Tue, Feb 5, 2013 at 12:38 PM, Andy Wingo <wingo@pobox.com> wrote:
>
> On Thu 17 Jan 2013 11:53, Marco Maggi <marco.maggi-ipsu@poste.it>
> writes:
> > The other Scheme implementations using a non-compacting
> > garbage collector
>
> Do we want to guarantee that our garbage collector will always be
> non-compacting, or force major interface changes if it is? (I realize
> we've already taken out smob mark procedures, but this is in some ways
> even harder to add back.)
Smob mark procedures are still supported. Changing the GC would be
quite difficult -- we would need to progress to the point that we had
significantly less C. That would be years away if it comes. Anyway, we
could have a compacting collector and still allow fixed-position
objects:
http://www.scheme.com/csug8/smgmt.html#./smgmt:h3
Andy
--
http://wingolog.org/
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-02-05 22:31 ` Ludovic Courtès
@ 2013-02-06 10:57 ` Andy Wingo
2013-02-08 14:10 ` Ludovic Courtès
0 siblings, 1 reply; 12+ messages in thread
From: Andy Wingo @ 2013-02-06 10:57 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guile-user
On Tue 05 Feb 2013 23:31, ludo@gnu.org (Ludovic Courtès) writes:
> This would be less of a problem if we had “immediate pointers” [0].
> Would the retagging in 2.1 allow this?
The tc3 itag allocation in wip-retagging is:
* tc3:
* 000: A heap object with a tag word on the heap
* 001: Some other immediate
* 010: A pair
* 011: Small integer (odd)
* 100: (Unallocated tc3.)
* 101: (Unallocated tc3.)
* 110: A struct
* 111: Small integer (even)
And SCM_IMP is:
#define SCM_IMP(x) (1 & SCM_UNPACK (x))
So perhaps 101 could be an immediate pointer. Dunno. Pretty nasty
stuff!
> (At FOSDEM, Luca Saiu rightfully noted that we could use 4-bit tags
> instead of 3-bit tags on 64-bit arches, which would give us the needed
> room here.)
GC_malloc still returns eight-byte aligned memory hunks on 64-bit, no?
Andy
--
http://wingolog.org/
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-02-06 8:36 ` Andy Wingo
@ 2013-02-06 13:25 ` Noah Lavine
0 siblings, 0 replies; 12+ messages in thread
From: Noah Lavine @ 2013-02-06 13:25 UTC (permalink / raw)
To: Andy Wingo; +Cc: Guile Mailing List
[-- Attachment #1: Type: text/plain, Size: 747 bytes --]
On Wed, Feb 6, 2013 at 3:36 AM, Andy Wingo <wingo@pobox.com> wrote:
> On Tue 05 Feb 2013 20:29, Noah Lavine <noah.b.lavine@gmail.com> writes:
>
> Smob mark procedures are still supported. Changing the GC would be
> quite difficult -- we would need to progress to the point that we had
> significantly less C. That would be years away if it comes. Anyway, we
> could have a compacting collector and still allow fixed-position
> objects:
>
I see. I hadn't thought about what a big project it would be. I was
imagining bytevector->pointer as a procedure that just set some
"fixed-position bit" on its object, so there wasn't any allocation.
However, that would fit weirdly with the "->" convention. It's probably not
worth worrying about.
Noah
[-- Attachment #2: Type: text/html, Size: 1247 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-02-06 10:57 ` Andy Wingo
@ 2013-02-08 14:10 ` Ludovic Courtès
2013-02-08 14:37 ` Andy Wingo
0 siblings, 1 reply; 12+ messages in thread
From: Ludovic Courtès @ 2013-02-08 14:10 UTC (permalink / raw)
To: Andy Wingo; +Cc: guile-user
Andy Wingo <wingo@pobox.com> skribis:
> On Tue 05 Feb 2013 23:31, ludo@gnu.org (Ludovic Courtès) writes:
>
>> This would be less of a problem if we had “immediate pointers” [0].
>> Would the retagging in 2.1 allow this?
>
> The tc3 itag allocation in wip-retagging is:
>
> * tc3:
> * 000: A heap object with a tag word on the heap
> * 001: Some other immediate
> * 010: A pair
> * 011: Small integer (odd)
> * 100: (Unallocated tc3.)
> * 101: (Unallocated tc3.)
> * 110: A struct
> * 111: Small integer (even)
>
> And SCM_IMP is:
>
> #define SCM_IMP(x) (1 & SCM_UNPACK (x))
>
> So perhaps 101 could be an immediate pointer. Dunno. Pretty nasty
> stuff!
Would be cool, I think. Maybe we should make an auction for unused tc3
tags? :-)
>> (At FOSDEM, Luca Saiu rightfully noted that we could use 4-bit tags
>> instead of 3-bit tags on 64-bit arches, which would give us the needed
>> room here.)
>
> GC_malloc still returns eight-byte aligned memory hunks on 64-bit, no?
Oh right:
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (use-modules (system foreign))
scheme@(guile-user)> (pointer->procedure '* (dynamic-func "GC_malloc" (dynamic-link)) (list size_t))
$2 = #<procedure 1b60ac0 (_)>
scheme@(guile-user)> ,use (srfi srfi-1)
scheme@(guile-user)> ,use (srfi srfi-26)
scheme@(guile-user)> (unfold (cut > <> 100) (lambda (i) (logand (pointer-address ($2 (* i 2)) ) #x15)) 1+ 0)
$7 = (0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 16 0 0 16 0 0 16 0 0 0 0 0 0 0 0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 0 16 0 16 0)
--8<---------------cut here---------------end--------------->8---
If needed, we could just round up all the addresses returned.
And with 16 cases, we could even put single-precision floats in a word,
chars, and who knows what!
The downside is that we’d have to maintain two tag systems.
Ludo’.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-02-08 14:10 ` Ludovic Courtès
@ 2013-02-08 14:37 ` Andy Wingo
2013-02-08 15:52 ` Ludovic Courtès
0 siblings, 1 reply; 12+ messages in thread
From: Andy Wingo @ 2013-02-08 14:37 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guile-user
Hi :)
On Fri 08 Feb 2013 15:10, ludo@gnu.org (Ludovic Courtès) writes:
>> GC_malloc still returns eight-byte aligned memory hunks on 64-bit, no?
>
> Oh right:
Does it? I couldn't tell from your test. I'm not a native unfolder :)
> And with 16 cases, we could even put single-precision floats in a word,
> chars, and who knows what!
Chars are already immediate :)
Cheers,
Andy
--
http://wingolog.org/
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI
2013-02-08 14:37 ` Andy Wingo
@ 2013-02-08 15:52 ` Ludovic Courtès
0 siblings, 0 replies; 12+ messages in thread
From: Ludovic Courtès @ 2013-02-08 15:52 UTC (permalink / raw)
To: Andy Wingo; +Cc: guile-user
Andy Wingo <wingo@pobox.com> skribis:
> On Fri 08 Feb 2013 15:10, ludo@gnu.org (Ludovic Courtès) writes:
>
>>> GC_malloc still returns eight-byte aligned memory hunks on 64-bit, no?
>>
>> Oh right:
>
> Does it? I couldn't tell from your test. I'm not a native unfolder :)
Yes, it’s always eight-byte aligned (even on 32-bit IIRC), and often
16-byte aligned.
>> And with 16 cases, we could even put single-precision floats in a word,
>> chars, and who knows what!
>
> Chars are already immediate :)
Right, well, strings then! ;-)
Ludo’.
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2013-02-08 15:52 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-01-17 10:53 on coding a portable R6RS package supporting Guile and its FFI Marco Maggi
2013-01-27 15:08 ` Ludovic Courtès
2013-01-28 3:00 ` Daniel Hartwig
2013-02-05 17:38 ` Andy Wingo
2013-02-05 19:29 ` Noah Lavine
2013-02-06 8:36 ` Andy Wingo
2013-02-06 13:25 ` Noah Lavine
2013-02-05 22:31 ` Ludovic Courtès
2013-02-06 10:57 ` Andy Wingo
2013-02-08 14:10 ` Ludovic Courtès
2013-02-08 14:37 ` Andy Wingo
2013-02-08 15:52 ` Ludovic Courtès
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).