* on coding a portable R6RS package supporting Guile and its FFI @ 2013-01-17 10:53 Marco Maggi 2013-01-27 15:08 ` Ludovic Courtès 2013-02-05 17:38 ` Andy Wingo 0 siblings, 2 replies; 12+ messages in thread From: Marco Maggi @ 2013-01-17 10:53 UTC (permalink / raw) To: guile-user Ciao, in my latest package[1] I am using Guile 2.0.7 and its FFI along with Mosh Scheme, Racket, Sagittarius Scheme, Vicare Scheme, Ypsilon Scheme. I have the following observations: * Guile does not come with the very simple SRFI 78: lightweight testing. I had to include it[2][3]. * It appears that there is no facility to handle "output arguments" from C functions; I mean the cases where a C function accepts as argument a pointer to variable that will be filled with some computed value. I am using a WITH-LOCAL-STORAGE[4] macro which is maybe ugly, but works. Such arguments are common, and represent a nuisance to handle. Racket has a sophisticated interface, complicated to use when writing adapter code. Something simpler but built in would be useful (this is the sort of thing a user does not want to think about: it should be an already solved problem). * Whenever a callout to C accepts a pointer argument: a bytevector argument is rejected. Is this not a useless complication? One can work around it by explicitly using BYTEVECTOR->POINTER, so everything is ready in Guile. The other Scheme implementations using a non-compacting garbage collector already support this feature and I find it truly convenient. * There are no raw memory getters and setters[5]? The fact that it is possible to create a wrapping bytevector is a plus for sure, but I find it inconvenient to allocate a bytevector when I do not need it (and raw getters and setters are really small functions). * The limit of 10 arguments for callouts to C is annoying. It forced me to exclude some SOFA functions resulting in amputated Guile support. TIA [1] <http://github.com/marcomaggi/r6rs-sofa/> [2] <http://github.com/marcomaggi/r6rs-sofa/tree/master/lib/guile/srfi> [3] <http://github.com/marcomaggi/vicare/blob/devel/doc/srfi-lightweight-testing.texi> [4] <http://github.com/marcomaggi/r6rs-sofa/blob/master/lib/sofa/compat.guile.sls> [5] <http://marcomaggi.github.com/docs/vicare.html/iklib-pointers.html> -- Marco Maggi ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-01-17 10:53 on coding a portable R6RS package supporting Guile and its FFI Marco Maggi @ 2013-01-27 15:08 ` Ludovic Courtès 2013-01-28 3:00 ` Daniel Hartwig 2013-02-05 17:38 ` Andy Wingo 1 sibling, 1 reply; 12+ messages in thread From: Ludovic Courtès @ 2013-01-27 15:08 UTC (permalink / raw) To: guile-user Hi Marco, Sorry for the delay. Marco Maggi <marco.maggi-ipsu@poste.it> skribis: > * Guile does not come with the very simple SRFI 78: > lightweight testing. I had to include it[2][3]. Patches welcome. :-) > * It appears that there is no facility to handle "output > arguments" from C functions; I mean the cases where a C > function accepts as argument a pointer to variable that > will be filled with some computed value. I am using a > WITH-LOCAL-STORAGE[4] macro which is maybe ugly, but > works. I typically roll my own allocation and dereference routines as well, such as ‘make-int-pointer’ and ‘dereference-int’ at: http://git.savannah.gnu.org/cgit/libchop.git/tree/guile2/chop/internal.scm#n255 Perhaps adding them to (system foreign) would help? > Such arguments are common, and represent a nuisance to > handle. Racket has a sophisticated interface, complicated > to use when writing adapter code. Something simpler but > built in would be useful (this is the sort of thing a user > does not want to think about: it should be an already > solved problem). I realize it would be good to have facilities already available for this. However, I’m not familiar with Racket’s FFI, and it’s not clear to me what a good generic API would be. For instance, one could imagine a layer on top of ‘pointer->procedure’ that would allow to specify whether pointer arguments really are output arguments. But then, you’d also have to allow programmers to tell how output arguments are to be allocated (“pointerless” or not, ‘malloc’, etc.), who owns them, what their life cycle is, etc. All in all, it’s always seemed easier to me to do that manually, with helpers specifically adapted to the C library I write bindings for. WDYT? > * Whenever a callout to C accepts a pointer argument: a > bytevector argument is rejected. Is this not a useless > complication? > > One can work around it by explicitly using > BYTEVECTOR->POINTER, so everything is ready in Guile. The > other Scheme implementations using a non-compacting > garbage collector already support this feature and I find > it truly convenient. Well, the ‘pointer’ type is useful, because it’s inherently a more low-level representation than bytevectors. That said, the FFI call could implicitly convert bytevectors to pointers. However, I generally prefer avoiding implicit type conversions like these, for clarify. Thoughts? > * There are no raw memory getters and setters[5]? There’s only ‘dereference-pointer’ now, but I agree we could add more of these, as well as pointer arithmetic operators. > * The limit of 10 arguments for callouts to C is annoying. > It forced me to exclude some SOFA functions resulting in > amputated Guile support. Agreed. Would you like to propose a patch in some of these areas? Thanks for your feedback, Ludo’. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-01-27 15:08 ` Ludovic Courtès @ 2013-01-28 3:00 ` Daniel Hartwig 0 siblings, 0 replies; 12+ messages in thread From: Daniel Hartwig @ 2013-01-28 3:00 UTC (permalink / raw) To: guile-user On 27 January 2013 23:08, Ludovic Courtès <ludo@gnu.org> wrote: > Hi Marco, > > Sorry for the delay. > > Marco Maggi <marco.maggi-ipsu@poste.it> skribis: >> * It appears that there is no facility to handle "output >> arguments" from C functions; I mean the cases where a C >> function accepts as argument a pointer to variable that >> will be filled with some computed value. I am using a >> WITH-LOCAL-STORAGE[4] macro which is maybe ugly, but >> works. > > I typically roll my own allocation and dereference routines as well, > such as ‘make-int-pointer’ and ‘dereference-int’ at: Likewise. > > http://git.savannah.gnu.org/cgit/libchop.git/tree/guile2/chop/internal.scm#n255 > > Perhaps adding them to (system foreign) would help? Yes, in these generic cases. > >> Such arguments are common, and represent a nuisance to >> handle. Racket has a sophisticated interface, complicated >> to use when writing adapter code. Something simpler but >> built in would be useful Pointer arguments are common, yes, but they have a variety of usage patterns across different APIs. This is evident in the complexity of the Racket API for dealing with this. In particular, the questions of who (caller or callee) allocates the memory and who deallocates the memory complicate the handling. In some APIs, the party in control of deallocation can change. >> (this is the sort of thing a user >> does not want to think about: it should be an already >> solved problem). Mapping any pointer interface requires some user thought to consider its unique behaviour. It is risky to abstract some of that process away from the programmer, even if only to accomodate a single, “most common” usage. IMO the best that could be done without complicating the FFI is to bring in a small set of generic helpers such as Ludo's make-int-pointer, etc.. > … > > All in all, it’s always seemed easier to me to do that manually, with > helpers specifically adapted to the C library I write bindings for. > > WDYT? Right. After considering the requirements for any given interface it is only a small amount of work to create the appropriate helpers. Being specific to the API under consideration, such helpers can present an optimal interface. > >> * Whenever a callout to C accepts a pointer argument: a >> bytevector argument is rejected. Is this not a useless >> complication? No, it is sensible type checking: bytevector is not pointer. >> >> One can work around it by explicitly using >> BYTEVECTOR->POINTER, so everything is ready in Guile. The >> other Scheme implementations using a non-compacting >> garbage collector already support this feature and I find >> it truly convenient. > > Well, the ‘pointer’ type is useful, because it’s inherently a more > low-level representation than bytevectors. > > That said, the FFI call could implicitly convert bytevectors to > pointers. However, I generally prefer avoiding implicit type > conversions like these, for clarify. > > Thoughts? This implicit conversion must be avoided. Instead, how about supporting supporting SRFI-4 types (s16vector, etc.) or typed arrays in the FFI. This would only cover some use cases, where the caller controls the memory and the foreign argument is a /typed/ pointer. This much could be implemented without seriously complicating the FFI. For “void *” it is still required to explicitly pass a pointer object, though it is not difficult to create helpers appropriate to the /particular/ interface. For callee allocated or deallocated memory it is still required to interface using pointer objects. SRFI-4 has the advantage that slots are always stored contiguously in memory. Typed array slots can be stored non-contiguously, and any such array would have to be rejected as “array-contents” does. Regards ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-01-17 10:53 on coding a portable R6RS package supporting Guile and its FFI Marco Maggi 2013-01-27 15:08 ` Ludovic Courtès @ 2013-02-05 17:38 ` Andy Wingo 2013-02-05 19:29 ` Noah Lavine 2013-02-05 22:31 ` Ludovic Courtès 1 sibling, 2 replies; 12+ messages in thread From: Andy Wingo @ 2013-02-05 17:38 UTC (permalink / raw) To: marco.maggi-ipsu; +Cc: guile-user On Thu 17 Jan 2013 11:53, Marco Maggi <marco.maggi-ipsu@poste.it> writes: > * Whenever a callout to C accepts a pointer argument: a > bytevector argument is rejected. Is this not a useless > complication? > > One can work around it by explicitly using > BYTEVECTOR->POINTER, so everything is ready in Guile. The > other Scheme implementations using a non-compacting > garbage collector already support this feature and I find > it truly convenient. I agree that we would do well to allow bytevectors wherever an argument type is a pointer. It avoids an allocation in a very common case without changing the expressiveness of the interface or adding significant additional overhead. Ludovic sounded like he could go either way on this point. Daniel brought up the argument that "a bytevector is not a pointer"; but for many purposes it's not a terrible thing to think of it that way. IMHO. Andy -- http://wingolog.org/ ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-02-05 17:38 ` Andy Wingo @ 2013-02-05 19:29 ` Noah Lavine 2013-02-06 8:36 ` Andy Wingo 2013-02-05 22:31 ` Ludovic Courtès 1 sibling, 1 reply; 12+ messages in thread From: Noah Lavine @ 2013-02-05 19:29 UTC (permalink / raw) To: Andy Wingo; +Cc: Guile Mailing List [-- Attachment #1: Type: text/plain, Size: 760 bytes --] Hello, That does sound very useful. The only thing that makes me worried is On Tue, Feb 5, 2013 at 12:38 PM, Andy Wingo <wingo@pobox.com> wrote: > On Thu 17 Jan 2013 11:53, Marco Maggi <marco.maggi-ipsu@poste.it> writes: > > The other Scheme implementations using a non-compacting > > garbage collector > Do we want to guarantee that our garbage collector will always be non-compacting, or force major interface changes if it is? (I realize we've already taken out smob mark procedures, but this is in some ways even harder to add back.) I'm not sure this is a big deal, but maybe we should consider what would happen in the future if we did want to change. Using bytevector->pointer is an acceptable cost to pay for flexibility, IMHO. Noah [-- Attachment #2: Type: text/html, Size: 1340 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-02-05 19:29 ` Noah Lavine @ 2013-02-06 8:36 ` Andy Wingo 2013-02-06 13:25 ` Noah Lavine 0 siblings, 1 reply; 12+ messages in thread From: Andy Wingo @ 2013-02-06 8:36 UTC (permalink / raw) To: Noah Lavine; +Cc: Guile Mailing List On Tue 05 Feb 2013 20:29, Noah Lavine <noah.b.lavine@gmail.com> writes: > On Tue, Feb 5, 2013 at 12:38 PM, Andy Wingo <wingo@pobox.com> wrote: > > On Thu 17 Jan 2013 11:53, Marco Maggi <marco.maggi-ipsu@poste.it> > writes: > > The other Scheme implementations using a non-compacting > > garbage collector > > Do we want to guarantee that our garbage collector will always be > non-compacting, or force major interface changes if it is? (I realize > we've already taken out smob mark procedures, but this is in some ways > even harder to add back.) Smob mark procedures are still supported. Changing the GC would be quite difficult -- we would need to progress to the point that we had significantly less C. That would be years away if it comes. Anyway, we could have a compacting collector and still allow fixed-position objects: http://www.scheme.com/csug8/smgmt.html#./smgmt:h3 Andy -- http://wingolog.org/ ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-02-06 8:36 ` Andy Wingo @ 2013-02-06 13:25 ` Noah Lavine 0 siblings, 0 replies; 12+ messages in thread From: Noah Lavine @ 2013-02-06 13:25 UTC (permalink / raw) To: Andy Wingo; +Cc: Guile Mailing List [-- Attachment #1: Type: text/plain, Size: 747 bytes --] On Wed, Feb 6, 2013 at 3:36 AM, Andy Wingo <wingo@pobox.com> wrote: > On Tue 05 Feb 2013 20:29, Noah Lavine <noah.b.lavine@gmail.com> writes: > > Smob mark procedures are still supported. Changing the GC would be > quite difficult -- we would need to progress to the point that we had > significantly less C. That would be years away if it comes. Anyway, we > could have a compacting collector and still allow fixed-position > objects: > I see. I hadn't thought about what a big project it would be. I was imagining bytevector->pointer as a procedure that just set some "fixed-position bit" on its object, so there wasn't any allocation. However, that would fit weirdly with the "->" convention. It's probably not worth worrying about. Noah [-- Attachment #2: Type: text/html, Size: 1247 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-02-05 17:38 ` Andy Wingo 2013-02-05 19:29 ` Noah Lavine @ 2013-02-05 22:31 ` Ludovic Courtès 2013-02-06 10:57 ` Andy Wingo 1 sibling, 1 reply; 12+ messages in thread From: Ludovic Courtès @ 2013-02-05 22:31 UTC (permalink / raw) To: guile-user Andy Wingo <wingo@pobox.com> skribis: > I agree that we would do well to allow bytevectors wherever an argument > type is a pointer. It avoids an allocation in a very common case > without changing the expressiveness of the interface or adding > significant additional overhead. I find it somewhat inelegant, though. This would be less of a problem if we had “immediate pointers” [0]. Would the retagging in 2.1 allow this? (At FOSDEM, Luca Saiu rightfully noted that we could use 4-bit tags instead of 3-bit tags on 64-bit arches, which would give us the needed room here.) Ludo’. [0] http://lists.gnu.org/archive/html/guile-devel/2011-01/msg00159.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-02-05 22:31 ` Ludovic Courtès @ 2013-02-06 10:57 ` Andy Wingo 2013-02-08 14:10 ` Ludovic Courtès 0 siblings, 1 reply; 12+ messages in thread From: Andy Wingo @ 2013-02-06 10:57 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guile-user On Tue 05 Feb 2013 23:31, ludo@gnu.org (Ludovic Courtès) writes: > This would be less of a problem if we had “immediate pointers” [0]. > Would the retagging in 2.1 allow this? The tc3 itag allocation in wip-retagging is: * tc3: * 000: A heap object with a tag word on the heap * 001: Some other immediate * 010: A pair * 011: Small integer (odd) * 100: (Unallocated tc3.) * 101: (Unallocated tc3.) * 110: A struct * 111: Small integer (even) And SCM_IMP is: #define SCM_IMP(x) (1 & SCM_UNPACK (x)) So perhaps 101 could be an immediate pointer. Dunno. Pretty nasty stuff! > (At FOSDEM, Luca Saiu rightfully noted that we could use 4-bit tags > instead of 3-bit tags on 64-bit arches, which would give us the needed > room here.) GC_malloc still returns eight-byte aligned memory hunks on 64-bit, no? Andy -- http://wingolog.org/ ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-02-06 10:57 ` Andy Wingo @ 2013-02-08 14:10 ` Ludovic Courtès 2013-02-08 14:37 ` Andy Wingo 0 siblings, 1 reply; 12+ messages in thread From: Ludovic Courtès @ 2013-02-08 14:10 UTC (permalink / raw) To: Andy Wingo; +Cc: guile-user Andy Wingo <wingo@pobox.com> skribis: > On Tue 05 Feb 2013 23:31, ludo@gnu.org (Ludovic Courtès) writes: > >> This would be less of a problem if we had “immediate pointers” [0]. >> Would the retagging in 2.1 allow this? > > The tc3 itag allocation in wip-retagging is: > > * tc3: > * 000: A heap object with a tag word on the heap > * 001: Some other immediate > * 010: A pair > * 011: Small integer (odd) > * 100: (Unallocated tc3.) > * 101: (Unallocated tc3.) > * 110: A struct > * 111: Small integer (even) > > And SCM_IMP is: > > #define SCM_IMP(x) (1 & SCM_UNPACK (x)) > > So perhaps 101 could be an immediate pointer. Dunno. Pretty nasty > stuff! Would be cool, I think. Maybe we should make an auction for unused tc3 tags? :-) >> (At FOSDEM, Luca Saiu rightfully noted that we could use 4-bit tags >> instead of 3-bit tags on 64-bit arches, which would give us the needed >> room here.) > > GC_malloc still returns eight-byte aligned memory hunks on 64-bit, no? Oh right: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> (use-modules (system foreign)) scheme@(guile-user)> (pointer->procedure '* (dynamic-func "GC_malloc" (dynamic-link)) (list size_t)) $2 = #<procedure 1b60ac0 (_)> scheme@(guile-user)> ,use (srfi srfi-1) scheme@(guile-user)> ,use (srfi srfi-26) scheme@(guile-user)> (unfold (cut > <> 100) (lambda (i) (logand (pointer-address ($2 (* i 2)) ) #x15)) 1+ 0) $7 = (0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 16 0 0 16 0 0 16 0 0 0 0 0 0 0 0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 0 16 0 16 0 16 0 16 0 0 0 0 0 0 0 0 0 16 0 16 0) --8<---------------cut here---------------end--------------->8--- If needed, we could just round up all the addresses returned. And with 16 cases, we could even put single-precision floats in a word, chars, and who knows what! The downside is that we’d have to maintain two tag systems. Ludo’. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-02-08 14:10 ` Ludovic Courtès @ 2013-02-08 14:37 ` Andy Wingo 2013-02-08 15:52 ` Ludovic Courtès 0 siblings, 1 reply; 12+ messages in thread From: Andy Wingo @ 2013-02-08 14:37 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guile-user Hi :) On Fri 08 Feb 2013 15:10, ludo@gnu.org (Ludovic Courtès) writes: >> GC_malloc still returns eight-byte aligned memory hunks on 64-bit, no? > > Oh right: Does it? I couldn't tell from your test. I'm not a native unfolder :) > And with 16 cases, we could even put single-precision floats in a word, > chars, and who knows what! Chars are already immediate :) Cheers, Andy -- http://wingolog.org/ ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: on coding a portable R6RS package supporting Guile and its FFI 2013-02-08 14:37 ` Andy Wingo @ 2013-02-08 15:52 ` Ludovic Courtès 0 siblings, 0 replies; 12+ messages in thread From: Ludovic Courtès @ 2013-02-08 15:52 UTC (permalink / raw) To: Andy Wingo; +Cc: guile-user Andy Wingo <wingo@pobox.com> skribis: > On Fri 08 Feb 2013 15:10, ludo@gnu.org (Ludovic Courtès) writes: > >>> GC_malloc still returns eight-byte aligned memory hunks on 64-bit, no? >> >> Oh right: > > Does it? I couldn't tell from your test. I'm not a native unfolder :) Yes, it’s always eight-byte aligned (even on 32-bit IIRC), and often 16-byte aligned. >> And with 16 cases, we could even put single-precision floats in a word, >> chars, and who knows what! > > Chars are already immediate :) Right, well, strings then! ;-) Ludo’. ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2013-02-08 15:52 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-01-17 10:53 on coding a portable R6RS package supporting Guile and its FFI Marco Maggi 2013-01-27 15:08 ` Ludovic Courtès 2013-01-28 3:00 ` Daniel Hartwig 2013-02-05 17:38 ` Andy Wingo 2013-02-05 19:29 ` Noah Lavine 2013-02-06 8:36 ` Andy Wingo 2013-02-06 13:25 ` Noah Lavine 2013-02-05 22:31 ` Ludovic Courtès 2013-02-06 10:57 ` Andy Wingo 2013-02-08 14:10 ` Ludovic Courtès 2013-02-08 14:37 ` Andy Wingo 2013-02-08 15:52 ` Ludovic Courtès
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).