Passing C pointers through guile

Passing C pointers through guile

[Maciek Godek]

Hi,
is there any portable and recommended way for passing C pointers around
in guile environment?
I think of something like scm_to_ptr (analogous to scm_to_int etc.).
Certainly such a value would be completely useless for the interpreter.
One can achieve simillar functionality using the aforementioned
scm_to_int (keeping track on the word length of the build target),
but it seems rather unnatural.

Perhaps the need for passing pointers is a sign of a weakness
in design of a given system, is it so?

very good regards
mg

Re: Passing C pointers through guile

[Ludovic Courtès]

Hi,

"Maciek Godek" <pstrychuj@gmail.com> writes:

> is there any portable and recommended way for passing C pointers around
> in guile environment?
> I think of something like scm_to_ptr (analogous to scm_to_int etc.).
> Certainly such a value would be completely useless for the interpreter.
> One can achieve simillar functionality using the aforementioned
> scm_to_int (keeping track on the word length of the build target),
> but it seems rather unnatural.

If the goal is to make a C object pointed to by some pointer available
to Scheme functions, the recommended (and simplest) way is to use a
"SMOB" (see the manual for details).  In practice, SMOBs incur some
storage overhead because they hold 4 machines words, while you may only
need one.

Another possibility is to use a "uo" field in a struct: it allows you to
have a struct field holding a machine word inaccessible to Scheme
functions (see the "Vtables" node of the manual).

Hope this helps,
Ludovic.

Re: Passing C pointers through guile

[Kjetil S. Matheussen]

Ludovic Court?s:
> Hi,
>
> "Maciek Godek" <pstrychuj@gmail.com> writes:
>
>> is there any portable and recommended way for passing C pointers around
>> in guile environment?
>> I think of something like scm_to_ptr (analogous to scm_to_int etc.).
>> Certainly such a value would be completely useless for the interpreter.
>> One can achieve simillar functionality using the aforementioned
>> scm_to_int (keeping track on the word length of the build target),
>> but it seems rather unnatural.
>
> If the goal is to make a C object pointed to by some pointer available
> to Scheme functions, the recommended (and simplest) way is to use a
> "SMOB" (see the manual for details).  In practice, SMOBs incur some
> storage overhead because they hold 4 machines words, while you may only
> need one.
>
> Another possibility is to use a "uo" field in a struct: it allows you to
> have a struct field holding a machine word inaccessible to Scheme
> functions (see the "Vtables" node of the manual).
>

I haven't heard of the "uo" field before, but at least using a SMOB
is really inconvenient, and using an unsigned long for storing
pointers is really convenient. (BTW. How does swig and gwrap handle
pointers?)

I agree with Maciek that it would at least be mind-comforting to have
functions like scm_to_ptr/etc, although not strictly necessary,
since using integers works just fine.

Re: Passing C pointers through guile

[Ludovic Courtès]

Hi,

"Kjetil S. Matheussen" <k.s.matheussen@notam02.no> writes:

> I haven't heard of the "uo" field before, but at least using a SMOB
> is really inconvenient, and using an unsigned long for storing
> pointers is really convenient. (BTW. How does swig and gwrap handle
> pointers?)

G-Wrap has "wrapped C types" (WCTs):

  http://www.nongnu.org/g-wrap/manual/Wrapping-a-C-Pointer-Type.html

IIRC, it uses a SMOB containing additional information behind the
scenes.

> I agree with Maciek that it would at least be mind-comforting to have
> functions like scm_to_ptr/etc, although not strictly necessary,
> since using integers works just fine.

`scm_{to,from}_uintptr ()' could be handy (patches welcome!).

That said, using a Scheme integer to represent a pointer wouldn't be
efficient (pointers would likely translate to bignums).

Thanks,
Ludovic.

Re: Passing C pointers through guile

[Greg Troxel]

"Kjetil S. Matheussen" <k.s.matheussen@notam02.no> writes:

> Ludovic Court?s:
>> Hi,
>>
>> "Maciek Godek" <pstrychuj@gmail.com> writes:
>>
>>> is there any portable and recommended way for passing C pointers around
>>> in guile environment?
>>> I think of something like scm_to_ptr (analogous to scm_to_int etc.).
>>> Certainly such a value would be completely useless for the interpreter.
>>> One can achieve simillar functionality using the aforementioned
>>> scm_to_int (keeping track on the word length of the build target),
>>> but it seems rather unnatural.
>>
>> If the goal is to make a C object pointed to by some pointer available
>> to Scheme functions, the recommended (and simplest) way is to use a
>> "SMOB" (see the manual for details).  In practice, SMOBs incur some
>> storage overhead because they hold 4 machines words, while you may only
>> need one.
>>
>> Another possibility is to use a "uo" field in a struct: it allows you to
>> have a struct field holding a machine word inaccessible to Scheme
>> functions (see the "Vtables" node of the manual).
>>
>
> I haven't heard of the "uo" field before, but at least using a SMOB
> is really inconvenient, and using an unsigned long for storing
> pointers is really convenient. (BTW. How does swig and gwrap handle
> pointers?)
>
> I agree with Maciek that it would at least be mind-comforting to have
> functions like scm_to_ptr/etc, although not strictly necessary,
> since using integers works just fine.

Does C guarantee that pointers fit in unsigned long?

I suspect SMOBs are the right answer.  Part of the point is to do the
right thing when the scheme object is garbage collected.  When
discarding a pointer often one should be freeing an object or
decrementing a refcount.
I've use SMOBs, and they really weren't that hard once I dug in.

Re: Passing C pointers through guile

[Kjetil S. Matheussen]

On Wed, 9 Jul 2008, Greg Troxel wrote:

> "Kjetil S. Matheussen" <k.s.matheussen@notam02.no> writes:
> 
> > Ludovic Court?s:
> >> Hi,
> >>
> >> "Maciek Godek" <pstrychuj@gmail.com> writes:
> >>
> >>> is there any portable and recommended way for passing C pointers around
> >>> in guile environment?
> >>> I think of something like scm_to_ptr (analogous to scm_to_int etc.).
> >>> Certainly such a value would be completely useless for the interpreter.
> >>> One can achieve simillar functionality using the aforementioned
> >>> scm_to_int (keeping track on the word length of the build target),
> >>> but it seems rather unnatural.
> >>
> >> If the goal is to make a C object pointed to by some pointer available
> >> to Scheme functions, the recommended (and simplest) way is to use a
> >> "SMOB" (see the manual for details).  In practice, SMOBs incur some
> >> storage overhead because they hold 4 machines words, while you may only
> >> need one.
> >>
> >> Another possibility is to use a "uo" field in a struct: it allows you to
> >> have a struct field holding a machine word inaccessible to Scheme
> >> functions (see the "Vtables" node of the manual).
> >>
> >
> > I haven't heard of the "uo" field before, but at least using a SMOB
> > is really inconvenient, and using an unsigned long for storing
> > pointers is really convenient. (BTW. How does swig and gwrap handle
> > pointers?)
> >
> > I agree with Maciek that it would at least be mind-comforting to have
> > functions like scm_to_ptr/etc, although not strictly necessary,
> > since using integers works just fine.
> 
> Does C guarantee that pointers fit in unsigned long?
> 

I don't know. But in practice: Yes.


> I suspect SMOBs are the right answer.  Part of the point is to do the
> right thing when the scheme object is garbage collected.  When
> discarding a pointer often one should be freeing an object or
> decrementing a refcount.
> I've use SMOBs, and they really weren't that hard once I dug in.
> 

Sure, if you just do this now and then, SMOBs aren't a problem.
But that doesn't change the fact that all the functionality SMOB
provides is overkill when the only thing you need is to hold
a pointer.

I just submitted a patch for unsigned ptr to guile-devel though,
and if that patch is accepted, there is nothing more to discuss:

For those who needs the full functionality of SMOB's, or need
the extra speed and memory advantages of SMOB's, they are free
to use SMOB's. If not, the functions scm_to/from_uintptr
can be used instead, which is a billion times easier to use.

Re: Passing C pointers through guile

[Ludovic Courtès]

Hi,

"Kjetil S. Matheussen" <k.s.matheussen@notam02.no> writes:

> On Wed, 9 Jul 2008, Greg Troxel wrote:

>> Does C guarantee that pointers fit in unsigned long?
>
> I don't know. But in practice: Yes.

That's usually the case but it's not guaranteed, which is why C99
provides `uintptr_t' in <stdint.h>.

> Sure, if you just do this now and then, SMOBs aren't a problem.
> But that doesn't change the fact that all the functionality SMOB
> provides is overkill when the only thing you need is to hold
> a pointer.

It's not overkill, it's exactly what you need: disjoint SMOB type,
`free' and `mark' procedures.

> For those who needs the full functionality of SMOB's, or need
> the extra speed and memory advantages of SMOB's, they are free
> to use SMOB's. If not, the functions scm_to/from_uintptr
> can be used instead, which is a billion times easier to use.

Not really: as mentioned on `guile-devel', you'll need to implement that
functionality (disjoint type, garbage collection) on top of your
integer.  One advantage is that this can be done in Scheme (except for
the `mark' procedure, but it's not always needed); the main drawback in
the context of Guile is poor performance.

Thanks,
Ludovic.

Re: Passing C pointers through guile

[Kjetil S. Matheussen]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1589 bytes --]


Ludovic Courtès:
> > Sure, if you just do this now and then, SMOBs aren't a problem.
> > But that doesn't change the fact that all the functionality SMOB
> > provides is overkill when the only thing you need is to hold
> > a pointer.
> 
> It's not overkill, it's exactly what you need: disjoint SMOB type,
> `free' and `mark' procedures.

My point is that SMOB's are overkill if you _don't_ need either free or 
mark.


> > For those who needs the full functionality of SMOB's, or need
> > the extra speed and memory advantages of SMOB's, they are free
> > to use SMOB's. If not, the functions scm_to/from_uintptr
> > can be used instead, which is a billion times easier to use.
> 
> Not really: as mentioned on `guile-devel', you'll need to implement that
> functionality (disjoint type, garbage collection) on top of your
> integer.  One advantage is that this can be done in Scheme (except for
> the `mark' procedure, but it's not always needed); the main drawback in
> the context of Guile is poor performance.

I'm just copying my answer to the guile-devel mailing list:

"
Point is that you very often don't need any kind of free functionality.
For example, if you create a gui widget, you probably have a callback
function which is called if the gui is closed. That callback
function can free any allocated memory. Another example from snd
is creating ladspa plugins (audio plugins in linux). Handlers
from those, plus variuos configuration stuff, is alive througout
the whole session and will be automatically freed when the program
closes.
"

Re: Passing C pointers through guile

[Ken Raeburn]

On Jul 9, 2008, at 12:55, Kjetil S. Matheussen wrote:
> On Wed, 9 Jul 2008, Greg Troxel wrote:
>> Does C guarantee that pointers fit in unsigned long?
> I don't know. But in practice: Yes.

According to various sources, 64-bit Windows uses an LLP64 model --  
meaning long is 32 bits, long long and pointers are 64 bits.  So, no.

Near as I can tell, C99 does not require that there be *any* integral  
type large enough to hold a pointer value (6.3.2.3 paragraph 6); and  
specifically, uintptr_t and intptr_t are optional types.  However, I  
expect any C99 implementation we're likely to run across will have  
such a type, and will define [u]intptr_t.  I don't have a copy of the  
C89 spec handy, though, and unfortunately that's where most compilers  
are these days.

In practice, it's also probably safe to use unsigned long long (or  
whatever Windows calls it) on the platforms that have it, and unsigned  
long on those that don't.  But testing compiler properties in autoconf  
and then using them in your installed headers may tie you to a  
particular compiler when more than one may be available (e.g.,  
vendor's compiler and gcc).

Ken

Re: Passing C pointers through guile

[Greg Troxel]

Ken Raeburn <raeburn@raeburn.org> writes:

> On Jul 9, 2008, at 12:55, Kjetil S. Matheussen wrote:
>> On Wed, 9 Jul 2008, Greg Troxel wrote:
>>> Does C guarantee that pointers fit in unsigned long?
>> I don't know. But in practice: Yes.
>
> According to various sources, 64-bit Windows uses an LLP64 model -- 
> meaning long is 32 bits, long long and pointers are 64 bits.  So, no.

I believe this is correct.  I have been the 64-bit portability weenie on
a large project at work, and been railing against int/* assignment.  The
overwhelming consensus among those of us that have actually dealt with
making programs run on non-ILP32 machines is that int/* assignments are
just plain wrong.

> Near as I can tell, C99 does not require that there be *any* integral
> type large enough to hold a pointer value (6.3.2.3 paragraph 6); and
> specifically, uintptr_t and intptr_t are optional types.  However, I
> expect any C99 implementation we're likely to run across will have
> such a type, and will define [u]intptr_t.  I don't have a copy of the
> C89 spec handy, though, and unfortunately that's where most compilers
> are these days.

My recent experience is that decent compilers are now essentially C99,
and that Microsoft compilers are mostly C99 with a a few defects.  Our
strategy has been to patch around the defective compilers by defining
the things we need somewhat like:

#if defined(LOSING_COMPILER_A)
#ifdef i386
typedef unsigned lont uintptr_t;
#else
#error DEFINE uintptr_t for yoru platfrom
#endif
#endif

and then just rely on the C99 definition.  Surprising little fixup has
been needed.

> In practice, it's also probably safe to use unsigned long long (or
> whatever Windows calls it) on the platforms that have it, and unsigned
> long on those that don't.  But testing compiler properties in autoconf
> and then using them in your installed headers may tie you to a
> particular compiler when more than one may be available (e.g.,
> vendor's compiler and gcc).

If we have to store a pointer we should just use void *.  I see Ken's
point about testing, but that quickly leads to madness as he notes.