make-c-struct and pointer->string

make-c-struct and pointer->string

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 638 bytes --]

Hello,

	guile 2.2.4.1-cdb19

,use (system foreign)

;; this fails

scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!")))
$16 = #<pointer 0x55a3d54d54d0>
scheme@(guile-user)> (parse-c-struct $16 (list '* '*))
$17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>)
scheme@(guile-user)> (map pointer->string $17)
$18 = ("?g?գU" "`!?գU")

;; but this works

scheme@(guile-user)> (map pointer->string (parse-c-struct (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) (list '* '*)))
$19 = ("hello " "there!")


David

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: make-c-struct and pointer->string

[Amirouche Boubekki]

[-- Attachment #1: Type: text/plain, Size: 1892 bytes --]

Eventually, I can reproduced the issue with guile-next from guix:

scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer
"hello ") (string->pointer "there!")))
$5 = #<pointer 0x1d1ca70>
scheme@(guile-user) [1]> (parse-c-struct $5 (list '* '*))
$6 = (#<pointer 0x166a950> #<pointer 0x166a9f0>)
scheme@(guile-user) [1]> (map pointer->string $6)
$7 = ("hello " "there!")
scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer
"hello") (string->pointer "there!")))
$8 = #<pointer 0x1ce2a40>
scheme@(guile-user) [1]> (parse-c-struct $8 (list '* '*))
$9 = (#<pointer 0x166a810> #<pointer 0x166a910>)
scheme@(guile-user) [1]> (map pointer->string $9)
$10 = ("hello" "there!")
scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer
"hello") (string->pointer "there!")))
$11 = #<pointer 0x1ce2a10>
scheme@(guile-user) [1]> (parse-c-struct $11 (list '* '*))
$12 = (#<pointer 0x166a850> #<pointer 0x166a690>)
scheme@(guile-user) [1]> (map pointer->string $12)
$13 = ("?<\"?\x0e\x7f" "@?f\x01")
scheme@(guile-user) [1]>
scheme@(guile-user)>


Le mar. 26 mars 2019 à 14:19, David Pirotte <david@altosw.be> a écrit :

> Hello,
>
>         guile 2.2.4.1-cdb19
>
> ,use (system foreign)
>
> ;; this fails
>
> scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer
> "hello ") (string->pointer "there!")))
> $16 = #<pointer 0x55a3d54d54d0>
> scheme@(guile-user)> (parse-c-struct $16 (list '* '*))
> $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>)
> scheme@(guile-user)> (map pointer->string $17)
> $18 = ("?g?գU" "`!?գU")
>
> ;; but this works
>
> scheme@(guile-user)> (map pointer->string (parse-c-struct (make-c-struct
> (list '* '*) (list (string->pointer "hello ") (string->pointer "there!")))
> (list '* '*)))
> $19 = ("hello " "there!")
>
>
> David
>

[-- Attachment #2: Type: text/html, Size: 2757 bytes --]

Re: make-c-struct and pointer->string

[Neil Jerram]

On Thu, 28 Mar 2019 at 15:15, Amirouche Boubekki
<amirouche.boubekki@gmail.com> wrote:
>
> Eventually, I can reproduced the issue with guile-next from guix:
>
> scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!")))
> $5 = #<pointer 0x1d1ca70>
> scheme@(guile-user) [1]> (parse-c-struct $5 (list '* '*))
> $6 = (#<pointer 0x166a950> #<pointer 0x166a9f0>)
> scheme@(guile-user) [1]> (map pointer->string $6)
> $7 = ("hello " "there!")
> scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello") (string->pointer "there!")))
> $8 = #<pointer 0x1ce2a40>
> scheme@(guile-user) [1]> (parse-c-struct $8 (list '* '*))
> $9 = (#<pointer 0x166a810> #<pointer 0x166a910>)
> scheme@(guile-user) [1]> (map pointer->string $9)
> $10 = ("hello" "there!")
> scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello") (string->pointer "there!")))
> $11 = #<pointer 0x1ce2a10>
> scheme@(guile-user) [1]> (parse-c-struct $11 (list '* '*))
> $12 = (#<pointer 0x166a850> #<pointer 0x166a690>)
> scheme@(guile-user) [1]> (map pointer->string $12)
> $13 = ("?<\"?\x0e\x7f" "@?f\x01")
> scheme@(guile-user) [1]>
> scheme@(guile-user)>
>
>
> Le mar. 26 mars 2019 à 14:19, David Pirotte <david@altosw.be> a écrit :
>>
>> Hello,
>>
>>         guile 2.2.4.1-cdb19
>>
>> ,use (system foreign)
>>
>> ;; this fails
>>
>> scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!")))
>> $16 = #<pointer 0x55a3d54d54d0>
>> scheme@(guile-user)> (parse-c-struct $16 (list '* '*))
>> $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>)
>> scheme@(guile-user)> (map pointer->string $17)
>> $18 = ("?g?գU" "`!?գU")
>>
>> ;; but this works
>>
>> scheme@(guile-user)> (map pointer->string (parse-c-struct (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) (list '* '*)))
>> $19 = ("hello " "there!")

I guess it may be GC.  I've had problems with code like

   (procedure->pointer void (lambda (...) ...) args...)

because procedure->pointer does not protect the (lambda ...) from being GC'd.

For example: http://git.savannah.nongnu.org/cgit/ossaulib.git/commit/?id=1d31a87b53e9375b5fdd5120b7d409e6dde6aa47

I wonder if there's a similar problem here with the string->pointer data.

Best wishes,
     Neil

Re: make-c-struct and pointer->string

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 2751 bytes --]

Hi Amirouche,

I should add to your own answer that you actually were unable to reproduce the
problem while using the latest 2.2 release tarball (the one that Ubuntu packages to
be (even) more precise), as one can see on your gnome paste earlier today [1]

So, if this is corect, there must be a commit out there that 'broke it'

Cheers,
David

	here below, a copy/paste of
	https://paste.gnome.org/pbm5ok3pd#line-7
	[ not sure how long gnome pastes remains visible ..

	[ in the paste below, which is not mine, I have no idea why, starting from
	[ line 15, guile entered a raised exception mode

[1]	amirouche  ~  12:59:20  guile
GNU Guile 2.2.4
Copyright (C) 1995-2017 Free Software Foundation, Inc.
 
Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.
 
Enter `,help' for help.
scheme@(guile-user)> (use-modules (system foreign))
scheme@(guile-user)> (define str-1 (string->pointer "Hello"))
scheme@(guile-user)> (define str-2 (string->pointer "there!"))
scheme@(guile-user)> (make-c-struct (list '* '*) (list str-1 str-2))
$1 = #<pointer 0x55f30df62dd0>
scheme@(guile-user) [1]> (parse-c-struct $1 (list '* '*))
$2 = (#<pointer 0x55f30de21510> #<pointer 0x55f30de21f90>)
scheme@(guile-user) [1]> str-1
$3 = #<pointer 0x55f30de21510>
scheme@(guile-user) [1]> (map pointer->string $2)
$4 = ("Hello" "there!")
scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello
 ") (string->pointer "there!"))) $5 = #<pointer 0x55f30e235aa0>
scheme@(guile-user) [1]> (parse-c-struct $5 (list '* '*))
$6 = (#<pointer 0x55f30de25e80> #<pointer 0x55f30de26a30>)
scheme@(guile-user) [1]> (map pointer->string $6)
$7 = ("hello " "there!")
scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello")
 (string->pointer "there!"))) $8 = #<pointer 0x55f30e076470>
scheme@(guile-user) [1]> (parse-c-struct $8 (list '* '*))
$9 = (#<pointer 0x55f30de28020> #<pointer 0x55f30de28bd0>)
scheme@(guile-user) [1]> (map pointer->string $9)
$10 = ("hello" "there!")
scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello")
 (string->pointer "there!"))) $11 = #<pointer 0x55f30e0453e0>
scheme@(guile-user) [1]> (parse-c-struct $11 (list '* '*))
$12 = (#<pointer 0x55f30de2a170> #<pointer 0x55f30de28bd0>)
scheme@(guile-user) [1]> (map pointer->string $12)
$13 = ("hello" "there!")
scheme@(guile-user) [1]>
Posted by Anonymous at 28 Mar 2019, 12:33:15 UTC Language: scheme • Views: 4
© The GNOME Project

GNOME Pastebin. Hosted by Red Hat. Powered by Sticky Notes © 2014 Sayak Banerjee.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: make-c-struct and pointer->string

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 1347 bytes --]

Hello Neil,

> I guess it may be GC.  I've had problems with code like
> 
>    (procedure->pointer void (lambda (...) ...) args...)
> 
> because procedure->pointer does not protect the (lambda ...) from being GC'd.
> 
> For example:
> http://git.savannah.nongnu.org/cgit/ossaulib.git/commit/?id=1d31a87b53e9375b5fdd5120b7d409e6dde6aa47
> 
> I wonder if there's a similar problem here with the string->pointer data.

Thanks for your help!

Following your explanation and example, I tried this and thought it would work then,
but it also failed:

GNU Guile 2.2.4.1-cdb19
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,use (system foreign)
scheme@(guile-user)> (define str-1 "Hello")
scheme@(guile-user)> (define str-2 "there!")
scheme@(guile-user)>  (make-c-struct (list '* '*) (list (string->pointer str-1) (string->pointer str-2)))
$2 = #<pointer 0x55ae02e57830>
scheme@(guile-user)> (parse-c-struct $2 (list '* '*))
$3 = (#<pointer 0x55ae02f9e3c0> #<pointer 0x55ae02f8b050>)
scheme@(guile-user)> (map pointer->string $3)
$4 = ("" "`\v?\x02?U")

Cheers,
David

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: make-c-struct and pointer->string

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 2987 bytes --]

Hi again,

> ...
> Following your explanation and example, I tried this and thought it would work
> then, but it also failed:

> GNU Guile 2.2.4.1-cdb19
 
> Enter `,help' for help.
> scheme@(guile-user)> ,use (system foreign)
> scheme@(guile-user)> (define str-1 "Hello")
> scheme@(guile-user)> (define str-2 "there!")
> scheme@(guile-user)>  (make-c-struct (list '* '*) (list (string->pointer str-1)
> (string->pointer str-2))) $2 = #<pointer 0x55ae02e57830>
> scheme@(guile-user)> (parse-c-struct $2 (list '* '*))
> $3 = (#<pointer 0x55ae02f9e3c0> #<pointer 0x55ae02f8b050>)
> scheme@(guile-user)> (map pointer->string $3)
> $4 = ("" "`\v?\x02?U")

Here are another couple of examples:

scheme@(guile-user)> ,use (system foreign)
scheme@(guile-user)> (define str-1 "Hello")
scheme@(guile-user)> (define str-2 "there!")
scheme@(guile-user)>  (make-c-struct (list '* '*) (list (string->pointer str-1) (string->pointer str-2)))
$2 = #<pointer 0x55844635a0b0>
scheme@(guile-user)> (parse-c-struct $2 (list '* '*))
$3 = (#<pointer 0x558445edf610> #<pointer 0x558445edf630>)
scheme@(guile-user)> (map pointer->string $3)
$4 = ("Hello" "there!")

But then:

scheme@(guile-user)> str-1
$5 = "Hello"
scheme@(guile-user)> str-2
$6 = "there!"
scheme@(guile-user)> (map pointer->string $3)
$7 = ("?\\?q\x7f" " ??E?U")
scheme@(guile-user)> (map pointer->string $3)
$8 = ("?\\?q\x7f" " ??E?U")

I guess that what the GC recollected are the pointers, or even $3, the list
Let's hold on it then:

scheme@(guile-user)> (define c-struct (make-c-struct (list '* '*) (list (string->pointer str-1) (string->pointer str-2))))
scheme@(guile-user)> (parse-c-struct c-struct (list '* '*))
$9 = (#<pointer 0x558445edf400> #<pointer0x558445ef8df0>)
scheme@(guile-user)> (map pointer->string $9)
$10 = ("\x02" "0??E?U")

Not good either. Let's hold on the parse-c-struct to then:

scheme@(guile-user)> (define parsed (parse-c-struct c-struct (list '* '*)))
scheme@(guile-user)> parsed
$16 = (#<pointer 0x558445edf400> #<pointer 0x558445ef8df0>)
scheme@(guile-user)> (map pointer->string $16)
$17 = ("0??E?U" "?\x01?F?U")
scheme@(guile-user)> (map pointer->string parsed)
$18 = ("\x03" "?\x01?F?U")

Didn't do it either
Ok, let's hold-on to the pointers (which I thought would not be necessary after
holding-on c-struct and/or parsed):

scheme@(guile-user)> (define p1 (string->pointer str-1))
scheme@(guile-user)> (define p2 (string->pointer str-2))
scheme@(guile-user)> (make-c-struct (list '* '*) (list p1 p2))
$19 = #<pointer 0x5584467a5d70>
scheme@(guile-user)> (parse-c-struct $19 (list '* '*))
$20 = (#<pointer 0x5584468b0190> #<pointer 0x5584468b4430>)
scheme@(guile-user)> p1
$21 = #<pointer 0x5584468b0190>
scheme@(guile-user)> p2
$22 = #<pointer 0x5584468b4430>
scheme@(guile-user)> (map pointer->string $20)
$23 = ("Hello" "there!")

That worked, but I wonder if this is what is expected from us (users)?

Cheers,
David

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: make-c-struct and pointer->string

[Neil Jerram]

[-- Attachment #1: Type: text/plain, Size: 3345 bytes --]

On Fri, 29 Mar 2019, 09:46 David Pirotte, <david@altosw.be> wrote:

> Hi again,
>
> > ...
> > Following your explanation and example, I tried this and thought it
> would work
> > then, but it also failed:
>
> > GNU Guile 2.2.4.1-cdb19
>
> > Enter `,help' for help.
> > scheme@(guile-user)> ,use (system foreign)
> > scheme@(guile-user)> (define str-1 "Hello")
> > scheme@(guile-user)> (define str-2 "there!")
> > scheme@(guile-user)>  (make-c-struct (list '* '*) (list
> (string->pointer str-1)
> > (string->pointer str-2))) $2 = #<pointer 0x55ae02e57830>
> > scheme@(guile-user)> (parse-c-struct $2 (list '* '*))
> > $3 = (#<pointer 0x55ae02f9e3c0> #<pointer 0x55ae02f8b050>)
> > scheme@(guile-user)> (map pointer->string $3)
> > $4 = ("" "`\v?\x02?U")
>
> Here are another couple of examples:
>
> scheme@(guile-user)> ,use (system foreign)
> scheme@(guile-user)> (define str-1 "Hello")
> scheme@(guile-user)> (define str-2 "there!")
> scheme@(guile-user)>  (make-c-struct (list '* '*) (list (string->pointer
> str-1) (string->pointer str-2)))
> $2 = #<pointer 0x55844635a0b0>
> scheme@(guile-user)> (parse-c-struct $2 (list '* '*))
> $3 = (#<pointer 0x558445edf610> #<pointer 0x558445edf630>)
> scheme@(guile-user)> (map pointer->string $3)
> $4 = ("Hello" "there!")
>
> But then:
>
> scheme@(guile-user)> str-1
> $5 = "Hello"
> scheme@(guile-user)> str-2
> $6 = "there!"
> scheme@(guile-user)> (map pointer->string $3)
> $7 = ("?\\?q\x7f" " ??E?U")
> scheme@(guile-user)> (map pointer->string $3)
> $8 = ("?\\?q\x7f" " ??E?U")
>
> I guess that what the GC recollected are the pointers, or even $3, the list
> Let's hold on it then:
>
> scheme@(guile-user)> (define c-struct (make-c-struct (list '* '*) (list
> (string->pointer str-1) (string->pointer str-2))))
> scheme@(guile-user)> (parse-c-struct c-struct (list '* '*))
> $9 = (#<pointer 0x558445edf400> #<pointer0x558445ef8df0>)
> scheme@(guile-user)> (map pointer->string $9)
> $10 = ("\x02" "0??E?U")
>
> Not good either. Let's hold on the parse-c-struct to then:
>
> scheme@(guile-user)> (define parsed (parse-c-struct c-struct (list '*
> '*)))
> scheme@(guile-user)> parsed
> $16 = (#<pointer 0x558445edf400> #<pointer 0x558445ef8df0>)
> scheme@(guile-user)> (map pointer->string $16)
> $17 = ("0??E?U" "?\x01?F?U")
> scheme@(guile-user)> (map pointer->string parsed)
> $18 = ("\x03" "?\x01?F?U")
>
> Didn't do it either
> Ok, let's hold-on to the pointers (which I thought would not be necessary
> after
> holding-on c-struct and/or parsed):
>
> scheme@(guile-user)> (define p1 (string->pointer str-1))
> scheme@(guile-user)> (define p2 (string->pointer str-2))
> scheme@(guile-user)> (make-c-struct (list '* '*) (list p1 p2))
> $19 = #<pointer 0x5584467a5d70>
> scheme@(guile-user)> (parse-c-struct $19 (list '* '*))
> $20 = (#<pointer 0x5584468b0190> #<pointer 0x5584468b4430>)
> scheme@(guile-user)> p1
> $21 = #<pointer 0x5584468b0190>
> scheme@(guile-user)> p2
> $22 = #<pointer 0x5584468b4430>
> scheme@(guile-user)> (map pointer->string $20)
> $23 = ("Hello" "there!")
>
> That worked, but I wonder if this is what is expected from us (users)?
>

Yes, that would seem very odd to me, as I thought that a 'pointer' really
was just an address.

(Hopefully some more light will be thrown when the maintainers are back
around again!)

Best wishes,
     Neil


> Cheers,
> David
>

[-- Attachment #2: Type: text/html, Size: 4804 bytes --]

Re: make-c-struct and pointer->string

[Mark H Weaver]

Hi David,

David Pirotte <david@altosw.be> writes:

> Hello,
>
> 	guile 2.2.4.1-cdb19
>
> ,use (system foreign)
>
> ;; this fails
>
> scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!")))
> $16 = #<pointer 0x55a3d54d54d0>
> scheme@(guile-user)> (parse-c-struct $16 (list '* '*))
> $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>)
> scheme@(guile-user)> (map pointer->string $17)
> $18 = ("?g?գU" "`!?գU")

The Guile manual states:

 -- Scheme Procedure: string->pointer string [encoding]
     Return a foreign pointer to a nul-terminated copy of STRING in the
     given ENCODING, defaulting to the current locale encoding.  The C
     string is freed when the returned foreign pointer becomes
     unreachable.

Note the last sentence.  When the returned foreign pointer (object)
becomes unreachable, the C string is freed.  The problem here is that
you're not keeping a reference to those foreign pointer objects.

If you look at the code in foreign.c, specifically
'scm_string_to_pointer' and 'scm_from_pointer', you'll see that the C
pointer is wrapped within a heap-allocated foreign pointer object, and a
finalizer is being set on that heap object.  'make-c-struct' copies the
C pointers from those foreign pointer objects, but not not keep a
reference to the objects themselves.

In a later message, you wrote:

> Following your explanation and example, I tried this and thought it would work then,
> but it also failed:
>
> GNU Guile 2.2.4.1-cdb19
> Copyright (C) 1995-2017 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guile-user)> ,use (system foreign)
> scheme@(guile-user)> (define str-1 "Hello")
> scheme@(guile-user)> (define str-2 "there!")
> scheme@(guile-user)>  (make-c-struct (list '* '*) (list (string->pointer str-1) (string->pointer str-2)))
> $2 = #<pointer 0x55ae02e57830>
> scheme@(guile-user)> (parse-c-struct $2 (list '* '*))
> $3 = (#<pointer 0x55ae02f9e3c0> #<pointer 0x55ae02f8b050>)
> scheme@(guile-user)> (map pointer->string $3)
> $4 = ("" "`\v?\x02?U")

Here are you specifically saving a reference to the Scheme string
objects, but that's not what's needed.  What's needed is to hold
references to the foreign pointer objects returned by 'string->pointer'
for as long as the C strings are needed.

      Regards,
        Mark

Re: make-c-struct and pointer->string

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 2954 bytes --]

Hi Mark,

Thanks for your help.

> > ...
> > scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer "hello
> > ") (string->pointer "there!"))) $16 = #<pointer 0x55a3d54d54d0>
> > scheme@(guile-user)> (parse-c-struct $16 (list '* '*))
> > $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>)
> > scheme@(guile-user)> (map pointer->string $17)
> > $18 = ("?g?գU" "`!?գU")  

> The Guile manual states:

>  -- Scheme Procedure: string->pointer string [encoding]
>      Return a foreign pointer to a nul-terminated copy of STRING in the
>      given ENCODING, defaulting to the current locale encoding.  The C
>      string is freed when the returned foreign pointer becomes
>      unreachable.

> Note the last sentence.  When the returned foreign pointer (object)
> becomes unreachable, the C string is freed.  The problem here is that
> you're not keeping a reference to those foreign pointer objects.

Yes, and I was assuming that make-c-struct was holding a reference to 'its
children', including pointers.

> If you look at the code in foreign.c, specifically ...

>  'make-c-struct' copies the C pointers from those foreign pointer objects, but not
> not keep a reference to the objects themselves.

To me, this sounds very counter intuitive, actually, it sounds like a bug,
make-c-struct should be holding a reference to the pointers it receives: i seems to
me that only when the c-struct itself becomes unreachable, that these pointers could
be freed?

If I am wrong, and it seems I am (wrong), we really should think about writing
something about this in the manual (and explaining why would be nice).

Cheers,
David

ps:	in the last part of my email, I did precisely what you suggest in your
	answer, and showed that it works - if we hold a reference to these pointers
	'by ourselves' - and asked if that was what is what was expected from us and
	Neil Jerram to (also) find that 'very odd'

> > ...
> > Ok, let's hold-on to the pointers (which I thought would not be necessary
> > after holding-on c-struct and/or parsed):

> > scheme@(guile-user)> (define p1 (string->pointer str-1))
> > scheme@(guile-user)> (define p2 (string->pointer str-2))
> > scheme@(guile-user)> (make-c-struct (list '* '*) (list p1 p2))
> > $19 = #<pointer 0x5584467a5d70>
> > scheme@(guile-user)> (parse-c-struct $19 (list '* '*))
> > $20 = (#<pointer 0x5584468b0190> #<pointer 0x5584468b4430>)
> > scheme@(guile-user)> p1
> > $21 = #<pointer 0x5584468b0190>
> > scheme@(guile-user)> p2
> > $22 = #<pointer 0x5584468b4430>
> > scheme@(guile-user)> (map pointer->string $20)
> > $23 = ("Hello" "there!")

> > That worked, but I wonder if this is what is expected from us (users)?

> Yes, that would seem very odd to me, as I thought that a 'pointer' really
> was just an address.

> (Hopefully some more light will be thrown when the maintainers are back
> around again!)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

Re: make-c-struct and pointer->string

[Mark H Weaver]

Hi David,

David Pirotte <david@altosw.be> writes:

>>  'make-c-struct' copies the C pointers from those foreign pointer objects, but not
>> not keep a reference to the objects themselves.
>
> To me, this sounds very counter intuitive, actually, it sounds like a bug,
> make-c-struct should be holding a reference to the pointers it receives: i seems to
> me that only when the c-struct itself becomes unreachable, that these pointers could
> be freed?

It's a reasonable suggestion, and I agree that the documentation should
be clarified on these points, which are quite tricky.

I thought about what would be required to implement your suggestion and
quickly ran into difficulties, and also the understanding that this is
not always the right behavior.

First of all, the address of the foreign pointer object is what we would
need to keep a reference to, and that's not the same as the C pointer
wrapped within that foreign pointer object.

This raises the question of where to store these additional references,
since we can't store them in any of the C struct fields defined by the
user.  The worse problem is that there's no way to keep these extra
references in sync with the C pointers in the C struct.  The C code
might overwrite one of those pointers, and move them to a different
structure somewhere else.

Also note that in many cases, foreign pointer objects are used only
transiently to copy a C pointer from one place to another.  If we
implemented your suggestion, these otherwise transient foreign pointer
objects would be kept alive for no purpose.

The other thing is that even if we had these suggested semantics, you
would still need to arrange to hold a reference to the foreign pointer
object returned by 'make-c-struct'.  It seems to me that the hard part
is deciding when these references can be released.  Once you've made
that determination, releasing N references is almost as easy as
releasing 1 reference.

Let me ask you this: in your current use case, what _should_ be the
policy on when to free those C strings?  What will become of that C
structure allocated with 'make-c-struct', and how will you find out when
the C code no longer needs it?

     Regards,
       Mark

Re: make-c-struct and pointer->string

[Mark H Weaver]

Hi David,

David Pirotte <david@altosw.be> writes:

>>  'make-c-struct' copies the C pointers from those foreign pointer objects, but not
>> not keep a reference to the objects themselves.
>
> To me, this sounds very counter intuitive, actually, it sounds like a bug,
> make-c-struct should be holding a reference to the pointers it receives: i seems to
> me that only when the c-struct itself becomes unreachable, that these pointers could
> be freed?

In my previous email I described some practical difficulties with
implementing this suggestion, but I should rather emphasize a more
important point:

'make-c-struct' is ultimately meant for passing data to C, and in
general you should assume that Boehm GC will _not_ see references from C
data structures to your GC-allocated objects.

In fact, Boehm GC does _not_ scan objects allocated using C 'malloc',
nor C++ 'new', nor 'make-c-struct'.  So, for purposes of garbage
collection, you should normally assume that passing pointers to C land
is equivalent to passing them to a block hole.

In general, when passing GC-allocated objects to C code, unless the C
code in question was written with Boehm GC in mind, you must always
think about how long the object must be kept alive, and arrange to keep
a reference to it for at least that long, and preferably no longer than
necessary or else you'll have space leaks.

I'm open to suggestions of how to make this better, although I'm also
fairly confident that interfacing between GC-managed and non-GC-managed
code is a thorny problem with no silver bullets.

       Mark

Re: make-c-struct and pointer->string

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 2051 bytes --]

Hello Mark,

> >>  'make-c-struct' copies the C pointers from those foreign pointer objects, but
> >> not not keep a reference to the objects themselves.  

> > To me, this sounds very counter intuitive, actually, it sounds like a bug,
> > make-c-struct should be holding a reference to the pointers it receives: i seems
> > to me that only when the c-struct itself becomes unreachable, that these
> > pointers could be freed?  

> In my previous email I described some practical difficulties with
> implementing this suggestion, but I should rather emphasize a more
> important point:

> 'make-c-struct' is ultimately meant for passing data to C, and in
> general you should assume that Boehm GC will _not_ see references from C
> data structures to your GC-allocated objects.

> In fact, Boehm GC does _not_ scan objects allocated using C 'malloc',
> nor C++ 'new', nor 'make-c-struct'.  So, for purposes of garbage
> collection, you should normally assume that passing pointers to C land
> is equivalent to passing them to a block hole.

> In general, when passing GC-allocated objects to C code, unless the C
> code in question was written with Boehm GC in mind, you must always
> think about how long the object must be kept alive, and arrange to keep
> a reference to it for at least that long, and preferably no longer than
> necessary or else you'll have space leaks.

> I'm open to suggestions of how to make this better, although I'm also
> fairly confident that interfacing between GC-managed and non-GC-managed
> code is a thorny problem with no silver bullets.

Answering the second of your 2 emails, but I did read the first as well ...

I agree with you, I think I somehow was only looking at one (of the many) way users
may use make-c-struct/parse-c-struct, and indeed I now see that my request would
even lead to mem problem, since most of the time, C may change these pointers or any
other struct field(s) content ...

Thanks again for your time and very good explanations.

Cheers,
David

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]