* make-c-struct and pointer->string @ 2019-03-26 13:14 David Pirotte 2019-03-28 15:15 ` Amirouche Boubekki 2019-03-30 20:26 ` Mark H Weaver 0 siblings, 2 replies; 12+ messages in thread From: David Pirotte @ 2019-03-26 13:14 UTC (permalink / raw) To: guile-devel [-- Attachment #1: Type: text/plain, Size: 638 bytes --] Hello, guile 2.2.4.1-cdb19 ,use (system foreign) ;; this fails scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) $16 = #<pointer 0x55a3d54d54d0> scheme@(guile-user)> (parse-c-struct $16 (list '* '*)) $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>) scheme@(guile-user)> (map pointer->string $17) $18 = ("?g?գU" "`!?գU") ;; but this works scheme@(guile-user)> (map pointer->string (parse-c-struct (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) (list '* '*))) $19 = ("hello " "there!") David [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-26 13:14 make-c-struct and pointer->string David Pirotte @ 2019-03-28 15:15 ` Amirouche Boubekki 2019-03-28 15:42 ` Neil Jerram 2019-03-28 15:48 ` David Pirotte 2019-03-30 20:26 ` Mark H Weaver 1 sibling, 2 replies; 12+ messages in thread From: Amirouche Boubekki @ 2019-03-28 15:15 UTC (permalink / raw) To: David Pirotte; +Cc: guile-devel [-- Attachment #1: Type: text/plain, Size: 1892 bytes --] Eventually, I can reproduced the issue with guile-next from guix: scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) $5 = #<pointer 0x1d1ca70> scheme@(guile-user) [1]> (parse-c-struct $5 (list '* '*)) $6 = (#<pointer 0x166a950> #<pointer 0x166a9f0>) scheme@(guile-user) [1]> (map pointer->string $6) $7 = ("hello " "there!") scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello") (string->pointer "there!"))) $8 = #<pointer 0x1ce2a40> scheme@(guile-user) [1]> (parse-c-struct $8 (list '* '*)) $9 = (#<pointer 0x166a810> #<pointer 0x166a910>) scheme@(guile-user) [1]> (map pointer->string $9) $10 = ("hello" "there!") scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello") (string->pointer "there!"))) $11 = #<pointer 0x1ce2a10> scheme@(guile-user) [1]> (parse-c-struct $11 (list '* '*)) $12 = (#<pointer 0x166a850> #<pointer 0x166a690>) scheme@(guile-user) [1]> (map pointer->string $12) $13 = ("?<\"?\x0e\x7f" "@?f\x01") scheme@(guile-user) [1]> scheme@(guile-user)> Le mar. 26 mars 2019 à 14:19, David Pirotte <david@altosw.be> a écrit : > Hello, > > guile 2.2.4.1-cdb19 > > ,use (system foreign) > > ;; this fails > > scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer > "hello ") (string->pointer "there!"))) > $16 = #<pointer 0x55a3d54d54d0> > scheme@(guile-user)> (parse-c-struct $16 (list '* '*)) > $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>) > scheme@(guile-user)> (map pointer->string $17) > $18 = ("?g?գU" "`!?գU") > > ;; but this works > > scheme@(guile-user)> (map pointer->string (parse-c-struct (make-c-struct > (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) > (list '* '*))) > $19 = ("hello " "there!") > > > David > [-- Attachment #2: Type: text/html, Size: 2757 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-28 15:15 ` Amirouche Boubekki @ 2019-03-28 15:42 ` Neil Jerram 2019-03-29 8:48 ` David Pirotte 2019-03-28 15:48 ` David Pirotte 1 sibling, 1 reply; 12+ messages in thread From: Neil Jerram @ 2019-03-28 15:42 UTC (permalink / raw) To: Amirouche Boubekki; +Cc: guile-devel, David Pirotte On Thu, 28 Mar 2019 at 15:15, Amirouche Boubekki <amirouche.boubekki@gmail.com> wrote: > > Eventually, I can reproduced the issue with guile-next from guix: > > scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) > $5 = #<pointer 0x1d1ca70> > scheme@(guile-user) [1]> (parse-c-struct $5 (list '* '*)) > $6 = (#<pointer 0x166a950> #<pointer 0x166a9f0>) > scheme@(guile-user) [1]> (map pointer->string $6) > $7 = ("hello " "there!") > scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello") (string->pointer "there!"))) > $8 = #<pointer 0x1ce2a40> > scheme@(guile-user) [1]> (parse-c-struct $8 (list '* '*)) > $9 = (#<pointer 0x166a810> #<pointer 0x166a910>) > scheme@(guile-user) [1]> (map pointer->string $9) > $10 = ("hello" "there!") > scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello") (string->pointer "there!"))) > $11 = #<pointer 0x1ce2a10> > scheme@(guile-user) [1]> (parse-c-struct $11 (list '* '*)) > $12 = (#<pointer 0x166a850> #<pointer 0x166a690>) > scheme@(guile-user) [1]> (map pointer->string $12) > $13 = ("?<\"?\x0e\x7f" "@?f\x01") > scheme@(guile-user) [1]> > scheme@(guile-user)> > > > Le mar. 26 mars 2019 à 14:19, David Pirotte <david@altosw.be> a écrit : >> >> Hello, >> >> guile 2.2.4.1-cdb19 >> >> ,use (system foreign) >> >> ;; this fails >> >> scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) >> $16 = #<pointer 0x55a3d54d54d0> >> scheme@(guile-user)> (parse-c-struct $16 (list '* '*)) >> $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>) >> scheme@(guile-user)> (map pointer->string $17) >> $18 = ("?g?գU" "`!?գU") >> >> ;; but this works >> >> scheme@(guile-user)> (map pointer->string (parse-c-struct (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) (list '* '*))) >> $19 = ("hello " "there!") I guess it may be GC. I've had problems with code like (procedure->pointer void (lambda (...) ...) args...) because procedure->pointer does not protect the (lambda ...) from being GC'd. For example: http://git.savannah.nongnu.org/cgit/ossaulib.git/commit/?id=1d31a87b53e9375b5fdd5120b7d409e6dde6aa47 I wonder if there's a similar problem here with the string->pointer data. Best wishes, Neil ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-28 15:42 ` Neil Jerram @ 2019-03-29 8:48 ` David Pirotte 2019-03-29 9:46 ` David Pirotte 0 siblings, 1 reply; 12+ messages in thread From: David Pirotte @ 2019-03-29 8:48 UTC (permalink / raw) To: Neil Jerram; +Cc: guile-devel [-- Attachment #1: Type: text/plain, Size: 1347 bytes --] Hello Neil, > I guess it may be GC. I've had problems with code like > > (procedure->pointer void (lambda (...) ...) args...) > > because procedure->pointer does not protect the (lambda ...) from being GC'd. > > For example: > http://git.savannah.nongnu.org/cgit/ossaulib.git/commit/?id=1d31a87b53e9375b5fdd5120b7d409e6dde6aa47 > > I wonder if there's a similar problem here with the string->pointer data. Thanks for your help! Following your explanation and example, I tried this and thought it would work then, but it also failed: GNU Guile 2.2.4.1-cdb19 Copyright (C) 1995-2017 Free Software Foundation, Inc. Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. This program is free software, and you are welcome to redistribute it under certain conditions; type `,show c' for details. Enter `,help' for help. scheme@(guile-user)> ,use (system foreign) scheme@(guile-user)> (define str-1 "Hello") scheme@(guile-user)> (define str-2 "there!") scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer str-1) (string->pointer str-2))) $2 = #<pointer 0x55ae02e57830> scheme@(guile-user)> (parse-c-struct $2 (list '* '*)) $3 = (#<pointer 0x55ae02f9e3c0> #<pointer 0x55ae02f8b050>) scheme@(guile-user)> (map pointer->string $3) $4 = ("" "`\v?\x02?U") Cheers, David [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-29 8:48 ` David Pirotte @ 2019-03-29 9:46 ` David Pirotte 2019-03-29 19:08 ` Neil Jerram 0 siblings, 1 reply; 12+ messages in thread From: David Pirotte @ 2019-03-29 9:46 UTC (permalink / raw) To: Neil Jerram; +Cc: guile-devel [-- Attachment #1: Type: text/plain, Size: 2987 bytes --] Hi again, > ... > Following your explanation and example, I tried this and thought it would work > then, but it also failed: > GNU Guile 2.2.4.1-cdb19 > Enter `,help' for help. > scheme@(guile-user)> ,use (system foreign) > scheme@(guile-user)> (define str-1 "Hello") > scheme@(guile-user)> (define str-2 "there!") > scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer str-1) > (string->pointer str-2))) $2 = #<pointer 0x55ae02e57830> > scheme@(guile-user)> (parse-c-struct $2 (list '* '*)) > $3 = (#<pointer 0x55ae02f9e3c0> #<pointer 0x55ae02f8b050>) > scheme@(guile-user)> (map pointer->string $3) > $4 = ("" "`\v?\x02?U") Here are another couple of examples: scheme@(guile-user)> ,use (system foreign) scheme@(guile-user)> (define str-1 "Hello") scheme@(guile-user)> (define str-2 "there!") scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer str-1) (string->pointer str-2))) $2 = #<pointer 0x55844635a0b0> scheme@(guile-user)> (parse-c-struct $2 (list '* '*)) $3 = (#<pointer 0x558445edf610> #<pointer 0x558445edf630>) scheme@(guile-user)> (map pointer->string $3) $4 = ("Hello" "there!") But then: scheme@(guile-user)> str-1 $5 = "Hello" scheme@(guile-user)> str-2 $6 = "there!" scheme@(guile-user)> (map pointer->string $3) $7 = ("?\\?q\x7f" " ??E?U") scheme@(guile-user)> (map pointer->string $3) $8 = ("?\\?q\x7f" " ??E?U") I guess that what the GC recollected are the pointers, or even $3, the list Let's hold on it then: scheme@(guile-user)> (define c-struct (make-c-struct (list '* '*) (list (string->pointer str-1) (string->pointer str-2)))) scheme@(guile-user)> (parse-c-struct c-struct (list '* '*)) $9 = (#<pointer 0x558445edf400> #<pointer0x558445ef8df0>) scheme@(guile-user)> (map pointer->string $9) $10 = ("\x02" "0??E?U") Not good either. Let's hold on the parse-c-struct to then: scheme@(guile-user)> (define parsed (parse-c-struct c-struct (list '* '*))) scheme@(guile-user)> parsed $16 = (#<pointer 0x558445edf400> #<pointer 0x558445ef8df0>) scheme@(guile-user)> (map pointer->string $16) $17 = ("0??E?U" "?\x01?F?U") scheme@(guile-user)> (map pointer->string parsed) $18 = ("\x03" "?\x01?F?U") Didn't do it either Ok, let's hold-on to the pointers (which I thought would not be necessary after holding-on c-struct and/or parsed): scheme@(guile-user)> (define p1 (string->pointer str-1)) scheme@(guile-user)> (define p2 (string->pointer str-2)) scheme@(guile-user)> (make-c-struct (list '* '*) (list p1 p2)) $19 = #<pointer 0x5584467a5d70> scheme@(guile-user)> (parse-c-struct $19 (list '* '*)) $20 = (#<pointer 0x5584468b0190> #<pointer 0x5584468b4430>) scheme@(guile-user)> p1 $21 = #<pointer 0x5584468b0190> scheme@(guile-user)> p2 $22 = #<pointer 0x5584468b4430> scheme@(guile-user)> (map pointer->string $20) $23 = ("Hello" "there!") That worked, but I wonder if this is what is expected from us (users)? Cheers, David [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-29 9:46 ` David Pirotte @ 2019-03-29 19:08 ` Neil Jerram 0 siblings, 0 replies; 12+ messages in thread From: Neil Jerram @ 2019-03-29 19:08 UTC (permalink / raw) To: David Pirotte; +Cc: guile-devel [-- Attachment #1: Type: text/plain, Size: 3345 bytes --] On Fri, 29 Mar 2019, 09:46 David Pirotte, <david@altosw.be> wrote: > Hi again, > > > ... > > Following your explanation and example, I tried this and thought it > would work > > then, but it also failed: > > > GNU Guile 2.2.4.1-cdb19 > > > Enter `,help' for help. > > scheme@(guile-user)> ,use (system foreign) > > scheme@(guile-user)> (define str-1 "Hello") > > scheme@(guile-user)> (define str-2 "there!") > > scheme@(guile-user)> (make-c-struct (list '* '*) (list > (string->pointer str-1) > > (string->pointer str-2))) $2 = #<pointer 0x55ae02e57830> > > scheme@(guile-user)> (parse-c-struct $2 (list '* '*)) > > $3 = (#<pointer 0x55ae02f9e3c0> #<pointer 0x55ae02f8b050>) > > scheme@(guile-user)> (map pointer->string $3) > > $4 = ("" "`\v?\x02?U") > > Here are another couple of examples: > > scheme@(guile-user)> ,use (system foreign) > scheme@(guile-user)> (define str-1 "Hello") > scheme@(guile-user)> (define str-2 "there!") > scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer > str-1) (string->pointer str-2))) > $2 = #<pointer 0x55844635a0b0> > scheme@(guile-user)> (parse-c-struct $2 (list '* '*)) > $3 = (#<pointer 0x558445edf610> #<pointer 0x558445edf630>) > scheme@(guile-user)> (map pointer->string $3) > $4 = ("Hello" "there!") > > But then: > > scheme@(guile-user)> str-1 > $5 = "Hello" > scheme@(guile-user)> str-2 > $6 = "there!" > scheme@(guile-user)> (map pointer->string $3) > $7 = ("?\\?q\x7f" " ??E?U") > scheme@(guile-user)> (map pointer->string $3) > $8 = ("?\\?q\x7f" " ??E?U") > > I guess that what the GC recollected are the pointers, or even $3, the list > Let's hold on it then: > > scheme@(guile-user)> (define c-struct (make-c-struct (list '* '*) (list > (string->pointer str-1) (string->pointer str-2)))) > scheme@(guile-user)> (parse-c-struct c-struct (list '* '*)) > $9 = (#<pointer 0x558445edf400> #<pointer0x558445ef8df0>) > scheme@(guile-user)> (map pointer->string $9) > $10 = ("\x02" "0??E?U") > > Not good either. Let's hold on the parse-c-struct to then: > > scheme@(guile-user)> (define parsed (parse-c-struct c-struct (list '* > '*))) > scheme@(guile-user)> parsed > $16 = (#<pointer 0x558445edf400> #<pointer 0x558445ef8df0>) > scheme@(guile-user)> (map pointer->string $16) > $17 = ("0??E?U" "?\x01?F?U") > scheme@(guile-user)> (map pointer->string parsed) > $18 = ("\x03" "?\x01?F?U") > > Didn't do it either > Ok, let's hold-on to the pointers (which I thought would not be necessary > after > holding-on c-struct and/or parsed): > > scheme@(guile-user)> (define p1 (string->pointer str-1)) > scheme@(guile-user)> (define p2 (string->pointer str-2)) > scheme@(guile-user)> (make-c-struct (list '* '*) (list p1 p2)) > $19 = #<pointer 0x5584467a5d70> > scheme@(guile-user)> (parse-c-struct $19 (list '* '*)) > $20 = (#<pointer 0x5584468b0190> #<pointer 0x5584468b4430>) > scheme@(guile-user)> p1 > $21 = #<pointer 0x5584468b0190> > scheme@(guile-user)> p2 > $22 = #<pointer 0x5584468b4430> > scheme@(guile-user)> (map pointer->string $20) > $23 = ("Hello" "there!") > > That worked, but I wonder if this is what is expected from us (users)? > Yes, that would seem very odd to me, as I thought that a 'pointer' really was just an address. (Hopefully some more light will be thrown when the maintainers are back around again!) Best wishes, Neil > Cheers, > David > [-- Attachment #2: Type: text/html, Size: 4804 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-28 15:15 ` Amirouche Boubekki 2019-03-28 15:42 ` Neil Jerram @ 2019-03-28 15:48 ` David Pirotte 1 sibling, 0 replies; 12+ messages in thread From: David Pirotte @ 2019-03-28 15:48 UTC (permalink / raw) To: Amirouche Boubekki; +Cc: guile-devel [-- Attachment #1: Type: text/plain, Size: 2751 bytes --] Hi Amirouche, I should add to your own answer that you actually were unable to reproduce the problem while using the latest 2.2 release tarball (the one that Ubuntu packages to be (even) more precise), as one can see on your gnome paste earlier today [1] So, if this is corect, there must be a commit out there that 'broke it' Cheers, David here below, a copy/paste of https://paste.gnome.org/pbm5ok3pd#line-7 [ not sure how long gnome pastes remains visible .. [ in the paste below, which is not mine, I have no idea why, starting from [ line 15, guile entered a raised exception mode [1] amirouche ~ 12:59:20 guile GNU Guile 2.2.4 Copyright (C) 1995-2017 Free Software Foundation, Inc. Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. This program is free software, and you are welcome to redistribute it under certain conditions; type `,show c' for details. Enter `,help' for help. scheme@(guile-user)> (use-modules (system foreign)) scheme@(guile-user)> (define str-1 (string->pointer "Hello")) scheme@(guile-user)> (define str-2 (string->pointer "there!")) scheme@(guile-user)> (make-c-struct (list '* '*) (list str-1 str-2)) $1 = #<pointer 0x55f30df62dd0> scheme@(guile-user) [1]> (parse-c-struct $1 (list '* '*)) $2 = (#<pointer 0x55f30de21510> #<pointer 0x55f30de21f90>) scheme@(guile-user) [1]> str-1 $3 = #<pointer 0x55f30de21510> scheme@(guile-user) [1]> (map pointer->string $2) $4 = ("Hello" "there!") scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) $5 = #<pointer 0x55f30e235aa0> scheme@(guile-user) [1]> (parse-c-struct $5 (list '* '*)) $6 = (#<pointer 0x55f30de25e80> #<pointer 0x55f30de26a30>) scheme@(guile-user) [1]> (map pointer->string $6) $7 = ("hello " "there!") scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello") (string->pointer "there!"))) $8 = #<pointer 0x55f30e076470> scheme@(guile-user) [1]> (parse-c-struct $8 (list '* '*)) $9 = (#<pointer 0x55f30de28020> #<pointer 0x55f30de28bd0>) scheme@(guile-user) [1]> (map pointer->string $9) $10 = ("hello" "there!") scheme@(guile-user) [1]> (make-c-struct (list '* '*) (list (string->pointer "hello") (string->pointer "there!"))) $11 = #<pointer 0x55f30e0453e0> scheme@(guile-user) [1]> (parse-c-struct $11 (list '* '*)) $12 = (#<pointer 0x55f30de2a170> #<pointer 0x55f30de28bd0>) scheme@(guile-user) [1]> (map pointer->string $12) $13 = ("hello" "there!") scheme@(guile-user) [1]> Posted by Anonymous at 28 Mar 2019, 12:33:15 UTC Language: scheme • Views: 4 © The GNOME Project GNOME Pastebin. Hosted by Red Hat. Powered by Sticky Notes © 2014 Sayak Banerjee. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-26 13:14 make-c-struct and pointer->string David Pirotte 2019-03-28 15:15 ` Amirouche Boubekki @ 2019-03-30 20:26 ` Mark H Weaver 2019-03-31 10:38 ` David Pirotte 1 sibling, 1 reply; 12+ messages in thread From: Mark H Weaver @ 2019-03-30 20:26 UTC (permalink / raw) To: David Pirotte; +Cc: guile-devel Hi David, David Pirotte <david@altosw.be> writes: > Hello, > > guile 2.2.4.1-cdb19 > > ,use (system foreign) > > ;; this fails > > scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer "hello ") (string->pointer "there!"))) > $16 = #<pointer 0x55a3d54d54d0> > scheme@(guile-user)> (parse-c-struct $16 (list '* '*)) > $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>) > scheme@(guile-user)> (map pointer->string $17) > $18 = ("?g?գU" "`!?գU") The Guile manual states: -- Scheme Procedure: string->pointer string [encoding] Return a foreign pointer to a nul-terminated copy of STRING in the given ENCODING, defaulting to the current locale encoding. The C string is freed when the returned foreign pointer becomes unreachable. Note the last sentence. When the returned foreign pointer (object) becomes unreachable, the C string is freed. The problem here is that you're not keeping a reference to those foreign pointer objects. If you look at the code in foreign.c, specifically 'scm_string_to_pointer' and 'scm_from_pointer', you'll see that the C pointer is wrapped within a heap-allocated foreign pointer object, and a finalizer is being set on that heap object. 'make-c-struct' copies the C pointers from those foreign pointer objects, but not not keep a reference to the objects themselves. In a later message, you wrote: > Following your explanation and example, I tried this and thought it would work then, > but it also failed: > > GNU Guile 2.2.4.1-cdb19 > Copyright (C) 1995-2017 Free Software Foundation, Inc. > > Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. > This program is free software, and you are welcome to redistribute it > under certain conditions; type `,show c' for details. > > Enter `,help' for help. > scheme@(guile-user)> ,use (system foreign) > scheme@(guile-user)> (define str-1 "Hello") > scheme@(guile-user)> (define str-2 "there!") > scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer str-1) (string->pointer str-2))) > $2 = #<pointer 0x55ae02e57830> > scheme@(guile-user)> (parse-c-struct $2 (list '* '*)) > $3 = (#<pointer 0x55ae02f9e3c0> #<pointer 0x55ae02f8b050>) > scheme@(guile-user)> (map pointer->string $3) > $4 = ("" "`\v?\x02?U") Here are you specifically saving a reference to the Scheme string objects, but that's not what's needed. What's needed is to hold references to the foreign pointer objects returned by 'string->pointer' for as long as the C strings are needed. Regards, Mark ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-30 20:26 ` Mark H Weaver @ 2019-03-31 10:38 ` David Pirotte 2019-04-01 6:23 ` Mark H Weaver 2019-04-01 8:50 ` Mark H Weaver 0 siblings, 2 replies; 12+ messages in thread From: David Pirotte @ 2019-03-31 10:38 UTC (permalink / raw) To: Mark H Weaver; +Cc: guile-devel [-- Attachment #1: Type: text/plain, Size: 2954 bytes --] Hi Mark, Thanks for your help. > > ... > > scheme@(guile-user)> (make-c-struct (list '* '*) (list (string->pointer "hello > > ") (string->pointer "there!"))) $16 = #<pointer 0x55a3d54d54d0> > > scheme@(guile-user)> (parse-c-struct $16 (list '* '*)) > > $17 = (#<pointer 0x55a3d5d12170> #<pointer 0x55a3d5d0a640>) > > scheme@(guile-user)> (map pointer->string $17) > > $18 = ("?g?գU" "`!?գU") > The Guile manual states: > -- Scheme Procedure: string->pointer string [encoding] > Return a foreign pointer to a nul-terminated copy of STRING in the > given ENCODING, defaulting to the current locale encoding. The C > string is freed when the returned foreign pointer becomes > unreachable. > Note the last sentence. When the returned foreign pointer (object) > becomes unreachable, the C string is freed. The problem here is that > you're not keeping a reference to those foreign pointer objects. Yes, and I was assuming that make-c-struct was holding a reference to 'its children', including pointers. > If you look at the code in foreign.c, specifically ... > 'make-c-struct' copies the C pointers from those foreign pointer objects, but not > not keep a reference to the objects themselves. To me, this sounds very counter intuitive, actually, it sounds like a bug, make-c-struct should be holding a reference to the pointers it receives: i seems to me that only when the c-struct itself becomes unreachable, that these pointers could be freed? If I am wrong, and it seems I am (wrong), we really should think about writing something about this in the manual (and explaining why would be nice). Cheers, David ps: in the last part of my email, I did precisely what you suggest in your answer, and showed that it works - if we hold a reference to these pointers 'by ourselves' - and asked if that was what is what was expected from us and Neil Jerram to (also) find that 'very odd' > > ... > > Ok, let's hold-on to the pointers (which I thought would not be necessary > > after holding-on c-struct and/or parsed): > > scheme@(guile-user)> (define p1 (string->pointer str-1)) > > scheme@(guile-user)> (define p2 (string->pointer str-2)) > > scheme@(guile-user)> (make-c-struct (list '* '*) (list p1 p2)) > > $19 = #<pointer 0x5584467a5d70> > > scheme@(guile-user)> (parse-c-struct $19 (list '* '*)) > > $20 = (#<pointer 0x5584468b0190> #<pointer 0x5584468b4430>) > > scheme@(guile-user)> p1 > > $21 = #<pointer 0x5584468b0190> > > scheme@(guile-user)> p2 > > $22 = #<pointer 0x5584468b4430> > > scheme@(guile-user)> (map pointer->string $20) > > $23 = ("Hello" "there!") > > That worked, but I wonder if this is what is expected from us (users)? > Yes, that would seem very odd to me, as I thought that a 'pointer' really > was just an address. > (Hopefully some more light will be thrown when the maintainers are back > around again!) [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 484 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-31 10:38 ` David Pirotte @ 2019-04-01 6:23 ` Mark H Weaver 2019-04-01 8:50 ` Mark H Weaver 1 sibling, 0 replies; 12+ messages in thread From: Mark H Weaver @ 2019-04-01 6:23 UTC (permalink / raw) To: David Pirotte; +Cc: guile-devel Hi David, David Pirotte <david@altosw.be> writes: >> 'make-c-struct' copies the C pointers from those foreign pointer objects, but not >> not keep a reference to the objects themselves. > > To me, this sounds very counter intuitive, actually, it sounds like a bug, > make-c-struct should be holding a reference to the pointers it receives: i seems to > me that only when the c-struct itself becomes unreachable, that these pointers could > be freed? It's a reasonable suggestion, and I agree that the documentation should be clarified on these points, which are quite tricky. I thought about what would be required to implement your suggestion and quickly ran into difficulties, and also the understanding that this is not always the right behavior. First of all, the address of the foreign pointer object is what we would need to keep a reference to, and that's not the same as the C pointer wrapped within that foreign pointer object. This raises the question of where to store these additional references, since we can't store them in any of the C struct fields defined by the user. The worse problem is that there's no way to keep these extra references in sync with the C pointers in the C struct. The C code might overwrite one of those pointers, and move them to a different structure somewhere else. Also note that in many cases, foreign pointer objects are used only transiently to copy a C pointer from one place to another. If we implemented your suggestion, these otherwise transient foreign pointer objects would be kept alive for no purpose. The other thing is that even if we had these suggested semantics, you would still need to arrange to hold a reference to the foreign pointer object returned by 'make-c-struct'. It seems to me that the hard part is deciding when these references can be released. Once you've made that determination, releasing N references is almost as easy as releasing 1 reference. Let me ask you this: in your current use case, what _should_ be the policy on when to free those C strings? What will become of that C structure allocated with 'make-c-struct', and how will you find out when the C code no longer needs it? Regards, Mark ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-03-31 10:38 ` David Pirotte 2019-04-01 6:23 ` Mark H Weaver @ 2019-04-01 8:50 ` Mark H Weaver 2019-04-02 18:03 ` David Pirotte 1 sibling, 1 reply; 12+ messages in thread From: Mark H Weaver @ 2019-04-01 8:50 UTC (permalink / raw) To: David Pirotte; +Cc: guile-devel Hi David, David Pirotte <david@altosw.be> writes: >> 'make-c-struct' copies the C pointers from those foreign pointer objects, but not >> not keep a reference to the objects themselves. > > To me, this sounds very counter intuitive, actually, it sounds like a bug, > make-c-struct should be holding a reference to the pointers it receives: i seems to > me that only when the c-struct itself becomes unreachable, that these pointers could > be freed? In my previous email I described some practical difficulties with implementing this suggestion, but I should rather emphasize a more important point: 'make-c-struct' is ultimately meant for passing data to C, and in general you should assume that Boehm GC will _not_ see references from C data structures to your GC-allocated objects. In fact, Boehm GC does _not_ scan objects allocated using C 'malloc', nor C++ 'new', nor 'make-c-struct'. So, for purposes of garbage collection, you should normally assume that passing pointers to C land is equivalent to passing them to a block hole. In general, when passing GC-allocated objects to C code, unless the C code in question was written with Boehm GC in mind, you must always think about how long the object must be kept alive, and arrange to keep a reference to it for at least that long, and preferably no longer than necessary or else you'll have space leaks. I'm open to suggestions of how to make this better, although I'm also fairly confident that interfacing between GC-managed and non-GC-managed code is a thorny problem with no silver bullets. Mark ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: make-c-struct and pointer->string 2019-04-01 8:50 ` Mark H Weaver @ 2019-04-02 18:03 ` David Pirotte 0 siblings, 0 replies; 12+ messages in thread From: David Pirotte @ 2019-04-02 18:03 UTC (permalink / raw) To: Mark H Weaver; +Cc: guile-devel [-- Attachment #1: Type: text/plain, Size: 2051 bytes --] Hello Mark, > >> 'make-c-struct' copies the C pointers from those foreign pointer objects, but > >> not not keep a reference to the objects themselves. > > To me, this sounds very counter intuitive, actually, it sounds like a bug, > > make-c-struct should be holding a reference to the pointers it receives: i seems > > to me that only when the c-struct itself becomes unreachable, that these > > pointers could be freed? > In my previous email I described some practical difficulties with > implementing this suggestion, but I should rather emphasize a more > important point: > 'make-c-struct' is ultimately meant for passing data to C, and in > general you should assume that Boehm GC will _not_ see references from C > data structures to your GC-allocated objects. > In fact, Boehm GC does _not_ scan objects allocated using C 'malloc', > nor C++ 'new', nor 'make-c-struct'. So, for purposes of garbage > collection, you should normally assume that passing pointers to C land > is equivalent to passing them to a block hole. > In general, when passing GC-allocated objects to C code, unless the C > code in question was written with Boehm GC in mind, you must always > think about how long the object must be kept alive, and arrange to keep > a reference to it for at least that long, and preferably no longer than > necessary or else you'll have space leaks. > I'm open to suggestions of how to make this better, although I'm also > fairly confident that interfacing between GC-managed and non-GC-managed > code is a thorny problem with no silver bullets. Answering the second of your 2 emails, but I did read the first as well ... I agree with you, I think I somehow was only looking at one (of the many) way users may use make-c-struct/parse-c-struct, and indeed I now see that my request would even lead to mem problem, since most of the time, C may change these pointers or any other struct field(s) content ... Thanks again for your time and very good explanations. Cheers, David [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-04-02 18:03 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-03-26 13:14 make-c-struct and pointer->string David Pirotte 2019-03-28 15:15 ` Amirouche Boubekki 2019-03-28 15:42 ` Neil Jerram 2019-03-29 8:48 ` David Pirotte 2019-03-29 9:46 ` David Pirotte 2019-03-29 19:08 ` Neil Jerram 2019-03-28 15:48 ` David Pirotte 2019-03-30 20:26 ` Mark H Weaver 2019-03-31 10:38 ` David Pirotte 2019-04-01 6:23 ` Mark H Weaver 2019-04-01 8:50 ` Mark H Weaver 2019-04-02 18:03 ` David Pirotte
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).