how to access c-array member in c-structure?

how to access c-array member in c-structure?

[Park SungMin]

I read https://www.gnu.org/software/guile/manual/html_node/Foreign-Structs.html ..

follow c struct

struct foo {
    int a;
    int b[4];
}
then 
(let ((foo-type (list int (make-list 4 int))))
   (parse-c-struct (make-c-struct foo-type (list 1 (list 1 2 3 4))) foo-type))
is run well.

but,
typedef struct {
  int a;
  int b;
  int c[20000];
} Foo;

Foo make_foo(int a, int b) {
  Foo foo = {.a = a, .b = b};
  return foo;
}

(let ((c-type (list int int (make-list 20000 int))))
    (parse-c-struct
     ((pointer->procedure
       c-type
       (dynamic-func "make_foo" libfoo)
       (list int int))
      4 11)
     c-type))

also run well…but seems less effective.(if more bigger size…2000000??) 
 is it possible pass or return as bytevector or pointer to c-array?
(let ((c-type (list int int ‘*)) ….)  is different type….(when i try it, guile shutdown)

Re: how to access c-array member in c-structure?

[Taylan Ulrich Bayırlı/Kammer]

Park SungMin <byulparan_eng@icloud.com> writes:

> (let ((c-type (list int int (make-list 20000 int))))
>     (parse-c-struct
>      ((pointer->procedure
>        c-type
>        (dynamic-func "make_foo" libfoo)
>        (list int int))
>       4 11)
>      c-type))
>
> also run well…but seems less effective.(if more bigger size…2000000??) 

You could create the type object once at program startup and use it
many times.

    (define foo-type
      (list int int (make-list 2000000 int)))
    
    (define make-foo
      (pointer->procedure foo-type
                          (dynamic-func "make_foo" libfoo)
                          (list int int)))
    
    (parse-c-struct (make-foo 4 11) foo-type)
    (parse-c-struct (make-foo 12 5) foo-type)
    (parse-c-struct (make-foo 8 91) foo-type)
    ...

Taylan

Re: how to access c-array member in c-structure?

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Mar 30, 2016 at 10:03:28AM +0200, Taylan Ulrich Bayırlı/Kammer wrote:
> Park SungMin <byulparan_eng@icloud.com> writes:
> 
> > (let ((c-type (list int int (make-list 20000 int))))
> >     (parse-c-struct
> >      ((pointer->procedure
> >        c-type
> >        (dynamic-func "make_foo" libfoo)
> >        (list int int))
> >       4 11)
> >      c-type))
> >
> > also run well…but seems less effective.(if more bigger size…2000000??) 
> 
> You could create the type object once at program startup and use it
> many times.
> 
>     (define foo-type
>       (list int int (make-list 2000000 int)))

If I understood Park SungMin right, the problem is rather with the
representation of a huge array as a list?

regards
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlb7iU0ACgkQBcgs9XrR2kY8iwCdG3h68BefLs69cBsc0M0gC4l8
0WwAnRpSNt2YVQASj3SwCi9Ct34zUMm5
=fB2/
-----END PGP SIGNATURE-----

Re: how to access c-array member in c-structure?

[Taylan Ulrich Bayırlı/Kammer]

<tomas@tuxteam.de> writes:

> On Wed, Mar 30, 2016 at 10:03:28AM +0200, Taylan Ulrich Bayırlı/Kammer wrote:
>> Park SungMin <byulparan_eng@icloud.com> writes:
>> 
>> > (let ((c-type (list int int (make-list 20000 int))))
>> >     (parse-c-struct
>> >      ((pointer->procedure
>> >        c-type
>> >        (dynamic-func "make_foo" libfoo)
>> >        (list int int))
>> >       4 11)
>> >      c-type))
>> >
>> > also run well…but seems less effective.(if more bigger size…2000000??) 
>> 
>> You could create the type object once at program startup and use it
>> many times.
>> 
>>     (define foo-type
>>       (list int int (make-list 2000000 int)))
>
> If I understood Park SungMin right, the problem is rather with the
> representation of a huge array as a list?

Oh I see.

For me, the following works:

    (define type (list int (make-list 10 int)))

    (define struct (make-c-struct type (list 42 (make-list 10 42))))

    (parse-c-struct struct (list int '*))
      => (1 #<pointer 0x200000002>) 

Is it different when the struct really comes from the C library?

Taylan

Re: how to access c-array member in c-structure?

[Park SungMin]

>> If I understood Park SungMin right, the problem is rather with the
>> representation of a huge array as a list?

> 

correct! (sorry..my poor english)

I think huge c-array should be representation as bytevector or pointer object.

(define type (list int (make-list 10 int)))

(define struct (make-c-struct type (list 42 (make-list 10 42))))

(parse-c-struct struct (list int '*))
=> (42 #<pointer 0x2a0000002a>)

but….when i access to that pointer object
(pointer->bytevector (cadr (parse-c-struct struct (list int '*)))
		     (* 10 (sizeof int)))

=> shutdown guile!

with  my test C function..

typedef struct {
  int a;
  int b;
  int c[2000];
} Foo;

Foo make_foo(int a, int b) {
  Foo foo = {.a = a, .b = b};
  return foo;
}

((pointer->procedure
   (list int int '*)
   (dynamic-func "make_foo" libfoo)
   (list int int))
 10 10)

shutdown…. I also test in SBCL(cffi-libffi)
(cffi:defcstruct foo
  (a :int)
  (b :int)
  (c :int :count 2000))

(cffi:defcfun "make_foo" (:struct foo)
  (a :int)
  (b :int))

(make-foo 10 20)  => (C #.(SB-SYS:INT-SAP #X0CC6C0A8) B 20 A 10)

but when I change to 
(cffi:defcstruct foo
  (a :int)
  (b :int)
  (c :pointer))

when call (make-foo 10 20), then shut down.
but It seems consequent, array and pointer are different type.


> On Mar 30, 2016, at 6:03 PM, Taylan Ulrich Bayırlı/Kammer <taylanbayirli@gmail.com> wrote:
> 
> <tomas@tuxteam.de> writes:
> 
>> On Wed, Mar 30, 2016 at 10:03:28AM +0200, Taylan Ulrich Bayırlı/Kammer wrote:
>>> Park SungMin <byulparan_eng@icloud.com> writes:
>>> 
>>>> (let ((c-type (list int int (make-list 20000 int))))
>>>>    (parse-c-struct
>>>>     ((pointer->procedure
>>>>       c-type
>>>>       (dynamic-func "make_foo" libfoo)
>>>>       (list int int))
>>>>      4 11)
>>>>     c-type))
>>>> 
>>>> also run well…but seems less effective.(if more bigger size…2000000??) 
>>> 
>>> You could create the type object once at program startup and use it
>>> many times.
>>> 
>>>    (define foo-type
>>>      (list int int (make-list 2000000 int)))
>> 
>> If I understood Park SungMin right, the problem is rather with the
>> representation of a huge array as a list?
> 
> Oh I see.
> 
> For me, the following works:
> 
>    (define type (list int (make-list 10 int)))
> 
>    (define struct (make-c-struct type (list 42 (make-list 10 42))))
> 
>    (parse-c-struct struct (list int '*))
>      => (1 #<pointer 0x200000002>) 
> 
> Is it different when the struct really comes from the C library?
> 
> Taylan
> 

Re: how to access c-array member in c-structure?

[Taylan Ulrich Bayırlı/Kammer]

Park SungMin <byulparan_eng@icloud.com> writes:

>>> If I understood Park SungMin right, the problem is rather with the
>>> representation of a huge array as a list?
>
> correct! (sorry..my poor english)
>
> I think huge c-array should be representation as bytevector or pointer
> object.
>
> (define type (list int (make-list 10 int)))
>
> (define struct (make-c-struct type (list 42 (make-list 10 42))))
>
> (parse-c-struct struct (list int '*))
> => (42 #<pointer 0x2a0000002a>)
>
> but….when i access to that pointer object
> (pointer->bytevector (cadr (parse-c-struct struct (list int '*)))
> 		     (* 10 (sizeof int)))
>
> => shutdown guile!

I see.  The problem here is, a struct like

    struct {
      int a;
      int b;
      int c[2000];
    } foo;

declares a memory region of 2002 consecutive ints, whereas

    struct {
      int a;
      int b;
      int *c;
    } bar;

declares a memory region of two consecutive ints followed by one
pointer.  These are not the same thing.  The C language partly upholds
an illusion that they're same because "foo.c" implicitly returns a
pointer to the first integer in the array (i.e. the third integer in the
struct, after a and b).  On the other hand, "bar.c" returns the pointer
value that's stored in the struct.

When we tell Guile to treat foo like bar, it takes an int from the array
and treats it like a pointer.  So dereferencing it segfaults.

I don't know what a simple solution would be.

My bytestructures library *might* help, though it's not properly
integrated with the FFI yet.

    https://github.com/taylanub/scheme-bytestructures

Here's an example of how to use it in this case:

|> ,use (system foreign)
|> (define type (list int int (make-list 2000 int)))
|> (define struct (make-c-struct type (list 1 2 (make-list 2000 3))))
|> ,use (bytestructures guile)
|> (define bs-type (bs:struct `((a ,int) (b ,int) (c ,(bs:vector 2000 int)))))
WARNING: (guile-user): `int' imported from both (system foreign) and (bytestructures guile)
|> (define size (bytestructure-descriptor-size bs-type))
|> (define bs-struct
     (make-bytestructure (pointer->bytevector struct size) 0 bs-type))
|> (bytestructure-ref bs-struct 'a)
$2 = 1
|> (bytestructure-ref bs-struct 'b)
$3 = 2
|> (bytestructure-ref bs-struct 'c 0)
$5 = 3
|> (bytestructure-ref bs-struct 'c 1)
$6 = 3
|> (bytestructure-ref bs-struct 'c 2)
$7 = 3

As you see in the warning, the bytestructures library overrides some
variables of Guile's FFI library.  I usually import the FFI library with

    (use-modules ((system foreign) #:prefix ffi:))

to work around that issue.  (So now the original int is "ffi:int".)

I don't know if the library is ideal for your use-case, but if you
decide to use it or want to try it out, ask for help any time. :-)

Taylan

Re: how to access c-array member in c-structure?

[Park SungMin]

[-- Attachment #1: Type: text/plain, Size: 3299 bytes --]


Thank you!! this library is very good solution for me!
It seems very useful.


> On Mar 31, 2016, at 4:54 AM, Taylan Ulrich Bayırlı/Kammer <taylanbayirli@gmail.com> wrote:
> 
> Park SungMin <byulparan_eng@icloud.com <mailto:byulparan_eng@icloud.com>> writes:
> 
>>>> If I understood Park SungMin right, the problem is rather with the
>>>> representation of a huge array as a list?
>> 
>> correct! (sorry..my poor english)
>> 
>> I think huge c-array should be representation as bytevector or pointer
>> object.
>> 
>> (define type (list int (make-list 10 int)))
>> 
>> (define struct (make-c-struct type (list 42 (make-list 10 42))))
>> 
>> (parse-c-struct struct (list int '*))
>> => (42 #<pointer 0x2a0000002a>)
>> 
>> but….when i access to that pointer object
>> (pointer->bytevector (cadr (parse-c-struct struct (list int '*)))
>> 		     (* 10 (sizeof int)))
>> 
>> => shutdown guile!
> 
> I see.  The problem here is, a struct like
> 
>    struct {
>      int a;
>      int b;
>      int c[2000];
>    } foo;
> 
> declares a memory region of 2002 consecutive ints, whereas
> 
>    struct {
>      int a;
>      int b;
>      int *c;
>    } bar;
> 
> declares a memory region of two consecutive ints followed by one
> pointer.  These are not the same thing.  The C language partly upholds
> an illusion that they're same because "foo.c" implicitly returns a
> pointer to the first integer in the array (i.e. the third integer in the
> struct, after a and b).  On the other hand, "bar.c" returns the pointer
> value that's stored in the struct.
> 
> When we tell Guile to treat foo like bar, it takes an int from the array
> and treats it like a pointer.  So dereferencing it segfaults.
> 
> I don't know what a simple solution would be.
> 
> My bytestructures library *might* help, though it's not properly
> integrated with the FFI yet.
> 
>    https://github.com/taylanub/scheme-bytestructures <https://github.com/taylanub/scheme-bytestructures>
> 
> Here's an example of how to use it in this case:
> 
> |> ,use (system foreign)
> |> (define type (list int int (make-list 2000 int)))
> |> (define struct (make-c-struct type (list 1 2 (make-list 2000 3))))
> |> ,use (bytestructures guile)
> |> (define bs-type (bs:struct `((a ,int) (b ,int) (c ,(bs:vector 2000 int)))))
> WARNING: (guile-user): `int' imported from both (system foreign) and (bytestructures guile)
> |> (define size (bytestructure-descriptor-size bs-type))
> |> (define bs-struct
>     (make-bytestructure (pointer->bytevector struct size) 0 bs-type))
> |> (bytestructure-ref bs-struct 'a)
> $2 = 1
> |> (bytestructure-ref bs-struct 'b)
> $3 = 2
> |> (bytestructure-ref bs-struct 'c 0)
> $5 = 3
> |> (bytestructure-ref bs-struct 'c 1)
> $6 = 3
> |> (bytestructure-ref bs-struct 'c 2)
> $7 = 3
> 
> As you see in the warning, the bytestructures library overrides some
> variables of Guile's FFI library.  I usually import the FFI library with
> 
>    (use-modules ((system foreign) #:prefix ffi:))
> 
> to work around that issue.  (So now the original int is "ffi:int".)
> 
> I don't know if the library is ideal for your use-case, but if you
> decide to use it or want to try it out, ask for help any time. :-)
> 
> Taylan


[-- Attachment #2: Type: text/html, Size: 45598 bytes --]