packed structures usefulness

packed structures usefulness

[Marco Maggi]

Ciao,
  I am in the position to add, with relatively
little effort, a module to GEE to handle
packed data structures. This means that,
as example of a still unimplemented API, doing:

(define *triplet*
  (make-gstruct-type '((one . int)
	       (two . double)
	       (three . (int64 . 3)))))

(define S (make-gstruct *triplet*))

(set! (one S) 123)
(set! (two S) 1.2)
(set! (three S 2) 44)

one can define a SMOB whose internal
representation is equivalent to the C language
type:

struct triplet {
  int one;
  double two;
  int64_t three[3];
};

and then access its fields.

Remembering that, IMHO, there is no way to
mimic the C structure fields alignment from a
Scheme level inteface, I wonder if such a module
would be useful or not.

The conversion between Scheme level values and
C level values takes time, so I am not sure about
the value of the reduced memory usage.

Notice that I have already written a module
to implement variable-length vectors of C language
types, so that way of saving memory is already
available to me.

--
Marco Maggi

"Now feel the funk blast!"
Rage Against the Machine - "Calm like a bomb"




_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: packed structures usefulness

[Ludovic Courtès]

Hi,

"Marco Maggi" <marco.maggi-ipsu@poste.it> writes:

>   I am in the position to add, with relatively
> little effort, a module to GEE to handle
> packed data structures. This means that,
> as example of a still unimplemented API, doing:
>
> (define *triplet*
>   (make-gstruct-type '((one . int)
> 	       (two . double)
> 	       (three . (int64 . 3)))))
>
> (define S (make-gstruct *triplet*))
>
> (set! (one S) 123)
> (set! (two S) 1.2)
> (set! (three S 2) 44)
>
> one can define a SMOB whose internal
> representation is equivalent to the C language
> type:
>
> struct triplet {
>   int one;
>   double two;
>   int64_t three[3];
> }; 
>
> and then access its fields.
>
> Remembering that, IMHO, there is no way to
> mimic the C structure fields alignment from a
> Scheme level inteface, I wonder if such a module
> would be useful or not.

I'm not convinced.  :-)

Most "modern" C libraries use opaque types, typically pointers to
structs, so you rarely get to access the fields directly.

(BTW, note that Guile's structs have a specified C layout in terms of
`SCM' fields, though.)

Thanks,
Ludovic.



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: packed structures usefulness

[Thien-Thi Nguyen]

() "Marco Maggi" <marco.maggi-ipsu@poste.it>
() Fri,  2 Nov 2007 09:12:05 +0100

   I wonder if such a module would be useful or not.

i think so, if it can handle char, short, and bitfields.
presuming it has length and signedness checks, it would
be nice to have those configurable, as well (ie., error
vs warn vs auto-normalize).

thi


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: packed structures usefulness

[Neil Jerram]

"Marco Maggi" <marco.maggi-ipsu@poste.it> writes:

> one can define a SMOB whose internal
> representation is equivalent to the C language
> type:
>
> struct triplet {
>   int one;
>   double two;
>   int64_t three[3];
> }; 

I implemented something like this for an application whose basic
control mechanism is message passing using flat C structures ("flat"
=> no pointers), because I wanted to be able to build and manipulate
the structures from Scheme.

I handled the padding issue by autogenerating structure definitions
like this (from the header files that define the structures):

struct struct_desc triplet_desc[] =
{
  { "one", offsetof(struct triplet, one), sizeof(one) },
  { "two", offsetof(struct triplet, two), sizeof(two) },
  ...
};

(It was more complex than shown here, to handle embedded structures,
array fields, etc., but I'm sure you get the idea.)

Then writing C code to read these definitions and export them to the
Scheme level.

I didn't use SMOBs.  On the Scheme level the structure is just a
uniform byte array, and there are procedures for reading and writing
fields (identified by name), using the structure definition
information.

So to answer your actual question: yes, I think this is useful, but
only in a rather specific kind of application context.

(One could also consider using this approach for reading and writing
binary network protocols, but that feels less robust to me than the
traditional approach of reading and writing a sequential byte stream.)

Regards,
        Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: packed structures usefulness

[Marco Maggi]

"ludo@gnu.org" wrote:
> "Marco Maggi" writes:
>> Remembering that, IMHO, there is no way to
>> mimic the C structure fields alignment from a
>> Scheme level inteface, I wonder if such a module
>> would be useful or not.
>
>I'm not convinced.  :-)
>
>Most  "modern"  C   libraries  use  opaque  types,
>typically pointers  to structs, so  you rarely get
>to access the fields directly.


"Thien-Thi Nguyen" wrote:
>> I wonder if such a module would be useful or not.
>
>i  think so,  if it  can handle  char,  short, and
>bitfields.  presuming it has length and signedness
>checks,   it   would  be   nice   to  have   those
>configurable,  as  well  (ie.,  error vs  warn  vs
>auto-normalize).


"Neil Jerram" wrote:
>I   implemented  something   like   this  for   an
>application  whose   basic  control  mechanism  is
>message passing using flat C structures ("flat" =>
>no pointers), because I wanted to be able to build
>and manipulate the structures from Scheme.
>[...]
>So to  answer your  actual question: yes,  I think
>this is useful, but only in a rather specific kind
>of application context.


(Thanks (expt 10 3)). I have decided to enqueue it
in  my  to  do  list.   I  will  write  it  as  an
independent module, but  with the specific purpose
of  having a tool  to write  an interface  to Ralf
Engelschall's OSSP MM: a library for shared memory
usage [1].

It is a small project  that I started one year ago
and then  put aside, because  I had no  clear idea
about  how  to  define  a "mask"  of  data  fields
through  which  "look"  at  the  allocated  shared
memory.

I will use (sub)SMOBs and define the struct format
from the Scheme level only,  I do not want to make
it  a way  to "see"  a foreign  C struct  from the
Scheme  level.  There  should  be  no  problem  to
register   an  assertion   function   and  set/get
conversion functions.

But...  ahem...  bitfields?  These are *boring* to
code... ;-)


[1] <http://www.ossp.org/pkg/lib/mm/>

--
Marco Maggi

"Now feel the funk blast!"
Rage Against the Machine - "Calm like a bomb"




_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: packed structures usefulness

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 278 bytes --]

() "Marco Maggi" <marco.maggi-ipsu@poste.it>
() Tue,  6 Nov 2007 21:53:31 +0100

   But...  ahem...  bitfields?  These are *boring* to
   code... ;-)

you're right.  forget i mentioned it.
i now[0] think byte-granularity is enough.

thi

_______________________________________

[-- Attachment #2: work in progress X-ok-C-no --]
[-- Type: application/x-tar, Size: 94365 bytes --]

[-- Attachment #3: Type: text/plain, Size: 140 bytes --]

_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user