wip: cdata module for dealing with C types in bytevectors

[Matt Wette <matt.wette@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6383 bytes --]

Hi All,

I have been working on a module, an alternative to Scheme Bytestructures, say, that
can be used for cross-platform development.  It's still in progress but I am pretty far along.
I thought I'd share some aspects of the design.  I am currently working on ctype-equal? and
adding test code that generates, compiles and calls C code.

Matt

Introduction
************

Matt Wette
June 2024

Introduction
============

The ‘cdata’ module, with its partner ‘arch-info’, provides a way to work
with data and associated types originating from C libraries.  It supports
non-native machine architectures using a global ‘*arch*’ parameter.
Size and alignment is tracked for all types.  There are type classes for
all C type classes: base, struct, union, array, pointer, and function.
Beyond size and alignment, base type objects carry a symbolic tag to
determine the appropriate bytevector procedures: there is no type
aliasing.  Support for C base types is handled by the ‘cbase’ type
object generator which converts them to underlying types.  For example,
on a 64 bit little endian architecture, ‘cbase’ would convert
‘uintptr_t’ to a base class ctype with underlying type info symbol
‘u64le’.

    The module is hopefully easy to use.  One uses the procedures
‘cbase’, ‘cstruct’, ‘cunion’, ‘cpointer’, ‘carray’, and ‘cfunction’ to
generate c-type objects, and then ‘make-cdata’ to generate C data
objects.  Access to component data is provided by the ‘cdata-ref’
procedure and mutation is accomplished via the ‘cdata-set!’ procedure.
With respect to Guile’s ffi interface, one can use ‘ctype->ffi’ to
convert to FFI type specifiers required for use of
‘foreign-library-function’.

    Example:
      (define t1 (cstruct '((tv_sec long) (tv_usec long))))
      (define gettod
        (foreign-library-function
         #f "gettimeofday"
         #:return-type (ctype->ffi (cbase 'int))
         #:arg-types (map ctype->ffi (list (cpointer t1) (cpointer 'void)))))

      (define d1 (make-cdata t1))
      (gettod (cdata-ref (cdata& d1)) %null-pointer)
      (format #t "time: ~s ~s\n" (cdata-ref d1 'tv_sec) (cdata-ref d1 'tv_usec))
      time: 1719062561 676365

Handling Machine Architectures
==============================

Needs love ...

      (define tx (with-arch 'riscv64
                   (cstruct '((a long) (b int)))))

Constructing Types
==================

The procedures used to create C types are described in the following.
Type construction is machine architecture (i.e., parameter ‘*arch*’)
dependent.

  -- Procedure: cbase name
      Creates a C base type, given the symbolic NAME for that type (e.g.,
      ‘unsigned-int’).

  -- Procedure: cstruct field-list [#:packed? pp]
      Creates a C struct with field list in the form ‘((name type) ...)’
      where NAME is a symbolic name (or ‘#f’) and TYPE is a c-type.
      Anonymous structs (and unions) are specified using ‘#f’ for the
      field name.  For packed structures add ‘#:packed #t’.

  -- Procedure: cunion field-list
      Creates a C union with field list in the same form as ‘cstruct’,
      except bitfield types are not allowed.

  -- Procedure: cpointer type
      Create a pointer type referencing TYPE.  A non-negative selector
      can be used with this type to generate an increment.  Say you have,
      in C, the variables ‘x’ defined from ‘int *x;’ which represents an
      array in memory.  Then the expression to reference the fourth
      element is ‘x[3]’ or, equivalently, ‘*(x+3)’.  In Guile, this
      expression would be ‘(cdata* (cdata-ref x 3))’.

  -- Procedure: carray type length
      Create an array of TYPE with LENGTH.  If LENGTH is zero, the array
      length is unbounded (so be careful).

  -- Procedure: cfunction ret-type arg-types
      This is work to go.

  -- Record: cbitfield type width
      This type is only created internally in the ‘cstruct’ procedure,
      but a ‘cdata-sel’ operation can yeild a result of this type which,
      when provided as argument to ‘cdata-ref’ or ‘cdata-set!’, will do
      the right thing.
           (define st (cstruct '((a int) (b short 3))))
           (define sd (make-cdata st))
           (define sd.b (cdata-sel sd 'b)) ;; => #<cdata 0x12345678 bitfield>
           (cdata-set! sd.b 3)
           (cdata-ref sd.b) => 3

    As a special case to deal with void pointers, the expression
‘(cpointer 'void)’ can be used.  Data object of this type would be
converted to Guiles ‘pointer’ type by the procedure ‘cdata->ffi’.  The
expression ‘(cbase 'void*)’ will generate the associated integer type
used for pointers.

Working with Data
=================

These procedures can be used to manipulate data.  They are not dependent
on the global ‘*arch*’ parameter.

  -- Procedure: make-cdata type [value]
      This procedure creates an object of type ‘<cdata>’.

  -- Procedure: cdata-ref data [tag ...]
      Return the Scheme value for cdata.  Since we provide no equivalent
      for structures in Scheme, cdata should have type class base,
      pointer, or array.  To get the value of a struct field, you should
      use the following form:
           (cdata-ref (cdata-ref struct-data 'a 'b 'c))

  -- Procedure: cdata-set! data value [tag ...]
           (cdata-set! struct-data 1 'a 'b 'c))

  -- Procedure: cdata-sel data tag ...
      Return a new ‘cdata’ object representing the associated selection.
      For example,
           dat1 -> <cdata 0x12345678 struct>
           (cdata-ref dat1 'a 'b 'c) -> <cdata 0x12345700> f64le>
      TODO: do we need something to return ‘(values bv ix ct)’

  -- Procecure: cdata* data
      to be documented: generates deferenced data object

  -- Procecure: cdata& data
      to be documented: generates pointer data object

Other Procedures
================

  -- Procedure: ctype-equal?
      to be documented This procedure is not dependent on ‘*arch*’.

Guile FFI Support
=================

  -- Procedure: ctype->ffi-type type
      Convert a _ctype_ to the (integer) code for the associated FFI
      type.

References
==========

   1. Guile Manual:<https://www.gnu.org/software/guile/manual>
   2. Scheme Bytestructures:
      <https://github.com/TaylanUB/scheme-bytestructures>


[-- Attachment #2: Type: text/html, Size: 6735 bytes --]