Re: Bytestructures: a "type system" for bytevectors

Re: Bytestructures: a "type system" for bytevectors

[Matt Wette]

 
> On Jun 21, 2016, at 5:53 AM, Matt Wette <matt.wette@gmail.com> wrote:
> 
>> On Jun 21, 2016, at 12:50 AM, Taylan Ulrich Bayırlı/Kammer <taylanbayirli@gmail.com> wrote:
>> 
>> Matt Wette <matt.wette@gmail.com> writes:
>> 
>>> nyacc is an all-guile implementation of yacc and comes with a c99
>>> parser, available from www.nongnu.org. 
>>> The parser outputs parse trees in sxml format. It is beta-level code.
>>> 
>>> Matt
>> 
>> Wow!  That covers a big chunk of the task, if I implement it from
>> scratch.  In fact, given I don't have to deal with typedefs and such for
>> doing something like Lua's FFI, it covers most of the task.
> 
> There is code to expand typedefs.   Check “stripdown” routine in (nyacc lang c99 util2).  It also provides a keyword arg (#:keep) to provide a list of typedefs to not expand.

I should expand a little.  The nyacc c99 parser provides functionality that I think will be valuable to FFI:
1) preserves file context: the parse tree for included files is stuffed under an associated cpp-stmt include node
2) provides utility to expand typedef references into base types (with “keepers” arg to select which typedef references to preserve)
3) provides parser argument to choose which defines get expanded
4) provides utility to unwrap declarations

There is still a lot to do to support FFI.  E.g., what to do about system dependencies (e.g., is “int" 32 bits or 64 bits)

My stuff is all still pretty rough right now.   Here is some demo code for features (1) and (2).  

;; demo.scm

(use-modules (nyacc lang c99 parser))
(use-modules (nyacc lang c99 util1))
(use-modules (nyacc lang c99 util2))
(use-modules (ice-9 pretty-print))

(let* ((tree (with-input-from-file "ex1.h" parse-c99))
       (ud1 (reverse (tree->udict tree)))) ;; decl's only in ex1.h
  (display "\ntree:\n")
  (pretty-print tree)
  (display "\nud1:\n")
  (pretty-print ud1)
  (let ((ud0 (tree->udict (merge-inc-trees! tree)))) ;; decl's in all
    (for-each
     (lambda (pair)
       (let* ((udecl (cdr pair))
	      (tdexp (expand-decl-typerefs udecl ud0 #:keep '("int32_t")))
	      (mspec (udecl->mspec tdexp)))
	 (display "\ntdexp:\n")
	 (pretty-print tdexp)))
     ud1)))

=================================================
// ex1.h
#include "ex0.h"

typedef struct {
  double d;
  foo_t x;
} bar_t;

bar_t ftn1(int*);
======================================================
// ex0.h
typedef int int32_t;
typedef int32_t foo_t;
======================================================
tree:
(trans-unit
  (comment " ex1.h")
  (cpp-stmt
    (include
      "\"ex0.h\""
      (trans-unit
        (comment " ex0.h")
        (decl (decl-spec-list
                (stor-spec (typedef))
                (type-spec (fixed-type "int")))
              (init-declr-list (init-declr (ident "int32_t"))))
        (decl (decl-spec-list
                (stor-spec (typedef))
                (type-spec (typename "int32_t")))
              (init-declr-list (init-declr (ident "foo_t")))))))
  (decl (decl-spec-list
          (stor-spec (typedef))
          (type-spec
            (struct-def
              (field-list
                (comp-decl
                  (decl-spec-list
                    (type-spec (float-type "double")))
                  (comp-declr-list (comp-declr (ident "d"))))
                (comp-decl
                  (decl-spec-list (type-spec (typename "foo_t")))
                  (comp-declr-list (comp-declr (ident "x"))))))))
        (init-declr-list (init-declr (ident "bar_t"))))
  (decl (decl-spec-list (type-spec (typename "bar_t")))
        (init-declr-list
          (init-declr
            (ftn-declr
              (ident "ftn1")
              (param-list
                (param-decl
                  (decl-spec-list (type-spec (fixed-type "int")))
                  (param-declr (abs-declr (pointer))))))))))
ud1:
(("bar_t"
  decl
  (decl-spec-list
    (stor-spec (typedef))
    (type-spec
      (struct-def
        (field-list
          (comp-decl
            (decl-spec-list
              (type-spec (float-type "double")))
            (comp-declr-list (comp-declr (ident "d"))))
          (comp-decl
            (decl-spec-list (type-spec (typename "foo_t")))
            (comp-declr-list (comp-declr (ident "x"))))))))
  (init-declr (ident "bar_t")))
 ("ftn1"
  decl
  (decl-spec-list (type-spec (typename "bar_t")))
  (init-declr
    (ftn-declr
      (ident "ftn1")
      (param-list
        (param-decl
          (decl-spec-list (type-spec (fixed-type "int")))
          (param-declr (abs-declr (pointer)))))))))

tdexp:
(decl (decl-spec-list
        (stor-spec (typedef))
        (type-spec
          (struct-def
            (field-list
              (comp-decl
                (decl-spec-list
                  (type-spec (float-type "double")))
                (comp-declr (ident "d")))
              (comp-decl
                (decl-spec-list (type-spec (typename "int32_t")))
                (comp-declr (ident "x")))))))
      (init-declr (ident "bar_t")))

tdexp:
(decl (decl-spec-list
        (type-spec
          (struct-def
            (field-list
              (comp-decl
                (decl-spec-list
                  (type-spec (float-type "double")))
                (comp-declr (ident "d")))
              (comp-decl
                (decl-spec-list (type-spec (typename "int32_t")))
                (comp-declr (ident "x")))))))
      (init-declr
        (ftn-declr
          (ident "ftn1")
          (param-list
            (param-decl
              (decl-spec-list (type-spec (fixed-type "int")))
              (param-declr (abs-declr (pointer))))))))