Bytestructures 1.0.0

Bytestructures 1.0.0

[Taylan Ulrich Bayırlı/Kammer]

I'm proud to make the first official release of Bytestructures, which is
long overdue.

The release brings no new features since the void pointers feature that
was pushed to the git repo on September 29, but I've since added support
for GNU Autotools, making it easy to package bytestructures on most GNU
distributions.  I've been told a Fedora package was made and works now.

The tarball release for 1.0.0:
https://github.com/TaylanUB/scheme-bytestructures/releases/download/v1.0.0/bytestructures-1.0.0.tar.gz


What is bytestructures?
=======================

To get a complete overview, tutorial-like introduction, detailed API
specification, or other in-depth information about bytestructures, you
are strongly advised to read the README.md file that can be found in the
git repository as well as in release tarballs.  If you tolerate unfree
JavaScript code, It can be read on the GitHub project page as well,
where it's nicely formatted into HTML:

    https://github.com/TaylanUB/scheme-bytestructures/

However, here's a short summary:

Scheme has a data type called a "bytevector" which is a dumb array of
bytes.  It's useful for, for instance, I/O operations or raw memory
access.  In particular, it's possible in Guile to receive a memory
pointer from the Foreign Function Interface, then create a bytevector
object that serves as a "window" into the raw bytes behind a data
structure created by a C library.

The bytestructures system offers a structured way to access contents of
bytevectors, based on layout information you provide.  The system is
highly flexible, but the core library provides functionality that
closely resembles the type system of C, and correctly implements the
rules of struct padding and alignment as far as I'm aware (bug reports
and patches welcome), supporting even bitfields, which you'll find in
few if any other FFI libraries to my knowledge. :-)

Here's an example of a simple C data structure:

    #include <stdint.h>

    typedef struct point {
        uint64_t x;
        uint64_t y;
    } point_t;

    point_t points[1000];

And here's the bytestructure equivalent in Guile Scheme:

    (import (bytestructures guile))

    (define bs:point
      (bs:struct `((x ,uint64)
                   (y ,uint64))))

    (define points
      (bytestructure (bs:vector 1000 bs:point)))

Here's how you might use the C data structure:

    points[n].y = 42;

And here's the equivalent Scheme code:

    (bytestructure-set! points n 'y 42)


How is the performance?
=======================

If you use the regular bytestructure-set! and bytestructure-ref forms,
you will suffer run-time overhead to access bytevector contents.  This
may be insignificant for many use-cases, but if you find yourself in a
situation where higher performance is desired, you can hoist *all*
overhead incurred by bytestructures to compile-time by using the
alternative macro API.

This naturally puts some constraints on the programmer.  Most notably,
the layout information you provide (concretely, the "bytestructure
descriptor" object you create by calling e.g. bs:struct) must be
available at compile-time.  Normally all you need to do for this is to
put your bytestructure descriptor definitions into a separate module;
then importing that module in Guile will make it available at compile
time as well as run-time.

An example:

    ;; Contents of point.scm:

    (define-module (taylan point))

    (import (bytestructures guile))

    (define-public bs:point
      (bs:struct `((x ,uint64)
                   (y ,uint64))))

    ;; point.scm ends here


    ;; Contents of program.scm:

    (import (rnrs bytevectors))
    (import (taylan point))

    (define-bytestructure-accessors (bs:vector 1000 bs:point)
      points-unwrap points-ref points-set!)

    (define points
      (make-bytevector
        (* 1000 (bytestructure-descriptor-size bs:point))))

    ;; ...

    (points-set! points n y 42)

    (display (points-ref points n y))  ;prints 42
    (newline)

    ;; program.scm ends here


Portability to other Scheme implementations
===========================================

The library is known to also work with Chibi, Gauche, Kawa, and Larceny.
Further, any other Scheme implementation that supports R7RS-small should
be able to run the library with little or no porting effort.

However, the macro API is only available when the underlying Scheme
implementation supports syntax-case, and the following sub-modules are
unique to Guile:

- (bytestructures guile pointer)
- (bytestructures guile cstring-pointer)


Future development
==================

You can expect development to be dormant for the foreseeable future
unless when there are bug reports or feature requests, as I'm busy with
other things in life and fairly content with the current feature-set.

The API can be considered highly stable.

The internals may one day be overhauled, in particular to use a
different record type system that supports subtyping.

Another long-term goal is to support use of ports instead of bytevectors
in combination with bytestructures, though no concrete plan exists.


Questions, comments, etc. are welcome. :-)

Taylan

Bytestructures 1.0.1

[Taylan Ulrich Bayırlı/Kammer]

Amazingly, a bug that existed for months has been coincidentally
discovered hours after I made the release. :-))

The alignment of vectors and unions within structs was not being
calculated correctly.  Fortunately, it was a quick fix.

So here's 1.0.1 already!

https://github.com/TaylanUB/scheme-bytestructures/releases/download/v1.0.1/bytestructures-1.0.1.tar.gz


Taylan

Re: Bytestructures 1.0.1

[Alex Vong]

taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes:

> Amazingly, a bug that existed for months has been coincidentally
> discovered hours after I made the release. :-))
>
> The alignment of vectors and unions within structs was not being
> calculated correctly.  Fortunately, it was a quick fix.
>
> So here's 1.0.1 already!
>
> https://github.com/TaylanUB/scheme-bytestructures/releases/download/v1.0.1/bytestructures-1.0.1.tar.gz
>
>
> Taylan

It is even more amazing that I've installed 1.0.0 after I wake up and
now I have to install 1.0.1 ...

I can confirm the configure script works properly on both versions.

Thanks,
Alex

Re: Bytestructures 1.0.1

[Amirouche Boubekki]

On 2018-01-08 00:45, taylanbayirli@gmail.com wrote:
> Amazingly, a bug that existed for months has been coincidentally
> discovered hours after I made the release. :-))
> 
> The alignment of vectors and unions within structs was not being
> calculated correctly.  Fortunately, it was a quick fix.
> 
> So here's 1.0.1 already!
> 
> https://github.com/TaylanUB/scheme-bytestructures/releases/download/v1.0.1/bytestructures-1.0.1.tar.gz
> 

Thanks a lot, I will test it and work on a guix package.

Thanks again!

Re: Bytestructures 1.0.1

[Catonano]

2018-01-08 0:45 GMT+01:00 Taylan Ulrich Bayırlı/Kammer <
taylanbayirli@gmail.com>:

> Amazingly, a bug that existed for months has been coincidentally
> discovered hours after I made the release. :-))
>
> The alignment of vectors and unions within structs was not being
> calculated correctly.  Fortunately, it was a quick fix.
>
> So here's 1.0.1 already!
>
> https://github.com/TaylanUB/scheme-bytestructures/
> releases/download/v1.0.1/bytestructures-1.0.1.tar.gz
>
>
> Taylan
>
>

thank you Taylan and Amirouche :-)

I should update my dependency on Bytesrtuctures in my tiny proect to read
soreadsheet files (I can't remember its name now)

I don't know. I'm not that willing to do that now ;-)