From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: taylanbayirli@gmail.com (Taylan Ulrich =?utf-8?Q?Bay=C4=B1rl=C4=B1?= =?utf-8?Q?=2FKammer?=) Newsgroups: gmane.lisp.guile.user Subject: Bytestructures: a "type system" for bytevectors Date: Sun, 30 Aug 2015 18:32:53 +0200 Message-ID: <87r3mkg2iy.fsf@T420.taylan> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1440952390 5109 80.91.229.3 (30 Aug 2015 16:33:10 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 30 Aug 2015 16:33:10 +0000 (UTC) To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Sun Aug 30 18:33:06 2015 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZW5XW-000467-N1 for guile-user@m.gmane.org; Sun, 30 Aug 2015 18:33:06 +0200 Original-Received: from localhost ([::1]:59396 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZW5XW-0007ww-K6 for guile-user@m.gmane.org; Sun, 30 Aug 2015 12:33:06 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:60354) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZW5XN-0007wf-Af for guile-user@gnu.org; Sun, 30 Aug 2015 12:32:58 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZW5XL-0008FO-Ub for guile-user@gnu.org; Sun, 30 Aug 2015 12:32:57 -0400 Original-Received: from mail-wi0-x236.google.com ([2a00:1450:400c:c05::236]:32812) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZW5XL-0008F3-NC for guile-user@gnu.org; Sun, 30 Aug 2015 12:32:55 -0400 Original-Received: by wicpl12 with SMTP id pl12so10683890wic.0 for ; Sun, 30 Aug 2015 09:32:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:user-agent:mime-version :content-type; bh=8mgVlbPmGlz91RnO4gA+RCd3I4teWXibOvDP4Ni2U8M=; b=EaZarXcma7NsA74KhGBHLMXQ+gfFt6qOetwFRtnEUofGbUvJj0nag8HkyGd9zQcahp eecmxrx8BGkXngyWWCYP7jd1DwNCvZ8UoePa7i139VtMisQDSqDNZ5L1cr1OzseNnti7 qPAXWV/PUgOFYmuez3Oz8cg/mbYo6Qx6fEw17a5pVkaQoNh2UhwRBNhH8baKKHA9QXWX N3rS7kKORPZ7+XdyYoiwPPJZY61Xoe+R9lzgxhs0TrxPRKhsNePxrN7iv1RKFtwxOayT KXZLwhbvzWTWZu5a2I7cnwrrsW3aHp123+x2MYo1qqS6p13osAhcGSoZcS3muOHdEN3p /6RQ== X-Received: by 10.194.93.198 with SMTP id cw6mr24187286wjb.113.1440952375013; Sun, 30 Aug 2015 09:32:55 -0700 (PDT) Original-Received: from T420.taylan ([2a02:908:c32:4740:221:ccff:fe66:68f0]) by smtp.gmail.com with ESMTPSA id jr5sm18072721wjc.14.2015.08.30.09.32.53 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 30 Aug 2015 09:32:53 -0700 (PDT) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a00:1450:400c:c05::236 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: guile-user-bounces+guile-user=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.user:11969 Archived-At: https://github.com/taylanub/scheme-bytestructures (I don't endorse GitHub, but I gave in after Gitorious went down.) I had started working on this project around two years ago but it had a pretty strange and complex API, an unreadable README, high overhead for something you might want to use in high-performance settings (byte crunching), and a bit too hung up on being standards compliant. I resumed working on it around a week ago, and solved all these issues! (I still do my best to keep R7RS compliance, but Guile is my prime *real* target.) So what is it? === Structured access to bytevector contents === Don't tell C-standard pedants, but the type system of C is nothing more than a thin layer over your computer memory: a huge sequence of bytes. Compilers are allowed some crazy things as per the strict standards language, but if you define a struct { uint16_t x; int8_t y[3]; }, you essentially declare that you will be working with memory-snippets of 5 bytes, whose first two bytes stand for an unsigned 16-bit integer, and the other three bytes for an array of three signed 8-bit integers. That's all there is to it, and you could populate those bytes directly, one by one: the_struct_t my_struct = { a, b, c, d, e, f } Not sensible in practice for various reasons (endianness, compiler optims, unreadable code, etc.), but it's good to be aware of the basic idea: structs, arrays, unions, integer types, whatever, all types in C are just windows upon byte-sequences. Some programs will offer a stable ABI, indeed making promises about the byte-by-byte structure of their data types. It might also be a piece of hardware giving you bytes under such a structure, and you can declare the structure via C's type system to put some sanity into your hardware-interfacing code. Now in Scheme we have some *proper* abstraction. We can fully forget about the bytes and work with a purely logical notion of objects. But still we have bytevectors, a type encapsulating a raw sequence of bytes, because they are useful for various purposes, be it talking with a C library, or with a piece of hardware. But Scheme doesn't offer anything like C's type system for bytevectors. There's the bytevector-foo-ref/set! functions, and there's SRFI-4, but there's nothing as generic as C's type system. Now there is. :-) (Bit-fields not yet supported.) The struct example from above is now: (bs:struct `((x ,uint16) (y ,(bs:vector 3 int8)))) and that returns an object (a "bytestructure descriptor") which you can use as part of other type definitions (the uin16 ant int8 there are types provided for convenience by the library; they could have been defined by the user as well), or use to access data in a bytevector logically via struct field names and vector indices. (bytestructure my-struct 'y 2) ;the last uint8 There is even a "pointer" type, made possible by our FFI module. Be warned though, you can write Scheme programs that segfault! For when you need maximal performance from your code, there's a macro-generating macro which will arrange for all the type stuff to happen at compile time. Granted, you lose some flexibility. For more details, refer to the README. I'm still shy of offering an absolutely final API, but I'm pretty happy with the state of things and would love it if some people played with it. Consider this a beta phase announcement or so. Tell me any nontrivial issues you see with the library. That's all for now. Thanks in advance for any feedback! Happy hacking, Taylan