From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: HiPhish Newsgroups: gmane.lisp.guile.user Subject: [ANN] guile-msgpack: MessagePack for GNU Guile (+ help needed) Date: Wed, 19 Sep 2018 14:33:22 +0200 Message-ID: <3401551.pIYnLFp0Xl@aleksandar-ixtreme-m5740> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7Bit X-Trace: blaine.gmane.org 1537360310 7965 195.159.176.226 (19 Sep 2018 12:31:50 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 19 Sep 2018 12:31:50 +0000 (UTC) To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Wed Sep 19 14:31:46 2018 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g2bdy-0001wA-02 for guile-user@m.gmane.org; Wed, 19 Sep 2018 14:31:46 +0200 Original-Received: from localhost ([::1]:45174 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g2bg4-0003PQ-9T for guile-user@m.gmane.org; Wed, 19 Sep 2018 08:33:56 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51793) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g2bfe-0003PK-UT for guile-user@gnu.org; Wed, 19 Sep 2018 08:33:32 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g2bfb-0006om-Oh for guile-user@gnu.org; Wed, 19 Sep 2018 08:33:30 -0400 Original-Received: from mout01.posteo.de ([185.67.36.65]:53890) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g2bfb-0006nd-8k for guile-user@gnu.org; Wed, 19 Sep 2018 08:33:27 -0400 Original-Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 9A93320E52 for ; Wed, 19 Sep 2018 14:33:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.de; s=2017; t=1537360404; bh=dhnzcrl4SYfP7SY3IURoVzihitD7WlpDt28XcUueFug=; h=From:To:Subject:Date:From; b=XpzYULuOZKvsS+ZAaH/V8z02bJewn7KzuySZwjCuUYeY0TnaBRLhVQCaSsZ55JBos jEOQMpp3huVpkiuOozqsLvE72mqxY7DOEMUyJpTfTtaVcjft38HKN23jurJ3689v9S CmII9xyqwaoW9VQphjWcarrwud1Kehq1wwzbgyzXVFF3Nu0KM61fGQYivwMAH2TF5t ikOZ57yXtvAxMzIDR00z+PtVUKdK/dbk+iqJvOWNo4YL3hHzwpU4AxPWCfzwjWh+5D 007iETM258Xpb2iQTLSDAy6qz9sXaE0dFM1kYzelyMd85R5vBW8OlI8f3EafW76guy CARReIPlct2Rg== Original-Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 42FfR7520yz6tmH for ; Wed, 19 Sep 2018 14:33:23 +0200 (CEST) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 185.67.36.65 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:14888 Archived-At: Hello Schemers, I am pleased to announce to the public a project I have been working on for a while: an implementation of the MessagePack[1] data serialization format for GNU Guile. https://gitlab.com/HiPhish/guile-msgpack ## What is MessagePack ## MessagePack is a specification for how to turn data into byte and vice-versa. Think of it like JSON, except instead of human readability it stresses speed and size efficiency. The format is binary and well suited for data exchange between processes. It also forms the basis of the MessagePack RPC protocol[2]. ## About the Guile implementation ## My implementation is written in pure Guile Scheme, so it should work out of the box. You can try it out without installing anything. Just clone the repo and give it a try: $ guile -L . scheme@(guile-user)> ,use (msgpack) scheme@(guile-user)> (pack "hello" 1337 #t) $1 = #vu8(165 104 101 108 108 111 205 5 57 195) As you can see, the three objects passed to the `pack` procedure have been turned into one big bytevector. We could now unpack this bytevector again, write it to a file, or send if off to a port. Since using MessagePack with ports is a frequent task there is also a `pack!` procedure which takes in a port to pack to as well. (define hodgepodge (vector 1 2 '#(3 #t) "foo")) (pack! hodgepodge (current-output-port)) To get our object back we `unpack` it: (unpack #vu8(#xA5 #x68 #x65 #x6C #x6C #x6F)) ; Returns "hello" (unpack! (current-input-port)) The readme goes into more detail and the (Texinfo) manual has the complete documentation. Building it is simple enough with the included makefile. ## What's next? ## Next is your feedback! Once the library has settled down I would like to make it into a Guix package. From there I can then start building a MessagePack RPC implementation and finally a Neovim client which will allow people to write plugins for Neovim in GNU Guile. (yes, Vim plugins written in Lisp, who would have thought?) ## Help needed ## This is my first time making a full project in Guile, so I would appreciate it if someone with more experience could look over my code. I have written down some things I could think of in the todo file. In particular I would like to know: How much is portability a concern? I know Guile implements *some* or r6rs, but I wasn't paying much attention to that. Is it something worth considering or should I just treat Guile as its own language that just so happens to be based on Scheme? The extension type `ext` (msgpack/ext.scm) is a pair of a signed 8-bit integer and a bytevector. The constructor does not enforce this condition, the two slots can be really anything. What it the proper way of enforcing this in Guile? I know Common Lisp has type declarations and Racket has contracts, but what does Guile have? The `pack` procedure takes any number of objects and packs them into a large bytevector. However, the `unpack` procedure only returns the first object it unpacks. Is there a way of making it unpack as many as it can? I thought of `values`, but you would need to know the number of values in advance. Also the caller would have to know in advance how many objects he is going to get unpacked. As I said, there is more in the todo file, but the other questions are under the hood, not user-visible. ## The inevitable panhandling ## I don't want to go into this too much, because no one likes to read it, but if you like my MessagePack implementation I would really appreciate if you could spare some cash. Links are in the readme; at the moment I only have Liberapay set up, if anyone can recommend me a service for one-time donations that would be cool. All the services I could find were about fundraising for charity and stuff, not what I was looking for. _______________________________________________________________________________ [1] https://msgpack.org/ [2] https://github.com/msgpack-rpc/msgpack-rpc