HI all, I added a new gem, it' a c based serializer and deserializer of guile data which is relative feature rich and will ping pong between scheme and c-land in order to be able to be fiber friendly. So currently the benchmarks are for the following copy operations, ,time (load (dump (make-bytevector N 1)) 1s for ~ 1GB (test of memcpy) ,time (load (dump (make-vector N 1)) 1s for ~ 100MB (will be less pipelined) ,time (load (dump (iota N)) 1s for ~ 50MB (will create in total N cons cells with 5 bytes) The scheme based version of these serializers and deserializers was able to perform 5MB per second, but thats with much higher allocation overhead This means that we match pythons dump and load functions. I will add those to my fibers piping tool Currently supported data structures all numbers yes even bignums all immediates ('(), booleans characters and whatnot) bytevectors strings (wide strings and narrow strings) symbols keywords pairs vectors the location of the c-code is in guile-persist: https://gitlab.com/tampe/guile-persist (use-modules (persist persistance)) => atom-load-bv and atom-dump-bv is defined. This library is used by python-on-guile and I will replace the current dump and load with this in the end. Also this tool will be added to the fiber piping system I am working with in stis-engine. On Sat, Aug 14, 2021 at 2:07 PM Stefan Israelsson Tampe < stefan.itampe@gmail.com> wrote: > Next is to add support for > 1. self referential data structures like #1 = '(1 2 3 #1), string > compression > 2. Structs > 3 .Goops Classes and Objects > > Then I will add > 1. Lambdas > and then Continuations to be able to serialize (if you turn off unboxing > optimizationsas in guile) > > After that a toolbox to customize tserializing just as with python > > Note all this is supported in python-on-guile, just that the new code is > better suited for proper streaming > and the code is much nicer in my personal view so you do have all these > features already if you use pythons > dumps and loads commands together with string->fpipe and fpipe->string. > > The new thing is essentially stream compression and streams to zmq's. > Stream compression wll come to > python as well as the zlib python module is very rudementary > >