I added a new gem, it' a c based serializer and deserializer of guile data which is relative feature rich and will ping pong
between scheme and c-land in order to be able to be fiber friendly. So currently the benchmarks are for the following copy
,time (load (dump (make-vector N 1))
1s for ~ 100MB (will be less pipelined)
,time (load (dump (iota N))
1s for ~ 50MB (will create in total N cons cells with 5 bytes)
The scheme based version of these serializers and deserializers was able to perform 5MB per second, but thats with much higher allocation overhead
This means that we match pythons dump and load functions. I will add those to my fibers piping tool
Currently supported data structures
all numbers yes even bignums
all immediates ('(), booleans characters and whatnot)
bytevectors
strings (wide strings and narrow strings)
symbols
keywords
pairs
vectors
the location of the c-code is in
(use-modules (persist persistance)) => atom-load-bv and atom-dump-bv is defined.
This library is used by python-on-guile and I will replace the current dump and load with this in the end. Also this tool
will be added to the fiber piping system I am working with in stis-engine.