Hello guilers!

I was wondering if we should make an effort to replace guile's C-based write
function with a more "schemy" solution that allows us to use write/display
on
large scheme data structures without stalling the fibers system and be very
fast and feature rich at the same time.

I did a take some months ago with the repository found at:
https://gitlab.com/tampe/guile-persist

To import this library's write/display just make sure that it is installed
and use
(use-modules (ice-9 write))

Then use write and display from that module just as you usually do in guile.

Now the solution I suggest we look at is to combine C routines with scheme
to get the best of both worlds. In C we get quite a lot of speed from
taking advantage of SIMD instructions like testing if 16 bytes are all in
the range [0,127] in one instruction and the coded library tries to do good
for all kinds of encodings and type of scheme strings (hence quite large).
Some string operations will be in an order of 100X faster with this library
than guiles own C implementation. Now C is bad in two ways. It is a less
powerful language than Scheme and harder to maintain. However quite a lot
of the power of scheme can be saved by separating more logic into scheme
and just focusing the C code on smaller parts. Also we would like to use
scheme because it interoperates well with fibers. As it is now, writing a
huge scheme datastructure freezes the thread if all is in C. But this does
not mean that it is impossible to use C. By instead trampolining into C
serializing chunks of data we can improve the behavior in a fibers context
quite dramatically. The reason to still use C is speed though and that
guile cannot take advantage of SIMD instructions. This should in the end
change with time but no work is currently planned for this and gcc today
has a really nice infrastructure towards such constructs in C.

Here are some blog posts about the writer I propose that we look into,
http://itampe.com/display-strings.html
http://itampe.com/printing-doubles.html

This code handles doubles and floats better than guiles C based writer.
First it fixes a bug when we write out a bytevector of f32 currently guile
ends up writing them as doubles which creates a lot of noise in the
printout. And secondly we can tell the writer to print floats in hex format
which guile cannot (we also have i reader in the repository that can read
hex floats) we actually can write in any of base 2,4,8,16,32 base
floats/doubles

The drawback is most likely tougher code to maintain, but thanks to gcc's
support for SIMD instructionsm, not too bad in my view.

WDYT?