persistent memory for Guile

persistent memory for Guile

[Terence Kelly]

Dear Colleagues,

An ongoing thread in the gnu-prog-discuss mailing list is discussing 
persistent memory for GNU projects.  Circa late February, Jacob Bachmeyer 
suggested on that list that Guile might benefit from persistent memory. 
After consulting the Guile manual, I'm writing now to outline what this 
might mean and to invite interested persons in the Guile community to 
discuss further.

First, a note on terminology:  By "persistent memory" I do *not* mean 
newfangled non-volatile memory hardware, e.g., Intel Optane or NVDIMMs. 
Instead, "persistent memory" here refers to a purely software abstraction 
that can be implemented on conventional hardware --- ordinary computers 
with volatile DRAM memory and conventional block storage (HDDs, SSDs).

The easiest way to vividly explain the possibilities for Guile might be to 
point out the persistent memory feature recently added to the GNU AWK 
(gawk) interpreter.  The new persistent memory feature of gawk is called 
"pm-gawk."  When you run an AWK script in the gawk interpreter, you may 
activate pm-gawk by specifying a file containing a persistent heap where 
pm-gawk will hold AWK variables and functions.  AWK scripts themselves are 
*not* modified in any way to activate pm-gawk.  A persistent heap lives in 
a file on disk and hangs around until deleted.  It may be re-used by 
subsequent runs of the same AWK script or by completely different AWK 
scripts, i.e., it's a way to pass variables and functions between 
unrelated scripts.  As a side effect, pm-gawk gives AWK (the moral 
equivalent of) a REPL.  The Persistent Memory Gawk User Manual explains 
the details; see the Quick Start section for a succinct concrete 
illustration of use:

https://www.gnu.org/software/gawk/manual/pm-gawk/

If Guile did something similar, the details might differ from pm-gawk, but 
the benefits would hopefully be the same: Efficient, transparent, and 
nearly effortless persistence-to-disk for arbitrary Scheme code, at the 
cost of small and fairly easy changes to Guile.  Implementing pm-gawk was 
remarkably easy --- under 100 lines of code added/changed in the initial 
proof-of-concept prototype, most of which was integrating a 
malloc-compatible persistent memory library into the gawk interpreter:

https://dl.acm.org/doi/pdf/10.1145/3534855

Another precedent for persistence in Scheme is a project that used an 
older persistent heap of mine to retrofit persistence onto a Scheme 
interpreter:

https://github.com/tvcutsem/schemeken

If you think that a similar persistence capability might be useful for 
Guile, let's talk.

Thank you.

Terence Kelly
https://dl.acm.org/profile/81100523747