Re: Serialize list to disk

[Kevin Rodgers <ihs_4664@yahoo.com>]

JD Smith wrote:
> I have a large (1MB) XML routine definition file which can be parsed
> with xml-parse-file into a fairly large list.  I need to walk over
> this list and transform it in place to a more useful internal format.
> All of this takes 5-7secs on reasonably speedy machines (of which
> about 4s occurs just in xml-parse-file).  This list must be read in
> during each Emacs session in which my IDLWAVE programming mode is
> used, but the list is constant, so it would seem natural to perform
> the XML conversion just once, and then serialize the list to disk in
> some compact binary (but hopefully portable) format for future
> sessions.  Before it was reborn as XML, this data structure was
> auto-generated in lisp and byte-compiled, and it loaded in a fraction
> of a second.  That is no longer an option, so I'm stuck with the XML.
> 
> Is there a simple input/output mechanism in Emacs for static list
> structures, that would allow me to write a compact representation on
> disk, and quickly recover it in future sessions, without going through
> the lengthy XML conversion process?  One wrinkle: much of this data
> structure is repetitive, so to save on memory and speed, many of the
> strings inside it are "sinterned" into custom hash variables.

Of course!

To write it out:

(let ((routine (xml-parse-file XML-FILE)))
   (with-temp-file LISP-FILE
     (pp object (current-buffer))))

Then to read it in:

(let ((routine (read (find-file-noselect LISP-FILE))))
   ...)

> The list is basically many, many cells similar to:
> 
> <ROUTINE name="WV_CWT" link="WV_CWT.html">
>   <SYNTAX name="Result = WV_CWT(Array, Family, Order )" type="func" />
>   <ARGUMENT name="Array" link="WV_CWT.html#wp1009345" />
>   <ARGUMENT name="Family" link="WV_CWT.html#wp1009347" />
>   <ARGUMENT name="Order" link="WV_CWT.html#wp1009349" />
>   <KEYWORD name="DOUBLE" link="WV_CWT.html#wp1009356" />
>   <KEYWORD name="DSCALE" link="WV_CWT.html#wp1009358" />
>   <KEYWORD name="NSCALE" link="WV_CWT.html#wp1009484" />
>   <KEYWORD name="PAD" link="WV_CWT.html#wp1009489" />
>   <KEYWORD name="SCALE" link="WV_CWT.html#wp1009494" />
>   <KEYWORD name="START_SCALE" link="WV_CWT.html#wp1009568" />
> </ROUTINE>
> 
> which gets lispified as:
> 
> (ROUTINE ((name . "WV_CWT") (link . "WV_CWT.html")) (SYNTAX ((name
> . "Result = WV_CWT(Array, Family, Order )") (type . "func")) "")
> (ARGUMENT ((name . "Array") (link . "WV_CWT.html#wp1009345")) "")
> (ARGUMENT ((name . "Family") (link . "WV_CWT.html#wp1009347")) "")
> (ARGUMENT ((name . "Order") (link . "WV_CWT.html#wp1009349")) "")
> (KEYWORD ((name . "DOUBLE") (link . "WV_CWT.html#wp1009356")) "")
> (KEYWORD ((name . "DSCALE") (link . "WV_CWT.html#wp1009358")) "")
> (KEYWORD ((name . "NSCALE") (link . "WV_CWT.html#wp1009484")) "")
> (KEYWORD ((name . "PAD") (link . "WV_CWT.html#wp1009489")) "")
> (KEYWORD ((name . "SCALE") (link . "WV_CWT.html#wp1009494")) "")
> (KEYWORD ((name . "START_SCALE") (link . "WV_CWT.html#wp1009568"))
> ""))
> 
> Not sure why that final null string is added to the list (possibly the
> space before "/>"?).

No, that is normal XML syntax:  "<SYNTAX ... />" is an empty element
just like "<SYNTAX ...></SYNTAX>".

-- 
Kevin Rodgers