persistent obarray?

persistent obarray?

[Drew Adams]

I have a large obarray (length 102701). To create it, I walk through a
(large) file of words, with many duplicates, interning each word traversed.
This takes 2-3 minutes. Currently, I create the obarrary when a given major
mode is entered for the first time.

I'm wondering if it would speed things up to write the completed obarray to
a Lisp file, and then read that file instead of creating the obarray as I do
now.

What are my options for doing that (to see if it is faster)? If I just write
the obarray, it will be read in as an ordinary vector, and, IIUC, that is
not a way to create an obarry (you must use intern). I could convert it to a
list, write that out, read it back in, and map intern over the list after
reading it. Is there another option?

Does it sound like this (write + read) would be worth trying? If so, is
there anything to be gained by byte-compiling the Lisp file? It would
contain only a list (or a setq or defvar with the list as value).

RE: persistent obarray?

[Drew Adams]

    I have a large obarray (length 102701). To create it, I walk
    through a (large) file of words, with many duplicates, interning
    each word traversed. This takes 2-3 minutes. Currently, I create
    the obarrary when a given major mode is entered for the first time.

    I'm wondering if it would speed things up to write the
    completed obarray to a Lisp file, and then read that file
    instead of creating the obarray as I do now.

    What are my options for doing that (to see if it is faster)?
    If I just write the obarray, it will be read in as an
    ordinary vector, and, IIUC, that is not a way to create an
    obarry (you must use intern). I could convert it to a
    list, write that out, read it back in, and map intern over the
    list after reading it. Is there another option?

    Does it sound like this (write + read) would be worth trying?
    If so, is there anything to be gained by byte-compiling the
    Lisp file? It would contain only a list (or a setq or defvar
    with the list as value).

Well, I tried it (haven't tried byte-compiling yet). It's a _zillion_ times
faster. Unless someone sees a better way, this is what I'm doing now:

Write:

(with-temp-file "foo"
  (mapatoms (lambda (s) (push s my-list)) my-obarray)
  (pp my-list (current-buffer)))

Read:

(if (not (file-readable-p "foo"))
    (make-my-obarray)    ; Create obarray from scratch
  (let ((buf (find-file-noselect "foo" 'nowarn 'raw))
        (obarray my-obarray))
    (unwind-protect
         (setq my-list (read buf))
      (kill-buffer buf))))

Re: persistent obarray?

[Stefan Monnier]

>   (let ((obarray my-obarray))
>     (unwind-protect
>          (setq my-list (read buf))
>       (kill-buffer buf))))

The let-binding of obarray is the main trick, yes.


        Stefan