Against sqlite3!!! (Was: sqlite3)

Against sqlite3!!! (Was: sqlite3)

[Qiantan Hong]

[-- Attachment #1: Type: text/plain, Size: 77 bytes --]

I’ve attached a pure Emacs Lisp implementation of persistent key value store.

[-- Attachment #2: resist!.el --]
[-- Type: application/octet-stream, Size: 1624 bytes --]

;;; resist!.el --- Against SQLite3!  -*- lexical-binding: t; -*-

(defvar kv-store-table)
(defmacro --kv (key value)
  (puthash key value kv-store-table)
  nil)
(cl-defstruct (kv-store (:constructor make-kv-store-1)) path table)
(defun make-kv-store (path)
  (let* ((kv-store (make-kv-store-1 :path path))
         (kv-store-table (make-hash-table :test 'equal))
         need-compactification)
    (when (file-exists-p path)
      (condition-case c
          (load-file path)
        (end-of-file
         ;; We might encounter trailing unbalanced form if Emacs
         ;; crashed in the middle of `kv-put'.  We compact the file
         ;; and fix unbalanced form as a side effect
         (setq need-compactification t)))
      (setf (kv-store-table kv-store) kv-store-table))
    (when need-compactification
      (compact-kv-store kv-store))
    kv-store))
(defsubst kv-put-log (key value kv-store)
  (let ((print-length nil) (print-level nil))
    (prin1 `(--kv ,key ,value) (current-buffer)))
  (insert "\n"))
(defun compact-kv-store (kv-store)
  ;; dump the full content of kv-store-table at once
  ;; to compress the log
  (with-temp-buffer
    (maphash (lambda (key value) (kv-put-log key value kv-store))
             (kv-store-table kv-store))
    (let ((file-precious-flag t))
      (write-file (kv-store-path kv-store)))))
(defun kv-put (key value kv-store)
  (with-temp-buffer
    (kv-put-log key value kv-store)
    (append-to-file nil nil (kv-store-path kv-store)))
  (puthash key value (kv-store-table kv-store)))
(defun kv-get (key kv-store)
  (gethash key (kv-store-table kv-store)))
(provide 'resist!)

Against sqlite3!!! (Was: sqlite3)

[Qiantan Hong]

[-- Attachment #1.1: Type: text/plain, Size: 81 bytes --]

I’ve attached a pure Emacs Lisp implementation of persistent key value store.

[-- Attachment #1.2: Type: text/html, Size: 852 bytes --]

[-- Attachment #2: resist!.el --]
[-- Type: application/octet-stream, Size: 1667 bytes --]

;;; resist!.el --- Against SQLite3!  -*- lexical-binding: t; -*-

(defvar kv-store-table)
(defmacro --kv (key value)
  (puthash key value kv-store-table)
  nil)
(cl-defstruct (kv-store (:constructor make-kv-store-1)) path table)
(defun make-kv-store (path)
  (let* ((kv-store (make-kv-store-1 :path path))
         (kv-store-table (make-hash-table :test 'equal))
         need-compactification)
    (when (file-exists-p path)
      (condition-case c
          (load-file path)
        (end-of-file
         ;; We might encounter trailing unbalanced form if Emacs
         ;; crashed in the middle of `kv-put'.  We compact the file
         ;; and fix unbalanced form as a side effect
         (setq need-compactification t)))
      (setf (kv-store-table kv-store) kv-store-table))
    (when need-compactification
      (compact-kv-store kv-store))
    kv-store))
(defsubst kv-put-log (key value kv-store)
  (let ((print-length nil) (print-level nil))
    (prin1 `(--kv ,key ,value) (current-buffer)))
  (insert "\n"))
(defun compact-kv-store (kv-store)
  ;; dump the full content of kv-store-table at once
  ;; to compress the log
  (with-temp-buffer
    (maphash (lambda (key value) (kv-put-log key value kv-store))
             (kv-store-table kv-store))
    (let ((file-precious-flag t))
      (write-file (kv-store-path kv-store)))))
(defun kv-put (key value kv-store)
  (with-temp-buffer
    (kv-put-log key value kv-store)
    (append-to-file nil nil (kv-store-path kv-store)))
  (puthash key value (kv-store-table kv-store)))
(defun kv-get (key kv-store)
  (gethash key (kv-store-table kv-store)))
(provide 'resist!)

[-- Attachment #3: ATT00001.htm --]
[-- Type: text/html, Size: 332 bytes --]

Re: Against sqlite3!!! (Was: sqlite3)

[Qiantan Hong]

A first unscientific benchmark, people!

(defvar test (make-kv-store "~/Projects/test.eld”))
(loop for i below 10000
      do (kv-put i '(1 3 3) test))

(measure-time
 (make-kv-store "~/Projects/test.eld"))
"0.025423"

Hua, 0.025s for loading 10k entries, by abusing built-in C load directly.

> On Dec 7, 2021, at 12:13 AM, Qiantan Hong <qhong@MIT.EDU> wrote:
> 
>> I’ve attached a pure Emacs Lisp implementation of persistent key value store.
> <resist!.el>
> Seems to stuck for too long, so I’ve resent it — sorry for the spam.

Re: Against sqlite3!!! (Was: sqlite3)

[Zhu Zihao]

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]


Actually, Emacs can serialize/deserialize hash table directly via
prin1-to-string & read

```
(let ((ht (make-hash-table)))
  (puthash "test" "value" ht)
  (format "%S" ht))
```

You can use `read` to "parse" the string returned by that snippet and
get a hash table.

Qiantan Hong <qhong@mit.edu> writes:

>  I’ve attached a pure Emacs Lisp implementation of persistent key value store.
>
> [4. resist!.el --- application/emacs-lisp; resist!.el]...
>
> [5. ATT00001.htm --- text/html; ATT00001.htm]...


-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]