unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Against sqlite3!!! (Was: sqlite3)
@ 2021-12-07  7:44 Qiantan Hong
  0 siblings, 0 replies; 4+ messages in thread
From: Qiantan Hong @ 2021-12-07  7:44 UTC (permalink / raw)
  To: tomas@tuxteam.de; +Cc: emacs-devel@gnu.org

[-- Attachment #1: Type: text/plain, Size: 77 bytes --]

I’ve attached a pure Emacs Lisp implementation of persistent key value store.

[-- Attachment #2: resist!.el --]
[-- Type: application/octet-stream, Size: 1624 bytes --]

;;; resist!.el --- Against SQLite3!  -*- lexical-binding: t; -*-

(defvar kv-store-table)
(defmacro --kv (key value)
  (puthash key value kv-store-table)
  nil)
(cl-defstruct (kv-store (:constructor make-kv-store-1)) path table)
(defun make-kv-store (path)
  (let* ((kv-store (make-kv-store-1 :path path))
         (kv-store-table (make-hash-table :test 'equal))
         need-compactification)
    (when (file-exists-p path)
      (condition-case c
          (load-file path)
        (end-of-file
         ;; We might encounter trailing unbalanced form if Emacs
         ;; crashed in the middle of `kv-put'.  We compact the file
         ;; and fix unbalanced form as a side effect
         (setq need-compactification t)))
      (setf (kv-store-table kv-store) kv-store-table))
    (when need-compactification
      (compact-kv-store kv-store))
    kv-store))
(defsubst kv-put-log (key value kv-store)
  (let ((print-length nil) (print-level nil))
    (prin1 `(--kv ,key ,value) (current-buffer)))
  (insert "\n"))
(defun compact-kv-store (kv-store)
  ;; dump the full content of kv-store-table at once
  ;; to compress the log
  (with-temp-buffer
    (maphash (lambda (key value) (kv-put-log key value kv-store))
             (kv-store-table kv-store))
    (let ((file-precious-flag t))
      (write-file (kv-store-path kv-store)))))
(defun kv-put (key value kv-store)
  (with-temp-buffer
    (kv-put-log key value kv-store)
    (append-to-file nil nil (kv-store-path kv-store)))
  (puthash key value (kv-store-table kv-store)))
(defun kv-get (key kv-store)
  (gethash key (kv-store-table kv-store)))
(provide 'resist!)

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Against sqlite3!!! (Was: sqlite3)
       [not found] <MN2PR12MB3391BC76A0D05236AC76C94E946E9@MN2PR12MB3391.namprd12.prod.outlook.com>
@ 2021-12-07  8:13 ` Qiantan Hong
  2021-12-07  9:14   ` Qiantan Hong
  2021-12-07 13:45   ` Zhu Zihao
  0 siblings, 2 replies; 4+ messages in thread
From: Qiantan Hong @ 2021-12-07  8:13 UTC (permalink / raw)
  To: emacs-devel@gnu.org


[-- Attachment #1.1: Type: text/plain, Size: 81 bytes --]

I’ve attached a pure Emacs Lisp implementation of persistent key value store.

[-- Attachment #1.2: Type: text/html, Size: 852 bytes --]

[-- Attachment #2: resist!.el --]
[-- Type: application/octet-stream, Size: 1667 bytes --]

;;; resist!.el --- Against SQLite3!  -*- lexical-binding: t; -*-

(defvar kv-store-table)
(defmacro --kv (key value)
  (puthash key value kv-store-table)
  nil)
(cl-defstruct (kv-store (:constructor make-kv-store-1)) path table)
(defun make-kv-store (path)
  (let* ((kv-store (make-kv-store-1 :path path))
         (kv-store-table (make-hash-table :test 'equal))
         need-compactification)
    (when (file-exists-p path)
      (condition-case c
          (load-file path)
        (end-of-file
         ;; We might encounter trailing unbalanced form if Emacs
         ;; crashed in the middle of `kv-put'.  We compact the file
         ;; and fix unbalanced form as a side effect
         (setq need-compactification t)))
      (setf (kv-store-table kv-store) kv-store-table))
    (when need-compactification
      (compact-kv-store kv-store))
    kv-store))
(defsubst kv-put-log (key value kv-store)
  (let ((print-length nil) (print-level nil))
    (prin1 `(--kv ,key ,value) (current-buffer)))
  (insert "\n"))
(defun compact-kv-store (kv-store)
  ;; dump the full content of kv-store-table at once
  ;; to compress the log
  (with-temp-buffer
    (maphash (lambda (key value) (kv-put-log key value kv-store))
             (kv-store-table kv-store))
    (let ((file-precious-flag t))
      (write-file (kv-store-path kv-store)))))
(defun kv-put (key value kv-store)
  (with-temp-buffer
    (kv-put-log key value kv-store)
    (append-to-file nil nil (kv-store-path kv-store)))
  (puthash key value (kv-store-table kv-store)))
(defun kv-get (key kv-store)
  (gethash key (kv-store-table kv-store)))
(provide 'resist!)

[-- Attachment #3: ATT00001.htm --]
[-- Type: text/html, Size: 332 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Against sqlite3!!! (Was: sqlite3)
  2021-12-07  8:13 ` Qiantan Hong
@ 2021-12-07  9:14   ` Qiantan Hong
  2021-12-07 13:45   ` Zhu Zihao
  1 sibling, 0 replies; 4+ messages in thread
From: Qiantan Hong @ 2021-12-07  9:14 UTC (permalink / raw)
  To: emacs-devel@gnu.org

A first unscientific benchmark, people!

(defvar test (make-kv-store "~/Projects/test.eld”))
(loop for i below 10000
      do (kv-put i '(1 3 3) test))

(measure-time
 (make-kv-store "~/Projects/test.eld"))
"0.025423"

Hua, 0.025s for loading 10k entries, by abusing built-in C load directly.

> On Dec 7, 2021, at 12:13 AM, Qiantan Hong <qhong@MIT.EDU> wrote:
> 
>> I’ve attached a pure Emacs Lisp implementation of persistent key value store.
> <resist!.el>
> Seems to stuck for too long, so I’ve resent it — sorry for the spam.


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Against sqlite3!!! (Was: sqlite3)
  2021-12-07  8:13 ` Qiantan Hong
  2021-12-07  9:14   ` Qiantan Hong
@ 2021-12-07 13:45   ` Zhu Zihao
  1 sibling, 0 replies; 4+ messages in thread
From: Zhu Zihao @ 2021-12-07 13:45 UTC (permalink / raw)
  To: Qiantan Hong; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]


Actually, Emacs can serialize/deserialize hash table directly via
prin1-to-string & read

```
(let ((ht (make-hash-table)))
  (puthash "test" "value" ht)
  (format "%S" ht))
```

You can use `read` to "parse" the string returned by that snippet and
get a hash table.

Qiantan Hong <qhong@mit.edu> writes:

>  I’ve attached a pure Emacs Lisp implementation of persistent key value store.
>
> [4. resist!.el --- application/emacs-lisp; resist!.el]...
>
> [5. ATT00001.htm --- text/html; ATT00001.htm]...


-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-12-07 13:45 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-07  7:44 Against sqlite3!!! (Was: sqlite3) Qiantan Hong
     [not found] <MN2PR12MB3391BC76A0D05236AC76C94E946E9@MN2PR12MB3391.namprd12.prod.outlook.com>
2021-12-07  8:13 ` Qiantan Hong
2021-12-07  9:14   ` Qiantan Hong
2021-12-07 13:45   ` Zhu Zihao

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).