New package: resist!

New package: resist!

[Qiantan Hong]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

This package implements persistence facility.
It provides two level of interfaces:
- A high-level persistent variable facility
- A low-level persistent key-value store facility

The persistent variable facility detects changes of values
persistent variables in an idle timer and persist the changes
into a persistent key-value store.
Multiple methods for detecting and computing the changes are
provided. See `make-persistent-variable' for details.

The persistent key-value store provides the following functions:
- Creating and compressing store: `make-kv-store', `compact-kv-store'
- Put, remove and look up key value pairs: `kv-put', `kv-rem', `kv-get',
- List operations: `kv-push', `kv-delete'
See their docstrings for details.
All changes are persisted immediately into external storage.


[-- Attachment #2: resist!.el --]
[-- Type: application/octet-stream, Size: 10405 bytes --]

;;; resist! --- Against SQLite3!  -*- lexical-binding: t; -*-

;; Copyright (C) 2021 Free Software Foundation, Inc.

;; Author: Qiantan Hong <qhong@alum.mit.edu>
;; Maintainer: Qiantan Hong <qhong@alum.mit.edu>
;; Keywords: persistence database
;; Version: 0.0.1

;; This file is part of GNU Emacs.

;; GNU Emacs is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:
;; This package implements persistence facility.
;; It provides two level of interfaces:
;; - A high-level persistent variable facility
;; - A low-level persistent key-value store facility

;; The persistent variable facility detects changes of values
;; persistent variables in an idle timer and persist the changes
;; into a persistent key-value store.
;; Multiple methods for detecting and computing the changes are
;; provided. See `make-persistent-variable' for details.

;; The persistent key-value store provides the following functions:
;; - Creating and compressing store: `make-kv-store', `compact-kv-store'
;; - Put, remove and look up key value pairs: `kv-put', `kv-rem', `kv-get',
;; - List operations: `kv-push', `kv-delete'
;; See their docstrings for details.
;; All changes are persisted immediately into external storage.

;;; Code:

(require 'cl-lib)

(defcustom persistent-variable-idle-time 1
  "Time in seconds to wait before writing out persistent variables.

The effect normally takes place after restarting Emacs, or
restarting `persistent-variable-idle-timer' manually."
  :type 'number)

(defcustom persistent-variable-store-filename (concat user-emacs-directory ".persistent-variables")
  "Filename of the key value store that backs up persistent variables."
  :type 'file)

(cl-defstruct (kv-store (:constructor make-kv-store-1)) path table)
(defun make-kv-store (path)
  "Create a key value store backed by file PATH.
If file PATH does not exist, create it and return an empty key value store.
If file PATH exists, load its content into a key value store and return it."
  (let* ((kv-store (make-kv-store-1 :path path))
         (kv-store-table (make-hash-table :test 'equal))
         need-compactification)
    (when (file-exists-p path)
      (condition-case nil
          (with-temp-buffer
            (insert-file-contents path)
            (while (< (point) (1- (point-max))) ; exclude trailing newline
              (let ((entry (read (current-buffer))))
                (pcase (car entry)
                  ('++ (puthash (cadr entry) (caddr entry) kv-store-table))
                  ('-- (remhash (cadr entry) kv-store-table))
                  ('l+ (push (caddr entry) (gethash (cadr entry) kv-store-table)))
                  ('l- (puthash (cadr entry)
                                (delete (caddr entry) (gethash (cadr entry) kv-store-table))
                                kv-store-table))))))
        (end-of-file
         ;; We might encounter trailing unbalanced form if Emacs
         ;; crashed in the middle of `kv-put'.  We compact the file
         ;; and fix unbalanced form as a side effect
         (setq need-compactification t)))
      (setf (kv-store-table kv-store) kv-store-table))
    (when need-compactification
      (compact-kv-store kv-store))
    kv-store))
(defsubst kv--log (form)
  (let ((print-length nil) (print-level nil))
    (prin1 form (current-buffer)))
  (insert "\n"))
(defun compact-kv-store (kv-store)
  "Compress the log for KV-STORE.
Do this by dumping the full content of (kv-store-table KV-STORE) at once."
  ;; dump the full content of kv-store-table at once
  ;; to compress the log
  (with-temp-buffer
    (maphash (lambda (key value) (kv--log (list '++ key value)))
             (kv-store-table kv-store))
    (let ((file-precious-flag t))
      (write-file (kv-store-path kv-store)))))

(defsubst kv-put-1 (key value kv-store)
  (kv--log (list '++ key value))
  (puthash key value (kv-store-table kv-store)))
(defsubst kv-rem-1 (key kv-store)
  (kv--log (list '-- key))
  (remhash key (kv-store-table kv-store)))
(defsubst kv-push-1 (key value kv-store)
  (kv--log (list 'l+ key value))
  (push value (gethash key (kv-store-table kv-store))))
(defsubst kv-delete-1 (key value kv-store)
  (kv--log (list 'l- key value))
  (puthash key (delete value (gethash key (kv-store-table kv-store))) (kv-store-table kv-store)))
(defmacro kv--persist-now (kv-store &rest body)
  (declare (indent 1) (debug ([&rest form] body)))
  `(with-temp-buffer
     ,@body
     (let ((inhibit-message t))
       (write-region nil nil (kv-store-path ,kv-store) t 'silence))))

(defun kv-put (key value kv-store)
  "Associate KEY with VALUE in KV-STORE.
The operation is immediately persisted."
  (kv--persist-now kv-store
    (kv-put-1 key value kv-store)))
(defun kv-rem (key kv-store)
  "Remove KEY from KV-STORE.
The operation is immediately persisted."
  (kv--persist-now kv-store
    (kv-rem-1 key kv-store)))
(defun kv-push (key value kv-store)
  "Add VALUE to the list associated with KEY in KV-STORE.
The operation is immediately persisted."
  (kv--persist-now kv-store
    (kv-push-1 key value kv-store)))
(defun kv-delete (key value kv-store)
  "Remove VALUE from the list associated with KEY in KV-STORE.
The operation is immediately persisted."
  (kv--persist-now kv-store
    (kv-delete-1 key value kv-store)))
(defun kv-get (key kv-store &optional dflt)
  "Look up KEY in KV-STORE and return its associated value.
If KEY is not found, return DFLT which defaults to nil."
  (gethash key (kv-store-table kv-store) dflt))

(defvar inhibit-ask-user-about-lock nil)
(defun inhibit-ask-user-about-lock-advice (orig-func file opponent)
  (if inhibit-ask-user-about-lock
      (signal 'file-locked (list file opponent))
    (funcall orig-func file opponent)))
(advice-add 'ask-user-about-lock :around #'inhibit-ask-user-about-lock-advice)

(defvar persistent-variable-list nil "List of persistent variables.")
(defvar persistent-variable-kv-store (make-kv-store persistent-variable-store-filename))
(defvar persistent-variable-idle-timer (run-with-idle-timer persistent-variable-idle-time t 'persistent-variable-demon))

(defvar persistent-variable-unbound-marker (gensym))
(defun persistent-variable-demon ()
  "Persist any changes of variables in `persistent-variable-list'.

The operation may fail if `persistent-variable-store-filename' is locked.
In such cases, return nil.  If the operation succeeds, return t."
  (dolist (variable persistent-variable-list)
    (with-temp-buffer
      (setq buffer-file-truename (kv-store-path persistent-variable-kv-store))
      (condition-case nil
          (progn
            (setf (buffer-modified-p (current-buffer)) t) ; otherwise `lock-buffer' would do nothing
            (let ((create-lockfiles t)
                  (inhibit-ask-user-about-lock t))
              (lock-buffer))
            (condition-case nil
              (pcase (get variable 'persistence-method)
                ('eql
                 (unless (eql (kv-get variable persistent-variable-kv-store) (symbol-value variable))
                   (kv-put-1 variable (symbol-value variable) persistent-variable-kv-store)))
                ('equal
                 (unless (equal (kv-get variable persistent-variable-kv-store) (symbol-value variable))
                   (kv-put-1 variable (copy-tree (symbol-value variable) t) persistent-variable-kv-store)))
                ('set
                 (let ((old-list (kv-get variable persistent-variable-kv-store))
                       (new-list (symbol-value variable)))
                   (let ((deleted-items (cl-set-difference old-list new-list))
                         (added-items (cl-set-difference new-list old-list)))
                     (mapc (lambda (value) (kv-delete-1 variable value persistent-variable-kv-store)) deleted-items)
                     (mapc (lambda (value) (kv-push-1 variable value persistent-variable-kv-store)) added-items)))))
              (void-variable
               (unless (eq (kv-get variable persistent-variable-kv-store persistent-variable-unbound-marker)
                           persistent-variable-unbound-marker)
                 (kv-rem variable persistent-variable-kv-store))))
            (write-region nil nil buffer-file-truename t 'silent)
            t)
        (file-locked
         (message "Giving up storing persistent variables this time, because %s is locked."
                  (kv-store-path persistent-variable-kv-store))
         nil)))))
(add-hook 'kill-emacs-hook #'persistent-variable-demon)

(cl-defun make-persistent-variable (variable &optional (method 'eql))
  "Make VARIABLE persistent.

If there is an existing entry in `persistent-variable-kv-store',
set the value of VARIABLE to the value in the key value store.

METHOD specifies how VARIABLE is saved and restored,
and can be one of the following:
- `eql': saves the value of VARIABLE if it is not `eql' to last saved value.
- `equal': saves the value of VARIABLE if it is not `equal' to last saved value.
- `set': saves the set difference of the value of VARIABLE compared to last saved value.
Membership test is done using `eql'."
  (cl-pushnew variable persistent-variable-list)
  (let ((persistent-value (kv-get variable persistent-variable-kv-store persistent-variable-unbound-marker)))
    (unless (eq persistent-value persistent-variable-unbound-marker)
      (set variable persistent-value)))
  (setf (get variable 'persistence-method) method))
(defun kill-persistent-variable (variable)
  "Make VARIABLE no longer persistent."
  (setq persistent-variable-list (delq variable persistent-variable-list))
  variable)

(provide 'resist!)
;;; resist!.el ends here

Re: New package: resist!

[Karl Fogel]

On 08 Dec 2021, Qiantan Hong wrote:
> This package implements persistence facility.
> It provides two level of interfaces:
> - A high-level persistent variable facility
> - A low-level persistent key-value store facility
>
> The persistent variable facility detects changes of values
> persistent variables in an idle timer and persist the changes
> into a persistent key-value store.
> Multiple methods for detecting and computing the changes are
> provided. See `make-persistent-variable' for details.
>
> The persistent key-value store provides the following functions:
> - Creating and compressing store: `make-kv-store', 
> `compact-kv-store'
> - Put, remove and look up key value pairs: `kv-put', `kv-rem', 
> `kv-get',
> - List operations: `kv-push', `kv-delete'
> See their docstrings for details.
> All changes are persisted immediately into external storage.

Does this package require the entire store to be loaded into 
memory in order for a single value to be looked up or stored?

It looks like it does (based on reading `kv-get', etc), but I 
might be missing something.

Best regards,
-Karl

Re: New package: resist!

[Qiantan Hong]

> Does this package require the entire store to be loaded into memory in order for a single value to be looked up or stored?
> 
> It looks like it does (based on reading `kv-get', etc), but I might be missing something.

It is. According to my benchmark I don’t think this is a problem
for most Emacs use cases (it loads 10k entry within 0.03s,
I’ve tested longer list).

I do have an idea how to do it without loading the entire store,
by dividing the store into several “buckets” based on hash value
of they keys (and each bucket can be dynamically splitted,
essentially forming a trie). But I don’t know if it’s worth it.

Maybe let’s put the package in use first and see if that’s
necessary? It can be added (mostly) without changing
the interface. The only change I can see is, we may want
kv-store-filename to become kv-store-directory so
those bucket files are stored under it. Or we can retain
the current interface and add suffix to kv-store-filename.
Which one is more preferable?

Re: New package: resist!

[Óscar Fuentes]

Qiantan Hong <qhong@mit.edu> writes:

>> Does this package require the entire store to be loaded into memory in order for a single value to be looked up or stored?
>> 
>> It looks like it does (based on reading `kv-get', etc), but I might be missing something.
>
> It is. According to my benchmark I don’t think this is a problem
> for most Emacs use cases (it loads 10k entry within 0.03s,
> I’ve tested longer list).

Was it with a cold disk cache? SSD or HDD? :-)

I'm afraid that you are focusing too much on a single metric.

You don't mention how large is each record, but anyway 10k records is a
tiny number for any serious database application. And then we have
memory pressure: apart from raw RAM usage, IIRC Emacs' garbage collector
traverses live objects, which means that Emacs becomes slower as the
info it holds grows. Then we have the persistence requirement: if we
want frequent, persistent updates to the database, having to write the
database's entire content each time we want to persist a change is
unacceptable. Then, if we want concurrent access from multiple sessions
(which itself requires some level of support for atomic operations) your
approach of reading and saving a big file becomes unfeasible in
practice, unless for tiny databases.

Don't get me wrong, I'm also worried about introducing sqlite support in
core. I'm afraid that it will be misused, a solution that creates
problems, so to speak, but it is undeniable that there are reasonable
uses for a proper database on Emacs.

Re: New package: resist!

[Qiantan Hong]

> 
> Was it with a cold disk cache? SSD or HDD? :-)
> 
> I'm afraid that you are focusing too much on a single metric.
> 
> You don't mention how large is each record, but anyway 10k records is a
> tiny number for any serious database application. And then we have
> memory pressure: apart from raw RAM usage, IIRC Emacs' garbage collector
> traverses live objects, which means that Emacs becomes slower as the
> info it holds grows.
10k is tiny for business records, but IMO big for Emacs packages.
About the size dependency on record, the takeaway is that the speed
is almost exactly the READ speed, so for loading Lisp values,
it’s basically impossible to beat because you always need to convert
from string to Lisp values even using other database.

Instead, database will benefit from loading lazily, but I can do it too.
It’s unclear to me whether it’s necessary, but if it’s proven to be true,
I could add it.

> Then we have the persistence requirement: if we
> want frequent, persistent updates to the database, having to write the
> database's entire content each time we want to persist a change is
> unacceptable.
This is a common misunderstanding. Resist! is a log structured store
and only writes out increments. It’s even probably “cleverer” than 
stock databases because it understands some Lisp (e.g. list operations)
and you can teach it to understand more.

Maybe to avoid future confusion I should mention this in package header?

Re: New package: resist!

[Qiantan Hong]

> Then, if we want concurrent access from multiple sessions
> (which itself requires some level of support for atomic operations) your
> approach of reading and saving a big file becomes unfeasible in
> practice, unless for tiny databases.
It is not doing that. Currently it has LWW semantics for “basic”
kv, and set semantics for “list” kv.

Emacs lock file is slow so kv-* is not doing locking and leave the
responsibility to caller. persistent-value-* does locking.

Writes from other instances is not immediately visible, but I could
make that happen. Not sure if it’s useful though, unless someone
want to (ab)use it as an IPC channel.

Re: New package: resist!

[Tassilo Horn]

Karl Fogel <kfogel@red-bean.com> writes:

> Does this package require the entire store to be loaded into memory in
> order for a single value to be looked up or stored?
>
> It looks like it does (based on reading `kv-get', etc), but I might be
> missing something.

It does, but I think usually every package/component would have its own
store.  So I don't think that the foobar package loads its complete
store when accessing foobar-value-1 is no big deal.

Bye,
Tassilo

Re: New package: resist!

[Qiantan Hong]

> On Dec 9, 2021, at 1:15 AM, Tassilo Horn <tsdh@gnu.org> wrote:
> 
> Karl Fogel <kfogel@red-bean.com> writes:
> 
>> Does this package require the entire store to be loaded into memory in
>> order for a single value to be looked up or stored?
>> 
>> It looks like it does (based on reading `kv-get', etc), but I might be
>> missing something.
> 
> It does, but I think usually every package/component would have its own
> store.  So I don't think that the foobar package loads its complete
> store when accessing foobar-value-1 is no big deal.

The persistent-variables facility I added however maintains a global store.

Will this turns out to be a problem?
If so, I can either implement general bucket-splitting, or
for the persistent-variable-*, use one store for each package.
Which one makes more sense?

Re: New package: resist!

[Tassilo Horn]

Qiantan Hong <qhong@mit.edu> writes:

>> It does, but I think usually every package/component would have its
>> own store.  So I don't think that the foobar package loads its
>> complete store when accessing foobar-value-1 is no big deal.
>
> The persistent-variables facility I added however maintains a global
> store.

Oh, I haven't checked the new version.

> Will this turns out to be a problem?

Possibly, I don't know.  But what's wrong with simply having
`make-persistent-variable' receive the store as an argument?  That would
allow me to have a foo--store where all my persistent foo-variables of
my foo package reside.  If all of them get loaded at once, that's
totally ok.

Bye,
Tassilo

Re: New package: resist!

[Stefan Monnier]

> You don't mention how large is each record, but anyway 10k records is a
> tiny number for any serious database application.

`resist!` is designed to make persistent data that is usually stored in
Emacs's heap.  If you database is too large that you clearly don't want
to use that.  So far, all the examples I've seen mentioned here (Gnus
registry, savehist, ...) are databases which are currently happily kept
in the Lisp heap, so they should do fine with `resist!`.


        Stefan

Re: New package: resist!

[Stefan Kangas]

Qiantan Hong <qhong@mit.edu> writes:

> This package implements persistence facility

Any chance for a more descriptive name than "resist!.el"?  Thanks.

Re: New package: resist!

[Qiantan Hong]

> Possibly, I don't know.  But what's wrong with simply having
> `make-persistent-variable' receive the store as an argument?  That would
> allow me to have a foo--store where all my persistent foo-variables of
> my foo package reside.  If all of them get loaded at once, that's
> totally ok.
Good idea, I’ll add that in the next revision.

>> This package implements persistence facility
> 
> Any chance for a more descriptive name than "resist!.el"?  Thanks.
Resist! seems very descriptive to me:
It’s the recursive acronym of Resist! pErSISTs!

Jokes aside, persist.el or persistent.el or persistence.el might be
a few candidates, but not sure if they’re used, or reserved
because these are quite broad terms.

Re: New package: resist!

[Qiantan Hong]

The package is now hosted at https://code.librehq.com/qhong/resist/

I think it will be useful to have this package either on GNU Elpa
or Emacs core, or maybe both (for forward compatibility).
Shall we push forward?

>  But what's wrong with simply having
> `make-persistent-variable' receive the store as an argument?
Added.

I’ve also added an automatic compacting daemon.

There’s some remaining technical question:
- What’s the right group to put the customizations?
- currently I ask user to invoke kv-compact-demon-start
  and persistent-variable-demon-start explicitly to
  start/restart the services.
  Are there better way?
  Not sure if those makes sense as minor modes.

Re: New package: resist!

[Stefan Monnier]

> The package is now hosted at https://code.librehq.com/qhong/resist/
>
> I think it will be useful to have this package either on GNU Elpa
> or Emacs core, or maybe both (for forward compatibility).
> Shall we push forward?

Do you have write access so you can add it yourself?
If not, let me know.

I just looked at the code and I think it should be cleaner w.r.t its use
of namespace: the file is `resist!.el` and it defines macros and
functions using the `kv-` prefix as well as the `persistent-` prefix; we
should unify those 3 names.

[ Also the advice functions should have a name that starts with the
  package's namespace prefix.  ]

> There’s some remaining technical question:
> - What’s the right group to put the customizations?
> - currently I ask user to invoke kv-compact-demon-start
>   and persistent-variable-demon-start explicitly to
>   start/restart the services.
>   Are there better way?
>   Not sure if those makes sense as minor modes.

Why not start them lazily when the data is changed (e.g. within
`kv--ensure-transaction`)?


        Stefan

Re: New package: resist!

[Qiantan Hong]

> I just looked at the code and I think it should be cleaner w.r.t its use
> of namespace: the file is `resist!.el` and it defines macros and
> functions using the `kv-` prefix as well as the `persistent-` prefix; we
> should unify those 3 names.
> 
> [ Also the advice functions should have a name that starts with the
>  package's namespace prefix.  ]
I’m thinking the “package” as an extension to Elisp language itself,
so I didn’t have package prefix in mind. Thus the name like `compact-kv-store`.

Maybe the file should be named `persistence.el`?

>> There’s some remaining technical question:
>> - What’s the right group to put the customizations?
>> - currently I ask user to invoke kv-compact-demon-start
>>  and persistent-variable-demon-start explicitly to
>>  start/restart the services.
>>  Are there better way?
>>  Not sure if those makes sense as minor modes.
> 
> Why not start them lazily when the data is changed (e.g. within
> `kv--ensure-transaction`)?
Sure, that will work, thanks.

Re: New package: resist!

[Philip Kaludercic]

Qiantan Hong <qhong@mit.edu> writes:

> I’m thinking the “package” as an extension to Elisp language itself,
> so I didn’t have package prefix in mind. Thus the name like `compact-kv-store`.
>
> Maybe the file should be named `persistence.el`?

I like this name better!

-- 
	Philip Kaludercic

RE: [External] : Re: New package: resist!

[Drew Adams]

> > Maybe the file should be named `persistence.el`?
> I like this name better!

andyetshepersisted.el ;-)

https://en.wikipedia.org/wiki/Nevertheless,_she_persisted

Re: New package: resist!

[Stefan Monnier]

> I’m thinking the “package” as an extension to Elisp language itself,
> so I didn’t have package prefix in mind. Thus the name like `compact-kv-store`.

IMO, *very* few things deserve to be in the "global" namespace beside
`setq`, `lambda`, ...

> Maybe the file should be named `persistence.el`?

I don't have an opinion on the name, as long as you pick one and use it
for both the filename and all the definitions therein ;-)

>> Why not start them lazily when the data is changed (e.g. within
>> `kv--ensure-transaction`)?
> Sure, that will work, thanks.

If you do that, then you can even use a timer that's not repeated, and
instead just make sure subsequent changes will re-start the timer.


        Stefan

Re: New package: resist!

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Date: Fri, 10 Dec 2021 19:24:36 +0000
> Cc: Karl Fogel <kfogel@red-bean.com>, Tassilo Horn <tsdh@gnu.org>,
>  Stefan Monnier <monnier@iro.umontreal.ca>,
>  "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> 
> Qiantan Hong <qhong@mit.edu> writes:
> 
> > I’m thinking the “package” as an extension to Elisp language itself,
> > so I didn’t have package prefix in mind. Thus the name like `compact-kv-store`.
> >
> > Maybe the file should be named `persistence.el`?
> 
> I like this name better!

I suggest 'against-persistence.el'.

Re: New package: resist!

[Alexandre Garreau]

Le vendredo, 10-a de decembro 2021, 20-a horo kaj 57:21 CET Eli Zaretskii 
a écrit :
> > From: Philip Kaludercic <philipk@posteo.net>
> > Date: Fri, 10 Dec 2021 19:24:36 +0000
> > Cc: Karl Fogel <kfogel@red-bean.com>, Tassilo Horn <tsdh@gnu.org>,
> > 
> >  Stefan Monnier <monnier@iro.umontreal.ca>,
> >  "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> > 
> > Qiantan Hong <qhong@mit.edu> writes:
> > > I’m thinking the “package” as an extension to Elisp language itself,
> > > so I didn’t have package prefix in mind. Thus the name like
> > > `compact-kv-store`.
> > > 
> > > Maybe the file should be named `persistence.el`?
> > 
> > I like this name better!
> 
> I suggest 'against-persistence.el'.

persist: it’s shorter, so shorter identifiers

maybe even prsist

Re: New package: resist!

[Qiantan Hong]

> persist: it’s shorter, so shorter identifiers
> 
> maybe even prsist
Sounds good. I’ll do prsist then.

Best,
Qiantan

Re: New package: resist!

[Philip Kaludercic]

Qiantan Hong <qhong@mit.edu> writes:

>> persist: it’s shorter, so shorter identifiers
>> 
>> maybe even prsist
> Sounds good. I’ll do prsist then.

(I know this is bike-shedding, but...) What is the point of leaving out
a single vowel?  Doesn't using a "real" or at least pronounceable word
avoid typos and make it easier to remember?

> Best,
> Qiantan
>

-- 
	Philip Kaludercic

Re: New package: resist!

[Qiantan Hong]

> On Dec 10, 2021, at 12:34 PM, Philip Kaludercic <philipk@posteo.net> wrote:
> 
> Qiantan Hong <qhong@mit.edu> writes:
> 
>>> persist: it’s shorter, so shorter identifiers
>>> 
>>> maybe even prsist
>> Sounds good. I’ll do prsist then.
> 
> (I know this is bike-shedding, but...) What is the point of leaving out
> a single vowel?  Doesn't using a "real" or at least pronounceable word
> avoid typos and make it easier to remember?
With completion there’re no other package starting with `prs`, but
there is `perl-mode` that also starts with `per`, so `prs` save 25%
of the key strokes in this case. But maybe it doesn’t worth it.

Another factor: prsist is obviously a fake word, but persist is a real *verb*.
persist-variable-* looks stranger to me (it’s obviously grammatically incorrect)
than prsist-variable-*.

Being said that, your argument is also sound and I don’t really have a strong 
opinion on either of them. I’ll wait to see if there’s more comments. If not,
I’ll decide it using a random number generator.

Best,
Qiantan

Re: New package: resist!

[Alexandre Garreau]

Le vendredo, 10-a de decembro 2021, 21-a horo kaj 34:38 CET Philip 
Kaludercic a écrit :
> Qiantan Hong <qhong@mit.edu> writes:
> >> persist: it’s shorter, so shorter identifiers
> >> 
> >> maybe even prsist
> > 
> > Sounds good. I’ll do prsist then.
> 
> (I know this is bike-shedding, but...) What is the point of leaving out
> a single vowel?  Doesn't using a "real" or at least pronounceable word
> avoid typos and make it easier to remember?

if you spell out the R like p’R-sist, you get the pronounciation of 
persist.  Plus it’s a common tradition to leave out some vowels to make 
identifiers smaller, such as cnt, ptr, itr (notice there’s still a vowel), 
etc.

prsst could be possible, but the double s is weird, it’s less 
recognizable, and prsist looks more like “resist” than “persist” (there’s 
no “e” in the “wrong” place).

I didn’t think of what Qiantan said about completion and verb meaning, but 
it looks sound to me.  Originally I wasn’t so convinced because “persist” 
looked at least a little more beautiful, I proposed it because I thought 
it since “resist” was suggested to be change into something more 
meaningful.

Re: New package: resist!

[Dmitry Gutov]

On 10.12.2021 22:57, Eli Zaretskii wrote:
> I suggest 'against-persistence.el'.

Viva la Persistance!

RE: [External] : Re: New package: resist!

[Drew Adams]

> > Maybe the file should be named `persistence.el`?
> 
> I don't have an opinion on the name, as long as you
> pick one and use it for both the filename and all
> the definitions therein ;-)

Use the same name for the file and its definitions?

I don't think so.  Did you perhaps mean use the same
name for the file and as a _prefix_ for the names of
the things it defines?

Even then, I don't think so.  AFAIK, there's no stated
convention regarding Lisp file names.  But there is a
convention regarding a library's prefix.

So a file name such as `persist.el', in combination
with a prefix such as `pers-', or `kv-, or `xyz-', or
anything else not already taken, would be conventional.

On the other hand, a prefix that in some way reflects
the library (its name or something else) is generally
helpful for users, mnemonically.

RE: [External] : Re: New package: resist!

[Drew Adams]

> >> maybe even prsist
> > Sounds good. I’ll do prsist then.
> 
> (I know this is bike-shedding, but...) What is the point of leaving out
> a single vowel?  Doesn't using a "real" or at least pronounceable word
> avoid typos and make it easier to remember?

One person's bike-shedding is another's
matter of helpfulness and convenience.

But yes, +1.

If you're looking for a library that persists
things, your search is more likely to find
"persist" than "prsist".  MUCH more.

Re: New package: resist!

[Joost Kremers]

On Fri, Dec 10 2021, Alexandre Garreau wrote:
>> > Qiantan Hong <qhong@mit.edu> writes:
>> > > I’m thinking the “package” as an extension to Elisp language itself,
>> > > so I didn’t have package prefix in mind. Thus the name like
>> > > `compact-kv-store`.
>> > > 
>> > > Maybe the file should be named `persistence.el`?
[...]
> persist: it’s shorter, so shorter identifiers

Note, though, there's already a package on GNU ELPA called "persist":

https://elpa.gnu.org/packages/persist.html

-- 
Joost Kremers
Life has its moments

Re: [External] : Re: New package: resist!

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Before we add this package to Emacs in any way, we should look at its
interface and think about whether to change the details.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: New package: resist!

[Qiantan Hong]

> Note, though, there's already a package on GNU ELPA called "persist":
> 
> https://elpa.gnu.org/packages/persist.html
Oops, now I have no good idea for name.

> I just looked at the code and I think it should be cleaner w.r.t its use
> of namespace: the file is `resist!.el` and it defines macros and
> functions using the `kv-` prefix as well as the `persistent-` prefix; we
> should unify those 3 names.
After second thought, I think it’s quite hard to reconcile them in a descriptive
way. Does that mean I should split the file into a kv.el and persistent.el?

Is it acceptable that there’re some slight deviation from the
prefix tradition, like make-persistent-variable and kill-persistent-variable?

If to put the package on ELPA, should we also split into two packages?
I think it’s logically one suite of persistence facility though, in that
case, is it good to have two file named kv.el and persistent.el
in a package called Persistence (or to avoid confusion with Persist,
screw it and still call it Resist!)?


Best,
Qiantan

Re: New package: resist!

[Tassilo Horn]

Qiantan Hong <qhong@mit.edu> writes:

>> Note, though, there's already a package on GNU ELPA called "persist":
>> 
>> https://elpa.gnu.org/packages/persist.html
> Oops, now I have no good idea for name.

Maybe pkv or pkvs for persistent key-value store?

Bye,
Tassilo

Re: New package: resist!

[Stefan Monnier]

Qiantan Hong [2021-12-11 09:33:04] wrote:
>> Note, though, there's already a package on GNU ELPA called "persist":
>> https://elpa.gnu.org/packages/persist.html
> Oops, now I have no good idea for name.

If you look at persist.el (always a good idea when a package by the
same name shows up), you'll see it has very similar aims.
`org-persist.el` as well.

So it looks like a good opportunity to see if/how you can improve your
package such that it can be either build on top of those packages, or
completely replace some of those packages, or replace some part of
those packages.

> Is it acceptable that there’re some slight deviation from the
> prefix tradition, like make-persistent-variable and kill-persistent-variable?

I hate those deviations, to be honest.  And they're incompatible with
the new symbol shorthand feature.


        Stefan

Re: New package: resist!

[Qiantan Hong]

> If you look at persist.el (always a good idea when a package by the
> same name shows up), you'll see it has very similar aims.
> `org-persist.el` as well.
> 
> So it looks like a good opportunity to see if/how you can improve your
> package such that it can be either build on top of those packages, or
> completely replace some of those packages, or replace some part of
> those packages.
My current plan is to split what’s in resist!.el into two packages
- kv.el, which provides persistent key value store, and should
  support multiple backends. My log store would be one,
  what’s in persist.el already would be one, and potentially
  SQLite3/gdbm/recutils... ones in the future.
- My persistent variable stuff is of different flavor from those
  already in persist.el, mine being more transparent.
  I hope I can let my stuff join persist-* namespace,
  and replace what’s already in persist.el with wrappers
  for backward compatibility.
  But I would need to hear back from author of persist.el
  first.

Best,
Qiantan

Re: New package: resist!

[Alexandre Garreau]

Le sabato, 11-a de decembro 2021, 15-a horo kaj 6:14 CET Stefan Monnier a 
écrit :
> Qiantan Hong [2021-12-11 09:33:04] wrote:
> >> Note, though, there's already a package on GNU ELPA called "persist":
> >> https://elpa.gnu.org/packages/persist.html
> > 
> > Oops, now I have no good idea for name.
> 
> If you look at persist.el (always a good idea when a package by the
> same name shows up), you'll see it has very similar aims.
> `org-persist.el` as well.
> 
> So it looks like a good opportunity to see if/how you can improve your
> package such that it can be either build on top of those packages, or
> completely replace some of those packages, or replace some part of
> those packages.

Btw, eieio-persistent (which you didn’t cite in this mail) also stores one 
value per file, just as persist.el, so they’re technically compatible.

I don’t know how org-persist stores data, however.

But it would be nice if they all shared a common implementation, so it 
would be possible to configure a single backend for all of them at will 
(possible, you could still accept that eieio uses something different from 
org-persist, since they’re on different levels, and you could also configure 
per package, I guess), without for the user to have to multiply its 
internal knowledge of more and more software.

Re: New package: resist!

[Qiantan Hong]

> Btw, eieio-persistent (which you didn’t cite in this mail) also stores one 
> value per file, just as persist.el, so they’re technically compatible.
I think eieio-persistent occupies a weird position here.
It’s not really an implementation of a persistent (key value) store,
it’s more like an object oriented prin1.
I’m not sure how to incorporate eieio-persistent into our program yet.

Re: New package: resist!

[Alexandre Garreau]

Le sabato, 11-a de decembro 2021 22-a horo kaj 13:04 CET, vous avez écrit 
:
> > Btw, eieio-persistent (which you didn’t cite in this mail) also stores
> > one value per file, just as persist.el, so they’re technically
> > compatible.
> I think eieio-persistent occupies a weird position here.
> It’s not really an implementation of a persistent (key value) store,
> it’s more like an object oriented prin1.
> I’m not sure how to incorporate eieio-persistent into our program yet.

It’s very weird because you define one file per object… so yeah it’s not a 
store at all, it just makes objects persist.  But if anyone wants objects 
to persist all in a single store, instead of taking up a lot of different 
spare files (or at least to a predefined directory), they might want to use 
your logging kv, sqlite, gdbm, etc.

What should be implemented in eieio-persistent is a way to define a default 
place to store objects.  By default it would be, for instance, a 
directory, which would be not an object slot but a class attribute.  It 
would make so that when you create an object without specifying a “file” 
slot, the “directory” slot would be used to automatically generate a file 
in it.  And then the directory would be used to restore objects at 
startup.  There would be a global default directory defined for the class 
eieio-persistent, inherited by all persistent objects, and customizable 
through customize.  But you could define child class to eieio-persistent 
with other directories (or subdirectories, if it’s a relative path).  
Possibly, if we want to be strictly compatible with the current (by 
default, if we don’t specify a “file” slot, non-working) behavior, we could 
define a eieio-stored subclass to eieio-persistent, and only this one would 
act as I said.

Then, we could also make customizable the fact that it wouldn’t use one 
file per object, just as persist.el, but also use another backend.

But we should hack eieio-base so that eieio-persistent functions use 
persist.el

Re: New package: resist!

[Alexandre Garreau]

Le sabato, 11-a de decembro 2021, 22-a horo kaj 26:13 CET Alexandre 
Garreau a écrit :
> Le sabato, 11-a de decembro 2021 22-a horo kaj 13:04 CET, vous avez
> écrit
> > > Btw, eieio-persistent (which you didn’t cite in this mail) also
> > > stores
> > > one value per file, just as persist.el, so they’re technically
> > > compatible.
> > 
> > I think eieio-persistent occupies a weird position here.
> > It’s not really an implementation of a persistent (key value) store,
> > it’s more like an object oriented prin1.
> > I’m not sure how to incorporate eieio-persistent into our program yet.
> 
> It’s very weird because you define one file per object… so yeah it’s not
> a store at all, it just makes objects persist.  But if anyone wants
> objects to persist all in a single store, instead of taking up a lot of
> different spare files (or at least to a predefined directory), they
> might want to use your logging kv, sqlite, gdbm, etc.

Moreover, eieio persistence mechanism doesn’t allow to *automatically* 
restore objects.  So you have to manually restore the file name somewhere 
else (wait what’s the point of saving if you still have something else to 
save (necessarily by another mean) in order to be able to restore?).  A 
such mechanism should, for instance, take the filename and use that as a 
variable name to put the object into.

Re: New package: resist!

[Michael Heerdegen]

Qiantan Hong <qhong@mit.edu> writes:

> It’s not really an implementation of a persistent (key value) store,
> it’s more like an object oriented prin1.

How does your package handle values whose print syntax can't just be
`read' to recreate the value?

Michael.

Re: New package: resist!

[Alexandre Garreau]

Le dimanĉo, 12-a de decembro 2021, 2-a horo kaj 5:52 CET Michael Heerdegen 
a écrit :
> Qiantan Hong <qhong@mit.edu> writes:
> 
> 
> 
> > It’s not really an implementation of a persistent (key value) store,
> > it’s more like an object oriented prin1.
> 
> How does your package handle values whose print syntax can't just be
> `read' to recreate the value?

that’s an issue for pretty much any persistence package, and it would just 
as well be while using sqlite, persist, eieio-persistent or anything else.

You must for each of these edge cases make a special condition to invent 
some sort of print value, and recognize it back when reading it…  That is, 
you must invent a way to prin1 it anyway.  I mean storing data necessarily 
requires some sort of serialization (be it native dumping).  It’s part of 
the philosophy of lisp anyway to make anything readable and printable.

Now we could try to see what would happen concretely: what kind of values 
are you thinking to?

Re: New package: resist!

[Qiantan Hong]

>> It’s not really an implementation of a persistent (key value) store,
>> it’s more like an object oriented prin1.
> 
> How does your package handle values whose print syntax can't just be
> `read' to recreate the value?
Currently I haven’t account for it, it will prob — it’s easy to add
a `condition-case` that catches error and warns in the next revision.
Do you have any better suggestions?

> You must for each of these edge cases make a special condition to invent 
> some sort of print value, and recognize it back when reading it…  That is, 
> you must invent a way to prin1 it anyway. 
AFAIU, eieio-persistent looks almost like a generic prin1.
However it doesn’t work on native Lisp objects —
it basically does the thing what we need but with a very weird interface
(e.g. it expect the object itself to remember where it should write to,
which makes it basically impossible to define sensible method on native objects).
An interface like 
(object-write THIS &optional PRINTCHARFUN) makes much more sense to me.

How can we fix this? If this is fixed, we could use such extensible prin1 for all
persistence operation.

Also another namespace problem: the extensible prin1 probably shouldn’t live
in persist-*, because kv-* will depend on this extensible prin1. 
But it now becomes unclear what namespace it should lives in (kv-* doesn’t make
much sense IMO).

I’m now more convinced that this whole matter is more of an Elisp language
extension rather than a package. If we insist to not utilize global namespace,
it probably would need to be divided into lots of namespace.
Is it agreed that each file should use only one namespaces?
Then we would also need to scatter code into quite a few files.

Re: New package: resist!

[Qiantan Hong]

>> It’s not really an implementation of a persistent (key value) store,
>>> it’s more like an object oriented prin1.
>> 
>> How does your package handle values whose print syntax can't just be
>> `read' to recreate the value?
> Currently I haven’t account for it, it will prob 
* it will probably just error when loading the store.

Re: New package: resist!

[Michael Heerdegen]

Alexandre Garreau <galex-713@galex-713.eu> writes:

> > > It’s not really an implementation of a persistent (key value) store,
> > > it’s more like an object oriented prin1.
> > 
> > How does your package handle values whose print syntax can't just be
> > `read' to recreate the value?
>
> that’s an issue for pretty much any persistence package, and it would just 
> as well be while using sqlite, persist, eieio-persistent or anything
> else.

Yes, but it's the hard part.  It is what makes saving the Gnus registry
slow, and it is what makes eieio-persistent look "like an object
oriented prin1".

> Now we could try to see what would happen concretely: what kind of values 
> are you thinking to?

The Gnus registry is one example.  Apart from that: most values,
actually, apart from trivial ones: lists, hash-tables, structs.  Most
applications will not just need to save integers and strings.

Michael.

Re: New package: resist!

[Qiantan Hong]

> actually, apart from trivial ones: lists, hash-tables, structs.  Most
> applications will not just need to save integers and strings.
Lists have readable print syntax in almost every Lisps,
and hash-tables and structs both have readable print syntax
in Emacs Lisp

— that’s one aspect where Emacs Lisp is better than Common Lisp.

Re: New package: resist!

[Michael Heerdegen]

Qiantan Hong <qhong@mit.edu> writes:

> Lists have readable print syntax in almost every Lisps,
> and hash-tables and structs both have readable print syntax
> in Emacs Lisp

It's onyl that you can't use it if one of the elements does not have
readable print syntax.  So you have to traverse the list, and you end up
doing the same as eieio-persistent needs to.

Michael.

Re: New package: resist!

[Alexandre Garreau]

Le dimanĉo, 12-a de decembro 2021, 2-a horo kaj 35:48 CET Michael 
Heerdegen a écrit :
> Alexandre Garreau <galex-713@galex-713.eu> writes:
> 
> 
> 
> 
> > > How does your package handle values whose print syntax can't just be
> > > `read' to recreate the value?
> 
> 
> 
> > that’s an issue for pretty much any persistence package, and it would
> > just as well be while using sqlite, persist, eieio-persistent or
> > anything else.
> 
> Yes, but it's the hard part.  It is what makes saving the Gnus registry
> slow, and it is what makes eieio-persistent look "like an object
> oriented prin1".

No, as said before you’re conflating two separated issues: variable 
persistence, and element-by-element retrieving (from a kv store).  We use 
a kv-store for variable persistence, but that feature is different from the 
later one because you save each value once in its entirety, and then fetch 
it back once in its entirety.  So reading or printing it is actually 
necessary, you cannot make it faster, and it’s not a bottleneck (I/O is 
the bottleneck then), you have to serialize anyway.  When you want to 
fetch individual elements without restoring the whole value in its 
entirety (or perhaps doing just as if it was, in a lazy way, but that’s 
yet again a new feature which has not been discussed yet, and which enters 
in competition with the kernel’s paging/swaping features), you need to 
destructure it manually, you don’t do that automatically, and you want its 
restoration to be coordinated in some determined way… that’s not variable 
persistence.  So the problem you’re talking about, where “read” and 
“write” are slow, is not variable persistence.

I feel like the features we came to discuss are so abstract people keep 
confusing and conflating them together.  They all in some way slightly 
overtake one onto the other, so it makes it understanding overall even 
more difficult…

> > Now we could try to see what would happen concretely: what kind of
> > values are you thinking to?
> 
> The Gnus registry is one example.  Apart from that: most values,
> actually, apart from trivial ones: lists, hash-tables, structs.  Most
> applications will not just need to save integers and strings.

That’s a separate problem, again.  These value are perfectly printable.  
You just claim we can store them so that they’re more efficient to 
(partially) retrieve.  And that’s not a problem solveable by variable 
persistence (since the concept of variable persistence is you restore back 
the entire variable into memory), so it’s not even related to the current 
feature being discussed.

If you want to solve that, you have to use the lower level feature we 
discussed before, a kv storage mechanism, to store your hashtable or 
alist.  A struct is by definition limited so there’s little interest in 
storing it strangely (but you store an array of them you may want a 
relational database, indeed… except until now we only discussed about 
needs of a simple kv storage, which is lower level than a relational 
database).  A *general* list, or an array, may be amenable to a hashtable/
alist with succesive increments as keys, but if you have any more adapted 
idea I’d like to hear it (because otherwise it looks a little overkill).

But you then have to *manually* prepare a store for that, and manually 
split up your sequence into that store.

We cannot apply that to the issue discussed now, variable persistence, 
hence any variable, because that would mean simply recursively make all 
lisp objects which are sequences stored into kv stores (because, as you 
know, a hashtable can contain another hashtable, and two sequences can 
share elements), that’s just overkill, it consists in changing the entire 
elisp runtime, and in the end you just replace every variable with a query 
to a relational database… which is interesting, but has not been discussed 
until now.  You would make one table for each type, and pointers would be 
replaced by foreign keys.  Is that what you want? is that what you’re 
talking about?  That would be the “most efficient way” if we wanted to store 
all emacs ram to disk, and yet be able to retrieve it while being sure to 
do it most efficiently wrt what a relational database can do, with the least 
reading from disk and the least stuff kept into memory.

The ideas in this discussion start getting very weird… now I’m wondering 
if it would be possible, in a perfect world, to at the same time describe 
perfectly the native way of storing variables/(PI/repositionable)pointers/
number in memory so it can directly be dumped to disk and portably be 
decoded at will, and design a relational database format so that it can be 
mmaped to memory and be used to represent variables and complex objects 
directly (or the otherway around: encode, and defragment/fragment/group 
(through action of a special garbage collector) objects into memory so 
that if it’s dumped to disk it gives a most efficient relational database 
data, and yet be effecient to read (like data often used together would be 
near each other)), and use that to represent all data on a computer, for 
every programming language/runtime/software…

Re: New package: resist!

[Alexandre Garreau]

Le dimanĉo, 12-a de decembro 2021, 2-a horo kaj 52:56 CET Michael 
Heerdegen a écrit :
> Qiantan Hong <qhong@mit.edu> writes:
> 
> 
> 
> > Lists have readable print syntax in almost every Lisps,
> > and hash-tables and structs both have readable print syntax
> > in Emacs Lisp
> 
> It's onyl that you can't use it if one of the elements does not have
> readable print syntax.  So you have to traverse the list, and you end up
> doing the same as eieio-persistent needs to.

But then lists are not the “elements which do not have readable print 
syntax”, while we asked you example of such elements.

Yes, you need to traverse the list, but to print a list you traverse it 
anyway.  The idea of an “extensible print” is that during that recursive 
traversing that print already does, if it encounters a “object that it 
doesn’t know how to print”, we teach it to use a function that knows how 
to print it in a such way that it can be read again.

Re: New package: resist!

[Michael Heerdegen]

Alexandre Garreau <galex-713@galex-713.eu> writes:

> Yes, you need to traverse the list, but to print a list you traverse it 
> anyway.  The idea of an “extensible print” is that during that recursive 
> traversing that print already does, if it encounters a “object that it 
> doesn’t know how to print”, we teach it to use a function that knows how 
> to print it in a such way that it can be read again.

This is what eieio-persistent does and what makes it slow.  What would
your package would have to offer then?

Michael.

Re: New package: resist!

[Alexandre Garreau]

Le dimanĉo, 12-a de decembro 2021, 3-a horo kaj 11:29 CET Michael 
Heerdegen a écrit :
> Alexandre Garreau <galex-713@galex-713.eu> writes:
> 
> 
> 
> > Yes, you need to traverse the list, but to print a list you traverse
> > it
> > anyway.  The idea of an “extensible print” is that during that
> > recursive traversing that print already does, if it encounters a
> > “object that it doesn’t know how to print”, we teach it to use a
> > function that knows how to print it in a such way that it can be read
> > again.
> 
> This is what eieio-persistent does and what makes it slow.  What would
> your package would have to offer then?

That’s what any serialisation method do.  It’s necessary.  Unless your 
data is compacted adjacently into memory (some garbage collectors do that, 
not all of them) so you can “serialize” by directly dumping a block of 
memory onto disk.  But nobody do that, because it’s unportable.

So my question is: compared to what?

Our package (actually there would likely be between 2 and 4 packages, with 
between 3 and 4 layers of abstraction, but we’re talking about the highest 
one) offer an interface so that some chosen variable are saved and restored 
between emacs’ sessions.  It’s not even a question of rapidity.  And 
there’s no “faster competitor”.

If you want not to have to access objects, which are all recursive, and 
all of arbitrary dimension, and since they’re lisp, all able to contain 
arbitrary type of data, there’s not even *one* relational database that 
can handle it correctly now (or maybe is that what nosql is about? dunno).  
You are talking about an inexistent feature nobody ever talked about, and 
which has not been implemented before by anybody.

Re: New package: resist!

[Qiantan Hong]

> On Dec 11, 2021, at 5:59 PM, Alexandre Garreau <galex-713@galex-713.eu> wrote:
> 
> If you want to solve that, you have to use the lower level feature we 
> discussed before, a kv storage mechanism, to store your hashtable or 
> alist.  A struct is by definition limited so there’s little interest in 
> storing it strangely (but you store an array of them you may want a 
> relational database, indeed… except until now we only discussed about 
> needs of a simple kv storage, which is lower level than a relational 
> database).  A *general* list, or an array, may be amenable to a hashtable/
> alist with succesive increments as keys, but if you have any more adapted 
> idea I’d like to hear it (because otherwise it looks a little overkill).
> 
> But you then have to *manually* prepare a store for that, and manually 
> split up your sequence into that store.
In fact I think I can do better for these “second-level” key-value mapping
pattern, e.g. a persistent variable bound to a hash-table/alist.
I can translate them to a list key in the .persistent-variables store,
e.g. (variable-symbol key-in-the-hash-table). We can even support more 
indirections.

> This is what eieio-persistent does and what makes it slow.  What would
> your package would have to offer then?
Please read the package description.


Best,
Qiantan

Re: New package: resist!

[Michael Heerdegen]

Alexandre Garreau <galex-713@galex-713.eu> writes:

>  and in the end you just replace every variable with a query to a
> relational database… which is interesting, but has not been discussed
> until now.  You would make one table for each type, and pointers would
> be replaced by foreign keys.  Is that what you want? is that what
> you’re talking about?

No, not really.  I wanted to point to the problems of saving Elisp data
I know from eieio-persistent, but I see you are aware of them.  Since I
did not really understand the purpose of your package yet let me have a
look now.

Michael.