From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Karl Fogel Newsgroups: gmane.emacs.devel Subject: Re: sqlite3 Date: Mon, 06 Dec 2021 14:16:20 -0800 Message-ID: <87pmq9751n.fsf@red-bean.com> References: <87tufmjyai.fsf@gnus.org> <875ys2lyt0.fsf@yahoo.com> <897C2ADC-9F9D-4D3E-B427-773EC7CE561C@mit.edu> Reply-To: Karl Fogel Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="16497"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.90 (gnu/linux) Cc: "emacs-devel@gnu.org" To: Qiantan Hong Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Dec 06 23:17:07 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1muMIF-00043R-4Q for ged-emacs-devel@m.gmane-mx.org; Mon, 06 Dec 2021 23:17:07 +0100 Original-Received: from localhost ([::1]:39476 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1muMID-0000OM-Jj for ged-emacs-devel@m.gmane-mx.org; Mon, 06 Dec 2021 17:17:05 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:33116) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1muMHa-0007vJ-SZ for emacs-devel@gnu.org; Mon, 06 Dec 2021 17:16:26 -0500 Original-Received: from sanpietro.red-bean.com ([45.79.25.59]:49962) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1muMHY-0007BT-IG for emacs-devel@gnu.org; Mon, 06 Dec 2021 17:16:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=red-bean.com; s=202005newsp; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Message-ID:In-Reply-To:Date:Reply-To:References:Subject:Cc:To: From:Sender:Content-ID:Content-Description; bh=++w2o8Sk8EEU5MeAgBPs4dVL7oVwq51196hGMHNLOks=; t=1638828983; x=1640038583; b=kB8N/GyhcGkVgKlfhRSgmo23d7YhEQz/GBXKhOrtyr22OjBj6vH5pLQ/YpMdRzDz15n6GpKqSyY iSmKUdxy4k+FxLxPgOZM8R0IGY+bOneM73aFDpNzqbb92km8asct30ZeJnrxw/7SXT70UC0o9EwoP M8ekdVhq3a9d36Xidrtxui2okCVkmNc0XTM8gusCTQNJ1vIrOndc2FZY+vLMI/OJ6hr3UzqZW1XP3 kRNBNRVpsb5mCmAoWwNhHVe9ZmjqFUIbXmeDtiYVyweypcDi/onnEeRJmHYRzM1dSvohSGyba8emd cxEEG+DW9fZPBBvoHzovCigbxBegBpsqic5A==; Original-Received: from [198.44.217.39] (port=41358 helo=klen) by sanpietro.red-bean.com with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1muMHV-00Awlr-Lc; Mon, 06 Dec 2021 22:16:21 +0000 In-Reply-To: (Qiantan Hong's message of "Mon, 6 Dec 2021 19:59:05 +0000") Received-SPF: pass client-ip=45.79.25.59; envelope-from=kfogel@red-bean.com; helo=sanpietro.red-bean.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:281184 Archived-At: On 06 Dec 2021, Qiantan Hong wrote: > As for key value store, if you really want a persistent > kv store with O(1) insert, couldn=E2=80=99t we just use something like=20 > the following super simple (skeleton) code? > > (defvar kv-store-table) > (cl-defstruct (kv-store (:constructor make-kv-store-1)) path=20 > table) > (defun make-kv-store (path) > (let* ((kv-store (make-kv-store-1 :path path)) > (kv-store-table (make-hash-table :test 'equal))) > (when (file-exists-p path) > (load-file path) > ;; and some error handling to ignore clear trailing=20 > un-balanced form > ;; just in case Emacs crash in the middle of a kv-put > ) > (setf (kv-store-table kv-store) kv-store-table) > kv-store)) > (defun compact-kv-store (kv-store) > ;; dump the full content of kv-store-table at once > ;; to compress the log and speed up loading > ) > (defun kv-put (key value kv-store) > (with-temp-buffer > (print `(puthash ',key ',value kv-store-table)=20 > (current-buffer)) > (append-to-file nil nil (kv-store-path kv-store))) > (puthash key value (kv-store-table kv-store))) > (defun kv-get (key kv-store) > (gethash key (kv-store-table kv-store))) As a data point: I wrote a similar generic data-persistence mechanism a while ago,=20 because I had exactly the kind of need Lars described in his OP.=20 The code has just been living in my .emacs [1]. I never tried to=20 package it up as an independent thing, and I'm sure that with more=20 & wiser heads we'd easily come up with a better system. I suspect=20 that I'm not the only person who implemented a personal=20 persistence mechanism :-). Just having _one_ persistence mechanism (as opposed to everyone's=20 personal improvisation) would be an improvement. I liked Lars's=20 sqlite3 proposal, but it doesn't have to be sqlite3 and I don't=20 have a strong enough opinion to argue hard for that versus some=20 more Lisp-y solution. However, whatever the system is, it shouldn't require loading the=20 entire data store into memory. That's a key test that sqlite3=20 passes; if we do something else instead, it should also pass that=20 test. Best regards, -Karl [1] https://svn.red-bean.com/repos/kfogel/trunk/.emacs, search for "A generic cross-session data persistence mechanism."