From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lars Ingebrigtsen Newsgroups: gmane.emacs.devel Subject: Re: sqlite3 Date: Tue, 14 Dec 2021 18:46:38 +0100 Message-ID: <87lf0nnkox.fsf@gnus.org> References: <87tufmjyai.fsf@gnus.org> <87lf0nr2b4.fsf@gnus.org> <87tufbnnlq.fsf@gnus.org> <853a500f-ae15-ccd7-561c-a5c03afcaae5@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="37711"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: Dmitry Gutov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Dec 14 18:47:51 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mxBtz-0009Qo-9b for ged-emacs-devel@m.gmane-mx.org; Tue, 14 Dec 2021 18:47:47 +0100 Original-Received: from localhost ([::1]:50232 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxBtx-0006OF-D5 for ged-emacs-devel@m.gmane-mx.org; Tue, 14 Dec 2021 12:47:45 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:60892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxBt0-00051D-KI for emacs-devel@gnu.org; Tue, 14 Dec 2021 12:46:46 -0500 Original-Received: from [2a01:4f9:2b:f0f::2] (port=32906 helo=quimby.gnus.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxBsy-00087N-MZ for emacs-devel@gnu.org; Tue, 14 Dec 2021 12:46:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID :In-Reply-To:Date:References:Subject:Cc:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=592bMdSbTiFHSuP9awk4gdBbtaoUzlWmzRtvlamJBjw=; b=bBtXY/QYeMjgg6HNZaglv1WwTx ehRVle/iui+4qk6QKhL8lUXsqW1aLNv0qwH1Rx0G5eyxtiPT1GtMNzciAFx8WwouDfIpSB2wEqou/ 2eRB09jxSR8UN0WCLKqvhdJm/5lFHWjXoMNMix+dnQN+UusTB6DpboJGqxU/zc3mmy4A=; Original-Received: from [84.212.220.105] (helo=xo) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mxBst-0004VF-7Y; Tue, 14 Dec 2021 18:46:42 +0100 Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAJ1BMVEX9/f3S0tMgHx4O DAyPj5BSUVGwr7AtLCtgX19wb3C+v8KAf3////+zg4Q9AAAAAWJLR0QMgbNRYwAAAAd0SU1FB+UM DhErIP+IrkUAAAGASURBVDjLvZJNT8JAEIZnE6hy2wZS4WjXhHjdsYI3iR+EKyWCNzCA4WbakHI0 FCLe/AD0B/gn/HduW2y7pRx1mnab98nMuzuzACIIo+BH5qYwbqIfgWBvQFa7cuw4mAYLLJWBhXEw CBblheS4BEZE9ayuAe5RAkPQ1r45NWXQE9WOqAr5hiWDrngvjFbPXCdK0dztso68Z/abMgCFusjb 1GWYAAD7TKcgrPUkgLuJ6ABW2iwBvuwWiIxyNZlRGtcAXLTP3WQpQr2M45WdBF6DTeRbuwraj5gK CNsBFPwL8CqBXKifxmUKeyHoxsElef/VDfkUB2GCQWWL6OD8XErJ45Z7zfuokTlWfVmrV4GolGYj 4NsXRU0gbxMt1txKoHsbJzHZByX/5xFooRkDHDLBDA6hsGrExoGwmf0JeUIpIKw5YKlAuBdZKngA zU0FxrA/TQVYsdJLcRY8Omfi1vEInI0WnVln8T3/nM9nz4uPKMPSRTi6ZZWdsuVYgDviP8APOLuy Sfn2JkEAAAAldEVYdGRhdGU6Y3JlYXRlADIwMjEtMTItMTRUMTc6NDM6MzIrMDA6MDD7coEkAAAA JXRFWHRkYXRlOm1vZGlmeQAyMDIxLTEyLTE0VDE3OjQzOjMyKzAwOjAwii85mAAAAABJRU5ErkJg gg== X-Now-Playing: Winston Tong's _Theoretically Chinese_: "Reports from the Heart" In-Reply-To: <853a500f-ae15-ccd7-561c-a5c03afcaae5@yandex.ru> (Dmitry Gutov's message of "Tue, 14 Dec 2021 20:32:59 +0300") X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a01:4f9:2b:f0f::2 (failed) Received-SPF: pass client-ip=2a01:4f9:2b:f0f::2; envelope-from=larsi@gnus.org; helo=quimby.gnus.org X-Spam_score_int: -35 X-Spam_score: -3.6 X-Spam_bar: --- X-Spam_report: (-3.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:281929 Archived-At: Dmitry Gutov writes: >> sqlite files hash >> Read/write value speed =E2=9A=84 =E2=9A=85 =E2=9A=80 >> Read/write value mem =E2=9A=85 =E2=9A=84 =E2=9A=80 >> List all values speed =E2=9A=85 =E2=9A=80 =E2=9A=85 >> List all values mem =E2=9A=83 =E2=9A=81 =E2=9A=83 >> Ease of moving around =E2=9A=84 =E2=9A=80 =E2=9A=85 > > I'm not 100% sure how to interpret (is a higher value for "mem" better > or worse?), but it seems like, at least, for the original scenario of > having large data sets sqlite might still be optimal. =E2=9A=85 is better than =E2=9A=84 etc. > OK, but it's still the same number of writes, more or less? Yup.=20=20 > IO is the slow part of most programs, and when it comes to an SQL > database, it might have to do an update in multiple places (e.g. the > data and the index), rather than do one smooth write. > > Might also depend on the size of the write (how big the values are). I think the main issue is probably that sqlite tries really hard to maintain data integrity, so it does fsyncing etc. (More or less according to what pragmas you give it.) It's nice in general, but probably overkill for this use case. > Speaking of the latter scheme, I might be missing some details, but > sqlite should provide better atomicity guarantess in the same of being > interrupted mid-write. Like, if we have one-file-per-value, then the > total list of keys must live somewhere, and they can get > desynchronized. There is no total list of keys in multisession.el -- you define separate objects that are each instantiated to disk. --=20 (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no