From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Matt Armstrong Newsgroups: gmane.emacs.devel Subject: Re: persistent data feature Date: Fri, 10 Dec 2021 11:32:01 -0800 Message-ID: <87fsr0gssu.fsf@rfc20.org> References: <87tufmjyai.fsf@gnus.org> <877dcil2sj.fsf@ericabrahamsen.net> <87czm98qi1.fsf@gnu.org> <87o85tcwm0.fsf@ericabrahamsen.net> <874k7ljwkr.fsf@gnus.org> <87fsr5cuzq.fsf@ericabrahamsen.net> <878rwx8mdn.fsf@gnu.org> <87r1aphuei.fsf@gnus.org> <837dcex6ub.fsf@gnu.org> <87bl1p10js.fsf@gnus.org> <87a6h8y5j6.fsf@gnus.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18646"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Richard Stallman , eric@ericabrahamsen.net, cesar.mena@gmail.com, emacs-devel@gnu.org, Pip Cet , eliz@gnu.org To: Stefan Monnier , Lars Ingebrigtsen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Dec 10 20:33:52 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mvleS-0004cH-7N for ged-emacs-devel@m.gmane-mx.org; Fri, 10 Dec 2021 20:33:52 +0100 Original-Received: from localhost ([::1]:51198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvleR-0003dt-30 for ged-emacs-devel@m.gmane-mx.org; Fri, 10 Dec 2021 14:33:51 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:47500) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvlcr-0002vH-Nn for emacs-devel@gnu.org; Fri, 10 Dec 2021 14:32:13 -0500 Original-Received: from relay8-d.mail.gandi.net ([217.70.183.201]:33137) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvlcp-0001ay-6H; Fri, 10 Dec 2021 14:32:13 -0500 Original-Received: (Authenticated sender: matt@rfc20.org) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id D15171BF205; Fri, 10 Dec 2021 19:32:04 +0000 (UTC) Original-Received: from matt by naz with local (Exim 4.95) (envelope-from ) id 1mvlcf-002RQr-Bi; Fri, 10 Dec 2021 11:32:01 -0800 In-Reply-To: Received-SPF: pass client-ip=217.70.183.201; envelope-from=matt@rfc20.org; helo=relay8-d.mail.gandi.net X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:281617 Archived-At: Stefan Monnier writes: >> But the current large data structures are in things like the Gnus >> registry, which can grow arbitrarily large (and slow, unless you trim >> it). And my .ecompleterc is 4MB. > > 4MB doesn't sound like it should be a problem, and I don't see much > benefit in having ecompleterc grow much further (by its nature it > contains a lot of "irrelevant" data and it's good to trim it every once > in a while for that reason). Also having it too large would slow down > completion itself, so I suspect that storing it in a sql database would > not help. > > The gnus-registry is a much more compelling example (especially since > there's no reason we should regularly scan it from start to end, we > only really ever need to look up specific entries, so the indexing of > the database should allow answering those queries efficiently without > having to read the whole DB from the disk (tho nowadays reading 100MB > from the disk is basically instantaneous anyway). > > So, IIRC the only current example is gnus-registry. That sounds good. Another example is Org Roam (https://www.orgroam.com/), which uses sqlite to manage its index of Org metadata.