From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Alexandre Garreau Newsgroups: gmane.emacs.devel Subject: Re: New package: resist! Date: Sun, 12 Dec 2021 02:59:38 +0100 Message-ID: <3759747.ti9LBWOQnn@galex-713.eu> References: <4660513.oQR9SAZQBz@galex-713.eu> <87fsqyy58r.fsf@web.de> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1938"; mail-complaints-to="usenet@ciao.gmane.io" To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Dec 12 03:00:26 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mwEA6-0000Io-6J for ged-emacs-devel@m.gmane-mx.org; Sun, 12 Dec 2021 03:00:26 +0100 Original-Received: from localhost ([::1]:45128 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mwEA4-0002M9-UG for ged-emacs-devel@m.gmane-mx.org; Sat, 11 Dec 2021 21:00:25 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:33286) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwE9R-0001ZN-Nr for emacs-devel@gnu.org; Sat, 11 Dec 2021 20:59:45 -0500 Original-Received: from [2a00:5884:8305::1] (port=36450 helo=galex-713.eu) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwE9P-0004Dm-8Q for emacs-devel@gnu.org; Sat, 11 Dec 2021 20:59:45 -0500 Original-Received: from gal by galex-713.eu with local (Exim 4.94.2) (envelope-from ) id 1mwE9K-001pXI-SM for emacs-devel@gnu.org; Sun, 12 Dec 2021 02:59:38 +0100 In-Reply-To: <87fsqyy58r.fsf@web.de> X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:5884:8305::1 (failed) Received-SPF: pass client-ip=2a00:5884:8305::1; envelope-from=galex-713@galex-713.eu; helo=galex-713.eu X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:281722 Archived-At: Le diman=C4=89o, 12-a de decembro 2021, 2-a horo kaj 35:48 CET Michael=20 Heerdegen a =C3=A9crit : > Alexandre Garreau writes: >=20 >=20 >=20 >=20 > > > How does your package handle values whose print syntax can't just be > > > `read' to recreate the value? >=20 >=20 >=20 > > that=E2=80=99s an issue for pretty much any persistence package, and it= would > > just as well be while using sqlite, persist, eieio-persistent or > > anything else. >=20 > Yes, but it's the hard part. It is what makes saving the Gnus registry > slow, and it is what makes eieio-persistent look "like an object > oriented prin1". No, as said before you=E2=80=99re conflating two separated issues: variable= =20 persistence, and element-by-element retrieving (from a kv store). We use=20 a kv-store for variable persistence, but that feature is different from the= =20 later one because you save each value once in its entirety, and then fetch= =20 it back once in its entirety. So reading or printing it is actually=20 necessary, you cannot make it faster, and it=E2=80=99s not a bottleneck (I/= O is=20 the bottleneck then), you have to serialize anyway. When you want to=20 fetch individual elements without restoring the whole value in its=20 entirety (or perhaps doing just as if it was, in a lazy way, but that=E2=80= =99s=20 yet again a new feature which has not been discussed yet, and which enters= =20 in competition with the kernel=E2=80=99s paging/swaping features), you need= to=20 destructure it manually, you don=E2=80=99t do that automatically, and you w= ant its=20 restoration to be coordinated in some determined way=E2=80=A6 that=E2=80=99= s not variable=20 persistence. So the problem you=E2=80=99re talking about, where =E2=80=9Cr= ead=E2=80=9D and=20 =E2=80=9Cwrite=E2=80=9D are slow, is not variable persistence. I feel like the features we came to discuss are so abstract people keep=20 confusing and conflating them together. They all in some way slightly=20 overtake one onto the other, so it makes it understanding overall even=20 more difficult=E2=80=A6 > > Now we could try to see what would happen concretely: what kind of > > values are you thinking to? >=20 > The Gnus registry is one example. Apart from that: most values, > actually, apart from trivial ones: lists, hash-tables, structs. Most > applications will not just need to save integers and strings. That=E2=80=99s a separate problem, again. These value are perfectly printa= ble. =20 You just claim we can store them so that they=E2=80=99re more efficient to= =20 (partially) retrieve. And that=E2=80=99s not a problem solveable by variab= le=20 persistence (since the concept of variable persistence is you restore back= =20 the entire variable into memory), so it=E2=80=99s not even related to the c= urrent=20 feature being discussed. If you want to solve that, you have to use the lower level feature we=20 discussed before, a kv storage mechanism, to store your hashtable or=20 alist. A struct is by definition limited so there=E2=80=99s little interes= t in=20 storing it strangely (but you store an array of them you may want a=20 relational database, indeed=E2=80=A6 except until now we only discussed abo= ut=20 needs of a simple kv storage, which is lower level than a relational=20 database). A *general* list, or an array, may be amenable to a hashtable/ alist with succesive increments as keys, but if you have any more adapted=20 idea I=E2=80=99d like to hear it (because otherwise it looks a little overk= ill). But you then have to *manually* prepare a store for that, and manually=20 split up your sequence into that store. We cannot apply that to the issue discussed now, variable persistence,=20 hence any variable, because that would mean simply recursively make all=20 lisp objects which are sequences stored into kv stores (because, as you=20 know, a hashtable can contain another hashtable, and two sequences can=20 share elements), that=E2=80=99s just overkill, it consists in changing the = entire=20 elisp runtime, and in the end you just replace every variable with a query= =20 to a relational database=E2=80=A6 which is interesting, but has not been di= scussed=20 until now. You would make one table for each type, and pointers would be=20 replaced by foreign keys. Is that what you want? is that what you=E2=80=99= re=20 talking about? That would be the =E2=80=9Cmost efficient way=E2=80=9D if w= e wanted to store=20 all emacs ram to disk, and yet be able to retrieve it while being sure to=20 do it most efficiently wrt what a relational database can do, with the leas= t=20 reading from disk and the least stuff kept into memory. The ideas in this discussion start getting very weird=E2=80=A6 now I=E2=80= =99m wondering=20 if it would be possible, in a perfect world, to at the same time describe=20 perfectly the native way of storing variables/(PI/repositionable)pointers/ number in memory so it can directly be dumped to disk and portably be=20 decoded at will, and design a relational database format so that it can be= =20 mmaped to memory and be used to represent variables and complex objects=20 directly (or the otherway around: encode, and defragment/fragment/group=20 (through action of a special garbage collector) objects into memory so=20 that if it=E2=80=99s dumped to disk it gives a most efficient relational da= tabase=20 data, and yet be effecient to read (like data often used together would be= =20 near each other)), and use that to represent all data on a computer, for=20 every programming language/runtime/software=E2=80=A6