From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet Newsgroups: gmane.emacs.devel Subject: Re: sqlite3 Date: Thu, 9 Dec 2021 19:39:02 +0000 Message-ID: References: <87tufmjyai.fsf@gnus.org> <877dcil2sj.fsf@ericabrahamsen.net> <87czm98qi1.fsf@gnu.org> <87o85tcwm0.fsf@ericabrahamsen.net> <874k7ljwkr.fsf@gnus.org> <87fsr5cuzq.fsf@ericabrahamsen.net> <878rwx8mdn.fsf@gnu.org> <87r1aphuei.fsf@gnus.org> <87wnke4p21.fsf@gnus.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="9861"; mail-complaints-to="usenet@ciao.gmane.io" Cc: eric@ericabrahamsen.net, cesar mena , Richard Stallman , emacs-devel@gnu.org To: Lars Ingebrigtsen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Dec 09 20:42:57 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mvPJg-0002QM-VS for ged-emacs-devel@m.gmane-mx.org; Thu, 09 Dec 2021 20:42:57 +0100 Original-Received: from localhost ([::1]:49840 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mvPJf-0003aU-NJ for ged-emacs-devel@m.gmane-mx.org; Thu, 09 Dec 2021 14:42:55 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:58460) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mvPGh-00029e-9X for emacs-devel@gnu.org; Thu, 09 Dec 2021 14:39:51 -0500 Original-Received: from [2a00:1450:4864:20::135] (port=40751 helo=mail-lf1-x135.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mvPGf-0004Fw-7Y; Thu, 09 Dec 2021 14:39:50 -0500 Original-Received: by mail-lf1-x135.google.com with SMTP id l22so14000080lfg.7; Thu, 09 Dec 2021 11:39:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=W9IYvQIYnMbcGj/rE+K+PoBTPm4LCJ5jUPv9S8l4ilE=; b=Ya0VtwL9Zb9LUEipQf9QmTpKNcEK92j008x1JL2XmrlYvRE71tCGaQDQr8/ndrEmzh CXr1gazcWyILZnRhfqur/9zW/MA3T/u6qA6O0zoZfrv5MKtmN593CdG7f5LEiZMdd9eO zUA/Un0JbkDDI3v9UCW+EXyveG+V5x/p1GnAyBYjtgdqIDfCUKU4YR+iTm/6uQfnAUS6 fdNa1yuDJYrKFL878/MH7jO/U5ymHy5525zlnQB5ItnzGMwqYMw8f9346XwNp6b/qkCt TXwwTvQwug9HwQ77QR3bv63CyS60/4/zokkuda7t9uOkruc/t7FIh9XmIxiTktpiiQXd GPow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=W9IYvQIYnMbcGj/rE+K+PoBTPm4LCJ5jUPv9S8l4ilE=; b=UpC+uB8RzIkgZX3SZl9qqovLEW+BbtrVMJohm9MWxmqMBgefACM0lnX0ciY3hGJ5tl 7Qmj+cMxlmCkfv9tGZzu+4qPy3zKONd2Xonu7Ml7PhNHcLp7MkCdlyUVT/4s2V0kbKwg rphjmKg51N2ECKD6P1gFl065CXbRlI72JUty2OyIfZTxCm4wfc+t9Zpui2fa0A13bq1f nrnpY2DffN7IAQ0aP33tNrmEkjxKCqGOwNTEMMfoA7G+M6Y3Gg4KmRrMz6BWDHAXQs7Y bYn2WgEAzNSxvL1v6yr3BzvFQdFgE4x449kUGaVYBd9Vbn96OlHrfHf+AcNrgBH2796H Bw8A== X-Gm-Message-State: AOAM532UoRMUjZnwdPRpTGBQ1lwwoVjCRespsMxhJb/4sfLBHH5e6FfT iWvrE5aWV74aAeafbml0D035Gfp4YQ72OU0nci4= X-Google-Smtp-Source: ABdhPJwJCsVMdDalel7qSKxWTWMGAYpqwTdhKVHjtRgiMNFP4IfjrZkru4W/Bcn9wiCK766dSfphSK4SAZJq9X0T+18= X-Received: by 2002:a05:6512:228a:: with SMTP id f10mr7647581lfu.463.1639078780303; Thu, 09 Dec 2021 11:39:40 -0800 (PST) In-Reply-To: <87wnke4p21.fsf@gnus.org> X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::135 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::135; envelope-from=pipcet@gmail.com; helo=mail-lf1-x135.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:281536 Archived-At: On Thu, Dec 9, 2021 at 12:09 AM Lars Ingebrigtsen wrote: > Pip Cet writes: > > The proposal is to have Emacs store some user data in some binary > > format that cannot be readily inspected, diffed, backed up, > > version-controlled, shared, altered, or understood. (Or archived, > > published, indexed, checksummed, ...) > > The slope is pretty slippery from .emacs to .newsrc.eld to Gnus registry > storage to .sqlite files. I agree that .newsrc.eld and the Gnus registry, as you describe them, have problems that need to be fixed. Making those permanently unfixable by switching to a binary-only format seems like a bad idea to me, unless there are overwhelming advantages that I'm just failing to see so far. > The .emacs file with the Customize settings is usually something a user > can read and edit and diff with some confidence. I'm glad you're not proposing to convert that! > The .newsrc.eld file, which is a couple of huge alists with complicated > nesting, is very hard to read, edit or diff, even thought it's "text". Then we need to work on improving its format. > Putting it in VC is meaningless. How so? I usually have my home directory under version control, and git's xdelta implementation should deal with alists just fine (and produce readable output at sub-line granularity)... > The Gnus registry, which is a dumped hash table, can't meaningfully be > read or edited by anybody unless that person is an expert on hash table > representation and what it all means. Then we need to improve that. It's not a huge change to allow dumping hash tables in predictable order (rather than predictable-unless-the-bucket-count-changed order) and at one tuple per line; are there additional issues? > And it's not diffable or VC-able > without special software. Possibly so. That still means you're making problems that are currently coincidental fundamentally unfixable. > An .sqlite file isn't pretending to be user editable in vi, but there's > a plethora of software out there do do all things you'd expect, and (of How much of it is GNU software? > course) Emacs will come with a mode to inspect and edit these files more > free-form. Indeed, I would have considered it a good idea to introduce such a mode well before proposing (as you might or might not have done) that random packages start using a new library. > They will, in the end, be a lot more editable than a hash > table dump: This mode will allow you to say "delete all values that > refer to 'foo' no matter where it's stored in this file", and that's not > something that's anybody implemented for all these "text" formats that > Emacs uses today. You're touching on an important point here: if we go the sqlite route, efforts will be split between improving things for text files (something that is desperately necessary) and improving things for sqlite databases. Again, I'm not saying Emacs is currently perfect (blasphemy!), but at least it's fixable in principle, without duplicating too much effort. > As for archiving, publishing, indexing or checksumming: Doing that with > an .sqlite file is a lot more meaningful than a file containing a hash > table dump. (I don't think sqlite (or sqlite3) is considered an acceptable format by most archives, or publishers. ASCII text is.) I don't see how indexing an sqlite file, without first converting it into text format, gives you anything valuable: does sqlite even guarantee long strings won't be split up arbitrarily? And a checksum that changes with no underlying changes in the data is problematic. Usually not a problem with ELisp dumps, but it usually is with sqlite files. I must confess my experience with sqlite is not very recent; back then, it fsync()ed heavily (and unnecessarily), created additional files that needed to be kept in sync with each other, and often ended up corrupting the databases anyway. Even if those issues have been fixed, I feel almost zero consideration has been given to alternatives. I must repeat that interacting with existing sqlite files is something Emacs should be able to do, but creating new ones for Emacs purposes is quite a different proposal, and by my reading it's what you actually suggested. And, merely as a request, I'd still like to hear a further convincing case for all this being necessary in the first place. So far I've heard "Gnus" and "emoji favorites". I can't currently use the former and the latter is inaccessible to me on multiple levels, so I'm left with my "binary junk in .emacs.d" impression of what you're actually proposing. Pip