Re: sqlite3

[Pip Cet <pipcet@gmail.com>]

On Wed, Dec 8, 2021 at 6:57 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Wed, 8 Dec 2021 18:36:24 +0000
> > Cc: eric@ericabrahamsen.net, cesar mena <cesar.mena@gmail.com>,
> >  Richard Stallman <rms@gnu.org>, emacs-devel@gnu.org
> >
> > > Just read the first one.  It explains everything.sq
> >
> > I disagree, FWIW. There's very little in that message that goes beyond
> > "let's use sqlite!"
>
> Why is that a problem?

Lars said his message explains everything. I say it doesn't. Whether
that is a problem or not is up to you.

> > The proposal is to have Emacs store some user data in some binary
> > format
>
> No, the proposal is to have Emacs _capable_ of storing data in a DB.

See, apparently, not even that is made clear in the message.

However, having reread it, I maintain that Lars is indeed proposing to
have Emacs store etc. etc. I just don't see another way of reading it.

> We didn't yet seriously discuss whether some core features will
> actually use that, and if so, which ones.

I agree that the necessary discussion has not taken place.

> > that cannot be readily inspected, diffed, backed up,
> > version-controlled, shared, altered, or understood. (Or archived,
> > published, indexed, checksummed, ...)
>
> That's incorrect: tools exist that can browse sqlite databases and
> compare them.

"readily".

> > Those are huge disadvantages. The benefits are not clear to me at all,
> > but they seem to revolve around the time it takes for an interrupted
> > or terminated session to restart?
>
> No, the benefit is to be able to store and retrieve data from a
> persistent DB.  Python has it, Lua has it, JS has it, so why
> shouldn't Emacs be able to have it?

That's not the proposal as I read it.

> > I don't think there are any applications for which the trade off (gain
> > some performance; lose some user freedom) is worth it, but even if
> > there are, we should look harder at alternatives which allow us to
> > gain similar amounts of performance without littering unsuspecting
> > users' .emacs.ds with binary-only files.
>
> If you are right, then this capability will remain largely unused.
> Like Lisp threads, for example.  Let's talk in a couple of years.

I don't believe in the infinite wisdom of package maintainers,
particularly when it comes to not doing a technically nifty thing that
happens to have actual side effects.

> > And we need to be careful about introducing this "as an option", of
> > course, because that means two versions of many packages' core code
> > need to be maintained, only one of which will be tested by users who
> > have not had the time to customize the defaults.
>
> No, it means the packages that need such a capability will _require_
> Emacs with sqlite support.  Like any package that needs to display
> SVG requires Emacs with librsvg support, for example.

Same effect: users will be forced to have to deal with binary files,
because they either don't have the option not to use them or won't
notice they're using them until it's too late. Let's avoid that.

> > I've tested a little and at least in those initial tests, sqlite seems
> > to actually delete data from sqlite files when the corresponding rows
> > are deleted, but I'd like to mention this explicitly because I'm not
> > sure there's actually a guarantee of it in place: if sqlite ever keeps
> > storing data that the user thought they deleted, that makes it
> > unsuitable for storing the kind of data I would expect in .emacs.d.
>
> Your file system doesn't actually erase the data from the disk,
> either, as you probably well know.  That doesn't mean we cannot use
> disk I/O in Emacs.

I think the danger of people accidentally sharing a file without
considering what invisible extra information it might contain
(particularly if there's no good reason for such invisible information
to be present in the first place) is MUCH larger than that of people
deliberately sharing an image of their decrypted file system volume,
expecting only current file system-visible content to be preserved.

The first happens on a regular basis with blacked-out PDFs, Microsoft
Word and Microsoft Excel documents, and other such abominations. I've
not heard of the latter happening on any significant scale.

The former makes it unsuitable for .emacs.d. The latter only makes it
important to keep educating people about what file systems do and do
not do.

> > Of course, none of this is meant to stop people experimenting in any
> > way, but I don't see any package that should realistically be able to
> > make use of it without an opt-in option and probably an extra warning.
>
> I don't see any reason for these precautions.  We never did that for
> any other optional feature that requires an optional library.

You're right that features of Emacs were usually introduced by turning
them on by default, then breaking the "off" option because no one used
it anymore (see pdumper). I don't think that this approach has been
hugely successful, certainly not enough to justify repeating that
mistake in this case.

Pip