Re: Unboxed package manager

[Lynn Winebarger <owinebar@gmail.com>]

On Mon, Mar 20, 2023 at 2:11 PM Jonas Bernoulli <jonas@bernoul.li> wrote:
>
> Lynn Winebarger <owinebar@gmail.com> writes:
>
> > I think I'm going to hack something together starting with advice on the
> > existing package management and taking some inspiration from the design of
> > Jonas Bernoulli's epkg and emir packages for tracking installed packages
> > and component files in a SQLite database.
>
> If I were to start over now, I wouldn't write Closql.  At the time it
> made a lot of sense because I knew nothing about databases and because
> it allowed me to switch out the old data store ("everything is *its own*
> file") without changing internal interfaces much.  Moving from files to
> a database did wonders for performance, so at first I didn't mind the
> performance impact of the OO interface on top of the database.
> Meanwhile I have moved away from the OO interface for anything that
> deals with more than one package at a time, turning many rows into
> EIEIO objects is a bit costly.

Hmm - I had thought it might be an interesting exercise to get
acquainted with Emacs's version of CLOS, but maybe not.  However, just
reviewing the way you've organized the package data is probably going
to be useful.  For example, trying to understand the best way to
assess whether a particular version of a package is an upgrade.  I
don't know about the behavior of package.el in master, but
historically, if there is a version of a package on MELPA and the same
package is available on GNU or NONGNU ELPA, package.el will treat the
version on GNU/NONGNU as an upgrade even though the one on MELPA is
more recent.  I assume it has something to do with the comparison of
commit hashes or dates with more traditional version numbers, but
that's just a guess.

>
> When I switched to SQLite, Emacs had no built-in support (coming in
> Emacs 29) and there also was no module, so EmacSQL was the natural
> choice.  I am the maintainer of that now, so I definitely think it
> serves a purpose, but I do have some reservations.
>
> The next release will feature new backends that use the built-in support
> and a module, but if I were to start now, I probably would go with the
> built-in support directly.
>
> EmacSQL allows writing SQL using vectors instead of concatenating
> strings, which is nice, but for people just getting started with SQL, it
> has the disadvantage that you now have to learn two things, SQL and the
> almost SQL vector syntax, which isn't 100% complete and doesn't map 1:1.
>
> The main limitation of EmacSQL is that it stores everything (except
> NULL) as a string.  This is why I would probably avoid it now, because
> it limits interoperability with anything that doesn't use EmacSQL.

That's only because it was designed to interact with sqlite through a
pipe to the shell program, though, right?  It seems like a method for
compiling sexpr-type representations of sql queries into statements
usable with the builtin support would still be useful, and not limited
in the same way - since the returned values do not require serializing
as text by the sqlite shell then parsing them in Elisp.

[ Digression...]
Just looking at the src/sqlite.c in master, as there is no other
documentation of the sqlite support I can see, only a simplified form
of the core API is supported - not unreasonable for an initial
release.  The only noticeable absence I see, based on a cursory review
of the sqlite3 API spec, is that a select query cannot be reset.
Maybe because each db connection is associated with at most one
prepared statement at a time by the design of the Lisp_Sqlite
pseudovector?

Other than that, I note that rows are returned as lists rather than
arrays, which makes the semantics more incompatible with emacsql than
it really has to be.  Can that be changed before 29 is released?

The associated sqlite-mode looks interesting.  I only wonder why it
doesn't derive from tabulated-list mode instead of directly from
special.  Tabulated list mode would seem to be made for things like
database tables.

Lynn