Re: Cuirass news

[ludo@gnu.org (Ludovic Courtès)]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> I've now forked guile-sqlite3 and put the stuff that makes sense for a
> low-level binding there (not sqlite-exec - that would still be in guix-cuirass).
>
> These are:
> - stmts hash-table is now in guile-sqlite3's <db>.
> - sqlite-finalize adapted to update <db>'s stmts list
> - sqlite-finalize* variant which doesn't update <db>'s stmts list
> (because the updating by value is slow)
> - sqlite-prepare* macro which stores and reuses existing <stmt>s.
> - sqlite-bind-args convenience function which just binds all the parameters
> in sequence.
> - SQLITE_CONSTRAINT and SQLITE_CONSTRAINT_PRIMARYKEY, moved from guix-cuirass
>
> https://notabug.org/dannym/guile-sqlite3

Good you’re giving it some love.  :-)

I’ve already applied two commits in civodul/guile-sqlite3.  I think the
statement cache requires some more work though (see below).

> If that's OK I'll replace the reference in guix-master, or we could do a
> pull request to the civodul repository.

I think we should stick to a single repo for guile-sqlite3, though it
prolly shouldn’t be called “civodul/”.  Perhaps you can create
guile-sqlite3/guile-sqlite3 and add the two of us plus Andy there for a
start?

>> I’m not sure what ‘sqlsym’ is.  Apparently it’s a symbol derived from
>> the SQL statement, right?  I don’t think it’s necessary.
>
> Yes, the idea is to prevent it from having to do string comparison/hashing
> to find an existing stmt.  It's interned at macro-expansion time - so at
> runtime it shouldn't traverse the sql text after the stmt is created.
>
>> Instead you can simply make that hash table a regular (non-weak) hash
>> table that maps strings (SQL text) to prepared statements.  You’d also
>> need to use ‘hash-set!’ and ‘hash-ref’ instead of ‘hashq-set!’ and
>> ‘hash-ref’ since strings should be compared with ‘equal?’, not ‘eq?’.
>
> That's what I tried to avoid, having it compare 80 character strings
> every time I want to reuse a prepared statement. :-)
>
> What I wanted more is something like a new thread-local variable being
> declared every time a new SQL statement is used, otherwise the existing
> variable being used.

The problem is that interned symbols are potentially not GC’d (though I
think with Guile 2.2 and its weak sets they may be subject to GC.)

The other issue is that we’d still be caching potentially a lot more
than needed.  For instance, we don’t know whether a statement is a
one-off statement (used only once, for instance because it’s derived
from user parameters passed through the HTTP API or something), or if
it’s a statement that’s regularly used.  Excessive caching could be a
real issue for long-running processes like cuirass.

Thinking more about it, I’m inclined to not try to be smart and instead
let users explicitly ask for caching when they want to.

WDYT?

Thanks,
Ludo’.