From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Cuirass news Date: Sun, 28 Jan 2018 22:47:34 +0100 Message-ID: <87fu6pzmdl.fsf@gnu.org> References: <877es6x5xj.fsf@gnu.org> <87lggmjjgo.fsf@gmail.com> <87k1w6jjak.fsf@gmail.com> <87h8raxeym.fsf@gnu.org> <20180126153005.259a75e8@scratchpost.org> <87zi4z1eb0.fsf@gnu.org> <20180127181852.42f0bcbf@scratchpost.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:39653) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1efunc-0005Yp-9q for guix-devel@gnu.org; Sun, 28 Jan 2018 16:47:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1efunb-0008Ah-2C for guix-devel@gnu.org; Sun, 28 Jan 2018 16:47:40 -0500 Received: from hera.aquilenet.fr ([2a0c:e300::1]:49658) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1efuna-0008AV-Qy for guix-devel@gnu.org; Sun, 28 Jan 2018 16:47:38 -0500 In-Reply-To: <20180127181852.42f0bcbf@scratchpost.org> (Danny Milosavljevic's message of "Sat, 27 Jan 2018 18:18:52 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Danny Milosavljevic Cc: guix-devel Hi Danny, Danny Milosavljevic skribis: > I've now forked guile-sqlite3 and put the stuff that makes sense for a > low-level binding there (not sqlite-exec - that would still be in guix-cu= irass). > > These are: > - stmts hash-table is now in guile-sqlite3's . > - sqlite-finalize adapted to update 's stmts list > - sqlite-finalize* variant which doesn't update 's stmts list > (because the updating by value is slow) > - sqlite-prepare* macro which stores and reuses existing s. > - sqlite-bind-args convenience function which just binds all the paramete= rs > in sequence. > - SQLITE_CONSTRAINT and SQLITE_CONSTRAINT_PRIMARYKEY, moved from guix-cui= rass > > https://notabug.org/dannym/guile-sqlite3 Good you=E2=80=99re giving it some love. :-) I=E2=80=99ve already applied two commits in civodul/guile-sqlite3. I think= the statement cache requires some more work though (see below). > If that's OK I'll replace the reference in guix-master, or we could do a > pull request to the civodul repository. I think we should stick to a single repo for guile-sqlite3, though it prolly shouldn=E2=80=99t be called =E2=80=9Ccivodul/=E2=80=9D. Perhaps you= can create guile-sqlite3/guile-sqlite3 and add the two of us plus Andy there for a start? >> I=E2=80=99m not sure what =E2=80=98sqlsym=E2=80=99 is. Apparently it=E2= =80=99s a symbol derived from >> the SQL statement, right? I don=E2=80=99t think it=E2=80=99s necessary. > > Yes, the idea is to prevent it from having to do string comparison/hashing > to find an existing stmt. It's interned at macro-expansion time - so at > runtime it shouldn't traverse the sql text after the stmt is created. > >> Instead you can simply make that hash table a regular (non-weak) hash >> table that maps strings (SQL text) to prepared statements. You=E2=80=99= d also >> need to use =E2=80=98hash-set!=E2=80=99 and =E2=80=98hash-ref=E2=80=99 i= nstead of =E2=80=98hashq-set!=E2=80=99 and >> =E2=80=98hash-ref=E2=80=99 since strings should be compared with =E2=80= =98equal?=E2=80=99, not =E2=80=98eq?=E2=80=99. > > That's what I tried to avoid, having it compare 80 character strings > every time I want to reuse a prepared statement. :-) > > What I wanted more is something like a new thread-local variable being > declared every time a new SQL statement is used, otherwise the existing > variable being used. The problem is that interned symbols are potentially not GC=E2=80=99d (thou= gh I think with Guile 2.2 and its weak sets they may be subject to GC.) The other issue is that we=E2=80=99d still be caching potentially a lot more than needed. For instance, we don=E2=80=99t know whether a statement is a one-off statement (used only once, for instance because it=E2=80=99s derived from user parameters passed through the HTTP API or something), or if it=E2=80=99s a statement that=E2=80=99s regularly used. Excessive caching = could be a real issue for long-running processes like cuirass. Thinking more about it, I=E2=80=99m inclined to not try to be smart and ins= tead let users explicitly ask for caching when they want to. WDYT? Thanks, Ludo=E2=80=99.