From: ludo@gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym@scratchpost.org>
Cc: guix-devel <guix-devel@gnu.org>
Subject: Re: Cuirass news
Date: Sat, 27 Jan 2018 17:01:23 +0100 [thread overview]
Message-ID: <87zi4z1eb0.fsf@gnu.org> (raw)
In-Reply-To: <20180126153005.259a75e8@scratchpost.org> (Danny Milosavljevic's message of "Fri, 26 Jan 2018 15:30:05 +0100")
Hi Danny,
Danny Milosavljevic <dannym@scratchpost.org> skribis:
> I saw that (cuirass database) has some problems with sql injection.
> I defused it a little, see attached patch.
Yes, I was unhappy with that, glad you fixed it. :-)
> While we are at it, we can also reuse prepared statements (using the
> sqltext as key to find the right one).
Indeed, good idea!
> I also monitor sqlite accesses now - maybe that's overkill (see "with-mutex").
We don’t need mutexes: a given <db> is only ever used from one thread at
a time. Sometimes we have several threads accessing the database, but
they do so through different handlers, which SQLite handles correctly.
Some comments:
> From b8fdd9c4e3a11f11c8d948ee07b2003fa4981f81 Mon Sep 17 00:00:00 2001
> From: Danny Milosavljevic <dannym@scratchpost.org>
> Date: Fri, 26 Jan 2018 15:16:04 +0100
> Subject: [PATCH] database: Make 'sqlite-exec' reuse the prepared statement.
> Tags: patch
>
> * src/cuirass/database.scm (%sqlite-exec): Delete variable.
> (<db>): New variable.
> (%wrap-db): New variable.
> (%sqlite-prepare): New variable.
> (%sqlite-bind-args): New variable.
> (%sqlite-fetch-all): New variable.
> (sqlite-exec): Modify.
> (db-init): Use %wrap-db.
> (db-open): Use %wrap-db.
> (db-close): Modify.
> (db-add-specification): Adjust for prepared statement parameters.
> (db-get-specifications): Adjust for prepared statement parameters.
> (db-add-derivation): Adjust for prepared statement parameters.
> (db-get-derivation): Adjust for prepared statement parameters.
> (db-add-evaluation): Adjust for prepared statement parameters.
> (db-add-build): Adjust for prepared statement parameters.
> (db-update-build-status!): Adjust for prepared statement parameters.
> (db-get-build): Adjust for prepared statement parameters.
> (db-get-builds): Adjust for prepared statement parameters.
> (db-get-stamp): Adjust for prepared statement parameters.
> (db-add-stamp): Adjust for prepared statement parameters.
[...]
> +(define (%wrap-db native-db)
> + (db native-db (make-mutex) (make-weak-key-hash-table)))
> +
> +(define (%sqlite-prepare db sqlsym sqltext)
> + (with-mutex (db-lock db)
> + (let ((stmt (sqlite-prepare (db-native-db db) sqltext)))
> + (hashq-set! (db-stmts db) sqlsym stmt)
> + stmt)))
I’m not sure what ‘sqlsym’ is. Apparently it’s a symbol derived from
the SQL statement, right? I don’t think it’s necessary.
Instead you can simply make that hash table a regular (non-weak) hash
table that maps strings (SQL text) to prepared statements. You’d also
need to use ‘hash-set!’ and ‘hash-ref’ instead of ‘hashq-set!’ and
‘hash-ref’ since strings should be compared with ‘equal?’, not ‘eq?’.
However, could the hash table grow indefinitely if there are always new
statements prepared?
> +(define (%sqlite-bind-args stmt args)
> + (let ((argsi (zip (iota (length args)) args)))
> + (for-each (match-lambda ((i arg)
> + (sqlite-bind stmt (1+ i) arg)))
> + argsi)))
You can make it (note the indentation of ‘match-lambda’):
(for-each (match-lambda
((i arg)
(sqlite-bind stmt (1+ i) arg)))
(iota (length args))
args)
> (define-syntax sqlite-exec
> - ;; Note: Making it a macro so -Wformat can do its job.
> (lambda (s)
> - "Wrap 'sqlite-prepare', 'sqlite-step', and 'sqlite-finalize'. Send to given
> -SQL statement to DB. FMT and ARGS are passed to 'format'."
> (syntax-case s ()
> - ((_ db fmt args ...)
> - #'(%sqlite-exec db (format #f fmt args ...)))
> - (id
> - (identifier? #'id)
> - #'(lambda (db fmt . args)
> - (%sqlite-exec db (apply format #f fmt args)))))))
> + ((_ db sqltext arg ...) (string? (syntax->datum #'sqltext))
> + #`(let* ((sqlsym (quote #,(datum->syntax #'here (string->symbol (string-trim (syntax->datum #'sqltext))))))
> + (stmt (or (hashq-ref (db-stmts db) sqlsym)
> + (%sqlite-prepare db sqlsym sqltext))))
> + (with-mutex (db-lock db)
> + (%sqlite-bind-args stmt (list arg ...))
> + (%sqlite-fetch-all stmt))))
I think we can turn ‘sqlite-exec’ back into a procedure. The only
reason to make it a macro was to have -Wformat support, as noted in the
comment.
Otherwise LGTM.
Could you prepare an updated patch to address these and to remove the
mutex?
Thank you!
Ludo’.
next prev parent reply other threads:[~2018-01-27 16:01 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-24 22:12 Cuirass news Ludovic Courtès
2018-01-25 10:55 ` Mathieu Othacehe
2018-01-25 10:59 ` Mathieu Othacehe
2018-01-25 13:09 ` Ludovic Courtès
2018-01-26 14:30 ` Danny Milosavljevic
2018-01-27 16:01 ` Ludovic Courtès [this message]
2018-01-27 17:18 ` Danny Milosavljevic
2018-01-27 19:12 ` Danny Milosavljevic
2018-01-28 21:47 ` Ludovic Courtès
2018-01-28 22:23 ` Danny Milosavljevic
2018-01-29 9:57 ` Andy Wingo
2018-02-08 13:37 ` Ludovic Courtès
2018-02-08 16:29 ` Danny Milosavljevic
2018-02-08 22:21 ` Ludovic Courtès
2018-02-08 23:05 ` Danny Milosavljevic
2018-02-09 6:17 ` Gábor Boskovits
2018-02-09 9:41 ` Ludovic Courtès
2018-02-09 11:29 ` Danny Milosavljevic
2018-02-09 16:53 ` Ludovic Courtès
2018-02-09 17:06 ` Danny Milosavljevic
2018-02-10 11:18 ` Ludovic Courtès
2018-02-13 9:12 ` Danny Milosavljevic
2018-02-14 13:43 ` Ludovic Courtès
2018-02-14 23:17 ` Ludovic Courtès
2018-02-19 15:35 ` Danny Milosavljevic
2018-02-19 15:35 ` [PATCH] database: Simplify 'db-get-builds' Danny Milosavljevic
2018-02-19 17:52 ` [PATCH] database: db-get-builds: Inline output selection Danny Milosavljevic
2018-02-19 22:08 ` Cuirass news Danny Milosavljevic
2018-03-02 13:21 ` Ludovic Courtès
2018-03-02 22:06 ` Ludovic Courtès
2018-03-02 23:29 ` Danny Milosavljevic
2018-02-14 23:21 ` Ludovic Courtès
2018-01-25 21:06 ` Ricardo Wurmus
2018-01-26 11:12 ` Ludovic Courtès
2018-01-25 22:28 ` Danny Milosavljevic
2018-01-26 10:47 ` Ludovic Courtès
2018-01-28 12:33 ` Cuirass frontend Danny Milosavljevic
2018-01-29 17:42 ` Ludovic Courtès
2018-01-26 0:46 ` Cuirass news Danny Milosavljevic
2018-01-27 17:27 ` Danny Milosavljevic
2018-01-28 21:48 ` Ludovic Courtès
2018-01-26 17:55 ` Jan Nieuwenhuizen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87zi4z1eb0.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=dannym@scratchpost.org \
--cc=guix-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.