Problems with guile-sqlite3

Problems with guile-sqlite3

[Detlev Zundel]

Hi,

having a all new shiny, I wanted to do some random tests with it and
turned to Andys sqlite3 bindings[1].  Compiling and installing worked
like a breeze but I'm having some beginners trouble.

Looking into sqlite3.scm I gather that the following session should
open up a connection to my "mydb" database:

,----
| scheme@(guile-user)> ,use (sqlite3)
| scheme@(guile-user)> (sqlite-open "mydb" SQLITE_OPEN_READONLY)
| ;;; <stdin>:2:0: warning: possibly unbound variable `SQLITE_OPEN_READONLY'
| <unnamed port>:1:0: In procedure #<procedure 9335250 at <current input>:2:0 ()>:
| <unnamed port>:1:0: In procedure module-lookup: Unbound variable: SQLITE_OPEN_READONLY
`----

Hm ok, so the constants seem not to be exported, right?

,----
| scheme@(guile-user)> (define db (sqlite-open "mydb" 1))
| <unnamed port>:0:0: In procedure #<procedure 9bf9690 at <current input>:1:0 ()>:
| <unnamed port>:0:0: Throw to key `sqlite-error' with args `(sqlite-open 14 "Unable to open the database file")'.
`----

But that is strange, I'm sure I have this file.  So lets do an "strace
-e open guile" and see what guile accesses:

,----
| ....
| open("/usr/local/lib/guile/2.0/ccache/language/bytecode/spec.go", O_RDONLY) = 91
| open("/opt/src/git/guile-sqlite3/tests/mydb\315\201", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
| <unnamed port>:0:0: In procedure #<procedure 8ce37d0 at <current input>:1:0 ()>:
| <unnamed port>:0:0: Throw to key `sqlite-error' with args `(sqlite-open 14 "Unable to open the database file")'.
`----

Huh, what are those characters after "mydb"?  Funnily enough, if I use
filenames longer than 4 characters it works.  Can someone hit me with a
clue-stick please?

Thanks
  Detlev

[1] git://gitorious.org/guile-sqlite3/guile-sqlite3.git

-- 
I think that level of generalization is too abstract for useful thinking.
             -- Richard Stallman in <E19N344-0006Q9-Bt@fencepost.gnu.org>

Re: Problems with guile-sqlite3

[Andy Wingo]

On Thu 31 Mar 2011 00:52, Detlev Zundel <dzu@denx.de> writes:

> | scheme@(guile-user)> (sqlite-open "mydb" SQLITE_OPEN_READONLY)
> | ;;; <stdin>:2:0: warning: possibly unbound variable `SQLITE_OPEN_READONLY'
> | <unnamed port>:1:0: In procedure #<procedure 9335250 at <current input>:2:0 ()>:
> | <unnamed port>:1:0: In procedure module-lookup: Unbound variable: SQLITE_OPEN_READONLY
> `----
>
> Hm ok, so the constants seem not to be exported, right?

Hah, it appears not.  Want to fix it?  Fork the project on github and
send me a merge request :)

> ,----
> | scheme@(guile-user)> (define db (sqlite-open "mydb" 1))
> | <unnamed port>:0:0: In procedure #<procedure 9bf9690 at <current input>:1:0 ()>:
> | <unnamed port>:0:0: Throw to key `sqlite-error' with args `(sqlite-open 14 "Unable to open the database file")'.
> `----
>
> But that is strange, I'm sure I have this file.  So lets do an "strace
> -e open guile" and see what guile accesses:
>
> ,----
> | ....
> | open("/usr/local/lib/guile/2.0/ccache/language/bytecode/spec.go", O_RDONLY) = 91
> | open("/opt/src/git/guile-sqlite3/tests/mydb\315\201", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
> | <unnamed port>:0:0: In procedure #<procedure 8ce37d0 at <current input>:1:0 ()>:
> | <unnamed port>:0:0: Throw to key `sqlite-error' with args `(sqlite-open 14 "Unable to open the database file")'.
> `----
>
> Huh, what are those characters after "mydb"?  Funnily enough, if I use
> filenames longer than 4 characters it works.  Can someone hit me with a
> clue-stick please?

Interesting.  It seems that the string->pointer code is doing the wrong thing:

(define (string->utf8-pointer s)
  (bytevector->pointer (string->utf8 s)))

Indeed, there's no null-termination on this string.  I guess we need to
copy into a bytevector that is longer and provide a NUL byte.  Want to
patch that one too?

Cheers,

Andy
-- 
http://wingolog.org/

Re: Problems with guile-sqlite3

[Detlev Zundel]

Hi Andy,

> On Thu 31 Mar 2011 00:52, Detlev Zundel <dzu@denx.de> writes:
>
>> | scheme@(guile-user)> (sqlite-open "mydb" SQLITE_OPEN_READONLY)
>> | ;;; <stdin>:2:0: warning: possibly unbound variable `SQLITE_OPEN_READONLY'
>> | <unnamed port>:1:0: In procedure #<procedure 9335250 at <current input>:2:0 ()>:
>> | <unnamed port>:1:0: In procedure module-lookup: Unbound variable: SQLITE_OPEN_READONLY
>> `----
>>
>> Hm ok, so the constants seem not to be exported, right?
>
> Hah, it appears not.  Want to fix it?  Fork the project on github and
> send me a merge request :)

Ok, I'll see what I can do without asking for more help.

>> ,----
>> | scheme@(guile-user)> (define db (sqlite-open "mydb" 1))
>> | <unnamed port>:0:0: In procedure #<procedure 9bf9690 at <current input>:1:0 ()>:
>> | <unnamed port>:0:0: Throw to key `sqlite-error' with args
>> | (sqlite-open 14 "Unable to open the database file")'.
>> `----
>>
>> But that is strange, I'm sure I have this file.  So lets do an "strace
>> -e open guile" and see what guile accesses:
>>
>> ,----
>> | ....
>> | open("/usr/local/lib/guile/2.0/ccache/language/bytecode/spec.go", O_RDONLY) = 91
>> | open("/opt/src/git/guile-sqlite3/tests/mydb\315\201",
>> | O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
>> | <unnamed port>:0:0: In procedure #<procedure 8ce37d0 at <current input>:1:0 ()>:
>> | <unnamed port>:0:0: Throw to key `sqlite-error' with args
>> | (sqlite-open 14 "Unable to open the database file")'.
>> `----
>>
>> Huh, what are those characters after "mydb"?  Funnily enough, if I use
>> filenames longer than 4 characters it works.  Can someone hit me with a
>> clue-stick please?
>
> Interesting.  It seems that the string->pointer code is doing the wrong thing:
>
> (define (string->utf8-pointer s)
>   (bytevector->pointer (string->utf8 s)))
>
> Indeed, there's no null-termination on this string.  I guess we need to
> copy into a bytevector that is longer and provide a NUL byte.  Want to
> patch that one too?

Ok, I will at least try ;)

Thanks
  Detlev

-- 
Modern methods of production have given us the possibility of ease and
security for all;  we have chosen, instead,  to have overwork for some
and starvation for others.
                                    -- Bertrand Russell

Re: Problems with guile-sqlite3

[Detlev Zundel]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

Hi Andy,

>> Indeed, there's no null-termination on this string.  I guess we need to
>> copy into a bytevector that is longer and provide a NUL byte.  Want to
>> patch that one too?

The attached patches work for me.

As a followup I'd really like to put a few statments into tests below
test/.  Can anyone point me to what functions (i.e. assert,...) I should
use in such tests?

Thanks
  Detlev

-- 
The proprietary-Unix players proved so ponderous, so blind, and so inept at
marketing that Microsoft was able to grab away a large part of their market
with the shockingly inferior technology of its Windows operating system.
                   -- "A Brief History of Hackerdom" by Eric Steven Raymond

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Export-SQLITE_-constants.patch --]
[-- Type: text/x-diff, Size: 1078 bytes --]

From bf3993bbb3800cc6b0e353e1b59928d78ddab125 Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Thu, 31 Mar 2011 18:14:06 +0200
Subject: [PATCH 1/2] Export SQLITE_* constants.

Signed-off-by: Detlev Zundel <dzu@denx.de>
---
 sqlite3.scm |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/sqlite3.scm b/sqlite3.scm
index c326da4..711fd26 100644
--- a/sqlite3.scm
+++ b/sqlite3.scm
@@ -38,7 +38,24 @@
             sqlite-fold
             sqlite-map
             sqlite-reset
-            sqlite-finalize))
+	    sqlite-finalize
+
+	    SQLITE_OPEN_READONLY
+	    SQLITE_OPEN_READWRITE
+	    SQLITE_OPEN_CREATE
+	    SQLITE_OPEN_DELETEONCLOSE
+	    SQLITE_OPEN_EXCLUSIVE
+	    SQLITE_OPEN_MAIN_DB
+	    SQLITE_OPEN_TEMP_DB
+	    SQLITE_OPEN_TRANSIENT_DB
+	    SQLITE_OPEN_MAIN_JOURNAL
+	    SQLITE_OPEN_TEMP_JOURNAL
+	    SQLITE_OPEN_SUBJOURNAL
+	    SQLITE_OPEN_MASTER_JOURNAL
+	    SQLITE_OPEN_NOMUTEX
+	    SQLITE_OPEN_FULLMUTEX
+	    SQLITE_OPEN_SHAREDCACHE
+	    SQLITE_OPEN_PRIVATECACHE))
 
 ;;
 ;; Utils
-- 
1.7.4.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-Correctly-null-terminate-strings-that-we-pass-out.patch --]
[-- Type: text/x-diff, Size: 799 bytes --]

From 565070537b435f4cc597d304b02dfff0aa7daaa4 Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Thu, 31 Mar 2011 18:15:12 +0200
Subject: [PATCH 2/2] Correctly null terminate strings that we pass out.

Signed-off-by: Detlev Zundel <dzu@denx.de>
---
 sqlite3.scm |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/sqlite3.scm b/sqlite3.scm
index 711fd26..76ba4b5 100644
--- a/sqlite3.scm
+++ b/sqlite3.scm
@@ -61,7 +61,11 @@
 ;; Utils
 ;;
 (define (string->utf8-pointer s)
-  (bytevector->pointer (string->utf8 s)))
+  (let* ((len (string-length s))
+	 (bv (make-bytevector (+ len 1))))
+    (bytevector-copy! (string->utf8 s) 0 bv 0 len)
+    (array-set! bv 0 len)
+    (bytevector->pointer bv)))
 
 (define strlen
   (pointer->procedure size_t
-- 
1.7.4.1

Re: Problems with guile-sqlite3

[David Pirotte]

Hello,

Just tried guile-sqlite3  - after applying the 2 patches of Detlev - and succeeded with

	sqlite-open, sqlite-prepare, sqlite-column-names [and sqlite-close]

but

	scheme@(guile-user)> (sqlite-step stmt)
	ERROR: In procedure pointer->bytevector:
	ERROR: In procedure pointer->bytevector: null pointer dereference

	Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
	scheme@(guile-user) [1]> ,bt

	In sqlite/sqlite3.scm:
	   408:29  2 (sqlite-row #<<sqlite-stmt> pointer: #<pointer 0x89f224…>)
	   387:12  1 (#<procedure 90025c0 at sqlite/sqlite3.scm:377:4 (stmt…> …)
	In unknown file:
	           0 (pointer->bytevector #<pointer 0x0> 0 #<undefined> #<un…>)

Cheers,
David

;; --

Le Thu, 31 Mar 2011 18:18:55 +0200,
Detlev Zundel <dzu@denx.de> a écrit :

> Hi Andy,
> 
> >> Indeed, there's no null-termination on this string.  I guess we need to
> >> copy into a bytevector that is longer and provide a NUL byte.  Want to
> >> patch that one too?
> 
> The attached patches work for me.
> 
> As a followup I'd really like to put a few statments into tests below
> test/.  Can anyone point me to what functions (i.e. assert,...) I should
> use in such tests?
> 
> Thanks
>   Detlev

Re: Problems with guile-sqlite3

[Detlev Zundel]

Hi David,

> Just tried guile-sqlite3  - after applying the 2 patches of Detlev - and succeeded with
>
> 	sqlite-open, sqlite-prepare, sqlite-column-names [and sqlite-close]
>
> but
>
> 	scheme@(guile-user)> (sqlite-step stmt)
> 	ERROR: In procedure pointer->bytevector:
> 	ERROR: In procedure pointer->bytevector: null pointer dereference
>
> 	Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
> 	scheme@(guile-user) [1]> ,bt
>
> 	In sqlite/sqlite3.scm:
> 	   408:29  2 (sqlite-row #<<sqlite-stmt> pointer: #<pointer 0x89f224…>)
> 	   387:12  1 (#<procedure 90025c0 at sqlite/sqlite3.scm:377:4 (stmt…> …)
> 	In unknown file:
> 	           0 (pointer->bytevector #<pointer 0x0> 0 #<undefined> #<un…>)

Thanks for testing.  Unfortunately currently I'm not able to reproduce
your findings - in my little test I was successful in reading records
from a small database.

Can you show me the exact calls and the database you used?

Thanks
  Detlev

-- 
``The number of UNIX installations has grown to 10,
with more expected.''     Unix Programmers Manual -- 1972
The number of UNIX variants has grown to dozens,
with more expected.                               -- 2001

Re: Problems with guile-sqlite3

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 644 bytes --]

Le Fri, 01 Apr 2011 11:12:03 +0200,
Detlev Zundel <dzu@denx.de> a écrit :

> Hi David,
> 
> > Just tried guile-sqlite3  - after applying the 2 patches of Detlev - and
> > succeeded with
> > ...

> Thanks for testing.  Unfortunately currently I'm not able to reproduce
> your findings - in my little test I was successful in reading records
> from a small database.
> 
> Can you show me the exact calls and the database you used?

Hello Detlev,
Andy,

I spotted where the problem comes from: it is when a text value is empty. I have
produced a small but complete example [attached]: can you reproduce it ?

Thanks,
David

[-- Attachment #2: empty-string.scm --]
[-- Type: text/x-scheme, Size: 928 bytes --]


(define-module (empty-string)
  :use-module (ice-9 format)
  :use-module (sqlite3))

(export db
	create
	insert
	query-1
	query-2
	stmt)

(define stmt #f)


#!

April the 1st, 2011

  origin/HEAD -> origin/master
  origin/master

Retreiving an empty string triggers an error

!#


;;;
;;; Opening a test db
;;;

(define db (sqlite-open "/tmp/sqlite.db" 6))

(define create
  "create table test (
     reference integer primary key,
     name      text,
     firstname text,
     email     text
   );")

(define insert
  "insert into test 
        values (1,'BLAISE','Virginie','');")

(define query-1 "select reference, name, firstname from test;")
(define query-2 "select * from test;")

(for-each (lambda (sql-stmt)
	    (set! stmt (sqlite-prepare db sql-stmt))
	    (format #t "~S~%" (sqlite-step stmt))
	    (sqlite-finalize stmt))
    (list create
	  insert
	  query-1
	  query-2))


;;;
;;; Closing
;;;

(sqlite-close db)

Re: Problems with guile-sqlite3

[Detlev Zundel]

[-- Attachment #1: Type: text/plain, Size: 1355 bytes --]

Hi David,

> I spotted where the problem comes from: it is when a text value is empty. I have
> produced a small but complete example [attached]: can you reproduce it ?

Yes, thanks for the test case, I can now reproduce it and hopefully the
attached patches work for you also.

The first one fixes the ignorance of 'proc' of the 'sqlite-map'
procedure.  It took me a while to figure this one out ;)

The second patch extends 'tests/basic.test' with some statements that
make the problem discovered by you show up.

The third patch fixes the problem for me.  Andy, is this the 'right way'
to test for null pointers?  I could not find a cleaner solution in my
limited search...

The fourth patch fixes a problem I discovered while playing with
sqlite-bind.  Really this should generate more testcases.

Andy, I have now cloned your repo at gitorious[1] for you to pull if
that's easier.  I am perfectly happy to rework patches if anything needs
to be done for that.

Thanks
  Detlev

[1] https://gitorious.org/~dzu/guile-sqlite3/guile-sqlite3-dzu

-- 
Bacchus, n. A convenient deity invented by the ancients as an excuse for
getting drunk.
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu@denx.de

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Make-sqlite-map-really-apply-the-procedure-argument.patch --]
[-- Type: text/x-diff, Size: 592 bytes --]

From 1018e0617b08c0f79556ea1e6188963ce01bb952 Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Mon, 4 Apr 2011 18:11:58 +0200
Subject: [PATCH 1/4] Make sqlite-map really apply the procedure argument

---
 sqlite3.scm |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/sqlite3.scm b/sqlite3.scm
index 3369e02..2492d15 100644
--- a/sqlite3.scm
+++ b/sqlite3.scm
@@ -425,4 +425,5 @@
           seed))))
 
 (define (sqlite-map proc stmt)
-  (reverse! (sqlite-fold cons '() stmt)))
+  (map proc
+       (reverse! (sqlite-fold cons '() stmt))))
-- 
1.7.4.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-Add-some-basic-functionality-testing.patch --]
[-- Type: text/x-diff, Size: 2072 bytes --]

From f45fc5c709d241a147e061e92062993cce1b4841 Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Mon, 4 Apr 2011 18:12:39 +0200
Subject: [PATCH 2/4] Add some basic functionality testing

---
 tests/basic.test |   43 ++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 42 insertions(+), 1 deletions(-)

diff --git a/tests/basic.test b/tests/basic.test
index 1e5a3e0..46b395c 100644
--- a/tests/basic.test
+++ b/tests/basic.test
@@ -1,6 +1,6 @@
 ;;;; basic.test ---      -*- mode: scheme; coding: utf-8; -*-
 ;;;;
-;;;; 	Copyright (C) 2010 Andy Wingo  <wingo@pobox.com>
+;;;;   Copyright (C) 2011 Detlev Zundel <dzu@denx.de>
 ;;;;
 ;;;; This library is free software; you can redistribute it and/or
 ;;;; modify it under the terms of the GNU Lesser General Public
@@ -17,6 +17,47 @@
 ;;;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
 
 (define-module (tests basic-test)
+  #:use-module (ice-9 format)
   #:use-module (sqlite3))
 
+(define (sqlite-exec db sql)
+  (let ((stmt (sqlite-prepare db sql)))
+    (sqlite-map display stmt)))
+
+;; Cleanup database so we can check creation
+(define db-name "tests/simple.db")
+(if (file-exists? db-name)
+    (begin
+      (format #t "Removing leftover database ~a~%" db-name)
+      (delete-file db-name)))
+
+(format #t "Creating test database ~a:" db-name)
+(define db (sqlite-open db-name (logior SQLITE_OPEN_CREATE
+                                        SQLITE_OPEN_READWRITE)))
+(format #t "~40tOk~%")
+
+(format #t "Creating table 'project':")
+(sqlite-exec
+ db
+ "create table project (
+      reference integer primary key,
+      name   text,
+      website   text
+  )")
+(format #t "~40tOk~%")
+
+(format #t "Inserting a dataset:")
+(sqlite-exec db "insert into project values (1, 'Guile', '')")
+(format #t "~40tOk~%")
+
+(format #t "Reading dataset: ")
+(sqlite-exec db "select * from project")
+(format #t "~40tOk~%")
+
+(sqlite-close db)
+
+(format #t "Removing database ~a" db-name)
+(delete-file db-name)
+(format #t "~40tOk~%")
+
 (exit 0)
-- 
1.7.4.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-Fix-interpretation-of-null-pointers-from-the-library.patch --]
[-- Type: text/x-diff, Size: 1205 bytes --]

From 97aae36202ca7ba0abec5cf0da95a56ac706805f Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Mon, 4 Apr 2011 18:13:18 +0200
Subject: [PATCH 3/4] Fix interpretation of null pointers from the library

---
 sqlite3.scm |   10 +++++++---
 1 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/sqlite3.scm b/sqlite3.scm
index 2492d15..9e48ff5 100644
--- a/sqlite3.scm
+++ b/sqlite3.scm
@@ -373,11 +373,15 @@
          (value-double (stmt-pointer stmt) i))
         ((3) ; SQLITE3_TEXT
          (let ((p (value-blob (stmt-pointer stmt) i)))
-           (utf8->string
-            (pointer->bytevector p (value-bytes (stmt-pointer stmt) i)))))
+           (if (eq? p %null-pointer)
+               ""
+               (utf8->string
+                (pointer->bytevector p (value-bytes (stmt-pointer stmt) i))))))
         ((4) ; SQLITE_BLOB
          (let ((p (value-blob (stmt-pointer stmt) i)))
-           (pointer->bytevector p (value-bytes (stmt-pointer stmt) i))))
+           (if (eq? p %null-pointer)
+               (make-bytevector 0)
+               (pointer->bytevector p (value-bytes (stmt-pointer stmt) i)))))
         ((5) ; SQLITE_NULL
          #f)))))
 
-- 
1.7.4.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #5: 0004-Fix-sqlite-bind.patch --]
[-- Type: text/x-diff, Size: 1293 bytes --]

From c37f372990d9edddd859e9b999eb38151ce33cf8 Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Mon, 4 Apr 2011 18:09:06 +0200
Subject: [PATCH 4/4] Fix sqlite-bind.

- sqlite-transient needs to be a pointer
- fix typo for string case
---
 sqlite3.scm |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/sqlite3.scm b/sqlite3.scm
index 9e48ff5..97550b4 100644
--- a/sqlite3.scm
+++ b/sqlite3.scm
@@ -290,7 +290,8 @@
                     int
                     (dynamic-func "sqlite3_bind_null" libsqlite3)
                     (list '* int)))
-        (sqlite-transient (make-bytevector (sizeof '*) #xff)))
+        (sqlite-transient (bytevector->pointer
+                           (make-bytevector (sizeof '*) #xff))))
     (lambda (stmt key val)
       (assert-live-stmt! stmt)
       (let ((idx (key->index stmt key))
@@ -300,7 +301,7 @@
           (bind-blob p idx (bytevector->pointer val) (bytevector-length val)
                      sqlite-transient))
          ((string? val)
-          (let ((bv ((string->utf8 val))))
+          (let ((bv (string->utf8 val)))
             (bind-text p idx (bytevector->pointer bv) (bytevector-length bv)
                        sqlite-transient)))
          ((and (integer? val) (exact? val))
-- 
1.7.4.1

Re: Problems with guile-sqlite3

[Andy Wingo]

On Mon 04 Apr 2011 18:22, Detlev Zundel <dzu@denx.de> writes:

>> I spotted where the problem comes from: it is when a text value is empty. I have
>> produced a small but complete example [attached]: can you reproduce it ?
>
> Yes, thanks for the test case, I can now reproduce it and hopefully the
> attached patches work for you also.

I have added you to the gitorious repo.  Feel free to push your patches
there :-)

Cheers,

Andy
-- 
http://wingolog.org/

Re: Problems with guile-sqlite3

[Detlev Zundel]

Hi Andy,

> On Mon 04 Apr 2011 18:22, Detlev Zundel <dzu@denx.de> writes:
>
>>> I spotted where the problem comes from: it is when a text value is empty. I have
>>> produced a small but complete example [attached]: can you reproduce it ?
>>
>> Yes, thanks for the test case, I can now reproduce it and hopefully the
>> attached patches work for you also.
>
> I have added you to the gitorious repo.  Feel free to push your patches
> there :-)

Hey thanks!  Still I'd welcome if you at least look over my changes and
give me some hints if you see room for improvements - most of what I did
in scheme in the past was in scsh ;)

As I said in my previous mail, I'd like to know if there is a better way
to handle the null pinter problem reported by Dave, e.g. does guile have
a null-pointer? or something comparable more gentle on the eyes than (eq
%null-pointer ..)?  If not, would it make sens to add such a predicate?

Thanks
  Detlev

-- 
Once the  implementation is  up and running,  it is still a trick to keep it
running.  In a normal,  closed implementation,  this is not a problem;  in a
system with metaobject protocols [..] there is the potential for spectacular
failure modes if certain situations are not properly anticipated.
                                       -- The Art of the Metaobject Protocol
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu@denx.de

Re: Problems with guile-sqlite3

[Detlev Zundel]

Hi,

>> I have added you to the gitorious repo.  Feel free to push your patches
>> there :-)
>
> Hey thanks!  Still I'd welcome if you at least look over my changes and
> give me some hints if you see room for improvements - most of what I did
> in scheme in the past was in scsh ;)
>
> As I said in my previous mail, I'd like to know if there is a better way
> to handle the null pinter problem reported by Dave, e.g. does guile have
> a null-pointer? or something comparable more gentle on the eyes than (eq
> %null-pointer ..)?  If not, would it make sens to add such a predicate?

Ok, I found 'null-pointer?' and pushed all my changes to the repo[1].

Thanks
  Detlev

[1] https://gitorious.org/guile-sqlite3/guile-sqlite3

-- 
error compiling committee.c: too many arguments to function

Re: Problems with guile-sqlite3

[David Pirotte]

Hi Detlev,

I am sorry for the very loooooong delay in responding, but yes it did work for me as
well, many thanks for your patches.

I'd love to have a './configure --prefix=... && make install' [even though there is
only only file for the time being], could you help me doing the setup or point me
a tutorial a could read with regards these matters?

Thanks,
David

;; --

Le Mon, 04 Apr 2011 18:22:18 +0200,
Detlev Zundel <dzu@denx.de> a écrit :

> Hi David,
> 
> > I spotted where the problem comes from: it is when a text value is empty. I have
> > produced a small but complete example [attached]: can you reproduce it ?
> 
> Yes, thanks for the test case, I can now reproduce it and hopefully the
> attached patches work for you also.
> 
> The first one fixes the ignorance of 'proc' of the 'sqlite-map'
> procedure.  It took me a while to figure this one out ;)
> 
> The second patch extends 'tests/basic.test' with some statements that
> make the problem discovered by you show up.
> 
> The third patch fixes the problem for me.  Andy, is this the 'right way'
> to test for null pointers?  I could not find a cleaner solution in my
> limited search...
> 
> The fourth patch fixes a problem I discovered while playing with
> sqlite-bind.  Really this should generate more testcases.
> 
> Andy, I have now cloned your repo at gitorious[1] for you to pull if
> that's easier.  I am perfectly happy to rework patches if anything needs
> to be done for that.
> 
> Thanks
>   Detlev
> 
> [1] https://gitorious.org/~dzu/guile-sqlite3/guile-sqlite3-dzu
> 

Re: Problems with guile-sqlite3

[Detlev Zundel]

Hi David,

> I am sorry for the very loooooong delay in responding, but yes it did work for me as
> well, many thanks for your patches.

Ok, thanks for the confirmation.

> I'd love to have a './configure --prefix=... && make install' [even though there is
> only only file for the time being], could you help me doing the setup or point me
> a tutorial a could read with regards these matters?

Duew to using autoconf, the project already takes care that it hooks
into the guile which is accessible at "./configure" times.  "make
install" works for me out of the box:

  dzu@ohwell:guile-sqlite3[2]$ sudo make install
  [sudo] password for dzu: 
  make[1]: Entering directory `/opt/src/git/guile-sqlite3'
  make[1]: Nothing to be done for `install-exec-am'.
  test -z "/usr/local/share/guile/site/2.0" || /bin/mkdir -p "/usr/local/share/guile/site/2.0"
   /usr/bin/install -c -m 644  sqlite3.scm '/usr/local/share/guile/site/2.0/.'
  test -z "/usr/local/lib/guile/2.0/ccache" || /bin/mkdir -p "/usr/local/lib/guile/2.0/ccache"
   /usr/bin/install -c -m 644  sqlite3.go '/usr/local/lib/guile/2.0/ccache/.'
  make[1]: Leaving directory `/opt/src/git/guile-sqlite3'
  dzu@ohwell:guile-sqlite3[2]$ 

What else do you want? ;)

Cheers
  Detlev

-- 
Soon in a source code repository near you. 
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu@denx.de

Re: Problems with guile-sqlite3

[Andy Wingo]

On Wed 20 Apr 2011 17:51, Detlev Zundel <dzu@denx.de> writes:

>> I'd love to have a './configure --prefix=... && make install' [even though there is
>> only only file for the time being], could you help me doing the setup or point me
>> a tutorial a could read with regards these matters?
>
> Duew to using autoconf, the project already takes care that it hooks
> into the guile which is accessible at "./configure" times.  "make
> install" works for me out of the box:

Perhaps David was missing the fact that you need to run `autoreconf
-vif' to generate ./configure.

Cheers,

Andy
-- 
http://wingolog.org/

Re: Problems with guile-sqlite3

[David Pirotte]

Hello,

Le Thu, 21 Apr 2011 14:17:15 +0200,
Andy Wingo <wingo@pobox.com> a écrit :

> On Wed 20 Apr 2011 17:51, Detlev Zundel <dzu@denx.de> writes:

> ...
> Perhaps David was missing the fact that you need to run `autoreconf
> -vif' to generate ./configure.

Yes I did miss that, thank you! I am thinking, would it not be interesting to create
a directory sqlite under site/2.0 so that other files may go there in the future
too ? Anyway, just a thought.

> Cheers,
> Andy

dito,
David

Re: Problems with guile-sqlite3

[Andy Wingo]

On Thu 31 Mar 2011 18:18, Detlev Zundel <dzu@denx.de> writes:

> Hi Andy,
>
>>> Indeed, there's no null-termination on this string.  I guess we need to
>>> copy into a bytevector that is longer and provide a NUL byte.  Want to
>>> patch that one too?
>
> The attached patches work for me.

Thanks.  Please detabify the first one.  Can you resubmit the second one
also using the string->pointer encoding argument that I just pushed to
Guile?  Thanks :-)

> As a followup I'd really like to put a few statments into tests below
> test/.  Can anyone point me to what functions (i.e. assert,...) I should
> use in such tests?

Sigh, it's a good question.  Guile has "(test-suite lib)" internally,
which is quite nice, but not public.  Externally there are many things
but none as nice.

I guess I would say to just add some test scripts using the normal
automake mechanism, and make them (exit 1) if there is an error.  You
won't get a nice summary but it's better than nothing.

Cheers,

Andy
-- 
http://wingolog.org/

Re: Problems with guile-sqlite3

[Detlev Zundel]

[-- Attachment #1: Type: text/plain, Size: 1392 bytes --]

Hi Andy,

> On Thu 31 Mar 2011 18:18, Detlev Zundel <dzu@denx.de> writes:
>
>> Hi Andy,
>>
>>>> Indeed, there's no null-termination on this string.  I guess we need to
>>>> copy into a bytevector that is longer and provide a NUL byte.  Want to
>>>> patch that one too?
>>
>> The attached patches work for me.
>
> Thanks.  Please detabify the first one.  

Sure, no problem.

> Can you resubmit the second one also using the string->pointer
> encoding argument that I just pushed to Guile?  Thanks :-)

Ah, that makes the code a lot easier ;)

>> As a followup I'd really like to put a few statments into tests below
>> test/.  Can anyone point me to what functions (i.e. assert,...) I should
>> use in such tests?
>
> Sigh, it's a good question.  Guile has "(test-suite lib)" internally,
> which is quite nice, but not public.  Externally there are many things
> but none as nice.
>
> I guess I would say to just add some test scripts using the normal
> automake mechanism, and make them (exit 1) if there is an error.  You
> won't get a nice summary but it's better than nothing.

Ok, I needed to read up on the "normal automake mechanism", but I think
I understand.  The third patch prepares for real test cases ;)

Thanks
  Detlev

-- 
1. What is the best thing about Unix?
A: The community.
2. What is the worst thing about Unix?
A: That there are so many communities.         (Rob Pike)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Export-SQLITE_-constants.patch --]
[-- Type: text/x-diff, Size: 1197 bytes --]

From 5eef5b889a1e00b05a424fa27618b76a66a576fe Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Thu, 31 Mar 2011 18:14:06 +0200
Subject: [PATCH 1/3] Export SQLITE_* constants.

Signed-off-by: Detlev Zundel <dzu@denx.de>
---
 sqlite3.scm |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/sqlite3.scm b/sqlite3.scm
index c326da4..8a1e6f1 100644
--- a/sqlite3.scm
+++ b/sqlite3.scm
@@ -38,7 +38,24 @@
             sqlite-fold
             sqlite-map
             sqlite-reset
-            sqlite-finalize))
+            sqlite-finalize
+
+            SQLITE_OPEN_READONLY
+            SQLITE_OPEN_READWRITE
+            SQLITE_OPEN_CREATE
+            SQLITE_OPEN_DELETEONCLOSE
+            SQLITE_OPEN_EXCLUSIVE
+            SQLITE_OPEN_MAIN_DB
+            SQLITE_OPEN_TEMP_DB
+            SQLITE_OPEN_TRANSIENT_DB
+            SQLITE_OPEN_MAIN_JOURNAL
+            SQLITE_OPEN_TEMP_JOURNAL
+            SQLITE_OPEN_SUBJOURNAL
+            SQLITE_OPEN_MASTER_JOURNAL
+            SQLITE_OPEN_NOMUTEX
+            SQLITE_OPEN_FULLMUTEX
+            SQLITE_OPEN_SHAREDCACHE
+            SQLITE_OPEN_PRIVATECACHE))
 
 ;;
 ;; Utils
-- 
1.7.4.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-Fix-and-simplify-foreign-string-conversions.patch --]
[-- Type: text/x-diff, Size: 1055 bytes --]

From 2be1beddc9d52664482d9d3e7f57e0a976baf5de Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Thu, 31 Mar 2011 18:15:12 +0200
Subject: [PATCH 2/3] Fix and simplify foreign string conversions

Use the versions of string->pointer and pointer->string that accept an
encoding parameter.

The previous version of string->utf8-pointer missed the null termination
of the string.

Signed-off-by: Detlev Zundel <dzu@denx.de>
---
 sqlite3.scm |    9 ++-------
 1 files changed, 2 insertions(+), 7 deletions(-)

diff --git a/sqlite3.scm b/sqlite3.scm
index 8a1e6f1..3369e02 100644
--- a/sqlite3.scm
+++ b/sqlite3.scm
@@ -61,15 +61,10 @@
 ;; Utils
 ;;
 (define (string->utf8-pointer s)
-  (bytevector->pointer (string->utf8 s)))
-
-(define strlen
-  (pointer->procedure size_t
-                      (dynamic-pointer "strlen" (dynamic-link))
-                      '(*)))
+  (string->pointer s "utf-8"))
 
 (define (utf8-pointer->string p)
-  (utf8->string (pointer->bytevector p (strlen p))))
+  (pointer->string p -1 "utf-8"))
 
 
 ;;
-- 
1.7.4.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-Fix-command-line-parameter-for-test-execution.patch --]
[-- Type: text/x-diff, Size: 721 bytes --]

From 00c4543c9844ac584929a16e4d2d44b577b6dea1 Mon Sep 17 00:00:00 2001
From: Detlev Zundel <dzu@denx.de>
Date: Fri, 1 Apr 2011 16:30:44 +0200
Subject: [PATCH 3/3] Fix command line parameter for test execution

Signed-off-by: Detlev Zundel <dzu@denx.de>
---
 Makefile.am |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index 509d1be..ae9a6a6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -24,6 +24,6 @@ SUFFIXES = .scm .go
 
 TESTS = 					\
 	tests/basic.test
-TESTS_ENVIRONMENT = $(abs_top_builddir)/env $(GUILE) --no-autocompile
+TESTS_ENVIRONMENT = $(abs_top_builddir)/env $(GUILE) --no-auto-compile
 
 EXTRA_DIST = $(SOURCES) $(NOCOMP_SOURCES) $(TESTS)
-- 
1.7.4.1

Re: Problems with guile-sqlite3

[Andy Wingo]

Applied the patches.  Thanks!

Andy
-- 
http://wingolog.org/

Re: Problems with guile-sqlite3

[Ludovic Courtès]

Hello!

Andy Wingo <wingo@pobox.com> writes:

> Interesting.  It seems that the string->pointer code is doing the wrong thing:

It works as advertised.  :-)

 -- Scheme Procedure: string->pointer string
     Return a foreign pointer to a nul-terminated copy of STRING in the
     current locale encoding.  The C string is freed when the returned
     foreign pointer becomes unreachable.

     This is the Scheme equivalent of `scm_to_locale_string'.

We could add ‘string->utf8-pointer’ to (system foreign), I guess.

Thanks,
Ludo’.

Re: Problems with guile-sqlite3

[Andy Wingo]

Hi :)

On Thu 31 Mar 2011 16:28, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>> It seems that the string->pointer code is doing the wrong thing:
>
> It works as advertised.  :-)

Sorry, I didn't mean to accuse that function, as I didn't even use it!

  (define (string->utf8-pointer s)
    (bytevector->pointer (string->utf8 s)))

That said, adding an #:encoding argument to string->pointer and
pointer->string sounds like a good idea.

Andy
-- 
http://wingolog.org/