unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
* New library: guile-wikidata
       [not found] <5ffe968620f1d2d940b7db2b1900dc43@riseup.net>
@ 2018-12-09  9:11 ` swedebugia
  2018-12-09 11:08   ` tomas
  0 siblings, 1 reply; 12+ messages in thread
From: swedebugia @ 2018-12-09  9:11 UTC (permalink / raw)
  To: Guile User

Hi

I worked hard for a few days playing with guile. 

Pre-release now at https://gitlab.com/swedebugia/guile-wikidata

Next step is to implement looking up the entities for the result and
filter on instance of (the properties). E.g. software. Or query wikidata
with SPARQL and guile-sparql.

:D

What do you think?
I tried reuse code and abstract away a la Guile. This is my first
program so feel free to give it a review and ideas for improvement.

It depends on (guix import json) right now to be able to fetch and
return an alist and I'm keen on removing this dependency before the
first release.

Happy hacking with wikidata!

Cheers
swedebugia

-------- Original Message --------
Subject: New library: guile-wikidata
Date: 2018-12-04 23:03
From: swedebugia@riseup.net
To: guix-devel <guix-devel@gnu.org>

Hi

I worked hard for a few hours playing with guile and came up with this.

Next step is to implement looking up the entities for the result and
filter on instance of (the properties). E.g. software.

:D

What do you think?
I tried reuse code and abstract away in a a la Guile. This is my first
program so feel free to give it a review and ideas for improvement.

It depends on (guix import json) right now and I think we should
upstream those two json-to-alist functions to guile-json.

Example output:

scheme@(guile-user) [25]> (show-first-result "gcc")
1.: Q178940: GNU Compiler Collection: compiler system with support for
various programming languages

scheme@(guile-user) [25]> (show-first-result "emacs")
1.: Q189722: Emacs: family of text editors

scheme@(guile-user) [25]> (show-first-result "guile")
1.: Q601244: Guile: character from the Street Fighter fighting game
series

scheme@(guile-user) [25]> (show-first-result "openssh")
1.: Q847062: OpenSSH: set of computer programs providing encrypted
communication sessions

scheme@(guile-user) [25]> (show-first-result "automake")
1.: Q1324275: Automake: tool for generating GNU Standards-compliant
Makefiles


Not so reliable output for bioinformatics packages:

scheme@(guile-user) [25]> (show-first-result "aragorn")
1.: Q180322: Aragorn: character from the Lord of the Rings
$19 = #t
scheme@(guile-user) [25]> (show-first-result "bamm")
1.: Q12819323: Bamm: Uzbek name of deep and dark sounds of stringed
instruments
$20 = #t
scheme@(guile-user) [25]> (show-first-result "bamtools")
1.: Q30426432: BamTools: a C++ API and toolkit for analyzing and
managing BAM files.: scientific article
$21 = #t
scheme@(guile-user) [25]> (show-first-result "bcftools")
1.: Q31041251: BCFtools/RoH: a hidden Markov model approach for
detecting autozygosity from next-generation sequencing data.:\
 scientific article
$22 = #t
scheme@(guile-user) [25]> (show-first-result "bedops")
1.: Q36076319: BEDOPS: high-performance genomic feature operations.:
scientific article
$23 = #t
scheme@(guile-user) [25]> (show-first-result "blast+")
1.: Q179057: explosion: sudden release of energy through high
temperatures and gas expansion
$24 = #t

-- 
Cheers 
Swedebugia



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: New library: guile-wikidata
  2018-12-09  9:11 ` New library: guile-wikidata swedebugia
@ 2018-12-09 11:08   ` tomas
  2018-12-09 21:26     ` Arne Babenhauserheide
  0 siblings, 1 reply; 12+ messages in thread
From: tomas @ 2018-12-09 11:08 UTC (permalink / raw)
  To: guile-user

[-- Attachment #1: Type: text/plain, Size: 322 bytes --]

On Sun, Dec 09, 2018 at 01:11:05AM -0800, swedebugia@riseup.net wrote:
> Hi
> 
> I worked hard for a few days playing with guile. 
> 
> Pre-release now at https://gitlab.com/swedebugia/guile-wikidata

This is pretty exciting. Hoping to find a relief from my stern project
manager...

Thanks & cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: New library: guile-wikidata
  2018-12-09 11:08   ` tomas
@ 2018-12-09 21:26     ` Arne Babenhauserheide
  2018-12-11  0:32       ` swedebugia
  0 siblings, 1 reply; 12+ messages in thread
From: Arne Babenhauserheide @ 2018-12-09 21:26 UTC (permalink / raw)
  To: tomas; +Cc: guile-user

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]


tomas@tuxteam.de writes:

> On Sun, Dec 09, 2018 at 01:11:05AM -0800, swedebugia@riseup.net wrote:
>> Hi
>> 
>> I worked hard for a few days playing with guile. 
>> 
>> Pre-release now at https://gitlab.com/swedebugia/guile-wikidata
>
> This is pretty exciting. Hoping to find a relief from my stern project
> manager...

That looks pretty cool — I didn’t know wikidata.

The search procedure looks like it wants to be in the readme as an
example :-)

You could add a header and exported main function to also use this module as script file:

Header:

#!/usr/bin/env bash
# -*- scheme -*
exec -a "$0" guile -L "$(dirname "$0")" -e '(wikidata)' -c '' "$@"
;; !# ;; this ends the inline comment started by the hashbang


module:

...
  #:export (show main))


main:

(define (main args)
  (if (null? (cdr args)) (format #t "usage: ... ~s" (first args))
    (let ((query (first args))
          (count (if (> (len args) 1) (second args) 10)))
         (search query count))))


Add `chmod +x wikidata.scm` and you can run it as

./wikidata.scm <query> [<count>]


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1076 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: New library: guile-wikidata
  2018-12-09 21:26     ` Arne Babenhauserheide
@ 2018-12-11  0:32       ` swedebugia
  2018-12-11 10:29         ` Roel Janssen
  0 siblings, 1 reply; 12+ messages in thread
From: swedebugia @ 2018-12-11  0:32 UTC (permalink / raw)
  To: Arne Babenhauserheide; +Cc: guile-user, guile-user

On 2018-12-09 22:26, Arne Babenhauserheide wrote:
> tomas@tuxteam.de writes:
> 
>> On Sun, Dec 09, 2018 at 01:11:05AM -0800, swedebugia@riseup.net wrote:
>>> Hi
>>>
>>> I worked hard for a few days playing with guile.
>>>
>>> Pre-release now at https://gitlab.com/swedebugia/guile-wikidata
>>
>> This is pretty exciting. Hoping to find a relief from my stern project
>> manager...
> 
> That looks pretty cool — I didn’t know wikidata.
> 
> The search procedure looks like it wants to be in the readme as an
> example :-)
> 
> You could add a header and exported main function to also use this
> module as script file:
> 
> Header:
> 
> #!/usr/bin/env bash
> # -*- scheme -*
> exec -a "$0" guile -L "$(dirname "$0")" -e '(wikidata)' -c '' "$@"
> ;; !# ;; this ends the inline comment started by the hashbang
> 
> 
> module:
> 
> ...
>   #:export (show main))
> 
> 
> main:
> 
> (define (main args)
>   (if (null? (cdr args)) (format #t "usage: ... ~s" (first args))
>     (let ((query (first args))
>           (count (if (> (len args) 1) (second args) 10)))
>          (search query count))))
> 
> 
> Add `chmod +x wikidata.scm` and you can run it as
> 
> ./wikidata.scm <query> [<count>]
> 
> 
> Best wishes,
> Arne

Thanks for the tips. 

I now implemented sparql queries as well. See
https://gitlab.com/swedebugia/guile-wikidata

-- 
Cheers 
Swedebugia



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: New library: guile-wikidata
  2018-12-11  0:32       ` swedebugia
@ 2018-12-11 10:29         ` Roel Janssen
  2018-12-13 15:01           ` swedebugia
  0 siblings, 1 reply; 12+ messages in thread
From: Roel Janssen @ 2018-12-11 10:29 UTC (permalink / raw)
  To: swedebugia; +Cc: guile-user, guile-user

On 11-12-18 01:32, swedebugia@riseup.net wrote:
> On 2018-12-09 22:26, Arne Babenhauserheide wrote:
>> tomas@tuxteam.de writes:
>>
>>> On Sun, Dec 09, 2018 at 01:11:05AM -0800, swedebugia@riseup.net wrote:
>>>> Hi
>>>>
>>>> I worked hard for a few days playing with guile.
>>>>
>>>> Pre-release now at https://gitlab.com/swedebugia/guile-wikidata
>>>
>>> This is pretty exciting. Hoping to find a relief from my stern project
>>> manager...
>>
>> That looks pretty cool — I didn’t know wikidata.
>>
>> The search procedure looks like it wants to be in the readme as an
>> example :-)
>>
>> You could add a header and exported main function to also use this
>> module as script file:
>>
>> Header:
>>
>> #!/usr/bin/env bash
>> # -*- scheme -*
>> exec -a "$0" guile -L "$(dirname "$0")" -e '(wikidata)' -c '' "$@"
>> ;; !# ;; this ends the inline comment started by the hashbang
>>
>>
>> module:
>>
>> ...
>>    #:export (show main))
>>
>>
>> main:
>>
>> (define (main args)
>>    (if (null? (cdr args)) (format #t "usage: ... ~s" (first args))
>>      (let ((query (first args))
>>            (count (if (> (len args) 1) (second args) 10)))
>>           (search query count))))
>>
>>
>> Add `chmod +x wikidata.scm` and you can run it as
>>
>> ./wikidata.scm <query> [<count>]
>>
>>
>> Best wishes,
>> Arne
> 
> Thanks for the tips.
> 
> I now implemented sparql queries as well. See
> https://gitlab.com/swedebugia/guile-wikidata
> 

"guile-wikidata" looks cool!  And I'm glad to see you've added SPARQL 
support as well.  It would be great to integrate it with "guile-sparql" 
if you find the time for it.  The "guile-sparql" code is quite short, 
and hopefully readable enough.. :)

I've been working on a web interface to work with RDF/SPARQL (written 
mostly in Guile Scheme, which uses "guile-sparql" as well).  You might 
find it interesting:
https://github.com/UMCUGenetics/sparqling-genomics

The Guile Scheme parts are in the "web" subfolder.

Kind regards,
Roel Janssen




^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: New library: guile-wikidata
  2018-12-11 10:29         ` Roel Janssen
@ 2018-12-13 15:01           ` swedebugia
  2018-12-13 16:06             ` Trouble parsing a response (Was: Re: New library: guile-wikidata) swedebugia
  0 siblings, 1 reply; 12+ messages in thread
From: swedebugia @ 2018-12-13 15:01 UTC (permalink / raw)
  To: Roel Janssen; +Cc: guile-user, guile-user

[-- Attachment #1: Type: text/plain, Size: 1906 bytes --]

On 2018-12-11 11:29, Roel Janssen wrote:
> On 11-12-18 01:32, swedebugia@riseup.net wrote:
snip

>> I now implemented sparql queries as well. See
>> https://gitlab.com/swedebugia/guile-wikidata
>>
> 
> "guile-wikidata" looks cool!  And I'm glad to see you've added SPARQL
> support as well.  It would be great to integrate it with
> "guile-sparql" if you find the time for it.  The "guile-sparql" code
> is quite short, and hopefully readable enough.. :)

Do you mean integrate only the SPARQL-part or the whole? I agree to the
first statement :) but the other wikidata specifics are perhaps better
kept on its own.

Actually the endpoint is a Blazegraph-server and my get-query code does
not seem to work right (see the bug mentioned in the latest commit).

Does guile-sparql work with Blazegraph? If yes we don't need the (buggy)
sparql-get-query-code I wrote.

I could not get it to work and rolled my own as a teach-yourself-scheme
learning-by-doing pet project :) 

I tried with the file attached but got this because the driver does not
support URIs but only host, port, type, token:

...
In web/client.scm:
    461:0  2 (http-post "http://http://query.wikidata.org/sparql:80…" …)
    181:7  1 (open-socket-for-uri _)
In unknown file:
           0 (getaddrinfo "http" "http" 0 #<undefined> #<undefined> #)

ERROR: In procedure getaddrinfo:
In procedure getaddrinfo: Name or service not known


> 
> I've been working on a web interface to work with RDF/SPARQL (written
> mostly in Guile Scheme, which uses "guile-sparql" as well).  You might
> find it interesting:
> https://github.com/UMCUGenetics/sparqling-genomics
> 
> The Guile Scheme parts are in the "web" subfolder.

Thanks for the tip :)

Additionally I have a suggestion for guile-sparql. Could you generate a
pdf from the texi-manuel and include in the repo?
-- 
Cheers 
Swedebugia

[-- Attachment #2: sdb-test.scm --]
[-- Type: text/plain, Size: 230 bytes --]

(use-modules (sparql driver)
             (sparql lang))

(sparql-query
 "SELECT ?item 
WHERE 
{
?item wdt:P31 wd:Q146.
}
LIMIT 10
"
 #:host "http://query.wikidata.org/sparql"
;; #:port 80
 #:type "text/xml"
 ;;  #:token "..."
 )

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Trouble parsing a response (Was: Re: New library: guile-wikidata)
  2018-12-13 15:01           ` swedebugia
@ 2018-12-13 16:06             ` swedebugia
  2018-12-13 22:03               ` Roel Janssen
  0 siblings, 1 reply; 12+ messages in thread
From: swedebugia @ 2018-12-13 16:06 UTC (permalink / raw)
  To: Roel Janssen; +Cc: guile-user, guile-user

[-- Attachment #1: Type: text/plain, Size: 3721 bytes --]

On 2018-12-13 16:01, swedebugia@riseup.net wrote:
snip

> 
> I tried with the file attached but got this because the driver does not
> support URIs but only host, port, type, token:

Ah, I saw now that you already implemented URI on master :)
https://github.com/roelj/guile-sparql/blob/master/sparql/driver.scm

When I try calling this
;; Example query to wikidata listing cats                               
                                       
(sparql-query
 "SELECT ?item                                                          
                                       
WHERE                                                                   
                                       
{                                                                       
                                       
?item wdt:P31 wd:Q146.                                                  
                                       
}                                                                       
                                       
LIMIT 10                                                                
                                       
"
 #:uri "https://query.wikidata.org/sparql"
 ;; #:port 80                                                           
                                       
 #:type "application/sparql-results+json"
 ;;  #:token "..."                                                      
                                       
 #:store-backend 'blazegraph
 )

I get this fine result:
#<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers:
((date . #<date nanosecond: 0 second: 12 minute: 32 hour: 15 day: 13
month: 12 year: 2018 zone-offset: 0>) (content-type
application/sparql-results+json (charset . "utf-8")) (transfer-encoding
(chunked)) (connection close) (server . "nginx/1.13.6") (x-served-by .
"wdqs1005") (access-control-allow-origin . "*") (cache-control public
(max-age . 300)) (vary accept accept-encoding) (x-varnish . "644531744,
572094009, 417977651") (via "1.1 varnish (Varnish/5.1)" "1.1 varnish
(Varnish/5.1)" "1.1 varnish (Varnish/5.1)") (accept-ranges bytes) (age .
0) (x-cache . "cp1079 pass, cp3030 pass, cp3030 pass") (x-cache-status .
"pass") (server-timing . "cache;desc=\"pass\"")
(strict-transport-security . "max-age=106384710; includeSubDomains;
preload") (set-cookie .
"WMF-Last-Access=13-Dec-2018;Path=/;HttpOnly;secure;Expires=Mon, 14 Jan
2019 12:00:00 GMT") (set-cookie .
"WMF-Last-Access-Global=13-Dec-2018;Path=/;Domain=.wikidata.org;HttpOnly;secure;Expires=Mon,
14 Jan 2019 12:00:00 GMT") (x-analytics . "https=1;nocookies=1")
(x-client-ip . "83.185.90.53")) port: #<input-output: file 8c37e70>>

My problem now is that I don't know how to separate the header from the
port-file.

Ah, reading here
https://www.gnu.org/software/guile/manual/html_node/Responses.html#Responses
I found (response-port).

Unfortunately this only took me one step further as I run into this
instead when trying to parse the port with (json->scm):

Backtrace:
           7 (apply-smob/1 #<catch-closure 9769550>)
In ice-9/boot-9.scm:
    705:2  6 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
In ice-9/eval.scm:
    619:8  5 (_ #(#(#<directory (guile-user) 9759910>)))
In ice-9/boot-9.scm:
   2312:4  4 (save-module-excursion _)
  3831:12  3 (_)
In sdb-test.scm:
     24:1  2 (_)
In json/parser.scm:
   311:18  1 (json-read-number _)
   148:28  0 (read-number _)

json/parser.scm:148:28: In procedure read-number:
Throw to key `json-invalid' with args `(#<json-parser port:
#<input-output: string 98dea80>>)'.

Maybe this is a bug in (json)?


Cheers 
Swedebugia

[-- Attachment #2: sdb-test.scm --]
[-- Type: text/plain, Size: 727 bytes --]

(use-modules (sparql driver)
             (sparql util)
	     (json)
	     (web response))

;; Example query to wikidata listing cats
(define response (sparql-query
  "SELECT ?item 
WHERE 
{
?item wdt:P31 wd:Q146.
}
LIMIT 10
"
  #:uri "https://query.wikidata.org/sparql"
  ;; #:port 80
  #:type "application/sparql-results+json"
  ;;  #:token "..."
  #:store-backend 'blazegraph
  ))


(display
 (json->scm
  (response-port response)))

;; (json->scm
;;  (sparql-query
;;   "SELECT ?item 
;; WHERE 
;; {
;; ?item wdt:P31 wd:Q146.
;; }
;; LIMIT 10
;; "
;;   #:uri "https://query.wikidata.org/sparql"
;;   ;; #:port 80
;;   #:type "application/sparql-results+json"
;;   ;;  #:token "..."
;;   #:store-backend 'blazegraph
;;   ))

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Trouble parsing a response (Was: Re: New library: guile-wikidata)
  2018-12-13 16:06             ` Trouble parsing a response (Was: Re: New library: guile-wikidata) swedebugia
@ 2018-12-13 22:03               ` Roel Janssen
  2018-12-13 22:29                 ` Trouble parsing a response swedebugia
                                   ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Roel Janssen @ 2018-12-13 22:03 UTC (permalink / raw)
  To: swedebugia; +Cc: guile-user, guile-user



On 13-12-18 17:06, swedebugia@riseup.net wrote:
> On 2018-12-13 16:01, swedebugia@riseup.net wrote:
> snip
> 
>>
>> I tried with the file attached but got this because the driver does not
>> support URIs but only host, port, type, token:
> 
> Ah, I saw now that you already implemented URI on master :)
> https://github.com/roelj/guile-sparql/blob/master/sparql/driver.scm
> 
> When I try calling this
> ;; Example query to wikidata listing cats
>                                         
> (sparql-query
>   "SELECT ?item
>                                         
> WHERE
>                                         
> {
>                                         
> ?item wdt:P31 wd:Q146.
>                                         
> }
>                                         
> LIMIT 10
>                                         
> "
>   #:uri "https://query.wikidata.org/sparql"
>   ;; #:port 80
>                                         
>   #:type "application/sparql-results+json"
>   ;;  #:token "..."
>                                         
>   #:store-backend 'blazegraph
>   )
> 
> I get this fine result:
> #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers:
> ((date . #<date nanosecond: 0 second: 12 minute: 32 hour: 15 day: 13
> month: 12 year: 2018 zone-offset: 0>) (content-type
> application/sparql-results+json (charset . "utf-8")) (transfer-encoding
> (chunked)) (connection close) (server . "nginx/1.13.6") (x-served-by .
> "wdqs1005") (access-control-allow-origin . "*") (cache-control public
> (max-age . 300)) (vary accept accept-encoding) (x-varnish . "644531744,
> 572094009, 417977651") (via "1.1 varnish (Varnish/5.1)" "1.1 varnish
> (Varnish/5.1)" "1.1 varnish (Varnish/5.1)") (accept-ranges bytes) (age .
> 0) (x-cache . "cp1079 pass, cp3030 pass, cp3030 pass") (x-cache-status .
> "pass") (server-timing . "cache;desc=\"pass\"")
> (strict-transport-security . "max-age=106384710; includeSubDomains;
> preload") (set-cookie .
> "WMF-Last-Access=13-Dec-2018;Path=/;HttpOnly;secure;Expires=Mon, 14 Jan
> 2019 12:00:00 GMT") (set-cookie .
> "WMF-Last-Access-Global=13-Dec-2018;Path=/;Domain=.wikidata.org;HttpOnly;secure;Expires=Mon,
> 14 Jan 2019 12:00:00 GMT") (x-analytics . "https=1;nocookies=1")
> (x-client-ip . "83.185.90.53")) port: #<input-output: file 8c37e70>>
> 
> My problem now is that I don't know how to separate the header from the
> port-file.
> 
> Ah, reading here
> https://www.gnu.org/software/guile/manual/html_node/Responses.html#Responses
> I found (response-port).
>

Here's what I did in SPARQLing-genomics:
https://github.com/UMCUGenetics/sparqling-genomics/blob/master/web/www/pages/query-response.scm#L86

So basically:

(use-modules
   (ice-9 receive)
   (ice-9 rdelim)
   (web response))

(receive (header port)
   ;; Note: "text/csv" is the only format that is consistent for 
multiple SPARQL back-ends (Virtuoso, BlazeGraph, ...)
   (sparql-query ... #:type "text/csv")
   (if (= (response-code header) 200) ; This means the query went OK.
     (call-some-function port)
     #f)) ; Deal with errors at the #f.

(define (call-some-function port)
   (let ((line (read-line port)))
     (if (eof-object? line)
       #t
       (begin
         (format #t "Line: ~a~%" line)
         ;; Tail-recurse until we have processed each line.
         (call-some-function port)))))

The SPARQLing-genomics code deals with more error codes, and processes 
the lines in a more useful way.

> Unfortunately this only took me one step further as I run into this
> instead when trying to parse the port with (json->scm):
> 
> Backtrace:
>             7 (apply-smob/1 #<catch-closure 9769550>)
> In ice-9/boot-9.scm:
>      705:2  6 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
> In ice-9/eval.scm:
>      619:8  5 (_ #(#(#<directory (guile-user) 9759910>)))
> In ice-9/boot-9.scm:
>     2312:4  4 (save-module-excursion _)
>    3831:12  3 (_)
> In sdb-test.scm:
>       24:1  2 (_)
> In json/parser.scm:
>     311:18  1 (json-read-number _)
>     148:28  0 (read-number _)
> 
> json/parser.scm:148:28: In procedure read-number:
> Throw to key `json-invalid' with args `(#<json-parser port:
> #<input-output: string 98dea80>>)'.
> 
> Maybe this is a bug in (json)?

It looks like the JSON response is not (only) JSON, or simply invalid.
Maybe the "text/xml" or "text/csv" content-type will work better for 
you.  I noticed that each back-end provides their own structure for XML 
and JSON, so I used the somewhat quirky CSV format as a work-for-all 
response type.

I hope this helps.

Kind regards,
Roel Janssen



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Trouble parsing a response
  2018-12-13 22:03               ` Roel Janssen
@ 2018-12-13 22:29                 ` swedebugia
  2018-12-14 15:59                   ` Roel Janssen
  2018-12-26 19:49                 ` Trouble parsing a response (Was: Re: New library: guile-wikidata) swedebugia
  2019-01-03 12:25                 ` swedebugia
  2 siblings, 1 reply; 12+ messages in thread
From: swedebugia @ 2018-12-13 22:29 UTC (permalink / raw)
  To: Roel Janssen; +Cc: guile-user, guile-user

On 2018-12-13 23:03, Roel Janssen wrote:
> On 13-12-18 17:06, swedebugia@riseup.net wrote:
>> On 2018-12-13 16:01, swedebugia@riseup.net wrote:
>> snip
>>
>>>
>>> I tried with the file attached but got this because the driver does not
>>> support URIs but only host, port, type, token:
>>
>> Ah, I saw now that you already implemented URI on master :)
>> https://github.com/roelj/guile-sparql/blob/master/sparql/driver.scm
>>
>> When I try calling this
>> ;; Example query to wikidata listing cats
>>                                         (sparql-query
>>   "SELECT ?item
>>                                         WHERE
>>                                         {
>>                                         ?item wdt:P31 wd:Q146.
>>                                         }
>>                                         LIMIT 10
>>                                         "
>>   #:uri "https://query.wikidata.org/sparql"
>>   ;; #:port 80
>>                                           #:type "application/sparql-results+json"
>>   ;;  #:token "..."
>>                                           #:store-backend 'blazegraph
>>   )
>>
>> I get this fine result:
>> #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers:
>> ((date . #<date nanosecond: 0 second: 12 minute: 32 hour: 15 day: 13
>> month: 12 year: 2018 zone-offset: 0>) (content-type
>> application/sparql-results+json (charset . "utf-8")) (transfer-encoding
>> (chunked)) (connection close) (server . "nginx/1.13.6") (x-served-by .
>> "wdqs1005") (access-control-allow-origin . "*") (cache-control public
>> (max-age . 300)) (vary accept accept-encoding) (x-varnish . "644531744,
>> 572094009, 417977651") (via "1.1 varnish (Varnish/5.1)" "1.1 varnish
>> (Varnish/5.1)" "1.1 varnish (Varnish/5.1)") (accept-ranges bytes) (age .
>> 0) (x-cache . "cp1079 pass, cp3030 pass, cp3030 pass") (x-cache-status .
>> "pass") (server-timing . "cache;desc=\"pass\"")
>> (strict-transport-security . "max-age=106384710; includeSubDomains;
>> preload") (set-cookie .
>> "WMF-Last-Access=13-Dec-2018;Path=/;HttpOnly;secure;Expires=Mon, 14 Jan
>> 2019 12:00:00 GMT") (set-cookie .
>> "WMF-Last-Access-Global=13-Dec-2018;Path=/;Domain=.wikidata.org;HttpOnly;secure;Expires=Mon,
>> 14 Jan 2019 12:00:00 GMT") (x-analytics . "https=1;nocookies=1")
>> (x-client-ip . "83.185.90.53")) port: #<input-output: file 8c37e70>>
>>
>> My problem now is that I don't know how to separate the header from the
>> port-file.
>>
>> Ah, reading here
>> https://www.gnu.org/software/guile/manual/html_node/Responses.html#Responses
>> I found (response-port).
>>
> 
> Here's what I did in SPARQLing-genomics:
> https://github.com/UMCUGenetics/sparqling-genomics/blob/master/web/www/pages/query-response.scm#L86
> 
> So basically:
> 
> (use-modules
>   (ice-9 receive)
>   (ice-9 rdelim)
>   (web response))
> 
> (receive (header port)
>   ;; Note: "text/csv" is the only format that is consistent for
> multiple SPARQL back-ends (Virtuoso, BlazeGraph, ...)
>   (sparql-query ... #:type "text/csv")
>   (if (= (response-code header) 200) ; This means the query went OK.
>     (call-some-function port)
>     #f)) ; Deal with errors at the #f.
> 
> (define (call-some-function port)
>   (let ((line (read-line port)))
>     (if (eof-object? line)
>       #t
>       (begin
>         (format #t "Line: ~a~%" line)
>         ;; Tail-recurse until we have processed each line.
>         (call-some-function port)))))

This would be a really nice addition to the guile manual. Can I submit
it as a patch crediting you?

> 
> The SPARQLing-genomics code deals with more error codes, and processes
> the lines in a more useful way.
> 
>> Unfortunately this only took me one step further as I run into this
>> instead when trying to parse the port with (json->scm):
>>
>> Backtrace:
>>             7 (apply-smob/1 #<catch-closure 9769550>)
>> In ice-9/boot-9.scm:
>>      705:2  6 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
>> In ice-9/eval.scm:
>>      619:8  5 (_ #(#(#<directory (guile-user) 9759910>)))
>> In ice-9/boot-9.scm:
>>     2312:4  4 (save-module-excursion _)
>>    3831:12  3 (_)
>> In sdb-test.scm:
>>       24:1  2 (_)
>> In json/parser.scm:
>>     311:18  1 (json-read-number _)
>>     148:28  0 (read-number _)
>>
>> json/parser.scm:148:28: In procedure read-number:
>> Throw to key `json-invalid' with args `(#<json-parser port:
>> #<input-output: string 98dea80>>)'.
>>
>> Maybe this is a bug in (json)?
> 
> It looks like the JSON response is not (only) JSON, or simply invalid.
> Maybe the "text/xml" or "text/csv" content-type will work better for
> you.  I noticed that each back-end provides their own structure for
> XML and JSON, so I used the somewhat quirky CSV format as a
> work-for-all response type.

Ok, good to know. Is this documented in your guile-sparql docs?

> 
> I hope this helps.

Thanks a lot for taking the time to write this. :)
I already met receive in the npm importer but I did not really learn to
use it yet. So whenever I see multiple objects returned I can part them
with receive! Nice.

Guile is super nice. And fast it seems when you get it right.
-- 
Cheers 
Swedebugia



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Trouble parsing a response
  2018-12-13 22:29                 ` Trouble parsing a response swedebugia
@ 2018-12-14 15:59                   ` Roel Janssen
  0 siblings, 0 replies; 12+ messages in thread
From: Roel Janssen @ 2018-12-14 15:59 UTC (permalink / raw)
  To: swedebugia; +Cc: guile-user, guile-user



On 13-12-18 23:29, swedebugia@riseup.net wrote:
> On 2018-12-13 23:03, Roel Janssen wrote:
>> On 13-12-18 17:06, swedebugia@riseup.net wrote:
>>> On 2018-12-13 16:01, swedebugia@riseup.net wrote:
>>> snip
>>>
>>>>
>>>> I tried with the file attached but got this because the driver does not
>>>> support URIs but only host, port, type, token:
>>>
>>> Ah, I saw now that you already implemented URI on master :)
>>> https://github.com/roelj/guile-sparql/blob/master/sparql/driver.scm
>>>
>>> When I try calling this
>>> ;; Example query to wikidata listing cats
>>>                                          (sparql-query
>>>    "SELECT ?item
>>>                                          WHERE
>>>                                          {
>>>                                          ?item wdt:P31 wd:Q146.
>>>                                          }
>>>                                          LIMIT 10
>>>                                          "
>>>    #:uri "https://query.wikidata.org/sparql"
>>>    ;; #:port 80
>>>                                            #:type "application/sparql-results+json"
>>>    ;;  #:token "..."
>>>                                            #:store-backend 'blazegraph
>>>    )
>>>
>>> I get this fine result:
>>> #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers:
>>> ((date . #<date nanosecond: 0 second: 12 minute: 32 hour: 15 day: 13
>>> month: 12 year: 2018 zone-offset: 0>) (content-type
>>> application/sparql-results+json (charset . "utf-8")) (transfer-encoding
>>> (chunked)) (connection close) (server . "nginx/1.13.6") (x-served-by .
>>> "wdqs1005") (access-control-allow-origin . "*") (cache-control public
>>> (max-age . 300)) (vary accept accept-encoding) (x-varnish . "644531744,
>>> 572094009, 417977651") (via "1.1 varnish (Varnish/5.1)" "1.1 varnish
>>> (Varnish/5.1)" "1.1 varnish (Varnish/5.1)") (accept-ranges bytes) (age .
>>> 0) (x-cache . "cp1079 pass, cp3030 pass, cp3030 pass") (x-cache-status .
>>> "pass") (server-timing . "cache;desc=\"pass\"")
>>> (strict-transport-security . "max-age=106384710; includeSubDomains;
>>> preload") (set-cookie .
>>> "WMF-Last-Access=13-Dec-2018;Path=/;HttpOnly;secure;Expires=Mon, 14 Jan
>>> 2019 12:00:00 GMT") (set-cookie .
>>> "WMF-Last-Access-Global=13-Dec-2018;Path=/;Domain=.wikidata.org;HttpOnly;secure;Expires=Mon,
>>> 14 Jan 2019 12:00:00 GMT") (x-analytics . "https=1;nocookies=1")
>>> (x-client-ip . "83.185.90.53")) port: #<input-output: file 8c37e70>>
>>>
>>> My problem now is that I don't know how to separate the header from the
>>> port-file.
>>>
>>> Ah, reading here
>>> https://www.gnu.org/software/guile/manual/html_node/Responses.html#Responses
>>> I found (response-port).
>>>
>>
>> Here's what I did in SPARQLing-genomics:
>> https://github.com/UMCUGenetics/sparqling-genomics/blob/master/web/www/pages/query-response.scm#L86
>>
>> So basically:
>>
>> (use-modules
>>    (ice-9 receive)
>>    (ice-9 rdelim)
>>    (web response))
>>
>> (receive (header port)
>>    ;; Note: "text/csv" is the only format that is consistent for
>> multiple SPARQL back-ends (Virtuoso, BlazeGraph, ...)
>>    (sparql-query ... #:type "text/csv")
>>    (if (= (response-code header) 200) ; This means the query went OK.
>>      (call-some-function port)
>>      #f)) ; Deal with errors at the #f.
>>
>> (define (call-some-function port)
>>    (let ((line (read-line port)))
>>      (if (eof-object? line)
>>        #t
>>        (begin
>>          (format #t "Line: ~a~%" line)
>>          ;; Tail-recurse until we have processed each line.
>>          (call-some-function port)))))
> 
> This would be a really nice addition to the guile manual. Can I submit
> it as a patch crediting you?

Sure!

>>
>> The SPARQLing-genomics code deals with more error codes, and processes
>> the lines in a more useful way.
>>
>>> Unfortunately this only took me one step further as I run into this
>>> instead when trying to parse the port with (json->scm):
>>>
>>> Backtrace:
>>>              7 (apply-smob/1 #<catch-closure 9769550>)
>>> In ice-9/boot-9.scm:
>>>       705:2  6 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
>>> In ice-9/eval.scm:
>>>       619:8  5 (_ #(#(#<directory (guile-user) 9759910>)))
>>> In ice-9/boot-9.scm:
>>>      2312:4  4 (save-module-excursion _)
>>>     3831:12  3 (_)
>>> In sdb-test.scm:
>>>        24:1  2 (_)
>>> In json/parser.scm:
>>>      311:18  1 (json-read-number _)
>>>      148:28  0 (read-number _)
>>>
>>> json/parser.scm:148:28: In procedure read-number:
>>> Throw to key `json-invalid' with args `(#<json-parser port:
>>> #<input-output: string 98dea80>>)'.
>>>
>>> Maybe this is a bug in (json)?
>>
>> It looks like the JSON response is not (only) JSON, or simply invalid.
>> Maybe the "text/xml" or "text/csv" content-type will work better for
>> you.  I noticed that each back-end provides their own structure for
>> XML and JSON, so I used the somewhat quirky CSV format as a
>> work-for-all response type.
> 
> Ok, good to know. Is this documented in your guile-sparql docs?

No.  I think what SPARQL endpoints actually return is beyond the scope 
of guile-sparql, and may change in the future anyway.

>>
>> I hope this helps.
> 
> Thanks a lot for taking the time to write this. :)
> I already met receive in the npm importer but I did not really learn to
> use it yet. So whenever I see multiple objects returned I can part them
> with receive! Nice.

Exactly.  I'm glad my snippet helped.

Kind regards,
Roel Janssen



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Trouble parsing a response (Was: Re: New library: guile-wikidata)
  2018-12-13 22:03               ` Roel Janssen
  2018-12-13 22:29                 ` Trouble parsing a response swedebugia
@ 2018-12-26 19:49                 ` swedebugia
  2019-01-03 12:25                 ` swedebugia
  2 siblings, 0 replies; 12+ messages in thread
From: swedebugia @ 2018-12-26 19:49 UTC (permalink / raw)
  To: Roel Janssen; +Cc: guile-user, guile-user

[-- Attachment #1: Type: text/plain, Size: 5840 bytes --]

On 2018-12-13 23:03, Roel Janssen wrote:
> 
> 
> On 13-12-18 17:06, swedebugia@riseup.net wrote:
>> On 2018-12-13 16:01, swedebugia@riseup.net wrote:
>> snip
>>
>>>
>>> I tried with the file attached but got this because the driver does not
>>> support URIs but only host, port, type, token:
>>
>> Ah, I saw now that you already implemented URI on master :)
>> https://github.com/roelj/guile-sparql/blob/master/sparql/driver.scm
>>
>> When I try calling this
>> ;; Example query to wikidata listing cats
>> (sparql-query
>>   "SELECT ?item
>> WHERE
>> {
>> ?item wdt:P31 wd:Q146.
>> }
>> LIMIT 10
>> "
>>   #:uri "https://query.wikidata.org/sparql"
>>   ;; #:port 80
>>   #:type "application/sparql-results+json"
>>   ;;  #:token "..."
>>   #:store-backend 'blazegraph
>>   )
>>
>> I get this fine result:
>> #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers:
>> ((date . #<date nanosecond: 0 second: 12 minute: 32 hour: 15 day: 13
>> month: 12 year: 2018 zone-offset: 0>) (content-type
>> application/sparql-results+json (charset . "utf-8")) (transfer-encoding
>> (chunked)) (connection close) (server . "nginx/1.13.6") (x-served-by .
>> "wdqs1005") (access-control-allow-origin . "*") (cache-control public
>> (max-age . 300)) (vary accept accept-encoding) (x-varnish . "644531744,
>> 572094009, 417977651") (via "1.1 varnish (Varnish/5.1)" "1.1 varnish
>> (Varnish/5.1)" "1.1 varnish (Varnish/5.1)") (accept-ranges bytes) (age .
>> 0) (x-cache . "cp1079 pass, cp3030 pass, cp3030 pass") (x-cache-status .
>> "pass") (server-timing . "cache;desc=\"pass\"")
>> (strict-transport-security . "max-age=106384710; includeSubDomains;
>> preload") (set-cookie .
>> "WMF-Last-Access=13-Dec-2018;Path=/;HttpOnly;secure;Expires=Mon, 14 Jan
>> 2019 12:00:00 GMT") (set-cookie .
>> "WMF-Last-Access-Global=13-Dec-2018;Path=/;Domain=.wikidata.org;HttpOnly;secure;Expires=Mon, 
>>
>> 14 Jan 2019 12:00:00 GMT") (x-analytics . "https=1;nocookies=1")
>> (x-client-ip . "83.185.90.53")) port: #<input-output: file 8c37e70>>
>>
>> My problem now is that I don't know how to separate the header from the
>> port-file.
>>
>> Ah, reading here
>> https://www.gnu.org/software/guile/manual/html_node/Responses.html#Responses 
>>
>> I found (response-port).
>>
> 
> Here's what I did in SPARQLing-genomics:
> https://github.com/UMCUGenetics/sparqling-genomics/blob/master/web/www/pages/query-response.scm#L86 
> 
> 
> So basically:
> 
> (use-modules
>    (ice-9 receive)
>    (ice-9 rdelim)
>    (web response))
> 
> (receive (header port)
>    ;; Note: "text/csv" is the only format that is consistent for 
> multiple SPARQL back-ends (Virtuoso, BlazeGraph, ...)
>    (sparql-query ... #:type "text/csv")
>    (if (= (response-code header) 200) ; This means the query went OK.
>      (call-some-function port)
>      #f)) ; Deal with errors at the #f.
> 
> (define (call-some-function port)
>    (let ((line (read-line port)))
>      (if (eof-object? line)
>        #t
>        (begin
>          (format #t "Line: ~a~%" line)
>          ;; Tail-recurse until we have processed each line.
>          (call-some-function port)))))
> 
> The SPARQLing-genomics code deals with more error codes, and processes 
> the lines in a more useful way.
> 
>> Unfortunately this only took me one step further as I run into this
>> instead when trying to parse the port with (json->scm):
>>
>> Backtrace:
>>             7 (apply-smob/1 #<catch-closure 9769550>)
>> In ice-9/boot-9.scm:
>>      705:2  6 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
>> In ice-9/eval.scm:
>>      619:8  5 (_ #(#(#<directory (guile-user) 9759910>)))
>> In ice-9/boot-9.scm:
>>     2312:4  4 (save-module-excursion _)
>>    3831:12  3 (_)
>> In sdb-test.scm:
>>       24:1  2 (_)
>> In json/parser.scm:
>>     311:18  1 (json-read-number _)
>>     148:28  0 (read-number _)
>>
>> json/parser.scm:148:28: In procedure read-number:
>> Throw to key `json-invalid' with args `(#<json-parser port:
>> #<input-output: string 98dea80>>)'.
>>
>> Maybe this is a bug in (json)?
> 
> It looks like the JSON response is not (only) JSON, or simply invalid.
> Maybe the "text/xml" or "text/csv" content-type will work better for 
> you.  I noticed that each back-end provides their own structure for XML 
> and JSON, so I used the somewhat quirky CSV format as a work-for-all 
> response type.

I tried again today on guile 2.2.4 on parabola with guile installed from 
guix and got this:

sdb@parabola:~/src/guile-sparql$ guile -L . -s test.scm
guile: warning: failed to install locale
;;; note: source file /home/egil/src/guile-sparql/test.scm
;;;       newer than compiled 
/home/egil/.cache/guile/ccache/2.2-LE-8-3.A/home/egil/src/guile-sparql/test.scm.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/egil/src/guile-sparql/test.scm
;;; compiled 
/home/egil/.cache/guile/ccache/2.2-LE-8-3.A/home/egil/src/guile-sparql/test.scm.go
Backtrace:
            5 (apply-smob/1 #<catch-closure 23002c0>)
In ice-9/boot-9.scm:
     705:2  4 (call-with-prompt _ _ #<procedure default-prompt-handle?>)
In ice-9/eval.scm:
     619:8  3 (_ #(#(#<directory (guile-user) 23c9140>)))
In ice-9/boot-9.scm:
    2312:4  2 (save-module-excursion _)
   3831:12  1 (_)
In test.scm:
       8:0  0 (_)

test.scm:8:0: Throw to key `vm-error' with args `(vm-run "Wrong number 
of values returned to continuation (expected ~a)" (2))'.

Note: to reproduce you need to run it with guile-sparql from git.

Any idea whats wrong?

-- 
Cheers Swedebugia

[-- Attachment #2: test.scm --]
[-- Type: text/x-scheme, Size: 1217 bytes --]

;; Example query to wikidata listing cats                               
(use-modules
 (sparql driver)
 (ice-9 receive)
 (ice-9 rdelim)
 (web response))

(define q
  (sparql-query
   "
SELECT ?item
WHERE                                                                   
{                                                                       
?item wdt:P31 wd:Q146.                                                  
}                                                                       
LIMIT 10                                                                
"
   #:uri "https://query.wikidata.org/sparql"
   #:type "text/csv"
   #:store-backend 'blazegraph))
(display q)

(receive (header port)
    ;; Note: "text/csv" is the only format that is consistent for multiple SPARQL back-ends (Virtuoso, BlazeGraph, ...)
    q
  (if (= (response-code header) 200) ; This means the query went OK.
    (call-some-function port)
    #f)) ; Deal with errors at the #f.

(define (call-some-function port)
  (let ((line (read-line port)))
    (if (eof-object? line)
      #t
      (begin
        (format #t "Line: ~a~%" line)
        ;; Tail-recurse until we have processed each line.
        (call-some-function port))))) 

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Trouble parsing a response (Was: Re: New library: guile-wikidata)
  2018-12-13 22:03               ` Roel Janssen
  2018-12-13 22:29                 ` Trouble parsing a response swedebugia
  2018-12-26 19:49                 ` Trouble parsing a response (Was: Re: New library: guile-wikidata) swedebugia
@ 2019-01-03 12:25                 ` swedebugia
  2 siblings, 0 replies; 12+ messages in thread
From: swedebugia @ 2019-01-03 12:25 UTC (permalink / raw)
  To: Roel Janssen; +Cc: guile-user, guile-user

[-- Attachment #1: Type: text/plain, Size: 6646 bytes --]

On 2018-12-13 23:03, Roel Janssen wrote:
> 
> 
> On 13-12-18 17:06, swedebugia@riseup.net wrote:
>> On 2018-12-13 16:01, swedebugia@riseup.net wrote:
>> snip
>>
>>>
>>> I tried with the file attached but got this because the driver does not
>>> support URIs but only host, port, type, token:
>>
>> Ah, I saw now that you already implemented URI on master :)
>> https://github.com/roelj/guile-sparql/blob/master/sparql/driver.scm
>>
>> When I try calling this
>> ;; Example query to wikidata listing cats
>> (sparql-query
>>   "SELECT ?item
>> WHERE
>> {
>> ?item wdt:P31 wd:Q146.
>> }
>> LIMIT 10
>> "
>>   #:uri "https://query.wikidata.org/sparql"
>>   ;; #:port 80
>>   #:type "application/sparql-results+json"
>>   ;;  #:token "..."
>>   #:store-backend 'blazegraph
>>   )
>>
>> I get this fine result:
>> #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers:
>> ((date . #<date nanosecond: 0 second: 12 minute: 32 hour: 15 day: 13
>> month: 12 year: 2018 zone-offset: 0>) (content-type
>> application/sparql-results+json (charset . "utf-8")) (transfer-encoding
>> (chunked)) (connection close) (server . "nginx/1.13.6") (x-served-by .
>> "wdqs1005") (access-control-allow-origin . "*") (cache-control public
>> (max-age . 300)) (vary accept accept-encoding) (x-varnish . "644531744,
>> 572094009, 417977651") (via "1.1 varnish (Varnish/5.1)" "1.1 varnish
>> (Varnish/5.1)" "1.1 varnish (Varnish/5.1)") (accept-ranges bytes) (age .
>> 0) (x-cache . "cp1079 pass, cp3030 pass, cp3030 pass") (x-cache-status .
>> "pass") (server-timing . "cache;desc=\"pass\"")
>> (strict-transport-security . "max-age=106384710; includeSubDomains;
>> preload") (set-cookie .
>> "WMF-Last-Access=13-Dec-2018;Path=/;HttpOnly;secure;Expires=Mon, 14 Jan
>> 2019 12:00:00 GMT") (set-cookie .
>> "WMF-Last-Access-Global=13-Dec-2018;Path=/;Domain=.wikidata.org;HttpOnly;secure;Expires=Mon, 
>>
>> 14 Jan 2019 12:00:00 GMT") (x-analytics . "https=1;nocookies=1")
>> (x-client-ip . "83.185.90.53")) port: #<input-output: file 8c37e70>>
>>
>> My problem now is that I don't know how to separate the header from the
>> port-file.
>>
>> Ah, reading here
>> https://www.gnu.org/software/guile/manual/html_node/Responses.html#Responses 
>>
>> I found (response-port).
>>
> 
> Here's what I did in SPARQLing-genomics:
> https://github.com/UMCUGenetics/sparqling-genomics/blob/master/web/www/pages/query-response.scm#L86 
> 
> 
> So basically:
> 
> (use-modules
>    (ice-9 receive)
>    (ice-9 rdelim)
>    (web response))
> 
> (receive (header port)
>    ;; Note: "text/csv" is the only format that is consistent for 
> multiple SPARQL back-ends (Virtuoso, BlazeGraph, ...)
>    (sparql-query ... #:type "text/csv")
>    (if (= (response-code header) 200) ; This means the query went OK.
>      (call-some-function port)
>      #f)) ; Deal with errors at the #f.
> 
> (define (call-some-function port)
>    (let ((line (read-line port)))
>      (if (eof-object? line)
>        #t
>        (begin
>          (format #t "Line: ~a~%" line)
>          ;; Tail-recurse until we have processed each line.
>          (call-some-function port)))))
> 
> The SPARQLing-genomics code deals with more error codes, and processes 
> the lines in a more useful way.
> 
>> Unfortunately this only took me one step further as I run into this
>> instead when trying to parse the port with (json->scm):
>>
>> Backtrace:
>>             7 (apply-smob/1 #<catch-closure 9769550>)
>> In ice-9/boot-9.scm:
>>      705:2  6 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
>> In ice-9/eval.scm:
>>      619:8  5 (_ #(#(#<directory (guile-user) 9759910>)))
>> In ice-9/boot-9.scm:
>>     2312:4  4 (save-module-excursion _)
>>    3831:12  3 (_)
>> In sdb-test.scm:
>>       24:1  2 (_)
>> In json/parser.scm:
>>     311:18  1 (json-read-number _)
>>     148:28  0 (read-number _)
>>
>> json/parser.scm:148:28: In procedure read-number:
>> Throw to key `json-invalid' with args `(#<json-parser port:
>> #<input-output: string 98dea80>>)'.
>>
>> Maybe this is a bug in (json)?
> 
> It looks like the JSON response is not (only) JSON, or simply invalid.
> Maybe the "text/xml" or "text/csv" content-type will work better for 
> you.  I noticed that each back-end provides their own structure for XML 
> and JSON, so I used the somewhat quirky CSV format as a work-for-all 
> response type.
> 
> I hope this helps.

You were right!

I debugged away and got this in the end after many trial and errors:

$ env |grep ssl
GIT_SSL_CAINFO=/home/egil/.guix-profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_DIR=/home/egil/.guix-profile/etc/ssl/certs
$ guile --version
guile (GNU Guile) 2.2.4
$ guix --version
guix (GNU Guix) 0.16.0 <- installed from binary 0.16 on parabola.

$ guile -s test2.scm
Line: 1aa
Line: item
Line: http://www.wikidata.org/entity/Q28114532
Line: http://www.wikidata.org/entity/Q28114535
Line: http://www.wikidata.org/entity/Q28665865
Line: http://www.wikidata.org/entity/Q28792126
Line: http://www.wikidata.org/entity/Q30600575
Line: http://www.wikidata.org/entity/Q42442324
Line: http://www.wikidata.org/entity/Q43260736
Line: http://www.wikidata.org/entity/Q48895080
Line: http://www.wikidata.org/entity/Q49581026
Line: http://www.wikidata.org/entity/Q50378472
Line:
Line: 0
Line:
Backtrace:
            9 (apply-smob/1 #<catch-closure 18835c0>)
In ice-9/boot-9.scm:
     705:2  8 (call-with-prompt _ _ #<procedure default-prompt-handler 
(k proc)>)
In ice-9/eval.scm:
     619:8  7 (_ #(#(#<directory (guile-user) 18f2140>)))
In ice-9/boot-9.scm:
    2312:4  6 (save-module-excursion _)
   3831:12  5 (_)
In test2.scm:
     51:14  4 (read #<input-output: string 1c22d20>)
In ice-9/rdelim.scm:
    195:24  3 (read-line _ _)
In unknown file:
            2 (%read-line #<input-output: string 1c22d20>)
In web/client.scm:
    142:24  1 (read! #vu8(48 13 10 13 10 103 47 101 110 116 105 116 121 
47 81 52 56 56 57 53 48 56 48 13 10 104 116 116 112 58 47 47 119 119 119 
46 119 ?) ?)
In unknown file:
            0 (get-bytevector-some #<input-output: string 1cf51c0>)

ERROR: In procedure get-bytevector-some:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum The TLS 
connection was non-properly terminated.> read_from_session_record_port)'.

Can anyone replicate this? (run the attachment)
Is this a bug in guile?
How do I ignore this error?

-- 
Cheers Swedebugia

[-- Attachment #2: test2.scm --]
[-- Type: text/x-scheme, Size: 2011 bytes --]

;; Example query to wikidata listing cats                               
(use-modules
 (ice-9 receive)
 (ice-9 rdelim)
 (ice-9 textual-ports)
 (web response)
 (web client)
 (web uri))

(define q2
  "
SELECT ?item
WHERE                                                                   
{                                                                       
?item wdt:P31 wd:Q146.                                                  
}                                                                       
LIMIT 10                                                                
")
(define post-url "https://query.wikidata.org/sparql")

(define json "application/sparql-results+json")
(define csv "text/csv") 
(define type csv)

(define* (old-url-encoding input #:optional (index 0) (output ""))
  (if (< (string-length input) 3)
      (string-append output input)
      (let ((triple (substring/read-only input index (+ index 3))))
        (if (string= triple "%20")
            (old-url-encoding (string-drop input 3)
                              0
                              (string-append output "+"))
            (old-url-encoding (string-drop input 1)
                              0
                              (string-append output
                                             (string
                                              (string-ref input index))))))))

(define r
  (let ((query q2))
   (http-post post-url
              #:body (string-append "query=" (old-url-encoding
                                              (uri-encode query)))
              #:streaming? #t
              #:headers
              `((user-agent   . "GNU Guile")
		(content-type . (application/x-www-form-urlencoded))
		(accept       . ((,(string->symbol type))))))))

(define (read port)
  (let ((line (read-line port)))
    (if (eof-object? line)
	#t
	(begin
          (format #t "Line: ~a~%" line)
          ;; Tail-recurse until we have processed each line.
          (read port)))))

(read (response-port r))


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2019-01-03 12:25 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <5ffe968620f1d2d940b7db2b1900dc43@riseup.net>
2018-12-09  9:11 ` New library: guile-wikidata swedebugia
2018-12-09 11:08   ` tomas
2018-12-09 21:26     ` Arne Babenhauserheide
2018-12-11  0:32       ` swedebugia
2018-12-11 10:29         ` Roel Janssen
2018-12-13 15:01           ` swedebugia
2018-12-13 16:06             ` Trouble parsing a response (Was: Re: New library: guile-wikidata) swedebugia
2018-12-13 22:03               ` Roel Janssen
2018-12-13 22:29                 ` Trouble parsing a response swedebugia
2018-12-14 15:59                   ` Roel Janssen
2018-12-26 19:49                 ` Trouble parsing a response (Was: Re: New library: guile-wikidata) swedebugia
2019-01-03 12:25                 ` swedebugia

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).