From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: swedebugia Newsgroups: gmane.lisp.guile.user Subject: Re: Trouble parsing a response (Was: Re: New library: guile-wikidata) Date: Thu, 3 Jan 2019 13:25:37 +0100 Message-ID: <568bae0a-0012-bd5e-d251-9b03aee23245@riseup.net> References: <5ffe968620f1d2d940b7db2b1900dc43@riseup.net> <9f876db761468c16d1a6e0ee27c30821@riseup.net> <20181209110814.GA20268@tuxteam.de> <87tvjms87o.fsf@web.de> <33b7a105-8529-1043-c5bd-5b7fccefc042@gnu.org> <413312f3b999443b86b604ea9beff618@riseup.net> <7ebe1814-8aff-2d09-dc54-f978b4d16fe0@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------DE18C75242A050E58064E54C" X-Trace: blaine.gmane.org 1546517847 30262 195.159.176.226 (3 Jan 2019 12:17:27 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 3 Jan 2019 12:17:27 +0000 (UTC) Cc: guile-user , guile-user@gnu.org To: Roel Janssen Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Thu Jan 03 13:17:23 2019 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gf1w9-0007iU-4X for guile-user@m.gmane.org; Thu, 03 Jan 2019 13:17:21 +0100 Original-Received: from localhost ([127.0.0.1]:52934 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf1yF-0002U2-Su for guile-user@m.gmane.org; Thu, 03 Jan 2019 07:19:31 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:33125) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf1xs-0002TO-4Z for guile-user@gnu.org; Thu, 03 Jan 2019 07:19:09 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gf1xq-0000nu-Nk for guile-user@gnu.org; Thu, 03 Jan 2019 07:19:08 -0500 Original-Received: from mx1.riseup.net ([198.252.153.129]:41376) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gf1xl-0000lr-HP; Thu, 03 Jan 2019 07:19:01 -0500 Original-Received: from piha.riseup.net (piha-pn.riseup.net [10.0.1.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.riseup.net (Postfix) with ESMTPS id 5BC811A0142; Thu, 3 Jan 2019 04:19:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1546517940; bh=8n9ZJqKCeNTYGIlGhxkvT6p4212PF6sJk+9o2vH1HFc=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=FDPrzQF2K/mLyvAHg38iDOq0FcWD0HwIxLstXP1rSq5qBfpPwWj3anY0IAGb2h8zT bpsz+xiDVti/uXZvnzk9WO6KTHUxwv9FAeO5Iqqcdj3M4CEuZr46TXUbPRDsT/xgN5 Ovlc1clpGXl27AwGjVrGv5W27YBxZDijHPddSi7c= X-Riseup-User-ID: 208BA5526D7EC07C3D9603833859572D05A6A174CC3A30E16A0BF2A04676E179 Original-Received: from [127.0.0.1] (localhost [127.0.0.1]) by piha.riseup.net with ESMTPSA id 06F6E1C05C6; Thu, 3 Jan 2019 04:18:58 -0800 (PST) In-Reply-To: <7ebe1814-8aff-2d09-dc54-f978b4d16fe0@gnu.org> Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 198.252.153.129 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:15189 Archived-At: This is a multi-part message in MIME format. --------------DE18C75242A050E58064E54C Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 2018-12-13 23:03, Roel Janssen wrote: >=20 >=20 > On 13-12-18 17:06, swedebugia@riseup.net wrote: >> On 2018-12-13 16:01, swedebugia@riseup.net wrote: >> snip >> >>> >>> I tried with the file attached but got this because the driver does n= ot >>> support URIs but only host, port, type, token: >> >> Ah, I saw now that you already implemented URI on master :) >> https://github.com/roelj/guile-sparql/blob/master/sparql/driver.scm >> >> When I try calling this >> ;; Example query to wikidata listing cats >> (sparql-query >> =C2=A0 "SELECT ?item >> WHERE >> { >> ?item wdt:P31 wd:Q146. >> } >> LIMIT 10 >> " >> =C2=A0 #:uri "https://query.wikidata.org/sparql" >> =C2=A0 ;; #:port 80 >> =C2=A0 #:type "application/sparql-results+json" >> =C2=A0 ;;=C2=A0 #:token "..." >> =C2=A0 #:store-backend 'blazegraph >> =C2=A0 ) >> >> I get this fine result: >> #< version: (1 . 1) code: 200 reason-phrase: "OK" headers: >> ((date . #> month: 12 year: 2018 zone-offset: 0>) (content-type >> application/sparql-results+json (charset . "utf-8")) (transfer-encodin= g >> (chunked)) (connection close) (server . "nginx/1.13.6") (x-served-by . >> "wdqs1005") (access-control-allow-origin . "*") (cache-control public >> (max-age . 300)) (vary accept accept-encoding) (x-varnish . "644531744= , >> 572094009, 417977651") (via "1.1 varnish (Varnish/5.1)" "1.1 varnish >> (Varnish/5.1)" "1.1 varnish (Varnish/5.1)") (accept-ranges bytes) (age= . >> 0) (x-cache . "cp1079 pass, cp3030 pass, cp3030 pass") (x-cache-status= . >> "pass") (server-timing . "cache;desc=3D\"pass\"") >> (strict-transport-security . "max-age=3D106384710; includeSubDomains; >> preload") (set-cookie . >> "WMF-Last-Access=3D13-Dec-2018;Path=3D/;HttpOnly;secure;Expires=3DMon,= 14 Jan >> 2019 12:00:00 GMT") (set-cookie . >> "WMF-Last-Access-Global=3D13-Dec-2018;Path=3D/;Domain=3D.wikidata.org;= HttpOnly;secure;Expires=3DMon,=20 >> >> 14 Jan 2019 12:00:00 GMT") (x-analytics . "https=3D1;nocookies=3D1") >> (x-client-ip . "83.185.90.53")) port: #> >> >> My problem now is that I don't know how to separate the header from th= e >> port-file. >> >> Ah, reading here >> https://www.gnu.org/software/guile/manual/html_node/Responses.html#Res= ponses=20 >> >> I found (response-port). >> >=20 > Here's what I did in SPARQLing-genomics: > https://github.com/UMCUGenetics/sparqling-genomics/blob/master/web/www/= pages/query-response.scm#L86=20 >=20 >=20 > So basically: >=20 > (use-modules > =C2=A0 (ice-9 receive) > =C2=A0 (ice-9 rdelim) > =C2=A0 (web response)) >=20 > (receive (header port) > =C2=A0 ;; Note: "text/csv" is the only format that is consistent for=20 > multiple SPARQL back-ends (Virtuoso, BlazeGraph, ...) > =C2=A0 (sparql-query ... #:type "text/csv") > =C2=A0 (if (=3D (response-code header) 200) ; This means the query wen= t OK. > =C2=A0=C2=A0=C2=A0 (call-some-function port) > =C2=A0=C2=A0=C2=A0 #f)) ; Deal with errors at the #f. >=20 > (define (call-some-function port) > =C2=A0 (let ((line (read-line port))) > =C2=A0=C2=A0=C2=A0 (if (eof-object? line) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 #t > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (begin > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (format #t "Line: ~a~%" lin= e) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ;; Tail-recurse until we ha= ve processed each line. > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (call-some-function port)))= )) >=20 > The SPARQLing-genomics code deals with more error codes, and processes=20 > the lines in a more useful way. >=20 >> Unfortunately this only took me one step further as I run into this >> instead when trying to parse the port with (json->scm): >> >> Backtrace: >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7 (= apply-smob/1 #) >> In ice-9/boot-9.scm: >> =C2=A0=C2=A0=C2=A0=C2=A0 705:2=C2=A0 6 (call-with-prompt _ _ #) >> In ice-9/eval.scm: >> =C2=A0=C2=A0=C2=A0=C2=A0 619:8=C2=A0 5 (_ #(#(#))) >> In ice-9/boot-9.scm: >> =C2=A0=C2=A0=C2=A0 2312:4=C2=A0 4 (save-module-excursion _) >> =C2=A0=C2=A0 3831:12=C2=A0 3 (_) >> In sdb-test.scm: >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 24:1=C2=A0 2 (_) >> In json/parser.scm: >> =C2=A0=C2=A0=C2=A0 311:18=C2=A0 1 (json-read-number _) >> =C2=A0=C2=A0=C2=A0 148:28=C2=A0 0 (read-number _) >> >> json/parser.scm:148:28: In procedure read-number: >> Throw to key `json-invalid' with args `(#> #>)'. >> >> Maybe this is a bug in (json)? >=20 > It looks like the JSON response is not (only) JSON, or simply invalid. > Maybe the "text/xml" or "text/csv" content-type will work better for=20 > you.=C2=A0 I noticed that each back-end provides their own structure fo= r XML=20 > and JSON, so I used the somewhat quirky CSV format as a work-for-all=20 > response type. >=20 > I hope this helps. You were right! I debugged away and got this in the end after many trial and errors: $ env |grep ssl GIT_SSL_CAINFO=3D/home/egil/.guix-profile/etc/ssl/certs/ca-certificates.c= rt SSL_CERT_DIR=3D/home/egil/.guix-profile/etc/ssl/certs $ guile --version guile (GNU Guile) 2.2.4 $ guix --version guix (GNU Guix) 0.16.0 <- installed from binary 0.16 on parabola. $ guile -s test2.scm Line: 1aa Line: item Line: http://www.wikidata.org/entity/Q28114532 Line: http://www.wikidata.org/entity/Q28114535 Line: http://www.wikidata.org/entity/Q28665865 Line: http://www.wikidata.org/entity/Q28792126 Line: http://www.wikidata.org/entity/Q30600575 Line: http://www.wikidata.org/entity/Q42442324 Line: http://www.wikidata.org/entity/Q43260736 Line: http://www.wikidata.org/entity/Q48895080 Line: http://www.wikidata.org/entity/Q49581026 Line: http://www.wikidata.org/entity/Q50378472 Line: Line: 0 Line: Backtrace: 9 (apply-smob/1 #) In ice-9/boot-9.scm: 705:2 8 (call-with-prompt _ _ #) In ice-9/eval.scm: 619:8 7 (_ #(#(#))) In ice-9/boot-9.scm: 2312:4 6 (save-module-excursion _) 3831:12 5 (_) In test2.scm: 51:14 4 (read #) In ice-9/rdelim.scm: 195:24 3 (read-line _ _) In unknown file: 2 (%read-line #) In web/client.scm: 142:24 1 (read! #vu8(48 13 10 13 10 103 47 101 110 116 105 116 121=20 47 81 52 56 56 57 53 48 56 48 13 10 104 116 116 112 58 47 47 119 119 119=20 46 119 ?) ?) In unknown file: 0 (get-bytevector-some #) ERROR: In procedure get-bytevector-some: Throw to key `gnutls-error' with args `(# read_from_session_record_port)'. Can anyone replicate this? (run the attachment) Is this a bug in guile? How do I ignore this error? --=20 Cheers Swedebugia --------------DE18C75242A050E58064E54C Content-Type: text/x-scheme; name="test2.scm" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="test2.scm" ;; Example query to wikidata listing cats (use-modules (ice-9 receive) (ice-9 rdelim) (ice-9 textual-ports) (web response) (web client) (web uri)) (define q2 " SELECT ?item WHERE { ?item wdt:P31 wd:Q146. } LIMIT 10 ") (define post-url "https://query.wikidata.org/sparql") (define json "application/sparql-results+json") (define csv "text/csv") (define type csv) (define* (old-url-encoding input #:optional (index 0) (output "")) (if (< (string-length input) 3) (string-append output input) (let ((triple (substring/read-only input index (+ index 3)))) (if (string= triple "%20") (old-url-encoding (string-drop input 3) 0 (string-append output "+")) (old-url-encoding (string-drop input 1) 0 (string-append output (string (string-ref input index)))))))) (define r (let ((query q2)) (http-post post-url #:body (string-append "query=" (old-url-encoding (uri-encode query))) #:streaming? #t #:headers `((user-agent . "GNU Guile") (content-type . (application/x-www-form-urlencoded)) (accept . ((,(string->symbol type)))))))) (define (read port) (let ((line (read-line port))) (if (eof-object? line) #t (begin (format #t "Line: ~a~%" line) ;; Tail-recurse until we have processed each line. (read port))))) (read (response-port r)) --------------DE18C75242A050E58064E54C--