From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?S=C3=A9bastien?= Rey-Coyrehourcq Newsgroups: gmane.lisp.guile.user Subject: Re: Some help needed to use curl lib to download binary file Date: Mon, 01 Aug 2022 09:48:47 +0200 Message-ID: <87tu6wz7o0.fsf@univ-rouen.fr> References: <874jz01erv.fsf@univ-rouen.fr> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="===-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18563"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.6.11; emacs 28.1.90 Cc: guile-user@gnu.org To: Vivien Kraus Original-X-From: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Mon Aug 01 10:33:25 2022 Return-path: Envelope-to: guile-user@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oIQrc-0004aq-Kr for guile-user@m.gmane-mx.org; Mon, 01 Aug 2022 10:33:24 +0200 Original-Received: from localhost ([::1]:36860 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oIQra-0003QK-VS for guile-user@m.gmane-mx.org; Mon, 01 Aug 2022 04:33:23 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:57426) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oIQqD-0003OO-Du for guile-user@gnu.org; Mon, 01 Aug 2022 04:32:01 -0400 Original-Received: from smtp-out-2.univ-rouen.fr ([193.52.152.97]:7006 helo=mailhoc.univ-rouen.fr) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oIQq8-0001ny-9t for guile-user@gnu.org; Mon, 01 Aug 2022 04:31:57 -0400 Original-Received: from localhost (i16-les01-ix2-62-35-41-231.sfr.lns.abo.bbox.fr [62.35.41.231]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailhoc.univ-rouen.fr (Postfix) with ESMTPSA id B49E91D3F; Mon, 1 Aug 2022 10:31:45 +0200 (CEST) In-reply-to: X-Spamd-Bar: -- Authentication-Results: ORIGINATING; auth=pass smtp.auth=reycoseb smtp.mailfrom=sebastien.rey-coyrehourcq@univ-rouen.fr X-Rspamd-Server: mailhoc X-Rspamd-Queue-Id: B49E91D3F X-Spamd-Result: default: False [-2.20 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM(-0.00)[-0.827,0]; SIGNED_PGP(-2.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~]; ASN(0.00)[asn:5410, ipnet:62.34.0.0/15, country:FR]; MID_RHS_MATCH_FROM(0.00)[] Received-SPF: pass client-ip=193.52.152.97; envelope-from=sebastien.rey-coyrehourcq@univ-rouen.fr; helo=mailhoc.univ-rouen.fr X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL=1.31, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.io gmane.lisp.guile.user:18469 Archived-At: --===-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: multipart/mixed; boundary="==-=-=" --==-=-= Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Thanks a lot for this very detailled explanation Vivien, that=E2=80=99s hel= p me a lot to understand what happens here !! I=E2=80=99m trying to continue my learning of Lisp/Scheme like language, th= is is a new fascinating world to explore :) Best regards, Vivien Kraus writes: > Hello, > > I see in the paste: > >> ;; function taken on >> >> ;; then adapted to use cookie jar >> (define-public (http-get url cookie-exist) >> =C2=A0 ;; Create a Curl handle >> =C2=A0 (let ((handle (curl-easy-init))) >> =C2=A0=C2=A0=C2=A0 ;; Set the URL from which to get the data >> =C2=A0=C2=A0=C2=A0 (curl-easy-setopt handle =E2=80=99url url) >> =C2=A0=C2=A0=C2=A0 (if cookie-exist >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (curl-easy-setopt handle =E2= =80=99cookie =E2=80=9Ccookie.txt=E2=80=9D) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (curl-easy-setopt handle =E2= =80=99cookiejar =E2=80=9Ccookie.txt=E2=80=9D)) >> >> =C2=A0=C2=A0=C2=A0 ;; Request that the HTTP headers be included in the r= esponse >> =C2=A0=C2=A0=C2=A0 (curl-easy-setopt handle =E2=80=99header #t) >> =C2=A0=C2=A0=C2=A0 ;; Get the result as a Latin-1 string >> =C2=A0=C2=A0=C2=A0 (let* ((response-string (curl-easy-perform handle)) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ;; Create a= string port from the response >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (response-p= ort (open-input-string response-string)) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ;; Have the= (web response) module to parse the response >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (response (= read-response response-port)) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (body (utf8= ->string (read-response-body response)))) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (close response-port) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ;; Have the (web response) module extract= the body from the >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ;; response >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (values response body)))) > > So here the call expects the response to be UTF-8 text. If it is a > binary file that you are downloading, the function will raise an > exception. Guile has that python3 feeling where you are supposed to > know in advance whether what you are using is text or binary, which is > hurting you here.=C2=A0 > > However, you can avoid the problem by either having bytevectors > everywhere, so removing the call to utf8->string and bind the =E2=80=9Cbo= dy=E2=80=9D > variable directly to (read-response-body response), or pretend that you > know better than guile and pretend that it is latin-1-encoded, so you > lose no information and guile won=E2=80=99t complain. In that case, load = (ice-9 > iconv) and replace (utf8->string =E2=80=A6) with (bytevector->string =E2= =80=A6 =E2=80=9CISO- > 8859-1=E2=80=9D). > > If you go the first route, you get a bytevector back. If you go the > second one, you get a string, but you must remember that it contains > raw bytes and not text (unless the response body was indeed text). > > There are some cases when you might want to have such strings-that- > contain-binary-or-text, such as if you want to use the strings API on > them and the bytevector API does not provide what you want, or you want > to interface with NUL-terminated strings. In other cases you might > prefer bytevectors. Anyway, there is no encoding cost to convert > between strings-that-contain-binary-or-toxt and bytevectors, it is as > simple as a copy. > >> ;; write content into file >> (call-with-output-file =E2=80=9Cdownload.zip=E2=80=9D (lambda (current-o= utput-port) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 (get-file get-file-link) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 (put-bytevector (current- >> output-port) body))) > > call-with-output-file calls its function argument with a port. Plus, > you just ignore the result of get-file, so instead you should do > something with it. > > If you chose to have get-file return a bytevector, you can do: > > (call-with-output-file =E2=80=9Cdownload.zip=E2=80=9D > =C2=A0 (lambda (port) > =C2=A0=C2=A0=C2=A0 (put-bytevector port (get-file get-file-link))) > =C2=A0 #:binary #t) > > If get-file returns a string, you can do: > > (call-with-output-file =E2=80=9Cdownload.zip=E2=80=9D > =C2=A0 (lambda (port) > =C2=A0=C2=A0=C2=A0 (put-string port (get-file get-file-link))) > =C2=A0 #:encoding =E2=80=9CISO-8859-1=E2=80=9D) > > Best regards, > > Vivien --==-=-=-- --=-=-=-- --===-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJcBAEBCABGFiEEQdBRWS1ZqcB6tN8l3FXLa3BDQW4FAmLnj28oHHNlYmFzdGll bi5yZXktY295cmVob3VyY3FAdW5pdi1yb3Vlbi5mcgAKCRDcVctrcENBbtPxD/4w 4FpfNCUxzioeTJ9aZAG/Y09Kaf4f/f+F0IFcArjo+5D/tivcN/t35uJPIC1pnz5N qMPI18Fh4zLZpZNTguUWyL0g1epOjrFgXv3LhO7ZsdAX1Tni1koD6mk0dUN3j+L/ +Eku4a7mYfwn/dtopGdZDNsyPf/zGvcRHb9FdlVvmtCPUgS/NPwBzwZDgEfJxEAd 9xblYRov80n6y6R/J67Q3IzpRAJOksCv4/8RLTo0EKs9X2Dxg4YFhXJCvicBDyBz KSiJFS98hmRiwL1iwSoJeEtvLk67f2aZh98ONJQf+BubE1tR/+1Q5hbxXJISHydc RQXvRhyYGutlTWff8/HAqjssUSeouf35lPfz+q+r1Y0tPLBoMCUy3QDO5yrXeIA2 JnEkMwNqD1Ik+lxYzVFtz0eCP0ub5rPHl/qMUDAbaEO+HHRt+cBP7F+u2FxWlgy5 Rmk/SSRzWoPhvEHeMPVGurMfnwFmPd+ARBbkCjIrzA7d+B10HaXWSQj1sOG905zj 8W2qNOQBywzmO3y8q242vmUau/P1r8HXsK7Iz5q1IJ5j3XptEjJzyYMeFrKWZu0t 2zHLPlTyKO2m7bdouWgc+En9azI2wVKEjOPrzOETkDb0PnGvBnonYhbuZkbhRZp0 tBeTVDQ3k6fLwLgwMIwAf8ocHbcKesSSZ9kW7wzqXw== =m0zp -----END PGP SIGNATURE----- --===-=-=--