unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
From: Vivien Kraus <vivien@planete-kraus.eu>
To: "Sébastien Rey-Coyrehourcq"
	<sebastien.rey-coyrehourcq@univ-rouen.fr>,
	guile-user@gnu.org
Subject: Re: Some help needed to use curl lib to download binary file
Date: Fri, 29 Jul 2022 12:25:53 +0200	[thread overview]
Message-ID: <ef81da2784f52a94e260ba3aa7d1bee96b172339.camel@planete-kraus.eu> (raw)
In-Reply-To: <874jz01erv.fsf@univ-rouen.fr>

Hello,

I see in the paste:

> ;; function taken on
> https://gist.github.com/amirouche/138a27bdbef5a672a0135f90ca26ec41
> ;; then adapted to use cookie jar
> (define-public (http-get url cookie-exist)
>   ;; Create a Curl handle
>   (let ((handle (curl-easy-init)))
>     ;; Set the URL from which to get the data
>     (curl-easy-setopt handle 'url url)
>     (if cookie-exist
>         (curl-easy-setopt handle 'cookie "cookie.txt")
>         (curl-easy-setopt handle 'cookiejar "cookie.txt"))
> 
>     ;; Request that the HTTP headers be included in the response
>     (curl-easy-setopt handle 'header #t)
>     ;; Get the result as a Latin-1 string
>     (let* ((response-string (curl-easy-perform handle))
>            ;; Create a string port from the response
>            (response-port (open-input-string response-string))
>            ;; Have the (web response) module to parse the response
>            (response (read-response response-port))
>            (body (utf8->string (read-response-body response))))
>       (close response-port)
>       ;; Have the (web response) module extract the body from the
>       ;; response
>       (values response body))))

So here the call expects the response to be UTF-8 text. If it is a
binary file that you are downloading, the function will raise an
exception. Guile has that python3 feeling where you are supposed to
know in advance whether what you are using is text or binary, which is
hurting you here. 

However, you can avoid the problem by either having bytevectors
everywhere, so removing the call to utf8->string and bind the "body"
variable directly to (read-response-body response), or pretend that you
know better than guile and pretend that it is latin-1-encoded, so you
lose no information and guile won’t complain. In that case, load (ice-9
iconv) and replace (utf8->string …) with (bytevector->string … "ISO-
8859-1").

If you go the first route, you get a bytevector back. If you go the
second one, you get a string, but you must remember that it contains
raw bytes and not text (unless the response body was indeed text).

There are some cases when you might want to have such strings-that-
contain-binary-or-text, such as if you want to use the strings API on
them and the bytevector API does not provide what you want, or you want
to interface with NUL-terminated strings. In other cases you might
prefer bytevectors. Anyway, there is no encoding cost to convert
between strings-that-contain-binary-or-toxt and bytevectors, it is as
simple as a copy.

> ;; write content into file
> (call-with-output-file “download.zip” (lambda (current-output-port)
>                                         (get-file get-file-link)
>                                         (put-bytevector (current-
> output-port) body)))

call-with-output-file calls its function argument with a port. Plus,
you just ignore the result of get-file, so instead you should do
something with it.

If you chose to have get-file return a bytevector, you can do:

(call-with-output-file "download.zip"
  (lambda (port)
    (put-bytevector port (get-file get-file-link)))
  #:binary #t)

If get-file returns a string, you can do:

(call-with-output-file "download.zip"
  (lambda (port)
    (put-string port (get-file get-file-link)))
  #:encoding "ISO-8859-1")

Best regards,

Vivien




  reply	other threads:[~2022-07-29 10:25 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-07-29  8:41 Some help needed to use curl lib to download binary file Sébastien Rey-Coyrehourcq
2022-07-29 10:25 ` Vivien Kraus [this message]
2022-08-01  7:48   ` Sébastien Rey-Coyrehourcq

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/guile/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ef81da2784f52a94e260ba3aa7d1bee96b172339.camel@planete-kraus.eu \
    --to=vivien@planete-kraus.eu \
    --cc=guile-user@gnu.org \
    --cc=sebastien.rey-coyrehourcq@univ-rouen.fr \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).