Re: Maintaining implementations of similar utility functions like json-fetch

[ludo@gnu.org (Ludovic Courtès)]

Hello Jelle!

Jelle Licht <jlicht@fsfe.org> skribis:

> Sorry for the delay. Cue the drum roll; Attached is my initial draft of
> this patch. I initially wanted to split it up into 2 or more patches, but
> could not make this work in a way that I could wrap my head around.

Better late than never!  :-)  It looks good to me.

> Also, there is yet another 'json-fetch'-like function implemented in
> `guix/ci.scm', but I was not sure whether the error-handling facilities
> would be applicable there.

I’d leave (guix ci) as-is.  It’s used by (guix scripts weather), which
has custom HTTP error handling in this case.

> From c60686975df2999906118c3a26cc9c2cef2a93b2 Mon Sep 17 00:00:00 2001
> From: Jelle Licht <jlicht@fsfe.org>
> Date: Sun, 10 Jun 2018 20:35:39 +0200
> Subject: [PATCH] import: json: Consolidate duplicate json-fetch functionality.
>
> * guix/import/json.scm (json-fetch): Return a list or hash table.
>   (json-fetch-alist): New procedure.
> * guix/import/github.scm (json-fetch*): Remove.
>   (latest-released-version): Use json-fetch.
> * guix/import/cpan.scm (module->dist-name): Use json-fetch-alist.
>   (cpan-fetch): Likewise.
> * guix/import/crate.scm (crate-fetch): Likewise.
> * guix/import/gem.scm (rubygems-fetch): Likewise.
> * guix/import/pypi.scm (pypi-fetch): Likewise.
> * guix/import/stackage.scm (stackage-lts-info-fetch): Likewise.

LGTM!

> --- a/guix/import/json.scm
> +++ b/guix/import/json.scm
> @@ -22,15 +22,25 @@
>    #:use-module (guix http-client)
>    #:use-module (guix import utils)
>    #:use-module (srfi srfi-34)
> -  #:export (json-fetch))
> +  #:export (json-fetch
> +            json-fetch-alist))
>  
>  (define (json-fetch url)
> -  "Return an alist representation of the JSON resource URL, or #f on failure."
> +  "Return a representation of the JSON resource URL (a list or hash table), or
> +#f if URL returns 403 or 404."
>    (guard (c ((and (http-get-error? c)
> -                  (= 404 (http-get-error-code c)))
> -             #f))                       ;"expected" if package is unknown
> -    (let* ((port (http-fetch url #:headers '((user-agent . "GNU Guile")
> -                                             (Accept . "application/json"))))
> -           (result (hash-table->alist (json->scm port))))
> +                  (let ((error (http-get-error-code c)))
> +                    (or (= 403 error)
> +                        (= 404 error))))
> +             #f))
> +    ;; Note: many websites returns 403 if we omit a 'User-Agent' header.
> +    (let* ((port   (http-fetch url #:headers '((user-agent . "GNU Guile")
> +                                               (Accept . "application/json"))))
> +           (result (json->scm port)))
>        (close-port port)
>        result)))
> +
> +(define (json-fetch-alist url)
> +  "Return an alist representation of the JSON resource URL, or #f if URL
> +returns 403 or 404."
> +  (hash-table->alist (json-fetch url)))

There’s an issue: some of the values in the hash tables may themselves
be hash tables as well, so we’d need to convert them as well.

The problem was already there though (and apparently it’s not much of a
problem :-)), so we can address it later.

Thank you!

Ludo’.