From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Maintaining implementations of similar utility functions like json-fetch Date: Sun, 10 Jun 2018 21:54:10 +0200 Message-ID: <87k1r6ct99.fsf@gnu.org> References: <87po5v1dy0.fsf@gnu.org> <87shafd1gl.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:41467) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fS6Pm-0005r5-9P for guix-devel@gnu.org; Sun, 10 Jun 2018 15:54:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fS6Pl-0001lf-90 for guix-devel@gnu.org; Sun, 10 Jun 2018 15:54:14 -0400 In-Reply-To: (Jelle Licht's message of "Sun, 10 Jun 2018 20:50:09 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Jelle Licht Cc: guix-devel Hello Jelle! Jelle Licht skribis: > Sorry for the delay. Cue the drum roll; Attached is my initial draft of > this patch. I initially wanted to split it up into 2 or more patches, but > could not make this work in a way that I could wrap my head around. Better late than never! :-) It looks good to me. > Also, there is yet another 'json-fetch'-like function implemented in > `guix/ci.scm', but I was not sure whether the error-handling facilities > would be applicable there. I=E2=80=99d leave (guix ci) as-is. It=E2=80=99s used by (guix scripts weat= her), which has custom HTTP error handling in this case. > From c60686975df2999906118c3a26cc9c2cef2a93b2 Mon Sep 17 00:00:00 2001 > From: Jelle Licht > Date: Sun, 10 Jun 2018 20:35:39 +0200 > Subject: [PATCH] import: json: Consolidate duplicate json-fetch functiona= lity. > > * guix/import/json.scm (json-fetch): Return a list or hash table. > (json-fetch-alist): New procedure. > * guix/import/github.scm (json-fetch*): Remove. > (latest-released-version): Use json-fetch. > * guix/import/cpan.scm (module->dist-name): Use json-fetch-alist. > (cpan-fetch): Likewise. > * guix/import/crate.scm (crate-fetch): Likewise. > * guix/import/gem.scm (rubygems-fetch): Likewise. > * guix/import/pypi.scm (pypi-fetch): Likewise. > * guix/import/stackage.scm (stackage-lts-info-fetch): Likewise. LGTM! > --- a/guix/import/json.scm > +++ b/guix/import/json.scm > @@ -22,15 +22,25 @@ > #:use-module (guix http-client) > #:use-module (guix import utils) > #:use-module (srfi srfi-34) > - #:export (json-fetch)) > + #:export (json-fetch > + json-fetch-alist)) >=20=20 > (define (json-fetch url) > - "Return an alist representation of the JSON resource URL, or #f on fai= lure." > + "Return a representation of the JSON resource URL (a list or hash tabl= e), or > +#f if URL returns 403 or 404." > (guard (c ((and (http-get-error? c) > - (=3D 404 (http-get-error-code c))) > - #f)) ;"expected" if package is unknown > - (let* ((port (http-fetch url #:headers '((user-agent . "GNU Guile") > - (Accept . "application/json= ")))) > - (result (hash-table->alist (json->scm port)))) > + (let ((error (http-get-error-code c))) > + (or (=3D 403 error) > + (=3D 404 error)))) > + #f)) > + ;; Note: many websites returns 403 if we omit a 'User-Agent' header. > + (let* ((port (http-fetch url #:headers '((user-agent . "GNU Guile") > + (Accept . "application/js= on")))) > + (result (json->scm port))) > (close-port port) > result))) > + > +(define (json-fetch-alist url) > + "Return an alist representation of the JSON resource URL, or #f if URL > +returns 403 or 404." > + (hash-table->alist (json-fetch url))) There=E2=80=99s an issue: some of the values in the hash tables may themsel= ves be hash tables as well, so we=E2=80=99d need to convert them as well. The problem was already there though (and apparently it=E2=80=99s not much = of a problem :-)), so we can address it later. Thank you! Ludo=E2=80=99.