From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:44679) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2WuF-0005pE-GV for guix-patches@gnu.org; Fri, 14 Feb 2020 04:05:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j2WuE-0002a1-8Q for guix-patches@gnu.org; Fri, 14 Feb 2020 04:05:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:54873) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j2WuE-0002Zu-56 for guix-patches@gnu.org; Fri, 14 Feb 2020 04:05:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j2WuD-0007rx-V6 for guix-patches@gnu.org; Fri, 14 Feb 2020 04:05:01 -0500 Subject: [bug#39547] [PATCH] website: Provide JSON sources list used by Software Heritage. Resent-Message-ID: MIME-Version: 1.0 References: <20200210170418.32076-1-zimon.toutoune@gmail.com> <87pnehk1de.fsf@gnu.org> In-Reply-To: <87pnehk1de.fsf@gnu.org> From: zimoun Date: Fri, 14 Feb 2020 10:04:01 +0100 Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 39547@debbugs.gnu.org Hi Ludo, Thank you for the review. On Fri, 14 Feb 2020 at 09:40, Ludovic Court=C3=A8s wrote: > zimoun skribis: > > Format discussed here . > > > > * website/apps/packages/builder.scm (sources-json-builder): New procedu= re. > > [...] > > > +(define (sources-json-builder) > > + "Return a JSON page listing all the sources." > > Please add a reference to > here. ok > > + (define (origin->json origin) > > + (define method > > + (origin-method origin)) > > + > > + (define uri ;represented as string > > + (origin-uri origin)) > > + > > + (define (mirror->url uri) > > + (uri->string (car (maybe-expand-mirrors uri %mirrors)))) > > + > > + (define (resolve urls) > > + (let* ((url (car urls)) > > + (uri (string->uri url)) > > + (rest (cdr urls))) > > + (case (uri-scheme uri) > > + ((mirror) (mirror->url uri)) > > + ((http) url) > > + ((https) url) > > + (else > > + (if (null? rest) > > + url > > + (resolve rest)))))) > > + > > + `((type . ,(cond ((eq? url-fetch method) 'url) > > + ((eq? git-fetch method) 'git) > > + ((eq? svn-fetch method) 'svn) > > + (else #nil))) > > + ,@(cond ((eq? url-fetch method) > > + `(("url" . ,(match uri > > + ((? string? url) (mirror->url (string->= uri url))) > > + ((urls ...) (resolve urls)))))) > > + ((eq? git-fetch method) > > + `(("git_url" . ,(git-reference-url uri)))) > > + ((eq? svn-fetch method) > > + `(("svn_url" . ,(svn-reference-url uri)))) > > + (else '())) > > + ,@(if (eq? method git-fetch) > > + `(("git_ref" . ,(git-reference-commit uri))) > > + '()) > > + ,@(if (eq? method svn-fetch) > > + `(("svn_revision" . ,(svn-reference-revision > > + uri))) > > + '()))) > > Could you, in a first patch, move =E2=80=98origin->json=E2=80=99 out of > =E2=80=98packages-json-builder=E2=80=99, and in a second patch, add mirro= r-expansion > feature? Yes, I will try but I am a bit lost. There is: - packages-json-builder that I did not modified - sources-json-builder which the "adaptation" of the former to output to the sources.json format. Well, do you want a refactor of 'origin->json' shared by the 2 "{sources,packages}-json-builder"? > For mirror:// expansion, I think you can just write something like: > > (define uris > (append-map (cut maybe-expand-mirrors <> %mirrors) > (match url > ((_ ...) (map string->uri url)) > (_ (list (string->uri url)))))) > > and then pick the first element of the list. Yes, it is better. :-) > In parallel, I=E2=80=99d recommend suggesting a format change or addition= that > would allow us to provide all the URLs. :-) Ok :-) > With those changes in place, I think we=E2=80=99ll be ready to go! Working on updating the package Julia, I have seen that some patches are 'origin' and live for example upstream (https://blablab/project.git/patches/fancy-name.patch) and the current patch will not list them. Therefore, if upstream disappears and/or change in-place the patches, Guix would not be able to re-build in the future (time-machine). I am trying to implement a recursive exporter. Well, I will try to make that the v2 contains your suggestions and all the patches. Cheers, simon