From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id UKgpLt33O2HoYwEAgWs5BA (envelope-from ) for ; Sat, 11 Sep 2021 02:27:09 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id gP7eKd33O2HQHQAAbx9fmQ (envelope-from ) for ; Sat, 11 Sep 2021 00:27:09 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 507E8134CD for ; Sat, 11 Sep 2021 02:27:09 +0200 (CEST) Received: from localhost ([::1]:38412 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mOqrM-0008L6-Fr for larch@yhetil.org; Fri, 10 Sep 2021 20:27:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45532) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mOqrG-0008Ke-ER for guix-patches@gnu.org; Fri, 10 Sep 2021 20:27:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:56019) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mOqrG-0000Uz-6a for guix-patches@gnu.org; Fri, 10 Sep 2021 20:27:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mOqrG-00053G-3O for guix-patches@gnu.org; Fri, 10 Sep 2021 20:27:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50515] [PATCH 2/2] website: Add 'computed-origin-method' packages to 'sources.json'. Resent-From: zimoun Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 11 Sep 2021 00:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50515 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50515@debbugs.gnu.org Cc: zimoun Received: via spool by 50515-submit@debbugs.gnu.org id=B50515.163131998219354 (code B ref 50515); Sat, 11 Sep 2021 00:27:02 +0000 Received: (at 50515) by debbugs.gnu.org; 11 Sep 2021 00:26:22 +0000 Received: from localhost ([127.0.0.1]:39331 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOqqc-000526-0f for submit@debbugs.gnu.org; Fri, 10 Sep 2021 20:26:22 -0400 Received: from mail-wr1-f53.google.com ([209.85.221.53]:35646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mOqqX-00051a-QM for 50515@debbugs.gnu.org; Fri, 10 Sep 2021 20:26:19 -0400 Received: by mail-wr1-f53.google.com with SMTP id i23so4967841wrb.2 for <50515@debbugs.gnu.org>; Fri, 10 Sep 2021 17:26:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vwrGUIXLJn4AliOmzs+v9XSS3uJLqjn9QpjTHO8By9M=; b=UTc60ixqoFDufwyEsATmiYo09Nu69kvFsi6HTE8gNZPGloEZvGMtl+6xDC4Vw34tzp pYWLEOWio9NY8oPm1oJ2VzQYQnuhhI365JOOTZcxszfharVP1PEuCIXn/DFNnGj5QDPd h1Y4tkAMZjyrcjUzYQRscsQcyruvT5Z3fbEFvkaEUuaR6fMxfu91uWdfe179jAwEGSij vGXE+ALOG6dAw6IqRV8PPkMHTCyEPc4TQRAZWyX/fLyGsd/ZHA3Gp9lSwAqPZ3yFDDdc xEF6l1hgFH5ah1JiPHBElozKA9YqvOL9XMmmboKe5ZCh9uAhJzu8FBLnsVDNMJkuOiFr cS+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vwrGUIXLJn4AliOmzs+v9XSS3uJLqjn9QpjTHO8By9M=; b=aucPoGFcjilX7OglfbSe3007ig8sWcp2YXDPuBlGWjZSIKQjx1jPPuzCwv1PPfUrxQ t7ZsbKongYqpdHvisJ49R3C+/7LS6Yf5/dLzHpJpgDuTNhKllgyR/ONr4bQQjkMI51Nj eIm8raAcwaQXTWOMBbwfrk8wtbRz8YdwJlC+ojzL22bp8EwetZxdV1iYYIeOZbScc9tL lhFi+fBOtLRNi1Ek1gWWvWxODo6fNwxb9/Wn6tvONH+Wa/qC+OPbCWIheSvZVOeO03zz B1l5a2yipNQTP+ayj+eZ2yrzD3OBElwlYAnLk+hj4E5y06dEou9ldHd7TJ23vfYwdMLG SQuA== X-Gm-Message-State: AOAM533k/hQ35LJ3bXn0Uf6MgkNHC2VP0INw+gmQAIs2Ev7+/VNTT0jz F8OF+5tBF2nhR3OQSBx2z5sb8OOZbkA= X-Google-Smtp-Source: ABdhPJwVQkQg3Llb58m653LbNcaOsHTdDJ4K7I3EDBDprc0mctyd/Q8w8LlICbnMLQ/ZF44ocB3xHA== X-Received: by 2002:a5d:5262:: with SMTP id l2mr417880wrc.190.1631319972031; Fri, 10 Sep 2021 17:26:12 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id x21sm181561wmc.14.2021.09.10.17.26.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 17:26:11 -0700 (PDT) From: zimoun Date: Sat, 11 Sep 2021 02:26:08 +0200 Message-Id: <20210911002608.14074-2-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210911002608.14074-1-zimon.toutoune@gmail.com> References: <20210911002608.14074-1-zimon.toutoune@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1631320029; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=vwrGUIXLJn4AliOmzs+v9XSS3uJLqjn9QpjTHO8By9M=; b=OJ6zBe6UHGz/RmEy4NpE9kWfkDWpJ881rFCmcMM8R7mGTWqCkfU66NqNIqEc9nx9vqCCzB IItIzrw4Rdsz/WBuNsL98DDOrK59nZk+QfNOWqeSZVl7jgjxUXUvbUBRXcUOakuypsiukY ToJyD330J9jOlRrBE0NBn3ksZozm5qiov4bc7+0K8KwC5rt+ftYShcybnp0zTacOawEWas kApwYIH3MRYXfWXDauCSPGbz852c1z8ED7AV8EtUIJQQAT0N5wHgei7EID8T6NT941PvDi yErbMDhIOlBpUbwYWcVuna7pXc0x4Jwxg6GMusvSrFnkgGUTPgogH0sH4a8pqw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1631320029; a=rsa-sha256; cv=none; b=IminjZQkl2rJjcl3Vp4a4VElyR6xVv9EdC1GStmZ9Wbig+H8fDsXXovzfJqN/vI+9OvC1/ eM0nfgWRvzsfxV/G5EFx8nO+JJ5hSGgNmXygFx/a1NYB6BTYg731v0lVCkh9IgDtP1PRB4 9QAOVnuV4avwNBLIDAOH4r8Y570szNhjPjMH2S1guJjLPqDqg/sV2IRkArI+y7ED0zS0+S N5qOiigcwyAd000/gMVlEMgE6LFUdZP0lgU7yEgevDMmjeSGlpCDC3l0PM03c0PPJcSo7O FblujYtVtirZv7KV+fMzvhSfV2t0zt4/xbtyYovLuk8xsiJb35MPuDHiY6OzCw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=UTc60ixq; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -1.31 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=UTc60ixq; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 507E8134CD X-Spam-Score: -1.31 X-Migadu-Scanner: scn0.migadu.com X-TUID: OYmW8W7mOiG2 With Guix 9875f9bca3976bf3576eab9be42164fde454597e, the packages considered are IceCat and the Linux kernel; see: gnu/packages/gnuzilla.scm and gnu/packages/linux.scm. * website/apps/packages/builder.scm (gexp-references): Unexported variable from the module '(guix gexp)'. (origin->json): Add 'computed-origin-method' case. (package-json-builder): Adjust. (sources-json-builder): Idem. [flatten]: New procedure. --- website/apps/packages/builder.scm | 134 +++++++++++++++++++----------- 1 file changed, 84 insertions(+), 50 deletions(-) diff --git a/website/apps/packages/builder.scm b/website/apps/packages/builder.scm index fb53215..ecf958a 100644 --- a/website/apps/packages/builder.scm +++ b/website/apps/packages/builder.scm @@ -2,7 +2,7 @@ ;;; Copyright © 2017 Ludovic Courtès ;;; Copyright © 2019 Ricardo Wurmus ;;; Copyright © 2019 Nicolò Balzarotti -;;; Copyright © 2020 Simon Tournier +;;; Copyright © 2020, 2021 Simon Tournier ;;; ;;; Initially written by sirgazil ;;; who waives all copyright interest on this file. @@ -49,11 +49,14 @@ #:use-module ((guix base64) #:select (base64-encode)) #:use-module ((guix describe) #:select (current-profile)) #:use-module ((guix config) #:select (%guix-version)) + #:use-module (guix gexp) #:use-module (json) #:use-module (ice-9 match) #:use-module ((web uri) #:select (string->uri uri->string)) #:export (builder)) +;;; Required by 'origin->json' for 'computed-origin-method' corner cases +(define gexp-references (@@ (guix gexp) gexp-references)) ;;; ;;; Application builder. @@ -98,7 +101,7 @@ (define method (origin-method origin)) - (define uri ;represented as string + (define uri (origin-uri origin)) (define (resolve urls) @@ -106,53 +109,67 @@ (append-map (cut maybe-expand-mirrors <> %mirrors) (map string->uri urls)))) - `((type . ,(cond ((or (eq? url-fetch method) - (eq? url-fetch/tarbomb method) - (eq? url-fetch/zipbomb method)) 'url) - ((eq? git-fetch method) 'git) - ((or (eq? svn-fetch method) - (eq? svn-multi-fetch method)) 'svn) - ((eq? hg-fetch method) 'hg) - (else #nil))) - ,@(cond ((or (eq? url-fetch method) + (match uri + ((? promise? promise) ;computed-origin-method + (match (force promise) + ((? gexp? g) + (map origin->json + (filter-map (match-lambda + ((? gexp-input? thing) + (match (gexp-input-thing thing) + ((? origin? o) o) + (_ #f))) + (_ #f)) + (gexp-references g)))) + (_ `((type . #nil))))) + (_ ;represented as string + `((type . ,(cond ((or (eq? url-fetch method) + (eq? url-fetch/tarbomb method) + (eq? url-fetch/zipbomb method)) 'url) + ((eq? git-fetch method) 'git) + ((or (eq? svn-fetch method) + (eq? svn-multi-fetch method)) 'svn) + ((eq? hg-fetch method) 'hg) + (else #nil))) + ,@(cond ((or (eq? url-fetch method) + (eq? url-fetch/tarbomb method) + (eq? url-fetch/zipbomb method)) + `(("urls" . ,(list->vector + (resolve + (match uri + ((? string? url) (list url)) + ((urls ...) urls))))))) + ((eq? git-fetch method) + `(("git_url" . ,(git-reference-url uri)))) + ((eq? svn-fetch method) + `(("svn_url" . ,(svn-reference-url uri)))) + ((eq? svn-multi-fetch method) + `(("svn_url" . ,(svn-multi-reference-url uri)))) + ((eq? hg-fetch method) + `(("hg_url" . ,(hg-reference-url uri)))) + (else '())) + ,@(if (or (eq? url-fetch method) (eq? url-fetch/tarbomb method) (eq? url-fetch/zipbomb method)) - `(("urls" . ,(list->vector - (resolve - (match uri - ((? string? url) (list url)) - ((urls ...) urls))))))) - ((eq? git-fetch method) - `(("git_url" . ,(git-reference-url uri)))) - ((eq? svn-fetch method) - `(("svn_url" . ,(svn-reference-url uri)))) - ((eq? svn-multi-fetch method) - `(("svn_url" . ,(svn-multi-reference-url uri)))) - ((eq? hg-fetch method) - `(("hg_url" . ,(hg-reference-url uri)))) - (else '())) - ,@(if (or (eq? url-fetch method) - (eq? url-fetch/tarbomb method) - (eq? url-fetch/zipbomb method)) - (let* ((content-hash (origin-hash origin)) - (hash-value (content-hash-value content-hash)) - (hash-algorithm (content-hash-algorithm content-hash)) - (algorithm-string (symbol->string hash-algorithm))) - `(("integrity" . ,(string-append algorithm-string "-" - (base64-encode hash-value))))) - '()) - ,@(if (eq? method git-fetch) - `(("git_ref" . ,(git-reference-commit uri))) - '()) - ,@(if (eq? method svn-fetch) - `(("svn_revision" . ,(svn-reference-revision uri))) - '()) - ,@(if (eq? method svn-multi-fetch) - `(("svn_revision" . ,(svn-multi-reference-revision uri))) - '()) - ,@(if (eq? method hg-fetch) - `(("hg_changeset" . ,(hg-reference-changeset uri))) - '()))) + (let* ((content-hash (origin-hash origin)) + (hash-value (content-hash-value content-hash)) + (hash-algorithm (content-hash-algorithm content-hash)) + (algorithm-string (symbol->string hash-algorithm))) + `(("integrity" . ,(string-append algorithm-string "-" + (base64-encode hash-value))))) + '()) + ,@(if (eq? method git-fetch) + `(("git_ref" . ,(git-reference-commit uri))) + '()) + ,@(if (eq? method svn-fetch) + `(("svn_revision" . ,(svn-reference-revision uri))) + '()) + ,@(if (eq? method svn-multi-fetch) + `(("svn_revision" . ,(svn-multi-reference-revision uri))) + '()) + ,@(if (eq? method hg-fetch) + `(("hg_changeset" . ,(hg-reference-changeset uri))) + '()))))) (define (packages-json-builder) "Return a JSON page listing all packages." @@ -167,7 +184,12 @@ ,@(if cpe-name `(("cpe_name" . ,cpe-name)) '()) ,@(if cpe-version `(("cpe_version" . ,cpe-version)) '()) ,@(if (origin? (package-source package)) - `(("source" . ,(origin->json (package-source package)))) + `(("source" . ,(let ((json (origin->json (package-source package)))) + (match json + ((('type . x) other ...) + json) + ((head tail ...) ;multi-origin + head))))) ;XXXX: Improve this approximation '()) ("synopsis" . ,(package-synopsis package)) ,@(if (package-home-page package) @@ -195,11 +217,23 @@ (define (package->json package) `(,@(if (origin? (package-source package)) (origin->json (package-source package)) - `(("type" . "no-origin") + `((type . "no-origin") ("name" . ,(package-name package)))))) + (define (flatten lst) + ;; Convert nested lists to simple list + `(,@(if (null? lst) + '() + (match lst + ((head tail ...) + (match head + ((('type . x) other ...) + (cons head (flatten tail))) + (_ + (append (flatten head) (flatten tail))))))))) + (make-page "sources.json" - `(("sources" . ,(list->vector (map package->json (all-packages)))) + `(("sources" . ,(list->vector (flatten (map package->json (all-packages))))) ("version" . "1") ("revision" . ,(match (current-profile) -- 2.29.2