From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id gHdQOmQQh2DjTgAAgWs5BA (envelope-from ) for ; Mon, 26 Apr 2021 21:11:32 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id gLPhNWQQh2D3cwAAB5/wlQ (envelope-from ) for ; Mon, 26 Apr 2021 19:11:32 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2792614EBD for ; Mon, 26 Apr 2021 21:11:32 +0200 (CEST) Received: from localhost ([::1]:55810 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lb6dn-0001ES-7h for larch@yhetil.org; Mon, 26 Apr 2021 15:11:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lb6dd-0001E0-7g for guix-devel@gnu.org; Mon, 26 Apr 2021 15:11:21 -0400 Received: from mout02.posteo.de ([185.67.36.66]:40999) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lb6da-0001lY-Fn for guix-devel@gnu.org; Mon, 26 Apr 2021 15:11:21 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id DCDA22400FE for ; Mon, 26 Apr 2021 21:11:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1619464275; bh=zEX3cqLLs3QOp33/9FbVMP66PShmhcozqE+M2xEsTJ4=; h=Date:From:To:Cc:Subject:From; b=lzh1JYxiF0cps1ILZMJZwz9fXNzD1SRCTHhHDd3bC5Nc5U9mQAutu9uIahAnPMk4W S6hMnDr5gdudLKgVbp7zxGVBC/tzqaOWJ/84VO+DvrOGGOuXNSCFhLUoQJkvSk7/+c TV8Hdbz3RPf7y2aCPlpxqZ4nvdrWe4ERBMUqVyHDN2lGg35fKBPlS2xcIJdAOfmjZq 8c121ATjf+h8CJ8V4ZH0sQBHsunblen/BpET3gV3cQ2Xc1fy04bJnEdwa8rwcrTvw6 9HhRgsMosGgSiGV4ObPkV1aGUdezyGGKEhOEUz/iuLMHrB/1GopiXj6yDjpxF3R6j0 MJ0fLKJxT7s4w== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4FTZHh70Qpz9rxR; Mon, 26 Apr 2021 21:11:12 +0200 (CEST) Date: Mon, 26 Apr 2021 19:11:03 +0000 From: Luciana Lima Brito To: Christopher Baines Subject: Re: Outreachy - Guix Data Service: questions about improving the data for derivation comparisons. Message-ID: <20210426161103.0647ee3f@lubrito> In-Reply-To: <87o8e1foly.fsf@cbaines.net> References: <20210421152914.461bbdbb@lubrito> <87bla6iwli.fsf@cbaines.net> <20210422170040.37941083@lubrito> <875z0ehyl3.fsf@cbaines.net> <20210422180208.28473e41@lubrito> <87zgxqggw9.fsf@cbaines.net> <20210423181551.01c42455@lubrito> <87bla4hdto.fsf@cbaines.net> <20210425171507.6a259017@lubrito> <87o8e1foly.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/0iDPGj9vSQZV5Rce3kWMcoK" Received-SPF: pass client-ip=185.67.36.66; envelope-from=lubrito@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1619464292; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=xiNpj6ttNxqXrA+YJJ7Cnilc3DM+ABitBQr1V8LZWx0=; b=WBegEzVBPfvfdtoTRzhYd23Tae++RmbtWo8sWAMIjBfRByZkW1FLZGrHqhW6svIWksT2TR wfxH3cLbniRLzCVof9WHIZqTMqgia3LETthom9QEWEVpPXodoJ3qQUK32XC+n3Zq/ITHXx UtcmmYLSCaxsI2JV0yD4nI8ZQcYKKXXNjnsYxxOY2M6suK3XdfrS8naQxNrYhRrDG06X6B nCNcbhAbUCsO/LLklnPZ43GyLN/X7rT9jdAhnPuEgA0RfdwIUSvQJqPbBXoHf6sKZUjsfO 1lBc16JUtqpefq26+f0c33NC8f9SJAMD4jn5ZxxxomKiKDPwNuiuscqvXD3Rzg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1619464292; a=rsa-sha256; cv=none; b=P7i70fnZDeA+M/UEFGUih1k3tJT5c5GuZT/BYr8abO1P6J8yBdso/AEUWScPMXIeJnClbK sVMcPzvSKKE6Inm+9+ucE4O0AbQ4OCvmHrKBJhpDKhEoq0Fge5ZhJ2OBC61x6kU05U/R7W dHZLb7xjGYy55DbSVEGfDjY8Pfr+fsiO2zHnTvNRR31fQRM+4JvxVemjns902zupT5tjFe rB0awn0kdJJ3ga6w2ikHHTbKGdR2IoKpT6aTXh3KZyG0ZoXzyfTuQFc2asanLsZaZ/pt8I HdsWQJyvFpybOH/mZQE/7HCoNidG5FE41HmDXs/A6dPFPXVJDmOnvuMCsrcjjw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=lzh1JYxi; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -2.65 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=lzh1JYxi; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 2792614EBD X-Spam-Score: -2.65 X-Migadu-Scanner: scn0.migadu.com X-TUID: oHPn/VFdmtHo --MP_/0iDPGj9vSQZV5Rce3kWMcoK Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline On Mon, 26 Apr 2021 09:15:37 +0100 Christopher Baines wrote: > So, one advantage of alists over lists is that the code is probably > less brittle when adding elements say, since code parsing the list > will probably break with a new element, but this is probably less > likely to happen with an alist. > > However, this will happen with an alist if match is used to pick > elements out. I'd suggest using assq-ref or similar to pluck elements > out. Ok, I changed that on the html.scm. > I'd consider these options first probably: > > - Could the data coming from derivation-differences-data have vectors > where appropriate already? The HTML code would probably need to be > adjusted, but I think that's fine. I tried this for days but with no success. Maybe the only way would be to tweak group-to-alist, but it touches many places, and I didn't want to mess with it. > - Could this be written in a form like: > > ,@(map (lambda (name) > ...) > '(outputs inputs sources arguments)) This only make sense to me inside render-json (because of the ,@), but I think the code would be less clean and "arguments" would appear in a different order. What I did was bind the result of a function similar to this in the let. Well, this way made things much shorter. I'm sending a new patch for you to review. -- Best Regards, Luciana Lima Brito MSc. in Computer Science --MP_/0iDPGj9vSQZV5Rce3kWMcoK Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0001-Change-handling-of-queried-data-for-derivations-comp.patch >From d0e221f5aef2892582bdcb73aceadf937b50e45f Mon Sep 17 00:00:00 2001 From: Luciana Brito Date: Sun, 25 Apr 2021 15:17:33 -0300 Subject: [PATCH] Change handling of queried data for derivations comparison. comparison.scm: return query data for derivation comparison as an alist, instead of list. html.scm: Access derivation differences data using assq-ref. controller.scm: generalize map for outputs/inputs/sources/arguments. --- guix-data-service/comparison.scm | 68 ++++++++-------- guix-data-service/web/compare/controller.scm | 82 ++++---------------- guix-data-service/web/compare/html.scm | 53 ++++++------- 3 files changed, 75 insertions(+), 128 deletions(-) diff --git a/guix-data-service/comparison.scm b/guix-data-service/comparison.scm index e5e1955..1f47c38 100644 --- a/guix-data-service/comparison.scm +++ b/guix-data-service/comparison.scm @@ -158,19 +158,23 @@ GROUP BY 1, 2, 3, 4, 5")) (let ((parsed-derivation-ids (map string->number (parse-postgresql-array-string derivation_ids)))) - (list output-name - path - hash-algorithm - hash - recursive - (append (if (memq base-derivation-id - parsed-derivation-ids) - '(base) - '()) - (if (memq target-derivation-id - parsed-derivation-ids) - '(target) - '())))))) + `((output-name . ,output-name) + (path . ,path) + ,@(if (string? hash-algorithm) + `((hash-algorithm . ,hash-algorithm)) + `((hash-algorithm . ()))) + ,@(if (string? hash) + `((hash . ,hash)) + `((hash . ()))) + (recursive . ,(string=? recursive "t")) + ,(append (if (memq base-derivation-id + parsed-derivation-ids) + '(base) + '()) + (if (memq target-derivation-id + parsed-derivation-ids) + '(target) + '())))))) (exec-query conn query))) (define (derivation-inputs-differences-data conn @@ -202,16 +206,16 @@ INNER JOIN derivations ON derivation_outputs.derivation_id = derivations.id (let ((parsed-derivation-ids (map string->number (parse-postgresql-array-string derivation_ids)))) - (list derivation_file_name - derivation_output_name - (append (if (memq base-derivation-id - parsed-derivation-ids) - '(base) - '()) - (if (memq target-derivation-id - parsed-derivation-ids) - '(target) - '())))))) + `((derivation_file_name . ,derivation_file_name) + (derivation_output_name . ,derivation_output_name) + ,(append (if (memq base-derivation-id + parsed-derivation-ids) + '(base) + '()) + (if (memq target-derivation-id + parsed-derivation-ids) + '(target) + '())))))) (exec-query conn query))) (define (derivation-sources-differences-data conn @@ -235,15 +239,15 @@ GROUP BY derivation_source_files.store_path")) (let ((parsed-derivation-ids (map string->number (parse-postgresql-array-string derivation_ids)))) - (list store_path - (append (if (memq base-derivation-id - parsed-derivation-ids) - '(base) - '()) - (if (memq target-derivation-id - parsed-derivation-ids) - '(target) - '())))))) + `((store_path . ,store_path) + ,(append (if (memq base-derivation-id + parsed-derivation-ids) + '(base) + '()) + (if (memq target-derivation-id + parsed-derivation-ids) + '(target) + '())))))) (exec-query conn query))) (define* (package-derivation-differences-data conn diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index 895bb40..287ae97 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,79 +588,25 @@ '(application/json text/html) mime-types) ((application/json) - (let ((outputs - (map - (lambda (label items) - (cons label - (list->vector - (map - (match-lambda - ((name path hash-alg hash recursive) - `((name . ,name) - (path . ,path) - ,@(if (string? hash-alg) - `((hash-algorithm . ,hash-alg)) - '()) - ,@(if (string? hash) - `((hash . ,hash)) - '()) - (recursive . ,(string=? recursive "t"))))) - (or items '()))))) - '(base target common) - (let ((output-groups (assq-ref data 'outputs))) - (list (assq-ref output-groups 'base) - (assq-ref output-groups 'target) - (assq-ref output-groups 'common))))) - - (inputs - (map - (lambda (label items) - (cons label - (list->vector - (map - (match-lambda - ((derivation output) - `((derivation . ,derivation) - (output . ,output)))) - (or items '()))))) - '(base target common) - (let ((input-groups (assq-ref data 'inputs))) - (list (assq-ref input-groups 'base) - (assq-ref input-groups 'target) - (assq-ref input-groups 'common))))) - - (sources - (map - (lambda (label items) - (cons label - (list->vector - (map - (match-lambda - ((derivation) - `((derivation . ,derivation)))) - (or items '()))))) - '(base target common) - (let ((source-groups (assq-ref data 'sources))) - (list (assq-ref source-groups 'base) - (assq-ref source-groups 'target) - (assq-ref source-groups 'common))))) - - (arguments - (map - (match-lambda - ((label args ...) - `(,label . ,(list->vector args)))) - (assq-ref data 'arguments)))) - + (let ((data-groups + (map (lambda (name) + (cons name + (map + (match-lambda + ((label args ...) + `(,label . ,(list->vector args)))) + (assq-ref data name)))) + '(outputs inputs sources arguments)))) + (render-json `((base . ((derivation . ,base-derivation))) (target . ((derivation . ,target-derivation))) - (outputs . ,outputs) - (inputs . ,inputs) - (sources . ,sources) + (outputs . ,(assq-ref data-groups 'outputs)) + (inputs . ,(assq-ref data-groups 'inputs)) + (sources . ,(assq-ref data-groups 'sources)) (system . ,(assq-ref data 'system)) (builder . ,(assq-ref data 'builder)) - (arguments . ,arguments) + (arguments . ,(assq-ref data-groups 'arguments)) (environment-variables . ,(assq-ref data 'environment-variables))) #:extra-headers http-headers-for-unchanging-content))) diff --git a/guix-data-service/web/compare/html.scm b/guix-data-service/web/compare/html.scm index 5b5fe0a..22e2dfc 100644 --- a/guix-data-service/web/compare/html.scm +++ b/guix-data-service/web/compare/html.scm @@ -487,27 +487,24 @@ (th "Hash") (th "Recursive"))) (tbody - ,@(let ((base-outputs (assq-ref outputs 'base)) - (target-outputs (assq-ref outputs 'target)) - (common-outputs (assq-ref outputs 'common))) - (append-map - (lambda (label items) - (map - (match-lambda - ((name path hash-algorithm hash recursive) - `(tr - (td ,label) - (td ,name) - (td (a (@ (href ,path)) - ,(display-store-item path))) - (td ,hash-algorithm) - (td ,hash) - (td ,recursive)))) - (or items '()))) - (list base target "Common") - (list (assq-ref outputs 'base) - (assq-ref outputs 'target) - (assq-ref outputs 'common)))))))) + ,@(append-map + (lambda (label items) + (map + (match-lambda + ((alist ...) + `(tr + (td ,label) + (td ,(assq-ref alist 'output-name)) + (td (a (@ (href ,(assq-ref alist 'path))) + ,(display-store-item (assq-ref alist 'path)))) + (td ,(assq-ref alist 'hash-algorithm)) + (td ,(assq-ref alist 'hash)) + (td ,(assq-ref alist 'recursive))))) + (or items '()))) + (list base target "Common") + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common))))))) (h2 "Inputs") ,@(let ((inputs (assq-ref data 'inputs))) `((table @@ -522,12 +519,12 @@ (lambda (label items) (map (match-lambda - ((derivation outputs) + ((alist ...) `(tr (td ,label) - (td (a (@ (href ,derivation)) - ,(display-store-item derivation))) - (td ,outputs)))) + (td (a (@ (href ,(assq-ref alist 'derivation_file_name))) + ,(display-store-item (assq-ref alist 'derivation_file_name)))) + (td ,(assq-ref alist 'derivation_output_name))))) (or items '()))) (list base target) (list (assq-ref inputs 'base) @@ -546,11 +543,11 @@ (lambda (label items) (map (match-lambda - ((file) + ((alist ...) `(tr (td ,label) - (td (a (@ (href ,file)) - ,(display-store-item file)))))) + (td (a (@ (href ,(assq-ref alist 'store_path))) + ,(display-store-item (assq-ref alist 'store_path))))))) (or items '()))) (list base target "Common") (list (assq-ref sources 'base) -- 2.30.2 --MP_/0iDPGj9vSQZV5Rce3kWMcoK--