From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id sKKUDnoNiGCmjgAAgWs5BA (envelope-from ) for ; Tue, 27 Apr 2021 15:11:22 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id CEVYCnoNiGB9UgAA1q6Kng (envelope-from ) for ; Tue, 27 Apr 2021 13:11:22 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 57ED334FB1 for ; Tue, 27 Apr 2021 15:11:21 +0200 (CEST) Received: from localhost ([::1]:57538 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lbNUm-0000iK-HH for larch@yhetil.org; Tue, 27 Apr 2021 09:11:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47364) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lbNTy-0000gK-Is for guix-devel@gnu.org; Tue, 27 Apr 2021 09:10:30 -0400 Received: from mout01.posteo.de ([185.67.36.65]:41845) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lbNTv-0007Rg-EG for guix-devel@gnu.org; Tue, 27 Apr 2021 09:10:30 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id E401C240029 for ; Tue, 27 Apr 2021 15:10:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1619529022; bh=RoYbuHfv7tbvWYtCN10T/RULTgt6ZaJH8GPhJBv2G2w=; h=Date:From:To:Cc:Subject:From; b=Bjf97HcE6oXtY3vnCATYwYovkVxlTGA5nC39G43q58FuGWnwT4Nj4da2vvHYL27f+ i4rV/+h2OXAhAKGV9KNE3MrQYLn+BaVEBxF0OqDLjMhVFswknBRFWrjkXF/KxDhCQ/ 6DveBx+Z6V+EP46PQkWG2FRFC987pQMp3B4ghx57pThBySDIB8/zm1l9XujOWsJoFA f1loCCbaGiN/7rqfsz4vbMlpDaaqmCV1Vb4NFSSBSF8Q7uWtcQMZNt/IpZwsDAFaIx k7s3vnhDULs2M5n3CwyIsCdO7vHKM/wpoLic/g2U2nz/hibVQLYrXvJ90tU0swQDFe J4vLFjVeV9YMA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4FV2Dl4pcSz6tmQ; Tue, 27 Apr 2021 15:10:15 +0200 (CEST) Date: Tue, 27 Apr 2021 13:10:01 +0000 From: Luciana Lima Brito To: Christopher Baines Subject: Re: Outreachy - Guix Data Service: questions about improving the data for derivation comparisons. Message-ID: <20210427101001.0242f90e@lubrito> In-Reply-To: <87im48g2s1.fsf@cbaines.net> References: <20210421152914.461bbdbb@lubrito> <87bla6iwli.fsf@cbaines.net> <20210422170040.37941083@lubrito> <875z0ehyl3.fsf@cbaines.net> <20210422180208.28473e41@lubrito> <87zgxqggw9.fsf@cbaines.net> <20210423181551.01c42455@lubrito> <87bla4hdto.fsf@cbaines.net> <20210425171507.6a259017@lubrito> <87o8e1foly.fsf@cbaines.net> <20210426161103.0647ee3f@lubrito> <87im48g2s1.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/2NXyTSVF6Lu.mU0SFmDdNH+" Received-SPF: pass client-ip=185.67.36.65; envelope-from=lubrito@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1619529081; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=R/noXzhIbZzP1UdhjS2uDql4yunxc74csUIbw2vVTkU=; b=AglwHdOp7QPUFoQ2ry84HVs7xePG840ep4LMiAQxz6gn+gcGzRZRJpoBtxI1edVuiQvoXH mxqlxEC/WL51VssXe/izv7K9JFnFx/MQJzPNx3U2uCHsqBv5d/C168HjrmOlGl0UhA/f38 HlqtgyZSaMJrx9hnGqtSltXE13n9fIsu+Tk3at4Nv4aZ5RatHIQGK35+weGS5lHiD4Mpne t5vSklwdht36iWTNSIQb45EQONqsYa+ms+x08J58Hov3iJow/YnHZe5o7geoH+Ijrn+x7G aNz0yeoqtcRFuDs3MhgHX8IiO/vKLiQB0pWFRZ0cJjgTg6S1XhUHBrO+EJUY6A== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1619529081; a=rsa-sha256; cv=none; b=fWIsKjIKP6v9zt8Jr/SWnYbZjuuEtOascrn7aQpm8o1/+mBEBJxj7B/4NEyZrCenarnF0X qDWLKUtDzmFNglYbTaU88DrXr2KoNQ+bphd+81NVQJmdpRi9N/S806rk1ZU9P4o8WoLaZG KWECv+nNdvdNSdBf3o0nN1vfFM/SYRUuITliDpKfuhmZezGzD8uOvah7SbAlyYfWNObwue FfasrrNwbSe4+Q6dvZi34WI2DuD2Xr1Qo2GYzdTYuMDGdrwDRNB00xtJnacRMNJHLU5Dls NNc4mEMRs+0Bum5BPc8vyrS0lK/3duVVjLQDpDU5tu1cY3Tjgr99JEODZLpdHA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=Bjf97HcE; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -2.65 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=Bjf97HcE; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 57ED334FB1 X-Spam-Score: -2.65 X-Migadu-Scanner: scn0.migadu.com X-TUID: mHWycUjFQeSG --MP_/2NXyTSVF6Lu.mU0SFmDdNH+ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline On Mon, 26 Apr 2021 22:21:50 +0100 Christopher Baines wrote: > > Rather than writing: > > (match-lambda > ((alist ...) > > I'd just use > > (lambda (alist) > > as I think that's equivalent right? Right, I did this. > >> I'd consider these options first probably: > >> > >> - Could the data coming from derivation-differences-data have > >> vectors where appropriate already? The HTML code would probably > >> need to be adjusted, but I think that's fine. > > > > I tried this for days but with no success. Maybe the only way would > > be to tweak group-to-alist, but it touches many places, and I > > didn't want to mess with it. > > Maybe add another procedure that combines group-to-alist but generates > an alist with vectors as the values? (group-to-alist/vector maybe). > I think using let is OK, but I think just unpacking data-groups as > you've called it directly in to the alist is fine (so ,@data-groups), > rather than picking out the elements. JSON objects are unordered, so > the ordering isn't something that really matters. > > If you do go down this route though, I'd probably add a comment saying > what things are being added to the outer most alist, just to make the > code quicker to read. Well, I went down the second route, now I'm calling the ,@data-groups and I added a comment explaining its use. The main point here is, the code is working and it looks nice, but to get the data with the vectors seems to be right too. I'm sending the new patch for your review and I'll wait for your call, if you think I should try the first route or not. -- Best Regards, Luciana Lima Brito MSc. in Computer Science --MP_/2NXyTSVF6Lu.mU0SFmDdNH+ Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0001-Change-handling-of-queried-data-for-derivations-comp.patch >From 03a70ac2e07f2eec35a9473d8930a9cbefa50b92 Mon Sep 17 00:00:00 2001 From: Luciana Brito Date: Sun, 25 Apr 2021 15:17:33 -0300 Subject: [PATCH] Change handling of queried data for derivations comparison. comparison.scm: return query data for derivation comparison as an alist, instead of list. html.scm: Access derivation differences data using assq-ref. controller.scm: generalize map for outputs/inputs/sources/arguments. --- guix-data-service/comparison.scm | 68 +++++++++-------- guix-data-service/web/compare/controller.scm | 78 +++----------------- guix-data-service/web/compare/html.scm | 62 +++++++--------- 3 files changed, 75 insertions(+), 133 deletions(-) diff --git a/guix-data-service/comparison.scm b/guix-data-service/comparison.scm index e5e1955..1f47c38 100644 --- a/guix-data-service/comparison.scm +++ b/guix-data-service/comparison.scm @@ -158,19 +158,23 @@ GROUP BY 1, 2, 3, 4, 5")) (let ((parsed-derivation-ids (map string->number (parse-postgresql-array-string derivation_ids)))) - (list output-name - path - hash-algorithm - hash - recursive - (append (if (memq base-derivation-id - parsed-derivation-ids) - '(base) - '()) - (if (memq target-derivation-id - parsed-derivation-ids) - '(target) - '())))))) + `((output-name . ,output-name) + (path . ,path) + ,@(if (string? hash-algorithm) + `((hash-algorithm . ,hash-algorithm)) + `((hash-algorithm . ()))) + ,@(if (string? hash) + `((hash . ,hash)) + `((hash . ()))) + (recursive . ,(string=? recursive "t")) + ,(append (if (memq base-derivation-id + parsed-derivation-ids) + '(base) + '()) + (if (memq target-derivation-id + parsed-derivation-ids) + '(target) + '())))))) (exec-query conn query))) (define (derivation-inputs-differences-data conn @@ -202,16 +206,16 @@ INNER JOIN derivations ON derivation_outputs.derivation_id = derivations.id (let ((parsed-derivation-ids (map string->number (parse-postgresql-array-string derivation_ids)))) - (list derivation_file_name - derivation_output_name - (append (if (memq base-derivation-id - parsed-derivation-ids) - '(base) - '()) - (if (memq target-derivation-id - parsed-derivation-ids) - '(target) - '())))))) + `((derivation_file_name . ,derivation_file_name) + (derivation_output_name . ,derivation_output_name) + ,(append (if (memq base-derivation-id + parsed-derivation-ids) + '(base) + '()) + (if (memq target-derivation-id + parsed-derivation-ids) + '(target) + '())))))) (exec-query conn query))) (define (derivation-sources-differences-data conn @@ -235,15 +239,15 @@ GROUP BY derivation_source_files.store_path")) (let ((parsed-derivation-ids (map string->number (parse-postgresql-array-string derivation_ids)))) - (list store_path - (append (if (memq base-derivation-id - parsed-derivation-ids) - '(base) - '()) - (if (memq target-derivation-id - parsed-derivation-ids) - '(target) - '())))))) + `((store_path . ,store_path) + ,(append (if (memq base-derivation-id + parsed-derivation-ids) + '(base) + '()) + (if (memq target-derivation-id + parsed-derivation-ids) + '(target) + '())))))) (exec-query conn query))) (define* (package-derivation-differences-data conn diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index 895bb40..a48b7c5 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,79 +588,23 @@ '(application/json text/html) mime-types) ((application/json) - (let ((outputs - (map - (lambda (label items) - (cons label - (list->vector - (map - (match-lambda - ((name path hash-alg hash recursive) - `((name . ,name) - (path . ,path) - ,@(if (string? hash-alg) - `((hash-algorithm . ,hash-alg)) - '()) - ,@(if (string? hash) - `((hash . ,hash)) - '()) - (recursive . ,(string=? recursive "t"))))) - (or items '()))))) - '(base target common) - (let ((output-groups (assq-ref data 'outputs))) - (list (assq-ref output-groups 'base) - (assq-ref output-groups 'target) - (assq-ref output-groups 'common))))) - - (inputs - (map - (lambda (label items) - (cons label - (list->vector - (map - (match-lambda - ((derivation output) - `((derivation . ,derivation) - (output . ,output)))) - (or items '()))))) - '(base target common) - (let ((input-groups (assq-ref data 'inputs))) - (list (assq-ref input-groups 'base) - (assq-ref input-groups 'target) - (assq-ref input-groups 'common))))) - - (sources - (map - (lambda (label items) - (cons label - (list->vector - (map - (match-lambda - ((derivation) - `((derivation . ,derivation)))) - (or items '()))))) - '(base target common) - (let ((source-groups (assq-ref data 'sources))) - (list (assq-ref source-groups 'base) - (assq-ref source-groups 'target) - (assq-ref source-groups 'common))))) - - (arguments - (map - (match-lambda - ((label args ...) - `(,label . ,(list->vector args)))) - (assq-ref data 'arguments)))) + (let ((data-groups + (map (lambda (name) + (cons name + (map + (match-lambda + ((label args ...) + `(,label . ,(list->vector args)))) + (assq-ref data name)))) + '(outputs inputs sources arguments)))) + ;data-groups returns four pairs/entries: outputs, inputs, sources and arguments. (render-json `((base . ((derivation . ,base-derivation))) (target . ((derivation . ,target-derivation))) - (outputs . ,outputs) - (inputs . ,inputs) - (sources . ,sources) + ,@data-groups (system . ,(assq-ref data 'system)) (builder . ,(assq-ref data 'builder)) - (arguments . ,arguments) (environment-variables . ,(assq-ref data 'environment-variables))) #:extra-headers http-headers-for-unchanging-content))) diff --git a/guix-data-service/web/compare/html.scm b/guix-data-service/web/compare/html.scm index 5b5fe0a..d144736 100644 --- a/guix-data-service/web/compare/html.scm +++ b/guix-data-service/web/compare/html.scm @@ -487,27 +487,23 @@ (th "Hash") (th "Recursive"))) (tbody - ,@(let ((base-outputs (assq-ref outputs 'base)) - (target-outputs (assq-ref outputs 'target)) - (common-outputs (assq-ref outputs 'common))) - (append-map - (lambda (label items) - (map - (match-lambda - ((name path hash-algorithm hash recursive) - `(tr - (td ,label) - (td ,name) - (td (a (@ (href ,path)) - ,(display-store-item path))) - (td ,hash-algorithm) - (td ,hash) - (td ,recursive)))) - (or items '()))) - (list base target "Common") - (list (assq-ref outputs 'base) - (assq-ref outputs 'target) - (assq-ref outputs 'common)))))))) + ,@(append-map + (lambda (label items) + (map + (lambda (alist) + `(tr + (td ,label) + (td ,(assq-ref alist 'output-name)) + (td (a (@ (href ,(assq-ref alist 'path))) + ,(display-store-item (assq-ref alist 'path)))) + (td ,(assq-ref alist 'hash-algorithm)) + (td ,(assq-ref alist 'hash)) + (td ,(assq-ref alist 'recursive)))) + (or items '()))) + (list base target "Common") + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common))))))) (h2 "Inputs") ,@(let ((inputs (assq-ref data 'inputs))) `((table @@ -521,13 +517,12 @@ ,@(append-map (lambda (label items) (map - (match-lambda - ((derivation outputs) - `(tr - (td ,label) - (td (a (@ (href ,derivation)) - ,(display-store-item derivation))) - (td ,outputs)))) + (lambda (alist) + `(tr + (td ,label) + (td (a (@ (href ,(assq-ref alist 'derivation_file_name))) + ,(display-store-item (assq-ref alist 'derivation_file_name)))) + (td ,(assq-ref alist 'derivation_output_name)))) (or items '()))) (list base target) (list (assq-ref inputs 'base) @@ -545,12 +540,11 @@ ,@(append-map (lambda (label items) (map - (match-lambda - ((file) - `(tr - (td ,label) - (td (a (@ (href ,file)) - ,(display-store-item file)))))) + (lambda (alist) + `(tr + (td ,label) + (td (a (@ (href ,(assq-ref alist 'store_path))) + ,(display-store-item (assq-ref alist 'store_path)))))) (or items '()))) (list base target "Common") (list (assq-ref sources 'base) -- 2.30.2 --MP_/2NXyTSVF6Lu.mU0SFmDdNH+--