From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id EFWqE/NBg2BEhAEAgWs5BA (envelope-from ) for ; Fri, 23 Apr 2021 23:53:55 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id IDllD/NBg2ADPAAA1q6Kng (envelope-from ) for ; Fri, 23 Apr 2021 21:53:55 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E646916BA1 for ; Fri, 23 Apr 2021 23:53:54 +0200 (CEST) Received: from localhost ([::1]:54404 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1la3kI-0002AU-3z for larch@yhetil.org; Fri, 23 Apr 2021 17:53:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38078) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1la3fZ-0006QJ-Df for guix-devel@gnu.org; Fri, 23 Apr 2021 17:49:01 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36644) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1la3fT-0007S2-Fb for guix-devel@gnu.org; Fri, 23 Apr 2021 17:49:01 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 328A127BC7D; Fri, 23 Apr 2021 22:48:54 +0100 (BST) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id f0308dab; Fri, 23 Apr 2021 21:48:53 +0000 (UTC) References: <20210421152914.461bbdbb@lubrito> <87bla6iwli.fsf@cbaines.net> <20210422170040.37941083@lubrito> <875z0ehyl3.fsf@cbaines.net> <20210422180208.28473e41@lubrito> <87zgxqggw9.fsf@cbaines.net> <20210423181551.01c42455@lubrito> User-agent: mu4e 1.4.15; emacs 27.1 From: Christopher Baines To: Luciana Lima Brito Subject: Re: Outreachy - Guix Data Service: questions about improving the data for derivation comparisons. In-reply-to: <20210423181551.01c42455@lubrito> Date: Fri, 23 Apr 2021 22:48:51 +0100 Message-ID: <87bla4hdto.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1619214835; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=C8khmdyzrcaToNZJo28Nx3drX6omUXR0bD82aIsr8qg=; b=nB65sbhzVr0Q/mmJWC5o9Hxdey0hWMsKeefxeySyMuEHIHo/asvqR8QQT5EVe6hAwv/9AS OWxAaJuGjhBoQyavvDmJ/NRV6PJ2zI+THDAUIAMXPJ1FLgd+VADP6m//J52FzkOGkK9O9h jkJMU3UTCpNR1Xz5tQEOQkusQNHMUxAV8/jr85IrfgnYpfh6sgbHXRmek0N/VVpg7nOLlx JuSqnCCSYPnZFRHmfaC34L/k4bJf2aqYbql4qEVpxkPQITEPPBqyYwxcbdKc+dBXoOdkg2 /eWSSyTWwshJRuIDVNsNQUEv51P+6tf46uNapouF0ZH+MSmJfwhrTd5IdAz75A== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1619214835; a=rsa-sha256; cv=none; b=jjdWTMAHspL9DZ4sweLZ9oRgLkj69UgerEFeyjSFrqnEPzHRZmIs/pjgBKPyjYJ+g8dJ28 04k4dPwGKh0/saQyo+06G7n7U7gcX6wtSi3XTjQUFxY6U8aTKOFyN3aB3dbSsBW1ovG9nw OUF5oGzWQ9sS4bRHdzdx8zDc1YJ8PPMx7gccY2/Krl3VKwb2eL64QbtASST11McQwFmYaV XkoajQrqbtIgxuv0JsaWnVGTU00j058F+AYHB1nGA8Aaydd84bgdo4QMVgw6sJLAIr7q6Q mO051qhYmpOi/i4Lz8EJs+ND2wEoKzGg7hD3yXki7BUU5hMttjfIKBm2lPldZQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -4.54 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: E646916BA1 X-Spam-Score: -4.54 X-Migadu-Scanner: scn0.migadu.com X-TUID: ZdroVOX+UB7w --=-=-= Content-Type: text/plain Luciana Lima Brito writes: > Hi, > > On Thu, 22 Apr 2021 22:15:34 +0100 > Christopher Baines wrote: > > I'm stuck. > >> I'd suggest avoiding '() as the value for hash and hash-algorithm when >> they're NULL in the database. One option here that I've used in some >> places is to return a alist rather than a list. This can simplify JSON >> output since it's often a alist that's desired, and can avoid breaking >> matches when they're matching on the list. >> >> Does that make sense? > > It is not so clear to me. I tried three ways, always testing on > "outputs": > > First, I tried using an alist within > the map function on derivation-outputs-differences-data > (comparison.scm), and I applied a list->vector on the result. It > worked, but I do believe that this is not the way we want it to be. > This way the label comes as another field, and I got the result like > this: > > code: > > `((label . ,(cond > ((and (memq base-derivation-id > parsed-derivation-ids) > (memq target-derivation-id > parsed-derivation-ids)) > 'common) > ((memq base-derivation-id parsed-derivation-ids) > 'base) > (else 'target))) > (name . ,output-name) > (path . ,path) > ,@(if (string? hash-algorithm) > `((hash-algorithm . ,hash-algorithm)) > '()) > ,@(if (string? hash) > `((hash . ,hash)) > '()) > (recursive . ,(string=? recursive "t"))) > > json outputs: > 0: > label: "base" > name: "foo" > path: "bar" > recursive: #f > > This way I only used the function derivation-outputs-differences-data, > not using group-to-alist or group-by-last-element. > > The second way I tried was to pass an alist as first element of the list > and append the labels (base or/and target): > > (list > `((name . ,output-name) > (path . ,path) > ,(if (string? hash-algorithm) > `(hash-algorithm . ,hash-algorithm) > '()) > ,(if (string? hash) > `(hash . ,hash) > '()) > (recursive . ,(string=? recursive "t"))) > > (append (if (memq base-derivation-id > parsed-derivation-ids) > '(base) > '()) > (if (memq target-derivation-id > parsed-derivation-ids) > '(target) > '()))) > > But this seemed to result in an extra parenthesis in the result, which > didn't work. I'd expect ,@(if ... rather than just ,(if ... and then having a list of a pair when you want the if to result in something. There's some documentation on quasiquoting (the ` thing) here: https://www.gnu.org/software/guile/manual/html_node/Expression-Syntax.html#index-quasiquote > So I tried a third way, using the pairs > individually in the list and the append. > > (list > `(name . ,output-name) > `(path . ,path) > (if (string? hash-algorithm) > `(hash-algorithm . ,hash-algorithm) > '()) > (if (string? hash) > `(hash . ,hash) > '()) > `(recursive . ,(string=? recursive "t")) > > (append (if (memq base-derivation-id > parsed-derivation-ids) > '(base) > '()) > (if (memq target-derivation-id > parsed-derivation-ids) > '(target) > '()))) > > > Furthermore, all these methods would require tweaking the html > processing. > > Here is an idea I just had. The first way I tried returned a > vector. This worked but kind of ugly. As I am already > getting the alist, I just thought about writing another function to > group by the first elements (the labels), something like this: > > (group-by-first-element l) > > where l is the alist I get from the first method I mentioned. > And then, I would have the following alist: > ((base (...)) > (target (...)) > (common ((...)(...)(...)))) > > which is the current alist we are using in controller.scm. > Problem is, I still don't know how to do this, and this seems somewhat > too long winded to get the proper alist. So, the current code copes with the general case where there can be any number of outputs, and some can be common between two derivations, and some other number can come from just the base or target derivation. The query will effectively return a list with an element for each output. I'd perhaps try to work within the current structure at first at least. If you change derivation-outputs-differences-data to return a list of alists rather than a list of lists, I'd look at adapting the group-to-alist call to work with that. The group-by-last-element procedure it passes in takes a list, and returns a pair to add to the alist. It seems like you need something like group-by-last-element, but group-by... something in a alist, which just like group-by-last-element, will take an alist, and return a pair, where the first element is the thing to use as key, and the second element of the pair is the alist tweaked to remove the key. Does that make sense? Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmCDQMNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XcB9xAAnUDWsYtjgb6EK/JmBpfgWon9rG6sPRC+ aRb/BftErfwSbf2Qp0ChFMfGoCkiSuLtwmYYG1Yyd0MFO8lfHogKDwFiXJ5WZM9z Loxxq5UcH4vxzWJd26LYrdWI+MCeDpLR1NiXYX61c1sgyPEMtu0LdBhhYizPti9F NSEyhziJePjIdSTyxGbszCiQBQrYPtU6jz3XeG69RGWls4XF4ewYa9O2oug3mgGu oLlywfWKKJIWtPmMsM2DMpxwaz5BZyrtsGRjhoQOo/1QnCYAWJ0Mf9n3iFaQgvBY G2TuY784pLSIDH1nbH1KKiixgZKgu0q2cNBq+3xb+y/LSX0vfLtoN8qAqp3kdBiR J0IdTedNaE3+k2Go7rWsxqoR1VIapvc6qKdAG/+T4xFViutFqJIswRn4HsD3LbSW 9Un0fm9wviZKreqMl68u9UNTirufK69wQ9ugimS/39pJzeyelnV46w4Fze7yiYbV +V5vwBQiLiKHmhaLJlY7O0bJZYzg/apsf4SQth1NE86oiivIG0j/1D1gLhFmL0WE ID4ZOZ7NRT9iJ+Ww8psbDUGrl1+DMbivBwsYyLI1s2BN3Hu2hKjZ7H/eypQw6Bc+ SZpPJ1KFmkEYKZz2VC+yCpqcB7wBpynjI3DI2CAAs270YagCs2YIEmOX/JXBp07a mdv752RX/H0= =Eg3k -----END PGP SIGNATURE----- --=-=-=--