Re: Outreachy - Guix Data Service: questions about improving the data for derivation comparisons.

[Christopher Baines <mail@cbaines.net>]

[-- Attachment #1: Type: text/plain, Size: 5387 bytes --]


Luciana Lima Brito <lubrito@posteo.net> writes:

> Hi,
>
> On Thu, 22 Apr 2021 22:15:34 +0100
> Christopher Baines <mail@cbaines.net> wrote:
>
> I'm stuck.
>
>> I'd suggest avoiding '() as the value for hash and hash-algorithm when
>> they're NULL in the database. One option here that I've used in some
>> places is to return a alist rather than a list. This can simplify JSON
>> output since it's often a alist that's desired, and can avoid breaking
>> matches when they're matching on the list.
>>
>> Does that make sense?
>
> It is not so clear to me. I tried three ways, always testing on
> "outputs":
>
> First, I tried using an alist  within
> the map function on derivation-outputs-differences-data
> (comparison.scm), and I applied a list->vector on the result. It
> worked, but I do believe that this is not the way we want it to be.
> This way the label comes as another field, and I got the result like
> this:
>
> code:
>
> `((label . ,(cond
> 		((and (memq base-derivation-id
> 			parsed-derivation-ids)
>               	      (memq target-derivation-id
> 			parsed-derivation-ids))
>                    'common)
>          	((memq base-derivation-id parsed-derivation-ids)
> 		   'base)
>               	(else 'target)))
>            (name . ,output-name)
>            (path . ,path)
>            ,@(if (string? hash-algorithm)
>                  `((hash-algorithm . ,hash-algorithm))
>                  '())
>            ,@(if (string? hash)
>                  `((hash . ,hash))
>                  '())
>            (recursive . ,(string=? recursive "t")))
>
> json outputs:
>    0:
> 	label: "base"
> 	name: "foo"
> 	path: "bar"
> 	recursive: #f
>
> This way I only used the function derivation-outputs-differences-data,
> not using group-to-alist or group-by-last-element.
>
> The second way I tried was to pass an alist as first element of the list
> and append the labels (base or/and target):
>
> (list
>          `((name . ,output-name)
>            (path . ,path)
>            ,(if (string? hash-algorithm)
>                `(hash-algorithm . ,hash-algorithm)
>                '())
>            ,(if (string? hash)
>                `(hash . ,hash)
>                '())
>            (recursive . ,(string=? recursive "t")))
>
>          (append (if (memq base-derivation-id
>                            parsed-derivation-ids)
>                      '(base)
>                      '())
>                  (if (memq target-derivation-id
>                            parsed-derivation-ids)
>                      '(target)
>                      '())))
>
> But this seemed to result in an extra parenthesis in the result, which
> didn't work.

I'd expect ,@(if ... rather than just ,(if ... and then having a list of
a pair when you want the if to result in something. There's some
documentation on quasiquoting (the ` thing) here:

  https://www.gnu.org/software/guile/manual/html_node/Expression-Syntax.html#index-quasiquote

> So I tried a third way, using the pairs
> individually in the list and the append.
>
> (list
>          `(name . ,output-name)
>          `(path . ,path)
>          (if (string? hash-algorithm)
>                  `(hash-algorithm . ,hash-algorithm)
>                  '())
>          (if (string? hash)
>                  `(hash . ,hash)
>                  '())
>          `(recursive . ,(string=? recursive "t"))
>
>          (append (if (memq base-derivation-id
>                            parsed-derivation-ids)
>                      '(base)
>                      '())
>                  (if (memq target-derivation-id
>                            parsed-derivation-ids)
>                      '(target)
>                      '())))
>
>
> Furthermore, all these methods would require tweaking the html
> processing.
>
> Here is an idea I just had. The first way I tried returned a
> vector. This worked but kind of ugly. As I am already
> getting the alist, I just thought about writing another function to
> group by the first elements (the labels), something like this:
>
> (group-by-first-element l)
>
> where l is the alist I get from the first method I mentioned.
> And then, I would have the following alist:
> ((base (...))
>  (target (...))
>  (common ((...)(...)(...))))
>
> which is the current alist we are using in controller.scm.
> Problem is, I still don't know how to do this, and this seems somewhat
> too long winded to get the proper alist.

So, the current code copes with the general case where there can be any
number of outputs, and some can be common between two derivations, and
some other number can come from just the base or target derivation.

The query will effectively return a list with an element for each
output.

I'd perhaps try to work within the current structure at first at least.

If you change derivation-outputs-differences-data to return a list of
alists rather than a list of lists, I'd look at adapting the
group-to-alist call to work with that.

The group-by-last-element procedure it passes in takes a list, and
returns a pair to add to the alist.

It seems like you need something like group-by-last-element, but
group-by... something in a alist, which just like group-by-last-element,
will take an alist, and return a pair, where the first element is the
thing to use as key, and the second element of the pair is the alist
tweaked to remove the key.

Does that make sense?

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]